Instrument Autonomy

Current Projects

• 

  Machine Learning Classifier for OCO-2/3 Bad Pixel Map

  A machine learning approach has been developed to enhance the bad pixel map for the imaging spectrometers of the Orbiting Carbon Observatory-2 and -3. The method characterizes each pixel's behavior through a collection of interpretable metrics, which are then utilized to construct a binary "bad-good" classifier. This approach has been demonstrated to significantly reduce the time and effort required for bad pixel detection, while simultaneously improving the accuracy of the resulting mask and enabling new insights into pixel behavior. The algorithm is currently being employed operationally in calibration processes to assess detector health and potential instrument changes.
• 

  Segmentation, Instance Tracking, and data Fusion Using multi-SEnsor imagery (SIT-FUSE)

  Accurate detection, segmentation and tracking of geophysical objects (fire, smoke, algal blooms) at high spatiotemporal resolution are essential to better understand and therefore model complicated Earth Systems interactions. Segmentation, Instance Tracking, and data Fusion Using multi-SEnsor imagery (SIT-FUSE) provides a self-supervised deep learning framework that allows users to segment and track instances of objects in single and multi-sensor scenes from orbital and suborbital instruments. By leveraging self-supervised learning, subject matter experts are kept in the loop while removing the need for per-instrument detection algorithms along with the labor intensive and sometimes error prone task of manual per-pixel labeling.
• 

  Science Yield improvemeNt via Onboard Prioritization and Summary of Information System (SYNOPSIS)

  SYNOPSIS is a framework for downlink prioritization that uses onboard estimates of science utility along with data product similarity assessments to select a downlink ordering that prioritizes products with high science utility while also capturing data with diverse characteristics to enable serendipitous discovery of novel or unexpected phenomena. SYNOPSIS will also account for user-specified dependencies that positively or negatively affect the utility of observations across instruments, such as observations of the same target from multiple instruments that provide complementary information.

Past Projects

• 

  Barefoot Rover

  Modern rovers rely heavily on visual images and telemetry, such as motor current, to assess the health and safety of the vehicle. Systems lacking in-situ instruments have less ability to assess the interaction between the terrain and the rover mobility system. The barefoot rover wheel carries a payload of an electrochemical impedance spectrometer (EIS), for hydration detection, and is wrapped in a grid of pressure sensors. The pressure grid gives continual context for the pressure signature of the ground underneath the rover. This can be used for high-level detections, like determining the terrain type the rover is operating on, or low-level detections, like determining the slip rate of the wheel. The combination of in-situ terrain and hydration information results in real-time extractions for both science and engineering.
• 

  Content-based On-board Summarization to Monitor Infrequent Change (COSMIC)

  The COSMIC (Capturing Onboard Summarization to Monitor Image Change) task will extend several proven, flight-heritage technologies from the Machine Learning & Instrument Autonomy group (MLIA, 398J) to create a robust, flexible orbital system for conducting planetary surveys and change monitoring in the Martian environment using imagery that would likely never be downlinked due to bandwidth constraints. The system will be designed to facilitate and support scientist-driven exploration of Mars, delivering planet-wide summarization databases, monitoring scientist-specified areas for change, autonomously detecting changes in non-monitored areas, and providing relevant, informative descriptions of onboard images to advise downlink prioritization. This system will demonstrate a key new exploration paradigm: “Observe much, return best.”
• 

  Responsive Onboard Science for the Europa Clipper Mission

  Upcoming missions to remote destinations like Jupiter’s moon Europa will operate at extreme distances from the Earth. The high-radiation environment limits their expected lifetimes, so they must make the most of every observing and downlink opportunity. We have developed methods to analyze data as it is collected during a Europa flyby to quickly identify content of interest: an active icy plume, surface mineral deposits, or any unexpected phenomena (scientific anomalies). Data with positive detections can be marked for high-priority downlink to Earth and inform the next steps in mission planning. The goal of this work is to enable data analysis and machine learning classifiers to operate onboard and increase the rate of exploration and discovery.
• 

  HiiHAT Automated Hyperspectral Image Summary

  We are developing algorithms for efficient automated summary and analysis of hyperspectral scenes. These are aimed primarily at the challenges of planetary science datasets.
• 

  Detecting Transient Surface Features with Dynamic Landmarking

  We have developed methods to dynamically and autonomously detect transient surface features, such as dust devil tracks or dark slope streaks on Mars, from images. Most prior work on this subject has relied on manual examination of image pairs. Exciting discoveries of new surface features such as gullies and impact craters have been made, usually serendipitously. How many more such features remain undiscovered in the massive volume of images being collected and returned? Automated methods can help reduce the manual effort needed to find and catalog new and interesting features. Previous techniques for automated analysis have focused on changes at the pixel level. They require an initial, sometimes slow, full registration between a candidate pair of images. Once the images are registered, subtracting one from the other yields changes. These are usually thresholded or subjected to further analysis to help filter out noise and other uninteresting 'changes'.
• 

  Near Earth Asteroid Scout (NEA Scout) Science Autonomy

  Small spacecraft are continually evolving in capability and mission complexity. As spacecraft size decreases, physical limitations present new challenges for mission designers. These include limited instrument aperture, low communications bandwidth, and reduced attitude control. Software techniques can address these limitations to retain the capabilities of larger spacecraft, in a small form factor. These techniques move the first order science analysis, which is traditionally completed on the ground, onboard the spacecraft. This can minimize the amount of data volume required for first order decision making. We present a collection of techniques designed for mitigating limited pointing stability for target acquisition, onboard image calibration and decision making, in a low bandwidth environment. The flight software implementations of these algorithms have been run on New Horizons, Rosetta and terrestrially acquired data sets. These experiments validate the capability to detect the target, through noise and target location uncertainty.
• 

  Onboard Autonomous Science Investigation System

  We have developed an onboard science analysis technology for increasing science return from missions. Our technology evaluates the geologic data gathered by the rover.
• 

  Operational Recommendations for Capturing History and Infusing Data Science (ORCHIDS)

  The ORCHIDS system provides a framework for capturing knowledge generated in a spacecraft mission system in formats ready for data science. This information spans planning artifacts, spacecraft telemetry, spacecraft event reporting (EVRs), commands uplinked, team staffing, ground system processing status, and human reporting artifacts. Additionally, ORCHIDS provides a system for metadata tracking, so subject matter experts can easily annotate the correct usage of their data for future data scientists. This significantly reduces the amount of time required for data scientists to learn about the data they’re working with. The ORCHIDS framework manages the time alignment and featurization of these data streams, so data scientists can minimize the ‘data wrangling’ necessary to conduct hypothesis testing and data exploration with their mission system customers.
• 

  Ocean Worlds Life Surveyor (OWLS)

  The Ocean Worlds Life Surveyor (OWLS) project is focused on finding life on watery moons in the solar system. All life we know of depends on liquid water, so ocean worlds are a natural target of scientific inquiry. The broader OWLS team is developing several instruments that can find and return evidence of microorganisms (think single cell bacteria or eukaryotes) from water samples. This will include microscopy videos (so we can literally look for swimming cells) as well as mass spectrometry data (to search for chemical signatures of life).
• 

  Planned Observations and Intelligent Science Experimentation (POISE)

  POISE will develop autonomous technologies that enable coordinated, targeted, adaptive observations across multiple observing systems guided by the estimated improvement to our model-based understanding, predictive skills, and scientific understanding of highest priority Earth processes in the Earth Sciences. To achieve this vision, we will bring together adaptive model-driven observation selection via automated tasking and intelligent instruments.