project goal

For most remote sensing applications, there is a tradeoff between temporal and spatial resolution. Either you have high resolution data that is infrequently sampled, or you can have data that is updated frequently, but at a lower resolution.

In this project, I was curious whether I could use an image super resolution neural network to improve the resolution of remote sensing data in the ocean.

methods

To test this technique, I needed an ocean dataset where there is detailed structure. I chose chlorophyll a concentrations, and sea surface temperatures as measured by the Aqua/MODIS satellite.
For context, chlorophyll a concentration is a proxy for the amount of phytoplankton in the ocean. For biological oceanography, phytoplankton concentrations are one of the most important features to monitor as it forms the base of the oceanic food chain.
The MODIS spectrometer aboard the Aqua satellite was launched in 2002. It images the earth every 1-2 days. I used level 3 preprocessed monthly composite images found here in order to fill in many of the gaps in coverage due to cloud cover, while still maintaining the spatial structure that is present in the chlorphyll and temperature distribution.
Because the sensing data is grayscale and the super resolution network expects an RGB image, I mapped the intensity to common matplotlib colormaps.
Level 3 products (products with the most preprocessing) were most accessible at 4km and 9km per pixel resolutions. For this test, I used the 9km data (low res) as the images to upsample, and 4km data (high res) as the ground truth image. I also compared the super resolution images against basic bicubic upsampling.
To see and run the code, you can launch a Google colab notebook here: Ocean Super Resolution

results

ocean temperature - bicubic upsampling vs. super resolution

chlorophyll a - high res baseline vs. super resolution

ocean temperature - bicubic upsampling vs. super resolution

ocean temperature - high res baseline vs. super resolution

conclusion

The network certainly accomplishes the goal of upsampling an image and sharpening the output. However, there are a fair number of artifacts present in the outputs.
It's questionable though whether the benefits of increased resolution would be strong enough to outweigh the downsides of introducing processing artifacts into the sample.

what would i do next

• I'd be curious whether the network could perform substantially better if trained specifically on remote sensing data. The network pereceptually performed better when using natural color Landsat imagery.
• Do different color maps impact the outcome? The network certainly uses color patterns from natural images in order to aid the upsampling process. Perhaps the "jet" and "viridis" color schemes are unfamiliar to the network.