Abstract

The study and prediction of space weather entails the analysis of solar images showing structures of the Sun’s atmosphere. When imaged from the Earth’s ground, images may be polluted by terrestrial clouds which hinder the detection of solar structures. We propose a new method to remove cloud shadows, based on a U-Net architecture, and compare classical supervision with conditional GAN. We evaluate our method on two different imaging modalities, using both real images and a new dataset of synthetic clouds. Quantitative assessments are obtained through image quality indices (RMSE, PSNR, SSIM, and FID). We demonstrate improved results with regards to the traditional cloud removal technique and a sparse coding baseline, on different cloud types and textures.

Data

We have released the data used in this project on Zenodo

Source Code

The source code for this project is hosted on the LIS-lab Gitlab.

Install

You can pip install directly from this repo:

pip install git+https://github.com/jaypmorgan/cloud-removal.git

or if you’ve cloned the repo to a local directory:

cd cloudremoval
pip install ./

Usage

Using the existing synthetic clouds dataset:

import dfp
from cloudremoval.dataset import SyntheticClouds, CloudsTransform

# download the data
dataset = SyntheticClouds(download=True)

# get only a single wavelength from the data
caii = dataset.filter(lambda row: dfp.has_props(row, {"type": "Ca II"}))

# split into train and test
train, test = caii.split(lambda row: dfp.has_props(row, {"subset": "train"}))

# get the first instance:
item = train[0]
inp1 = item.input
item.target
item.mask

# Add a transform
train.transform = CloudsTransform(hflip_p=0.5, vflip_p=0.5)
item = train[0]
inp2 = item.input

To create a model, or load one using existing model weights:

from cloudremoval.model import CloudRemover

# create a new model from scratch (i.e. random model weights)
model = CloudRemover()

# create a model using the existing weights
model = CloudRemover(pretrained=True)

# create a model using a different wavelength
model = CloudRemover(wavelength="H-alpha", pretrained=True)

# test making of predictions
dataset = SyntheticClouds(download=True, transform=CloudsTransform())
model = CloudRemover(pretrained=True)
out = model(dataset[0].input[None,...])*dataset[0].mask[None,...]

import matplotlib.pyplot as plt
plt.imshow(out[0,0].detach().cpu().numpy(), cmap="Greys_r")