Intelligentia artificiali tantum utere si exemplar in datis astronomicis eruditum est.
The use of AI (Artificial Intelligence) as part of the workflow in Astronomical image processing.
It has become increasingly common for astronomers to use Artificial Intelligence systems to aid in the processing of their image data. This is done, often with little thought to whether the results, however pleasing may be less than authentic in the final analysis.
There are AI systems in many modern image editing programs which often form the main basis of the advertising of the software. This is frequently because the programs have the ability to produce generatively, absolutely fake content to augment the original image. Whilst there may be some artistic merit in this, this kind of capability has no place in the processing of astronomical images. These programs have AI models that are trained on vast amounts of general photographic images rather than specifically on astronomical data. In my opinion such software should be avoided when using AI, and should be used only for general, traditional, mathematical processes that can be used for processes such as adjusting levels, curves, saturation, sharpness etc. Even then, as any experienced astronomer knows, even these processes should be used judiciously to avoid the introduction of artefacts into the image.
AI models used for in astronomical image processing
There are five areas in which AI models are used.
- Star removal.
- Background gradient removal.
- Noise reduction.
- Deconvolution... sharpening of stellar and non-stellar components in an image.
- Re-scaling.
All of these processes have traditional counterparts that do not involve AI and these mathematical processes should not be forgotten in one’s enthusiasm to get on board with AI!
An example is G’MIC Qt which can variously be run as a stand alone application or as a plugin for GIMP and other programs. This software contains a huge selection of procedures for the mathematical approach to different aspects of image processing. In this software it is possible to appreciate the different results produced by different mathematical ways of revealing details in any given image. This fact alone should caution a user against the over zealous use of any given procedure.
The programs with which G’MIC Qt can be used as a plugin are:
Adobe Photoshop
GIMP 2 and 3
Affinity Photo
PaintShop Pro
PhotoLine
XnView
IrfanView
And other 8bf-compatible hosts. An 8bf-compatible host is any graphics application that implements Adobe’s Photoshop filter API and can load filter plugins with the extension. When you drop a file into the host’s plugin folder, the application recognizes it as an additional filter option in its menu system.
Similarly, a user must guard against the over zealous use of AI procedures that they may use in their astronomical image processing. Many of these AI procedures have sliders that allow the user to determine the aggressiveness of the application of the AI technique to their data. There can be undesirable consequences from over aggressive application of an AI process.
The inherent dangers of using AI tools in astronomical image processing
Distortion of Data Integrity
Artificial intelligence tools often apply learned patterns to enhance images, but in doing so they can inadvertently change or erase genuine celestial features. Concerns have been raised that AI-driven star removal or deconvolution may misclassify faint nebular structures as artefacts and strip them away, compromising the integrity of the data
Introduction of Hallucinations and Artefacts
Because AI models “hallucinate” details to satisfy their training objectives, they can introduce spurious structures such as faint filaments, false stars, etc that never existed in the raw exposures. These artificial patterns risk being accepted as real structure and they compromise the data
Loss of Genuine Faint Signals
Advanced noise-reduction networks aim to distinguish signal from background noise, but overzealous filtering can remove legitimate low-contrast features. Small and very faint objects might be mistaken for noise and eliminated or simply vanish under aggressive smoothing.
Training Data Bias and Coverage Gaps
AI models rely on the datasets they’re trained on. For example if those datasets emphasize only bright objects, the models may under-perform or misbehave when confronted with a very dim object: Such bias can blind AI pipelines to anomalies that don’t fit the “expected” patterns. The big worry here is whether AI model training that has been done by individuals rather than huge organisations may have used training data that are not comprehensive enough to handle all contingencies encountered in an image
Overall, it is best to carefully use a combination of conventional and AI procedures for astronomical image processing, with constant reference to your original stacked image. Above all, one should not have blind faith in the results produced by Artificial Intelligence even if, in the end, they turn out to be acceptable.
As Darwin's 'bulldog', Thomas Huxley, the English biologist and anthropologist once said: “Skepticism is the highest duty and blind faith the one unpardonable sin.”
Using AI for Deconvolution
We shall look here at the results of three Deconvolution programs that use AI, but will concentrate on a new one called Theia, and compare its results with the other two, which are Seti Astro’s Cosmic Clarity and The PixInsight Process Blur Xterminator by R C Astro (Russel Croman Astrophotography).
Deconvolution is used to sharpen images, reducing blur and improving image clarity. It is the computational process of reversing the blurring effects introduced by a telescope’s optics and the atmosphere. It aims to recover the original image from a blurred or distorted version that has been convolved with a point spread function (PSF) which describes the blur, and corrupted by noise.
Estimating the PSF means reconstructing how a telescope and atmosphere smear a point source, the PSF. Some software uses adaptive PSF. Instead of assuming one static PSF, an adaptive model is built that adjusts to each region of the image. Stars are sampled at various locations to get PSF snapshots at multiple positions. Parameters are interpolated (size, ellipticity, halo fraction) between those samples. The locally tailored PSF is fed into the deconvolution routines.
The data we are going to use here are some unremarkable data on M17 captured with a Seestar S50. 458 x 10s exposures captured in EQ mode. The data were stacked in Pixinsight, Spectrophotometrically corrected, Automatic Dynamic background corrected and stretched in GIMP.
The original image untouched by AI
The image was then NoiseXterminator (AI) denoised and saved as a 32bit Fits file. The image was stretched in Gimp.
The Image at this stage
This linear 32 bit Fits file was deconvoluted in PixInsight with the AI BlurXterminator and finished in Gimp.
The BlurXterminator image
The 32 bit linear Fits file was deconvoluted in Seti Astro’s Cosmic Clarity in the SetiAstroSuite and finished in GIMP
The Cosmic Clarity image
Finally, the The 32 bit linear Fits file was deconvoluted in Theia
We determined that Theia, which is a Windows program would run perfectly in Wine with no delaying overheads. Theia is a standalone program, paid for but low cost. The GUI renders slightly different in Windows and in Fedora Linux.
Theia Windows version GUI
The Theia Linux (Wine) version GUI
The Theia Image
It will be noticed that each of the AI deconvoluted images of M17 is slightly different, and as Hamlet said ‘and therein lies the rub’. The soliloquy might have been ‘To AI or not to AI, that is the question”. For which, if any, of these AI renderings is correct? and are they more or less correct than say a conventional Richardson-Lucy deconvolution or a wavelet sharpening? Indeed, a Richardson-Lucy conventional deconvolution yields yet another variation on the theme:
Richardson-Lucy image
If we merge all three AI images with equal weight into an average AI image we get:
Average AI image
And, if we merge all three AI images and the Richardson-Lucy image with equal weight we get:
Three AIs plus Richardson-Lucy average image
Which image should we use?
By careful comparison with the original image, (untouched by AI or any other process that might make structural changes) non of the resulting images seem to contain any hallucinations in their structure, so the decision becomes ultimately an aesthetic one.
Maybe the fairest approach is to combine the results from the most pleasing deconvolutions having satisfied ourselves that nothing deleterious has happened to the image during the processing, AI or not.
As a parting comment I will mention a bizarre example of an AI hallucination that occurred when I was researching the Foraax palette in 2024. I asked a large language model AI (which shall remain nameless) about the origin of the name Foraax. It came up with a completely believable explanation. It said that Foraax was derived from a combination of the first names of the palette’s developers: Forrest Tanaka and Axel Mellinger both of whom are real amateur astronomers. However, I wrote to Axel Mellinger to ask him to confirm what the AI had said. Here is a slightly redacted copy of his reply:
Stephen,
while I do pursue astrophotography and have been doing a fair amount of
narrow-band imaging ***** I have no connection
with any Foraax palette. AI must have gotten it wrong this time!
*****
To the best of my knowledge, the ForaxX palette (note the different
spelling) was invented by someone with the Discord name "ForaxX":
*****
It's actually been on my to-do list to try out for a while.
Best regards,
Axel
The AI had skilfully constructed an answer from the names of two real astronomers in its AI hallucination. When I questioned it about its answer later on, it denied all knowledge of its answer and wasn’t able to help. Incidentally, there have been considerable improvements in large language models since this incident, and when asked the same question today, it produced a very detailed and correct answer.
The take home message is that if an AI had been working on an astronomical image, it could have hallucinated details that were not part of the original image data. Moreover, these details could have seemed completely authentic. Caveat emptor!
