For any airborne surveillance or reconnaissance unit, SAR teams or law enforcement aerial units, clear and well-defined imagery is vital to the mission’s success. Any distortion of an image can impact the accuracy of an operation and possibly even jeopardise the safety of the crew. In the worst-case scenario, this could cost lives.
This article will examine the effect known as ‘image artefacts’ and how new technology can help clarify and dehaze images. We will investigate the technology available to mission commanders and those who need that clarity of imagery in the field. We will also consider what this means for aircraft mission systems and how OPENSIGHT is responding to the issue of image artefacts with our own technology and image dehazing systems.
What are image artefacts?
Image artefacts are anomalies that can be seen in digital graphics and imagery, and in both stills and video capture. These artefacts can degrade and distort an image, rendering it unreliable at best and useless at worst. Image artefacts fall into several categories, each with their own particular issues. These include:
- Noise – this term is used quite commonly when it comes to image distortion. Noise includes variations in colour or brightness, especially in multi-spectral imagery. In digital imaging, noise shows up as a grainy or speckled appearance that can interfere with the clarity and definition of fine details.
- Chromatic Aberration – optical dispersion through a lens causes what is referred to as colour fringes. These are located at high-contrast boundaries (where a very dark object meets a very light area, for example.) CA can distort an image and reduce the area of clarity by hazing the edges of the frame.
- Distortion – this is a generic term for various image artefacts that can be the result of equipment malfunction or incorrect operational procedures.
- Compression artefacts – These tend to manifest during image compression, particularly when a file is saved as a JPEG or other format. Compression artefacts are quite diverse, ranging from colour degradation to blockiness and noise that can pixelate and distort an image.
What causes image artefacts?
The most common causes of image artefacts result from the capturing process. They can also arise during image processing or when a file is compressed for storage.
Compression distortion is one of the biggest challenges when processing and storing captured data. While JPEG and other storage systems reduce the amount of memory used to hold a file, they can also result in distortion as the image is compressed. In most cases, the image quality is acceptable. However, in certain instances, that compression process can significantly impact the quality of the image. Compression algorithms try to smooth these high-contrast edges, which can result in a halo effect or dotting.
The equipment used to capture the image at the source can also affect the usability or clarity of a picture or video. Other things that can directly influence the appearance of image artefacts include environmental factors (such as ambient temperature, overheating equipment) or incompatible software and hardware.
However, one of the most significant issues that airborne units, in particular, suffer from is the weather. Foggy conditions can dramatically reduce the clarity of imaging, regardless of how advanced the multi-spectral imaging camera being used.
Fog causes a hazy effect on images that can obscure not just fine details but also major landmarks, buildings, and target locations. Even if the aerial unit is above the fog level and flying in clear air, its equipment will struggle to punch through the layer of condensed moisture in the atmosphere and see the ground below.
Why does it matter?
Image clarity is crucial to operational success. In a search-and-rescue situation, aerial operators need as clear an image as possible to spot the slightest indication of a target’s location.
The same principle applies in a combat situation, where a badly distorted image could result in the wrong target being identified. Accurate and undistorted imagery is not only important but also mission-critical. While anomalies are factored into the equation, anything that can be done to remove them or eliminate their effect has to be implemented. This is where image dehazing comes in.
What is image dehazing?
To address the issue of visibility reduction caused by atmospheric conditions as well as image artefacts caused by other factors, several algorithms have been developed to tackle issues such as blockiness, distortion and visual ghosting. For external imaging, other factors can also cause images to degrade, such as fog, smoke and mist, resulting in colour distortion or, at its simplest, inability to see the target area. This, in turn, can have a marked impact on how an operator interprets an image.
Dehazing removes this fogging and restores the image without introducing any additional image artefacts in the process. This gives a sharper, clearer image that the operator can accurately interpret, whether in the field or back at a ground-based operating centre. Fundamentally, dehazing allows the software to improve the clarity of an image and reduce image artefacts or fragmenting.
Transforming images – Transformer-Based Image Restoration
At the heart of dehazing and image restoration is the groundbreaking Transformer-Based Image Restoration technology or Vision Transformers (ViT). This works to restore images to be as clear and artefact-free as possible, making them more useable regardless of mitigating factors such as poor visibility.
The importance of ViT cannot be underestimated. These effective systems have virtually replaced the previous Convolutional Neural Networks or CNNs. While CNNs still have a role in restoring low-quality imagery, ViTs are superseding them because of their more efficient nature, their ability to handle much larger data dumps and, most importantly, the additional level of feature learning that ViT demonstrates.
Transformer-based image restoration is also more flexible, allowing it to adjust to different restoration tasks with ease. The advanced learning process that ViT demonstrates means that it can effectively ‘learn on the job’, adapting and changing to real-world situations without being reprogrammed by the operator. Simply put, ViTs are plug-and-play systems that can be grandfathered onto existing and already familiar hardware. This also reduces the training time required to familiarise the operator with the system.
ViT supports a variety of Image Restoration tasks, including:
- Image Super-Resolution – to enhance an image beyond its previous limitations and to improve the quality by removing hazing and image artefacts
- Image Denoising – getting rid of one of the biggest issues with poor quality images, noise
- General Image Enhancement – improving the overall quality and clarity of an image
- Compression Artefact Reduction – removing image artefacts caused during the compression process when saving images as JPEG or other formatted files
- Deblurring – creating a cleaner, sharper image
- Adverse weather effects and Image Dehazing – clarifying and sharpening an image impacted by adverse weather conditions such as fog, smoke, or haze.
The advantages of Transformer-Based Image Restoration
One of the major advantages of Transformer-Based Image Restoration is that it can cope with larger datasets quickly and effectively. ViT’s ability to ‘learn’ on the job also enables it to adapt to changing parameters and real-time input variations. This is particularly important when coping with variable weather conditions such as fog or mist.
How FlySight is developing modular ViT for use in aerial reconnaissance
The advancements in image-dehazing technology are particularly applicable in aviation, giving operators a clearer image of targets and the ability to see through image obstacles such as morning fog, smoke or haze. The fact that this can not only ensure mission success but also minimise the risk of accidents (such as encountering unseen power lines or topographical obstacles) makes it an invaluable tool in all aircraft mission systems.
To this end, FlySight’s OPENSIGHT is embracing the cutting-edge technology of Augmented Reality and incorporating it into its turnkey Mission Console Solution. This AI system is designed to provide support for airborne payload operators, including the capture of high-resolution video flow.
By inserting ViT into this package, operators can complete their missions using improved geospatial situational awareness, better quality imaging and greater clarity, thanks to both image artefact elimination and a greater level of multiple information layers.
All of this technology uses modular systems that are compact enough to be used on airborne platforms. It also utilises already familiar hardware that operators intuitively understand and can quickly adapt to with minimal training.
Image artefacts may still affect some image capture, but with dehazing, everyday issues such as adverse weather conditions or operating theatres where smoke may be a problem can be reduced. The advancement of Transformer-Based Image Restoration is setting new standards in both military and civilian operations and has the potential to make a massive difference to mission success rates, as well as providing an enhanced level of safety for operators.
These reasons alone make the inclusion of ViT software an obvious choice for operators. The fact that it can be easily grandfathered into existing modular systems and uses equipment that is intuitive and easy to operate only excels its practicality for everything from Search & Rescue to military and law enforcement surveillance.
To learn more about OPENSIGHT, get in touch to learn more about this technology and the aircraft mission systems and software that Flysight.it offers.
