Infrared image iterative enhancement in frequency domain

Sergey Stankevich; Oleksandr Kondratov; Maksym Herda; Oleh Maslenko; Ievgen Saprykin

doi:10.3103/S0735272724070045

Authors

Sergey Stankevich Scientific Centre for Aerospace Research of the Earth, NAS of Ukraine, Kyiv, Ukraine https://orcid.org/0000-0002-0889-5764
Oleksandr Kondratov Defence Intelligence Research Institute, Kyiv, Ukraine https://orcid.org/0000-0002-6390-7158
Maksym Herda Defence Intelligence Research Institute, Kyiv, Ukraine https://orcid.org/0000-0002-6569-9284
Oleh Maslenko Defence Intelligence Research Institute, Kyiv, Ukraine https://orcid.org/0000-0001-6963-6574
Ievgen Saprykin Scientific Centre for Aerospace Research of the Earth, NAS of Ukraine, Kyiv, Ukraine https://orcid.org/0009-0004-2347-3756

DOI:

https://doi.org/10.3103/S0735272724070045

Keywords:

infrared image, iterative processing, frequency domain, enhancement core, isotropic resolution

Abstract

This paper proposes an approach to infrared image enhancement involving iterative wrapping of the enhancement core within an image upscale/downscale loop. It highlights and suppresses distortion and blurs fine image details. This is the aim of the current study. Therefore, the isotropic pixel resolution of the digital image is chosen as an evaluation criterion. This criterion attunes the pre-processing result to the visual interpretation requirements of the infrared image. All processing workflow is performed inside the frequency domain, which is beneficial regarding computational burden. The actual infrared UAV image processing demonstrates an improvement in isotropic pixel resolution by a factor of 1.9–2.1, which is slightly superior to known single-pass algorithms. The proposed approach is flexible and modular and can be easily implemented in ground-based and onboard infrared imagery processing.

Author Biography

Sergey Stankevich, Scientific Centre for Aerospace Research of the Earth, NAS of Ukraine, Kyiv

главный научный сотрудник

доктор технических наук

профессор

References

M. Vollmer, K. Möllmann, Infrared Thermal Imaging. Weinheim: Wiley, 2017, doi: https://doi.org/10.1002/9783527693306.

W. Si et al., “Recent advances in broadband photodetectors from infrared to terahertz,” Micromachines, vol. 15, no. 4, p. 427, 2024, doi: https://doi.org/10.3390/mi15040427.

F. Bao, S. Jape, A. Schramka, J. Wang, T. E. McGraw, Z. Jacob, “Why thermal images are blurry,” Opt. Express, vol. 32, no. 3, p. 3852, 2024, doi: https://doi.org/10.1364/OE.506634.

C. Lu, Y. Shi, “Infrared and visible image fusion: A survey of current research status,” in Proceedings of the 5th International Conference on Computer Information and Big Data Applications, 2024, pp. 883–887, doi: https://doi.org/10.1145/3671151.3671306.

S. A. Stankevich, M. S. Lubskyi, A. R. Lysenko, “Long-wave infrared remote sensing data spatial resolution enhancement using modulation transfer function fusion approach,” in 2021 International Conference on Information and Digital Technologies (IDT), 2021, pp. 89–94, doi: https://doi.org/10.1109/IDT52577.2021.9497630.

R. Dulski, P. Powalisz, M. Kastek, P. Trzaskawka, “Enhancing image quality produced by IR cameras,” in Proceedings of SPIE, 2010, p. 783415, doi: https://doi.org/10.1117/12.864979.

F. Hou, Y. Zhang, Y. Zhou, M. Zhang, B. Lv, J. Wu, “Review on infrared imaging technology,” Sustainability, vol. 14, no. 18, p. 11161, 2022, doi: https://doi.org/10.3390/su141811161.

H. Li, S. Wang, S. Li, H. Wang, S. Wen, F. Li, “Thermal infrared-image-enhancement algorithm based on multi-scale guided filtering,” Fire, vol. 7, no. 6, p. 192, 2024, doi: https://doi.org/10.3390/fire7060192.

Y. Li, L. Ma, S. Yang, Q. Fu, H. Sun, C. Wang, “Infrared image-enhancement algorithm for weak targets in complex backgrounds,” Sensors, vol. 23, no. 13, p. 6215, 2023, doi: https://doi.org/10.3390/s23136215.

J. Lange, T. Lange, “Optimal receive filter (Wiener filter),” in Fourier Transformation for Signal and System Description, Wiesbaden: Springer, 2022, pp. 51–56.

B. T. Swartz, H. Zheng, G. T. Forcherio, J. Valentine, “Broadband and large-aperture metasurface edge encoders for incoherent infrared radiation,” Sci. Adv., vol. 10, no. 6, 2024, doi: https://doi.org/10.1126/sciadv.adk0024.

A. Paul, T. Sutradhar, P. Bhattacharya, S. P. Maity, “Adaptive clip-limit-based bi-histogram equalization algorithm for infrared image enhancement,” Appl. Opt., vol. 59, no. 28, p. 9032, 2020, doi: https://doi.org/10.1364/AO.395848.

R. Yang, L. Chen, L. Zhang, Z. Li, Y. Lin, Y. Wu, “Image enhancement via special functions and its application for near infrared imaging,” Glob. Challenges, vol. 7, no. 7, 2023, doi: https://doi.org/10.1002/gch2.202200179.

W. Ma et al., “Infrared and visible image fusion technology and application: a review,” Sensors, vol. 23, no. 2, p. 599, 2023, doi: https://doi.org/10.3390/s23020599.

W. Ding, D. Bi, L. He, Z. Fan, “Infrared and visible image fusion method based on sparse features,” Infrared Phys. Technol., vol. 92, pp. 372–380, 2018, doi: https://doi.org/10.1016/j.infrared.2018.06.029.

Q. Fu, H. Fu, Y. Wu, “Infrared and visible image fusion based on mask and cross-dynamic fusion,” Electronics, vol. 12, no. 20, p. 4342, 2023, doi: https://doi.org/10.3390/electronics12204342.

E. A. Donia, E.-S. M. El-Rabaie, F. E. A. El-Samie, O. S. Faragallah, N. A. El-Hag, “Infrared image fusion for quality enhancement,” J. Opt., vol. 52, no. 2, pp. 658–664, 2023, doi: https://doi.org/10.1007/s12596-022-01018-4.

C. Sun, C. Zhang, N. Xiong, “Infrared and visible image fusion techniques based on deep learning: a review,” Electronics, vol. 9, no. 12, p. 2162, 2020, doi: https://doi.org/10.3390/electronics9122162.

J. Ma, W. Yu, P. Liang, C. Li, J. Jiang, “FusionGAN: A generative adversarial network for infrared and visible image fusion,” Inf. Fusion, vol. 48, pp. 11–26, 2019, doi: https://doi.org/10.1016/j.inffus.2018.09.004.

S. Zhong, L. Fu, F. Zhang, “Infrared image enhancement using convolutional neural networks for auto-driving,” Appl. Sci., vol. 13, no. 23, p. 12581, 2023, doi: https://doi.org/10.3390/app132312581.

L. Hu, L. Hu, M. Chen, “Edge-enhanced infrared image super-resolution reconstruction model under transformer,” Sci. Reports, vol. 14, no. 1, p. 15585, 2024, doi: https://doi.org/10.1038/s41598-024-66302-8.

K. Seshadrinathan et al., “Image quality assessment,” in The Essential Guide to Image Processing, Amsterdam: Elsevier, 2009, pp. 553–595.

M. Testolina, T. Ebrahimi, “Review of subjective quality assessment methodologies and standards for compressed images evaluation,” in Applications of Digital Image Processing XLIV, 2021, p. 37, doi: https://doi.org/10.1117/12.2597813.

D.-Y. Tsai, Y. Lee, E. Matsuyama, “Information entropy measure for evaluation of image quality,” J. Digit. Imaging, vol. 21, no. 3, pp. 338–347, 2008, doi: https://doi.org/10.1007/s10278-007-9044-5.

Y. Ding, S. Wang, D. Zhang, “Full‐reference image quality assessment using statistical local correlation,” Electron. Lett., vol. 50, no. 2, pp. 79–81, 2014, doi: https://doi.org/10.1049/el.2013.3365.

D. Varga, “No-reference image quality assessment with global statistical features,” J. Imaging, vol. 7, no. 2, p. 29, 2021, doi: https://doi.org/10.3390/jimaging7020029.

U. Sara, M. Akter, M. S. Uddin, “Image quality assessment through FSIM, SSIM, MSE and PSNR—A comparative study,” J. Comput. Commun., vol. 07, no. 03, pp. 8–18, 2019, doi: https://doi.org/10.4236/jcc.2019.73002.

A. Rubel, O. Ieremeiev, V. Lukin, J. Fastowicz, K. Okarma, “Combined no-reference image quality metrics for visual quality assessment optimized for remote sensing images,” Appl. Sci., vol. 12, no. 4, p. 1986, 2022, doi: https://doi.org/10.3390/app12041986.

B. Bondzulic, V. Petrovic, M. Andric, B. Pavlovic, “Gradient-based image quality assessment,” Acta Polytech. Hungarica, vol. 15, no. 4, pp. 83–99, 2018, doi: https://doi.org/10.12700/APH.15.4.2018.4.5.

S. Perfetto, J. Wilder, D. B. Walther, “Effects of spatial frequency filtering choices on the perception of filtered images,” Vision, vol. 4, no. 2, p. 29, 2020, doi: https://doi.org/10.3390/vision4020029.

J. C. Leachtenauer, W. Malila, J. Irvine, L. Colburn, N. Salvaggio, “General image-quality equation for infrared imagery,” Appl. Opt., vol. 39, no. 26, p. 4826, 2000, doi: https://doi.org/10.1364/AO.39.004826.

G. D. Boreman, Modulation Transfer Function in Optical and Electro-Optical Systems. Bellingham: SPIE, 2001, doi: https://doi.org/10.1117/3.419857.

R. Gaber, A. AbdElmgied, A. Kareem, “Performance evaluation of infrared image enhancement techniques,” Int. J. Adv. Comput. Sci. Technol., vol. 11, no. 2, pp. 1–11, 2022, doi: https://doi.org/10.30534/ijacst/2022/011122022.

D. K. Nagaiah, “Efficient performance analysis of image enhancement filtering methods using MATLAB,” Int. J. Innov. Technol. Explor. Eng., vol. 13, no. 2, pp. 1–5, 2024, doi: https://doi.org/10.35940/ijitee.B9777.13020124.

H. S. Prasantha, H. L. Shashidhara, K. N. B. Murthy, “Fast computation of image scaling algorithms using frequency domain approach,” in Advances in Intelligent Systems and Computing, New Delhi: Springer, 2013, pp. 201–208.

S. Sari, T. Shimamura, “Frequency domain Wiener filter for image denoising: derivation of a new power spectrum estimation method,” J. Signal Process., vol. 16, no. 1, pp. 79–85, 2012, doi: https://doi.org/10.2299/jsp.16.79.

O. Gazi, Understanding Digital Signal Processing, vol. 13. Singapore: Springer Singapore, 2018, doi: https://doi.org/10.1007/978-981-10-4962-0.

S. A. Stankevich, “Evaluation of the spatial resolution of digital aerospace image by the bidirectional point spread function parameterization,” in Advances in Intelligent Systems and Computing, Cham: Springer, 2021, pp. 317–327.

A. M. Haun, E. Peli, “Complexities of complex contrast,” in Proceedings of SPIE, 2012, p. 82920E, doi: https://doi.org/10.1117/12.915365.

A. R. Weeks, Fundamentals of Electronic Image Processing. Bellingham: SPIE, 1996, doi: https://doi.org/10.1117/3.227778.

C. Liu, S. Jia, H. Wu, D. Zeng, F. Cheng, S. Zhang, “A spatial-frequency domain associated image-optimization method for illumination-robust image matching,” Sensors, vol. 20, no. 22, p. 6489, 2020, doi: https://doi.org/10.3390/s20226489.

H. Bahonar, A. Mirzaei, S. Sadri, R. C. Wilson, “Graph embedding using frequency filtering,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 2, pp. 473–484, 2021, doi: https://doi.org/10.1109/TPAMI.2019.2929519.

Malik N.A., Chang C.-L., Chaudhary N.I., Kiani A.K. A review on computational heuristics in harmonics estimation: history, current topnotch, challenges and future prospects. Journal of Innovative Technology, 2024, vol. 6, no. 2, pp. 9-30. DOI: 10.29424/JIT.202409_6(2).0002

Y. Liu, R. C. Paffenroth, “CNNs in the frequency domain for image super-resolution,” in 2019 International Conference on Image and Video Processing, and Artificial Intelligence, 2019, p. 40, doi: https://doi.org/10.1117/12.2539288.

M. T. Pham, V. Q. Nguyen, C. D. Hoang, H. L. Vo, D. K. Phan, A. H. Nguyen, “Efficient complex valued neural network with Fourier transform on image denoising,” in The 5th International Conference on Future Networks & Distributed Systems, 2021, pp. 48–57, doi: https://doi.org/10.1145/3508072.3508081.

J. Lou, J. Ji, Q. Zhou, X. Li, “Research and analysis of infrared image enhancement algorithm based on fractional differentiation,” J. Phys. Conf. Ser., vol. 2187, no. 112049, 2022, doi: https://doi.org/10.1088/1742-6596/2187/1/012049.