Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Deep architecture for super-resolution and deblurring of text images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Image deblurring and super resolution attempts to restore images that have been degraded. We propose a joint technique for super resolution and deblurring to solve the problem of blur and low resolution in text images. This joint technique is based on the use of a Deep Convolutional Neural Network (Deep CNN). Deep CNN has achieved promising performance for single image super-resolution. In particular, the Deep CNN skip Connection and Network in Network (DCSCN) architecture has been successfully applied to natural images super-resolution. In this work we propose a model that jointly performs super-resolution and deblurring of low-resolution blurry text images based on DCSCN. Our model uses subsampled blurry images in the input and original sharp images as ground truth. The proposed architecture consists of a higher number of filters in the input CNN layer to a better analysis of the text details. The experimental results have achieved state-of-the-art performance in the peak-signal-to-noise ratio (PSNR), the structural similarity index measure (SSIM), the information fidelity criterion (IFC) and Visual Information Fidelity (VIF) metrics. Thus, we confirm that DCSCN provides satisfactory results for enhancement tasks on low blurry images. The quantitative and qualitative evaluation on different datasets proves the high performance of our model to reconstruct high-resolution and sharp text images with PSNR= 20.406, SSIM= 0.877, VIF= 0.351, IFC= 2.868 for scale 4 compared to DCSCN with PSNR= 15.553, SSIM= 0.621, VIF= 0.166 IFC= 1.129. In addition, in terms of computational time, our proposed method gives competitive performance compared to state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The datasets generated and analysed during during the experiments reported in Section 4 are available in the Figshare repository, https://doi.org/10.6084/m9.figshare.20049029. The metrics shown in Tables 2, 3, 4 and 5 are computed on these datasets.

Notes

  1. https://pypi.org/project/pytesseract/

References

  1. Albluwi F, Krylov V A, Dahyot R (2019) Super-resolution on degraded low-resolution images using convolutional neural networks. In: 2019 27th European signal processing conference (EUSIPCO). IEEE, pp 1–5

  2. Bansal M, Kumar M, Sachdeva M, Mittal A (2021) Transfer learning for image classification using vgg19: Caltech-101 image data set. J Ambient Intell Humaniz Comput, 1–12

  3. Bansal S, Bansal RK, Kumar R (2022) A novel upgraded uniform embedding technique for jpeg steganography. In: Intelligent sustainable systems. Springer, pp 723–730

  4. Dalal M, Juneja M (2018) Video steganography techniques in spatial domain—a survey. In: Proceedings of the international conference on computing and communication systems. Springer, pp 705–711

  5. Du B, Ren X, Chen S, Ren J, Cao D (2019) Image super-resolution and deblurring using generative adversarial network. In: Proceedings of the 2019 8th international conference on computing and pattern recognition, pp 266–271

  6. Du B, Ren X, Ren J (2019) CNN-based image super-resolution and deblurring. In: Proceedings of the 2019 international conference on video, signal and image processing, pp 70–74

  7. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034

  8. Hradiš M, Kotera J, Zemcık P, Šroubek F (2015) Convolutional neural networks for direct text deblurring. In: Proceedings of BMVC, vol 10, p 2

  9. Jiang K, Wang Z, Yi P, Jiang J (2020) Hierarchical dense recursive network for image super-resolution. Pattern Recogn 107:Article 107475

    Article  Google Scholar 

  10. Jiang X, Yao H, Zhao S (2017) Text image deblurring via two-tone prior. Neurocomputing 242:1–14

    Article  Google Scholar 

  11. Kaur R P, Jindal MK, Kumar M (2021) Text and graphics segmentation of newspapers printed in gurmukhi script: a hybrid approach. Vis Comput 37(7):1637–1659

    Article  Google Scholar 

  12. Li Y, Yang Z, Mao X, Wang Y, Li Q, Liu W, Wang Y (2019) GAN with pixel and perceptual regularizations for photo-realistic joint deblurring and super-resolution. In: Computer graphics international conference. Springer, pp 395–401

  13. Liang Z, Zhang D, Shao J (2019) Jointly solving deblurring and super-resolution problems with dual supervised network. In: 2019 IEEE International conference on multimedia and expo (ICME). IEEE, pp 790–795

  14. Lin M, Chen Q, Yan S (2013) Network in network. arXiv:1312.4400

  15. Liu H, Qin J, Fu Z, Li X, Han J (2020) Fast simultaneous image super-resolution and motion deblurring with decoupled cooperative learning. J Real-time Image Process 17:1787–1800

    Article  Google Scholar 

  16. Ljubenovic M, Zhuang L, Figueiredo MAT (2017) Class-adapted blind deblurring of document images. In: 2017 14th IAPR International conference on document analysis and recognition (ICDAR), vol 1. IEEE, pp 721–726

  17. Lumentut J S, Park I K (2020) Deep neural network for joint light field deblurring and super-resolution. In: International workshop on advanced imaging technology (IWAIT) 2020, vol 11515. International Society for Optics and Photonics, p 1151507

  18. Mei Y, Fan Y, Zhou Y, Huang L, Huang T S, Shi H (2020) Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5690–5699

  19. Neji H, Halima M B, Hamdani T M, Nogueras-Iso J, Alimi A M (2021) Blur2sharp: a gan-based model for document image deblurring. Int J Comput Intell Syst 14(1):1315–1321

    Article  Google Scholar 

  20. Pan J, Sun D, Pfister H, Yang M-H (2016) Blind image deblurring using dark channel prior. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1628–1636

  21. Quan Y, Yang J, Chen Y, Xu Y, Ji H (2020) Collaborative deep learning for super-resolving blurry text images. IEEE Trans Comput Imag 6:778–790

    Article  MathSciNet  Google Scholar 

  22. Shafiei F, Fekri-Ershad S (2020) Detection of lung cancer tumor in ct scan images using novel combination of super pixel and active contour algorithms. Traitement du Signal 37:6

    Article  Google Scholar 

  23. Shaheed K, Mao A, Qureshi I, Kumar M, Hussain S, Ullah I, Zhang X (2022) Ds-cnn: a pre-trained xception model based on depth-wise separable convolutional neural network for finger vein recognition. Expert Syst Appl 191:116288

    Article  Google Scholar 

  24. Sheikh H R, Bovik A C (2006) Image information and visual quality. IEEE Trans Image Process 15(2):430–444

    Article  Google Scholar 

  25. Sheikh H R, Bovik A C, De Veciana G (2005) An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Trans Image Process 14(12):2117–2128

    Article  Google Scholar 

  26. Sun J, Xu Z, Shum H-Y (2008) Image super-resolution using gradient profile prior. In: 2008 IEEE Conference on computer vision and pattern recognition. IEEE, pp 1–8

  27. Timofte R, Smet V D, Gool L V (2014) A+: adjusted anchored neighborhood regression for fast super-resolution. In: Asian conference on computer vision. Springer, pp 111–126

  28. Tran Hanh TM, Ho-Phuoc T (2019) Deep Laplacian pyramid network for text images super-resolution. In: 2019 IEEE-RIVF International conference on computing and communication technologies (RIVF). IEEE, pp 1–6

  29. Wang C, Wang X, Zhang J, Zhang L, Bai X, Ning X, Zhou J, Hancock E (2022) Uncertainty estimation for stereo matching based on evidential deep learning. Pattern Recogn 124:108498

    Article  Google Scholar 

  30. Wang S, Zhang L, Liang Y, Pan Q (2012) Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2216–2223

  31. Wang Z, Bovik A C (2009) Mean squared error: love it or leave it? A new look at signal fidelity measures. IEEE Signal Process Mag 26(1):98–117

    Article  Google Scholar 

  32. Wang Z, Bovik A C, Sheikh H R, Simoncelli E P (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  33. Xu X, Sun D, Pan J, Zhang Y, Pfister H, Yang M-H (2017) Learning to super-resolve blurry face and text images. In: Proceedings of the IEEE international conference on computer vision, pp 251–260

  34. Yamanaka J, Kuwashima S, Kurita T (2017) Fast and accurate image super resolution by deep CNN with skip connection and network in network. In: International conference on neural information processing. Springer, pp 217–225

  35. Yang C-Y, Liu S, Yang M-H (2013) Structured face hallucination. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1099–1106

  36. Yang C-Y, Yang M-H (2013) Fast direct super-resolution by simple functions. In: Proceedings of the IEEE international conference on computer vision, pp 561–568

  37. Yang C-H, Chang L-W (2020) Deblurring and super-resolution using deep gated fusion attention networks for face images. In: ICASSP 2020-2020 IEEE International conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 1623–1627

  38. Yang J, Wright J, Huang T S, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  Google Scholar 

  39. Yun J U, Jo B, Park I K (2020) Joint face super-resolution and deblurring using generative adversarial network. IEEE Access 8:159661–159671

    Article  Google Scholar 

  40. Zeng K, Zheng H, Qu Y, Qu X, Bao L, Chen Z (2018) Single image super-resolution with learning iteratively non-linear mapping between low-and high-resolution sparse representations. In: 2018 24th International conference on pattern recognition (ICPR). IEEE, pp 507–512

  41. Zhang H, Yang J, Zhang Y, Huang T S (2010) Non-local kernel regression for image and video restoration. In: European conference on computer vision. Springer, pp 566–579

  42. Zhang X, Dong H, Hu Z, Lai W-S, Wang F, Yang M-H (2018) Gated fusion network for joint image deblurring and super-resolution. In: British machine vision conference 2018, BMVC 2018, Newcastle, UK, September 3-6, 2018. BMVA Press

  43. Zhang X, Wang F, Dong H, Guo Y (2018) A deep encoder-decoder networks for joint deblurring and super-resolution. In: 2018 IEEE International conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 1448–1452

Download references

Acknowledgements

We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Quadro P6000 GPU used for this research.

Funding

The research leading to these results has been partially supported by the Ministry of Higher Education and Scientific Research of Tunisia under the grant agreement number LR11ES48, the Spanish Regional Government of Aragon (project T59_23R), and the Spanish Ministry of Science and Innovation (project PID2020-113353RB-I00). We deeply acknowledge Taif University for Supporting this study through Taif University Researchers Supporting Project number (TURSP-2020/327), Taif University, Taif, Saudi Arabia.

Author information

Authors and Affiliations

  1. National Engineering School of Gabes (ENIG), University of Gabes, Gabes, Tunisia

    Hala Neji

  2. REGIM Lab (Research Groups in Intelligent Machines), University of Sfax, National Engineering School of Sfax (ENIS), BP 1173, 3038, Sfax, Tunisia

    Mohamed Ben Halima, Tarek M. Hamdani & Adel M. Alimi

  3. Aragon Institute of Engineering Research (I3A), University of Zaragoza, Zaragoza, Spain

    Javier Nogueras-Iso

  4. Department of Computer Science, College of Computers and Information Technology, Taif University, P.O.Box. 11099, Taif, 21944, Saudi Arabia

    Abdulrahman M. Qahtani

  5. College of Applied Computer Science, King Saud University, Riyadh, Saudi Arabia

    Omar Almutiry & Habib Dhahri

  6. Department of Electrical and Electronic Engineering Science, Faculty of Engineering and the Built Environment, University of Johannesburg, Johannesburg, South Africa

    Adel M. Alimi

Authors
  1. Hala Neji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed Ben Halima
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Javier Nogueras-Iso
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tarek M. Hamdani
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Abdulrahman M. Qahtani
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Omar Almutiry
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Habib Dhahri
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Adel M. Alimi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hala Neji.

Ethics declarations

Conflict of Interests

The authors declare that they have no competing financial interests or personal relationships that could have influenced the work reported in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Neji, H., Halima, M.B., Nogueras-Iso, J. et al. Deep architecture for super-resolution and deblurring of text images. Multimed Tools Appl 83, 3945–3961 (2024). https://doi.org/10.1007/s11042-023-15340-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15340-x

Keywords

Search

Navigation