Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

RVPNet: A real time unstructured road vanishing point detection algorithm using attention mechanism and global context information

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The detection of the vanishing point (VP) in unstructured road is crucial for the advancement of autonomous vehicle technology. However, due to the inadequate fusion of intra-level features and high computational requirements of existing CNN-based road VP detection methods, a model named RVPNet is proposed in this paper. To begin, the proposed algorithm adopts the architecture of encoder-decoder combined lightweight backbone to extract unstructured road features efficiently. Second, the Simple Residual Pyramid Pooling Module (SRPPM) is designed in this model to obtain cross-path global contextual information with low computational cost. And a Dual Attention-based Feature Aggregation Module (DAFAM) is proposed to obtain better inter-level feature representations. Finally, the offset loss is introduced to compensate for the inherent offset errors caused by the output stride of the heatmap. The experimental results show that the average detection error rate of our approach is only 0.03128 on the Kong dataset, and the average processing time reaches 238 FPS. The average detection error rate of our approach based is only 0.03600 on the Moghhadam dataset. Compared with the state-of-the-art methods, the proposed approach achieves the highest detection accuracy and speed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The datasets generated during and analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Alon Y, Ferencz A, Shashua A (2006) Off-road path following using region classification and geometric projection constraints. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), vol 1. IEEE, pp 689–696

  2. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  PubMed  Google Scholar 

  3. Bang S, Park S, Kim H, Kim H (2019) Encoder–decoder network for pixellevel road crack detection in black-box images. Comput-Aided Civil Infrastruct Eng 34(8):713–727

    Article  Google Scholar 

  4. Cai Y, Wang Z, Luo Z, Yin B, Du A, Wang H, Zhang X, Zhou X, Zhou E, Sun J (2020) Learning delicate local representations for multiperson pose estimation. In: Computer vision–ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16. Springer, pp 455–472

  5. Chang C-K, Zhao J, Itti L (2018) Deepvp: Deep learning for vanishing point detection on 1 million street view images. In: 2018 IEEE international conference on robotics and automation (ICRA). IEEE, pp 4496–4503

  6. Chen GH, Zhou W, Wang FJ, Xiao BJ, Dai SF (2013) Lane detection based on improved canny detector and least square fitting. In: Advanced materials research, vol 765. Trans Tech Publ, pp 2383–2387

  7. Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  PubMed  Google Scholar 

  8. Ding W, Li Y (2015) Efficient vanishing point detection method in complex urban road environments. IET Comput Vision 9(4):549–558

    Article  Google Scholar 

  9. Ebrahimpour R, Rasoolinezhad R, Hajiabolhasani Z, Ebrahimi M (2012) Vanishing point detection in corridors: using hough transform and k-means clustering. IET Comput Vision 6(1):40–51

    Article  MathSciNet  Google Scholar 

  10. Fei J, Peng K, Heidenreich P, Bieder F, Stiller C (2021) Pillarsegnet: Pillar-based semantic grid map estimation using sparse lidar data. In: 2021 IEEE intelligent vehicles symposium (IV). IEEE, pp 838–844

  11. Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 3146–3154

  12. Huang Z, Wei Y, Wang X, Liu W, Huang TS, Shi H (2021) Alignseg: feature-aligned segmentation networks. IEEE Trans Pattern Anal Mach Intell 44(1):550–557

    PubMed  Google Scholar 

  13. Huang S, Lu Z, Cheng R, He C (2021) Fapn: feature-aligned pyramid network for dense image prediction. In: Proceedings of the IEEE/CVF international conference on computer vision. pp 864–873

  14. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 7132–7141

  15. Ji Y, Zhang H, Zhang Z, Liu M (2021) Cnn-based encoder-decoder networks for salient object detection: a comprehensive review and recent advances. Inf Sci 546:835–857

    Article  MathSciNet  Google Scholar 

  16. Kingma DP, Ba JA, Adam J (2020) A method for stochastic optimization. arxiv 2014. 106. arXiv:1412.6980

  17. Kocur V, Ftáčnik M (2021) Traffic camera calibration via vehicle vanishing point detection. In: Artificial neural networks and machine learning–ICANN 2021: 30th international conference on artificial neural networks, Bratislava, Slovakia, September 14–17, 2021, Proceedings, Part V 30. Springer, pp 628–639

  18. Kogan H, Maurer R, Keshet R (2009). Vanishing points estimation by selfsimilarity. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE, pp 755–761

  19. Kong H, Audibert J-Y, Ponce J (2010) General road detection from a single image. IEEE Trans Image Process 19(8):2211–2220

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  20. Kong H, Akakin HC, Sarma SE (2013) A generalized laplacian of gaussian filter for blob detection and its applications. IEEE Trans Cybern 43(6):1719–1733

    Article  PubMed  Google Scholar 

  21. Kortli Y, Marzougui M, Bouallegue B, Bose JSC, Rodrigues P, Atri M (2017) A novel illumination-invariant lane detection system. In: 2017 2nd international conference on anti-cyber crimes (ICACC). IEEE, pp 166– 171

  22. Law H, Deng J (2018) Cornernet: detecting objects as paired keypoints. In: Proceedings of the European conference on computer vision (ECCV). pp 734–750

  23. Lee J, Kim D, Ponce J, Ham B (2019) Sfnet: learning object-aware semantic correspondence. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 2278–2287

  24. Lee S, Kim J, Shin Yoon J, Shin S, Bailo O, Kim N, Lee T- H, Seok Hong H, Han S-H, So Kweon I (2017) Vpgnet: vanishing point guided network for lane and road marking detection and recognition. In: Proceedings of the IEEE international conference on computer vision. pp 1947–1955

  25. Li Y, Tong G, Sun A, Ding W (2018) Road extraction algorithm based on intrinsic image and vanishing point for unstructured road image. Robot Auton Syst 109:86–96

    Article  Google Scholar 

  26. Liu Y-B, Zeng M, Meng Q-H (2020) Unstructured road vanishing point detection using convolutional neural networks and heatmap regression. IEEE Trans Instrum Meas 70:1–8

    Google Scholar 

  27. Liu N, Han J, Yang M-H (2018) Picanet: Learning pixel-wise contextual attention for saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 3089–3098

  28. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 3431–3440

  29. Moghadam P, Starzyk JA, Wijesoma WS (2011) Fast vanishing-point detection in unstructured environments. IEEE Trans Image Process 21(1):425–430

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  30. Newell A, Yang K, Deng J (2016) Stacked hourglass networks for human pose estimation. In: Computer vision–ECCV 2016: 14th European conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VIII 14. Springer, pp 483–499

  31. Peng K, Fei J, Yang K, Roitberg A, Zhang J, Bieder F, Heidenreich P, Stiller C, Stiefelhagen R (2022) Mass: multi-attentional semantic segmentation of lidar data for dense top-view understanding. IEEE Trans Intell Transp Syst 23(9):15824–15840

    Article  Google Scholar 

  32. Peng J, Liu Y, Tang S, Hao Y, Chu L, Chen G, Wu Z, Chen Z, Yu Z, Du Y et al. (2022) Pp-liteseg: a superior real-time semantic segmentation model. arXiv:2204.02681

  33. Qin Z, Zhang P, Wu F, Li X (2021) Fcanet: Frequency channel attention networks. In: Proceedings of the IEEE/CVF international conference on computer vision. pp 783–792

  34. Rasmussen C (2008) Roadcompass: following rural roads with vision+ ladar using vanishing point tracking. Auton Robots 25:205–229

    Article  Google Scholar 

  35. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer, pp 234–241

  36. Shen S, Wang S, Wang L, Wei H (2022) A refined-line-based method to estimate vanishing points for vision-based autonomous vehicles. Vehicles 4(2):314–325

    Article  Google Scholar 

  37. Shruthiba A, Deepu R, Dhomne A (2021) A cnn approach to detect vanishing point by considering street lane. In: 2021 IEEE international conference on computation system and information technology for sustainable solutions (CSITSS). pp 1–5. https://doi.org/10.1109/CSITSS54238.2021.9683182

  38. Shuai Y, Tiantian Y, Guodong Y, Zize L (2017) Regression convolutional network for vanishing point detection. In: 2017 32nd youth academic annual conference of Chinese association of automation (YAC). IEEE, pp 634– 638

  39. Song Q, Mei K, Huang R (2021) Attanet: attention-augmented network for fast and accurate scene parsing. Proceedings of the AAAI Conference on Artificial Intelligence 35:2567–2575

    Article  Google Scholar 

  40. Varma G, Subramanian A, Namboodiri A, Chandraker M, Jawahar C (2019) Idd: a dataset for exploring problems of autonomous navigation in unconstrained environments. In: 2019 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1743–1751

  41. Wang C-Y, Bochkovskiy A, Liao H-YM (2022) Yolov7: trainable bag-offreebies sets new state-of-the-art for real-time object detectors. arXiv:2207.02696

  42. Wang Y, Shen D, Teoh EK (2000) Lane detection using spline model. Pattern Recognit Lett 21(8):677–689

    Article  ADS  Google Scholar 

  43. Wang Y, Teoh EK, Shen D (2004) Lane detection and tracking using bsnake. Image and Vision Computing 22(4):269–280

    Article  Google Scholar 

  44. Wang P, Fang Z, Zhao S, Chen Y, Zhou M, An S (2021) Vanishing point aided lidar-visual-inertial estimator. In: 2021 IEEE international conference on robotics and automation (ICRA). IEEE, pp 13120–13126

  45. Wang W, Shen J, Dong X, Borji A (2018) Salient object detection driven by fixation prediction. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1711–1720

  46. Wang Q, Wu B, Zhu P, Li P, Zuo W, Hu Q (2020) Eca-net: efficient channel attention for deep convolutional neural networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 11534–11542

  47. Woo S, Park J, Lee J-Y, Kweon IS (2018) Cbam: convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV). pp 3–19

  48. Wu P-C, Chang C-Y, Lin CH (2014) Lane-mark extraction for automobiles under complex conditions. Pattern Recognit 47(8):2756–2767

    Article  ADS  Google Scholar 

  49. Wu Z, Fu W, Xue R, Wang W (2016) A novel line space voting method for vanishing-point detection of general road images. Sensors 16(7):948

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  50. Yang G, Wang Y, Yang J, Lu Z (2019) Fast and robust vanishing point detection using contourlet texture detector for unstructured road. IEEE Access 7:139358–139367

    Article  Google Scholar 

  51. Yu C, Xiao B, Gao C, Yuan L, Zhang L, Sang N, Wang J (2021) Lite-hrnet: a lightweight high-resolution network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 10440–10450

  52. Yu Z, Zhu L (2019) Roust vanishing point detection based on the combination of edge and optical flow. In: 2019 4th Asia-pacific conference on intelligent robot systems (ACIRS). pp 184–188. https://doi.org/10.1109/ACIRS.2019.8936016

  53. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2019) Unet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging 39(6):1856–1867

    Article  PubMed  PubMed Central  Google Scholar 

  54. Zhou X, Wang D, Krähenbühl, P (2019) Objects as points. arXiv:1904.07850

Download references

Acknowledgements

This research was funded by the Natural Science Foundation of Shandong Province for Key Project under Grant ZR2020KF006, the National Natural Science Foundation of China under Grant 62273164, the Development Program Project of Youth Innovation Team of Institutions of Higher Learning in Shandong Province, and the China Postdoctoral Science Foundation under Grant 2019M662407.

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, University of Jinan, Jinan, 250022, China

    Yu Liu, Xue Fan, Shiyuan Han, Jin Zhou, Xiaohui Yang & Zhongtao Li

Authors
  1. Yu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xue Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shiyuan Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jin Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaohui Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhongtao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shiyuan Han.

Ethics declarations

Conflicts of interests/competing interests

The authors have no financial or proprietary interests in any material discussed in this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Yu Liu and Xue Fan contributed equally to this work.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Y., Fan, X., Han, S. et al. RVPNet: A real time unstructured road vanishing point detection algorithm using attention mechanism and global context information. Multimed Tools Appl 83, 28263–28280 (2024). https://doi.org/10.1007/s11042-023-16447-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16447-x

Keywords

Search

Navigation