Study on Road Weather Recognition Method Based on Road Segmentation
吕慈明1, 刘电2, 张秀杰2
1. Beijing Zhuhai North Branch of Guangdong Expressway Co., Ltd., Shaoguan Guangzhou 512737, China; 2. Guangzhou Run One Traffic Information Co., Ltd., Guangzhou Guangdong 510665, China
Study on Road Weather Recognition Method Based on Road Segmentation
LÜ Ming-ci1, LIU Dian2, ZHANG Xiu-jie2
1. Beijing Zhuhai North Branch of Guangdong Expressway Co., Ltd., Shaoguan Guangzhou 512737, China; 2. Guangzhou Run One Traffic Information Co., Ltd., Guangzhou Guangdong 510665, China
摘要In order to realize the accurate recognition of weather images in road scenes, a road weather recognition method based on road segmentation is proposed, and a sort of road segmentation fusion network (RSFN), with the overall road weather image features and road features, is established by designing a method for extracting road area characteristics combining with the semantic segmentation model. First, the original images are preprocessed through the road segmentation network to obtain the binary images, and the road area information is obtained by using the convolutional feature mask (CFM). Subsequently, a convolutional neural network, which is composed of overall network branches and road network branches, is established and used to extract the overall image area features and focus on extracting the road weather features respectively. In view of the extracted irregular road characteristics, the overall image features and the road local features are fused with CFM. Finally, the weather recognition of key road areas is carried out through a fully connected layer, and the recognition of 5 types of weather (cloudy, sunny, foggy, rainy, and snowy) is realized considering the overall weather recognition. A road multi-class weather dataset (RMWD) is established by collecting real surveillance videos of expressway in multiple urban areas with different road sections and weather conditions, and compared with different network models on testing results. The result shows that (1) under the situation of decrease in parameters and computations, the RSFN weather recognition algorithm has accuracy rate and recall rate of 85.40% and 80.30% that improved by at least 3.97% and 3.86% respectively; (2) the key areas of extracting features from network models are placed in roads by using the road weather recognition method based on road segmentation, and the effective extraction of road weather features is realized by using RSFN algorithm, which can be applied to real-time and accurate weather recognition in road scenes effectively.
Abstract：In order to realize the accurate recognition of weather images in road scenes, a road weather recognition method based on road segmentation is proposed, and a sort of road segmentation fusion network (RSFN), with the overall road weather image features and road features, is established by designing a method for extracting road area characteristics combining with the semantic segmentation model. First, the original images are preprocessed through the road segmentation network to obtain the binary images, and the road area information is obtained by using the convolutional feature mask (CFM). Subsequently, a convolutional neural network, which is composed of overall network branches and road network branches, is established and used to extract the overall image area features and focus on extracting the road weather features respectively. In view of the extracted irregular road characteristics, the overall image features and the road local features are fused with CFM. Finally, the weather recognition of key road areas is carried out through a fully connected layer, and the recognition of 5 types of weather (cloudy, sunny, foggy, rainy, and snowy) is realized considering the overall weather recognition. A road multi-class weather dataset (RMWD) is established by collecting real surveillance videos of expressway in multiple urban areas with different road sections and weather conditions, and compared with different network models on testing results. The result shows that (1) under the situation of decrease in parameters and computations, the RSFN weather recognition algorithm has accuracy rate and recall rate of 85.40% and 80.30% that improved by at least 3.97% and 3.86% respectively; (2) the key areas of extracting features from network models are placed in roads by using the road weather recognition method based on road segmentation, and the effective extraction of road weather features is realized by using RSFN algorithm, which can be applied to real-time and accurate weather recognition in road scenes effectively.
吕慈明, 刘电, 张秀杰. Study on Road Weather Recognition Method Based on Road Segmentation[J]. Journal of Highway and Transportation Research and Development, 2023, 17(3): 26-35.
LÜ Ming-ci, LIU Dian, ZHANG Xiu-jie. Study on Road Weather Recognition Method Based on Road Segmentation. Journal of Highway and Transportation Research and Development, 2023, 17(3): 26-35.
[1] LEE J, LEE M I, AHN J B. Importance of Ocean Initial Conditions of Late Autumn on Winter Seasonal Prediction Skill in Atmosphere-land-ocean-sea Ice Coupled Forecast System[J]. Climate Dynamics, 2022,58(11/12):3427-3440. [2] LI Xiao-yong, LIU Kuan-zong, ZHOU Xiang-hua, et al. Study on Weather Phenomena Identification Based on Lightweight Convolutional Neural Network[J]. Information Recording Materials, 2021,22(8):78-80.(in Chinese) [3] XIE K, HUANG L, ZHANG W, et al. A CNN-based Multi-task Framework for Weather Recognition with Multi-s cale Weather Cues[J]. Expert Systems with Application, 2022,198:116689. [4] HOU Hui-rong. Study on Method for Weather Recognition Based on Feature Selection[D]. Changchun:Northeast Normal University, 2017.(in Chinese) [5] CORDTS M, OMRAN M, RAMOS S, et al. The Cityscapes Dataset for Semantic Urban Scene Understanding[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas:IEEE, 2016:3213-3223. [6] ZHOU B, ZHAO H, PUIG X, et al. Semantic Understanding of Scenes through the ADE20K Dataset[J]. International Journal of Computer Vision, 2019, 127(3):302-321. [7] HE K, ZHANG X, REN S, et al. Deep Residual Learning for Image Recognition[J]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas:IEEE, 2016:770-778. [8] ZHAO H, SHI J, QI X, et al. Pyramid Scene Parsing Network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu:IEEE, 2017:2881-2890. [9] QIANG B, CHEN R, ZHOU M, et al. Convolutional Neural Networks-based Object Detection Algorithm by Jointing Semantic Segmentation for Images[J]. Sensors, 2020, 20(18):5080. [10] WANG Zhi-hong, WANG Shao-bo, YAN Li-rong, et al. Pavement Type Recognition Based on Semantic Segmentation Model[J]. Journal of Highway and Transportation Research and Development, 2021,38(1):128-134.(in Chinese) [11] HE K, ZHANG X, REN S, et al. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9):1904-1916. [12] ZHAO J, FANG Y, LI G. Recurrence along Depth:Deep Convolutional Neural Networks with Recurrent Layer Aggregation[C]//35th Conference on Neural Information Processing Systems (NeurIPS), Virtual Conference. Sydney:Neural Information Processing Systems Foundation, Inc., 2021:11852. [13] ZUO Jie-ge, LIU Xiao-ming, CAI Bing. Outdoor Image Weather Recognition Based on Image Blocks and Feature Fusion[J]. Computer Science, 2022,49(3):197-203.(in Chinese) [14] LU Z, WHALEN I, DHEBAR Y, et al. Multi-objective Evolutionary Design of Deep Convolutional Neural Networks for Image Classification[J]. IEEE Transactions on Evolutionary Computation,2021, 25(2):277-291. [15] WU Z, HE S. Improvement of the AlexNet Networks for Large-scale Recognition Applications[J]. Iranian Journal of Science and Technology, Transactions of Electrical Engineering, 2021,45(2):493-503. [16] HASSON Y, VAROL G, TZIONAS D, et al. Learning Joint Reconstruction of Hands and Manipulated Objects[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach:IEEE, 2019:11807-11816. [17] CHATTOPADHYAY A, SARKAR A, HOWLADER P, et al. Grad-CAM++:Generalized Gradient-based Visual Explanations for Deep Convolutional Networks[J]. Computer Vision and Pattern Recognition, 2018(3):11063. [18] SATTARZADEH S, SUDHAKAR M, PLATANIOTIS K N, et al. Integrated Grad-CAM:Sensitivity-aware Visual Explanation of Deep Convolutional Networks via Integrated Gradient-based Scoring[C]//IEEE International Conference on Acoustics, Speech and Signal Processing. Toronto:IEEE,2021:1775-1779.
2023, Vol. 17 
