Automatic detection and recognition of traffic signs plays a crucial role in management of the traffic-sign inventory. It provides an accurate and timely way to manage traffic-sign inventory with a minimal human effort. In the computer vision community the recognition and detection of traffic signs is a well-researched problem. A vast majority of existing approaches perform well on traffic signs needed for advanced driver-assistance and autonomous systems. However, this represents a relatively small number of all traffic signs (around 50 categories out of several hundred) and performance on the remaining set of traffic signs, which are required to eliminate the manual labor in traffic-sign inventory management, remains an open question.
We address the issue of detecting and recognizing a large number of traffic-sign categories suitable for automating traffic-sign inventory management. We adopt a convolutional neural network (CNN) approach, the Mask R-CNN, to address the full pipeline of detection and recognition with automatic end-to-end learning. We propose several improvements that are evaluated on the detection of traffic signs and result in an improved overall performance. This approach is applied to detection of 200 traffic-sign categories represented in our novel dataset, the DFG-TS dataset. Results are reported on highly challenging traffic-sign categories that have not yet been considered in previous works. With an error rates below 3% the proposed approach is sufficient for deployment in practical applications of the traffic-sign inventory management.
The DFG-TS dataset
We propose a novel challenging dataset with 200 traffic-sign categories spread over 13000 traffic-sign instances and 7000 high-resolution images. The dataset represents a novel benchmark for a complex traffic-sign detection and recognition task with a large number of classes having a low inter-category and high intra-category appearance variability.
The dataset was created in collaboration with a Slovenian company DFG Consulting d.o.o.
DOWNLOAD DFG-TS dataset.
Code download
Our improvements to the Mask R-CNN are available in a forked Detectron framework on our GitHub repository.
Results
Three state-of-the-art detection models are evaluated on DFG-TS dataset:
- Faster R-CNN
- Mask R-CNN
- Mask R-CNN with our improvements
Overall, the deep-learning-based approach is able to achieve extremely good performance for many traffic-sign categories, including several complex ones with large intra-class variability. Large error rates for problematic traffic-sign categories are mostly due to similarity to other categories, wide viewing angles and large occlusions.
However, those issues do not pose a problem for the application of maintaining an accurate record of the traffic-sign inventory. They can be mitigated by the detection over several video frames or matching 3D locations from stereo cameras. In particular, this system is already being deployed for traffic-sign inventory management on Slovenian roads. However, the proposed solution is also applicable to other problems requiring the capability of traffic-sign detection such as autonomous driving and advanced driver-assistance systems.
Demo
Using Mask R-CNN trained on DFG dataset and LDP tracker in forward and backward tracking (offline processing), run on the streets of Ljubljana: