This project is about optical character recognition of seven-segment displays in real-time, dynamic video streams for the purpose of quality assurance and error detection. A high-level concept was developed and then implemented. Results show high recognition rates even under adverse lighting conditions and high amount of movement.
OCR, Optical character recognition, Computer Vision, OpenCV
Many industrial measuring devices only have a seven-segment display and lack any other standardized digital output. The goal of this project was to develop a prototypical application for optical character recognition for a pharmaceutical company, to help in quality assurance and error detection
Im industriellen Umfeld besitzen zahlreiche Messgeräte klassische Sieben-Segment-Displays, wie sie von älteren Taschenrechnern oder Digitaluhren bekannt sind. Diese Displays variieren unter anderem in Bezug auf Größe, Farbe und Intensität der Darstellung. Des Weiteren besitzen zahlreiche der verwendeten Messgeräte keinen zusätzlichen, standardisierten digitalen Ausgang, so dass die Messgrössen in der Regel lediglich visuell abgelesen werden können.
Im Rahmen eines Forschungsprojektes mit der Firma Augmenticon im Bereich der Pharmazie untersucht das IIT die Unterstützung, Kontrolle und Dokumentation notwendiger Arbeitsschritte in der Produktion unter Verwendung von tragbaren Augmented-Reality-Devices wie der MS Hololens, oder eines mobilen Geräts (Tablet/Handy). Die Detektion der Zahlenwerte von digitalen Displays (z.B. Waagen) ist dabei für die Fehlererkennung und der Dokumentation hilfreich.
Overall, the detection results have been excellent, even under especially challenging situations. The program could still successfully detect numbers under conditions where it was too dark for the human eye to see the number, as well as in situations where video was intentionally overexposed. High movement situations were simulated by vigorously shaking an external webcam. Applying those findings to the AR headset use case leads to the conclusion that serious neck injury would occur before excessive movement negatively affects detection rates. Detection was also still possible when the display had a very small absolute size in pixels, once again showing similar or better capabilities than the human eye.
This paper introduced a scoring algorithm for seven-segment OCR. This allows to select only results above a certain level of confidence, which is important for the intended use as a quality assurance tool. The results show that the proposed algorithm works even under adverse light conditions and under high motion. The newly introduced concept of Otsu linearization was shown to yield better results in some use cases.
Augmenticon AG
Dominik Russenberger
Hilko Cords, hilko.cord@fhnw.ch
Yves Simmen, yves.simmen@fhnw.ch
