Enhanced Optical Character Recognition by Optical Sensor Combined with BERT and Cosine Similarity Scoring (Student Abstract)

Optical character recognition(OCR) is the technology to identify text characters embedded within images. Conventional OCR models exhibit performance degradation when performing with noisy images. To solve this problem, we propose a novel model, which combines computer vision using optical sensor with natural language processing by bidirectional encoder representations from transformers(BERT) and cosine similarity scoring. The proposed model uses a confidence rate to determine whether to utilize optical sensor alone or BERT/cosine similarity scoring combined with the optical sensor. Experimental results show that the proposed model outperforms approximately 4.34 times better than the conventional OCR.

Authors

• Woohyeon Moon Korea Advanced Institute of Science and Technology
• Sarvar Nengroo Korea Advanced Institute of Science and Technology
• Taeyoung Kim Korea Advanced Institute of Science and Technology
• Jihui Lee Korea Advanced Institute of Science and Technology
• Seungah Son Korea Advanced Institute of Science and Technology
• Dongsoo Har Korea Advanced Institute of Science and Technology

Keywords

• BERT
• Cosine Similarity
• CV
• NLP
• OCR