Mathematical Formula OCR Using Physics-Informed Neural Networks (PINNs)

Abstract

The multidimensional structure, level of structure, and rigid rules of syntax used to define the symbolic notation make mathematical expression recognition a complex problem. The traditional OCR techniques have problems in preserving the spatial correlations and this results in errors of precedence of operators, matching of symbols and assembly of structure. The current paper presents the PINN-OCR is a physics-aware neural network that jointly performs convolutional feature extraction, bidirectional sequence modeling, and a Transformer-based decoder to which it adds the structural constraints of the model as differentiable. The constraints lead the model to the use of mathematical validity by positional reasoning, bracket consistency, and layout-aware arrangement of symbols during training. The system was assessed using datasets with CROHME and IM2LATEX-100K data and it is capable of generalization to both handwritten and printed expressions at very high levels as compared to baseline architectures. The hybrid scheme suggested puts down a strong line of structured OCR mechanism by using a structured and information-driven learning strategy alongside domain-pertinent mathematical regulations.