Solving Algebraic Problems with Geometry Diagrams Using Syntax-Semantics Diagram Understanding

Solving Algebraic Problems with Geometry Diagrams (APGDs) poses a significant challenge in artificial intelligence due to the complex and diverse geometric relations among geometric objects. Problems typically involve both textual descriptions and geometry diagrams, requiring a joint understanding of these modalities. Although considerable progress has been made in solving math word problems, research on solving APGDs still cannot discover implicit geometry knowledge for solving APGDs, which limits their ability to effectively solve problems. In this study, a systematic and modular three-phase scheme is proposed to design an algorithm for solving APGDs that involve textual and diagrammatic information. The three-phase scheme begins with the application of the state-transformer paradigm, modeling the problem-solving process and effectively representing the intermediate states and transformations during the process. Next, a generalized APGD-solving approach is introduced to effectively extract geometric knowledge from the problem’s textual descriptions and diagrams. Finally, a specific algorithm is designed focusing on diagram understanding, which utilizes the vectorized syntax-semantics model to extract basic geometric relations from the diagram. A method for generating derived relations, which are essential for solving APGDs, is also introduced. Experiments on real-world datasets, including geometry calculation problems and shaded area problems, demonstrate that the proposed diagram understanding method significantly improves problem-solving accuracy compared to methods relying solely on simple diagram parsing.