CN111191647A - Standard formula identification method based on image processing - Google Patents

Standard formula identification method based on image processing Download PDF

Info

Publication number
CN111191647A
CN111191647A CN201911365677.4A CN201911365677A CN111191647A CN 111191647 A CN111191647 A CN 111191647A CN 201911365677 A CN201911365677 A CN 201911365677A CN 111191647 A CN111191647 A CN 111191647A
Authority
CN
China
Prior art keywords
formula
image processing
method based
sub
standard
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN201911365677.4A
Other languages
Chinese (zh)
Inventor
周培培
侯幸林
蔡纪鹤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Changzhou Institute of Technology
Original Assignee
Changzhou Institute of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Changzhou Institute of Technology filed Critical Changzhou Institute of Technology
Priority to CN201911365677.4A priority Critical patent/CN111191647A/en
Publication of CN111191647A publication Critical patent/CN111191647A/en
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/635—Overlay text, e.g. embedded captions in a TV program
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Character Discrimination (AREA)
  • Character Input (AREA)

Abstract

The invention discloses a standard formula identification method based on image processing, which comprises the following steps: step 1, a formula to be identified; step 2, a horizontal formula after calibration; step 3, a formula after binarization processing is carried out; step 4, splitting each sub item; step 5, identifying a system by a formula; and 6, outputting the editable formula. The invention uses artificial intelligence method to output the identification result of the whole formula, and converts it into editable text, such as Latex, EdrawMath, Onenote or Word, or converts it into dynamic link library or loading item, or can be designed into software.

Description

Standard formula identification method based on image processing
Technical Field
The invention relates to the field of image processing, in particular to a standard formula identification method based on image processing.
Background
Formula editing is an extremely important part in text editing, however, large formula editing is complex and often occupies a large amount of time for text editors. For formulas in some texts (PDFs and pictures), editors need to copy all the texts, but the texts are often edited again in work, and a large amount of time is occupied, so that the design of the formula identification method has important significance.
Disclosure of Invention
1. Objects of the invention
The invention mainly takes pictures or captures screens aiming at formulas in pictures and PDF documents, and then converts the formulas into editable formula texts by an image processing method, and has high recognition rate and high recognition speed.
2. The technical scheme adopted by the invention
The invention discloses a standard formula identification method based on image processing, which comprises the following steps:
step 1, a formula to be identified;
step 2, a horizontal formula after calibration;
step 3, a formula after binarization processing is carried out;
step 4, splitting each sub item;
step 5, identifying a system by a formula;
and 6, outputting the editable formula.
And 5, identifying the formula by using image processing software:
5.1 transforming the intercepted or shot formula image to change the formula into a horizontal direction;
5.2 extracting a formula target by using an image processing algorithm;
5.3 using image processing algorithm to split the whole formula into sub-items;
5.4 recording the position corresponding relation among the sub items, wherein the specific rule is as follows:
the identification symbols and positions comprise left side, right side, upper right corner, upper and lower part type and root;
and 5.5, identifying the meaning of each sub-item according to the position relation among the sub-items, forming a logical ordering relation, classifying the relation into a method or a result, recording and storing the relation in the whole formula, and gradually reasoning from a small range to a large range after equal sign to infer the relation of each sub-item in the whole formula.
Further, said 5.2 uses image processing algorithm including threshold segmentation, binarization to extract formula object.
Furthermore, the 5.3 uses an image processing algorithm including a connected domain identification method.
Furthermore, in step 5.5, according to the position relation of each sub-item and the meaning of each sub-item, an artificial intelligence method is used for comprehensively outputting the recognition result of the whole formula and converting the recognition result into editable text comprising Latex, EdrawMath, Onenote or Word, or converting the recognition result into a dynamic link library or a loading item or independent software.
Further, step 1, the formula to be recognized is acquired by a camera.
Further, step 1, for the formula in the PDF document in the computer or the mobile phone, the formula target can be intercepted by using screenshot software as the input of the system on the premise of opening the file.
3. Advantageous effects adopted by the present invention
The invention uses artificial intelligence method to output the identification result of the whole formula, and converts it into editable text, such as Latex, EdrawMath, Onenote or Word, or converts it into dynamic link library or loading item, or can be designed into software.
Drawings
FIG. 1 is a flow chart of the present invention;
Detailed Description
The technical solutions in the examples of the present invention are clearly and completely described below with reference to the drawings in the examples of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without inventive step, are within the scope of the present invention.
The present invention will be described in further detail with reference to the accompanying drawings.
Example 1
The formula of the invention can be a handwritten formula (including pen writing, liquid crystal panel writing or screen handwriting and the like), and can also be a standard formula (Latex, EdrawMath, One note or Word and the like) edited by a formula editor;
the screen capture can be performed on a PDF document, a Word document or a picture opened on a computer, or can be performed on a PDF document, a Word document or a picture opened on mobile equipment;
the invention can be used as software on a computer or mobile equipment; the plug-in can be used as a plug-in and integrated in Word, WPS and other software; and can also be called as a dynamic link library or a loading item.
The invention aims at a handwritten formula and a formula of standard editing, uses an image processing algorithm for recognition, and converts the handwritten formula and the formula of standard editing into a text which can be edited, and mainly relates to the following aspects:
acquiring an image: aiming at a handwritten formula or a standard formula in a text, a target formula is obtained by adopting a photographing or screen capturing mode;
and (3) partitioning the formula: aiming at a formula image, dividing the formula image into a plurality of sub-items by using methods such as threshold segmentation, binarization, communication domain extraction and the like;
and (3) semantic recognition: analyzing the position relation of each sub-item and the specific meaning of each symbol, and converting the position relation into a formula text which can be directly edited by using an artificial intelligence method;
and (4) outputting the result: outputting editable formula text, software, dynamic link library or loading items.
The method specifically comprises the following steps:
1. a formula to be identified;
2. a calibrated horizontal formula;
3. a formula after binarization processing;
4. splitting each sub item;
5. a formula identification system;
6. and outputting the editable formula.
Example 2
1. Selecting a formula target to be identified for selection;
1.1 for handwritten formula (paper, liquid crystal panel, computer screen, etc.), shooting target with camera and intercepting target formula as system input
1.2 for the formula in the picture, capturing the formula target after taking a picture by using a camera as the input of the system
1.3 for the formula in PDF document in computer or mobile phone, on the premise of opening the file, intercepting the formula target by using the screenshot software as the input of the system
2 identifying formulas using image processing software
2.1, the intercepted or shot formula image is subjected to position, angle and other transformations, so that the formula is changed into the horizontal direction;
2.2 extracting the formula target by using an image processing algorithm (threshold segmentation, binarization and the like);
2.3 using image processing algorithm (communication domain identification, etc.) to split the whole formula into sub-items, such as numerator, denominator, operation symbol, bracket, etc.; e.g. y ═ x2+z3Splitting to obtain sub-items of 'y' '═ x' '' 2 '' + '' z '' '3', and the like;
2.4 recording the position correspondence between the sub-items, e.g. y ═ x2+z3In 'y' on the left side of '═ 2' on the upper right of 'x', etc.;
2.5 identifying the meaning of each sub-item according to the positional relationship between the sub-items, 'y' is on the left side of ═ and then 'y' is the calculation result, 'x' '2' '+' 'z' '3' is the calculation method, etc.;
and 3, comprehensively outputting the recognition result of the whole formula by using an artificial intelligence method according to the position relation and the meaning of each sub-item, and converting the recognition result into an editable text such as Latex, EdrawMath, Onenote or Word, or converting the recognition result into a dynamic link library or a loading item, wherein the recognition result can also be designed into software for use.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (7)

1.一种基于图像处理的标准公式识别方法，其特征在于：1. a standard formula recognition method based on image processing, is characterized in that: 步骤1.待识别的公式；Step 1. The formula to be identified; 步骤2.经过校准后的水平公式；Step 2. The calibrated horizontal formula; 步骤3.二值化处理后的公式；Step 3. The formula after binarization; 步骤4.拆分后的各个子项；Step 4. Each sub-item after splitting; 步骤5.公式识别系统；Step 5. Formula recognition system; 步骤6.输出的可编辑公式。Step 6. Editable formula for output. 2.根据权利要求1所述的基于图像处理的标准公式识别方法，其特征在于步骤5中使用图像处理的软件对公式进行识别：2. the standard formula identification method based on image processing according to claim 1 is characterized in that using the software of image processing in step 5 to identify formula: 5.1将截取或拍摄的公式图像进行变换，使公式变为水平方向；5.1 Transform the captured or photographed formula image so that the formula becomes horizontal; 5.2使用图像处理算法将公式目标提取；5.2 Use the image processing algorithm to extract the formula target; 5.3使用图像处理算法将公式整体拆分成子项；5.3 Use the image processing algorithm to split the formula as a whole into sub-items; 5.4记录各子项之间的位置对应关系，具体规则如下：5.4 Record the positional correspondence between each sub-item. The specific rules are as follows: 识别符号以及位置包括左侧、右侧、右上角、上下分式、根号下；Identification symbols and positions include left, right, upper right, upper and lower fractions, and under the root sign; 5.5根据各子项之间的位置关系，识别各子项的意义，形成逻辑排序关系，并归为方法或结果，将整个公式中的该类关系记录存储，并使用在等号后的从小范围到大范围的顺序逐步推理即可推断整个公式中每个子项的关系。5.5 According to the positional relationship between each sub-item, identify the meaning of each sub-item, form a logical sorting relationship, and classify it as a method or result, store this type of relationship record in the entire formula, and use the small range after the equal sign. Step-by-step reasoning to a large range of sequences can infer the relationship of each subterm in the entire formula. 3.根据权利要求1所述的基于图像处理的标准公式识别方法，其特征在于所述的5.2使用图像处理算法包括阈值分割，二值化将公式目标提取。3 . The standard formula recognition method based on image processing according to claim 1 , wherein the image processing algorithm used in 5.2 includes threshold segmentation and binarization to extract formula targets. 4 . 4.根据权利要求1所述的基于图像处理的标准公式识别方法，其特征在于所述的5.3使用图像处理算法包括联通域识别方法。4. The standard formula recognition method based on image processing according to claim 1, characterized in that the described 5.3 using image processing algorithm includes the Unicom domain recognition method. 5.根据权利要求1所述的基于图像处理的标准公式识别方法，其特征在于：步骤5.5中根据各子项的位置关系和各子项的意义，使用人工智能方法综合输出整个公式的识别结果，并将其转化为可编辑的文本，包括Latex、EdrawMath、Onenote或Word，或将其转换为动态链接库或加载项或独立软件。5. the standard formula identification method based on image processing according to claim 1, is characterized in that: in step 5.5, according to the positional relationship of each subitem and the meaning of each subitem, use artificial intelligence method synthetically output the identification result of whole formula , and convert it into editable text, including Latex, EdrawMath, Onenote or Word, or convert it into a dynamic link library or add-in or stand-alone software. 6.根据权利要求1所述的基于图像处理的标准公式识别方法，其特征在于：步骤1.待识别的公式采用相机获取。6 . The standard formula identification method based on image processing according to claim 1 , wherein: step 1. The formula to be identified is obtained by using a camera. 7 . 7.根据权利要求1所述的基于图像处理的标准公式识别方法，其特征在于：步骤1.对于电脑或手机中PDF文档中的公式，可以在打开文件的前提下，使用截图软件截取公式目标，作为系统的输入。7. the standard formula recognition method based on image processing according to claim 1, is characterized in that: step 1. for formula in the PDF document in computer or mobile phone, can under the premise of opening file, use screenshot software to intercept formula target , as the input to the system.
CN201911365677.4A 2019-12-26 2019-12-26 Standard formula identification method based on image processing Withdrawn CN111191647A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911365677.4A CN111191647A (en) 2019-12-26 2019-12-26 Standard formula identification method based on image processing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911365677.4A CN111191647A (en) 2019-12-26 2019-12-26 Standard formula identification method based on image processing

Publications (1)

Publication Number Publication Date
CN111191647A true CN111191647A (en) 2020-05-22

Family

ID=70709401

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911365677.4A Withdrawn CN111191647A (en) 2019-12-26 2019-12-26 Standard formula identification method based on image processing

Country Status (1)

Country Link
CN (1) CN111191647A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113326675A (en) * 2021-08-04 2021-08-31 江西风向标教育科技有限公司 Formula processing method and system for education resource library
CN114610405A (en) * 2022-03-03 2022-06-10 深圳盛显科技有限公司 Multi-application screen capture and network code output method, device, medium and product

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113326675A (en) * 2021-08-04 2021-08-31 江西风向标教育科技有限公司 Formula processing method and system for education resource library
CN114610405A (en) * 2022-03-03 2022-06-10 深圳盛显科技有限公司 Multi-application screen capture and network code output method, device, medium and product
CN114610405B (en) * 2022-03-03 2024-03-29 深圳盛显科技有限公司 Multi-application screen capturing and network code output method, equipment, medium and product

Similar Documents

Publication Publication Date Title
CN110689037B (en) Method and system for automatic object annotation using deep networks
CN110135411B (en) Business card recognition method and device
CA3027038C (en) Document field detection and parsing
JP5050075B2 (en) Image discrimination method
CN103946866B (en) The text detection that multilayer is connected component is used together with histogram
US20170039193A1 (en) Language generation from flow diagrams
US9940511B2 (en) Machine print, hand print, and signature discrimination
CN108229481B (en) Screen content analysis method, device, computing device and storage medium
JP7389824B2 (en) Object identification method and device, electronic equipment and storage medium
JP2024501642A (en) Detecting annotated regions of interest in images
CN111191647A (en) Standard formula identification method based on image processing
Marne et al. Identification of optimal optical character recognition (OCR) engine for proposed system
CN113065559A (en) Image comparison method and device, electronic equipment and storage medium
Nachamai Alphabet recognition of american sign language: a hand gesture recognition approach using sift algorithm
CN113628181A (en) Image processing method, image processing device, electronic equipment and storage medium
Dave et al. Ocr text detector and audio convertor
Revathi et al. Optical character recognition for handwritten Telugu Text
WO2024169656A1 (en) Self-monitoring-based video stream feature generation method, device, and storage medium
Dey et al. A comparative study of margin noise removal algorithms on marnr: A margin noise dataset of document images
Gladwin et al. Read Textual features in Images and convert to Editable form by extended use of Artificial Neural Networks, Deep learning and Maximally Stable Extremal Region techniques
CN118447522B (en) A PDF file component extraction device, method, electronic device and readable storage medium
Darahan et al. Real-Time Page Extraction for Document Digitization
US20150142784A1 (en) Retrieval device and method and computer program product
Joyakin iMask—An Artificial Intelligence Based Redaction Engine
Lakshmi et al. Modeling of Patterns with Spectral Data and Time-Varying PSO to Identify Concealed Character Strokes of Historical Manuscripts

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication

Application publication date: 20200522

WW01 Invention patent application withdrawn after publication