CN113869320A - A template-based key-value pair extraction method and system - Google Patents
A template-based key-value pair extraction method and system Download PDFInfo
- Publication number
- CN113869320A CN113869320A CN202111191056.6A CN202111191056A CN113869320A CN 113869320 A CN113869320 A CN 113869320A CN 202111191056 A CN202111191056 A CN 202111191056A CN 113869320 A CN113869320 A CN 113869320A
- Authority
- CN
- China
- Prior art keywords
- anchor point
- template
- bounding box
- value pair
- key
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 25
- 238000012937 correction Methods 0.000 claims abstract description 17
- 230000014509 gene expression Effects 0.000 claims abstract description 14
- 230000009466 transformation Effects 0.000 claims abstract description 12
- 238000001514 detection method Methods 0.000 claims abstract description 9
- 238000000034 method Methods 0.000 claims abstract description 7
- 239000000284 extract Substances 0.000 claims abstract 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Character Input (AREA)
- Processing Or Creating Images (AREA)
Abstract
本发明公开了一种基于模板的键值对提取方法及系统，接收图像，利用文本检测算法DB在图像中找到文本信息，在文本信息的周围形成包围盒；根据预先录入的模板在图像中找到关键词信息，根据关键词信息和包围盒的坐标计算出固定锚点；根据包围盒坐标以及固定锚点计算出可变锚点；根据固定锚点、可变锚点和模板对图像进行投影变换矫正；识别并提取矫正后的包围盒内的文本信息。根据关键词或正则表达式或自定义锚点函数，根据锚点信息进行投影变换矫正，适用于各种文本信息的矫正及提取，提升了图像文字识别的准确率，实际准确率和鲁棒性都优于已有的模板键值对匹配方法。
The invention discloses a template-based key-value pair extraction method and system. It receives an image, uses a text detection algorithm DB to find text information in the image, and forms a bounding box around the text information; Keyword information, the fixed anchor point is calculated according to the keyword information and the coordinates of the bounding box; the variable anchor point is calculated according to the bounding box coordinates and the fixed anchor point; the image is projected and transformed according to the fixed anchor point, variable anchor point and template Correction; identify and extract textual information within the corrected bounding box. According to keywords or regular expressions or custom anchor point functions, projection transformation correction is performed according to anchor point information, which is suitable for correction and extraction of various text information, and improves the accuracy, actual accuracy and robustness of image text recognition. Both are better than existing template key-value pair matching methods.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a key value pair extraction method and system based on a template.
Background
At present, after the characters of image data are identified, key information of the characters needs to be extracted according to business requirements, a plurality of key information of the image data are related to the typesetting position of the image data, and the relative position of the key information is usually fixed, so that a document template can be customized to specify a bounding box in which each key information is located, and perspective transformation is performed by determining the corresponding anchor point position in an actual image, so that the transformed actual image can be attached to the template. The existing image character recognition algorithm based on template matching has very strict matching rules for anchor points, the anchor points are required to be completely fixed and invariable characters, the fault tolerance rate is not allowed on the character recognition of actual images, and the corresponding anchor points cannot be matched otherwise. The method expands the rule of anchor point matching, optimizes the image recognition details after matching, and greatly improves the template matching rate and the character recognition accuracy rate of the image character recognition model.
Disclosure of Invention
The invention aims to at least solve one technical problem in the prior art, and therefore, the invention provides a key value pair extraction method and system based on a template.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
according to an embodiment of the first aspect of the present invention, a template-based key-value pair extraction method includes: s10, receiving an image, finding text information in the image by using a text detection algorithm DB, and forming a bounding box around the text information; s20, finding keyword information in the image according to a pre-entered template, and calculating a fixed anchor point according to the keyword information and the coordinates of the bounding box; s30, calculating a variable anchor point according to the bounding box coordinates and the fixed anchor point; s40, performing projective transformation correction on the image according to the fixed anchor point, the variable anchor point and the template; and S50, recognizing and extracting the text information in the corrected bounding box.
According to some embodiments of the present invention, the step S10 is preceded by a step S1, and the step S1 includes entering or deleting a template, wherein the template includes key typesetting and keyword information.
According to some embodiments of the invention, the step S20 further includes calculating coordinates of the anchor point according to a regular expression.
According to some embodiments of the invention, the step S20 further includes customizing the function of the anchor point, and optimizing the coordinates of the anchor point according to the coordinates of the bounding box.
According to some embodiments of the invention, the step S40 is further followed by a step S41, and the step S41 includes forming a bounding box around all the bounding boxes, and sequentially dividing the bounding box into a plurality of regions from top to bottom according to the template, wherein each region includes at least one key-value pair.
According to a second aspect of the present invention, a template-based key-value pair extraction system includes: the positioning module is used for receiving an image, finding text information in the image by utilizing a text detection algorithm DB, and forming a bounding box around the text information; the fixed anchor point generating module finds keyword information in the image according to a pre-entered template and calculates a fixed anchor point according to the keyword information and the coordinates of the bounding box; the variable anchor point calculating module is used for calculating a variable anchor point according to the bounding box coordinates and the fixed anchor point; the correction module is used for carrying out projection transformation correction on the image according to the fixed anchor points, the variable anchor points and the template; and the information extraction module is used for identifying and extracting the corrected text information in the bounding box.
According to some embodiments of the invention, the mobile terminal further comprises an entry module, wherein the entry module comprises an entry or deletion template, and the template comprises key typesetting and keyword information.
According to some embodiments of the present invention, the step of generating the anchor point further includes calculating coordinates of the anchor point according to a regular expression.
According to some embodiments of the invention, the anchor point generating module further comprises customizing a function of the anchor point to optimize coordinates of the anchor point according to coordinates of the bounding box.
According to some embodiments of the present invention, the system further comprises a key-value pair dividing module, wherein the key-value pair dividing module forms an enclosure frame around all the enclosures, and sequentially divides the enclosure frame into a plurality of regions from top to bottom according to the template, and each region comprises at least one key-value pair.
The template-based key value pair extraction method and system provided by the embodiment of the invention at least have the following beneficial effects: the method is suitable for correction and extraction of various text information, improves the accuracy of image character recognition, and has actual accuracy and robustness superior to the existing template key value pair matching method.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a block flow diagram of the present invention;
FIG. 2 is a schematic diagram of the invention forming bounding boxes and anchors;
fig. 3 is a schematic diagram of the present invention forming an enclosure frame.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention are further described below with reference to the accompanying drawings.
The technical solutions in the embodiments of the present invention will be fully described below, and it should be apparent that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides a key value pair extraction method based on a template, as shown in fig. 1, which is a flow chart of the invention, according to some embodiments, S10, receiving an image, finding text information in the image by using a text detection algorithm DB, and forming a bounding box around the text information; s20, finding keyword information in the image according to a pre-entered template, and calculating a fixed anchor point according to the keyword information and the coordinates of the bounding box; s30, calculating a variable anchor point according to the bounding box coordinates and the fixed anchor point; s40, performing projective transformation correction on the image according to the fixed anchor point, the variable anchor point and the template; and S50, recognizing and extracting the text information in the corrected bounding box.
Based on the above embodiment, an image is received, text information is found in the image by the text detection algorithm DB and a bounding box is formed around the text information, and keyword information is found in the text of the bounding box. The bounding box is a rectangular bounding box, coordinates of four points of the bounding box are extracted, and the coordinate position of the fixed anchor point is calculated around the keyword information according to the coordinates of the bounding box. As shown in fig. 2, the fixed anchor point is a black dot disposed on the left side of the keyword information, and is disposed at a midpoint of a short side of the bounding box, and the fixed anchor point is disposed on the left side and is not affected by the length of the keyword information, such as the length of a specific name and a specific place name. The white point on the right side is a variable anchor point, and the white point is used for calculating and positioning through the coordinates of the fixed anchor point and the coordinates of the bounding box, so that the number of the anchor points is increased, and the accuracy of projection transformation is improved. The variable anchor point can change the coordinates thereof according to specific key information, such as the length of a specific name and a specific place name. And calculating a projective transformation matrix through the fixed anchor points and the variable anchor points, and mapping the bounding box of the original picture onto the template picture for correction. After the image is corrected, the text information in the bounding box is identified, and after the identification is finished, if the text information has wrong expression modes, such as character missing, unrecognizable character recognition error and the like, the text information can be automatically corrected through manual correction and through a regular expression and collected text information.
According to some embodiments, the step S10 is preceded by a step S1, and the step S1 includes entering or deleting a template, where the template includes key typesetting and keyword information.
Based on the above embodiment, when unimportant text information exists in the image, for example, manually handwriting the unimportant text information onto the copy, after the copy image is shot by the high-speed shooting device, the keyword information can be more accurately located according to the keyword information and the key composition, and the key information is found according to the keyword information, wherein the keyword information is "name", "gender", "birth", "year", "month", "day", and the like, and the key information is specific name, place name, and the like before and after the keyword information.
According to some embodiments, the step S20 further includes calculating coordinates of the anchor points according to a regular expression.
According to some embodiments, the step S20 further includes customizing the function of the anchor point, and optimizing the coordinates of the anchor point according to the coordinates of the bounding box.
Based on the above embodiment, the computing method of the coordinates of the fixed anchor further includes computing according to a regular expression, which computes the coordinates of the non-located fixed anchor by collecting the coordinates of the plurality of located fixed anchors and by using the regular expression and the collected coordinates of the fixed anchors. The function of the fixed anchor point can be customized by a user and is used for calculating the coordinate of the fixed anchor point through the coordinates of the four points of the bounding box. Anchor point coordinates are obtained through multiple modes, and the method is more practical.
According to some embodiments, the step S40 is further followed by a step S41, and the step S41 includes forming a bounding box around all the bounding boxes, and sequentially dividing the bounding box into a plurality of regions from top to bottom according to the template, wherein each region includes at least one key-value pair.
Based on the above embodiment, as shown in fig. 3, a plurality of key-value pairs are respectively divided from top to bottom in the bounding box, the key-value pair of each region in the bounding box is extracted by identification, the key-value pair of each region is extracted after identification, and each region is set as a row, so that the distinction is facilitated. The extraction result is as follows: the name and the specific name are in a row, the gender and the specific gender, the ethnicity and the specific ethnicity are in a row, and the like, and the data are extracted into a computer text for storage.
On the basis of the above embodiments, the present embodiment provides a character correcting system based on bounding boxes. According to some embodiments, comprising: the positioning module is used for receiving an image, finding text information in the image by utilizing a text detection algorithm DB, and forming a bounding box around the text information; the fixed anchor point generating module finds keyword information in the image according to a pre-entered template and calculates a fixed anchor point according to the keyword information and the coordinates of the bounding box; the variable anchor point calculating module is used for calculating a variable anchor point according to the bounding box coordinates and the fixed anchor point; the correction module is used for carrying out projection transformation correction on the image according to the fixed anchor points, the variable anchor points and the template; and the information extraction module is used for identifying and extracting the corrected text information in the bounding box.
Based on the above embodiment, an image is received, text information is found in the image by the text detection algorithm DB and a bounding box is formed around the text information, and keyword information is found in the text of the bounding box. The bounding box is a rectangular bounding box, coordinates of four points of the bounding box are extracted, and the coordinate position of the fixed anchor point is calculated around the keyword information according to the coordinates of the bounding box. As shown in fig. 2, the fixed anchor point is a black dot disposed on the left side of the keyword information, and is disposed at a midpoint of a short side of the bounding box, and the fixed anchor point is disposed on the left side and is not affected by the length of the keyword information, such as the length of a specific name and a specific place name. The white point on the right side is a variable anchor point, and the white point is used for calculating and positioning through the coordinates of the fixed anchor point and the coordinates of the bounding box, so that the number of the anchor points is increased, and the accuracy of projection transformation is improved. The variable anchor point can change the coordinates thereof according to specific key information, such as the length of a specific name and a specific place name. And calculating a projective transformation matrix through the fixed anchor points and the variable anchor points, and mapping the bounding box of the original picture onto the template picture for correction. After the image is corrected, the text information in the bounding box is identified, and after the identification is finished, if the text information has wrong expression modes, such as character missing, unrecognizable character recognition error and the like, the text information can be automatically corrected through manual correction and through a regular expression and collected text information.
According to some embodiments, the system further comprises an entry module, wherein the entry module comprises an entry or deletion template, and the template comprises key typesetting and keyword information.
Based on the above embodiment, when unimportant text information exists in the image, for example, manually handwriting the unimportant text information onto the copy, after the copy image is shot by the high-speed shooting instrument, the keyword information can be more accurately positioned according to the keyword information and the key composition, the key information is found according to the keyword information, wherein the keyword information is "name", "gender", "birth", "year", "month", "day", and the like, and the key information is specific name of a person, place name, and the like before and after the keyword information.
According to some embodiments, the step of generating the anchor point further includes calculating coordinates of the anchor point according to a regular expression.
According to some embodiments, the anchor point generation module further comprises customizing a function of the anchor point to optimize coordinates of the anchor point according to coordinates of the bounding box.
Based on the above embodiment, the coordinate calculation mode of the fixed anchor point further includes calculating according to a regular expression, and the function of the fixed anchor point may also be customized by a user, and is used for calculating the coordinate of the fixed anchor point through the coordinates of the four points of the bounding box. Anchor point coordinates are obtained through multiple modes, and the method is more practical.
According to some embodiments, the system further comprises a key-value pair dividing module, wherein the key-value pair dividing module forms a bounding box around all the bounding boxes, and sequentially divides the bounding box into a plurality of regions from top to bottom according to the template, and each region comprises at least one key-value pair.
Based on the above embodiment, as shown in fig. 3, a plurality of key-value pairs are respectively divided from top to bottom in the bounding box, the key-value pair of each region in the bounding box is extracted by identification, the key-value pair of each region is extracted after identification, and each region is set as a row, so that the distinction is facilitated. The extraction result is as follows: the name and the specific name are in a row, the gender and the specific gender, the ethnicity and the specific ethnicity are in a row, and the like, and the data are extracted into a computer text for storage.
It will be evident to those skilled in the art that the invention is not limited to the exemplary embodiments described above, but that the present solution is capable of being embodied in other specific forms without departing from the essential characteristics thereof. Accordingly, the embodiments should be considered as exemplary and non-limiting.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111191056.6A CN113869320B (en) | 2021-10-13 | 2021-10-13 | A template-based key-value pair extraction method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111191056.6A CN113869320B (en) | 2021-10-13 | 2021-10-13 | A template-based key-value pair extraction method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113869320A true CN113869320A (en) | 2021-12-31 |
| CN113869320B CN113869320B (en) | 2024-12-03 |
Family
ID=78998912
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111191056.6A Active CN113869320B (en) | 2021-10-13 | 2021-10-13 | A template-based key-value pair extraction method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113869320B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114119349A (en) * | 2021-10-13 | 2022-03-01 | 广东金赋科技股份有限公司 | Image information extraction method, device and medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180285676A1 (en) * | 2015-09-11 | 2018-10-04 | Junyu Han | Method and apparatus for processing image information |
| WO2019238063A1 (en) * | 2018-06-15 | 2019-12-19 | 众安信息技术服务有限公司 | Text detection and analysis method and apparatus, and device |
| CN111126125A (en) * | 2019-10-15 | 2020-05-08 | 平安科技（深圳）有限公司 | Method, device and equipment for extracting target text in certificate and readable storage medium |
| CN111353492A (en) * | 2020-03-12 | 2020-06-30 | 上海合合信息科技发展有限公司 | Image identification and information extraction method and device for standardized document |
| CN111783770A (en) * | 2020-01-16 | 2020-10-16 | 北京沃东天骏信息技术有限公司 | Image rectification method, device and computer readable storage medium |
| US10896357B1 (en) * | 2017-12-29 | 2021-01-19 | Automation Anywhere, Inc. | Automatic key/value pair extraction from document images using deep learning |
| WO2021057138A1 (en) * | 2019-09-27 | 2021-04-01 | 支付宝(杭州)信息技术有限公司 | Certificate recognition method and apparatus |
| CN113269126A (en) * | 2021-06-10 | 2021-08-17 | 上海云扩信息科技有限公司 | Key information extraction method based on coordinate transformation |
-
2021
- 2021-10-13 CN CN202111191056.6A patent/CN113869320B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180285676A1 (en) * | 2015-09-11 | 2018-10-04 | Junyu Han | Method and apparatus for processing image information |
| US10896357B1 (en) * | 2017-12-29 | 2021-01-19 | Automation Anywhere, Inc. | Automatic key/value pair extraction from document images using deep learning |
| WO2019238063A1 (en) * | 2018-06-15 | 2019-12-19 | 众安信息技术服务有限公司 | Text detection and analysis method and apparatus, and device |
| WO2021057138A1 (en) * | 2019-09-27 | 2021-04-01 | 支付宝(杭州)信息技术有限公司 | Certificate recognition method and apparatus |
| CN111126125A (en) * | 2019-10-15 | 2020-05-08 | 平安科技（深圳）有限公司 | Method, device and equipment for extracting target text in certificate and readable storage medium |
| CN111783770A (en) * | 2020-01-16 | 2020-10-16 | 北京沃东天骏信息技术有限公司 | Image rectification method, device and computer readable storage medium |
| CN111353492A (en) * | 2020-03-12 | 2020-06-30 | 上海合合信息科技发展有限公司 | Image identification and information extraction method and device for standardized document |
| CN113269126A (en) * | 2021-06-10 | 2021-08-17 | 上海云扩信息科技有限公司 | Key information extraction method based on coordinate transformation |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114119349A (en) * | 2021-10-13 | 2022-03-01 | 广东金赋科技股份有限公司 | Image information extraction method, device and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113869320B (en) | 2024-12-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110232311B (en) | Method and device for segmenting hand image and computer equipment | |
| CN112699775B (en) | Certificate identification method, device, equipment and storage medium based on deep learning | |
| WO2021147252A1 (en) | Ocr-based table format recovery method and apparatus, electronic device, and storage medium | |
| CN111898411B (en) | Text image annotation system, method, computer device and storage medium | |
| WO2018108129A1 (en) | Method and apparatus for use in identifying object type, and electronic device | |
| WO2022001256A1 (en) | Image annotation method and device, electronic apparatus, and storage medium | |
| CN113569968B (en) | Model training method, target detection method, device, equipment and storage medium | |
| WO2018233055A1 (en) | Method and apparatus for entering policy information, computer device and storage medium | |
| WO2021147219A1 (en) | Image-based text recognition method and apparatus, electronic device, and storage medium | |
| US20130314437A1 (en) | Image processing apparatus, image processing method, and computer program | |
| KR20200010650A (en) | Deep Learning Based Automatic Gesture Recognition Method and System | |
| CN112396047B (en) | Training sample generation method and device, computer equipment and storage medium | |
| CN111353325A (en) | Key point detection model training method and device | |
| CN112287763A (en) | Image processing method, apparatus, device and medium | |
| CN113869320A (en) | A template-based key-value pair extraction method and system | |
| CN113657370B (en) | Character recognition method and related equipment thereof | |
| CN110414402B (en) | Gesture data labeling method and device, electronic equipment and storage medium | |
| WO2024140094A1 (en) | Paragraph determination method and apparatus for digital document, and electronic device and storage medium | |
| CN117496521A (en) | Method, system and device for extracting key information of table and readable storage medium | |
| WO2020244076A1 (en) | Face recognition method and apparatus, and electronic device and storage medium | |
| CN117058694A (en) | Image text structured extraction method and device | |
| CN116994251A (en) | Seal generation method, device, terminal equipment and storage medium | |
| CN113869313B (en) | A method and system for identifying information of image background filtering | |
| CN114692759A (en) | Video face classification method, device, equipment and storage medium | |
| CN114677743A (en) | Face rectification method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |