CN108806059B - Text region positioning method based on note alignment and eight-neighborhood connector offset correction of feature points - Google Patents
Text region positioning method based on note alignment and eight-neighborhood connector offset correction of feature points Download PDFInfo
- Publication number
- CN108806059B CN108806059B CN201810434069.3A CN201810434069A CN108806059B CN 108806059 B CN108806059 B CN 108806059B CN 201810434069 A CN201810434069 A CN 201810434069A CN 108806059 B CN108806059 B CN 108806059B
- Authority
- CN
- China
- Prior art keywords
- bill
- offset
- area
- alignment
- template
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012937 correction Methods 0.000 title claims abstract description 23
- 238000000034 method Methods 0.000 title claims abstract description 23
- 230000009466 transformation Effects 0.000 claims abstract description 18
- 239000011159 matrix material Substances 0.000 claims abstract description 11
- 238000000605 extraction Methods 0.000 claims abstract description 6
- 238000004364 calculation method Methods 0.000 claims abstract description 4
- 238000004891 communication Methods 0.000 claims description 7
- 230000000877 morphologic effect Effects 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 4
- 230000004807 localization Effects 0.000 abstract 1
- 238000010845 search algorithm Methods 0.000 abstract 1
- 238000013461 design Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07D—HANDLING OF COINS OR VALUABLE PAPERS, e.g. TESTING, SORTING BY DENOMINATIONS, COUNTING, DISPENSING, CHANGING OR DEPOSITING
- G07D7/00—Testing specially adapted to determine the identity or genuineness of valuable papers or for segregating those which are unacceptable, e.g. banknotes that are alien to a currency
- G07D7/20—Testing patterns thereon
- G07D7/2008—Testing patterns thereon using pre-processing, e.g. de-blurring, averaging, normalisation or rotation
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07D—HANDLING OF COINS OR VALUABLE PAPERS, e.g. TESTING, SORTING BY DENOMINATIONS, COUNTING, DISPENSING, CHANGING OR DEPOSITING
- G07D7/00—Testing specially adapted to determine the identity or genuineness of valuable papers or for segregating those which are unacceptable, e.g. banknotes that are alien to a currency
- G07D7/20—Testing patterns thereon
- G07D7/2016—Testing patterns thereon using feature extraction, e.g. segmentation, edge detection or Hough-transformation
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Character Input (AREA)
- Image Analysis (AREA)
Abstract
本发明涉及图像的技术领域，更具体地，涉及基于特征点的票据对齐和八邻域连通体偏移修正的文本区域定位方法。本发明提出了一种新颖且高效的双阶段文字区域定位方法，即第一阶段首先对票据图像标准化到统一尺寸，利用SIFT特征点匹配的思路，对每张待识别的票据进行特征提取，并与其对应的模板基准图进行特征点匹配，求得变换矩阵后进行投射变换，完成票据对齐。第二阶段利用已有的模板信息对其中一个信息区域使用八领域搜索连通体并将八领域内符合约定性质的连通体以一定顺序加入队列，并通过计算队列队首连通体的中心与该模板信息区中点的偏移量，推断出其余信息区的预测位置，并且通过对进行偏移修正后的各个区域的位置中是否存在合适的连通体来推断偏移量的计算是否准确。本方法结合了计算机视觉中特征点匹配算法以及基于位置搜索算法，实现了对模板信息区域的精确定位，尤其对套打偏移严重的票据有很好的效果。
The present invention relates to the technical field of images, and more particularly, to a text area localization method based on feature point alignment and eight-neighbor connected body offset correction. The invention proposes a novel and efficient two-stage text area positioning method, that is, the first stage first standardizes the bill image to a uniform size, and uses the idea of SIFT feature point matching to perform feature extraction for each bill to be identified, and Match the feature points with its corresponding template reference map, obtain the transformation matrix, and then perform projection transformation to complete the bill alignment. The second stage uses the existing template information to use the eight domains to search for connected objects in one of the information areas, and adds the connected objects that meet the agreed properties in the eight domains to the queue in a certain order. The offset of the midpoint of the information area is used to deduce the predicted positions of the remaining information areas, and whether the offset calculation is accurate is determined by checking whether there is a suitable connected body in the position of each area after offset correction. The method combines the feature point matching algorithm and the position-based search algorithm in computer vision, and realizes the precise positioning of the template information area, especially for bills with serious offset.
Description
Technical Field
The invention relates to the technical field of images, in particular to a text region positioning method based on note alignment of characteristic points and eight-neighborhood connector offset correction.
Background
In a system for automatically identifying bills based on templates, the position of the bill to be identified is often directly positioned according to the position of an information area defined in each type of template, and the positioning accuracy of the method for directly positioning the information area of the template is low. This is because each bill has its own features, and because the shooting angle and the completeness of the bill are different, the success rate of directly applying template information to locate the text is not high, which results in failure of subsequent character recognition.
Chinese patent publication No. CN104916034A discloses a bill recognition system and recognition method based on an interventionalisable template, where the location of the text region of interest in the system is the position information of the text directly using the original template, and no additional optimization operation is made for the text location. The method basically fails to locate the bill which has a difference with the shooting size of the template bill or has obvious overprinting effect.
Chinese patent publication No. CN107622255A discloses a method and system for positioning bill field based on position template and semantic template, which simply uses position template information in positioning of interested text, and the method is not suitable for bills with different bill size in image and template bill size due to difference in shooting angle. Meanwhile, the system carries out position correction by using a method of combining position information and bill mark fields for text positioning of the over printing type invoices, but the method requires that a characteristic field is defined for each type of invoices firstly and the relative offset is calculated by detecting the characteristic field of each invoice, and the method has limitation. The first method needs to manually design the characteristic field for each type of bill, which is time-consuming and labor-consuming, and cannot ensure that the characteristic mark can be designed for each type of bill; secondly, when the characteristic field is searched, the characteristic field is possibly failed to find due to recognition error, and then the text positioning is failed; thirdly, the adoption of the method is premised on that all text rows need to be recognized, and the efficiency is very low.
Disclosure of Invention
The invention aims to overcome at least one defect in the prior art, and provides a text region positioning method based on feature point bill alignment and eight-neighborhood connector offset correction. The whole set of positioning algorithm can solve the positioning problem of most bills, and the method has strong universality.
The technical scheme of the invention is as follows: a text region positioning method based on note alignment of characteristic points and eight-neighborhood connector offset correction comprises two parts of note alignment and eight-neighborhood connector offset correction based on characteristic point matching,
the bill alignment part based on feature point matching is the first stage of the whole text region positioning algorithm, firstly, the bill size is standardized, SIFT feature point extraction is carried out on the bill, then, feature matching is carried out on the bill feature points and the template bill feature points, a transformation matrix is obtained, projection transformation is carried out, and bill alignment in the first stage is completed;
the eight-neighborhood connector offset correction is the second stage of the text region positioning algorithm, on the basis that the bills are aligned, one information region is selected as a main operation region, and the positions of other regions of the template are corrected by using the offset error according to the offset error between the character region information on the original bill template and the connector queue, so that the accurate position of each information region is obtained.
Furthermore, the same type of bill alignment is realized by obtaining a transformation matrix through extraction and matching calculation of the feature points, and perspective transformation is performed according to the transformation matrix to complete bill alignment.
Further, an eight-neighborhood searching area is generated for a certain information area in the bill template, then binarization and morphological processing are carried out on the searching area, and the communicating bodies in the searching area are queued and stored according to a specified sequence, so that a main communicating body queue is generated.
Furthermore, the sequence of the communication bodies in the search area which are queued and stored according to the designated sequence is middle, right, left, upper, lower, upper right, lower right, upper left and lower left.
Further, the center coordinates of the connected body at the head of the queue are calculated, the offset between the coordinates and the center coordinates of the area in the template is calculated, the positions of the information areas in other templates are corrected according to the offset, the connected bodies are searched in the same eight neighborhoods of the calculated information areas and are queued, if the connected bodies meeting the conditions are positioned in the connected body queue in other positions, the offset is calculated correctly, otherwise, the connected bodies are dequeued, and the same operation is continuously performed on the next connected body in the main queue.
The note alignment is the first stage of the whole set of positioning algorithm, and the specific steps of the note alignment stage are as follows:
(1) firstly, the size standardization of the image of the bill to be processed is carried out, namely, the image of the bill to be processed is scaled to the specified size of the bill.
(2) Graying the bill image to be processed, extracting SIFT feature points, matching the SIFT feature points with the feature points of the corresponding template image, and further solving a transformation matrix H.
(3) And performing projection transformation on the image to be processed according to the transformation matrix H to obtain a transformed bill image, thereby finishing the alignment operation of the image to be processed and the reference image.
Generally speaking, for character positioning of common and non-sleeve printed bills, after bill alignment in the first stage is carried out, accurate text regions can be obtained by directly applying information of information region positions in a template. If the problem of character offset of the set of ticketing data needs to be solved, the second stage of the algorithm, namely the offset correction stage, needs to be carried out continuously, and the specific steps are as follows:
(1) and randomly selecting an information area in the template as a main operation area, and expanding the area to the eight directions of the periphery of the area by the size of the area to generate an eight-neighborhood search area.
(2) And carrying out binarization and corresponding morphological processing on the eight-neighborhood search area, and enqueuing and storing the connected bodies in the nine grids according to a certain rule.
(3) And calculating the center coordinate of the first communication body of the queue, and obtaining the offset K by taking the difference value between the center coordinate of the first communication body of the queue and the center coordinate of the area template information.
(4) And recalculating the specific position information of other information areas according to the offset K to obtain new information area position information, properly amplifying the position area for each new position, detecting whether a connected body meeting the appointed condition of the area exists in the area, if so, indicating that the offset is the real offset, completing the offset correction stage, and otherwise, performing the same offset correction operation on the next connected body in the queue.
Compared with the prior art, the beneficial effects are: the method makes full use of the higher similarity between the bills, and can finish the alignment of the bills by means of the characteristic points between the bills of the same kind. On the basis that the bills are aligned, the text is accurately positioned by using the characteristic that the relative position between the overprint fonts is unchanged and using the offset correction of the eight-neighborhood connected body. The method is suitable for wide bills, does not need to design characteristic marks in advance for auxiliary positioning, and is simple to operate and high in accuracy.
Drawings
FIG. 1 shows the algorithm framework of the present invention.
Fig. 2 shows a schematic diagram of an eight neighborhood search in the second stage of the algorithm.
Figure 3 shows the algorithmic process of the offset correction phase.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the patent; for the purpose of better illustrating the embodiments, certain features of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product; it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted. The positional relationships depicted in the drawings are for illustrative purposes only and are not to be construed as limiting the present patent.
As shown in FIG. 1, the scheme is divided into two stages of note alignment and offset correction. The specific steps of the note alignment stage include: (1) reading a bill image to be identified, classifying the bill image and obtaining corresponding bill template information; (2) graying the image to be recognized and standardizing the size of the image to be recognized to be the uniform size of the bill; (3) SIFT feature point extraction is carried out on the image, a feature point set of the image is matched with a feature point set of a template image, and a transformation matrix H is obtained; (4) and carrying out perspective transformation on the graph according to the transformation matrix H to realize the registration of the current bill and the template bill, namely realizing the alignment of the bills.
As shown in fig. 3, the specific steps of the bill offset correction phase include: (1) randomly selecting any information area in the template as a main operation area, and generating search areas with the same size as the information area in eight directions of the area, wherein the main operation area totally comprises nine grids as shown in fig. 2; (2) carrying out binarization and morphological processing on the search area of the nine grids, adhering text lines into a whole to form a communicating body by using expansion operation, and eliminating small-particle small noise by using corrosion operation; (3) and enqueuing and storing the communication bodies in the search area according to a certain sequence, wherein the enqueuing sequence determines the priority of the current communication body according to which of nine grids the center of each communication body falls, the priority of each grid is shown in FIG. 2, 1 is the highest priority, and 9 is the lowest priority. Determining an enqueuing sequence according to the priorities of the communicating bodies during enqueuing, wherein the communicating bodies with high priorities enqueue first; (4) calculating the center coordinate of the first communicating body of the queue, and obtaining an offset K by taking the difference value between the center coordinate and the area template information center coordinate; (5) recalculating the specific position information of other information areas according to the offset K to obtain new information area position information, and appropriately amplifying the position area for each new position, for example, multiplying the height and width of the area position defined by the template by 1.2 times; (6) and detecting whether a connected body meeting the appointed conditions of the area exists in the area, if so, indicating that the offset is the real offset, completing the offset correction stage, otherwise, performing the same offset correction operation on the next connected body in the queue until the offset calculation is successful, ending the offset correction stage, or ending the algorithm until the queue is empty and still cannot find out a proper offset, and returning positioning failure information.
It should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.
Claims (1)
1. A text region positioning method based on note alignment of characteristic points and eight-neighborhood connector offset correction is characterized by comprising two parts of note alignment and eight-neighborhood connector offset correction based on characteristic point matching,
the bill alignment part based on feature point matching is the first stage of the whole text region positioning algorithm, firstly, the bill size is standardized, SIFT feature point extraction is carried out on the bill, then, feature matching is carried out on the bill feature points and the template bill feature points, a transformation matrix is obtained, projection transformation is carried out, and bill alignment in the first stage is completed;
the bill alignment of the same kind is to obtain a transformation matrix through the extraction and matching calculation of the feature points, and to perform perspective transformation according to the transformation matrix to complete the bill alignment;
the eight-neighborhood connected body offset correction is the second stage of the text region positioning algorithm, on the basis that the bills are aligned, one information region is selected as a main operation region, the offset error between the character region information on the original bill template and the connected body queue is calculated, the offset error is used for correcting the positions of other regions of the template, and then the accurate position of each information region is obtained;
generating an eight-neighborhood search area for a certain information area in a bill template, then carrying out binarization and morphological processing on the search area, and queuing and storing the communicating bodies in the search area according to a specified sequence to generate a main communicating body queue;
the sequence of the communication bodies in the search area which are queued and stored according to the designated sequence is middle, right, left, upper, lower, upper right, lower right, upper left and lower left;
and calculating the center coordinate of the connected body at the head of the main connected body queue, calculating the offset of the coordinate and the center coordinate of the area in the template, correcting the position of the information area in other templates according to the offset, searching the connected bodies in the same eight neighborhoods of the calculated information area, enqueuing the connected bodies, indicating that the offset is calculated correctly if the connected bodies meeting the conditions are positioned in the main connected body queue at other positions, and if the connected bodies are not dequeued, continuing to perform the same operation on the next connected body in the main connected body queue.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810434069.3A CN108806059B (en) | 2018-05-08 | 2018-05-08 | Text region positioning method based on note alignment and eight-neighborhood connector offset correction of feature points |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810434069.3A CN108806059B (en) | 2018-05-08 | 2018-05-08 | Text region positioning method based on note alignment and eight-neighborhood connector offset correction of feature points |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108806059A CN108806059A (en) | 2018-11-13 |
| CN108806059B true CN108806059B (en) | 2020-05-22 |
Family
ID=64092020
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810434069.3A Active CN108806059B (en) | 2018-05-08 | 2018-05-08 | Text region positioning method based on note alignment and eight-neighborhood connector offset correction of feature points |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108806059B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109558846A (en) * | 2018-11-30 | 2019-04-02 | 厦门商集网络科技有限责任公司 | It is normalized based on OCR template and promotes self-defined template discrimination method and apparatus |
| CN109558844A (en) * | 2018-11-30 | 2019-04-02 | 厦门商集网络科技有限责任公司 | The method and apparatus of self-defined template discrimination is promoted based on image normalization |
| CN109800750A (en) * | 2019-01-25 | 2019-05-24 | 济南浪潮高新科技投资发展有限公司 | A kind of character area mask method of the text based on morphological image for rule composing |
| CN109886257B (en) * | 2019-01-30 | 2022-10-18 | 四川长虹电器股份有限公司 | Method for correcting invoice image segmentation result by adopting deep learning in OCR system |
| CN111275037B (en) * | 2020-01-09 | 2021-06-08 | 上海知达教育科技有限公司 | Bill identification method and device |
| CN111325669A (en) * | 2020-03-05 | 2020-06-23 | 北京远心科技有限责任公司 | Correction scale for oblique photography and oblique photography image correction method |
| CN111612967B (en) * | 2020-04-29 | 2023-03-14 | 武汉卓目科技有限公司 | Method and device for preprocessing CIS image of financial machine |
| SG10202004041SA (en) * | 2020-05-02 | 2021-12-30 | Alipay Labs Singapore Pte Ltd | An identification document authentication method and system |
| CN112132016B (en) * | 2020-09-22 | 2023-09-15 | 平安科技（深圳）有限公司 | Bill information extraction method and device and electronic equipment |
| CN117347367B (en) * | 2023-08-24 | 2024-09-03 | 广州鹰视信息科技有限公司 | Board card device positioning method, board card device detection method, device and medium |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100565559C (en) * | 2007-03-14 | 2009-12-02 | 中国科学院自动化研究所 | Image text location method and device based on connected component and support vector machine |
| CN101377847B (en) * | 2007-08-29 | 2010-06-02 | 中国科学院自动化研究所 | A method for document image registration and feature point selection |
| CN101872475B (en) * | 2009-04-22 | 2012-03-28 | 中国科学院自动化研究所 | A method for automatic registration of scanned document images |
| CN105469408A (en) * | 2015-11-30 | 2016-04-06 | 东南大学 | Building group segmentation method for SAR image |
| CN107103323B (en) * | 2017-03-09 | 2020-06-16 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | A Target Recognition Method Based on Image Contour Features |
| CN107832757B (en) * | 2017-11-03 | 2021-08-31 | 深圳航天信息有限公司 | Invoice image identification method |
-
2018
- 2018-05-08 CN CN201810434069.3A patent/CN108806059B/en active Active
Non-Patent Citations (2)
| Title |
|---|
| 《一种车牌字符位置侦测的新方法》;黎水平、贺建军;《江南大学学报(自然学版)》;20070630;第280-283页 * |
| 《基于八邻域边界跟踪的标号算法》;刘相滨、向坚持、阳波;《计算机工程与应用》;20011201;第125-126、132页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108806059A (en) | 2018-11-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108806059B (en) | Text region positioning method based on note alignment and eight-neighborhood connector offset correction of feature points | |
| Chen et al. | Seqtrack: Sequence to sequence learning for visual object tracking | |
| US11657631B2 (en) | Scalable, flexible and robust template-based data extraction pipeline | |
| US8412730B2 (en) | Image search apparatus and method thereof | |
| US9092697B2 (en) | Image recognition system and method for identifying similarities in different images | |
| CN102388392B (en) | Pattern recognition device | |
| WO2019042426A1 (en) | Augmented reality scene processing method and apparatus, and computer storage medium | |
| JP5468332B2 (en) | Image feature point extraction method | |
| WO2022198969A1 (en) | Method, apparatus, and device for recognizing seal text, and computer readable storage medium | |
| US6574351B1 (en) | Destination address area detection apparatus | |
| JPH10340316A (en) | On-line character recognition device | |
| CN105354533A (en) | Bag-of-word model based vehicle type identification method for unlicensed vehicle at gate | |
| CN109635729B (en) | A form identification method and terminal | |
| JP2011257963A (en) | Image processing device, and processing method and program thereof | |
| CN114936997A (en) | Detection method, detection device, electronic equipment and readable storage medium | |
| CN103235791B (en) | A kind of fingerprint matching optimum position method based on rank | |
| JP5304781B2 (en) | Image collation device, image collation feature storage medium, image collation method, and image collation program | |
| CN107729863B (en) | Human finger vein recognition method | |
| CN114863129A (en) | Instrument numerical analysis method, device, equipment and storage medium | |
| JP2012159990A (en) | Object identification system and method, and feature point position extraction system and method | |
| CN112287763A (en) | Image processing method, apparatus, device and medium | |
| JPWO2015068417A1 (en) | Image collation system, image collation method and program | |
| CN113869314B (en) | Image information extraction method and system for text direction clustering correction | |
| KR102492230B1 (en) | License plate detection apparatus and method using image warping | |
| US12165392B2 (en) | Estimation program, estimation device, generation method of detection model, learning method, and learning device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |