WO2022164031A1

WO2022164031A1 - Method and system for detecting character string by using high-dimensional polynomial regression

Info

Publication number: WO2022164031A1
Application number: PCT/KR2021/019343
Authority: WO
Inventors: 서재흥; 신승; 백영민
Original assignee: 네이버 주식회사; 라인 가부시키가이샤
Priority date: 2021-01-28
Filing date: 2021-12-17
Publication date: 2022-08-04
Also published as: KR20220109246A; KR102560051B1

Abstract

Disclosed are a method and system for detecting a character string by using high-dimensional polynomial regression. The method for detecting a character string, according to an embodiment, may comprise the steps of: setting boxes on text included in an input image; clustering the boxes by group; calculating a trajectory for each of boxes included in a same group; forming text lines according to a preset criterion by tracking the trajectory for each of the boxes; and processing regression analysis of a multidimensional polynomial via each of the text lines.

Description

String detection method and system using high-dimensional polynomial regression

The following description relates to a string detection method and system using high-dimensional polynomial regression.

While conventional Optical Character Reader (OCR) technologies generally focus on improving the performance of character detection, localization, and recognition, another aspect to consider is determining the order in which words in the OCR result are read.

Most commercial applications use the position (and/or rotation, grouping) of words to determine the order of words in OCR results. Positioning, rotation, and grouping of OCR results allows for relatively flat and uniformly rotated documents to be processed. However, this prior art has a problem in that the recognition accuracy is lowered when the page curvature is included in the document or the words are arranged in a non-linear manner.

Some studies use neural networks to estimate word order by combining correlations between positional encoding, language modeling, and OCR word outcomes. The use of neural networks can provide a more generalized and nuanced solution to the OCR reading order problem, and these networks can use modeled information with respect to language structure, context, and layout. However, such a network has a problem in that it requires a relatively large amount of resources and waiting time to be used in most commercial applications.

To provide a string detection method and system that can sequentially read OCR (Optical Character Reader) word results in consideration of the position and angle of each word in an image, and the curvature of a line or sentence formed by the words.

A method for detecting a character string in a computer device including at least one processor, the method comprising: setting, by the at least one processor, a box in text included in an image input; clustering, by the at least one processor, the boxes into groups; calculating, by the at least one processor, a trajectory for each box included in the same group; forming, by the at least one processor, a text line according to a preset criterion by tracing a trajectory for each of the boxes; and processing, by the at least one processor, a regression analysis of a multidimensional polynomial over each of the text lines.

According to one side, the preset criterion is that the trajectory of the first box extending in the first direction meets the second direction end of the second box (or the trajectory of the second box), or the direction opposite to the first direction Including a criterion in which a valid encounter occurs in which the trajectory of the first box extending in the second direction meets the end of the third box (or the trajectory of the third box) in the first direction in the second direction can be characterized as

According to another aspect, the forming of the text line may include: with respect to the first box, the second box (or the trajectory of the second box) or the third box (or the trajectory of the third box) is to be found. tracing the trajectory of the first box within a distance proportional to the length of the first box; and a response that an effective encounter occurs between the trajectory of the first box and the second box (or the trajectory of the second box), or the trajectory of the first box and the third box (or the trajectory of the third box) It may be characterized in that it comprises the step of forming a text line.

According to another aspect, the step of forming a text line in response to the occurrence of the valid encounter may include: the valid encounter occurs between the trajectory of the first box and the second box (or the third box), and the second When each of the first box and the second box (or the third box) is not connected to a specific text line, forming a new text line for the first box and the second box (or the third box) can be characterized as

According to another aspect, the step of forming a text line in response to the occurrence of the valid encounter may include: the valid encounter occurs between the trajectory of the first box and the second box (or the third box), and the second When any one of the first box and the second box (or the third box) is connected to a specific text line, the other one may be connected to the specific text line.

According to another aspect, the step of forming a text line in response to the occurrence of the valid encounter may include: the valid encounter occurs between the trajectory of the first box and the second box (or the third box), and the second When the first box and the second box (or the third box) are respectively connected to different text lines, the different text lines are combined to form one text line.

According to another aspect, the processing of the regression analysis comprises processing the regression analysis using three points of a left midpoint, a right midpoint and a center point of each box corresponding to one text line. can do.

According to another aspect, the processing of the regression analysis may be characterized in that a text line including one box is generated as a linear polynomial from the one box.

According to another aspect, a text line longer than half of the longest text line among polynomially parameterized text lines is set as a trusted text line, and the remaining text lines are set as an untrusted text line, and the regression analysis is processed. The performing may include filtering the multidimensional coefficients of the text lines through a weighted average of the multidimensional coefficients of the polynomial for the trusted text line.

According to another aspect, the filtering comprises: filtering a polynomial for the trust text line between the trust text lines; and filtering the polynomial for the untrusted line using the filtered polynomial for the trusted text line.

According to another aspect, the filtering may include calculating a weight for the weighted average so as to be proportional to a ratio of a length of a polynomial to a text line to a distance between centers of the polynomials.

According to another aspect, the method for detecting a character string includes, by the at least one processor, forming the text line whenever another box or a trajectory of the other box is found until the end of the input image is reached and repeating the step of processing the regression analysis.

Provided is a computer program stored in a computer-readable recording medium in combination with a computer device to execute the method on the computer device.

It provides a computer-readable recording medium in which a program for executing the method in a computer device is recorded.

at least one processor implemented to execute computer-readable instructions, wherein the at least one processor sets a box in text that an input image includes, clusters the box into groups, and assigns to the same group A computer characterized by calculating a trajectory for each included box, tracing the trajectory for each of the boxes to form a text line according to a preset criterion, and processing a multidimensional polynomial regression analysis through each of the text lines provide the device.

In consideration of the position and angle of each word in the image, and the curvature of the line or sentence formed by the words, the OCR (Optical Character Reader) word results can be read in order.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention.

2 is a block diagram illustrating an example of a computer device according to an embodiment of the present invention.

3 is a flowchart illustrating an example of a method for detecting a character string according to an embodiment of the present invention.

4 to 6 are views showing an example of rotation of the OCR box in an embodiment of the present invention.

7 is a diagram illustrating an example of a boundary point of an OCR box according to an embodiment of the present invention.

8 to 11 are diagrams illustrating examples of recognizing a text of an object including a curvature according to an embodiment of the present invention.

12 and 13 are diagrams illustrating an example of tracking a trajectory according to an embodiment of the present invention.

14 and 15 are diagrams illustrating examples of detection results of text lines derived by applying a character string detection method according to an embodiment of the present invention.

[Correction by Rule 91 06.01.2022]
16 to 22 are diagrams sequentially showing results of tracking and updating the trajectory of a box through a character string detection method according to an embodiment of the present invention.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

The character string detection system according to the embodiments of the present invention may be implemented by at least one computer device, and the character string detection method according to the embodiments of the present invention is performed through at least one computer device included in the character string detection system. can be The computer program according to an embodiment of the present invention may be installed and driven in the computer device, and the computer device may perform the string detection method according to the embodiments of the present invention under the control of the driven computer program. The above-described computer program may be stored in a computer-readable recording medium in combination with a computer device to cause the computer to execute the character string detection method.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention. The network environment of FIG. 1 shows an example including a plurality of

electronic devices

110 , 120 , 130 , 140 , a plurality of

servers

150 , 160 , and a network 170 . 1 is an example for explaining the invention, and the number of electronic devices or the number of servers is not limited as in FIG. 1 . In addition, the network environment of FIG. 1 only describes one example of environments applicable to the present embodiments, and the environment applicable to the present embodiments is not limited to the network environment of FIG. 1 .

The plurality of

electronic devices

110 , 120 , 130 , and 140 may be a fixed terminal implemented as a computer device or a mobile terminal. Examples of the plurality of

electronic devices

110 , 120 , 130 , 140 include a smart phone, a mobile phone, a navigation device, a computer, a notebook computer, a digital broadcasting terminal, a personal digital assistant (PDA), and a portable multimedia player (PMP). ), tablet PCs, etc. As an example, in FIG. 1 , the shape of a smartphone is shown as an example of the electronic device 110 , but in embodiments of the present invention, the electronic device 110 is substantially configured to be different through the network 170 using a wireless or wired communication method. It may refer to one of various physical computer devices capable of communicating with the

electronic devices

120 , 130 , 140 and/or the

servers

150 and 160 .

The communication method is not limited, and not only a communication method using a communication network (eg, a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network) that the network 170 may include, but also short-range wireless communication between devices may be included. For example, the network 170 may include a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), and a broadband network (BBN). , the Internet, and the like. In addition, the network 170 may include any one or more of a network topology including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree, or a hierarchical network, etc. not limited

Each of the

servers

150 and 160 communicates with the plurality of

electronic devices

110 , 120 , 130 , 140 and the network 170 through a computer device or a plurality of computers providing commands, codes, files, contents, services, etc. It can be implemented in devices. For example, the server 150 provides a service (eg, a content providing service, a group call service (or a voice conference service) to the plurality of

electronic devices

110 , 120 , 130 , and 140 connected through the network 170 ). , a messaging service, a mail service, a social network service, a map service, a translation service, a financial service, a payment service, a search service, etc.).

2 is a block diagram illustrating an example of a computer device according to an embodiment of the present invention. Each of the plurality of

electronic devices

110 , 120 , 130 , 140 or the

servers

150 and 160 described above may be implemented by the computer device 200 illustrated in FIG. 2 .

As shown in FIG. 2 , the computer device 200 may include a memory 210 , a processor 220 , a communication interface 230 , and an input/output interface 240 . The memory 210 is a computer-readable recording medium and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. Here, a non-volatile mass storage device such as a ROM and a disk drive may be included in the computer device 200 as a separate permanent storage device distinct from the memory 210 . Also, the memory 210 may store an operating system and at least one program code. These software components may be loaded into the memory 210 from a computer-readable recording medium separate from the memory 210 . The separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card. In another embodiment, the software components may be loaded into the memory 210 through the communication interface 230 instead of a computer-readable recording medium. For example, the software components may be loaded into the memory 210 of the computer device 200 based on a computer program installed by files received through the network 170 .

The processor 220 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. The instructions may be provided to the processor 220 by the memory 210 or the communication interface 230 . For example, the processor 220 may be configured to execute a received instruction according to a program code stored in a recording device such as the memory 210 .

The communication interface 230 may provide a function for the computer device 200 to communicate with other devices (eg, the storage devices described above) through the network 170 . For example, a request, command, data, file, etc. generated by the processor 220 of the computer device 200 according to a program code stored in a recording device such as the memory 210 is transmitted to the network ( 170) to other devices. Conversely, signals, commands, data, files, etc. from other devices may be received by the computer device 200 through the communication interface 230 of the computer device 200 via the network 170 . A signal, command, or data received through the communication interface 230 may be transmitted to the processor 220 or the memory 210 , and the file may be a storage medium (described above) that the computer device 200 may further include. persistent storage).

The input/output interface 240 may be a means for an interface with the input/output device 250 . For example, the input device may include a device such as a microphone, keyboard, or mouse, and the output device may include a device such as a display or a speaker. As another example, the input/output interface 240 may be a means for an interface with a device in which functions for input and output are integrated into one, such as a touch screen. The input/output device 250 may be configured as one device with the computer device 200 .

Also, in other embodiments, the computer device 200 may include fewer or more components than those of FIG. 2 . However, there is no need to clearly show most of the prior art components. For example, the computer device 200 may be implemented to include at least a portion of the above-described input/output device 250 or may further include other components such as a transceiver and a database.

Embodiments of the present invention can estimate the overall document curvature by inferring local curvature in text layout and self-filtering (or self-improving) the prediction. This allows you to determine the order of the text even if it is on a curved page or placed in an unconventional layout.

In addition, since embodiments of the present invention are not data-driven and therefore do not require particularly much data for language structure, context and layout, they are much simpler to implement even in resource-constrained environments and can be optimized to significantly reduce latency. can

3 is a flowchart illustrating an example of a method for detecting a character string according to an embodiment of the present invention. The character string detection method according to the present embodiment may be performed by the computer device 200 . In this case, the processor 220 of the computer device 200 may be implemented to execute a control instruction according to a code of an operating system included in the memory 210 or a code of at least one computer program. Here, the processor 220 causes the computer device 200 to perform steps 310 to 340 included in the method of FIG. 3 according to a control command provided by a code stored in the computer device 200 . can control

In operation 310, the computer device 200 may receive an image. Here, the image may include at least one text to be recognized using an optical character reader (OCR). Such an image may be input through the camera of the computer device 200, for example, when the computer device 200 is a terminal device including a camera. As another example, the computer device 200 may receive an image input through a camera in a terminal device including a camera through the network 170 .

In operation 320, the computer device 200 may set the OCR box to the text included in the input image. For example, the computer device 200 may assign an OCR box to each word recognized in the image. In this case, a "word" as a unit in which the OCR box is set may be understood as a group of texts separated by a distance, unlike a word as a dictionary meaning. For example, if the input image includes a character string such as “ABCD EFG”, “ABCD” may be recognized as a single word and an OCR box may be assigned, and “EFG” may be recognized as a single word and OCR A box may be given. In other words, a bundle of texts connected to each other may be recognized as a single word regardless of the meaning of the text, and bundles spaced apart from each other may be recognized as different words. Also, the computer device 200 may rotate the OCR boxes in a forward direction. The rotation of the OCR box will be described in more detail later with reference to FIGS. 4 to 6 .

In operation 330 , the computer device 200 may sample a boundary point of an OCR box for text included in an image as a control point for clustering. The boundary point of the OCR box will be described in more detail later with reference to FIG. 7 .

In operation 340 , the computer device 200 may cluster the OCR box using the sampled control points. For example, the computer device 200 may cluster the OCR boxes for each group by clustering the sampled control points using a clustering algorithm. The clustered OCR boxes may be individually processed in a process of text line detection through regression (hereinafter, a text line detection through regression (TLDR) process) to be described later.

In operation 350, the computer device 200 may calculate the trajectory of each OCR box included in the same group. In this case, the computer device 200 may calculate a major axis vector that is a trajectory of the OCR box based on the corner points of each OCR box. More specifically, the computer device 200 may calculate the principal axis vector as a linear vector using quartiles (quads) for each OCR box. When a boundary polygon is used as the OCR box, the computer device 200 may calculate the trajectory of each OCR box using a centerline and/or an nth-order polynomial corresponding to the centerline.

In operation 360, the computer device 200 may form a text line according to a preset criterion by tracking the trajectory of each OCR box. For example, the computer device 200 may track the trajectory within a distance proportional to the major length of each OCR box until another box is found for each OCR box. The preset criterion is that the trajectory of the first box that runs in the first direction meets the end of the second box (or the trajectory of the second box) in the second direction, or runs in a second direction opposite to the first direction. The trajectory of the first box may include a criterion in which a valid encounter occurs in which the trajectory of the first box meets the end of the third box (or the trajectory of the third box) in the first direction in the second direction.

In other words, the criterion for a "valid encounter" to occur is that the trajectory leading to the left or right of the first box must encounter another box (the second or third box) or the rightmost (or left) of the trajectory of the other box. can As a more specific example, any one of (1) and (2) below for the first OCR box, the second OCR box located to the left of the first OCR box, and the third OCR box located to the right of the first OCR box When the condition of is satisfied, it may be determined that a valid encounter occurs.

(1) A trajectory leading to the left with respect to the trajectory of the first OCR box is, within a transaction proportional to the main length of the first OCR box, a second OCR box located to the left of the first OCR box (or the right end of the trajectory).

(2) a third OCR box located to the right of the first OCR box (or the third OCR box in a transaction where the trajectory that runs to the right with respect to the trajectory of the first OCR box is proportional to the main length of the first OCR box) trajectory) at the left end.

In one example, the computer device 200 is proportional to the length of the first box until a second box (or trajectory of the second box) or a third box (or trajectory of the third box) is found for the first box. It is possible to track the trajectory of the first box within the distance. At this time, the computer device 200 generates an effective encounter between the trajectory of the first box and the second box (or the trajectory of the second box), or the trajectory of the first box and the third box (or the trajectory of the third box). Can form lines of text in response.

For example, it is assumed that a valid encounter with another OCR box (second OCR box or third OCR box) occurs on the left or right for the trajectory of the first OCR box. At this time, when each of the two OCR boxes (the first OCR box and the second OCR box or the first OCR box and the third OCR box) in which a valid encounter has occurred is not connected to a text line, the trajectories of the two OCR boxes A new line of text may be formed. In other words, the computer device 200 generates a valid encounter between the trajectory of the first box and the second box (or the third box), wherein each of the first box and the second box (or the third box) is a specific text line. If not connected to , a new text line for the first box and the second box (or the third box) can be formed.

In addition, when only one OCR box (eg, the first OCR box) of the two OCR boxes is connected to a specific text line (eg, the first text line), the other OCR box (eg, the second OCR box) The OCR box or the third OCR box) may be connected to the corresponding text line (the first OCR box). For example, in the computer device 200, a valid encounter occurs between the trajectory of the first box and the second box (or the third box), and any one of the first box and the second box (or the third box) is specified If it is linked to a line of text, you can link the other to a specific line of text.

Finally, if the two OCR boxes are respectively connected to different text lines (eg, the second text line and the third text line), the different text lines may be combined to form one text line. For example, in the computer device 200, an effective encounter occurs between the trajectory of the first box and the second box (or the third box), and the first box and the second box (or the third box) are different from each other. When connected to text lines, different text lines can be combined to form one text line.

In operation 370, the computer device 200 may process the regression analysis of the n-dimensional polynomial through each text line. The points used in the regression analysis can be obtained from the boundary of the OCR box or the midpoint of the principal axis derived from the boundary point. As an example, the computer device 200 may process a regression analysis using three points for each OCR box: a left midpoint, a right midpoint, and a midpoint of the OCR box. In this case, the OCR boxes may be boxes corresponding to one text line. Also, a text line containing only one OCR box is made from an OCR box that is not part of the text line, and the polynomial of this text line can be considered linear (one-dimensional).

In this case, a text line longer than half of the longest text line parameterized by the polynomial may be regarded as a trusted text line, and the remaining text lines may be regarded as an untrusted text line. The computer device 200 may filter the higher-order coefficients of the text line through a weighted average of the higher-order coefficients of the polynomial for the trusted text line. The polynomial for the trusted text line may first be filtered between the trusted text lines, and then the polynomial for the untrusted text line may be filtered by the filtered polynomial for the filtered trusted text line. Here, the weight may be defined to be proportional to (length of polynomial relative to text line)/(distance between centers of polynomials). When using regression on a quadratic polynomial, you can filter out quadratic coefficients, but when using regression on a higher nth polynomial, the nth coefficients are filtered out, or some of the coefficients between 2 and n are simultaneously can be filtered.

The filtered polynomial is now considered an “updated” trajectory and the computer device 200 may trace the updated trajectory as described above in step 360 . The next distance can be proportional to the length of the polynomial or up to the boundary of the image. In other words, the computer device 200 may track and update the trajectory, repeating

steps

360 and 370 whenever another OCR box or a line of text in another OCR box is found, until the end of the image is reached. .

In operation 380, the computer device 200 may provide the finally tracked and updated trajectory as a result of detecting a text line with respect to the input image. Based on the detection result of these text lines, the OCR word results can be read sequentially in consideration of the curvature of lines or sentences formed by words in the image.

4 to 6 are views showing an example of rotation of the OCR box in an embodiment of the present invention. 4 shows an example of an image 400 including slanted text. 5 shows an example of setting the OCR box to texts, and FIG. 6 shows an example of rotating the OCR boxes in the forward direction. At this time, as shown in FIG. 5 , the OCR boxes may be rotated in the forward direction by rotating the entire region including the OCR boxes, rather than rotating the respective OCR boxes.

7 is a diagram illustrating an example of a boundary point of an OCR box according to an embodiment of the present invention. For example, four vertices 711 to 714 of the box 700 may be used as boundary points. According to an embodiment, the center point 720 of the box 700 , the left midpoint 730 of the box 700 , and/or the right midpoint 740 of the box 700 may be further used as boundary points. According to an embodiment, other points obtained from the boundary line of the box 700 may be further used as boundary points. In addition, according to the embodiment, points on the main axis connecting the center point 720 of the box 700, the left midpoint 730 of the box 700, and the right midpoint 740 of the box 700 are further utilized as boundary points. it might be At this time, the length of the box 700 may be obtained as the length of the main axis (or the length of the line connecting the vertex 711 and the vertex 712 or the length of the line connecting the vertex 713 and the vertex 714).

8 to 11 are diagrams illustrating examples of recognizing a text of an object including a curvature according to an embodiment of the present invention. 8 shows an image 800 in which text of an object including curvature is listed according to the curvature of the object. In this case, as shown in FIG. 9 , the computer device 200 may set an OCR box for each word recognized in the image. Also, the computer device 200 may calculate a trajectory for each OCR box. FIG. 10 shows an example in which three

boxes

1010 , 1020 , and 1030 are enlarged and displayed among the boxes of FIG. 9 . Also, FIG. 10 shows an example in which the trajectory of the box 1020 meets the right end of the box 1010 on the left side and meets the left end of the box 1030 on the right side. In other words, it can be seen that a “valid encounter” has occurred for the trajectory of box 1020 . At this time, since all three

boxes

1010 , 1020 , and 1030 are not connected to a specific text line, a new text line for the three

boxes

1010 , 1020 and 1030 may be formed. For example, the trajectories for each of the three

boxes

1010 , 1020 , and 1030 may be combined to form one new text line. Assuming that trajectory 1040 is a line of text in another embodiment, it will be readily understood that

boxes

1010 and 1030 for which a valid query for box 1020 occurred may be connected to the line of text represented by trajectory 1040 . will be able

In this case, the computer device 200 may use the polynomial obtained by processing the regression analysis of the n-dimensional polynomial through the text line to which the three

boxes

1010 , 1020 , and 1030 are connected as the updated trajectory. 11 shows an example of such an updated trajectory 1110 . The tracking of the trajectory 1040 of the box 1020 may be made within a length proportional to the length of the box 1020 . As an example, tracking of trajectory 1040 may be made for each of the left and right sides within a length of one-half the length of box 1020 .

12 and 13 are diagrams illustrating an example of tracking a trajectory according to an embodiment of the present invention. When the distortion due to the curvature of the object is severe, there is a case where the trajectory of the box 1210 does not meet the box 1220 as in the

boxes

1210 and 1220 of FIG. 12 . In this case, a “valid encounter” may occur due to an encounter between the trajectory 1230 of the box 1210 and the trajectory 1240 of the box 1220 . In this case, a new text line may be formed by the combination of the two

trajectories

1230 and 1240 as shown in FIG. 12 , and the polynomial obtained by processing the regression analysis of the n-dimensional polynomial through the new text line will be utilized as the updated trajectory. can 13 shows an example of such an updated trajectory 1310 .

As already described, the computer device 200 may repeat tracking and updating of this trajectory whenever another OCR box or text line of another OCR box is found until the end of the image is reached.

14 and 15 are diagrams illustrating examples of detection results of text lines derived by applying a character string detection method according to an embodiment of the present invention. 14 shows an example of an input image, and FIG. 15 shows a detection result of a text line.

[Correction by Rule 91 06.01.2022]
16 to 22 are diagrams sequentially showing results of tracking and updating the trajectory of a box through a character string detection method according to an embodiment of the present invention. The image of FIG. 16 shows an example in which the OCR box is set. In this case, the OCR boxes of different groups are divided into boxes of different colors. 17 to 22 show examples in which curves as text lines are changed according to the curvature of an object (receipt) through tracking of the trajectory and updating of the trajectory through regression analysis of an n-dimensional polynomial. As already described, tracking and updating of the trajectory may be repeatedly performed whenever another box or a trajectory of the other box is found (valid encounter) until the end of the input image is reached.

As such, according to embodiments of the present invention, the OCR word results can be sequentially read in consideration of the position and angle of each word in the image, and the curvature of a line or sentence formed by the words. Further, according to embodiments of the present invention, by inferring local curvature in text layout and self-filtering prediction to estimate overall document curvature, the order of text even when text is on a curved page or laid out in an unconventional layout can be decided In addition, according to embodiments of the present invention, since it is not data-based and does not require a lot of data in particular for language structure, context and layout, it is much simpler to implement even in a resource-constrained environment and greatly reduces latency. can be optimized to be

The system or apparatus described above may be implemented as a hardware component or a combination of a hardware component and a software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. may be embodied in The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The medium may continuously store a computer executable program, or may be a temporary storage for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or several hardware combined, it is not limited to a medium directly connected to any computer system, and may exist distributed on a network. Examples of the medium include a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as CD-ROM and DVD, a magneto-optical medium such as a floppy disk, and those configured to store program instructions, including ROM, RAM, flash memory, and the like. In addition, examples of other media may include recording media or storage media managed by an app store that distributes applications, sites that supply or distribute various other software, or servers. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims

A method for detecting a character string in a computer device comprising at least one processor, the method comprising:

setting, by the at least one processor, a box in text included in the input image;

clustering, by the at least one processor, the boxes into groups;

calculating, by the at least one processor, a trajectory for each box included in the same group;

forming, by the at least one processor, a text line according to a preset criterion by tracing a trajectory for each of the boxes; and

processing, by the at least one processor, a regression analysis of a multidimensional polynomial over each of the lines of text;

A string detection method comprising
According to claim 1,

The preset criterion is that the trajectory of the first box extending in the first direction meets the second direction end of the second box (or the trajectory of the second box), or a second direction opposite to the first direction. wherein the trajectory of the first box leading to includes a criterion in which a valid encounter occurs in which the trajectory of the first box meets the end of the third box (or the trajectory of the third box) in the first direction in the second direction String detection method.
3. The method of claim 2,

Forming the text line comprises:

within a distance proportional to the length of the first box, until the second box (or the trajectory of the second box) or the third box (or the trajectory of the third box) is found for the first box tracking the trajectory of the first box; and

In response to a valid encounter occurring between the trajectory of the first box and the second box (or the trajectory of the second box), or between the trajectory of the first box and the third box (or the trajectory of the third box) Steps to form a line of text

A string detection method comprising a.
4. The method of claim 3,

forming a line of text in response to the occurrence of the valid encounter comprises:

The effective encounter occurs between the trajectory of the first box and the second box (or the third box), and each of the first box and the second box (or the third box) is connected to a specific text line. If not, a new text line for the first box and the second box (or the third box) is formed.
4. The method of claim 3,

forming a line of text in response to the occurrence of the valid encounter comprises:

The effective encounter occurs between the trajectory of the first box and the second box (or the third box), and any one of the first box and the second box (or the third box) is on a specific text line. When connected, the method for detecting a character string, characterized in that the other is connected to the specific text line.
4. The method of claim 3,

forming a line of text in response to the occurrence of the valid encounter comprises:

The effective encounter occurs between the trajectory of the first box and the second box (or the third box), and the first box and the second box (or the third box) are connected to different text lines, respectively , a character string detection method, characterized in that by combining the different text lines to form one text line.
According to claim 1,

The step of processing the regression analysis,

and processing the regression analysis using three points of a left midpoint, a right midpoint, and a center point of each box corresponding to one text line.
According to claim 1,

The step of processing the regression analysis,

A method for detecting a character string, characterized in that a text line including one box is generated from the one box as a linear polynomial.
According to claim 1,

Among the polynomially parameterized text lines, a text line longer than half the longest text line is set as a trusted text line, and the remaining text lines are set as an untrusted text line,

The step of processing the regression analysis,

filtering the multidimensional coefficients of the text lines through a weighted average of the multidimensional coefficients of the polynomial for the trusted text line;

A string detection method comprising a.
10. The method of claim 9,

The filtering step is

filtering a polynomial for the line of trust text between the lines of trust text; and

filtering the polynomial for the untrusted line using the polynomial for the filtered trusted text line.

A string detection method comprising a.
10. The method of claim 9,

The filtering step is

and calculating a weight for the weighted average so as to be proportional to the ratio of the length of the polynomial to the text line to the distance between the centers of the polynomials.
The method of claim 1,

repeating, by the at least one processor, forming the text line and processing the regression analysis whenever another box or a trajectory of the other box is found, until the end of the input image is reached step

A string detection method further comprising a.
A computer program stored in a computer-readable recording medium in combination with a computer device to cause the computer device to execute the method of any one of claims 1 to 12.
13. A computer-readable recording medium in which a computer program for executing the method of any one of claims 1 to 12 in a computer device is recorded.
at least one processor implemented to execute computer-readable instructions

including,

by the at least one processor,

Set a box to the text that the input image contains,

clustering the boxes by group,

Calculate the trajectory for each box included in the same group,

Tracing the trajectory for each of the boxes to form a text line according to a preset standard,

processing a regression analysis of a multidimensional polynomial through each of said lines of text;

A computer device characterized by a.
16. The method of claim 15,

The preset criterion is that the trajectory of the first box extending in the first direction meets the second direction end of the second box (or the trajectory of the second box), or a second direction opposite to the first direction. Containing criteria in which a valid encounter occurs in which the trajectory of the first box leading to the second direction meets the end of the third box (or the trajectory of the third box) in the first direction.

A computer device characterized by a.
16. The method of claim 15,

to process the regression analysis, by the at least one processor,

generating a text line containing a box as a linear polynomial from the one box

A computer device characterized by a.
16. The method of claim 15,

Among the polynomially parameterized text lines, a text line longer than half of the longest text line is set as a trusted text line, and the remaining text lines are set as an untrusted text line,

to process the regression analysis, by the at least one processor,

filtering the multidimensional coefficients of the text lines via a weighted average of the multidimensional coefficients of the polynomial for the confidence text line.

A computer device characterized by a.