TWI799050B - Electronic device and method for detecting and recognizing character - Google Patents

Abstract

An electronic device and a method for detecting and recognizing a character. The method includes: receiving a text image; detecting the text image to generate a plurality of bounding boxes corresponding to a plurality of characters respectively and calculating an inclined angle corresponding to the text image according to the plurality of bounding boxes; generating a plurality of recognition results corresponding to the plurality of characters respectively according to the plurality of bounding boxes; generating a plurality of orders corresponding to the plurality of characters respectively according to the inclined angle and the plurality of bounding boxes; matching the plurality of recognition results and the plurality of orders to generate a character string; and outputting the character string.

Description

Electronic device and method for detecting and recognizing text

The invention relates to an electronic device and method for detecting and recognizing characters.

Under the tide of digital transformation promoted by enterprises, the digitization of physical data is a basic and crucial step. Substantial data may include a large number of written documents, contracts, forms or signed documents. Users can perform character recognition on images of physical data to generate digital data. However, depending on the industry, there may be significant format differences between different users' entity data. In addition, the source of physical data may be generated by scanning a document by a scanner, or by a user taking a photo of a document with a camera. Format differences and source differences of images of physical data add many difficulties to character recognition.

The invention provides an electronic device and method for detecting and recognizing characters, which can accurately detect and recognize characters on the document when the document in the character image is tilted.

An electronic device for detecting and identifying characters of the present invention includes a processor, a storage medium and a transceiver. The transceiver receives text images. The storage medium stores multiple modules. The processor is coupled to the storage medium and the transceiver, and accesses and executes multiple modules, wherein the multiple modules include a character detection module, a character recognition module, a character sorting module, a matching module, and an output module Group. The character detection module detects the text image to generate a plurality of bounding boxes corresponding to the plurality of characters, and calculates an inclination angle corresponding to the text image according to the plurality of bounding boxes. The character recognition module generates a plurality of recognition results respectively corresponding to the plurality of characters according to the plurality of bounding boxes. The text sorting module generates a plurality of sequences corresponding to the plurality of bounding boxes according to the tilt angle and the plurality of bounding boxes. The matching module matches multiple recognition results and multiple sequences to generate a word string. The output module outputs the string through the transceiver.

In an embodiment of the present invention, the above-mentioned plurality of bounding frames includes a first bounding frame, wherein the character detection module responds that the second bounding frame of the plurality of bounding frames is closest to the first bounding frame frame and calculate the first tilt angle according to the first bounding box and the second bounding box, and calculate the tilt angle according to the first tilt angle.

In an embodiment of the present invention, the above character detection module calculates the tilt angle according to the first center point of the first bounding box and the second center point of the second bounding box.

In an embodiment of the present invention, the above-mentioned character detection module performs coordinate conversion on a plurality of coordinate parameters respectively corresponding to a plurality of bounding boxes according to the inclination angle to update the plurality of coordinate parameters, wherein the character sorting module according to The updated coordinate parameters generate multiple orders.

In an embodiment of the present invention, the above-mentioned text sorting module is configured to execute: select a first bounding box corresponding to the smallest vertical coordinate from multiple bounding boxes according to a plurality of coordinate parameters; A first coordinate parameter of the frame to select at least a certain bounding box from a plurality of bounding boxes; select a second bounding box corresponding to the smallest abscissa from at least a certain bounding box; and assign an order to the second bounding box to generate multiple bounding boxes sequence.

In an embodiment of the present invention, the above-mentioned text sorting module removes the second coordinate parameter of the second bounding box from the plurality of coordinate parameters in response to assigning an order to the second bounding box, thereby updating the plurality of coordinates parameters, wherein the text sorting module generates multiple sequences according to the updated multiple coordinate parameters.

In an embodiment of the present invention, the above-mentioned first coordinate parameter includes a first abscissa, a first ordinate corresponding to the first abscissa, a second abscissa, and a second ordinate corresponding to the second abscissa, wherein the first abscissa is smaller than the second abscissa, wherein the text sorting module is further configured to: set the first abscissa of the first coordinate parameter to zero to generate a reference bounding box; and respond to at least a certain bounding box with The overlapping area between the reference bounding boxes is greater than a threshold, and at least a certain bounding box is selected from the plurality of bounding boxes.

In an embodiment of the present invention, the above character detection module generates a plurality of bounding boxes according to a machine learning model.

In an embodiment of the present invention, the above character recognition module generates a plurality of recognition results according to the optical character recognition model.

In an embodiment of the present invention, the aforementioned inclination angle is greater than or equal to negative forty-five degrees and less than or equal to forty-five degrees.

In an embodiment of the present invention, the above-mentioned character image corresponds to a horizontal writing format.

A method for detecting and recognizing characters of the present invention includes: receiving a character image; detecting the character image to generate a plurality of bounding boxes corresponding to a plurality of characters, and calculating the corresponding character image according to the plurality of bounding boxes The inclination angle; generate multiple recognition results corresponding to multiple characters according to multiple bounding boxes; generate multiple sequences corresponding to multiple characters according to the tilt angle and multiple bounding boxes; match multiple recognitions result and multiple sequences to generate a string; and output a string.

Based on the above, the electronic device of the present invention can accurately detect the position of characters on the document when the document in the text image is tilted, so as to digitize the content of the document according to the position of the characters.

In order to make the content of the present invention more comprehensible, the following specific embodiments are taken as examples in which the present invention can actually be implemented. In addition, wherever possible, elements/components/steps using the same reference numerals in the drawings and embodiments represent the same or similar parts.

FIG. 1 shows a schematic diagram of an electronic device 100 for detecting and recognizing characters according to an embodiment of the present invention. The electronic device 100 may include a processor 110 , a storage medium 120 and a transceiver 130 .

The processor 110 is, for example, a central processing unit (central processing unit, CPU), or other programmable general purpose or special purpose micro control unit (micro control unit, MCU), microprocessor (microprocessor), digital signal processing Digital signal processor (DSP), programmable controller, application specific integrated circuit (ASIC), graphics processing unit (graphics processing unit, GPU), image signal processor (image signal processor, ISP) ), image processing unit (image processing unit, IPU), arithmetic logic unit (arithmetic logic unit, ALU), complex programmable logic device (complex programmable logic device, CPLD), field programmable logic gate array (field programmable gate array , FPGA) or other similar components or combinations of the above components. The processor 110 can be coupled to the storage medium 120 and the transceiver 130 , and access and execute multiple modules and various application programs stored in the storage medium 120 .

The storage medium 120 is, for example, any type of fixed or removable random access memory (random access memory, RAM), read-only memory (read-only memory, ROM), flash memory (flash memory) , hard disk drive (hard disk drive, HDD), solid state drive (solid state drive, SSD) or similar components or a combination of the above components, and are used to store multiple modules or various application programs executable by the processor 110 . In this embodiment, the storage medium 120 can store multiple modules including a character detection module 121, a character recognition module 122, a character sorting module 123, a matching module 124, and an output module 125. Will be explained later.

The transceiver 130 transmits and receives signals in a wireless or wired manner. The transceiver 130 may also perform operations such as low noise amplification, impedance matching, frequency mixing, up or down frequency conversion, filtering, amplification, and the like.

FIG. 2 shows a schematic diagram of multiple modules of the electronic device 100 according to an embodiment of the present invention. First, the character detection module 121 can receive the character image S1 through the transceiver 130 . The character image S1 is, for example, an RGB image. In one embodiment, the character image S1 corresponds to a horizontal writing format. Text image S1 may contain skewed documents. For example, a user can take a picture of a document with a photography device to generate a text image S1 including the document. Due to factors such as the shooting environment or poor shooting techniques, the document in the text image S1 may be tilted, thereby increasing the difficulty of character recognition.

FIG. 3 shows a schematic diagram of a text image S1 according to an embodiment of the present invention. The text image S1 may include a skewed document 30 . Since the document 30 is slanted in the text image S1, the characters on the document 30 are also slanted in the text image S1. Therefore, it is difficult for conventional character recognition technology to recognize the characters on the document 30 .

Referring back to FIG. 2 , the character detection module 121 can detect the character image S1 to generate a plurality of bounding boxes respectively corresponding to a plurality of characters. In one embodiment, the character detection module 121 can input the character image S1 into the machine learning model to generate a plurality of bounding boxes. For example, the character detection module 121 can input the character image S1 into an object detection model to generate a plurality of bounding boxes respectively corresponding to a plurality of characters.

The character detection module 121 can send the message S2 associated with a plurality of bounding boxes to the character recognition module 122, wherein each bounding box of the plurality of bounding boxes can enclose a character. The message S2 may include the serial number of the bounding box (or character) and the coordinate parameters of the bounding box. Table 1 is an example of the sequence numbers of the bounding boxes in FIG. 3 . Table 1 bounding box serial number 31 #1 32 #2 33 #3 34 #4 35 #5

The coordinate parameters of the bounding box may include coordinates of two vertices of the bounding box, where the two vertices may include a vertex with a minimum abscissa and minimum ordinate and a vertex with a maximum abscissa and maximum ordinate. Taking the bounding box 31 in FIG. 3 as an example, it is assumed that the horizontal axis is the X axis and the vertical axis is the Y axis. The bounding box 31 may include four vertices such as a vertex P1 , a vertex P2 , a vertex P3 and a vertex P4 . Among the four vertices, the vertex P1 has the smallest X coordinate and the smallest Y coordinate, and the vertex P4 has the largest X coordinate and the largest Y coordinate. Therefore, the coordinate parameters of the bounding box 31 may include the coordinates of the vertex P1 and the coordinates of the vertex P2.

代表定界框i的最小X座標，

代表定界框i的最小y座標，

代表定界框i的最大X座標並且

代表定界框i的最大y座標。舉例來說，

可代表定界框31的頂點P1的X座標，

可代表定界框31的頂點P1的Y座標，

可代表定界框31的頂點P4的X座標並且

可代表定界框31的頂點P4的Y座標。

…(1) In one embodiment, the character detection module 121 can convert the normalized coordinate parameters (that is, the coordinate parameters generated by the machine learning model) into actual coordinate parameters, and then send the actual coordinate parameters through the message S2 To character recognition module 122. The character recognition module 122 can perform character recognition according to the actual coordinate parameters. Taking the bounding box 31 of FIG. 3 as an example, assuming that the length of the text image S1 (or the projection length of the text image S1 on the X-axis) is L pixels, and the width of the text image S1 is W pixels, then the character detection The measurement module 121 can convert the coordinate parameter F of the bounding box 31 into a real coordinate parameter F' according to the equation (1), wherein

Represents the minimum X coordinate of the bounding box i,

represents the minimum y-coordinate of the bounding box i,

represents the maximum X coordinate of the bounding box i and

Represents the maximum y-coordinate of the bounding box i. for example,

Can represent the X coordinate of the vertex P1 of the bounding box 31,

can represent the Y coordinate of the vertex P1 of the bounding box 31,

can represent the X coordinate of the vertex P4 of the bounding box 31 and

may represent the Y coordinate of the vertex P4 of the bounding box 31 .

…(1)

The character recognition module 122 can obtain a plurality of bounding boxes respectively corresponding to a plurality of characters on the document 30 from the message S2. Then, the character recognition module 122 can generate a plurality of recognition results respectively corresponding to the plurality of characters according to the plurality of bounding boxes. Specifically, the character recognition module 122 can input the text image S1 including a plurality of bounding boxes into an optical character recognition model (optical character recognition, OCR) to generate a plurality of recognitions respectively corresponding to the plurality of bounding boxes. result. FIG. 4 shows a schematic diagram of executing a text image S1 including a plurality of bounding boxes according to an embodiment of the present invention. The character recognition module 122 can judge according to the optical character recognition model that the character in the bounding box 31 is "A", the character in the bounding box 32 is "B", and the character in the bounding box 33 is " C", the character in bounding box 34 is "D" and the character in bounding box 35 is "E".

The character recognition module 122 can send a plurality of recognition results to the matching module 124 through the message S4. The message S4 may include the serial number of the bounding box and the recognition result of the characters in the bounding box. Table 2 is an example of the sequence numbers and recognition results of the bounding boxes in FIG. 4 . Table 2 bounding box serial number Identification result 31 #1 A 32 #2 B 33 #3 C 34 #4 D. 35 #5 E.

32 72 #2

33 73 #3

34 74 #4

35 75 #5

Refer to Figure 2 and Figure 3. The character detection module 121 can calculate the angle of inclination of the document 30 in the text image S1 according to the plurality of bounding boxes in FIG. 3 . The character detection module 121 can send the message S3 associated with a plurality of bounding boxes and tilt angles to the character sorting module 123 . Specifically, the character detection module 121 may perform coordinate conversion on a plurality of coordinate parameters respectively corresponding to a plurality of bounding boxes according to the tilt angle to update the plurality of coordinate parameters. The character detection module 121 can send the updated coordinate parameters to the character sorting module 123 through the message S3. The message S3 may include the serial number of the bounding frame and the updated coordinate parameters of the bounding frame. The message S3 can be used to generate multiple bounding boxes as shown in FIG. 7A. Table 3 is an example of the serial number of the bounding box and the coordinate parameters of the updated (or transformed) bounding box. table 3 bounding box updated bounding box serial number Coordinate parameter 31 71 #1

32 72 #2

33 73 #3

34 74 #4

35 75 #5

FIG. 5 illustrates a flow chart of performing coordinate transformation according to an embodiment of the invention. After the character detection module 121 generates a plurality of bounding boxes of the text image S1, in step S501, the character detection module 121 can obtain the center point of the bounding box according to the coordinate parameters of the bounding box. Taking FIG. 3 as an example, the coordinate parameters of the bounding box 31 may include the coordinates of the vertex P1 and the coordinates of the vertex P4. The character detection module 121 can calculate the average of the coordinates of the vertex P1 and the coordinates of the vertex P4 to determine the center point C1 of the bounding box. Based on a similar method, the character detection module 121 can obtain the center point C2 of the bounding box 32 , the center point C3 of the bounding box 33 , the center point C4 of the bounding box 34 , and the center point C5 of the bounding box 35 .

為中心點C1的X座標，

為中心點C1的Y座標，

為中心點C2的X座標，並且

為中心點C2的Y座標。

…(2) In step S502, the character detection module 121 can calculate the inclination angle between the nearest bounding boxes. For example, the character detection module 121 may calculate the tilt angle θ1 according to the bounding box 31 and the bounding box 32 in response to the bounding box 32 among the plurality of bounding boxes being closest to the bounding box 31 . In one embodiment, the character detection module 121 can calculate the tilt angle θ1 according to the center point C1 of the bounding box 31 and the center point C2 of the bounding box 32 . Specifically, the character detection module 121 can calculate the tilt angle θ1 according to equation (2), where

is the X coordinate of the center point C1,

is the Y coordinate of the center point C1,

is the X coordinate of the center point C2, and

is the Y coordinate of the center point C2.

…(2)

Based on the same method as equation (2), the character detection module 121 can generate multiple tilt angles. The character detection module 121 can calculate the tilt angle θ2 according to the center point C2 of the bounding box 32 and the center point C3 of the bounding box 33 . The character detection module 121 can calculate the tilt angle θ3 according to the center point C4 of the bounding box 34 and the center point C5 of the bounding box 35 .

In step S503, the character detection module 121 can calculate the average inclination angle (ie: the average of the multiple inclination angles) according to the multiple inclination angles, and set the average inclination angle as the inclination angle of the document 30, wherein the inclination angle can be Greater than or equal to minus 45 degrees and less than or equal to 45 degrees, so that the character sorting module 123 can correctly determine the order of each character.

In step S504, the character detection module 121 may perform coordinate transformation on the coordinate parameters of the bounding box according to the tilt angle to update the coordinate parameters. For example, the character detection module 121 can multiply the coordinate parameter by the cosine of the tilt angle to update the coordinate parameter. After generating the updated coordinate parameters of the bounding box, the character detection module 121 can send the updated coordinate parameters to the character sorting module 123 through the message S3.

The character sorting module 123 can obtain a plurality of updated coordinate parameters respectively corresponding to a plurality of characters from the message S3. The text sorting module 123 can generate multiple sequences respectively corresponding to multiple bounding boxes according to the updated multiple coordinate parameters.

FIG. 6 is a flowchart illustrating a sequence of character generation according to an embodiment of the present invention. In step S601 , the text sorting module 123 can select a first bounding box corresponding to the smallest vertical coordinate from a plurality of bounding boxes according to a plurality of coordinate parameters. In other words, if among multiple bounding boxes, the ordinate of the reference point (for example: vertex or center point) of the first bounding box is smaller than the ordinate of the reference points of other bounding boxes, the text sorting module 123 The first bounding box may be selected from a plurality of bounding boxes.

7A and 7B illustrate a schematic diagram of a bounding box generated according to the message S3 according to an embodiment of the present invention. Referring to FIG. 7A, after the coordinate conversion is completed, bounding boxes 31, 32, 33, 34, and 35 in FIG. 3 are converted into bounding boxes 71, 72, 73, 74, and 75 in FIG. 7A, respectively. The text sorting module 123 can select the bounding box 72 corresponding to the smallest Y coordinate from the multiple bounding boxes in FIG. 7A .

In step S602, the text sorting module 123 can select at least a certain bounding box from a plurality of bounding boxes according to the first coordinate parameter of the first bounding box. Specifically, the first coordinate parameter may include a first abscissa, a first ordinate corresponding to the first abscissa, a second abscissa and a second ordinate corresponding to the second abscissa, wherein the first abscissa is less than The second abscissa, and the first ordinate is smaller than the second ordinate. The text sorting module 123 can set the first abscissa to zero to generate a reference bounding box. Next, the text sorting module 123 may select at least a certain bounding box from the plurality of bounding boxes in response to an overlapping area between at least a certain bounding box and the reference bounding box being greater than a threshold.

。文字排序模組123可根據定界框72的座標參數選出定界框71與定界框72。具體來說，文字排序模組123可將定界框的X座標

設為零以產生座標參數為

的參考定界框81。接著，文字排序模組123可響應於定界框71與參考定界框81之間的重疊區域大於定界框71的50%而選出定界框71。此外，文字排序模組123可響應於定界框72與參考定界框81之間的重疊區域大於定界框72的50%而選出定界框72。 Taking the bounding box 72 of Fig. 7A as an example, suppose that the bounding box 72 is selected as the first bounding box in step S601, the threshold is 50% of the bounding box, and the coordinate parameter of the bounding box 72 is

. The text sorting module 123 can select the bounding box 71 and the bounding box 72 according to the coordinate parameters of the bounding box 72 . Specifically, the text sorting module 123 can set the X coordinate of the bounding box

Set to zero to produce a coordinate parameter of

The reference bounding box 81. Next, the text sorting module 123 can select the bounding box 71 in response to the overlapping area between the bounding box 71 and the reference bounding box 81 being greater than 50% of the bounding box 71 . In addition, the text sorting module 123 may select the bounding box 72 in response to the overlapping area between the bounding box 72 and the reference bounding box 81 being greater than 50% of the bounding box 72 .

In step S603, the text sorting module 123 may select a second bounding box corresponding to the minimum abscissa from at least certain bounding boxes. In other words, if in at least a certain bounding box, the abscissa of the reference point (for example: vertex or center point) of the second bounding box is smaller than the abscissa of reference points of other bounding boxes, the text sorting module 123 may A second bounding box is selected from at least certain bounding boxes.

，並且定界框72的座標參數為

。假設參考點為定界框中具有最小橫座標的頂點。換句話說，定界框71的參考點的X座標為

，並且定界框72的參考點的X座標為

。在根據參考定界框81選出定界框71和定界框72後，文字排序模組123可響應於定界框71的X座標

小於定界框72的X座標

而從定界框71和定界框72兩者中選出定界框71以作為第二定界框。 Taking Fig. 7A as an example, the coordinate parameters of the bounding box 71 are

, and the coordinate parameters of the bounding box 72 are

. The reference point is assumed to be the vertex with the smallest abscissa in the bounding box. In other words, the X coordinate of the reference point of the bounding box 71 is

, and the X coordinate of the reference point of the bounding box 72 is

. After selecting the bounding box 71 and the bounding box 72 according to the reference bounding box 81, the text sorting module 123 can respond to the X coordinate of the bounding box 71

X coordinate smaller than bounding box 72

And the bounding box 71 is selected from both the bounding box 71 and the bounding box 72 as the second bounding box.

In step S604, the text sorting module 123 can assign a sequence to the second bounding box. The text sorting module 123 may remove the second coordinate parameter of the second bounding frame from the plurality of coordinate parameters in response to assigning the order to the second bounding frame, thereby updating the plurality of coordinate parameters.

For example, if the bounding box 71 is the first selected bounding box among all the bounding boxes in FIG. , the text sorting module 123 can assign the first order to the bounding box 71 . After allocating the bounding box 71 in the first order, the text sorting module 123 may remove the coordinate parameter of the bounding box 71 from the plurality of coordinate parameters, so as to update the plurality of coordinate parameters. The updated coordinate parameters can be used to generate bounding boxes as shown in FIG. 7B.

In step S605 , the text sorting module 123 can determine whether there is a bounding box that is not assigned a sequence. If there is still a bounding box that has not been assigned a sequence, step S601 is re-executed. The second iteration of the flow chart in FIG. 6 is described below with FIG. 7B. In step S601 , the text sorting module 123 may select the bounding box 72 corresponding to the smallest ordinate from the plurality of bounding boxes as the first bounding box. In step S602 , the text sorting module 123 can generate a reference bounding box 82 according to the bounding box 72 . The text sorting module 123 can select the bounding box 72 overlapping with the reference bounding box 82 from the multiple bounding boxes as at least a certain bounding box. In step S603 , the text sorting module 123 may select the bounding box 72 corresponding to the smallest abscissa from at least certain bounding boxes as the second bounding box. In step S604, the text sorting module 123 can assign the second order to the bounding box 72, and remove the coordinate parameter of the bounding box 72 from the plurality of coordinate parameters. In step S605 , the text sorting module 123 can determine whether there is a bounding box that is not assigned a sequence. After repeatedly executing the process of FIG. 6 , the text sorting module can assign a corresponding sequence to each bounding box or each character, thereby generating multiple sequences corresponding to multiple bounding boxes.

In step S605, if the text sorting module 123 judges that there is no bounding box that has not been assigned a sequence, the text sorting module 123 can send a plurality of sequences corresponding to multiple bounding boxes to the matching module through a message S5 Group 124. The message S5 may include the serial number of the bounding frame and the sequence of the bounding frame. Table 4 is an example of the sequence numbers of the bounding boxes and the order of the bounding boxes. Table 4 bounding box updated bounding box serial number order 31 71 #1 first order 32 72 #2 second order 33 73 #3 third order 34 74 #4 fourth order 35 75 #5 fifth order

Returning to FIG. 2 , the matching module 124 can receive the message S4 from the character recognition module 122 and the message S5 from the character sorting module 123 . The matching module 124 can obtain a plurality of serial numbers and a plurality of recognition results respectively corresponding to a plurality of bounding boxes from the message S4, and can obtain a plurality of serial numbers and a plurality of sequences corresponding to a plurality of bounding boxes respectively from the message S5 . The matching module 124 can match a plurality of recognition results with a plurality of sequences to generate a word string. The matching module 124 can send the string to the output module 125 through the message S6. The output module 125 can output the string through the transceiver 130 . For example, the matching module 124 can fuse the information in Table 2 and Table 4 to generate the fused information shown in Table 5. The matching module 124 can arrange a plurality of recognition results in order to generate the word string "ABCDE". table 5 bounding box serial number Identification result order 31 #1 A first order 32 #2 B second order 33 #3 C third order 34 #4 D. fourth order 35 #5 E. fifth order

FIG. 8 shows a flowchart of a method for detecting and recognizing characters according to an embodiment of the present invention, wherein the method can be implemented by the electronic device 100 shown in FIG. 1 . In step S801, a text image is received. In step S802, the text image is detected to generate a plurality of bounding boxes corresponding to the plurality of characters, and an inclination angle corresponding to the text image is calculated according to the plurality of bounding boxes. In step S803, a plurality of recognition results respectively corresponding to the plurality of characters are generated according to the plurality of bounding boxes. In step S804, a plurality of sequences respectively corresponding to the plurality of characters are generated according to the tilt angle and the plurality of bounding boxes. In step S805, a plurality of recognition results and a plurality of sequences are matched to generate a word string. In step S806, a character string is output.

To sum up, the electronic device of the present invention can acquire text images and recognize characters on documents in the text images. In addition, the electronic device can determine the inclination angle of the document according to the characters on the document, so as to correct the position of the bounding box of each character according to the inclination angle. The electronic device can determine the sequence corresponding to each character according to the corrected bounding box, so as to arrange the recognized characters to generate a word string. Accordingly, even if the document in the text image is skewed, the electronic device can still digitize the content of the document smoothly.

100: Electronic device 110: Processor 120: storage media 121:Character detection module 122:Character recognition module 123:Text sorting module 124: Matching module 125: Output module 130: Transceiver 30: File 31, 32, 33, 34, 35, 71, 72, 73, 74, 75: bounding box 81, 82: Reference bounding box C1, C2, C3, C4, C5: center point L: the length of the text image P1, P2, P3, P4: vertices S1: text image S2, S3, S4, S5, S6: message S501, S502, S503, S504, S601, S602, S603, S604, S605, S801, S802, S803, S804, S805, S806: steps W: the width of the text image θ1, θ2, θ3: tilt angle

FIG. 1 shows a schematic diagram of an electronic device for detecting and recognizing characters according to an embodiment of the present invention. FIG. 2 is a schematic diagram illustrating a plurality of modules of an electronic device according to an embodiment of the present invention. FIG. 3 is a schematic diagram illustrating a text image according to an embodiment of the present invention. FIG. 4 illustrates a schematic diagram of executing a text image including multiple bounding boxes according to an embodiment of the present invention. FIG. 5 illustrates a flow chart of performing coordinate transformation according to an embodiment of the invention. FIG. 6 is a flowchart illustrating a sequence of character generation according to an embodiment of the present invention. 7A and 7B illustrate a schematic diagram of a bounding box generated according to the message S3 according to an embodiment of the present invention. FIG. 8 shows a flow chart of a method for detecting and recognizing characters according to an embodiment of the present invention.

S801, S802, S803, S804, S805, S806: steps

Claims (9)

An electronic device for detecting and identifying characters, including: a transceiver for receiving text images; a storage medium for storing multiple modules; and a processor for coupling the storage medium and the transceiver, and accessing and executing all The plurality of modules, wherein the plurality of modules include: a character detection module, which detects the character image to generate a plurality of bounding boxes respectively corresponding to a plurality of characters, according to the plurality of defined The bounding box calculates an oblique angle corresponding to the text image, and performs coordinate transformation on a plurality of coordinate parameters respectively corresponding to the plurality of bounding boxes according to the oblique angle to update the plurality of coordinate parameters; character recognition The module generates a plurality of recognition results respectively corresponding to the plurality of characters according to the plurality of bounding boxes; A plurality of sequences of the plurality of bounding boxes; a matching module, matching the plurality of recognition results and the plurality of sequences to generate a word string; and an output module, outputting the word string through the transceiver, wherein The text sorting module selects the first bounding box corresponding to the minimum ordinate from the plurality of bounding boxes according to the updated coordinate parameters and obtains the first bounding box of the first bounding box. Coordinate parameters, wherein the first coordinate parameter includes a first abscissa, a first ordinate corresponding to the first abscissa, The second abscissa and the second ordinate corresponding to the second abscissa, wherein the first abscissa is smaller than the second abscissa, wherein the word sorting module uses the parameter of the first coordinate The first abscissa is set to zero to enlarge the first bounding box to generate a reference bounding box, and in response to a distance between at least one of the plurality of bounding boxes and the reference bounding box The overlapping area is greater than the threshold and selects the at least certain bounding box, wherein the text sorting module selects the second bounding box corresponding to the minimum abscissa from the at least certain bounding box, and selects the second bounding box for the second bounding box Distributing sequences to generate the plurality of sequences. The electronic device as claimed in claim 1, wherein the plurality of bounding boxes includes a third bounding box, wherein the character detection module is responsive to a fourth bounding box in the plurality of bounding boxes A first tilt angle is calculated according to the third bounding box and the fourth bounding box closest to the third bounding box, and the tilt angle is calculated according to the first tilt angle. The electronic device according to claim 2, wherein the character detection module calculates the inclination angle according to the first center point of the third bounding frame and the second center point of the fourth bounding frame . The electronic device according to claim 1, wherein the plurality of coordinate parameters include a second coordinate parameter corresponding to the second bounding box, wherein the text sorting module responds to the second bounding box assigning the order by removing the second coordinate parameter of the second bounding box from the plurality of coordinate parameters, thereby updating the plurality of coordinate parameters, wherein the text sorting module is based on the removed The second coordinate parameter The plurality of coordinate parameters of the number assigns an order to a third bounding box of the plurality of bounding boxes to generate the plurality of orders. The electronic device according to claim 1, wherein the character detection module generates the plurality of bounding boxes according to a machine learning model. The electronic device as claimed in claim 1, wherein the character recognition module generates the plurality of recognition results according to an optical character recognition model. The electronic device according to claim 1, wherein the tilt angle is greater than or equal to negative forty-five degrees and less than or equal to forty-five degrees. The electronic device as claimed in claim 1, wherein the character image corresponds to a horizontal writing format. A method for detecting and recognizing characters, comprising: receiving a character image; detecting the character image to generate a plurality of bounding boxes respectively corresponding to a plurality of characters, and calculating corresponding to the plurality of bounding boxes according to the plurality of bounding boxes The inclination angle of the text image, and according to the inclination angle, coordinate conversion is performed on a plurality of coordinate parameters respectively corresponding to the plurality of bounding boxes to update the plurality of coordinate parameters; A plurality of recognition results corresponding to the plurality of characters; generating a plurality of sequences respectively corresponding to the plurality of characters according to the tilt angle and the plurality of bounding boxes, including: according to the updated a plurality of coordinate parameters to select a first bounding box corresponding to the minimum ordinate from the plurality of bounding boxes and obtain the first bounding box The first coordinate parameter of the frame, wherein the first coordinate parameter includes a first abscissa, a first ordinate corresponding to the first abscissa, a second abscissa, and a second abscissa corresponding to the second ordinate, wherein said first abscissa is smaller than said second abscissa; said first bounding box is enlarged to generate a reference bounding box by setting said first abscissa of said first coordinate parameter to zero , and selecting at least a certain bounding box of the plurality of bounding boxes in response to an overlapping area between the reference bounding box and the reference bounding box being greater than a threshold; and selecting from the at least certain bounding boxes a second bounding box corresponding to the smallest abscissa, and assigning an order to the second bounding box to generate the plurality of orders; matching the plurality of recognition results and the plurality of orders to generate a word string; and Output the string.

Images