CN114663873A - Text region determination method and device, storage medium and electronic equipment - Google Patents

Abstract

The disclosure relates to the technical field of text detection, in particular to a text content determination method and device, a computer readable storage medium and an electronic device, wherein the method comprises the following steps: acquiring an initial image comprising a text, and determining a rectangular reference region comprising the text in the initial image, wherein one edge of the rectangular reference region is parallel to a reference direction in the initial image; determining a candidate region according to the rectangular reference region; determining text distribution information of each candidate area in the candidate areas; a target text region is determined in the candidate region based on the text distribution information. According to the technical scheme of the embodiment of the disclosure, under the condition of ensuring the precision, the calculation amount is reduced, and the calculation speed is increased.

Description

Text area determination method and device, storage medium and electronic device

technical field

The present disclosure relates to the technical field of information display, and in particular, to a text area determination method and apparatus, a computer-readable storage medium, and an electronic device.

Background technique

With the development of image recognition technology, the technology of determining a text area in an image to obtain the corresponding text is more and more widely used.

In the prior art method for determining a text area in an image to obtain corresponding text, there are problems of slow calculation speed or insufficient precision, that is, the accuracy and calculation speed cannot be taken into account.

It should be noted that the information disclosed in the above Background section is only for enhancement of understanding of the background of the present disclosure, and therefore may contain information that does not form the prior art that is already known to a person of ordinary skill in the art.

SUMMARY OF THE INVENTION

The purpose of the present disclosure is to provide a text area determination method, a text area determination device, a computer-readable medium and an electronic device, so as to reduce the amount of calculation and improve the calculation speed under the condition of ensuring the accuracy.

According to a first aspect of the present disclosure, there is provided a method for determining a text area, comprising: acquiring an initial image including the text, and determining a rectangular reference area including the text in the initial image, wherein the rectangular reference One edge of the area is parallel to the reference direction in the initial image; a candidate area is determined according to the rectangular reference area; text distribution information of each candidate area is determined in the candidate area; based on the text distribution information, The candidate region determines the target text region.

According to a second aspect of the present disclosure, there is provided an apparatus for determining a text area, comprising: a target detection module for acquiring an initial image including the text, and determining a rectangular reference area including the text in the initial image, Wherein, one edge of the rectangular reference area is parallel to the reference direction in the initial image; a first determination module is used to determine a candidate area according to the rectangular reference area; an information extraction module is used to determine in the candidate area text distribution information of each of the candidate regions; and a second determination module, configured to determine a target text region in the candidate region based on the text distribution information.

According to a third aspect of the present disclosure, there is provided a computer-readable medium on which a computer program is stored, and when the computer program is executed by a processor, implements the above-mentioned method.

According to a fourth aspect of the present disclosure, there is provided an electronic device, characterized by comprising: one or more processors; and a memory for storing one or more programs, when the one or more programs are executed by one or more When executed by the processor, one or more processors are caused to implement the above method.

In the method for determining a text area provided by an embodiment of the present disclosure, an initial image including text is acquired, and a rectangular reference area including text is determined in the initial image, wherein the edge of the rectangular reference area is parallel to the reference direction in the initial image Determine the candidate area according to the rectangular reference area; determine the text distribution information of each candidate area in the candidate area; determine the target text area in the candidate area based on the text distribution information, compared with the prior art, on the one hand, determine in the initial image The approximate rectangular reference area is obtained, and then the candidate area is determined in the rectangular reference area based on the geometric relationship. The target detection can start without the need to correct the image, which reduces the amount of calculation. At the same time, the edge of the rectangular reference area is parallel to the initial The reference direction in the image, that is, during detection, the angle of the detection frame can be consistent, which reduces the calculation amount in the detection process, further reduces the calculation amount, and improves the calculation speed. On the other hand, after the candidate region is determined, the target text region is determined based on the text distribution information of the candidate region, which can ensure the accuracy of the determination of the text region. That is, the present disclosure reduces the amount of calculation and improves the speed of determining the text area while ensuring the accuracy of determining the text area.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and for those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort. In the attached image:

Figure 1 shows a schematic diagram of a bank card image without a rotation angle;

Figure 2 shows a schematic diagram of a bank card image with a rotation angle;

3 shows a schematic diagram of an exemplary system architecture to which embodiments of the present disclosure may be applied;

FIG. 4 schematically shows a flowchart of a method for determining a text area in an exemplary embodiment of the present disclosure;

FIG. 5 schematically shows a flow chart of acquiring a rectangular reference area in an exemplary embodiment of the present disclosure;

FIG. 6 schematically shows a structural diagram of a rectangular reference area in an exemplary embodiment of the present disclosure;

FIG. 7 schematically shows a geometrical relationship diagram between an inscribed rectangle and a rectangular reference area in an exemplary embodiment of the present disclosure;

FIG. 8 schematically shows a geometric relationship diagram of another inscribed rectangle and a rectangular reference area in an exemplary embodiment of the present disclosure;

FIG. 9 schematically shows a target image corresponding to a rectangular reference area in an exemplary embodiment of the present disclosure;

FIG. 10 schematically shows a schematic diagram of the position of the inscribed rectangle abcd in the initial image in the exemplary embodiment of the present disclosure;

FIG. 11 schematically shows a target image corresponding to an inscribed rectangle abcd in an exemplary embodiment of the present disclosure;

FIG. 12 schematically shows the position of the inscribed rectangle efgj in the initial image in the exemplary embodiment of the present disclosure;

FIG. 13 schematically shows a target image corresponding to an inscribed rectangle efgj in an exemplary embodiment of the present disclosure;

FIG. 14 schematically shows a location map of key points in an exemplary embodiment of the present disclosure;

FIG. 15 schematically shows a position diagram of a straight line fitted according to key points in an exemplary embodiment of the present disclosure;

FIG. 16 schematically shows a position diagram of a minimum circumscribed rectangle in an exemplary embodiment of the present disclosure;

FIG. 17 schematically shows a flow chart of the preferred embodiment of the method for determining the text region of the present disclosure;

FIG. 18 schematically shows a schematic diagram of the composition of an apparatus for determining a text area in an exemplary embodiment of the present disclosure;

FIG. 19 shows a schematic diagram of an electronic device to which embodiments of the present disclosure may be applied.

Detailed ways

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments, however, can be embodied in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repeated descriptions will be omitted. Some of the block diagrams shown in the figures are functional entities that do not necessarily necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

In related technologies, with the popularity of portable electronic devices such as mobile phones and the large-scale construction of communication infrastructure such as 4G and 5G, mobile payment represented by mobile phone payment is becoming more and more popular, and to a certain extent, "cashless" has been realized. some characteristics of society.

The first step of mobile payment is that the user logs in the payment account on the mobile phone and binds his real-name bank card. One of the important steps is that the user enters his identity information and then enters the bank card number for verification. However, because bank cards generally have more digits, and their fonts are quite different from conventional printed and handwritten characters, there is a high probability of users entering wrongly, which leads to the need for users to repeatedly enter, confirm or even temporarily freeze the account, which is very affecting experience. Therefore, it is of great application value to study the automatic identification scheme of bank card number based on image recognition.

The current mainstream bank card number area detection solutions are divided into two categories: one is based on more traditional computer vision methods such as filters and morphological processing, which locate the card number area through edge detection, and the detection accuracy of this type of method is poor; the other The class method is based on the target detection class CNN (Convolutional Neural Networks, convolutional neural network). The model is trained through big data learning to have the ability to label independently and directly locate the target area. Larger, and generally can only locate horizontal rectangular boxes, so it is less effective for images with rotation. As shown in Figure 1 and Figure 2, when the image has a rotation angle, the image must be corrected to ensure the monitoring accuracy, and the amount of calculation is large. If the image is not corrected, the horizontal detection frame output by the CNN-based target detection scheme There are a large number of irrelevant background regions, which will affect the performance of the recognition module.

FIG. 3 shows a schematic diagram of a system architecture. The system architecture 300 may include a terminal 310 and a server 320 . The terminal 310 may be a terminal device such as a smart phone, a tablet computer, a desktop computer, a notebook computer, etc., and the server 320 generally refers to a background system that provides services related to text area determination in this exemplary embodiment, and may be one server or multiple servers. A cluster of servers. A connection may be formed between the terminal 310 and the server 320 through a wired or wireless communication link to perform data interaction.

In one embodiment, the above-mentioned text area determination method may be performed by the terminal 310 . For example, after the user uses the terminal 310 to shoot an image or the user selects an image in the album of the terminal 310, the terminal 310 determines the text area of the image, and outputs the target text area.

In one embodiment, the above-described text area determination method may be performed by the server 320 . For example, after the user uses the terminal 310 to shoot an image or the user selects an image in the album of the terminal 310 , the terminal 310 uploads the image to the server 320 , and the server 320 determines the text area of the image and returns the target text area to the terminal 310 .

It can be seen from the above that the execution subject of the text area determination method in this exemplary embodiment may be the above-mentioned terminal 310 or server 320, which is not limited in the present disclosure.

The text area determination method in this exemplary embodiment will be described below with reference to FIG. 4 , and FIG. 4 shows an exemplary flow of the text area determination method, which may include:

Step S410, acquiring an initial image including the text, and determining a rectangular reference area including the text in the initial image, wherein one edge of the rectangular reference area is parallel to a reference direction in the initial image ;

Step S420, determining a candidate region according to the geometric relationship of the rectangular reference region;

Step S430, determining the text distribution information of each candidate region in the candidate region;

Step S440, determining a target text region in the candidate region based on the text distribution information.

Based on the above method, on the one hand, the rectangular reference area is determined in the initial image, and then the candidate area is determined in the rectangular reference area based on the geometric relationship. The target detection can be started without the need to correct the image, which reduces the amount of calculation. , the edge of the rectangular reference area is parallel to the reference direction in the initial image, that is, during detection, the angle of the detection frame can be consistent, which reduces the amount of calculation in the detection process and improves the calculation speed. On the other hand, after the candidate region is determined, the target text region is determined based on the text distribution information of the candidate region, which can ensure the accuracy of the determination of the text region. That is, the present disclosure reduces the amount of calculation and improves the speed of determining the text area while ensuring the accuracy of determining the text area.

Each step in FIG. 4 will be described in detail below.

Referring to FIG. 4, in step S410, an initial image including the text is acquired, and a rectangular reference area including the text is determined in the initial image, wherein one edge of the rectangular reference area is parallel to the The reference orientation in the initial image.

In this exemplary embodiment, the initial image includes text, and the initial image may be an image corresponding to a card such as a bank card or an ID card, or a document image, and can also be customized according to user needs, which is not done in this exemplary embodiment. Specific restrictions.

When the above-mentioned initial image is an image corresponding to a bank card, an ID card, etc., the above-mentioned text may include a bank card number, an ID card number, and the like.

In this exemplary embodiment, the above-mentioned initial image may be a rectangle, a triangle, a circle, etc., and the shape of the above-mentioned initial image is not specifically limited in this exemplary embodiment. The initial image can be a 3-channel color image, or a 1-channel grayscale image, and can also be customized according to user requirements, which is not specifically limited in this exemplary implementation.

Wherein, the above-mentioned rectangular reference area is a rectangular area including the above-mentioned text, and the reference direction may be a user-defined direction, which is used to define the deflection angle between the rectangular reference area and the horizontal direction. For example, if the above-mentioned initial image is a rectangle , where the reference direction can be parallel to one of the edges of the above initial image.

In this exemplary embodiment, when acquiring the above-mentioned rectangular reference area, steps S510 to S530 may be included, and steps S510 to S530 will be described in detail below.

In step S510, target detection is performed on the initial image to obtain a plurality of intermediate text regions.

In step S520, the accuracy of each of the intermediate text regions and the confidence that each of the intermediate text regions includes text of a preset type are determined.

Specifically, referring to FIG. 6 , a rectangular coordinate system can be established first, and the initial image can be placed in the rectangular coordinate system. At this time, the reference direction can be a direction parallel to any coordinate axis, which can be detected by a rectangle. At this time, the edge of the above-mentioned rectangular detection frame is parallel to the above-mentioned reference direction, which defines the direction of the rectangular detection frame, and there is no need to use rectangular detection frames in other directions, which can reduce the number of rectangular detection frames. The number of detection frames reduces the amount of calculation in the detection process.

In the following, the above initial image is an image including a bank card, and the preset text type is a bank card number as an example for description. Referring to FIG. 6 , the processor can process the above initial image through multiple cascaded convolution, pooling and other operations. Then output n 6-dimensional vectors. Among them, n is related to the specific network structure, and n can be any positive integer greater than or equal to 1000 and less than or equal to 99999, such as 10000, 20000, etc. It can also be customized according to requirements, which is not specifically limited in this example implementation. . The form of a single prediction vector is (x, y, w, h, c ₁ , c ₂ ), where x, y, w, and h are positive numbers, representing the abscissa of the center point, the ordinate of the center point, and the center point of the middle text area, respectively. The width of the text area, the height of the intermediate text area, and then multiple intermediate text areas are obtained, and each 6 is a vector representing an intermediate text area.

In this exemplary embodiment, c ₁ and c ₂ are positive numbers between 0 and 1, respectively representing the confidence that the coordinates of the center point are accurate and the confidence that the area inside the frame is a bank card number.

In this exemplary embodiment, the product of the above-mentioned c ₁ and c ₂ may be used as the confidence level that the above-mentioned intermediate text region includes text of a preset type.

In step S530, the rectangular reference area is determined among the plurality of intermediate text areas according to the accuracy and the confidence.

In this exemplary implementation, after obtaining n intermediate text regions, the one with the largest product of c ₁ and c ₂ is taken as the above-mentioned rectangular reference region, that is, the intermediate text region with the highest confidence in the above-mentioned intermediate text region including text of the preset type . As shown in FIG. 6 , the prediction result is denoted as (X, Y, W, H, C1, C2), and the rectangular reference area is a rectangle ABCD.

In an example implementation of the present disclosure, a preset threshold may be set first, and the preset threshold may be 0.5, or 0.4, 0.6, etc., and can also be customized according to user requirements. In this example implementation No specific limitation is made.

In this exemplary implementation, taking the above preset threshold value of 0.5 as an example, the upper left corner of the initial image is taken as the coordinate origin, the upper left corner of the image is taken as the coordinate origin, the AB direction is the positive direction of the X axis, and the AD direction is the positive direction of the Y axis For illustration, if C1×C2 is smaller than the preset threshold, it means that the initial image does not contain a card number area, and subsequent operations may not be performed. If C1×C2 is greater than or equal to the above-mentioned preset threshold, the coordinates of the center point P of the above-mentioned rectangular reference area ABCD are determined to be (X, Y), and the coordinates of points A, B, C, and D are respectively (X-W/2, Y-H /2), (X+W/2, Y-H/2), (X+W/2, Y+H/2), (X-W/2, Y+H/2). Wherein W represents the length of the long side of the rectangular reference area, H represents the length of the short side of the rectangular reference area, that is, the length in the present disclosure represents the length of the long side, and the width represents the length of the short side. Using the above method to determine the rectangular reference area can reduce the calculation amount of the target detection algorithm while ensuring the accuracy.

After the above-mentioned rectangular reference area is obtained, step S420 may be executed.

In step S420, a candidate area is determined according to the rectangular reference area.

In this exemplary embodiment, the above-mentioned candidate area is a rectangular area that may include all texts in the rectangular reference area and whose area is less than or equal to the above-mentioned rectangular reference area. Specifically, the candidate area may be determined according to the geometric relationship of the rectangular reference area. The geometric relationship may include the length, width, aspect ratio, inscribed image, and circumscribed image of the rectangular reference area, which are not specifically limited in this exemplary implementation.

In one embodiment, after the rectangular reference area is determined, the inscribed rectangle of the rectangular reference area is determined first, and the inscribed rectangle of the rectangular reference area and the rectangular reference area itself may be used as the candidate area, that is, it may include The three candidate regions specifically include the rectangular reference region itself and the two inscribed rectangles of the rectangular reference region.

In an embodiment, if W and H of the above-mentioned rectangular reference area are not equal (that is, the rectangular reference area is not a square), the rectangular reference area only includes two inscribed rectangles. It should be noted that in this example implementation The four corners of the inscribed rectangle are located on the four sides of the rectangle reference area. Referring to FIG. 7 and FIG. 8 , the inscribed rectangles of the rectangular reference area ABCD may include abcd and efgj, wherein the two inscribed rectangles have the same size and opposite deflection directions. At this time, the coordinates of each point of abcd and efgj can be directly fitted with a computer to obtain the above two inscribed rectangles, and the above inscribed rectangle abcd, inscribed rectangle efgj and the rectangular reference area itself are used as the above candidate areas. Using the above inscribed rectangle as the above candidate area, on the one hand, can accurately locate the position of the text, on the other hand, the number of inscribed rectangles is fixed, which can directly reduce the number of candidate areas, thereby reducing the amount of calculation.

In an exemplary embodiment of the present disclosure, referring to FIG. 7 , the deflection angle ∠α of the inscribed rectangle relative to the rectangular reference area may be determined first, wherein the above deflection angle ∠α and the above ∠BDC have a one-to-one correspondence , for example, when the above ∠BDC=32°, ∠α=22°; when the above ∠BDC=20°, ∠α=15°. The corresponding relationship between ∠BDC and the above-mentioned deflection angle in the above-mentioned different rectangular reference regions can be determined first. After the above-mentioned ∠BDC is obtained, the above-mentioned deflection angle can be determined according to the above-mentioned corresponding relationship, and then each of the above-mentioned inscribed rectangles can be calculated based on the deflection angle. The coordinate value of the point.

In this exemplary embodiment, the above-mentioned correspondence between ∠BDC and the deflection angle may be accurate to 1 degree, or may be accurate to 0.5 degrees, 0.1 degrees, etc., which is not specifically limited in this exemplary embodiment.

In an example implementation, the preset aspect ratio of the text may also be determined first, and then the inscribed rectangle is determined based on the preset aspect ratio and the deflection angle. Specifically, the inscribed rectangle abcd is used as an example to perform Explanation, let the long side of the inscribed rectangle abcd be w, and the short side be h, w in this disclosure represents the length of the long side, h represents the length of the short side, assuming that the deflection angle is ∠α, ∠α is m of ∠BDC times, the following geometric logical relationship can be obtained:

∠α=m∠BDC

hsinα+wcosα=W

At the same time, the preset aspect ratio of the text, the preset aspect ratio of the text can be the preset aspect ratio of the text area, for example, w=kh, where k represents the preset aspect ratio, in this case, According to this, the values of h and w can be solved simultaneously from the system of equations. Then, the coordinates of the above four points abcd are calculated. For example, if the coordinates of point A are (x, y), then the coordinates of point a are (x, y+H-h*cosα), the coordinates of point b are (x+W-h*sinα,y), and the coordinates of point c are (x+W, y+h*cosα), the coordinates of point d are (x+h*sinα, y+H).

然后基于w＝kh既可以计算得到上述h、w的值，进而计算得到上述abcd四点的坐标。例如，假设A点的坐标是(x,y)，那么a点的坐标为

b点的坐标为

c点的坐标为

d点坐标为

In an exemplary embodiment of the present disclosure, if W and H of the above-mentioned rectangular reference area are equal (that is, the rectangular reference area is a square), at this time, the above-mentioned inscribed rectangle may be determined based on the above-mentioned preset aspect ratio and the above-mentioned deflection angle , and then determine the above-mentioned candidate area, when the above-mentioned rectangular reference area is a square, ∠α=∠BDC=45°, then we can get

Then, based on w=kh, the above-mentioned values of h and w can be calculated, and then the coordinates of the above-mentioned four points of abcd can be calculated. For example, if the coordinates of point A are (x, y), then the coordinates of point a are

The coordinates of point b are

The coordinates of point c are

The coordinates of point d are

In yet another exemplary embodiment, when the inscribed rectangle is determined based on the preset aspect ratio and the deflection angle, if the above-mentioned initial image corresponds to a bank card, it can be approximately determined that the above-mentioned ∠α=∠BDC.

At this point, the following geometric logical relationship can be obtained,

sinα=H/(W ² +H ² )1 ^/2

cosα=W/(W ² +H ² )1 ^/2

hsinα+wcosα=W

At the same time, by substituting w=kh into the above geometrical logical relationship, the values of h and w can be solved simultaneously from the equation system. Then, the coordinates of the above four points abcd are calculated. For example, if the coordinates of point A are (x, y), then the coordinates of point a are (x, y+H-h*cosα), the coordinates of point b are (x+W-h*sinα,y), and the coordinates of point c are (x+W, y+h*cosα), the coordinates of point d are (x+h*sinα, y+H).

In this exemplary embodiment, the above-mentioned preset aspect ratio is determined according to the initial image and the text in the initial image. For example, if the above-mentioned initial image is a bank card image, and the preset type text is a bank card number, the above-mentioned preset length The width ratio k may be 14, which is not specifically limited by the above-mentioned preset length-width ratio in this exemplary embodiment.

Based on the calculation process of the inscribed rectangle efgj, reference may be made to the calculation process of the inscribed rectangle abcd, which will not be repeated here.

After the two inscribed rectangles are obtained, the above-mentioned inscribed rectangle and the above-mentioned rectangle reference area itself are used as the above-mentioned candidate area, and then step S430 is executed.

In step S430, the text distribution information of each of the candidate regions is determined in the candidate regions.

The text distribution information is information used to represent the above-mentioned text distribution positions, and may include text distribution coefficients, distances between characters in the text, and the like, which are not specifically limited in this exemplary implementation.

In this exemplary embodiment, referring to FIG. 9 and FIG. 10 , after obtaining the plurality of candidate regions, the target image corresponding to each candidate region can be obtained, that is, the image of the candidate region is extracted from the initial image, and the target image corresponding to each candidate region can be obtained. The edge of its candidate region is also parallel to the above reference direction. Specifically, referring to FIG. 10 , taking the inscribed rectangle adcd as an example, it can be assumed that the four points corresponding to the candidate region in the original image are P ₁ , P ₂ , P ₃ , and P ₄ (where P ₁ is the closest Points in the upper left corner of the image, P ₁ , P ₂ , P ₃ , P ₄ are arranged in clockwise order), and their coordinate set is PS={(x ₁ , y ₁ ), (x ₂ , y ₂ ), (x ₃ , y ₃ ), (x ₄ , y ₅ )}, the width and height of the candidate area are w1, h1 respectively, then solve PS to point set PD={(0,0),(w1,0),(w1 ,h),(0,h1)} affine transformation matrix, and then perform the corresponding affine transformation on the original image. FIG. 9 shows the target image obtained when the rectangular reference area is used as the candidate area. Referring to FIG. 10 and FIG. 11 , the target image determined when the inscribed rectangle abcd is the candidate area, referring to FIGS. 12 and 13 , is the inscribed target image. The target image determined when the rectangle efgj is the candidate region.

At this time, the rotation angle of the text in the above target image can be determined. Specifically, referring to FIG. 14 , FIG. 15 and FIG. 16 , taking the candidate image as the rectangular reference area itself as an example, the processor can determine the above target image. Use the Shi Tomasi algorithm to detect the corner points of the candidate area to obtain multiple key points, and then obtain the minimum circumscribed rectangle corresponding to the text according to the above key points, take the angle between the minimum circumscribed rectangle S and the above candidate area as the above rotation angle, and use the above The rotation angle is denoted as V. In this example implementation, the maximum number of points for corner detection can be set to 100, the quality level is 0.005, and the minimum distance is 2, so that the corners can be concentrated in the area where the text is located, and the specific parameters of corner detection can also be customized according to user needs. Definitions, which are not specifically limited in this exemplary embodiment.

In this example implementation, the above-mentioned text distribution parameters include text distribution coefficients. Referring to FIG. 15 , after obtaining multiple key points, the least squares method can be used to perform straight line fitting on the above-mentioned key points, and the determined straight line is obtained. The slope K, and then the above text distribution coefficient R can be calculated according to the above slope and rotation angle. Specifically, R=VK.

After the above-mentioned text distribution information of the above-mentioned multiple candidate regions is calculated respectively, step S440 may be executed.

In step S440, a target text region is determined in the candidate region based on the text distribution information.

The target text area is determined according to the text distribution information, the rotation angle of the text can be determined by the text distribution information, the candidate area where the text in the target image is closest to the horizontal level is used as the target text area, or the text occupied in the candidate area can be determined by the text distribution information. Area ratio, take the closest occupied area as the target text area

In an exemplary real-time manner of the present disclosure, the candidate region with the smallest text distribution coefficient may be used as the target text region. The text in the candidate region with the smallest text distribution coefficient is the closest to the level, that is, the obtained target text region has the highest accuracy, and the above method can improve the accuracy of determining the target text region.

In another exemplary embodiment of the present disclosure, the processor may perform the following operations for each candidate area: first, divide the target image corresponding to the candidate area into a plurality of sub-areas, and then determine each of the above-mentioned sub-areas according to the above-mentioned text distribution coefficient The text density information of the region is calculated, and then the standard deviation of the text density information of each of the sub-regions in each of the candidate regions is calculated.

After the standard deviations of the plurality of candidate regions are obtained, the candidate region corresponding to the target image with the smallest standard deviation may be used as the target text region. The smaller the standard deviation of the text density of each region in the candidate region, the more uniform the text distribution is. Selecting the candidate region with the most uniform text distribution as the target text region can improve the accuracy of the determined target text region.

In yet another exemplary embodiment of the present disclosure, the processor may also directly use OCR (Optical Character Recognition, optical character recognition) to determine the target text region in the candidate region.

It should be noted that, the manners of determining the target text region in the candidate region may include various methods, the above is an exemplary description, and the present disclosure does not specifically limit it.

Further, referring to FIG. 17 , a specific example implementation is used to illustrate the above-mentioned method for determining the text area. First, step S1710 may be executed to acquire an initial image including the text, and determine in the initial image Include the rectangular reference area of the text; then step S1720 is executed, and the rectangular reference area and the inscribed rectangle are used as the candidate area. Specifically, when step S1720 is executed, step S1721 may be executed first to determine the above Whether the rectangular reference area is a rectangle or a square, if it is a rectangle, step S1722 is executed to determine a deflection angle according to the geometric relationship, and step S1723 is to determine the inscribed rectangle in the rectangular reference area based on the deflection angle. If the above-mentioned rectangular reference area is a square, step S1724 is executed to obtain a preset aspect ratio of the text, and a deflection angle is determined according to the geometric relationship; and step S1725, based on the preset aspect ratio and the deflection The corners define the inscribed rectangle in the rectangle reference area.

After obtaining the above inscribed rectangle, the text distribution coefficient in the text distribution information can be determined. Step S1730 can be executed to obtain the target image corresponding to each candidate region; multiple key points; then step S1750 is performed to determine the rotation angle of the text in the target image; and step S1760 is to perform straight line fitting based on the key points, and determine the slope of the straight line; the last step is to perform step S1770, The text distribution coefficient is determined based on the rotation angle and the slope.

After the above text distribution coefficient is obtained, step S1780 may be executed to determine the candidate area corresponding to the target image with the smallest absolute value of the text distribution coefficient as the target text area. to complete the determination of the target text area.

To sum up, compared with the prior art, on the one hand, an approximate rectangular reference area is determined in the initial image, and then a candidate area is determined in the rectangular reference area based on the geometric relationship, and you can start without correcting the image. Performing target detection reduces the amount of calculation. At the same time, the edge of the rectangular reference area is parallel to the reference direction in the initial image, that is, during detection, the angle of the detection frame can be consistent, which reduces the amount of calculation in the detection process. The calculation amount is further reduced and the calculation speed is improved. On the other hand, the inscribed rectangle of the rectangular reference area and itself are used as the candidate area, and the coordinate values of each vertex of the candidate area are calculated through the relevant geometric relationship. Under the condition of ensuring the accuracy, there is no need for the computer to do more complicated operations. , which further reduces the amount of computation. On the other hand, after the candidate area is determined, the target text area is determined based on the text distribution information of the candidate area. Specifically, the text distribution coefficient in the text distribution information is obtained through the rotation angle of the text, the key point you and the slope of the straight line, and the text is used. The distribution coefficient is used to determine the target text area, which can ensure the accuracy of the determination of the text area. That is, the present disclosure reduces the amount of calculation and improves the speed of determining the text area while ensuring the accuracy of determining the text area.

It should be noted that the above-mentioned drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiment of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the chronological order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, in multiple modules.

Further, referring to FIG. 18 , the embodiment of this example further provides a text area determination apparatus 1800 , including a target detection module 1810 , a first determination module 1820 , an information extraction module 1830 and an image generation module 1840 . in:

The object detection module 1810 can be used to obtain an initial image including text, and determine a rectangular reference area including text in the initial image, wherein one edge of the rectangular reference area is parallel to the reference direction in the initial image, specifically, using The rectangular detection frame performs target detection on the initial image to obtain multiple intermediate text areas; determines the accuracy of each intermediate text area and the confidence that each intermediate text area includes text of a preset type; The rectangular reference area is determined in the area.

The first determination module 1820 may be configured to determine a candidate area according to the rectangular reference area, specifically, may first determine the inscribed rectangle of the rectangular reference area; take the rectangular reference area as the candidate area, and use the inscribed rectangle as the candidate area. as the candidate area.

In an example implementation, when determining the inscribed rectangle of the rectangular reference area, the first determination module 1820 may first determine the deflection angle of the inscribed rectangle relative to the rectangular reference area according to the geometric relationship, and then determine the deflection angle of the inscribed rectangle in the rectangular reference area based on the deflection angle. Determine the inscribed rectangle in .

In this example embodiment, when determining the inscribed rectangle in the rectangle reference area based on the deflection angle, the first determination module 1820 may first obtain the preset aspect ratio of the text, and then determine the rectangle based on the preset aspect ratio and the deflection angle. The reference area determines the inscribed rectangle.

The information extraction module 1830 may be used to determine text distribution information of each candidate region in the candidate regions. Specifically, the text distribution information includes text distribution coefficients. When determining the text distribution information of each candidate region in the candidate region, the information extraction module 1830 can first obtain the target image corresponding to each candidate region; then determine the text in the target image. relative to the rotation angle of the candidate region; finally, the text distribution coefficient is determined according to the rotation angle.

In this exemplary embodiment, after determining the text distribution coefficient according to the rotation angle, the information extraction module 1830 may first perform corner detection on each target image to obtain multiple key points; then, perform straight line fitting based on the key points, and determine the Slope; finally, the text distribution coefficient is determined based on the rotation angle and the slope.

The image generation module 1840 may be used to determine the target text region in the candidate region based on the text distribution information.

In an example implementation, the image generation module 1840 is configured to determine the candidate region corresponding to the target image with the smallest absolute value of the text distribution coefficient as the target text region.

In another exemplary embodiment, the image generation module 1840 may be configured to first obtain the target image corresponding to each candidate area; then divide each target image into multiple sub-areas; secondly, determine the text in each sub-area according to the text distribution information density information; then, calculate the standard deviation of the text density information of each sub-region in each candidate region; finally, determine the candidate region corresponding to the target image with the smallest standard deviation as the target text region.

The specific details of each module in the above-mentioned apparatus have been described in detail in the method part of the implementation manner, and the undisclosed details can refer to the method part of the implementation manner, and thus will not be repeated.

Exemplary embodiments of the present disclosure also provide an electronic device for executing the above-described text area determination method, and the electronic device may be the above-described terminal 310 or server 320 . Generally, the electronic device may include a processor and a memory, the memory is used for storing executable instructions of the processor, and the processor is configured to execute the above-mentioned image text area determination method by executing the executable instructions.

The following takes the mobile terminal 1900 in FIG. 19 as an example to illustrate the structure of the electronic device. It will be understood by those skilled in the art that the configuration in Figure 19 can also be applied to stationary type devices, in addition to components specifically for mobile purposes.

As shown in FIG. 19 , the mobile terminal 1900 may specifically include: a processor 1901 , a memory 1902 , a bus 1903 , a mobile communication module 1904 , an antenna 1 , a wireless communication module 1905 , an antenna 2 , a display screen 1906 , a camera module 1907 , and an audio module 1908 , power module 1909 and sensor module 1910.

The processor 1901 may include one or more processing units, for example, the processor 210 may include an AP (Application Processor, application processor), a modem processor, a GPU (Graphics Processing Unit, graphics processor), an ISP (Image Signal Processor) , image signal processor), controller, encoder, decoder, DSP (Digital Signal Processor, digital signal processor), baseband processor and/or NPU (Neural-Network Processing Unit, neural network processor), etc. The text region determination method in this exemplary embodiment may be performed by an AP, a GPU or a DSP, and when the method involves processing related to a neural network, it may be performed by an NPU.

The processor 1901 may form a connection with the memory 1902 or other components through the bus 1903 .

Memory 1902 may be used to store computer-executable program code, which includes instructions. The processor 1901 executes various functional applications and data processing of the mobile terminal 1900 by executing the instructions stored in the memory 1902 . The memory 1902 may also store application data, such as storing images, videos, and other files.

The communication function of the mobile terminal 1900 can be implemented by the mobile communication module 1904, the antenna 1, the wireless communication module 1905, the antenna 2, the modem processor, the baseband processor, and the like. Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. The mobile communication module 204 can provide 2G, 3G, 4G, 5G and other mobile communication solutions applied on the mobile terminal 1900 . The wireless communication module 1905 can provide wireless communication solutions such as wireless local area network, Bluetooth, near field communication, etc. applied to the mobile terminal 1900 .

The sensor module 1910 may include a depth sensor 19101, a pressure sensor 19102, a gyroscope sensor 19103, an air pressure sensor 19104, etc., to implement corresponding sensing detection functions.

As will be appreciated by one skilled in the art, various aspects of the present disclosure may be implemented as a system, method or program product. Therefore, various aspects of the present disclosure can be embodied in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, which may be collectively referred to herein as implementations "circuit", "module" or "system".

Exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which a program product capable of implementing the above-described method of the present specification is stored. In some possible implementations, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code, when the program product runs on a terminal device, the program code is used to cause the terminal device to execute the above-mentioned procedures in this specification. Steps according to various exemplary embodiments of the present disclosure are described in the "Example Methods" section.

It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Furthermore, program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., as well as conventional procedural Programming Language - such as the "C" language or similar programming language. The program code may execute entirely on the user computing device, partly on the user device, as a stand-alone software package, partly on the user computing device and partly on a remote computing device, or entirely on the remote computing device or server execute on. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (eg, using an Internet service provider business via an Internet connection).

Other embodiments of the present disclosure will readily suggest themselves to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or techniques in the technical field not disclosed by the present disclosure . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the claims.

It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (12)

1. a text area determination method, is characterized in that, comprises: acquiring an initial image including text, and determining a rectangular reference area including the text in the initial image, wherein one edge of the rectangular reference area is parallel to a reference direction in the initial image; determining a candidate region according to the rectangular reference region; determining text distribution information of each of the candidate regions in the candidate regions; The target text area is determined based on the text distribution information. 2. The method according to claim 1, wherein the determining a rectangular reference area including the text in the initial image comprises: performing target detection on the initial image to obtain a plurality of intermediate text regions; determining the accuracy of each of the intermediate text regions and the confidence that each of the intermediate text regions includes text of a preset type; The rectangular reference area is determined among the plurality of intermediate text areas according to the accuracy and the confidence level. 3. The method according to claim 1, wherein the determining a candidate region according to the rectangular reference region comprises: determining the inscribed rectangle of the rectangular reference area; The rectangular reference area is used as the candidate area, and the inscribed rectangle is used as the candidate area. 4. The method according to claim 3, the determining the inscribed rectangle of the rectangular reference area, comprising: determining the deflection angle of the inscribed rectangle relative to the rectangular reference area according to the geometric relationship; The inscribed rectangle is determined in the rectangular reference area based on the deflection angle. 5. The method of claim 4, the determining the inscribed rectangle in the rectangular reference area based on the deflection angle comprising: Get the preset aspect ratio of the text; The inscribed rectangle is determined in the rectangle reference area based on the preset aspect ratio and the deflection angle. 6 . The method according to claim 1 , wherein the text distribution information comprises a text distribution coefficient, and the determining the text distribution information of each of the candidate regions in the candidate regions comprises: 6 . acquiring the target image corresponding to each of the candidate regions; determining the rotation angle of the text in the target image relative to the candidate region; The text distribution coefficient is determined according to the rotation angle. 7. The method according to claim 6, wherein the determining the text distribution coefficient according to the rotation angle comprises: Perform corner detection on each of the target images to obtain a plurality of key points; Fitting a straight line based on the key points, and determining the slope of the straight line; The text distribution coefficient is determined based on the rotation angle and the slope. 8. The method according to claim 6, wherein the determining the target text region in the candidate region based on the text distribution information comprises: The candidate area corresponding to the target image with the smallest absolute value of the text distribution coefficient is determined as the target text area. 9. The method according to claim 1, wherein the determining the target text region in the candidate region based on the text distribution information comprises: acquiring the target image corresponding to each of the candidate regions; dividing each described target image into a plurality of sub-regions; Determine text density information in each of the sub-regions according to the text distribution information; calculating the standard deviation of the text density information of each of the sub-regions in each of the candidate regions; The candidate region corresponding to the target image with the smallest standard deviation is determined as the target text region. 10. A device for determining a text area, comprising: an object detection module for acquiring an initial image including the text, and determining a rectangular reference area including the text in the initial image, wherein one edge of the rectangular reference area is parallel to the reference in the initial image direction; a first determination module, configured to determine a candidate region according to the geometric relationship of the rectangular reference region; an information extraction module; used for determining the text distribution information of each of the candidate regions in the candidate regions; The second determining module is configured to determine a target text region in the candidate region based on the text distribution information. 11 . A computer-readable storage medium on which a computer program is stored, characterized in that, when the program is executed by a processor, the method for determining a text area according to any one of claims 1 to 9 is implemented. 12. An electronic device, characterized in that, comprising: one or more processors; and a memory for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement any one of claims 1 to 9 The text area determination method described in item.

Images