JP6377214B2 - Text detection method and apparatus - Google Patents

Description

  The present invention relates primarily, but non-exclusively, to computer vision, image processing and image understanding, and more particularly to a text detection method and apparatus.

  The text in the natural scenery image and the moving image frame has important information for understanding and acquiring the visual content. Detecting text in images, especially in natural images and video frames, is a computer aid for visually impaired or foreigners and many computers such as automatic image and video acquisition and robot navigation in urban environments It is decisive for vision applications.

  Nevertheless, text detection in natural landscapes is a difficult subject. The main challenge lies in various texts with different fonts, sizes, skew angles and distortions. Environmental factors such as non-uniform lighting and reflections, bad lighting conditions, and complex backgrounds add further complexity.

  In related literature, a text detection method for detecting a text region in a natural landscape includes a step of generating individualized components from an image and a step of generating component groups by grouping components based on a certain rule. And then removing the non-text group by verifying the component group and recovering the text region (eg, text line, word) using the remaining text group.

  There are two main reasons why component grouping usually requires further verification. First, there is a noise component group composed of non-text components that happen to have a perceptual configuration similar to a text group. For example, non-text components that are spatially close and similar in appearance can also be grouped and stored together. Second, since multiple lines or multi-directional text is commonly seen in natural scenery, how to properly group text components is very important for text area detection performance. During the component grouping step, multiple assumptions are stored corresponding to less evidence to determine the layout pattern. Based on the above analysis, some text detection methods in public documents further include a group verification step. There, component groups are parsed and classified as text groups and non-text groups, and text regions (eg, text lines, words) are restored using only text groups, while non-text groups are removed. Let's go.

  For example, both the Chinese patent application 103077389 and the patent application 10418274429 disclosed how to validate component groups based on group level features and classifiers. Group level features typically describe a group from two perspectives: regularity and character likelihood. The former includes dispersion of components belonging to the group in terms of size, color, gap, and stroke width, and spatial arrangement of the components in the group. The character likelihood of a component in a group is usually measured by a character classifier, and then its value is integrated within the group. These group level features can be used as input feature vectors for text classifiers or as cascading rules. A text confidence value of the group is calculated based on the feature, and a group having a high text reliability is stored. However, performance depends on the features and training samples used in the classifier. It is difficult to reject non-text groups with high regularity, and it is also difficult to accept text groups that are different from the training sample.

  As a further example, both U.S. Pat. Nos. 8,320,674 and 6,563,949 disclosed how to validate component groups based on recognition results. Component groups are recognized by the OCR engine, and groups with low recognition confidence are rejected. For complex layout cases such as multi-line or multi-directional text, groups that satisfy the language model are saved. However, performance is highly related to the recognition engine and language model. Also, it takes time to recognize all the components in the group, especially when the amount of components is large.

  In fact, when considering the recognition result as one special feature of the group, the two types of prior art can be integrated. One common disadvantage is that each group is evaluated separately. That is, the global information of the image is ignored.

References The following references are referred to in the detailed description below.

L. Neumann and J. Matas, "On combining multiple segmentations in scene text recognition", International Conference on Document Analysis and Recognition (ICDAR), pp 523-527, 2013. Xu-cheng Yin, Xuwang Y., Kaizhu H., Hongwei Hao, "Robust text detection in natural scene images", IEEE Trans. On Pattern Analysis and Machine Intellignece, Vol. 36, No. 5, 2014. Boris Epshtein, Eyal Ofek, Yonatan Wexler, "Detecting text in natural scenes with stroke width transform", Computer Vision and Pattern Recognition, IEEE Computer Society Conference on, pp. 2963-2970, 2010. J. Matas, O. Chum, M. Urban and T. Pajdla, "Robust wide baseline stereo from maximally stable extremal regions", Proc. Of British Machine Vision Conference, pp. 384-396, 2002. Chang C C, Lin.CJ.LIBSVM: A library for support vector machines [J] .ACM Transactions on Intelligent Systems and Technology (TIST, 2003, 2 (3): 389-396.

Terminology The following terms appear frequently in this document and are defined in the detailed description below.
A component is a collection of pixels in an image that is spatially connected with similar colors, stroke widths, and gray scales.
A text component refers to the basic element of a character.
A component group refers to a collection of components that have a similar appearance and are arranged in a straight line.
A component connection refers to a component set that includes at least two adjacent components in a component group.
A text group refers to a component group composed of text components.
The text area refers to the bounding box or quadrilateral of the text group and is the output of text detection.
A global prime feature refers to a common feature shared by most component connections of a text group in an image. Usually it is the selected context information, for example in a direction of about 90 degrees.

  Accordingly, a novel text detection method and apparatus for improving text detection performance in images, particularly natural landscape images, is proposed in this disclosure. According to an aspect of the present invention, a component generation step of generating a component from an input image, a component grouping step of grouping components satisfying the similarity requirement to form a component group, and at least two neighbors in one component group Component connection extraction step for extracting component connections including components to be performed, a feature acquisition step for acquiring features of all component connections, and a component group as a text group and a non-text group based on the features acquired in the feature acquisition step A method is provided that includes a component group classification step for classifying and a text region generation step for generating a text region based on the text group. It is.

  The main novelty of the present invention resides in component group classification. The global prime features of one image text are extracted and used in component group classification. Global information used alone or combined with group level features is expected to improve the accuracy of text detection. The global prime feature is automatically selected from several features and can therefore be adapted to different scenes.

  The present invention is used to find a text region in a natural landscape image. It takes the original image file as input and generates a set of many rectangles (bounding boxes for text groups) as output. Compared to the prior art, the present invention can improve both accuracy and reproducibility at the same time cost.

It is a block diagram which shows the hardware constitutions of the computer system which performs embodiment of this invention. It is a block diagram which shows the structure of a text detection apparatus. It is a flowchart which shows the text detection method performed by a text detection apparatus. It is a flowchart which shows the method of classifying the component group which concerns on embodiment of this invention. FIG. 5A to FIG. 5C show examples of generation of candidate text components according to the embodiment of the present invention. FIG. 5A to FIG. 5C show examples of generation of candidate text components according to the embodiment of the present invention. FIG. 5A to FIG. 5C show examples of generation of candidate text components according to the embodiment of the present invention. The example of description of the component grouping result which concerns on embodiment of this invention is shown. 5 is a flowchart illustrating a method for acquiring a global prime feature according to an embodiment of the present invention. 6 is a flowchart illustrating a method for obtaining a global distribution of component connection directions according to an embodiment of the present invention. An example of the global distribution of component connection directions is shown. The example of acquisition of a global prime characteristic is shown. It is a flowchart which shows the method of selecting a feature in global prime feature extraction. 6 is a flowchart illustrating a method for acquiring a global prime feature based on a predetermined feature. FIG. 13A and FIG. 13B show an example of how a text area is generated according to the embodiment of the present invention. FIG. 13A and FIG. 13B show an example of how a text area is generated according to the embodiment of the present invention. It is a flowchart which shows the text information extraction method which concerns on embodiment of this invention. It is a block diagram which shows the text information extraction system which concerns on embodiment of this invention.

  Referring to the drawings listed above, this section describes specific embodiments and their detailed construction and operation. It should be noted that the embodiments described below are for illustrative purposes only and are not intended to be limiting. Therefore, the embodiment does not limit the scope of the present invention, and can be changed to various forms within the scope of the present invention. Those skilled in the art will recognize that, in light of the teachings herein, there are many equivalents to the exemplary embodiments described herein.

  FIG. 1 is a block diagram showing a hardware configuration of a computer system that executes an embodiment of the present invention.

  As shown in FIG. 1, the system includes at least a computer 100 including a CPU 101, a RAM 102, a ROM 103, a system bus 104, an input device 105, an output device 106, and a drive 107. For example, the computer 100 may be an image recognition device. Note that the computer 100 may include one or more computers, each of which may individually implement the corresponding functionality of the computer 100.

  The CPU 101 executes overall processing that is difficult to make a program held in the RAM 102 or the ROM 103. The RAM 102 is used as a temporary holding area when the CPU 101 executes various processes such as the embodiment of the present invention.

  The input device 105 allows a user to issue various instructions to the computer 100, and includes an imaging device (for example, a scanner and a digital camera), a user input interface, and a network interface.

  The output device 106 enables the user to output the text detection and the like of the present invention, and includes an output peripheral interface, a display device (for example, a monitor, a CRT, a liquid crystal display, or a graphic controller), and a printer.

  The driver 107 is used to drive a holding medium such as a hard disk, a memory card, or an optical disk (for example, a CD-ROM or a DVD ROM). For example, a program for performing image data or text detection processing is held in a holding medium and is driven by the driver 107.

  The system bus 104 connects the CPU 101, RAM 102, ROM 103, input device 105, output device 106, and driver 107. Data is communicated on the system bus 104. As used herein, the term “connected” means logically or physically connected indirectly or directly through one or more intermediaries.

  In general, the text detection inputs of the present invention are all types of images. For example, an image can be obtained by an imaging device such as a digital camera, a digital video camera, a sensor, or a scanning device (for example, a scanner or a multifunction device).

  The system shown in FIG. 1 is merely an example and is not intended to limit the present invention including its applications and uses. For example, when a program for performing text detection processing is activated, the CPU 101 performs the present invention such as the steps described in FIGS. 3 to 4, 7 to 8, 11 to 12, and 14. By performing all the disclosed steps, an input image is obtained from the input device 105, the component is extracted, the component is verified, and a text region is generated. Thereafter, the CPU 101 sends the result to the output device 106 through the system bus 104. The result may be stored in the RAM 102. The result may be sent to the remote computer via the network interface for use by other applications.

  Furthermore, it is possible to execute parts, devices, components and / or assemblies of the apparatus of the present invention, such as the apparatus shown in FIGS. 2 and 15, which are software, hardware, firmware or their Configured to perform text detection through any combination.

  FIG. 2 is a block diagram showing the configuration of the text detection apparatus. FIG. 3 is a flowchart showing a text detection method performed by the text detection apparatus shown in FIG. The CPU 101 executes the method of the present invention using a program and image data held in the RAM 102 or the ROM 103.

  As shown in FIG. 2, in this specification, the text detection device 200 includes an image input unit 201, a component generation unit 202, a component grouping unit 203, a component group classification unit 204, and a text region generation unit 205. And a text area output unit 206.

  The image input unit 201 is configured to acquire a natural scenery image acquired by the imaging device 207 or to acquire a natural scenery image held in a holding device (for example, a hard disk) for the text detection device 200. . The acquired image is seen as an input image, for example shown in FIG. 5A.

  The component generator 202 is configured to generate a set of candidate text components from the input image, which is described in S301 of FIG.

  In step S301, the component generation unit 202 generates a set of candidate text components. A component is usually a collection of pixels with similar colors (Non-Patent Document 1) or similar gray scales (Non-Patent Document 2) or similar stroke widths (Non-Patent Document 3) or spatially connected. is there. In the generation of components, several methods such as color clustering, adaptive binarization, and morphological processing may be used. According to an exemplary embodiment of the present invention, the component is generated from a grayscale image based on a Maximum Stable Extremal Region (MSER). As shown in FIG. 5B, the component is labeled as a dark gray rectangle. For example, components 501 and 502 are labeled as dark gray rectangles.

  For better results, apply component filtering after component generation to remove apparently non-text components. Commonly used features in component filtering are extracted from component size, component aspect ratio, component density, which is the component pixel occupancy within its bounding box, statistical characteristics of component stroke width, and component region Texture features (eg, wavelet, Gabor, LBP). These features may be used as a hierarchical filter or as an input to a learning classifier. According to an exemplary embodiment of the present invention, a support vector machine (hereinafter “SVM”) classifier (Non-Patent Document 5) is used to distinguish between text components and non-text components. As shown in FIG. 5C, some non-text components are removed after component filtering. For example, components 501 and 502 are removed because they are non-text components.

  The component grouping unit 203 is configured to group components, which will be described in step S302 of FIG.

  In step S302, the component grouping unit 203 constructs a component group by connecting candidate text components that satisfy the similarity requirement together. Feature values that describe the similarity between two components include distance features, difference features, and ratio features, such as spatial distance, gray scale difference, color difference, boundary contrast difference, bounding box height ratio. , Width ratio, stroke width ratio. The distance feature value is calculated by the normalized Euclidean distance between the centers of the components. All values of the difference feature are calculated by dividing the absolute value of the difference by the average value. All values of the ratio feature are calculated by dividing the maximum value of the specified property by the minimum value of the specified property. If the components do not belong to any group, they are considered noise components and are removed.

  FIG. 6 shows an explanatory example of the component grouping result according to the embodiment of the present invention. The text components shown in FIG. 6 are connected by lines and can be viewed as building a text component group. However, in addition to the expected true text group, there are groups that traverse the true text group, such as groups composed of characters in different text lines shown in FIG. There are still some groups composed of non-text components. For example, the non-text components of the window shown in FIG. 6 are connected by lines to build a non-text component group. In another example, the non-text components of the window and traffic light shown in FIG. 6 are connected by a line to build another non-text component group. Therefore, component group classification is necessary.

  The component group classification unit 204 is configured to classify component groups based on features acquired from a distribution of values of at least one candidate feature of all component connections. According to an exemplary embodiment of the present invention, the feature obtained from the distribution of values of at least one candidate feature of all component connections may be a global prime feature described in step S303 of FIG. . As used herein, a global prime feature refers to a common feature shared by most component connections of a text group. Usually it is the selected context information, for example in a direction of about 90 degrees.

  In step S303, the component group classification unit 204 first acquires a global prime feature, and then classifies the component group into a text group and a non-text group using the global prime feature as described in FIG.

  Reference is made to FIG. 4 which is a flowchart showing the method for classifying component groups according to the embodiment of the present invention and executed in step S303.

  In step S401, the component group classification unit 204 extracts a component connection from the component group. In this step, a component group is viewed as a collection of component connections, so that according to an exemplary embodiment of the invention, two adjacent components in a component group are extracted as a single component connection. In this specification, the component connection refers to a component set including at least two adjacent components in one component group.

  In step S402, the component group classification unit 204 calculates the text reliability of each component connection. The text confidence value for each component connection includes color similarity (eg, grayscale difference, color difference), size similarity, direction, spatial distance, boundary contrast difference, bounding box height ratio, width ratio, and stroke width. Calculated by a set of features extracted from component connections, such as ratios.

  According to an exemplary embodiment of the invention, text confidence values for component connections may be obtained by a pre-trained classifier for text and non-text components. A pretrained classifier is trained based on positive and negative samples. A positive sample is a component connection consisting of two adjacent components in one component group. A negative sample is a component connection consisting of one component in a group and one component outside that group (eg, in another group or a noise component). A binary classifier (eg, SVM) is used to classify the component connections, and then the output score is further converted to a text confidence value.

  In step S403, the component group classification unit 204 acquires global prime features of all component connections. Based on the component connections and their text confidence values, global prime features are obtained according to features automatically selected from a set of features as shown in FIG. 7 or predetermined features as shown in FIG.

  Referring to FIG. 7, shown therein is a flowchart illustrating a method for obtaining a global prime feature performed in step S403.

  In step S701, after acquiring the component connection, its text confidence value, and the set of candidate features for component connection, the component group classification unit 204 acquires the distribution of the value of the feature i in the set of candidate features. In the present embodiment, the distribution of the value of feature i is referred to as a global distribution. Candidate features i include the direction of component connection, the average foreground color of the component, the average background color of the component, the average boundary contrast, or the distance between two components in the component connection. In the present specification, for example, in FIG. 8, it is described that a global distribution of directions is acquired by setting the direction of component connection as a feature i.

  Referring to FIG. 8, shown therein is a flowchart illustrating a method for obtaining a global distribution of component connection directions performed in step S701.

  In step S801, the component group classification unit 204 acquires the direction of component connection by extracting the direction feature from the component connection.

  In step S802, the component group classification unit 204 acquires a text reliability weighted histogram of direction features based on all component connections in one image. According to an exemplary embodiment of the present invention, in each direction, the histogram values (y-axis) are weighted by their respective text confidence values calculated in step S402, each value of component connection feature i. Represents the frequency of each value of the feature i. The histogram value in each direction is shown in FIG. 9, for example, and the range of the direction is [0, 180] degrees.

  In step S803, the component group classification unit 204 finds the highest histogram bin using a sliding window as shown in FIG. The distribution is further quantified into “N” bins with a width “D”. According to an exemplary embodiment of the present invention, a sliding window of width “D” is used to minimize the impact of quantification errors on global prime characteristics. During the slide, the histogram values (y-axis) in the windows are summed and recorded, and the window with the highest value is the top histogram bin as shown in FIG. 9, and the other bins are determined accordingly. The

  In step S804, the component group classification unit 204 quantifies the histogram based on the highest-order histogram bin. The histogram is normalized so that the sum of the histogram values (y-axis) of all bins in the distribution is 1. All values of component connection feature i within a group of images contribute to a global distribution.

In step S702, the component group classification unit 204 selects a global prime feature based on the global distribution. As shown in FIG. 10, for example, in the present specification, it referred to the global distribution _{H = {h i, i =} 0,1, ... N} and. here,
“N” is the number of bins in the distribution.
h _top is the highest histogram bin of the distribution.
h _sec is the second histogram bin of the distribution.
f _top is the center value of the most significant bin on the x-axis.
“CL” is an abbreviation for component character likelihood. A single component CL is obtained by a character classifier such as an SVM classifier. On the other hand, “CL” of the bin is defined as an average score of all the components included in the bin component connection.

A feature derived from the component connection distribution is selected if the concentration of the component connection distribution is greater than a predetermined threshold. According to an exemplary embodiment of the present invention, a global prime feature is selected when the global distribution of component connections satisfies one of the following conditions:
1. The highest histogram bin h _top of the distribution is greater than a predetermined first threshold.
2. The average character likelihood CL (h _top ) of the component connection in the highest histogram bin of the distribution is
Bigger than.
3. A ratio h _top / h _sec of values of the highest histogram bin of the distribution and the second histogram bin of the distribution is larger than a predetermined second threshold value.

FIG. 10 shows an example of acquiring a global prime feature according to an embodiment of the present invention. If the distribution meets one of the above requirements, f _top is a global prime feature.

  In step S703, the component group classification unit 204 determines whether all the features have been processed. If other features remain, the process returns to step S701 to obtain a global distribution of other features in the candidate feature set. Otherwise, the process may proceed to step S704.

  In step S704, the component group classification unit 204 selects the feature with the most important global prime feature. The global prime features of the different features are compared and the feature with the most important global prime feature is selected.

  According to an exemplary embodiment of the present invention, FIG. 11 is a flowchart illustrating a method for selecting features in global prime feature extraction and performed in step S704.

  It should be noted that in step S1101, the global prime feature is associated with a feature because the global distribution of different features can change. According to an exemplary embodiment of the present invention, the feature used to obtain the global prime feature may be at least one of the following features: Direction, component color (eg, average foreground color, average background color), distance between two components in a component connection, average boundary contrast.

In step S1102, the component group classification unit 204 selects a global prime feature based on the global distribution of different features. In this specification, if there is a global prime feature as shown in step S1103, the method for selecting the feature is based on the value of h _top / h _sec . The larger the ratio, the more important the global prime characteristics. Therefore, the feature with the largest ratio h _top / h _sec will be selected as the global prime feature. That is, the component group classification unit 204 selects a global distribution with the maximum value of h _top / h _sec as shown in step S1104.

  After processing the candidate features one by one, the global prime features according to the different features are compared and the feature with the most important global prime feature is selected as shown in step S704. Therefore, the output of step S403 is a global prime feature according to the selected feature.

  FIG. 12 is a flowchart illustrating another method for obtaining a global prime feature based on a predetermined feature.

  Referring to FIG. 12, for example, after obtaining a predetermined feature in license plate recognition, the text group is basically along the horizontal direction, so the direction of component connection is used as the predetermined feature. In another example, in road sign recognition, the text and surrounding colors are often uniform and therefore are used as predetermined features. When the predetermined feature is the direction of component connection, the component group classification unit 204 uses the same method as described in step S701 and step S702, as shown in step S1201 and step S1202, to perform the feature globalization. Obtain a distribution and select global prime features based on the global distribution.

  Therefore, the output of step S403 is a global prime feature according to a predetermined feature.

  Examples of global prime features for a particular image are as follows: Most component connections within a text group are along a similar direction, which belongs to [-15, 15] degrees. Or, most component connections in a text group are of similar color / grayscale, with black or grayscale values belonging to [0, 30].

  In step S404, the component group classification unit 204 adjusts the text reliability of component connection. The text confidence value of the component connection is adjusted based on the global prime feature extracted in step S403. Component connections that follow the global prime feature are increased, while component connections that do not follow the global prime feature are reduced. Adjustment processing is performed by adjusting the text confidence value of the component connection.

  Since the global distribution obtained in step S701 or step S1201 has already been normalized, it is treated as a probability distribution of a predetermined feature, and the text confidence value of the component connection is adjusted based on the probabilistic reasoning.

Given that the probability that a component connection follows the global prime feature is h _top , the text confidence value of the component connection should be adjusted according to its deviation from the global prime feature. The normalized deviation is
It is expressed. here,
f _cur is a feature value of the current component connection.
“D” is the bin width of the feature distribution.

For example, if the global prime feature is about direction and f _top is the center of the top bin on the x axis and is 90 degrees, and f _cur is the current component connection direction and 10 degrees, The deviation between the two is 80 degrees.

If the probability that a component connection does not follow the global prime feature is 1-h _top , the text confidence value of the component connection should not be changed.

To summarize the above two cases, the text confidence value of the component connection is adjusted as follows.
here,
TC _org is the original text confidence value of the current component connection and is provided by the component connection classifier in step S402.
TC _adj is the adjusted text confidence value of the current component connection.
“W” is an adjustment factor that integrates the above two cases.
β is an adjustment parameter.
c is a compensation parameter.

  It can be seen that in component group classification, global prime features of component connections in an image are extracted and combined with local information (group level features).

In step S405, the component group classification unit 204 classifies the component group into a text group or a non-text group. Component groups are classified into text groups and non-text groups based on the group's text confidence value. Non-text groups are removed, while text groups are left to restore text areas. On the other hand, the text confidence value TC _g of component group is the average text confidence value component connections therein. TC _adj ⁱ is the adjusted text confidence value of the i th component connection in the group and is obtained in step S404.
Here, M is the amount of component connections in the group.

On the other hand, by extracting group-level features such as dispersion in terms of size, color, and stroke width of the components in the group and the spatial arrangement of the components in the group, measure another text confidence value for the group, This is represented as TC _f . Thus, the final text confidence value for the group is defined as the weighted sum of those two.
TC = ωTC _g + (1−ω) TC _f
Where 0 ≦ ω ≦ 1

  When ω = 0, only group level features are used in group classification. This is the same as the prior art that discloses how to verify component groups based on group level features and classifiers.

  When ω = 1, only global prime feature information is used, which is the first example described above. If 0 <ω <1, the global prime feature is integrated with the group level feature in the group classification, which is another example described above.

  A component group with a text reliability higher than a predetermined value remains determined as a text group, otherwise it is removed.

  13A and 13B show an example of generating a text area according to the embodiment of the present invention.

  As shown in FIG. 13A, after component group classification, a noise component group (eg, a non-text component group built by windows or traffic lights as shown in FIG. 6) and a false component group (eg, shown in FIG. 6). The text component group composed of characters of different text lines) is removed after step S303. It can be seen that the global prime feature extends the distinction between text groups and non-text groups. Therefore, the final result of component group classification is a good effect.

  The text region generation unit 205 is configured to generate a text region based on the remaining text group, which will be described in step S304 of FIG.

  In step S304, the text area generation unit 205 converts the remaining component group into a text area. The text region is usually generated based on the group component rectangle and the group text line. One exemplary implementation of this step is as follows.

First, a text line is obtained by least squares regression of the centers of all components in the group.
The upper limit line is then determined by parallel shifting of the text lines to cover the top points of the components in the group. Similarly, the lower limit line is determined.
Finally, the left limit line and the right limit line are determined by the rectangles of the leftmost component and the rightmost component in the group.

  As shown in FIG. 13B, the detected text area is an example of a generated text area.

  The text area output unit 206 is configured to output the result of the text area to the output device 106 (for example, an image recognition device) for further image processing such as information extraction and information recognition.

  In group classification, global prime features of text in an image are extracted and combined with local information (group level features). Global prime features are selected based on the distribution of features. Therefore, the present invention can be adapted to various landscapes.

  FIG. 14 shows a text information extraction method according to the embodiment of the present invention. The present invention can be used when text information is automatically extracted from images and moving images taken with a camera. As shown in FIG. 14, in block 1401, a text region from an input image or an input moving image is detected using a text detection method according to the text detection method described with reference to FIGS.

  At block 1402, text is extracted from the detected text region. Optionally, when detecting a text region from the input video, the text of the input video is tracked as shown in block 1404.

  In block 1403, text information is acquired by performing text recognition on the extracted text.

  FIG. 15 is a block diagram showing a text information extraction system according to the embodiment of the present invention.

  Referring to FIG. 15, a block diagram of a text information extraction system 1500 according to an embodiment of the present invention is shown. System 1500 is used to implement the method described with reference to FIG.

  As illustrated in FIG. 15, the system 1500 includes a text detection device 1501, an extraction device 1502, and a recognition device 1503.

  Text detection device 1501 is configured to detect a text region from an input image or input video and is the same as device 200 described with respect to FIG.

Extractor 1502 is configured to extract text from the detected text region.
The recognition device 1503 is configured to acquire text information by recognizing the extracted text.

  Optionally, system 1500 further comprises a tracking device 1504. The tracking device 1504 is configured to track text in the input video if the text detection device 1501 is configured to detect a text region from the input video.

  It will be appreciated that the parts and devices described above with respect to FIGS. 2 and 15 are exemplary and / or suitable modules for implementing the various steps. Modules are hardware units (such as processors and application specific integrated circuits) and / or software modules (such as computer programs). The above is not an exhaustive description of modules for implementing the various steps. However, if there are steps to perform a predetermined process, there is a corresponding functional module or unit (implemented by hardware and / or software) for implementing the same process. Combinations of the steps described above and below and units corresponding to these steps are included in the disclosure of the present invention so long as the technical solution comprising them is complete and applicable.

Claims (22)

A text detection method for detecting a text area of an image, comprising:
A component generation step for generating a component from the input image;
A component grouping step of grouping the components meeting similarity requirements to form a component group;
A component connection extraction step for extracting a component connection including at least two adjacent components in one component group;
A feature acquisition step of acquiring all component connection features;
Based on the feature of the component connection in one component group and the feature of the component connection in another component group different from the component group acquired in the feature acquisition step, the component group is a text group. Component group classification step for classifying into non-text groups;
A text region generating step for generating a text region based on the text group.   The method of claim 1, wherein the feature is obtained from a distribution of values of at least one of the candidate features for all component connections.   The component group classification step further includes a text confidence value calculation step of calculating a text confidence value of each component connection, and the component group classification step converts the component group into a text group or a non-text group based on the text confidence value. The method according to claim 1 or 2, wherein the classification is performed. The component group classification step further includes
A text confidence value adjustment step of adjusting the text confidence value of the component connection according to the feature acquired in the feature acquisition step;
The component group classification step may be configured to classify the component group as a text group or non-based based on a text confidence value of the component group and obtained from an adjusted text confidence value of all component connections in the group. The method of claim 3, wherein the method is classified into text groups.   The method of claim 2, wherein the feature is acquired in the feature acquisition step when a value of a histogram bin of the distribution is greater than a predetermined threshold.   The feature is acquired in the feature acquisition step when the average character likelihood of the component connection in the highest histogram bin of the distribution is greater than the average character likelihood of the component connection in any other histogram bin. The method described in 1.   The feature is acquired in the feature acquisition step when a ratio of values of the highest histogram bin of the distribution and a second histogram bin of the distribution is greater than a predetermined threshold. the method of.   The method of claim 2, wherein the candidate feature of all component connections is a predetermined feature or a set of selected features. The method of claim 2, wherein the candidate feature of all component connections is at least one of the following.
(1) Component connection direction,
(2) The average foreground color of the component,
(3) The average background color of the component,
(4) average boundary contrast, and (5) distance between two components in the component connection.   9. The method of claim 8, wherein the selected feature has a maximum value ratio of a top histogram bin and a second histogram bin.   4. The text confidence value of the component connection is calculated from a set of features extracted from the component connection and obtained by a pretrained classifier of text and non-text components. Method.   5. The method of claim 4, wherein the text confidence value of a component group is calculated as an average text confidence value of all component connections in it.   The method of claim 4, wherein the text confidence value of a component group is calculated as a weighted value of an average text confidence value of all component connections in the component group and a text confidence value determined based on a group level feature. The method of claim 13, wherein the group level feature is at least one of the following:
(1) Distribution of components in the group,
(2) component size, color or stroke width,
(3) Spatial arrangement of components in the group. A text detection device for detecting a text area of an image,
A component generator configured to generate components from an input image;
A component grouping unit configured to group the components that meet similarity requirements to form a component group;
The component connection including at least two adjacent components in one component group is extracted, the characteristics of all component connections are acquired, and the acquired characteristics of the component connection in one component group and the component group are A component group classifier configured to classify the component group into a text group and a non-text group based on the characteristics of component connections in different other component groups;
A text region generation unit configured to generate a text region based on the text group.   The apparatus of claim 15, wherein the feature is obtained from a distribution of values of at least one of the candidate features of all component connections.   The component group classification unit is further configured to calculate a text confidence value of each component connection, and the component group classification unit classifies the component group into a text group or a non-text group based on the text confidence value. The apparatus according to 15 or 16.   The component group classifier is further configured to adjust the text confidence value of a component connection according to the acquired characteristics, wherein the component group text confidence value is adjusted for all component connections in the group. The apparatus of claim 17, wherein the component group is classified into a text group or a non-text group based on a text confidence value obtained from a text confidence value.   A text region is detected from an input image or an input video using the text detection method according to any one of claims 1 to 14, the text is extracted from the detected text region, and extracted Recognizing text and obtaining text information;   The method according to claim 19, further comprising tracking the text in the input video when detecting the text region from the input video using the text detection method according to claim 1. .   The text detection device according to any one of claims 15 to 18 configured to detect a text region from an input image or an input moving image, and an extraction device configured to extract text from the detected text region. And a recognition device configured to acquire text information by recognizing the extracted text.   The system of claim 21, wherein if the text detection device is configured to detect the text region from the input video, the system further comprises a tracking device configured to track the text in the input video.