JPH11306282A - Method and device for extracting character string area - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enhance the speed of extraction of a binary image in a character string area when reading character information on a character plate in a specified format by using a camera as an input means. SOLUTION: A character string extracting device is provided with a camera 2, a character plate area extracting means 3 to automatically extract position information of a character area, a creating/storing means 10 of a template in an area to be extracted to create and store the template in the area to be extracted to specify an area to be threshold a processed, a thresholding means 4, an inclination/rotation angle estimating means 6 to estimate three-dimensional attitude and rotation in a plane for the image on the character plate, a template deforming means 1 of the area to be extracted to threshold the image and to deform the shape of the template in the area to be extracted according to the three-dimensional attitude and the rotation in the plane, and an inclination/rotation correcting means 5 of binary image to correct distortion due to three- dimensional inclination and the rotation in the plane only for black pixels existing within the range of the deformed template in the area to be extracted.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for extracting a character string area and an apparatus for implementing the method, and more particularly to a technique for reading character information on a character plate having a fixed format such as a name tag using a camera as an input means. The present invention relates to a technique for extracting a binary image of a character string area to be read.

[0002]

2. Description of the Related Art As shown in FIG. 16, a conventional character string region extracting device includes a camera 100 for photographing a scene image, a character plate region extracting means 200, and a binarizing unit for extracting the extracted character plate region. Binarizing means 300, binary image tilt / rotation correcting means 400 for correcting distortion due to three-dimensional tilt and rotation in a plane, character string area extracting means 50
0, and a tilt / rotation angle estimating means 600 for estimating a three-dimensional attitude and rotation in a plane.

FIG. 17 is a schematic diagram of an example of a name tag to be read as viewed from the front, and FIG. 18 is a diagram for explaining the contents of FIG. In FIG. 18, 101 is a company mark, 102 is an employee card, 103 is a line segment area, 104 is a photograph area, 105 is a character string area of a company name, 106 is a character string area of a department name, and 107 is a character of a name. It is a string area, and is a character string area to be read.

FIG. 19 is a schematic view showing an example of an image obtained by photographing the name tag of FIG. 17 with the camera 100, and there is distortion due to three-dimensional inclination.

FIG. 20 is a diagram showing an image in which only the pixels in the background area in the name tag are extracted from FIG. 19, and in this example,
It is assumed that the color information of the background area of the name tag is known in the form of, for example, RGB values (red, green, and blue values), and this extraction is performed using the color information.

FIG. 21 is a diagram showing a binary image in which pixels other than the background pixel in the area belonging to the name tag are black pixels, and all the rest are white pixels.

FIG. 22 is an image obtained by correcting the distortion and rotation caused by the inclination with respect to the image of FIG. However, in this example, since the rotation is very small, the effect of the rotation correction cannot be visually recognized.

FIG. 23 is a diagram showing the result of extracting only the character string region from FIG. According to this method, for example, a connected component of a black pixel is obtained by labeling, and a connected character component is combined to create a character pattern candidate, and a plurality of character pattern candidates having substantially the same size are almost constant in the horizontal direction. This can be done by selecting the arrangement at intervals as a reference for the character string area.

After the processing shown in FIG. 23, character extraction (separation of individual character patterns) and character recognition are performed on the binary image belonging to each character string area, and the binary image is collated with a word dictionary. Each character string area can be read. Among them, only the department and name which are the character strings to be read are sent to the subsequent processing.

The correction of the distortion and the rotation caused by the inclination is performed on all the black pixels in the binary image. However, this processing is a processing using a large amount of a trigonometric function and has a problem that the processing speed is reduced. (1997 IEICE National Convention, Information and Systems Society Conference, p.250, Title: Monocular Viewing Estimation of Character Plane Using Image Oblique Distortion)
reference).

In addition, for a binary image in which distortion and rotation due to tilt have been corrected for all regions, character strings are extracted, characters are cut out, characters are recognized, and words are collated in all regions. There is a problem that the processing speed is reduced.

[0012]

SUMMARY OF THE INVENTION As a result of studying the above prior art, the present inventor has found the following problems.

In the above-mentioned prior art, the correction of the distortion and the rotation caused by the inclination is performed for all the black pixels in the binary image. However, this processing is a processing that uses a large amount of trigonometric functions and has a large processing speed. There was a problem of lowering.

In addition, for a binary image in which distortion and rotation due to tilt have been corrected for all regions, character strings are extracted, characters are cut out, characters are recognized, and words are collated in all regions, so that the processing speed is reduced. There was a problem to do.

An object of the present invention is to provide a technique for reading character information on a character plate having a fixed format such as a name tag using a camera as an input means.
It is an object of the present invention to provide a technique capable of speeding up a value image extraction process.

The above and other objects and novel features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

[0017]

SUMMARY OF THE INVENTION Among the inventions disclosed in the present application, the outline of a representative one will be briefly described.
It is as follows.

(1) The position information of the region of the character string to be extracted is automatically extracted by analyzing the image of the character plate having a known three-dimensional posture presented as a sample. A learning process for creating and storing an extraction-needed area template specifying an area to be binarized for the position information, and a three-dimensional posture and rotation in a plane with respect to an image of a character plate to be input. Estimating and binarizing the image, and deforming the shape of the extraction required area template created in the learning process in accordance with the three-dimensional posture and rotation in the plane, 3 for only black pixels within the range of the template
This is a character string region extraction method including an input correction process for correcting distortion due to dimensional inclination and rotation in a plane.

(2) The camera that captures the scene image and the position information of the character string area to be extracted included in the scene image are
A character plate area extracting means for automatically extracting by analyzing an image of a character plate whose three-dimensional posture presented as a sample is known, and an area to be binarized of the extracted character plate area are designated. Extraction area template creating / storing means for creating and storing an extraction area template required, binarizing means for binarizing the designated character plate area, and A tilt / rotation angle estimating means for estimating a three-dimensional attitude and rotation in a plane, binarizing the image, and converting the shape of the extraction-required area template into the three-dimensional attitude and rotation in the plane Extracting area template deforming means for deforming according to the following, and distortion due to three-dimensional inclination only for black pixels within the range of the deformed extracting area template A character string region extracting device having a binary image tilt / rotation correction means for correcting the rotation of the plane.

That is, a feature of the present invention is to provide a means for performing a learning process for creating a required extraction area template from a learning sample image. In the input process,
The magnitude of the three-dimensional inclination and rotation of the character plate is estimated from the image, and the extraction-needed area template is transformed using the magnitude of the inclination and rotation to overlap the character plate image. That is, an area within the range of the extraction required area template is to be processed.

[0021]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The embodiments (examples) of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a block diagram showing a schematic configuration of a character string area extracting apparatus according to an embodiment (embodiment) of the present invention. FIG. 2 is a character string area extracting method by the character string area extracting apparatus of FIG. 6 is a flowchart showing the processing procedure of FIG.

The character string area extracting apparatus according to the present embodiment has the structure shown in FIG.
As shown in FIG. 7, an extraction required area template deforming means 1 for deforming an extraction required area template for designating an area to be binarized in a character plate area in accordance with a three-dimensional posture and rotation in a plane, and photographing a scene image Analyzing an image of a character plate whose three-dimensional posture is known, which presents, as a sample, position information of a region of a character string to be extracted included in a scene image captured by the camera 2 , A character plate area extracting means 3 for automatically extracting, a binarizing means 4 for binarizing the extracted character plate area, and correction of distortion due to three-dimensional inclination and rotation in a plane. Binary image tilt / rotation correction means 5 for performing a three-dimensional posture and tilt /
A rotation angle estimating unit 6, a character string region extracting unit 7, a character segmentation recognizing unit 8, a word matching unit 9, and an extraction required region template for designating a region of the character plate region to be binarized are created and stored. And an extraction area template creation / storage means 10.

FIG. 3 is a schematic diagram of an example of an image of a learning sample for creating an extraction area template.
Is a company mark, 102 is an employee ID, 103 is a line segment area,
104 is a photograph area, 105 is a character string area of a company name, 10
Reference numeral 6 denotes a character string area of the department name, and 107 denotes a character string area of the name, which are to be read.

In the learning process, it is assumed that the three-dimensional posture of the character plate has been determined.

FIGS. 4 to 7 are diagrams for explaining a method of creating a required extraction area template, and FIG. 8 is a diagram showing an example of the contents of a word dictionary used for word matching. The spelling of the word and information on whether or not each word is to be extracted are entered in the form of "necessary / unnecessary".

FIG. 9 is a diagram showing the results of character segmentation, character recognition, and word collation performed on each character string region extracted in FIG. Only the character string area 106 of the belonging department name and the character string area 107 of the name indicate "necessary", that is, the character string area to be extracted. Using this result, the result of setting the character string area 106 of the belonging department name and the character string area of the name character string area 107 as the required extraction area is shown in FIG.
Shown in However, for other character plates, the position of the character string and the number of characters are expected to be slightly different. Therefore, FIG. 11 shows an example in which a margin is provided around these two regions.
This is used as an extraction required area template. Formally, each region is represented by a rectangle, and the extraction-needed region template is represented by the coordinates of the four corners of each region.

The process of processing the example of the image in FIG. 3 using the extraction-necessary area template obtained by the above means will be described below.

FIGS. 12 to 15 are diagrams for explaining the subsequent processing steps. FIG. 12 is a diagram showing a result of transforming the shape of the character plate handling the extraction required area template shown in FIG. 11 into a shape when three-dimensionally inclined / rotated in a plane according to the appearance of the character plate. (However, it doesn't look like it hasn't rotated much). Formally, the transformation is performed by transforming the coordinates of the four corners of the rectangle representing each area.

FIG. 13 shows the binary image of FIG. 5 superimposed with the extraction required template of FIG. 12, and FIG.
FIG. 8 is a diagram showing a result of extracting an image within a range of an extraction required area template. FIG. 15 is a diagram showing the result of correcting distortion and rotation due to three-dimensional inclination similar to the related art with respect to the black pixels extracted in FIG.

Next, a method for extracting a character string area by the character string area extracting apparatus shown in FIG. 1 will be described with reference to FIG.

The processing procedure of the character string area extraction method according to the present embodiment is executed by the processing steps shown in FIG. That is, S201: First, a grayscale image including a character plate having a posture having a designated value as shown in FIG. 3 in a scene image captured by the image camera 2 is input as learning data.

S202: The character plate area extracting means 3 extracts a grayscale image of the character plate area as shown in FIG. 4 from the input grayscale image.

S203: As shown in FIG. 5, the binarizing means 4 creates a binary image from the image of the character plate area, in which the pixels in the background area of the plate are white and the pixels other than the background area are black.

S204: Binary image tilt / rotation correction means 5
Thus, as shown in FIG. 6, the distortion due to the three-dimensional inclination of the binary image of the character plate area is corrected.

S205: Binary image tilt / rotation correction means 5
, The rotation of the binary image in the character plate area is corrected.

S206: By the character string area extracting means 7,
As shown in FIG. 7, a character string region is extracted from a binary image of a character plate region in which distortion and rotation due to three-dimensional inclination have been corrected.

S207: The character extraction / recognition means 8 performs character extraction / character recognition processing on each character string area.

S208: As shown in FIG. 8, the word matching means 9 performs word matching on the result of the character segmentation / character recognition processing for each character string area, and generates the extraction area template required / storage means 10 As shown in FIG. 9, it is determined whether or not the character string area is to be extracted, and for the character string area to be extracted, a necessary extraction area template representing the location of the character plate in the image is created. And store it in memory.

S209: A gray-scale image including a character plate to be input is input.

S210: A shaded image of the character plate area is extracted.

S211: From the outline of the character plate area,
The three-dimensional attitude of the character plate and the rotation angle within the plane are calculated.

S212: Using the three-dimensional posture of the character plate and the rotation angle in the plane by the extraction area template transformation means 1, the shape of the extraction area template is changed in the same plane using the character. Deform to the shape when superimposed on the plate.

S213: The binarizing means 4 creates a binary image from the image of the character plate area, in which pixels in the background area of the plate are white and pixels other than the background area are black.

S214: The binary image of the character plate area is superimposed on the deformed extraction area template, and 3 pixels are added to the black pixels included in the area of the extraction area template.
The distortion due to the dimensional inclination and the rotation in the plane are corrected and output.

The processing procedure of the character string area extracting method according to the present embodiment can also be realized by a computer software program.

As described above, the present invention has been specifically described based on the above-described embodiments. However, the present invention is not limited to the above-described embodiments, and it is needless to say that various modifications can be made without departing from the scope of the present invention. It is.

[0048]

The effects obtained by typical aspects of the invention disclosed in the present application will be briefly described as follows.

According to the character string area extraction method having the learning processing step and the input correction processing step of the present invention, the distortion and rotation due to the inclination are corrected for the black pixels included in the limited area in the binary image. Since only this is performed, the processing speed can be improved.

Further, since the extraction of character strings, character segmentation, character recognition, and word collation are performed only on a limited area in the binary image, the processing speed can be improved.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of a character string region extraction device according to an embodiment (example) of the present invention.

FIG. 2 is a flowchart showing a processing procedure of a character string region extracting method by the character string region extracting device of FIG. 1;

FIG. 3 is a schematic diagram of an example of an image of a learning sample for creating a required extraction area template;

FIG. 4 is a diagram for explaining a method of creating a required extraction area template according to the embodiment;

FIG. 5 is a diagram for explaining a method of creating a required extraction area template according to the embodiment.

FIG. 6 is a diagram for explaining a method of creating an extraction required area template according to the embodiment;

FIG. 7 is a diagram for explaining a method of creating a required extraction area template according to the embodiment;

FIG. 8 is a diagram showing an example of the contents of a word dictionary used in word matching according to the embodiment.

FIG. 9 is a diagram showing a result of performing character segmentation, character recognition, and word collation for each character string region extracted in FIG. 7;

FIG. 10 is a diagram showing a result of using a character string area of a department area and a name area according to the embodiment as a required extraction area.

FIG. 11 is a diagram showing an example in which a margin is provided around these two regions because the position of the character string and the number of characters are expected to be slightly different in the present embodiment.

FIG. 12 is a diagram for explaining an input processing process according to the embodiment;

FIG. 13 is a diagram for explaining an input processing process according to the embodiment;

FIG. 14 is a diagram illustrating an input processing process according to the embodiment.

FIG. 15 is a diagram for explaining an input processing process according to the embodiment;

FIG. 16 is a block diagram showing a schematic configuration of a conventional character string area extraction device.

FIG. 17 is a schematic view of a conventional example of a name tag to be read as viewed from the front.

FIG. 18 is a diagram for explaining the contents of FIG. 17;

19 is a schematic diagram of an example of an image obtained by photographing the name tag of FIG. 17 with a camera.

FIG. 20 is a diagram showing an image in which only the pixels in the background area in the name tag are extracted from FIG. 19;

FIG. 21 is a diagram showing a binary image in which pixels other than the background pixel in a region belonging to a name tag are set as black pixels, and all the rest are expressed as white pixels.

FIG. 22 is an image obtained by correcting distortion and rotation due to tilt with respect to the image of FIG. 21;

FIG. 23 is a diagram showing a result of extracting only a character string region from FIG. 22;

[Explanation of symbols]

Reference numeral 1 denotes an extraction-required area template deforming means; 2 represents a camera;
... Character plate area extraction means, 4 ... Binarization means, 5 ... 2
Value image inclination / rotation correction means, 6 ... inclination / rotation angle estimation means, 7 ... character string area extraction means, 8 ... character cutout recognition means, 9 ... word collation means, 10 ... extraction area required template creation / storage means, 101 … Company mark, 102… Employee ID, 103… Line segment area, 104… Photo area, 105… Company name character string area, 106… Affiliation department name character string area,
107: Name character string area, 100: Camera, 200 ...
Character plate area extraction means, 300... Binarization means, 40
0: binary image inclination / rotation correction means, 500: character string area extraction means, 600: inclination / rotation angle estimation means.

Claims (2)

[Claims] 1. Position information of a character string area to be extracted is
Analyze the image of the character plate whose three-dimensional posture is known, presented as a sample, to automatically extract it and create an extraction required area template that specifies the area to be binarized for the extracted position information. The learning process in which the three-dimensional posture and the rotation in the plane are estimated with respect to the image of the character plate to be input,
The shape of the extraction required area template created by the learning process is transformed according to the three-dimensional posture and the rotation in the plane, and is within the range of the deformed extraction required area template. A character string region extraction method, comprising: an input correction processing step of correcting distortion due to three-dimensional inclination and rotation in a plane only for black pixels. 2. A camera that captures a scene image and an image of a character plate whose three-dimensional posture is known, which is presented as a sample, with position information of a character string area to be extracted included in the scene image being analyzed. Thereby, a character plate area extracting means for automatically extracting, and a required area template creating / storing means for creating and storing an extracted area template for designating an area to be binarized of the extracted character plate area Binarizing means for binarizing the designated character plate area, and inclination / rotation angle estimating means for estimating a three-dimensional attitude and rotation in a plane with respect to an image of the character plate to be input And binarize the image,
Means for changing the shape of the extraction-needed area template according to the three-dimensional posture and rotation in a plane; A character string region extraction device, comprising: a binary image inclination / rotation correction unit for correcting distortion due to three-dimensional inclination and rotation in a plane.