JP2004355184A - Extraction apparatus and extraction method for character/graphic region, program for the method, and recording medium for recording the program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently and precisely extract the inclusive rectangle of characters and graphics in an image. <P>SOLUTION: An image 100 is a correct answer data creation target image, such as inputted scene photos and photos in a town, and net-like grid construction processing 200 is constituted such that a net-like grid is constructed in the image. Inclusion rectangular extraction processing 300 extracts the inclusion rectangle of each character and each graphic. Then, supplementary information input processing 500 inputs supplementary information, such as image information, character fonts, character codes, and graphic names. Determination processing 700 determines whether processing is completed or not for all characters and graphics in the same net-like grid, and determination processing 800 determines whether processing is completed for all net-like grids in the image. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an apparatus and a method for extracting a rectangular circumscribed rectangular area (enclosing rectangle) including characters and graphics existing in an image using an input device such as a mouse or a keyboard.
[0002]
[Prior art]
The development of character recognition and figure recognition technologies includes various methods such as character codes of characters and figures existing in the image to be recognized, character fonts, figure names, positions of characters and figures, rectangular shapes of characters and figures, image information, etc. Data in which information is aggregated (hereinafter referred to as correct data) is required. This correct answer data may be used as learning data when creating a recognition template (hereinafter referred to as a dictionary), or may be used when measuring the correctness / error performance of a recognition result. This is important when developing recognition technology.
[0003]
In the correct answer data, the position and the included rectangular shape are described by the coordinates of the four vertices of the rectangle. Conventionally, using commercially available image processing software or the like (for example, see Non-Patent Documents 1 and 2), the following procedure is used. Coordinates were extracted.
[0004]
S1. The recognition target image is read into commercially available image processing software and displayed.
[0005]
S2. The rectangle is extracted by manually clicking the four vertices of the enclosing rectangle with a mouse using a rectangular region extraction tool often provided as a standard feature in commercially available image processing software.
[0006]
S3. The coordinates of the four vertices are manually read and recorded using a mouse pointer position display function often provided as a standard feature in commercially available image processing software.
[0007]
[Non-patent document 1]
Image processing software: Publisher: Paint Shop Pro, Jasc Software Inc. [0008]
[Non-patent document 2]
Image processing software: Publisher: Photoshop, Adobe
[Problems to be solved by the invention]
If each character or figure in the image has a rectangular outer frame in advance, it is possible to accurately extract the coordinates of the four vertices of the enclosing rectangle by the conventional method. There is no outer frame.
[0010]
In such a case, first, four sides of an object having a rectangular shape and a clear outer frame, such as a signboard on which characters and figures are written, are first extracted, and then these four sides are used as a reference, that is, these are extracted. By performing the parallel movement, the four sides of the rectangle including the characters and figures are determined, and the coordinates of the vertices at which the sides intersect are extracted. This will take a lot of work.
[0011]
In addition, since the image is perspective-transformed in principle, the four sides of the enclosing rectangle of characters and figures and the corresponding sides of the four sides of the signboard on which they are written are not completely parallel, but are straight lines. Are slightly different from each other, so that it is difficult to extract an inclusive rectangular shape with high accuracy.
[0012]
An object of the present invention is to provide a character / graphic region extracting apparatus, method, program, and recording medium that solve the above-mentioned problems.
[0013]
[Means for Solving the Problems]
The present invention is characterized by an apparatus, a method, a program, and a recording medium having the following functions for efficiently and accurately extracting a contained rectangle of a character or a figure.
[0014]
-Image reading and display function.
[0015]
・ A function to automatically construct a net-like grid.
[0016]
-Fine adjustment function of net-like grid.
[0017]
・ A function to give a mouse pointer a guide line along a net-like grid.
[0018]
-A function to determine the included rectangular shape by designating four vertices.
[0019]
・ Automatic output function of rectangular shape information.
[0020]
-Additional information input functions such as image information, character fonts, character codes, and graphic names as extended functions depending on the purpose of use, in addition to the basic functions 1 to 6 described above.
[0021]
(1) A character / graphic region extracting apparatus for extracting a rectangular inclusion rectangle that includes characters and graphics existing in an image,
A straight line or a characteristic point in an image, such as a frame of a signboard on which a character or figure is drawn, is acquired by image processing technology such as Hough transform or pattern matching, and a character or a figure is obtained based on the straight line or characteristic point. Means for constructing a net-like grid serving as a reference when extracting the inclusion rectangle of
A guide line along the net-like grid is superimposed and displayed on a pointer operated by an input device such as a mouse or a keyboard, and the enclosing rectangle of a character or a figure is formed based on the vertex of the enclosing rectangle specified by using the pointer. Means for extracting,
A character / figure region extraction device characterized by comprising:
[0022]
(2) In the character / graphic region extraction device according to (1),
A character / figure region extracting apparatus, comprising: a plurality of points input using the input device; and a unit configured to construct the net-like grid based on the points.
[0023]
(3) In the character / graphic region extraction device according to (1) or (2),
Characters / graphics comprising means for constructing a plurality of different net-like grids in an image, and superimposing and displaying a guide line along the net-like grid on a pointer in an area of each net-like grid. Region extraction device.
[0024]
(4) In the character / graphic region extraction device according to any one of (1) to (3),
A character / graphic region extracting apparatus comprising means for inputting supplementary information such as a character code of each character or graphic, a character font, a graphic name, or image information using an input device such as a mouse or a keyboard.
[0025]
(5) A character / graphic region extraction method for extracting a rectangular inclusion rectangle that includes characters and / or graphics present in an image,
A straight line or a characteristic point in an image, such as a frame of a signboard on which a character or figure is drawn, is acquired by image processing technology such as Hough transform or pattern matching, and a character or a figure is obtained based on the straight line or characteristic point. Constructing a net-like grid that serves as a reference when extracting the containing rectangle of
A guide line along the net-like grid is superimposed and displayed on a pointer operated by an input device such as a mouse or a keyboard, and the enclosing rectangle of a character or a figure is formed based on the vertex of the enclosing rectangle specified by using the pointer. The process of extracting,
A character / figure area extraction method characterized by comprising:
[0026]
(6) In the character / graphic area extraction method according to (5),
A method for extracting a character / graphic region, comprising a step of inputting a plurality of points using the input device and constructing the net-like grid based on the points.
[0027]
(7) In the character / graphic region extraction method according to the above (5) or (6),
Characters / graphics comprising a step of constructing a plurality of different net-like grids in an image, and superimposing and displaying a guide line along the net-like grid on a pointer in the area of each net-like grid. Region extraction method.
[0028]
(8) In the character / graphic area extraction method according to any one of (5) to (7),
A character / graphic area extraction method, comprising a step of inputting supplementary information such as a character code of each character or graphic, a character font, a graphic name, or image information using an input device such as a mouse or a keyboard.
[0029]
(9) A character / graphic region extraction program, characterized in that the processing procedure in the character / graphic region extraction method according to any one of (5) to (8) is configured to be executable by a computer.
[0030]
(10) A program for causing a computer to execute the processing procedure in the character / graphic area extraction method according to any one of (5) to (8) above is recorded on a computer-readable recording medium. Characteristic recording medium.
[0031]
BEST MODE FOR CARRYING OUT THE INVENTION
Embodiments of the present invention will be described in detail with reference to the drawings.
[0032]
FIG. 1 shows a processing procedure of the present embodiment. Reference numeral 100 denotes an image for which correct data is to be created, such as a landscape photograph or a photograph of a city, which is input by a photographing device or an image reading device. Reference numeral 200 denotes a net-like grid (for extracting a rectangle including characters and figures included in the image). A process for constructing a reference coordinate system), a process 300 for extracting an enclosing rectangle of each character or each figure, and a process 400 for enclosing a rectangle extracted by the enclosing rectangle extracting process 300 to a display or a storage device Shape information, 500 is input processing of additional information such as image information, character fonts, character codes, graphic names, etc., 600 is additional information input by the additional information input processing 500, and 700 is all characters in the same net-like grid. A process for determining whether or not the processing has been completed for the graphic, 800 indicates whether or not the processing has been completed for all net-like grids in the image It is a process of determining the.
[0033]
The image 100 is an image including characters and figures therein, and is, for example, a landscape photograph or a photograph of a city. As an example of the image 100, FIG. 2 shows a signboard photograph image in which a character string “Yokosuka City, Kanagawa Prefecture” is written.
[0034]
The net-like grid construction processing 200 is performed manually by using a line or a characteristic point such as a frame of a signboard on which a character or a figure obtained by the image processing technique is drawn, or an input device such as a mouse or a keyboard. This is a process for automatically constructing a net-like grid based on the set reference points.
[0035]
Here, a method for detecting a straight line such as a frame of a signboard from an image and creating a grid net based on the straight line will be described. As a method of detecting a straight line, Hough transform and pattern matching (reference document [1]) are well known methods.
[0036]
Reference 1 "Image Processing Standard Textbook, Image Information Education Promotion Association, pp. 187-190"
A net-like grid can be created by setting the intersection of a plurality of vertical straight lines extracted by the Hough transform and the intersection of a plurality of horizontal straight lines as vanishing points. FIG. 3 shows an example. In FIG. 3, 201, 202, 203, and 204 are extracted straight lines. A vanishing point 205 is formed from the intersection of the vertical straight lines 201 and 202, and another vanishing point 206 is formed from the intersection of the horizontal straight lines 203 and 204. I get it.
[0037]
In this embodiment, a method of manually inputting a reference point for constructing a net-like grid will be described below. FIG. 4 shows an example of the reference point. In the example of FIG. 4, four vertices at the upper left, lower left, lower right, and upper right of the frame existing on the signboard are set as reference points 207, 208, 209, and 210. If there is no such thing as the frame line in FIG. 4, that is, if the reference point is not clear, for example, the four vertices of the upper left, lower left, lower right, and lower right of the “eye” portion included in the characters “prefecture” It can be a reference point. The larger the distance between the reference points is, the more preferable it is to construct a net grid with higher accuracy. Even after the reference point has been input once, the position can be finely corrected using an input device. In addition, not only four reference points but also five or more reference points can be input in order to construct a more accurate net-like grid. However, in the present embodiment, four reference points will be described. Further, when a plurality of signboards appear in the photographic image, there are a plurality of net-like grids, and different reference points are input for each net-like grid.
[0038]
Next, a net grid is automatically constructed based on the input reference points. FIG. 5 shows an example of this processing. The reference points 207, 208, 209, and 210 in FIG. 5 are four reference points, and from the coordinates of these points, the vanishing points 211 and 212 of the net-like grid can be obtained by a simple geometric calculation.
[0039]
FIG. 6 shows an example in which grid lines at equal angular intervals starting from the obtained vanishing point are superimposed on the signboard photograph image of FIG. 2 and displayed. This grid line can be arbitrarily hidden. Further, even after the vanishing point is calculated once and the net-like grid is constructed, the reference point position can be finely corrected using the input device and the grid can be constructed again.
[0040]
Next, the inclusion rectangle extraction process 300 in FIG. 1 is a process of extracting the inclusion rectangle by manually specifying four vertices of the inclusion rectangle of a character or a figure using an input device. In order to efficiently and accurately extract the included rectangle, two mouse guide lines are generated along the net-like grid. By moving the mouse pointer to the four vertices of the containing rectangle with reference to the guide line, the vertices can be efficiently and accurately specified. FIG. 7 shows an example of a mouse pointer with a guide line.
[0041]
In this case, the inclusion rectangle shape information 400 is the coordinates of the four vertices of the inclusion rectangle extracted by the inclusion rectangle extraction processing 300.
[0042]
The additional information input processing 500 includes, if necessary, image information such as an image size and the number of colors, a character font, a character code, a graphic name, a net-like grid to which each character or graphic belongs, and a character string or graphic to which each character or graphic belongs. This is a process of manually inputting incidental information regarding columns, photo shooting locations, photo shooting times, and the like. The input of these pieces of information is performed by the input device at the same time as the process of extracting the containing rectangle.
[0043]
The supplementary information 600 is each supplementary information input by the supplementary information input processing 500 and is output as a file of an arbitrary designated format such as a text file or a CSV file.
[0044]
The same net-like grid process end determination process 700 is a process of manually inputting a determination result as to whether or not all characters and graphics in one net-like grid have been processed. If there is a character or graphic that has not been processed yet, the process returns to the inclusion rectangle extraction processing 300, and the inclusion rectangle extraction processing 300 and the accompanying information input processing 500 are repeatedly executed.
[0045]
The process end determination process 800 is a process of manually inputting a determination result as to whether or not the process has been completed for all net-like grids in an image. If there is a net-like grid that has not been processed yet, the process returns to the net-like grid reference point input processing 200, and the net-like grid construction processing 200, the inclusion rectangle extraction processing 300, and the supplementary information input processing 500 are repeatedly executed.
[0046]
By the processing described above, it is possible to efficiently and accurately extract the inclusion rectangles of the characters and figures in the image.
[0047]
FIG. 8 shows the configuration of a character / graphic region extracting apparatus according to an embodiment of the present invention. Each unit has a computer configuration that implements the processing functions of FIG. 1 by a software configuration. In FIG. 8, an image input unit 1 is a photographing device or an image reading device for inputting a correct data creation target image such as a landscape photograph or a street photograph. The net grid constructing unit 2 constructs a net grid for the image input from the image input unit 1. The inclusion rectangle extraction unit 3 extracts inclusion rectangle shape information of each character and each figure included in the image. The supplementary information input unit 4 performs a process of inputting supplementary information such as image information, a character font, a character code, and a graphic name for the extracted included rectangular shape information. The information output unit 5 outputs the inclusion rectangle information extracted by the inclusion rectangle extraction unit 3 and the supplementary information input by the supplementary information input unit 4 to a display or a storage device. The determination input unit 6 responds to a determination input as to whether processing has been completed for all characters and graphics in the same net-like grid and a determination input as to whether processing has been completed for all net-like grids in the image. Then, execution in the net-like grid construction unit 2 and the inclusion rectangle extraction unit 3 is instructed.
[0048]
The present invention can be used for geometric calibration in a three-dimensional information restoration technique for obtaining the shape and motion of a three-dimensional object reflected on an image using a two-dimensional image. In the three-dimensional information restoration, it is necessary to accurately obtain a process of projecting three-dimensional information into two-dimensional information by a camera or a video, and a camera model and various parameters thereof are calibrated. As one of the calibration methods, there is a method using a property of a vanishing point, and various conventional studies described below exist.
[0049]
(A) New Technology Communications, Computer Vision, pp. 42
(B) Caprile and V. Torre, Using vanishing points for camera calibration, IJCV, Vbl. 4, pp. 127-140.1990.
[0050]
(C) W. Chen and B.S. C. Jiang, 3-D camera calibration using vanishing point concept, PR, Vol. 24, pp. 57-67.1991.
[0051]
These conventional methods prepare an object drawn with parallel lines, and automatically extract vanishing points by analyzing an image taken of the object, but in the present invention, the judgment and input by hand, the outer frame of the signboard and the like The vanishing point can be extracted using various clues such as a rectangle inside the character. By applying the vanishing point extracted according to the present invention to the conventional method, simple and efficient calibration can be performed.
[0052]
Further, the present invention can also be used for editing such as distortion correction and rotation correction for images and videos shot by cameras and videos.
[0053]
In the present invention, a part or all of the processing functions of the method shown in FIG. 1 and the like can be configured as a program and executed using a computer. Also, a computer-readable recording medium such as a flexible disk, an MO, a ROM, and a memory card may be used to execute a program for realizing the processing function of each unit in the computer or a program for causing the computer to execute the processing procedure. , A CD, a DVD, a removable disk, or the like, and can be stored or provided, and can be distributed via a communication network such as the Internet.
[0054]
【The invention's effect】
As described above, according to the present invention, when extracting an inclusion rectangle of a character or figure, a coordinate system (net-like grid) serving as a reference of the inclusion rectangle is constructed, and a mouse pointer provided with guide lines along the coordinate system is provided. Is moved to specify the vertices of the enclosing rectangle, so that the enclosing rectangle of the character or figure in the image can be efficiently and accurately extracted. As a result, correct data for character recognition and graphic recognition technology development can be efficiently created.
[0055]
Further, in the present invention, the vanishing point extracted by hand can be used for a calibration method of a camera model and its parameters in restoring three-dimensional information from a two-dimensional image.
[0056]
Further, the present invention can be used for editing such as distortion correction and rotation correction for images and videos shot by a camera or video.
[Brief description of the drawings]
FIG. 1 is a processing procedure showing an embodiment of the present invention.
FIG. 2 is an example of a signboard photograph image in the embodiment.
FIG. 3 is an example of creating a net-like grid in the embodiment.
FIG. 4 is an example of a net-like grid reference point in the embodiment.
FIG. 5 is an example of a net-like grid construction process in the embodiment.
FIG. 6 is an example of a net-like grid in the embodiment.
FIG. 7 is an example of a mouse pointer with a guide line in the embodiment.
FIG. 8 is an apparatus configuration diagram showing an embodiment of the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Image input part 2 ... Net-like grid construction part 3 ... Inclusion rectangle extraction part 4 ... Additional information input part 5 ... Information output part 6 ... Judgment input part

Claims (10)

What is claimed is: 1. A character / graphic region extracting apparatus for extracting a rectangular inclusion rectangle that includes characters and graphics existing in an image,
A straight line or a characteristic point in an image, such as a frame of a signboard on which a character or figure is drawn, is acquired by image processing technology such as Hough transform or pattern matching, and a character or a figure is obtained based on the straight line or characteristic point. Means for constructing a net-like grid serving as a reference when extracting the inclusion rectangle of
A guide line along the net-like grid is superimposed and displayed on a pointer operated by an input device such as a mouse or a keyboard, and the enclosing rectangle of a character or a figure is formed based on the vertex of the enclosing rectangle specified by using the pointer. Means for extracting,
A character / figure region extraction device characterized by comprising: The character / graphic area extraction device according to claim 1,
A character / figure region extracting apparatus, comprising: a plurality of points input using the input device; and a unit configured to construct the net-like grid based on the points. 3. The character / graphic area extraction device according to claim 1,
Characters / graphics comprising means for constructing a plurality of different net-like grids in an image, and superimposing and displaying a guide line along the net-like grid on a pointer in an area of each net-like grid. Region extraction device. 4. The character / graphic area extraction device according to claim 1,
A character / graphic region extracting apparatus comprising means for inputting supplementary information such as a character code of each character or graphic, a character font, a graphic name, or image information using an input device such as a mouse or a keyboard. A character / graphic region extraction method for extracting a rectangular inclusion rectangle that includes characters and / or graphics present in an image,
A straight line or a characteristic point in an image, such as a frame of a signboard on which a character or figure is drawn, is acquired by image processing technology such as Hough transform or pattern matching, and a character or a figure is obtained based on the straight line or characteristic point. Constructing a net-like grid that serves as a reference when extracting the containing rectangle of
A guide line along the net-like grid is superimposed and displayed on a pointer operated by an input device such as a mouse or a keyboard, and the enclosing rectangle of a character or a figure is formed based on the vertex of the enclosing rectangle specified by using the pointer. The process of extracting,
A character / figure area extraction method characterized by comprising: The character / graphic area extraction method according to claim 5,
A method for extracting a character / graphic region, comprising a step of inputting a plurality of points using the input device and constructing the net-like grid based on the points. The character / graphic area extraction method according to claim 5 or 6,
Characters / graphics comprising a step of constructing a plurality of different net-like grids in an image, and superimposing and displaying a guide line along the net-like grid on a pointer in the area of each net-like grid. Region extraction method. 8. The character / graphic area extraction method according to claim 5,
A character / graphic area extraction method, comprising a step of inputting supplementary information such as a character code of each character or graphic, a character font, a graphic name, or image information using an input device such as a mouse or a keyboard. A character / graphic area extraction program, wherein the processing procedure in the character / graphic area extraction method according to any one of claims 5 to 8 is configured to be executable by a computer. A recording medium, wherein a program for causing a computer to execute the processing procedure in the character / graphic area extraction method according to any one of claims 5 to 8 is recorded on a computer-readable recording medium.

Images