JP2004151771A - Method of recognizing object in image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method of recognizing an object in an image for carrying out the proper processing of a recognized object image based on the parameter of the recognized object image. <P>SOLUTION: An image to be recognized is inputted, and a recognition template image is read from a database in which the recognition template image is stored, and the recognition template image is compared with the image to be recognized, so that object recognition can be carried out. Then, the recognized image obtained by the object recognition is used as the recognition template image, and the template image and the parameter in recognition are registered in the database so as to be associated with each other. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a technique for recognizing an object displayed in an image.
[0002]
[Prior art]
As a conventional technique for recognizing an object from digitally processed photographic image data, a large number of template images are prepared in advance in an information processing apparatus, and an image in a photographed image is compared with a comparison technique such as pattern matching. Those who recognized it as a specific object were common.
[0003]
As a conventional technique of this type, for example, in order to achieve both image processing and object recognition accuracy, an object is photographed using a plurality of cameras, and three-dimensional processing is performed based on the position states of these cameras, and the object is subjected to three-dimensional processing. There is something to grasp (see Patent Document 1).
[0004]
[Patent Document 1]
JP-A-2002-83297
[Problems to be solved by the invention]
An object of the present invention is to improve such a conventional technique, and to provide an object recognition technique capable of appropriately processing an object image based on parameters of the recognized object image.
[0006]
[Means for Solving the Problems]
First, the present invention automatically creates a template image from a recognized image. That is, the recognition target image is input, the recognition template image is read from the database storing the recognition template image, the object recognition is performed by comparing the recognition template image with the recognition target image, and the object recognition is performed. The recognition image is used as a recognition template image, and the template image is associated with parameters at the time of recognition and registered in the database. By performing such a process, for example, comparison can be performed with a template image created from the image to be recognized itself, and object recognition can be performed with high accuracy.
[0007]
Secondly, the present invention moves or deforms a recognition image obtained by object recognition based on parameters at the time of recognition. That is, the recognition target image is input, the recognition template image is read from the database storing the recognition template image, the object recognition is performed by comparing the recognition template image with the recognition target image, and the recognition image obtained by the object recognition is obtained. Is moved or deformed based on the parameter. By executing such processing, for example, it is possible to output an image obtained by moving the image of the object (recognition image) recognized at the corner of the image to the center or an image obtained by enlarging the recognition image.
[0008]
BEST MODE FOR CARRYING OUT THE INVENTION
<< Embodiment 1 >>
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0009]
FIG. 1 shows the system configuration of the present invention. As shown in FIG. 1, the system includes an input interface unit including a template image creation unit and a recognized image input unit, and an object image recognition engine unit.
[0010]
The present system is realized by executing an object recognition program on a general-purpose computer. More specifically, the functions of the input interface unit and the object image recognition engine unit are realized by the main memory, CPU (Central Processing Unit), and the like of the computer performing processing according to an object recognition program as described below. As an input device of this system (computer), coordinate input means such as a mouse or a digitizer, a keyboard, a scanner, a camera, a VTR, and the like can be used. The storage means includes a semiconductor memory, a hard disk device, and the like, and various programs for realizing the functions of the present embodiment, a template database, and an image to be recognized are stored in the storage means.
[0011]
Note that the present system is not limited to a software realized by a general-purpose computer, and may be configured by an electronic circuit (hardware) having the functions of the input interface unit and the object image recognition engine unit.
[0012]
(Template image creation unit)
The template image creation unit creates a template for recognition and inputs the template to the object image recognition engine unit. The function units include a template database, an image input unit, an image conversion unit, and a quantizer. And a template image developing unit.
[0013]
Here, the template database stores a template image for recognition.
The image input unit reads a template image for recognition from the database and receives an input.
[0014]
The image conversion unit converts the input image data (RGB color data) into a gray scale (256 levels, corresponding to 8 bits) or a luminance signal Y for efficient recognition processing.
[0015]
The quantizer quantizes the image according to the quantization table of FIG.
Further, the template image developing unit performs rotation or deformation of the template image according to the recognition purpose, such as recognition of the same image or similar image.
[0016]
The image registration unit registers the recognition image recognized by the object image recognition engine unit in the template database.
Specifically, the following processing is performed.
(1) In the case of the same image recognition, the template image read from the database is used as it is.
[0017]
{Circle around (2)} In the case of similar image recognition, the template image image data is normalized and used. In the normalization method, the x coordinate is normalized to 32 pixels, and the y coordinate is normalized according to the reduced scale.
[0018]
(3) If the template image is a line image, after the processing of (1) and (2) above, it is converted into a gray scale (256 levels, equivalent to 8 bits) or a luminance signal Y, which is further quantized as necessary and the efficiency is increased. A good recognition process.
[0019]
{Circle around (4)} In order to cope with the difference in the orientation of the object to be recognized, a plurality of template images are created from the recognized template image by transforming the image. In this method, first, a plurality of deformed template images for similar image recognition are created. By using the transformed recognition template image together, for example, the object image viewed from the horizontal direction or the vertical direction can be recognized together with the object front image.
The creation of the deformed template image is performed in the vertical and horizontal directions with respect to the basic recognition template image. This makes it possible to accurately perform recognition processing on the same object image even when viewed from the top, bottom, left, and right.
After the recognition template image is created, the recognition template image is deformed in the following procedure to enable recognition of a similar object image (modeled as a deformed object).
[0022]
Here, a modified template image for similar image recognition is created from the following commonly used equation.
x = ((X−x0) COSθ− (Y−y0) SINθ) / a + x0
y = ((X−x0) SINθ− (Y−y0) COSθ) / b + y0
[0023]
In the above equation, enlargement / reduction processing is performed by a factor in the horizontal direction and by a factor of b in the vertical direction around a point (x0, y0) on an arbitrary image. In addition, the rotation about the center point (x0, y0) can be performed by θ, but in the case of processing only in the vertical and horizontal directions, the calculation is performed with θ = 0. Here, the coordinates (x, y) are the enlarged / reduced image position, and the pixel data at the original image position (X, Y) corresponding to this is used as it is as the enlarged / reduced pixel. There is an advantage that can be.
[0024]
By using the modified recognition template image together, for example, the object image viewed from the horizontal direction or the vertical direction can be recognized together with the object front image;
[0025]
FIG. 3 is a specific example showing a method for creating a deformed image template image.
[0026]
In the figure, the original image template image is divided vertically into four parts and four parts horizontally, and then the respective modified image template images are created by changing the scale of each divided area. At this time, in the program, the scale of the base image template image and the scale for each deformation direction can be changed as a table.
[0027]
By creating such a deformed image template image, for example, when searching for all image files including a face image of the same person, inputting one image of the person Is also searchable. Thus, an efficient image search can be performed without preparing a plurality of template images for the search.
[0028]
In an actual program, the scale of the base image template image and the scale for each deformation direction can be changed as a table. Further, the number of divisions of the blocks is also variable.
[0029]
This is because the speed of improving the CPU performance of the computer is high, and a server equipped with a plurality of CPUs is considered. FIG. 4 shows an example of an actual deformed template image. In the figure, the base image template image is divided into four parts, and the scale ratio is changed in each divided area. That is, in the case of the right figure, the scale is 1.75 → 1.25 → 0.75 → 0.25 in order from the left area. By deforming in this way, even if only a face photograph image from the front of the person is given, the face state of the person facing left or right can be prepared as a template image.
[0030]
(Object image recognition engine)
The object recognition engine is a functional unit that recognizes an object in a recognition image based on various algorithms. Since there are various known techniques for the object recognition algorithm, the description is omitted here.
The output unit moves or deforms the recognized image recognized by the object recognition engine based on the parameter and outputs the image.
[0031]
(Recognized image input unit)
The recognized image input unit is an interface unit for inputting image data to be recognized, and includes a bitmap image input unit and an image conversion unit. These components are the same as the image input unit and the image conversion unit in the recognition image template image creation unit described with reference to FIG. If the input image is an analog signal, it is converted to a bitmap image so that the image processing is easy.
[0032]
(Object recognition method)
Next, an object recognition method in the present system will be described.
The object recognition method of the present system includes a first step of creating a template image from a recognized image, a second step of comparing the created template image with the recognized image and performing object recognition, and an object recognition method. There is a third step of outputting a recognition image.
[0033]
Hereinafter, each step will be described.
(1). Process of Creating Template Image from Recognized Image FIGS. 5 and 6 show the flow of processing.
When the process is started, first, the image input unit of the template image creation unit reads a default template image from the template database and inputs the default template image to the object image recognition engine unit (S1).
[0034]
Further, the image input unit of the recognized image input unit receives an input of the recognized image (S2). The recognized image input unit converts an image constituting the recognized image, that is, a moving image, for each frame and inputs the converted image to the object image recognition engine unit. Similarly, if the image to be recognized is a continuous still image, the image is input for each still image. Note that the following processing is the same for a still image that is continuous with the moving image, so only the moving image will be described. The object image recognition engine reads this frame into the object recognition engine (S3).
The object recognition engine performs object recognition by comparing the default recognition template image with the recognized image.
[0035]
If the features of the template image and the image to be recognized match, the matching part is extracted from the image to be recognized as a recognition image and output to the registration unit of the template creation unit together with the parameters at the time of recognition. The registration unit registers the recognized image as a template image in the template database together with the parameter (S4). As parameters registered at this time, as shown in FIG. 7, the frame position of the recognized image (indicating the number of the frame constituting the moving image), the recognized object position (where the recognized object is located in the frame, XY coordinates), the size of the recognition object (the size of the recognition object, the magnification of the template image, the ratio of the frame to the frame, etc.), the inclination of the recognition image (the rotation angle with respect to the template image), and thumbnail data (recognition) (A reduced image of a frame including an image).
[0036]
This recognition is repeated for each predetermined frame (S5).
When the recognition processing has been completed for all the frames, the processing of steps S2 to S5 is repeated for the next image to be recognized, and when all the images to be processed have been processed, the recognition processing ends (S6).
[0037]
(2). Process for Recognizing an Object Using the Created Template Image FIG. 8 shows the flow of the process.
When the creation of the template image is completed, a recognition process using the template image is started. First, the image input unit of the template image creation unit reads the template image created in the previous step from the template database and inputs the template image to the object image recognition engine unit (S11). At this time, the image conversion unit of the template image creation unit performs normalization based on the parameters (size) of the template image. In addition, the image developing unit performs a rotation process based on a parameter (inclination) of the template image so as to eliminate the inclination.
[0038]
Further, the image input unit of the recognized image input unit receives the input of the recognized image (S12). The image conversion unit appropriately converts each frame constituting the image to be recognized and inputs the converted image to the object image recognition engine unit. The object image recognition engine reads this frame into the object recognition engine (S13).
The object recognition engine performs object recognition by comparing the template image with the frame of the image to be recognized, and generates an image (recognition image) of a portion that matches the template image or a frame including the recognition image at the time of the recognition. The data is output to the output unit together with the parameters (S14). The parameters registered at this time are the same as those in FIG.
This recognition is repeated for each predetermined frame (S15). At this time, recognition may be performed for all frames or every predetermined number of frames, or the frame in which the template image is created in the above-described processing (1) based on the template image parameter (frame position), ie, the default frame The recognition may be performed on a frame that matches the template image.
[0039]
When the recognition processing has been completed for all frames, the processing of steps S12 to S15 is repeated for the next image to be recognized, and when all the images to be processed have been processed, the recognition processing ends (S16).
In the step of performing the object recognition, the first template image and the image to be recognized are compared to search for an object recognition candidate, and the image searched out and the second template image are compared and narrowed down. A narrowing-down search may be performed repeatedly and sequentially as necessary, such as a third template image and a fourth template image.
In the object recognition step, the object recognition may be performed by simultaneously comparing a plurality of template images and the image to be recognized.
[0040]
(3). The step of outputting the recognition image obtained by the object recognition moves or deforms the frame (recognition image) obtained by the object recognition based on the parameters obtained by the object recognition.
[0041]
For example,
Function 1. The position of the recognition object in the frame is determined based on the recognition image position, and the recognition object is moved so as to be displayed at the center.
Function 2. Based on the size of the recognition image, enlargement / reduction is performed to a predetermined size.
Function 3. The inclination of the recognized object is corrected based on the angle of the recognized image. That is, it is rotated and moved around the recognition image position.
FIG. 9 shows an example of moving or deforming by these functions.
FIG. 9A shows an original frame, and a portion surrounded by a square is a recognized image.
FIG. 9B is an example in which functions 1 and 2 are performed. In this way, it is possible to display the recognition object that is small on the edge of the screen in the center of the screen so as to be easily seen.
FIG. 9C shows an example in which functions 1, 2, and 3 are performed. With this function, it is possible to display an image recorded or photographed with a further inclination so that the image can be easily viewed without the inclination.
[0042]
As described above, according to the present embodiment, a template for recognition can be automatically created based on a default image without inputting a template image each time recognition processing is performed. Further, by performing recognition using a template created from the image to be recognized, more detailed recognition can be performed.
[0043]
For example, a default template (person) is compared with a movie or drama image (recognized image), a template image of a person is created from the recognized image, and whether or not the same person is determined using the created template image Is recognized. This makes it possible to extract the appearance scene of a specific person. Also, by recognizing non-identical people, the cast of the movie or drama can be extracted.
[0044]
Further, the image output means is not limited to display, and can be set arbitrarily, such as printing or transmission. Further, the recognition image can be displayed based on the parameters so as to be easily viewed.
[0045]
<< Embodiment 2 >>
FIG. 10 is a schematic configuration diagram of an object image recognition server system as the second embodiment.
As shown in FIG. 1, the image recognition server (image recognition device) is a so-called web server that connects to a user terminal via a network such as the Internet and provides a web page in HTML or the like. The image recognition server includes an image recognition system that performs an object recognition process on an image from the user terminal. Since the image recognition system is substantially the same as that of the first embodiment, the description of the overlapping part will be omitted.
[0046]
The template database of the image recognition system stores images of animals, actresses, singers, and the like, and parameters (comments related to the images) in association with each other.
[0047]
As shown in FIG. 11, when an image of a user's face is input from the user terminal as an image to be recognized to the image recognition server, for example, a template image of an animal is read out, compared with an object recognition engine, and is similar to the image to be recognized. The parameters (comments) of the template are output to the output unit.
[0048]
The output unit transmits the comment to the user terminal and displays the comment.
This makes it possible to provide services such as a personality check and a degree of similarity check with actors and singers.
[0049]
Note that the object recognition method, object recognition system, object recognition program, and object recognition device of the present invention are not limited to the above-described illustrated examples, and various changes can be made without departing from the gist of the present invention. Of course.
[0050]
For example, the same effects as those of the above-described embodiment can be obtained even with the configuration described below.
(Appendix 1)
Enter the image to be recognized,
Reading the recognition template image from the database storing the recognition template image,
Performing object recognition by comparing the recognition template image with the recognized image,
The recognition image obtained by the object recognition as a recognition template image,
A method for recognizing an object in an image, in which the template image is associated with parameters at the time of recognition and registered in the database.
[0051]
(Appendix 2)
2. The object recognition method in an image according to claim 1, wherein when the image to be recognized is composed of a plurality of continuous images, the object recognition is performed at predetermined intervals on the continuous images.
[0052]
(Appendix 3)
Enter the image to be recognized,
Reading the recognition template image from the database storing the recognition template image,
Performing object recognition by comparing the recognition template image with the recognized image,
A method for recognizing an object in an image, wherein the recognition image obtained by the object recognition is moved or deformed based on parameters obtained by the object recognition.
[0053]
(Appendix 4)
The object recognition method in an image according to any one of supplementary notes 1 to 3, wherein the parameter includes at least one of a frame position of the recognition image, a recognition object position, a recognition object size, and a tilt of the recognition object.
[0054]
(Appendix 5)
5. The method for recognizing an object in an image according to claim 1, wherein the recognition image is divided in a vertical or horizontal direction, and the recognition image is converted at a different scale for each of the divided areas.
[0055]
(Appendix 6)
X = ((X−x0) COS θ− (Y−y0) SINθ) / a + x0
y = ((X−x0) SINθ− (Y−y0) COSθ) / b + y0
(However, it is assumed that the image is multiplied by a in the horizontal direction and by b in the vertical direction around an arbitrary point x0, y0 on the image)
6. The method for recognizing an object in an image according to Supplementary Notes 1 to 5, wherein an enlargement process or a reduction process is performed based on the above equation.
[0056]
(Supplementary Note 7) A candidate for object recognition is found by comparing the first template image and the image to be recognized,
7. The method for recognizing an object in an image according to Supplementary Notes 1 to 6, wherein a plurality of images are sequentially compared with the image to be recognized, such as a second template image and a third template image, and a refined search is performed.
[0057]
(Supplementary note 8) The object recognition method in an image according to Supplementary notes 1 to 7, wherein object recognition is performed by simultaneously comparing at least two or more template images and the image to be recognized.
[0058]
(Appendix 9)
A computer-executable program for recognizing an object in a recognized image, wherein the recognition template image is read from a database storing a recognition template image,
Performing object recognition by comparing the recognition template image with the recognized image,
The recognition image obtained by the object recognition as a recognition template image,
A computer-executable program that associates the template image with a parameter at the time of recognition and registers the template image in the database.
[0059]
(Appendix 10)
10. The computer-executable program according to claim 9, wherein when the image to be recognized is composed of a plurality of continuous images, the object recognition is performed at predetermined intervals on the continuous images.
[0060]
(Appendix 11)
A computer-executable program for recognizing an object in a recognized image,
Enter the image to be recognized,
Reading the recognition template image from the database storing the recognition template image,
Performing object recognition by comparing the recognition template image with the recognized image,
A computer-executable program for moving or deforming a recognition image obtained by the object recognition based on parameters obtained by the object recognition.
[0061]
(Appendix 12)
12. The computer-executable program according to any one of supplementary notes 9 to 11, wherein the parameter includes at least one of a frame position of a recognition image, a recognition object position, a recognition object size, and a tilt of the recognition object.
[0062]
(Appendix 13)
An object recognition device that recognizes an object in a recognized image by comparing a recognition template image with the recognized image,
Means for receiving an input of a recognized image;
A database storing recognition template images,
Means for comparing the recognition template image read from the database with the recognized image,
An object recognition apparatus comprising: a recognition image obtained by the object recognition as a recognition template image; and a unit that associates the template image with a parameter at the time of recognition and registers the template image in the database.
[0063]
(Appendix 14)
14. The object recognition device according to claim 13, wherein when the image to be recognized is composed of a plurality of continuous images, the object recognition is performed on the continuous images at predetermined intervals.
[0064]
(Appendix 15)
An object recognition device that recognizes an object in a recognized image by comparing a recognition template image with the recognized image,
Means for receiving an input of a recognized image;
A database storing recognition template images,
Means for comparing the recognition template image read from the database with the recognized image,
Means for moving or deforming the recognition image obtained by the object recognition based on the parameters obtained by the object recognition.
[0065]
(Appendix 16)
The object recognition device according to any one of supplementary notes 13 to 15, wherein the parameter has at least one of a frame position of a recognition image, a recognition object position, a recognition object size, and a tilt of the recognition object.
In the present invention, the above configurations can be combined as much as possible.
【The invention's effect】
According to the present invention, it is possible to provide an object recognition technology that enables appropriate processing of an object image based on parameters of the recognized object image.
[Brief description of the drawings]
FIG. 1 is a diagram showing a system configuration of the present invention. FIG. 2 is a conversion formula of image conversion in an input interface unit. FIG. 3 is a diagram showing a method of transforming a template image. FIG. 4 is a diagram showing a specific example of a transformed template image. FIG. 5 is a diagram showing a method of creating a template image. FIG. 6 is an explanatory diagram of a method of creating a template image. FIG. 7 is a diagram showing parameters at the time of recognition. FIG. FIG. 10 is a diagram showing a movement / modification example of an image. FIG. 10 is a schematic configuration diagram of a second embodiment according to the present invention. FIG. 11 is an explanatory diagram of a system according to a second embodiment.

Claims (16)

Enter the image to be recognized,
Reading the recognition template image from the database storing the recognition template image,
Performing object recognition by comparing the recognition template image with the recognized image,
The recognition image obtained by the object recognition as a recognition template image,
A method for recognizing an object in an image, in which the template image is associated with parameters at the time of recognition and registered in the database. 2. The object recognition method in an image according to claim 1, wherein when the image to be recognized is composed of a plurality of continuous images, the object recognition is performed on the continuous images at predetermined intervals. Enter the image to be recognized,
Reading the recognition template image from the database storing the recognition template image,
Performing object recognition by comparing the recognition template image with the recognized image,
A method for recognizing an object in an image, wherein the recognition image obtained by the object recognition is moved or deformed based on parameters obtained by the object recognition. 4. The method according to claim 1, wherein the parameter includes at least one of a frame position of the recognized image, a recognized object position, a recognized object size, and a tilt of the recognized object. 5. The method according to claim 1, wherein the recognition image is divided in a vertical or horizontal direction, and the recognition image is converted at a different scale for each of the divided areas. X = ((X−x0) COS θ− (Y−y0) SINθ) / a + x0
y = ((X−x0) SINθ− (Y−y0) COSθ) / b + y0
(However, it is assumed that the image is multiplied by a in the horizontal direction and by b in the vertical direction around an arbitrary point x0, y0 on the image)
6. The method for recognizing an object in an image according to claim 1, wherein the enlargement processing or the reduction processing is performed based on the above equation. The first template image is compared with the image to be recognized to find candidates for object recognition,
The method of recognizing an object in an image according to claim 1, wherein a plurality of images are sequentially compared with the image to be recognized, such as a second template image and a third template image, and a refined search is performed. 8. The method for recognizing an object in an image according to claim 1, wherein the object recognition is performed by simultaneously comparing at least two or more template images and the image to be recognized. A computer-executable program for recognizing an object in a recognized image, wherein the recognition template image is read from a database storing a recognition template image,
Performing object recognition by comparing the recognition template image with the recognized image,
The recognition image obtained by the object recognition as a recognition template image,
A computer-executable object recognition program that associates the template image with parameters at the time of recognition and registers the template image in the database. The storage medium according to claim 9, wherein when the image to be recognized is composed of a plurality of continuous images, the object recognition is performed on the continuous images at predetermined intervals. A computer-executable program for recognizing an object in a recognized image,
Enter the image to be recognized,
Reading the recognition template image from the database storing the recognition template image,
Performing object recognition by comparing the recognition template image with the recognized image,
A computer-executable object recognition program for moving or deforming a recognition image obtained by the object recognition based on parameters obtained by the object recognition. The object recognition program according to any one of claims 9 to 11, wherein the parameter includes at least one of a frame position of a recognition image, a recognition object position, a recognition object size, and a tilt of the recognition object. An object recognition device that recognizes an object in a recognized image by comparing a recognition template image with the recognized image,
Means for receiving an input of a recognized image;
A database storing recognition template images,
Means for comparing the recognition template image read from the database with the recognized image,
An object recognition apparatus comprising: a recognition image obtained by the object recognition as a recognition template image; and a means for associating the template image with parameters at the time of recognition and registering the template image in the database. 14. The object recognition device according to claim 13, wherein when the image to be recognized is composed of a plurality of continuous images, the object recognition is performed on the continuous images at predetermined intervals. An object recognition device that recognizes an object in a recognized image by comparing a recognition template image with the recognized image,
Means for receiving an input of a recognized image;
A database storing recognition template images,
Means for comparing the recognition template image read from the database with the recognized image,
Means for moving or deforming the recognition image obtained by the object recognition based on the parameters obtained by the object recognition. 16. The object recognition device according to claim 13, wherein the parameter includes at least one of a frame position of the recognition image, a recognition object position, a recognition object size, and a tilt of the recognition object.

Images