JP3401511B2 - Computer-readable recording medium and image characteristic point extracting apparatus, in which a program for causing a computer to execute the method for extracting characteristic points of an image and the method for extracting the characteristic points of the image are recorded. - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to image processing for processing an image from a camera or the like, and more particularly to the field of image recognition for extracting feature points such as a person's face in an image.

[0002]

2. Description of the Related Art A video conferencing system has been put into practical use in which a plurality of people in a remote place conduct a conference by communication. However, in these systems, there is a problem that the amount of communication data increases when the image itself is sent. for that reason,
For example, a technique has been studied in which feature data relating to the line of sight, face orientation, facial expression, etc. of a target person is extracted at various locations and only the extracted data is transmitted to each other. On the receiving side, an image of the face of a virtual person is generated and displayed based on this data. As a result, the video conference can be efficiently performed while reducing the communication data amount.

Further, for example, in an educational system using broadcasting, it is desirable that the lecturer proceed with the lecture while watching the reactions of the students who are in various places. In this case, too, the amount of communication data increases when the images from various places are sent to the place where the lecturer is giving a lecture. In the first place, it is impractical to send the video of all the students when the number of students is large, and somehow the reaction of each student is extracted in each region, and only the information showing the reaction is sent to the instructor, and the instructor is sent. On the other hand, it is preferable to present it in the form of an abstract “reaction of the set of students”.

In order to realize such processing, it is necessary to recognize the facial expression, posture, line-of-sight direction and the like from the face image of the person. For that purpose, it is necessary to specify the position of the face and detect face parts such as eyes, nose, and mouth, in particular, the positions of the eyes, where the facial expression of the person is remarkably changed.

At present, as a technique for identifying and tracking the position of the entire face of a person from a video, a method of detecting and tracking a skin color using color information of the video has been proposed.
Further, as a simpler method, there is a method of detecting a face area by a difference between frames of a video, assuming that the background of the video has little movement and only a person moves.

[0006] As a technique for detecting the eyes after the approximate position of the entire face is detected in this way, a technique of using the matching of the light and dark distribution of the image in the face region with a template prepared in advance, the face region It has been proposed to find the position of the facial part by projecting the image in the vertical and horizontal directions.

[0007]

However, in the case of using template matching, it is necessary to prepare a large number of templates in order to perform accurate detection. Therefore, not only a large storage capacity is required, but also the processing time for matching is long, which is not suitable for real-time processing. Further, in template matching, for example, in order to accurately detect a face part when the face is tilted, that is, when the face is slightly rotated around the horizontal axis, it is necessary to prepare a template corresponding to the rotation. This will increase the number of templates and slow down the processing time, because different templates need to be prepared for different rotation angles. Such a problem similarly occurs in the detection of the position of the face part of the face that faces a little sideways. In addition, in order to deal with detection when the overall brightness of the image changes, it is necessary to prepare a new template for that.

Further, such a problem is not unique to the detection of the position of the face part, and is associated with the recognition of the type of various industrial products for the management at the manufacturing and distribution stage from the appearance image and the moving object. The same can occur in the recognition of the specific mark that is created.

Therefore, an object of the present invention is to realize an image feature point extracting apparatus which can obtain the feature information necessary for image recognition without being largely influenced by the tendency of the entire image, a method therefor, and a method using the computer. It is to provide a recording medium in which a program for performing the above is recorded.

Another object of the present invention is to provide an image feature point extraction apparatus capable of detecting the positions of face parts necessary for facial expression recognition without being significantly affected by the tendency of the entire image and the rotation of the image, a method therefor, and a method therefor. It is to provide a recording medium recording a program for realizing the method using a computer.

Still another object of the present invention is to provide an image feature point extraction device capable of extracting information for identifying an eye from a face image without being significantly affected by the tendency of the entire image and the rotation of the image. And a recording medium on which a program for implementing the method using a computer is recorded.

[0012]

According to a first aspect of the present invention, there is provided a method of extracting a feature point of an image which is a human face area.
Te comprising: providing a digital data value of each pixel in the image region of interest, for each pixel in the image region of interest, the value of the pixels on the predetermined closed curve surrounding the pixel The step of obtaining the relative amount of the specific wave number component indicated by the distribution curve and the specific wave number within the face area
The pixel in which the relative amount of the component locally shows the maximum is set in the face area.
And a step of extracting the feature points as points representing the eyebrows .

Since the characteristic points are extracted according to the relative amount of the specific wave number component indicated by the distribution curve of the pixel values on the closed curve, the characteristic points are extracted regardless of the tendency of the overall pixel value. it can.

According to a second aspect of the present invention, in addition to the structure of the first aspect of the present invention, the closed curve has a pixel at the center, and the closed curve corresponds to the size of the image area. A circle with a defined radius.

The wave number component of the distribution of pixel values on the circumference is a rotation invariant quantity. Therefore, in addition to the effect of the invention described in claim 1, the feature points can be extracted even if the target image is rotated.

According to a third aspect of the present invention, in addition to the configuration of the first or second aspect of the present invention, the method of extracting feature points of an image further comprises the step of obtaining a relative amount of a specific wave number component. Is a step of obtaining a relative amount of a specific wave number component by performing a discrete Fourier transform for each of the pixels in the target image area, using the value of each pixel on the closed curve surrounding the pixel as a sampling value. including.

The relative amount of wave number components can be obtained by performing an efficient operation called discrete Fourier transform. Therefore, in addition to the effect of the invention described in claim 1 or claim 2, feature points can be extracted quickly and efficiently.

According to a fourth aspect of the present invention, there is provided a method of extracting a feature point of an image, wherein the specific wave number component is a component having a wave number of 2 in addition to the configuration of the first aspect of the invention. Is.

Around a certain characteristic point, for example, in terms of lightness, there are two repetitions of "bright and dark" such as "bright, dark, bright, dark".
When there is a distribution of pixel values that appear repeatedly, the relative amount of the wave number 2 component increases, and the feature point surely exhibits the local maximum value in comparison with the surroundings. Therefore, in addition to the effect of the invention described in any one of claims 1 to 3, it is possible to reliably extract a feature point having a distribution of pixel values having two repetitions in the surroundings.

According to a fifth aspect of the present invention, there is provided a method of extracting a feature point of an image, wherein in addition to the configuration of the fourth aspect of the invention, a plurality of feature points extracted by the step of extracting the feature point Among the candidates of the above, only those in which the ratio of the spectral power of the component of wave number 1 to the spectral power of the component of wave number 2 of the distribution curve of the value of each pixel on the closed curve surrounding the candidate point is smaller than a predetermined threshold value. Is selected as a feature point.

For example, in terms of brightness, "bright, dark, bright, dark"
When there is a distribution of pixel values such that the repetition of “bright and dark” appears twice, the closer the distribution of pixel values on that circumference is to a sine curve, the closer The relative amount of the wave number 1 component becomes small, and the relative amount of the wave number 2 component becomes large. Therefore, the smaller the ratio is than the predetermined threshold value, the higher the possibility that the point is the feature point having the above-mentioned features. Therefore, in addition to the effect of the invention described in claim 4, it is possible to more reliably extract a feature point having a distribution of pixel values having two repetitions in the surroundings.

According to a sixth aspect of the present invention, there is provided a method of extracting feature points of an image, in addition to the configuration of the first aspect of the invention.
Te, the value of each pixel is Ru lightness der of each pixel.

In the human face area , the space between the eyebrows is bright in the surroundings,
Dark iteration Ru predetermined number appears. Therefore, in addition to the effect of the invention described in claim 1, it is a feature point of the human face.
The eyebrow can be extracted based on the wave number component of the distribution of the brightness values of the pixels around it.

A computer-readable recording medium according to a seventh aspect of the present invention is a computer-readable recording medium in which a program for causing a computer to execute a method of extracting a feature point in a target image area is recorded. The program is a recording medium, and the program includes a program portion that performs processing for preparing digital data of the value of each pixel in the target image area that is a human face area, and each of the pixels in the target image area. , The program part that performs the process of obtaining the relative amount of the specific wave number component indicated by the distribution curve of the pixel values on the predetermined closed curve surrounding the pixel , and the specific wave number component within the face area. The relative amount of is local
The pixel that shows the maximum is the point that represents the space between the eyebrows that is the feature point in the face area.
And a program part for performing a process of extracting as .

Since the feature points are extracted according to the relative amount of the specific wave number component indicated by the distribution curve of the pixel values on the closed curve, the feature points are extracted regardless of the tendency of the overall pixel value. it can.

In the computer-readable recording medium according to the invention described in claim 8, in addition to the configuration of the invention described in claim 7, the closed curve is determined according to the size of the image area with the pixel at the center. It is a circle with a radius.

The wave number component of the distribution of pixel values on the circumference is a rotation invariant quantity. Therefore, in addition to the effect of the invention described in claim 7, the feature points can be extracted even when the target image is rotated.

According to a ninth aspect of the present invention, there is provided a computer-readable recording medium, in addition to the configuration of the seventh or eighth aspect of the invention, a process for obtaining a relative amount of a specific wave number component. The program part to perform is the relative amount of a specific wave number component by performing a discrete Fourier transform for each pixel in the target image area by using the value of each pixel on the closed curve surrounding the pixel as a sampling value. Includes a program part for performing the process for obtaining

The relative amount of wave number components can be obtained by performing an efficient operation called discrete Fourier transform. Therefore, in addition to the effect of the invention described in claim 7 or claim 8, feature points can be extracted quickly and efficiently.

According to a tenth aspect of the present invention, there is provided a computer-readable recording medium, wherein the specific wave number component is a component having a wave number of 2 in addition to the configuration of the invention according to any of the seventh to ninth aspects. is there.

Around a certain feature point, for example, in terms of lightness, there are two repetitions of "bright and dark" such as "bright, dark, bright, dark".
When there is a distribution of pixel values that appear repeatedly, the relative amount of the wave number 2 component increases, and the feature point surely exhibits the local maximum value in comparison with the surroundings. Therefore, in addition to the effect of the invention described in any one of claims 7 to 9, it is possible to reliably extract feature points having a distribution of pixel values having two repetitions in the surroundings.

The program of a computer-readable recording medium according to an eleventh aspect of the present invention is extracted by a program portion that performs a process of extracting a feature point in addition to the configuration of the tenth aspect of the invention. Of the plurality of feature point candidates, the ratio of the spectral power of the wave number 1 component and the spectral power of the wave number 2 component of the distribution curve of the values of each pixel on the closed curve surrounding the candidate point is a predetermined threshold. It includes a program part for performing a process of selecting only those smaller than the value as feature points.

For example, in terms of brightness, "bright, dark, bright, dark"
When there is a distribution of pixel values such that the repetition of “bright and dark” appears twice, the closer the distribution of pixel values on that circumference is to a sine curve, the closer The relative amount of the wave number 1 component becomes small, and the relative amount of the wave number 2 component becomes large. Therefore, the smaller the ratio is than the predetermined threshold value, the higher the possibility that the point is the feature point having the above-mentioned features. Therefore, in addition to the effect of the invention described in the tenth aspect, it is possible to more reliably extract the feature points having the distribution of the values of the pixels having the repetition twice around.

[0034] Such computer-readable recording medium in the invention of claim 12, in addition to the structure of the invention according to claim 7, the value of each pixel is Ru lightness der of each pixel.

In the human face area , the space between the eyebrows is bright in the surroundings,
Dark iteration Ru predetermined number appears. Therefore, in addition to the effect of the invention described in claim 7, it is a feature point of the human face.
The eyebrow can be extracted based on the wave number component of the distribution of the brightness values of the pixels around it.

The image feature point extraction device according to the thirteenth aspect of the present invention is a device for preparing digital data of the value of each pixel in the target image region which is the human face region, and the target and For each of the pixels in the image area, the means for obtaining the relative amount of the specific wave number component indicated by the distribution curve of the values on the predetermined closed curve surrounding the pixel, and in the face area, The relative amount of a particular wavenumber component
The pixel showing the maximum locally is displayed as the feature point in the face area between the eyebrows.
Means for extracting the points .

Since the characteristic points are extracted according to the relative amount of the specific wave number component indicated by the distribution curve of the pixel values on the closed curve, the characteristic points are extracted regardless of the tendency of the overall pixel value. it can.

According to the image feature point extracting apparatus of the fourteenth aspect of the invention, in addition to the configuration of the thirteenth aspect of the invention, the closed curve is centered on the pixel and is determined according to the size of the image area. It is a circle with a radius.

The wave number component of the pixel value distribution on the circumference is a rotation invariant quantity. Therefore, in addition to the effect of the invention described in claim 13, the feature points can be extracted even when the target image is rotated.

The image feature point extracting device according to the invention of claim 15 is, in addition to the configuration of the invention of claim 13 or 14, a means for obtaining a relative amount of a specific wave number component. Is to obtain the relative amount of a specific wave number component by performing a discrete Fourier transform for each pixel in the target image area using the value of each pixel on the closed curve surrounding the pixel as a sampling value. Including means.

The relative amount of wave number components can be obtained by performing an efficient operation called discrete Fourier transform. Therefore, in addition to the effect of the invention described in claim 13 or 14, feature points can be extracted quickly and efficiently.

In the image feature point extraction device according to the invention of claim 16, in addition to the configuration of the invention of any one of claims 13 to 15, the specific wave number component is a component of wave number 2. .

For example, in terms of brightness, "bright, dark, bright, dark"
When there is a distribution of pixel values such that the repetition of “bright and dark” appears twice, the closer the distribution of pixel values on that circumference is to a sine curve, the closer The relative amount of the wave number 1 component becomes small, and the relative amount of the wave number 2 component becomes large. Therefore, the smaller the ratio is than the predetermined threshold value, the higher the possibility that the point is the feature point having the above-mentioned features. Therefore, in addition to the effects of the invention described in claims 13 to 15, it is possible to more reliably extract a feature point having a distribution of pixel values having two repetitions in the surroundings.

An image feature point extracting device according to the invention of claim 17 is the image feature point extracting device according to claim 16, further comprising a plurality of feature points extracted by means for extracting feature points. Among the candidates of the above, only those in which the ratio of the spectral power of the component of wave number 1 to the spectral power of the component of wave number 2 of the distribution curve of the value of each pixel on the closed curve surrounding the candidate point is smaller than a predetermined threshold Means for selecting as a feature point.

For example, in terms of brightness, "bright, dark, bright, dark"
When there is a distribution of pixel values such that the repetition of “bright and dark” appears twice, the closer the distribution of pixel values on that circumference is to a sine curve, the closer The relative amount of the wave number 1 component becomes small, and the relative amount of the wave number 2 component becomes large. Therefore, the smaller the ratio is than the predetermined threshold value, the higher the possibility that the point is the feature point having the above-mentioned features. Therefore, in addition to the effect of the invention described in claim 16, it is possible to more reliably extract a feature point having a distribution of pixel values having two repetitions in the periphery.

The image feature point extraction device according to the invention of claim 18, in addition to the configuration of the invention according to claim 1 3, the value of each pixel Ru lightness der of each pixel.

In the human face area , the space between the eyebrows is bright in the surroundings,
Dark iteration Ru predetermined number appears. Accordingly, claims 1 to 3
In addition to the effect, the feature point der human face in the embodiment described in
The eyebrow can be extracted based on the wave number component of the distribution of the brightness values of the surrounding pixels.

[0048]

BEST MODE FOR CARRYING OUT THE INVENTION [Hardware Configuration] An image feature point extracting apparatus according to a first embodiment of the present invention will be described below. This image feature point extraction device is realized by software executed on a computer such as a personal computer or a workstation, and is for detecting the position of eyes from the image of a person's face. FIG. 1 shows the appearance of this image feature point extraction device.

Referring to FIG. 1, this system 20 includes a CD
A computer main body 40 having a ROM (Compact Disc Read-Only Memory) drive 50 and an FD (Flexible Disk) drive 52; a display 42 as a display device connected to the computer main body 40; A keyboard 46 and a mouse 48 as input devices, and a computer main body 4
Camera 30 for capturing images, which is connected to 0. In the apparatus of this embodiment, the camera 30 is C
It is assumed that a video camera including a CD (solid-state image sensor) is used to perform processing for detecting the position of the eyes of a person who operates the system 20 in front of the camera 30.

FIG. 2 shows the configuration of the system 20 in a block diagram form. This system 2 as shown in FIG.
In addition to the CD-ROM drive 50 and the FD drive 52, the computer main body 40 constituting the CPU 0 has a CPU (Central Processing Uni) connected to the bus 66.
t) 56, ROM (Read Only Memory) 58, RAM
(Random Access Memory) 60 and hard disk 54
And an image capture device 68 for capturing an image from the camera 30. CD-ROM drive 50 has C
The D-ROM 62 is mounted. F for FD drive 52
D64 is installed.

As described above, the main part of the image feature point extracting apparatus is the computer hardware and the CPU 56.
It is realized by software executed by. Generally, such software is CD-ROM 62, FD
It is stored in a storage medium such as 64 and distributed, is read from the storage medium by the CD-ROM drive 50 or the FD drive 52, and is temporarily stored in the hard disk 54. Alternatively, when the device is connected to the network, the hard disk 5 is temporarily sent from the server on the network.
4 is copied. Then further hard disk 54
Is read from the RAM 60 to the RAM 60 and executed by the CPU 56. When connected to the network, it may be directly loaded into the RAM 60 and executed without being stored in the hard disk 54.

The hardware itself and the operating principle of the computer shown in FIGS. 1 and 2 are general. Therefore, the most essential part of the present invention is the software stored in the storage medium such as the FD drive 52, the FD 64, and the hard disk 54.

As a recent general tendency, various program modules are prepared as a part of the operating system of a computer, and the application program calls these modules in a predetermined arrangement when necessary and advances the processing. It is common. In such a case, such a module is not included in the software itself for realizing the image feature point extraction device, and the image feature point extraction device is realized only when the computer cooperates with the operating system. However, as long as a general platform is used, it is not necessary to distribute software that includes such modules, and the software itself that does not include those modules and a recording medium that records these software (and those software are distributed on the network). The data signal in the case) can be considered to constitute an embodiment. [Basic Principle] With reference to FIG. 3, in the device of the present embodiment, in the face of a person, between the eyebrows located between the eyes (in the following description, the center of a line segment connecting the centers of both eyes). The position of is called "between the eyebrows".) Below the eyebrows, "B
EP "(Between-Eyes-Point).

As shown in FIG. 3A, a circle with a certain radius is drawn around the eyebrows in the face image of a person, and the brightness of each pixel along the circumference is checked. Then, the result is as schematically shown in FIG. 3 (b). In FIG. 3B, the horizontal axis represents the position of each pixel in the circumferential direction, and the vertical axis represents the brightness of each pixel. It should be noted that the uppermost part of the circumference shown in FIG. 3A is the origin of the horizontal axis in FIG. 3B, and the circumference of FIG. In the graph of b), each pixel is arranged on the horizontal axis.

Referring to FIG. 3B, this graph is
It can be seen that the “vertices and valleys” are repeated twice, that is, the apex, the valley, the apex, and the valley. This meaning is shown in Figure 3.
It is obvious with reference to the face image shown in (a). In other words, in a human face image, if the human eye image is traced on the above-mentioned circumference centering on the eyebrow, first the forehead (high brightness), then the right eye (low brightness), then the nose (high brightness), then the A bright part and a dark part are alternately repeated twice, such as the left eye (low brightness) and finally the first frame (high brightness). In the face image, these features are most shown in the eyebrows, and other parts do not show such features much, and even if they are, they are smaller than the eyebrows.

Therefore, in the apparatus according to the present embodiment, it is assumed that such a lightness distribution exists around the eyebrows, and filtering is first performed by a filter called "ring DFT (discrete Fourier transform) filter" below. The BEP is detected, and thereafter, the positions of the eyes on both sides of the brow are detected based on the BEP. The first embodiment
The ring DFT transform used in the above will be described later.

The apparatus according to the present embodiment detects and tracks the eye position using software having the following control structure.

Referring to FIG. 4, an image is first acquired (step 80). Here, one frame image obtained from the camera 30 shown in FIG. 1 and FIG.
It is assumed that the image is digitally converted at 8 and stored in the image memory in the image capturing device 68, and the following processing is performed on this image. In the case of continuous processing, the following processing is repeated for each frame of the image obtained from the camera 30.

That is, at step 82, candidate points between the eyebrows are extracted from the image data for one frame by using the ring DFT filter described above. This processing will be described later with reference to FIG.

Next, it is determined whether the operation mode of the apparatus is the search mode or the tracking mode (step 84). The search mode is a mode for first determining the eye position as a premise of the processing for tracking the eye position. The tracking mode is an operation mode for always grasping the eye position by following changes in the eye position by processing only the vicinity of the eye position once determined in this way. At the beginning of this process, the eye position has not been determined yet, so the operation mode is set to the search mode. Therefore, hereinafter, the processing in the search mode will be described first, and then the processing in the tracking mode will be described.

In the search mode, among the candidate points (generally a plurality) between the eyebrows extracted in step 82,
Look for one that satisfies the condition that there are two dark areas (corresponding to the eyes) at symmetrical positions on both sides. Among the eyebrow candidate points, those that do not satisfy this condition are not adopted here.

In step 88, it is determined whether or not the eyes obtained as a result of the processing in step 86 are only one pair (that is, the eyebrows are only 1).
It is determined whether or not only points are detected. When only one pair has been obtained, the process for changing the operation mode thereafter to the tracking mode is performed (step 90), and the region near the eyes is saved as a template (step 98).
Then, the position of the determined eyebrow (or eye) is output (step 100), and the process returns to step 80. After that, the above processing is repeated for new frame data. However, the operation mode thereafter is the tracking mode.

On the other hand, if more than one pair of eyes are detected in the process of step 88, the process returns to step 80, and the above process is repeated for a new frame. However, in this case, the operation mode is still the search mode.

Once the processing is in the tracking mode,
As a result of the determination in step 84, the control proceeds to step 92.
In the process of step 82 preceding this, only the vicinity of the BEP obtained immediately before that is targeted for processing, whereby both the extraction process of candidate points and the subsequent processes can be speeded up. In step 92, the template (BEP in the previous frame) stored in step 98 is matched with all the BEP candidate points in the next frame. And the one that gave the best match is the only BEP
Will be the candidate point.

In step 94, it is judged whether or not the difference between the BEP candidate point that gave the best match in step 92 and the template is equal to or more than a predetermined threshold value. Even if the best match is given, it is often inappropriate to determine the candidate point as the BEP when the difference from the BEP in the immediately preceding frame is too large. The process in this step is a process for eliminating such an inappropriate BEP.

If it is determined in step 94 that the difference exceeds the predetermined threshold value (or is greater than or equal to the predetermined threshold value), the operation mode is set to the search mode in step 96, and the process proceeds to step 86. The subsequent processing is as already described for the search mode.

If it is determined in step 94 that the difference is less than or equal to the predetermined threshold value (or less than the predetermined threshold value), this candidate point is the BEP continuously tracked from the previous frame. Then, a new template corresponding to this BEP is saved (step 9).
8). Then, this BEP or the positions of the eyes on both sides thereof are output (step 100), and the control returns to step 80. The subsequent processing is as already described. [Extraction of Candidate Points Using Ring DFT Filter] In the extraction of the eyebrow candidate points performed in step 82, the ring DFT filter described above is used. Hereinafter, the process of step 82 will be described with reference to FIG.

First, at step 110, the image to be processed as shown in FIG. 6A is subjected to smoothing and vertical / horizontal ½ reduction processing, and the image as shown in FIG. 6B is obtained. To get In the experiment, for example, the brightness of 5 × 5 pixels around the target pixel was averaged to obtain the brightness of the pixel, and the image was reduced at the same time by selecting the target point. The smoothing is for removing noise (relatively high frequency components are included) included in the image. In particular,
In the detection of human BEP, since the spectrum power component of wave number 2 is calculated as described later, there is no possibility that this smoothing will delete information necessary for the subsequent processing. Further, by reducing the number of pixels to 1/4 in this process, it is possible to speed up the process. However, it may not be necessary to reduce the number of pixels if a sufficiently fast processor is used. Also, if a slower processor is used, it may be necessary to reduce (reduce the number of pixels) to a smaller image. However, if the image is reduced too much, the resolution will decrease, and as a result, the accuracy of BEP detection may decrease, so it is also useful to select an appropriate resolution by experiments.

Subsequently, a process of estimating the head region of the target person from the image thus obtained is performed (step 112). For this processing, color information is used as described above, an algorithm for tracking the skin color area is used, or the area that seems to have moved between two frames is extracted from the difference between the previous frame and the current frame, and this is extracted. It is also possible to use an algorithm for estimating a partial area. In the present embodiment, the difference between frames is used. Moreover, the region estimated here may be a region of any shape,
Considering the simplicity of area calculation, a rectangular area is suitable. However, depending on the conditions, it may be more efficient to use another shape. In addition, when the head is hardly moved, the inter-frame difference cannot be obtained. In that case, the head region estimated in the immediately preceding process is used assuming that the head has not moved.

Next, filtering using a ring DFT filter is performed within the range of the head region thus obtained. Specifically, for example, the following calculation is performed in order from a pixel on the upper left of the head region, with respect to a pixel on the circumference of a circle having a predetermined size and having the pixel at the center as a center. .

[0071]

[Equation 1]

In this equation, N is the number of points on the circumference, and k is the highest point on the circumference ("North Pole").
The position is a number corresponding to each point in a counterclockwise direction, where 0 is the point corresponding to. Also, fk (k = 0, ..., N−
1) is the lightness of the k-th pixel on the circumference, and i is the imaginary unit. This formula (1) is for the case where n = 2 of the DFT coefficient obtained by the following general discrete Fourier transform.

[0073]

[Equation 2]

By the conversion shown in the equation (1), the spectrum power component of the wave number 2 included in the above-mentioned fluctuation waveform of the lightness on the circumference (see FIG. 3B) is calculated. In the present embodiment, the radius of the circle is 7 pixels and N = 36. Since the size of the face area changes according to the distance between the person and the camera, if it is considered that there is a large amount of change in such distance, the face area size should be adjusted to the approximate size of the face area already obtained. The accuracy is improved by changing the radius of the circle. However, the radius may be fixed in advance if it is known that the person hardly makes such a movement.

By this calculation, the value of the spectral power of wave number 2 on the circumference of the center of the pixel is calculated for all the pixels in the head region. An example of a new image in which the brightness according to the value is assigned to each pixel is shown in FIG.
It shows in (b). As can be seen by referring to FIG.
In the distribution of the values obtained as a result of performing the above-mentioned calculation for each pixel, there is a particularly high value portion. It is considered that those portions have many wave number components of wave number 2 as described above on the circumference of the circumference. Therefore, they qualify as candidate points for BEP. In this way, a closed curve, which is typically a circle, centered on each target point on the image is traced, and the pixel value (including not only the lightness but also the hue, saturation, etc.) can be calculated on the closed curve. Obtaining information as shown in FIG. 8B by performing DFT on the other hand is called “filtering by a ring DFT filter” in the present invention.

From the distribution of values thus obtained as shown in FIG. 8B, the point showing the local maximum value is selected and B
It is set as a candidate point for EP (step 116). An example of the detected candidate points is shown in FIG. In this example, 27 candidate points are detected. The important thing to note here is this 2
That is, the true BEP is included in the seven. As described above, there is almost no difference around the true BEP, and there is a distribution of regions of bright, dark, bright, and dark. Therefore, as a result of the processing in step 114, it is almost certain that the true B
EP indicates a local maximum and, as a result, step 116.
Is extracted as a candidate point almost without exception. As described above, it is a characteristic of this method that the true BEP is almost certainly extracted. It should be noted that the threshold value for selection is empirically determined mainly according to the characteristics that the target image should have.

Then, in step 118, among the plurality of local maximum values, the local feature characteristic of BEP is considered,
A process of narrowing down BEP candidates is performed.

For example, in the actual BEP, in addition (amount)
There should be a bright area at the bottom and the bottom (nose), and dark areas at the left and right (both eyes). Therefore, the real part of the calculation result of equation (1) must be positive. Those that produce a non-positive real part are not BEPs and are excluded from the candidates.

For the same reason, considering an image obtained by projecting an image centered on the true BEP in the vertical and horizontal directions, the following can be said. Referring to FIG. 9, the distribution of light and dark is darkest in the center in the vertical direction and brightest in the center in the horizontal direction as shown in (a). Also, its distribution should be almost symmetrical about the center. Therefore, when there are a plurality of candidate points, the same projections in the vertical direction and the horizontal direction are created, and the ones that do not meet the above conditions are not adopted.

As another criterion, the center of gravity of the brightness of a small area centered on the BEP candidate point is calculated, and the center of gravity and BE are calculated.
If the distance to the P candidate point exceeds the threshold value, the BEP
Exclude candidate points.

Further, the candidate points can be narrowed down by using the following features of the ring DFT filter. That is, in the above general formula of Fn (formula (2)), n =
F1 is calculated for each pixel as 1. Then, the ratio (F1 / F2) with F2 obtained for each pixel is calculated, and the standard that the smaller this value is, the higher the probability of true BEP is used. It is considered that this value can be used as a reference to show how much the distribution of light and darkness on a circle centered on a pixel matches the ideal limit curve (how far they are separated) for the following reasons. .

The values calculated as n = 1, 2, ... In equation (2) indicate the spectral powers of the wave number components whose wave numbers are 1, 2 ,. If the distribution of light and darkness on the circumference ideally coincides with the sine curve when n = 2, Fn = 0 should be obtained for n other than 2. Of course, the distribution of light and darkness does not actually coincide with the sine curve, but if it is close to an ideal sine curve, F1 will have a small value and F2 will have a relatively large value. Therefore, it can be considered that if F1 / F2 is small, the light-dark distribution around the target pixel is close to the actual light-dark distribution around BEP, and if it is large, it is far away. This is why F1 / F2 can be used as a measure.

It should be noted that F3, F4, etc. should be 0 in the ideal distribution of light and dark like F1. Therefore, it is also possible to use F3 / F2, F4 / F2, or the like as a reference. However, they show a higher amount of wavenumber components and are therefore more susceptible to noise, so the results are less reliable than with F1 / F2.

Thereafter, steps 86 and 88 in FIG. 4 test whether this BEP is a true BEP or not. The result is shown in FIG. In this example, a plurality of BEP candidates remained (marked with “*”), but among them, regions that are considered to be eyes (marked with “+”) exist symmetrically on both sides in the center. "X" mark of the true BEP
Only. Thus, the true BEP is determined.

As described above, in the system of this embodiment, BEP detection is performed using the ring DFT filter. The ring DFT filter extracts feature points such as BEP from only wave number components existing in the light and dark distribution of the image. Therefore, there is a feature that it is less likely to be affected by the fluctuation of the overall brightness of the image. Further, even when the face is slightly tilted, the wave number component in the brightness distribution around a certain point is invariant to the rotation of the image. Therefore, using the above-described method, it is possible to realize the extraction of the feature points that are resistant to rotation. This is also the case when the face is turned slightly sideways. Even if the eyes are turned sideways so that they are barely visible, as long as both eyes are present in the image, the light-dark arrangement described above still exists around the eyebrows, so it is explained above. BEP can be almost certainly extracted using the method. Further, even if the target person has his eyes closed, the area is still darker than the forehead and nose, so that the BEP can be detected almost certainly by the above method. Therefore, it is possible to detect the BEP with high reliability and further the positions of the eyes on both sides of the BEP.

Furthermore, it will be apparent to those skilled in the art that the above-described feature point extraction method and apparatus are not limited to use only for facial feature point extraction. As described above, the ring DFT filter described above extracts the wave number components of the distribution of light and darkness on a circle centered on a certain pixel, and extracts the points that match the particular distribution of light and darkness as feature points. Therefore, for example, in the case of automatically recognizing and managing moving objects (products, automobiles, building materials) and the like, in order to recognize the posture, specific marks are previously attached to specific positions of those objects,
By performing the above-described feature point extraction, it is possible to recognize the positions and orientations of those objects. As a mark used at this time, a mark having a specific light and dark (or color) distribution around a certain point may be used. Further, the wave number component to be detected at this time is not limited to 2, and the wave number component corresponding to the number of repetitions of light and dark existing in the shape (pattern) of the mark may be detected.

Furthermore, in the above example, the DFT coefficient was calculated for points on the circumference centered on each pixel. However, the invention is not only applicable to points on the circumference. If it is a closed curve that has a predetermined positional relationship with the points to be extracted as feature points in advance, and if the distribution of light and dark on its circumference is known, use the above calculation on any other closed curve. You can go.
However, since it is the case of a circle that gives a robust result to rotation, the circle will often be optimal.

Furthermore, in the above-described embodiment, the wave number component in the light-dark distribution on one circumference centered on each pixel is used, but the number of circles used is not limited to one. It will also be apparent to those skilled in the art that none. For example, if it is known in advance that there should be different light and dark distributions around the feature point to be extracted in the regions at different distances from the center, then a plurality of circles (or closed curves) can be generated accordingly. The above-mentioned calculations may be performed respectively, and the feature points may be extracted by integrating the results of both calculations.

In the above example, the DFT is used to calculate the wave number component. Although it is considered that the use of DFT is the most efficient, the function required in the above example only needs to be able to extract the wave number component in the light-dark distribution on the circumference. Therefore, it will also be apparent to those skilled in the art that the technique that can be used is not limited to DFT, and that any function for extracting a wave number component can be used, including a general Fourier transform. .

Further, in the above-mentioned embodiment, the processing target of the ring DFT filter is the brightness of the pixel. However, the applicable object of the present invention is not limited to this. For example, a ring DFT is applied to values such as hue and saturation of each pixel.
You may perform filtering by a filter. Also,
Depending on the property that the characteristic point to be detected should have, it is possible to use a value obtained by performing a predetermined operation on the value of each pixel such as lightness, hue, and saturation as a filtering target.

The embodiments disclosed this time are to be considered as illustrative in all points and not restrictive. The scope of the present invention is shown not by the above description but by the claims, and is intended to include meanings equivalent to the claims and all modifications within the scope.

[Brief description of drawings]

FIG. 1 is an external view of a system according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a hardware configuration of a system according to an embodiment of the present invention.

FIG. 3 is a diagram for explaining the principle of the present invention.

FIG. 4 is a flowchart of processing executed by the system according to the first embodiment of the present invention.

FIG. 5 is a flowchart of a process of extracting a candidate point between eyebrows from image data.

FIG. 6 is a diagram showing images before and after smoothing / reduction.

FIG. 7 is a diagram showing a calculation path of a ring DFT filter.

FIG. 8 is a diagram showing a result of extraction processing of a candidate point between eyebrows by a ring DFT filter.

FIG. 9 is a schematic diagram for explaining local features between the eyebrows.

FIG. 10 is a diagram showing a result of a test between the eyebrows.

[Explanation of symbols]

20 Image feature point extraction device 30 cameras 40 computer body 42 monitors

Front page continuation (58) Fields surveyed (Int.Cl. ⁷ , DB name) G06T 7/40 G06T 7/00 300 JISST file (JOIS)

Claims (18)

(57) [Claims] 1. A step of preparing digital data of a value of each pixel in a target image area which is a human face area, and for each of the pixels in the target image area,
Determining a relative amount of a specific wave number component indicated by a distribution curve of the value of the pixel on a predetermined closed curve surrounding the pixel, and a relative amount of the specific wave number component within the face region. Is a station
The pixel showing the maximum locally is the feature point in the face area between the eyebrows
A method for extracting feature points of an image, the method including: 2. The method of extracting feature points of an image according to claim 1, wherein the closed curve is a circle centered on the pixel and having a radius determined according to a size of the image region. 3. The step of obtaining the relative amount of the specific wave number component, for each of the pixels in the image area of interest, sets the value of each of the pixels on the closed curve surrounding the pixel. The method for extracting feature points of an image according to claim 1 or 2, further comprising the step of determining a relative amount of the specific wave number component by performing a discrete Fourier transform as a sampling value. 4. The method of extracting feature points of an image according to claim 1, wherein the specific wave number component is a wave number 2 component. 5. A component having a wave number of 1 in a distribution curve of the value of each pixel on the closed curve surrounding the candidate points among a plurality of candidate feature points extracted by the step of extracting the characteristic points. 5. The method for extracting a feature point of an image according to claim 4, further comprising the step of selecting only a feature point having a ratio between the spectrum power of 1) and the spectrum power of the component of wave number 2 smaller than a predetermined threshold value as the feature point. 6. Before SL value of each pixel brightness der of each pixel
That, method of extracting feature points of the image according to 請 Motomeko 1. 7. A computer-readable recording medium having recorded thereon a program for causing a computer to execute a method of extracting a feature point in a target image area, the program being a human face area. There is a program portion that performs a process of preparing digital data of the value of each pixel in the target image area, and for each of the pixels in the target image area,
A program part for performing a process of obtaining a relative amount of a specific wave number component indicated by a distribution curve of the value of the pixel on a predetermined closed curve surrounding the pixel , Relative quantity is bureau
The pixel showing the maximum locally is the feature point in the face area between the eyebrows
A computer-readable recording medium that includes a program portion that performs a process of extracting as a point that represents . 8. The computer-readable recording medium according to claim 7, wherein the closed curve is a circle centered on the pixel and having a radius determined according to a size of the image area. 9. A program portion for performing a process of obtaining a relative amount of the specific wave number component, for each of the pixels in the target image area, the program on the closed curve surrounding the pixel. 9. The computer-readable recording medium according to claim 7, further comprising a program portion for performing a process of obtaining a relative amount of the specific wave number component by performing a discrete Fourier transform using a pixel value as a sampling value. . 10. The computer-readable recording medium according to claim 7, wherein the specific wave number component is a wave number 2 component. 11. The program further includes a distribution of the values of the respective pixels on the closed curve surrounding the candidate points among a plurality of characteristic point candidates extracted by a program part that performs processing for extracting the characteristic points. 11. The program part according to claim 10, further comprising a program portion for performing a process of selecting, as a characteristic point, only a curve having a ratio of the spectral power of the wave number 1 component and the spectral power of the wave number 2 component smaller than a predetermined threshold value. Computer readable recording medium. 12. Before SL value of each pixel is Ru Oh <br/> in brightness of each pixel, computer-readable recording medium according to claim 7
Body . 13. A means for preparing digital data of a value of each pixel in a target image area which is a human face area, and for each of the pixels in the target image area,
A means for obtaining a relative amount of a specific wave number component indicated by a distribution curve of values on a predetermined closed curve surrounding the pixel, and a relative amount of the specific wave number component in the face region are Station
The pixel showing the maximum locally is the feature point in the face area between the eyebrows
An image feature point extraction device including means for extracting as a point representing the image feature point. 14. The closed curve is centered on the pixel,
The image feature point extraction device according to claim 13, wherein the image feature point extraction device is a circle having a radius determined according to the size of the image region. 15. The means for determining the relative amount of the specific wave number component includes, for each of the pixels in the target image area, a pixel of each of the pixels on the closed curve that surrounds the pixel. The image feature point extraction apparatus according to claim 13 or 14, further comprising means for obtaining a relative amount of the specific wave number component by performing a discrete Fourier transform using the value as a sampling value. 16. The image feature point extraction apparatus according to claim 13, wherein the specific wave number component is a wave number 2 component. 17. Further, among a plurality of feature point candidates extracted by means for extracting as a feature point,
Only the characteristic points of which the ratio of the spectral power of the wave number 1 component and the spectral power of the wave number 2 component of the distribution curve of the value of each pixel on the closed curve surrounding the candidate point is smaller than a predetermined threshold value Including means for choosing as
The image feature point extraction device according to claim 16. 18. Before SL value of each pixel is Ru Oh <br/> in brightness of each pixel, image feature point extraction device according to 請 Motomeko 13.