JP2002056388A - Robot device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a robot device capable of discriminating a person even under an environment with inconstant lighting condition as domestic environment. SOLUTION: This device is provided with a person discrimination device comprising an image gaining means for gaining an image, a head detecting and tracking means for detecting a human's head from the image, a front face positioning means for gaining a front face image from the detected partial image of the head, a face characteristic extracting means for converting the front face image to characteristic quantity, a face discriminating means for discriminating the person from the characteristic quantity by the use of a discrimination dictionary and a discrimination dictionary memory means for storing the discrimination dictionary; a general control part for controlling the operation of a robot; a voice output means for generating a voice according to the instruction of the general control part; and a moving means for moving the robot according to the instruction of the general control part.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for identifying a person in an image, and more particularly to a technique for identifying a person by using a frontal face.
And a robot device for identifying a person.

[0002]

2. Description of the Related Art Several systems for identifying a person using a facial image have been proposed. Recent face detection,
The trend of the identification technology is described in, for example, Reference (1) (Shigeru Akamatsu, “Face Recognition by Computer-Survey”, IEICE Transactions, Vol.J80-D-II, No.8, pp.2031-2046 , Augus
t 1997). In general, a face identification system is configured to include a process of detecting a face from an image, a feature extraction process from a face pattern, and a person identification process of comparing a feature value with dictionary data.

A method for detecting a face image is described in reference (2).
(Shin Kosugi, "Search and Positioning of Faces in Scenes Using Multiple Pyramids for Individual Identification," IEICE Transactions, Vol.J77-D-II, No.4, pp.672-681, April 1994)
And template matching using a light and shade pattern as described in (3) (M. Turk, A.
Pentland, “Face Recognition on Using Eigenfac
es ", Proceedings of IEEE, CVPR91), an eigenvector projection distance method of a face image is known.

[0004] For example, Japanese Patent Application Laid-Open No. Hei 9-251534 proposes a method of detecting features such as eyes, nose, and mouth, and extracting a front face shading pattern from the positional relationship.

As a typical example of face detection, an eigenvector projection distance method by M. Turk will be described.

[0006] A lot of frontal face data (several hundred sheets)
Prepare An eigenvalue and an eigenvector are obtained using those pixel values as a feature vector. The p eigenvectors Vn (n = 1,... P) are obtained in descending order of the eigenvalue.

When the test image t is projected onto the eigenvector Vn, p projection values are obtained. By reconstructing a test image from these projection values and the eigenvector Vn, a reconstructed test image t 'is obtained.

[0008] If t is close to the face pattern, the reconstructed test image t 'can be an image close to the face pattern. Therefore, whether or not the face is a face is determined based on the distance scale Dt given by the following equation (1).

[0009]

There are two types of face identification features, one using geometric features of facial features such as eyes, nose, and mouth, and the other based on global shading pattern matching. As for the pattern, when the direction and expression of the face change, the positional relationship of the features also changes. Therefore, recently, the latter method of using a global shade pattern has become mainstream.

As a method for identifying and collating a face image, for example,
Reference (2) (Shin Kosugi, "Search and Positioning of Faces in a Scene Using Multiple Pyramids for Individual Identification", Transactions of the Institute of Electronics, Information and Communication Engineers, Vol.J77-D-II, No.4, pp. .672-68
1, April 1994) considers a light and shade pattern as a feature vector, and determines a category having a large inner product between the feature vectors as an identification result. In addition, the above reference (3) (M. Turk, A. Pentl)
and, “Face Recognition on Using Eigenfaces”, Pro
In ceedings of IEEE, CVPR91), a projection value of a face image onto an eigenvector is used as a feature vector, and a category having a small Euclidean distance is used as an identification result.

Conventionally, as a robot device having an image recognition function, for example, there is a device described in Japanese Patent Application No. 10-151591. This robot device can extract color information from an image and change its operation according to a color pattern. However, there is no functional means for recognizing a person.

[0013]

The above-described conventional system has the following problems.

The first problem is that a person cannot be identified in an environment where lighting conditions are not constant, such as a home environment.

The reason is that it is difficult to detect a face in a general environment. For example, the template matching method is difficult to detect unless the face pattern and the dictionary pattern in the image have almost the same density value, and if the lighting direction is slightly deviated or the person is different from the person in the dictionary, , Almost undetectable. on the other hand,
The eigenvector projection distance method has higher detection performance than template matching, but similarly fails to detect an image having a different illumination direction or having a complicated background.

Another reason why the person cannot be identified in an environment where the lighting conditions are not constant is that the conventional feature extraction method and the conventional identification method cannot absorb the variation in the characteristic amount due to the illumination variation.

Accordingly, the present invention has been made in view of the above problems, and an object of the present invention is to provide a robot apparatus capable of identifying a person in a general environment such as a home environment.

Therefore, another object of the present invention is to provide a robot device capable of stably identifying a person in a general environment.

[0019]

According to the present invention, there is provided a robot apparatus as a person identification apparatus, comprising: video acquisition means for acquiring an image; and head detection tracking for detecting a human head from the image. Means, a front face alignment means for obtaining a front face image from the detected partial image of the head, a face feature extraction means for converting the front face image into a feature quantity, and a person from the feature quantity using the identification dictionary. And an identification dictionary storage unit for storing an identification dictionary. And in the head detection tracking means, a monocular head rectangular coordinate detection means for detecting a head from one image,
A general control unit for controlling the operation of the robot, comprising: a general control unit that controls the operation of the robot; and a speaker that utters a voice according to an instruction from the general control unit. It is characterized by comprising a moving means for moving the robot in accordance with an instruction of the overall control unit, a facing distance sensor for measuring a distance to a preceding object, a touch sensor, a microphone, and a voice recognition means.

[0020] In the present invention, the overall control unit controls so as to speak with a different voice for each person when a person identification result is obtained.

[0021] In the present invention, the overall control unit may acquire a facing distance and a direction to a front object from the person identifying device, a unit acquiring a person identifying result, and wherein the facing distance is not less than a threshold value. In this case, the apparatus includes means for moving so as to approach the front object, and means for controlling the person identification result to be uttered in a different voice for each person when the facing distance is equal to or less than a threshold value.

[0022]

Next, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a diagram showing the configuration of an embodiment of a person identification device used in a robot device according to the present invention. Referring to FIG. 1, a person identification device 14 according to an embodiment of the present invention includes a video acquisition unit 2, a facing distance sensor 5, a person detection and identification unit 1, and a person detection and identification management unit 13.
And a dictionary data management unit 12. The image acquisition means 2 includes a right camera 3 and a left camera 4, and acquires respective camera image information. The facing distance sensor 5 is installed in the same direction as the optical axis of the camera, and measures a facing distance with an object in an image. Examples of the facing distance sensor 5 include an ultrasonic sensor and an infrared sensor. The person detection identification management unit 13 transmits an operation start command and an operation end command to the person detection
And sends a dictionary creation command.

A camera used in an embodiment of the present invention is, for example, a video camera, a digital CCD, or the like.
A photographing device including a camera or the like and capable of outputting a moving scene as a series of still images is generically called.

The person detecting and identifying means 1 includes a head detecting and tracking means 6, a front face aligning means 7, and a facial feature extracting means 8
, A face identification unit 9, an identification dictionary storage unit 10, and an identification result correction unit 11.

Upon receiving the operation start command from the person detection / identification management unit 13, the person detection / identification unit 1 loads the identification dictionary from the dictionary data management unit 12 into the identification dictionary storage unit 10, and then starts operation.

FIG. 18 is a flowchart for explaining the processing of the person detection / identification means 1 according to one embodiment of the present invention. The operation of the person detection and identification means 1 will be described with reference to FIGS.

First, based on the image information from the video acquisition means 2 and the reading value of the facing distance sensor 5, the head detection and tracking means 6 calculates the number of heads of the person in the current frame and the head rectangle. The coordinates are output (step S1).

Next, the number of detected heads is evaluated (step S2). If the number of detected heads is 0, the head of the next frame is input to perform video detection, and step S1 is continued until the number of detected heads becomes 1 or more.

When the number of detected heads is one or more, the detection result is transmitted to the front face positioning means 7. The front face alignment means 7 performs a search process of the face area (step S3),
It is determined whether a front face area has been found (step S4).

When a frontal face is found, a frontal face image which is a rectangular image at the center of the face is output. The processing in steps S3 and S4 is intended to eliminate erroneous detection of the head, extract only the video in which the person is facing the front of the camera, and send it to the subsequent processing. When the front face cannot be found, the processing is performed again from step S1.

When a front face is found, the face feature extraction means 8 converts the front face image into feature data (step S5).

As shown in FIG. 8, one example of the face feature extracting means 8 scans the frontal face image line by line from left to right, and scans the next line after finishing one line from top to bottom. It generates dimensional data (referred to as “raster scan”) and uses it as feature data. Alternatively, a method may be used in which filtering is performed using a primary differential filter or a secondary differential filter, and edge information is extracted and raster-scanned into feature data.

Next, the face identification means 9 performs face identification processing with reference to the dictionary data in the identification dictionary storage means 10 (step S6).

Next, the identification result correcting means 11 performs integration processing with the identification results for the past m frames (m is an integer of 2 or more) (step S7), and sends the result to the person detection identification management section 13. Output (Step S8).

At this time, the head detection and tracking means 6 (step S
In 1), when a plurality of head rectangles are detected and all of them are not processed (No branch of step S9), the processing is performed again from the front face alignment means 7 (step S3). The person detection / identification unit 1 ends upon receiving an end instruction from the person identification management unit 13 (step S1).
0). Until there is an end instruction, the processing is continued from step S1 again.

FIG. 2 is a diagram showing the configuration of the head detecting and tracking means 27 which constitutes one embodiment of the head detecting and tracking means 6 of FIG. Referring to FIG. 2, the head detection and tracking unit 27 includes a head detection unit 21, a head tracking unit 22, and a head rectangular coordinate storage unit 23. The head detecting unit 21 includes a head rectangular coordinate detecting unit 24, a left and right image matching unit 25, a facing distance integrating unit 31, and a facing distance evaluating unit 26.

FIG. 19 is a flowchart for explaining the processing of the head detecting and tracking means 27. The operation of the head detection and tracking means 27 according to one embodiment of the present invention will be described with reference to FIGS.

The image of the right camera, the image of the left camera, and the value read by the facing distance sensor are input to the head detecting means 21. The head detecting means 21 performs a process of detecting a human head from the input information, and outputs the coordinates of the head rectangle and the number of detected heads (step S10).

If the number of detected heads is 1 or more, the number of detected heads and the coordinates of the head rectangle are stored in the head rectangle coordinate storage means 23 and then output (step S18).

If the number of detected heads is 0, the head tracking means 22 converts the head rectangle information in the previous frame into
It is taken out from the head rectangle storage means 23 and head tracking processing is performed (step S19).

If the head tracking is successful, the number of heads that have been successfully tracked and the rectangular coordinates of the head are output. If the tracking fails, the number of detections 0 is output (step S20).

Next, the operation of the head detecting means 21 in step 10 of FIG. 19 will be described in detail. Head detection means 2
In step 1, either the left or right image is input to the head rectangle coordinate detection means 24, and the number of detected temporary heads and the coordinates of the temporary head rectangle are obtained (step S11).

The head rectangular coordinate detecting means 24 shown in FIG. 2 uses a right camera image. Next, the left-right image matching means 25 calculates the facing distance value based on the principle of stereo viewing using the obtained rectangular coordinates of the head and the images of the left and right cameras (step S12 in FIG. 19).

Referring to FIG. 6, the operation of the left and right image matching means 25 of FIG. 2 will be described. The head rectangle detected in the right camera image is referred to as a head detection rectangle 51. Then, using the image data in the head detection rectangle 51, the vicinity of the same detection coordinate position of the left camera image is searched. One example of a search method is template matching. Assuming that the grayscale value of the right camera image is FR (x, y), the grayscale value of the left camera image is FL (x, y), the horizontal size of the rectangle is Tw, and the vertical size is Th, The matching distance Dtm at (sx, sy) of the camera image is represented by the following equation (2).

[0046]

The above equation (2) represents the Euclidean distance between the partial images of the right camera and the left camera. The coordinates on the left camera image when Dtm is the smallest are set as a search result 52. When the search result 52 is obtained, the left and right rectangular coordinate values are compared, and the distance to the person's head is calculated.

Referring to FIG. 32, an example of a method for calculating the distance to the object will be described. FIG. 32 is a diagram of a situation in which a certain target object 403 is photographed using the left and right cameras, as viewed from above. A right camera 401 and a left camera 402 are installed in parallel at an interval C. Camera angle of view is θ
And the same on both sides. Let e be the horizontal length of the imaging surface of the camera. In this state, the object 4 is displayed on the right camera image.
03 is shown at the coordinate Xr, and in the left camera image is shown at the coordinate Xl. Note that the maximum horizontal size of an image is W pixels. At this time, the facing distance Z from the camera imaging plane to the symmetric object 403 can be calculated by the following equation (3).

Here, since e is usually a small value of less than 1 cm, it can be approximated as 0. As described above, the facing distance is calculated from the left and right camera images.

Then, in the left and right image matching means 25,
After the face-to-face distance is calculated by stereo vision, the face-to-face distance integrating means 31 performs a face-to-face distance integration process based on the output value of the face-to-face distance sensor 30 (step S in FIG. 19).
13). Experimentally, distance sensors such as ultrasonic sensors have very high accuracy when the distance is less than 1 m. On the other hand, 1m
The error tends to increase at the above-mentioned long distance.

The distance value calculated by stereo viewing is effective up to about 3 m depending on the angle of view of the camera.
If the distance is too short, the error tends to be rather large. Therefore, as a method of integrating the distance values of both, when the output of the facing distance sensor 30 is smaller than a certain threshold value T, the value of the facing distance sensor is adopted, and when the output of the facing distance sensor 30 is larger than the threshold value T, A method of employing a distance value by stereo vision is used.

After integrating the face-to-face distances, the face-to-face distance evaluation means 26 calculates the actual size of the head from the face-to-face distance value and the coordinates of the head rectangle in the image (step S14 in FIG. 19).

If the result of the calculation substantially matches the size of the human head, it is determined that the head has actually been detected. If the calculation result is significantly different from the actual head size, it is determined to be an erroneous detection (step S15 in FIG. 19). For example, when the horizontal size of the head is within 12 cm ± 2 cm and the vertical size is within 20 cm ± 4 cm, it is determined that the head is not the head.

If the size matches the actual size, the number of detections is increased by 1 (step S16 in FIG. 19).

If the temporary head rectangular coordinates that have not been evaluated remain (No branch of step S17 in FIG. 19), the processing is performed again from step S12. The head detecting means 21 outputs the number of detected heads and the rectangular coordinates of the head when all of the temporary head rectangular coordinates have been evaluated (Yes in step S19 in FIG. 19).

Next, the head rectangular coordinate detecting means 24 shown in FIG. 2 will be described. FIG. 4 is a diagram showing a configuration of the head rectangular coordinate detecting means 41 which constitutes one embodiment of the head rectangular coordinate detecting means 24. Referring to FIG. 4, a head rectangular coordinate detecting unit 41 includes a motion pixel detecting unit 42, a noise removing unit 47, a person number evaluating unit 43, and a top coordinate detecting unit 44.
Head lower coordinate detecting means 45, temporal coordinate detecting means 4
6 is provided.

FIG. 20 is a flowchart for explaining the processing of the head rectangular coordinate detecting means 41. The operation of the head rectangular coordinate detecting means 41 will be described with reference to FIG. 20, FIG. 4, and FIG.

First, the moving pixel detecting means 42 detects a moving pixel group in the screen. Input image data,
The difference from the image data input immediately before that is obtained, and a difference image g is generated (step S21).

Further, an integrated difference image G is obtained by adding the difference images g of the past m frames (m is an integer of 2 or more) and averaging them (step S22). Integrated difference image G
Has a pixel value of 0 in a non-moving area, and has a larger pixel value in a moving area.

Since the integrated difference image G contains a lot of sesame salt noise, the noise removing means 47 performs a noise removing process (step S23). Examples of the noise removal processing include expansion and contraction processing and median filter processing. These noise removal processes are common in the field of image processing, and processes that are well known to those skilled in the art are used, and thus detailed configurations are omitted.

Next, the number-of-persons evaluation means 43 in FIG. 4 evaluates how many people are on the screen. The operation of the number-of-persons evaluation means 43 will be described. FIG. 5 is a diagram for describing an example of obtaining the integrated difference image G.

First, a method for detecting only one person will be described. Assuming that the integrated difference image G48 has been obtained, it is first determined whether or not there is a moving area (step S24 in FIG. 20). Here, the moving area indicates an area occupied by a moving pixel. If there is no motion area, that is, if the integrated difference image G is all zero, the number of persons is determined to be zero. In other cases, the number of persons is one.

Next, a method for detecting a plurality of persons will be described. Assuming that the integrated difference image G49 is obtained, first, the presence or absence of a moving area is checked (step S24 in FIG. 20).
When there is no motion area, the number of persons is zero. If there is a moving area, the number of persons is determined with reference to the integrated difference image G (step S25 in FIG. 20). As a judgment method,
For example, there is a method in which the maximum value of the motion region width in the integrated difference image upper region 50 is one when the maximum value is smaller than a certain threshold value, and two when the maximum value is larger than a certain threshold value. When the number of persons is two, it is assumed that the persons are arranged side by side, and the integrated difference area G
Is divided into a partial area 1 and a partial area 2. It should be noted that a case where three or more persons are detected can be handled by increasing the number of divisions. When obtaining the head rectangle, the same processing described below (from step S26 to step S29 in FIG. 20) may be repeated for each of the partial area 1 and the partial area 2.

Next, the processing for obtaining the coordinates of the head rectangle from the integrated difference image G will be described. The motion area width 47 is obtained for each scan line (step S26 in FIG. 20).

The moving area width 47 represents the difference between the maximum and minimum values of the x coordinate of the moving area in each scan line.

Next, the Y coordinate of the crown is obtained by the crown coordinate detecting means 44 (step S27 in FIG. 20). As a method of obtaining the top coordinates, there is a method in which the minimum value of the Y coordinate of the motion area is set to the top.

Next, the Y coordinate of the bottom side of the head rectangle is obtained by the lower head coordinate detecting means 45 (step S28 in FIG. 20). As a method of obtaining the base coordinate of the head rectangle, a search is performed in a downward direction (Y direction) from the top of the head, and a line in which the moving area width 47 is smaller than the average value dm of the moving area width is obtained. May be used as the bottom of the rectangular head.

Next, the left and right x-coordinates of the head rectangle are obtained by the temporal coordinate detecting means 46 (step S2 in FIG. 20).
9). As a method of obtaining the left and right x-coordinates, a method of obtaining the coordinates of the left and right ends of the moving area in a line having the largest moving area width 47 in the range from the top to the lower part of the head may be used.

If the number of persons is two or more, the processing from step S26 to step S29 in FIG. 20 is repeated for each partial area.

Next, the operation of the head tracking means 22 of FIG. 2 will be described with reference to FIG. In the tracking process, the camera image used for detecting the rectangular coordinates of the head (the right camera image in FIG. 2)
Perform for First, the head rectangle coordinates 53 of the previous frame and the head rectangle image 55 of the previous frame are stored in the head rectangle storage unit 2.
Read from 3.

Next, in the current frame, a region near the head rectangular coordinates 53 of the previous frame is searched by template matching, and a portion having the smallest distance value is determined as a tracking result.

FIG. 3 is a diagram showing the configuration of the head detecting and tracking means 32 constituting another embodiment of the head detecting and tracking means 6 of FIG. Referring to FIG. 3, the head detecting and tracking means 32
The apparatus includes a head detection unit 33, a head rectangle storage unit 23, and a head tracking unit 22. The difference from the embodiment shown in FIG. 2 is that the head detecting means 33 has the head rectangular coordinate detecting means 24 and the face-to-face distance evaluating means 26, and the output of the monocular camera 34 and the face-to-face distance sensor 30. That is, detection is performed using That is, the head rectangle is evaluated using only the reading value of the facing distance sensor without considering the facing distance in left and right stereo vision.

As another embodiment of the head detecting and tracking means 6, a head detecting means having a structure in which a facing distance is obtained from information of only the left and right cameras without using a facing distance sensor and a head rectangle is evaluated. May be used. In this configuration,
In the head detecting means 21, the facing distance integrating means 3 shown in FIG.
1 is excluded.

FIG. 9 is a diagram showing the structure of the front face positioning means 61 which constitutes one embodiment of the front face positioning means 7 of FIG. Referring to FIG. 9, the front face alignment means 6
1 is a head rectangular cutting means 62 and a front face searching means 6
3 and a front face-likeness determination means 65.

The front face likeness determining means 65 includes a density dispersion determining means 66 and a threshold value processing means 67.

FIG. 21 is a flow chart for explaining the processing of the front face positioning means 61. The operation of the front face positioning means 61 will be described with reference to FIGS. When the image data, the rectangular coordinates of the head, and the facing distance are input, the front face positioning means 61 outputs a front face presence / absence flag and front face image data. The input image data is cut into a partial image based on the head rectangle by the head rectangle cutting means 62 (step S41). This partial image is called a “head rectangular image”.

Next, the frontal face search means 63 searches the head face rectangular image for a frontal face area, and outputs the distance between patterns or the similarity between the frontal face image and the standard face dictionary (step S42).

Next, the front face likeness determining means 65 determines whether or not the front face image is really a front face (step S43). If it is determined that the face is a front face, the front face presence / absence flag is set to “present”, and a front face image is output. If it is determined that the front face is not a front face, the front face presence / absence flag is set to “none”, and no front face image is output.

The front face likeness determination means 65 includes a density variance determination means 66 and a threshold value processing means 67.

The density variance judging means 66 obtains the variance of the gray value of the front face image data, and judges that it is not the front face if it is equal to or less than a certain threshold value (step S4 in FIG. 21).
4).

The density dispersion determining means 66 can eliminate a pattern such as a monotonous wall.

The threshold value means 67 determines the likeness of the frontal face by performing threshold processing on the distance or similarity between the patterns (step S45 in FIG. 21).

In the case of the inter-pattern distance value, it is determined that the face is not the front face when the distance is equal to or larger than the threshold value. In the case of the similarity, when the similarity is equal to or less than the threshold value, it is determined that the face is not the front face.

FIG. 12 is an explanatory diagram schematically showing the operation of the front face positioning means 61. Assuming that the head rectangle 151 has been detected, a head rectangle image 152 is generated in step S41 of FIG.

Next, in the face center portion search processing of step S42 in FIG. 21, a front face image 155 is obtained after the reduced head rectangular image 153 is generated.

The frontal face image is an image of the central part of the face as shown in the frontal face image 155 in FIG. 12, and the horizontal direction is a range that completely includes both eyes, and the vertical direction is from the eyebrows. It means an image in an area that includes the entire mouth.

FIG. 10 is a diagram showing the configuration of the frontal face searching means 71 constituting one embodiment of the frontal face searching means 63. Figure 1
0, the front face searching means 71 includes a head rectangular image storage means 89, a head intermediate size calculating means 88,
Image reducing means 90, intermediate size storing means 91, front face candidate extracting means 72, intermediate reduced image storing means 73,
Contrast correction means 74, eigenvector projection distance calculation means 75, standard face dictionary storage means 76, storage means 7
7, a minimum projection distance determination unit 78, a search range end determination unit 79, and a multi-resolution processing end determination unit 92.

The eigenvector projection distance calculation means 75 includes an average difference means 82, a vector projection value calculation means 83, a reconstruction calculation means 84, and a projection distance calculation means 85.

The standard face dictionary storage means 76 comprises a standard face average data storage means 80 and a standard face unique vector data storage means 81.

The storage means 77 comprises a minimum projection distance value storage means 86 and a front face gray value storage means 87.

FIG. 22 is a flowchart for explaining the processing of the front face searching means 71. The operation of the front face searching means 71 will be described with reference to FIGS. The head rectangle image data is held in the head rectangle image storage unit 89. First, the head intermediate size calculating means 88 calculates the intermediate reduced size of the rectangular head image with reference to the facing distance value and the size of the standard face dictionary data (step S101).

A processing example of the head intermediate size calculating means 88 will be described. The intermediate reduced size is shown as the vertical and horizontal sizes of the reduced head rectangular image 153 in FIG. The horizontal size of the head rectangular image 152 is Hw, and the vertical size is Hh. The horizontal size of the intermediate reduced size is Mw, and the vertical size is Mh
And The horizontal size of the front face image is Fw, and the vertical size is Fh. Fw and Fh are the same as the vertical and horizontal sizes of the front face search shape 154, and are uniquely determined for the standard face dictionary. Note that Hh, Hw, Mh, Mw, Fh, and Fw are all pixel sizes in pixel units.

The standard face dictionary corresponds to the front face image 155 shown in FIG.
6 is a dictionary for pattern recognition generated using the grayscale values of the frontal face area shown in FIG. The frontal face region means a region that completely includes both eyes in the horizontal direction and a region that includes the entire eyebrow to the mouth in the vertical direction. The frontal face area does not necessarily have to be rectangular, and can be realized by any continuous area including both eyes, nose, and mouth, such as an elliptical shape. However, if the shape is rectangular, the processing can be simplified and speeded up.
This is effective as a mounting mode. Therefore, the following description will be made assuming that the front face area is rectangular.

The actual vertical and horizontal size of the frontal face area is represented by RF
h and RFw, if a male adult, roughly RF
w = 10 cm and RFh = about 15 cm. On the other hand, the actual vertical and horizontal sizes RHh of the head rectangular image,
Since the facing distance Z is known, RHw can be calculated by the following equation (4). The variables in the following equation (4) correspond to FIG.

[0095]

Since the width e of the imaging surface is small, it is not a problem to normally ignore and calculate.

In order to search for a head rectangle image using the standard face dictionary, it is necessary to convert the head rectangle image to the same resolution as the standard face dictionary. The sizes after the conversion are the intermediate reduced sizes Mw and Mh. Mh and Mw can be obtained from the relational expression of the following expression (5).

[0098]

That is, the head intermediate size calculating means 88 can calculate one set of intermediate reduced sizes Mw and Mh by designating one set of RFw and RFh.
However, the size of the human frontal face varies from adult to child, female and male. Therefore, it is also possible to prepare a plurality of sets of RFw and RFh and calculate an intermediate reduced size corresponding to each set. By performing a plurality of calculations in advance, it is possible to process the frontal face search process in the subsequent stage with a plurality of intermediate reduced sizes. Searching with a plurality of intermediate reduced sizes can be interpreted as the same as searching for a head rectangle with a plurality of resolutions.

When the intermediate size is calculated by the head intermediate size calculating means 88, information on the intermediate size is stored in the intermediate size storing means 91.

Next, the minimum inter-pattern distance value Dmin is initialized to a value sufficiently larger than the normally obtained inter-pattern distance value (step S102 in FIG. 22).

One piece of information in the intermediate reduced size storage means 91 is selected, and the image reduction means 90 reduces the rectangular head image to the selected intermediate reduced size to obtain a reduced head rectangular image (step S103 in FIG. 22). ).

Next, the front face search positions SX and SY are initialized to 0 (step S104 in FIG. 22).

Next, the front face candidate extracting means 72 extracts front face candidate images at the search positions SX and SY (step S105 in FIG. 22).

Next, the contrast is corrected by the contrast correcting means 74 in order to correct the brightness of the front face candidate image (step S106 in FIG. 22).

An example of a specific method of contrast correction will be described. Assuming that the front face candidate image takes a value from 0 to vmax, the average of the pixel values is μ, and the standard deviation is σ, the conversion equation from the original image V to the contrast corrected image V ′ is as follows: ).

[0107]

Referring again to FIGS. 10 and 22, next, the eigenvector projection distance calculating means 75 calculates the eigenvector projection distance between the front face candidate image and the standard face pattern.
D is obtained (step S107).

Next, the minimum projection distance determining means 78 compares D and Dmin. At this time, if D is a value smaller than Dmin, the value is updated by substituting D for Dmin, and stored in the minimum projection distance value storage means 86. At the same time, the front face candidate image is stored in the front face gray value storage means 87 (step S108).

Next, in the search range end determining means 79, the search positions SX and SY are incremented (step S109), and it is determined whether or not all the head rectangles have been searched (step S110). If the search has not been completed, the processing is repeated from step 105 again.

When the search for the entire head rectangle is completed, the multi-resolution processing end determination means 92 determines whether or not the search has been performed for all intermediate reduced sizes (step S111). If there is an intermediate reduced size that has not been searched, step S is performed again using a different intermediate reduced size.
The processing is started from 103. When the search is completed for all intermediate reduced sizes, the frontal face search means 71 ends.

Next, the operation of the eigenvector projection distance calculation means 75 of FIG. 10 will be described.

The standard face dictionary storage means 76 stores standard face average data and standard face specific vector data.

FIG. 33 shows an example of an eigenvector projection distance calculation dictionary when the number of feature values is p. The eigenvector projection distance calculation dictionary includes p-dimensional eigenvector data E from the first to the p-th and an average value Ave of p feature amounts. When there are p feature amounts, the eigenvectors exist up to the p-th, but the projection distance is calculated using the first to the m-th.

The pixel values of the frontal face candidate image are raster-scanned as shown in FIG. 8 and converted into one-dimensional feature data. At this time, the product Fw × Fh of the vertical and horizontal sizes of the front face image
Must be p, which is the same as the feature amount of the dictionary. This is expressed as a vector X: X1, X2,. . . Xp.

First, the average difference means 82 subtracts the average vector Ave from the vector X. This is called the vector Y
And

[0117]

Next, the vector projection value calculating means 83 projects the vector Y onto m eigenvectors, and sets the projection values R1. . Find Rm. The projection value calculation method is shown in the following equation (8).

[0119]

Next, the reconstruction calculation means 84 (see FIG. 10)
At the projection values R1. . . The original feature amount Y is reconstructed using Rm and m eigenvectors, and the reconstructed vector is defined as Y ′. The calculation of the reconstruction is shown in the following equation (9).

[0121]

Next, the projection distance calculating means 85 obtains a Euclidean distance value between Y and Y 'according to the following equation (10). Thus, the projection distance D to the eigenvector E is
Is calculated.

[0123]

FIG. 11 is a diagram showing the configuration of the front face search means 101 which constitutes another embodiment of the front face search means 63 of FIG. Referring to FIG. 11, this front face searching means 101
The head rectangular image storage unit 89, the head intermediate size calculation unit 88, the image reduction unit 90, the intermediate size storage unit 91, the front face candidate extraction unit 72, the intermediate reduced image storage unit 73, the contrast Correction means 74, sum-of-products calculation means 102, standard face dictionary data storage means 104, storage means 105, maximum similarity determination means 103, search range end determination means 79, and multi-resolution processing end determination means 92
It is comprised including.

The storage unit 105 includes a maximum similarity storage unit 106 and a front face gray value storage unit 107.

FIG. 23 is a flowchart for explaining the processing of the front face searching means 101. The operation of the frontal face search means 101 will be described with reference to FIGS.
The head rectangle image data is held in the head rectangle image storage unit 89. First, the head intermediate size calculating means 88 calculates the intermediate reduced size of the head rectangular image with reference to the facing distance value and the size of the standard face dictionary data, and stores it in the intermediate size storage means 91 (step S121).

The method of calculating the intermediate reduced size is the same as that of the front face searching means 71. Next, the maximum similarity Smax is initialized to 0 (step S122).

One piece of information in the intermediate size storage means 91 is selected, and the image reducing means 90 reduces the rectangular head image to the selected intermediate reduced size to obtain a reduced rectangular head image (step S123).

Next, the front face search positions SX and SY are initialized to 0 (step S124).

Next, the front face candidate extracting means 72 extracts a front face candidate image at the search positions SX and SY (step S125).

Next, the contrast is corrected by the contrast correcting means 74 in order to correct the brightness of the front face candidate image (step S126).

Next, the sum-of-products calculating means 102 calculates the similarity S between the front face candidate image and the standard face pattern (step S127).

Next, the maximum similarity value judgment means 103 compares S with Smax. At this time, if S is a value larger than Smax, the value is updated by substituting S for Smax,
It is stored in the maximum similarity storage unit 106. At the same time, the front face candidate image is stored in the front face gray value storage means 107 (step S128).

Next, in the search range end determining means 79,
The search positions SX and SY are incremented (step S1
29) It is determined whether or not all the head rectangles have been searched (step S130). If the search has not been completed, the processing is repeated from step 125 again.

When the entire search range of the head rectangle has been searched, the multi-resolution processing end determination means 92 determines whether or not the search has been performed for all intermediate reduced sizes. If there is an intermediate reduced size that has not been searched, the process is restarted from step S123 using a different intermediate reduced size.

When the search is completed for all the intermediate reduced sizes, the frontal face searching means 101 ends.

Next, the operation of the product-sum operation means 102 for calculating the similarity with the standard face pattern shown in FIG. 11 will be described.

The sum-of-products calculation means 102 calculates the similarity S with reference to a linear discrimination dictionary for discriminating whether the face is a front face or not. FIG. 34 shows an example of the linear discrimination dictionary. FIG. 34 shows a dictionary for discriminating q classes. The standard face dictionary data storage means 104 is a dictionary for discriminating between a frontal face and the other two classes. Is equivalent to

The pixel values of the frontal face candidate image are raster-scanned as shown in FIG. 8 and converted into one-dimensional feature data. At this time, the product Fw × Fh of the vertical and horizontal sizes of the front face image
Must be p, which is the same as the feature amount of the dictionary. This is expressed as a vector X: X1, X2,. . . Xp.

The standard face dictionary data will be described on the assumption that the class of q = 1 represents the frontal face and the class of q = 2 represents the other. The similarity to the front face is represented by q =
It can be calculated by the following equation (11) using only the 1st row, that is, the class 1 identification coefficient 422.

[0141]

The product-sum calculation means 102 calculates the similarity by calculating the above equation (11).

FIG. 13 shows a face identification unit 111 which is an embodiment of the face identification unit 9 of FIG. 1, an identification dictionary storage unit 112 which is an embodiment of the identification dictionary storage unit 10 of FIG.
And an identification result correction means 113 which constitutes one embodiment of the identification result correction means 11 of FIG. Referring to FIG. 13, the face identification unit 111 includes a feature data storage unit 115, a product-sum operation unit 116, a maximum similarity person determination unit 118,
And a threshold processing unit 117. The identification dictionary storage unit 112 includes a registered person identification dictionary storage unit 119. The identification result correction means 113 has an identification result weighted average calculation means 114.

FIG. 24 is a flowchart for explaining the processing of the face identification means 111 and the identification result correction means 113. The operations of the face identification unit 111 and the identification result correction unit 113 will be described with reference to FIGS.

The characteristic data is input and stored in the characteristic data storage means 115 (step S51).

Next, the registered person identification dictionary storage means 119
The sum-of-products calculating means 116 calculates the degree of similarity to the registered person for each person by referring to the data (step S54).

The method of calculating the similarity is basically the same as the operation of the product-sum operation means 102 for calculating the similarity with the standard face pattern. However, the number of classes to be identified is the number of registered persons.

When data for q persons is registered as a dictionary, the registered person identification dictionary storage means 119 stores
The same number of data as the linear discrimination dictionary shown in FIG. 34 is held. Then, the product-sum operation unit 116 calculates q similarities Si: (i =
1,. . . q) is obtained.

[0149]

The method of identifying a pattern based on the magnitude of similarity obtained by the product-sum operation using the linear discriminant dictionary shown in FIG. 34 is referred to as “similarity discrimination by linear discriminant dictionary”.

Referring again to FIGS. 13 and 24, next, the maximum similarity person determination means 118 obtains the maximum value among the calculated q similarities, and obtains the corresponding person (step S55). ). That is, a person determined to be most similar to the feature data is obtained. The similarity at this time is called “maximum similarity”.

Next, in the threshold value processing means 117,
The maximum similarity is compared with a predetermined threshold (step S56).

When the maximum similarity is higher than the threshold, the face identifying means 111 determines that the person has been identified.
The ID number and the maximum similarity are output. If the maximum similarity is lower than the threshold, the registered person (person)
Is likely to be another person, the information "others" is output.

After receiving the identification result, the identification result correcting means 113 integrates the identification results in the past N frames by the identification result weighted average calculating means 114 (step S57). As an example of the operation of the identification result weighted average calculation means 114, there is a method of weighing the identification person ID and the similarity in the past N frames or the result of other person determination as follows.

Step A1: In the past N frames, when the number of frames at a fixed rate is “others”, it is determined as “others”. If it is determined that the person is not another person, the process proceeds to step A2.

Step A2: In the past N frames, it is assumed that a person i is determined to be a Ni frame (i =
1. . . q). The similarity-weighted average value of each person is calculated by the following equation (13). Si represents the similarity of the person i,
SSi represents a similarity-weighted average value of the person i. The person ID having the largest SSi is output as the identification result.

[0157]

The identification result correcting means 113 outputs the identification result integrated as described above.

FIG. 15 shows the structure of the face identification means 131 which constitutes another embodiment of the face identification means 9 of FIG. 1, and the structure of the identification dictionary storage means 132 which constitutes another embodiment of the identification dictionary storage means 10 of FIG. FIG. Referring to FIG. 15, the face identification unit 131 includes a feature data storage unit 115, an eigenvector stranger determination unit 133, a product-sum calculation unit 116, a maximum similarity person determination unit 118, and a threshold processing unit 117. Have.

The identification dictionary storage means 132 includes a dictionary storage means 134 for discriminating another person and a dictionary storage means 11 for registered person identification.
9 is provided.

FIG. 24 is a flowchart for explaining the processing of the face identifying means 131. The operation of the face identifying means 131 will be described with reference to FIGS. The feature data is input and stored in the feature data storage unit 115 (step S51).

Next, referring to the data stored in the dictionary storage means 134 for identifying others, the eigenvector and others identification means 133 is referred to.
Is the distance D between the pattern and the registered group of persons.
h is obtained (step S52), and if the inter-pattern distance Dh is larger than the threshold value, it is determined that another person is present (step S53). The inter-pattern distance Dh is calculated as the eigenvector projection distance. That is, the dictionary storage means for other persons stores an eigenvector dictionary created based on the registered feature data of all registered persons.

An example of the eigenvector dictionary is shown in FIG. 33, and the method of calculating the eigenvector projection distance is described in the description of the operation of the eigenvector projection distance calculation means 75.

If the input characteristic data is determined to be another person by the eigenvector other person discriminating means 133, the face discriminating means 131 immediately outputs "others" without going through the product-sum calculating means 116.

If the eigenvector / others determination unit 133 does not determine “other”, then the data in the registered person identification dictionary storage unit 119 is referred to, and the product-sum operation unit 116 determines the registered person. Is calculated for each person (step S54). If q persons are registered, q similarities Si: (i =
1,. . . q) is calculated.

Next, the maximum similarity person determination means 118 obtains the maximum value among the calculated q similarities, and obtains the corresponding person (step S55). That is, a person determined to be most similar to the feature data is obtained. The similarity at this time is called a maximum similarity. Next, the threshold value processing unit 117 compares the maximum similarity with a predetermined threshold value (step S56).

When the maximum similarity is higher than the threshold, the face identifying means 131 determines that the person has been identified, and
The ID number and the maximum similarity are output. When the maximum similarity is lower than the threshold value, it is highly likely that the registered person is another person who is not a registered person, so that information of “other person” is output.

FIG. 14 is a diagram showing a configuration of the dictionary data management section 121 which forms one embodiment of the dictionary data management section 12 of FIG. Referring to FIG. 14, the dictionary data management unit 121
Is provided with a personal characteristic data storage unit 122, an identification dictionary generation unit 123, and a selector 124.

The identification dictionary generating means 123 comprises a linear discriminating dictionary creating means 126 and an eigenvector dictionary creating means 127. Individual characteristic data storage means 122
Inside, there are the individual feature data areas 125 for the number of registered persons.

When the feature data and the person ID number are input, the dictionary data management unit 121 sorts the data by person ID by the selector 124 and inputs the data to the area corresponding to the person ID number in the personal feature data storage unit 122. The feature data is stored. If there is an instruction to add a new person ID, a new person-specific feature data area 125 is secured in the personal feature data storage means 122 and a new person ID number is assigned. If there is an instruction to delete an existing person ID, the individual characteristic data area 125 of the corresponding ID in the personal characteristic data storage unit 122 is discarded.

When the dictionary data management unit 121 receives the identification dictionary creation command, the identification dictionary generation unit 123 uses the data of the personal feature data storage unit 122 to register a registered person identification identification dictionary that is a linear discrimination dictionary. A dictionary for identifying others, which is an eigenvector dictionary, is generated. The identification dictionary for registered person identification is created by the linear discrimination dictionary creating means 126. The dictionary for identifying others is created by eigenvector dictionary creating means 12.
7 is created.

In the case where the face identifying means 9 in FIG. 1 has the configuration of the face identifying means 111 shown in FIG. 13, the dictionary for discriminating another person is unnecessary, so that the identification dictionary creating means 123 is used as the eigenvector dictionary creating means. 127 may be omitted.

FIG. 25 is a flowchart for explaining the registration processing to the dictionary data management unit 12 in FIG. 1 and 1
4 and FIG. 25, an operation when the dictionary data management unit 121 registers a new person in front of the camera will be described.

The person detection / identification management unit 13 sets a new person I
By specifying D, the dictionary management unit 121 is instructed to register a new person (step S61).

The dictionary data management unit 121 secures the individual-specific feature data area 125 corresponding to the specified ID in the personal feature data storage unit 122 (step S6).
2).

The person detection / identification management unit 13 acquires the specified number of pieces of feature data from the face feature extraction unit 8 in the person detection / identification unit 1 and sends it to the dictionary data management unit 121.

The dictionary data management unit 121 stores the acquired data in the individual-specific feature data area 125 corresponding to the new ID (step S63).

When the specified number of sheets have been obtained, the person detection identification management section 13 instructs the dictionary data management section 121 to create an identification dictionary (step S64).

Upon receiving the creation instruction, the dictionary data management unit 121 creates a registered person identification dictionary by the linear discriminating dictionary creation unit 126 of the identification dictionary creation unit 123 (step S65).

Next, a dictionary for identifying others is created by the eigenvector dictionary creating means 127 (step S6).
6).

Then, the created dictionary is output to and stored in the identification dictionary storage means 10 of the person detection / identification section (step S67). With the above processing, the registration processing of the new person ends.

FIG. 29 is a diagram showing a configuration of the linear discriminant dictionary creating means 311 which constitutes one embodiment of the linear discriminant dictionary creating means 126 of FIG. Referring to FIG. 29, the linear discriminant dictionary creation unit 311 includes a feature amount X storage unit 312, a variance-covariance matrix Cxx calculation unit 313, and an inverse matrix transformation unit 311.
14, a matrix multiplication unit 315, and a target variable Y storage unit 3.
17, the objective variable Y generating means 318, and the covariance matrix Cx
It comprises a y calculating means 319, a coefficient storing means 320, and a constant term calculating means 316.

Referring to FIGS. 31 and 34, a method of creating a linear discriminant dictionary will be described. FIG. 31 shows personal characteristic data X341 for n persons, and for q persons from person 1 to person q. In addition, personal characteristic data X3
The number of features of 41 is p. One line in FIG. 31 shows one piece of feature data. The personal characteristic data X341 is
In FIG. 14, individual characteristic data of one person is stored in the individual characteristic data storage unit 122, and individual characteristic data of one person is stored in the individual characteristic data area 125.

The target variable Y342 is a vector created for one piece of feature data and having elements for the number of persons to be identified. That is, in FIG. 31, since the person ID takes a value from 1 to q, the objective variable Y is from Y1 to Y
exist up to q. The value of the objective variable Y342 is 0 or 1 of 2
And the vector element of the person to which the feature data belongs is 1
And the others are 0. That is, in the case of the characteristic data of the person 2, only the Y2 element is 1 and the others are 0.

FIG. 34 is a diagram showing an example of the format of the linear discrimination dictionary. The linear discriminant dictionary includes two types of coefficients, a constant term 421 and a multiplication term 425. The matrix composed of the multiplication terms is represented by Aij (i = 1,... P, j = 1,.
q), a vector consisting of a constant term is represented by A0j (j =
1,. . . . q), the matrix Aij is obtained from the following equation (14).

[0186]

In the above equation (14), Cxx is the variance-covariance matrix using all of the personal characteristic data X341. This variance-covariance matrix Cxx is given by the following equation (1)
It is calculated in 5).

The elements of personal characteristic data X are represented by xij: (i
= 1. . . . N, j = 1,. . . expressed by p). N is the total number of data, and p is the number of features. In the example shown in FIG. 31,
Since there are n pieces of data per person, N = nq. x￣ represents an average value of x.

[0189]

Cxy is the personal characteristic data X and the objective variable Y
Is the covariance matrix with The covariance matrix Cxy is given by the following equation (1)
It is calculated according to 6). x￣, y￣ represents an average value of x and y.

[0191]

The constant term A0j is calculated according to the following equation (17).

[0193]

First, Cxx is calculated, and its inverse matrix is obtained. Next, Cxy is obtained. Finally, these matrices are multiplied to obtain a multiplication term matrix Aij.
Find j.

FIG. 29 is a view for explaining the processing of the linear discrimination dictionary creating means 311. With reference to FIG. 29, the operation of the linear discriminant dictionary creation means 311 will be described. The input personal characteristic data group is stored in the characteristic amount X storage unit 312.

Using the input personal characteristic data group and the person ID, the target variable Y
Is generated. The generated target variable Y is stored in the target variable Y storage unit 317.

Next, the variance-covariance matrix Cxx calculating means 31
At 3, the variance-covariance matrix Cxx is calculated.

Next, the inverse matrix conversion means 314 calculates an inverse matrix of the variance-covariance matrix Cxx.

Next, the covariance matrix Cxy calculating means 319 calculates the covariance matrix Cxy.

Next, the multiplication term Aij is calculated by the matrix multiplication means 315, and the multiplication term Aij is stored in the coefficient storage means 320.
Store the data.

Next, the constant term calculation means 316 calculates a constant term A0j and stores it in the coefficient storage means 320.

Finally, the data in the coefficient storage means 320 is output, and the processing ends.

Although FIG. 31 shows the binary data of 0 and 1, the linear discriminating dictionary creating means 311 uses 0 binary data.
It is also possible to use other binary data such as and 100.

FIG. 30 is a diagram showing the eigenvector dictionary creating means 331 which forms one embodiment of the eigenvector dictionary creating means 127 of FIG. Eigenvector dictionary creation means 33
1 is a feature amount storage unit 332 and a feature amount average calculation unit 3
33, a variance-covariance matrix calculation unit 334, an eigenvector calculation unit 335, and a coefficient storage unit 336.

A method for creating an eigenvector dictionary will be described with reference to FIGS. 31 and 33. Personal characteristic data X
The elements of xij: (i = 1 ... N, j =
1,. . . expressed by p).

First, an average value for each feature amount of X is obtained. Next, the variance-covariance matrix Cxx of X is calculated by the above equation (15).
Is calculated using An eigenvector of the variance-covariance matrix Cxx is obtained. The method of obtaining the eigenvector is widely known by those skilled in the art, and is not directly related to the present invention, and thus the details thereof are omitted.

With the above operation, an eigenvector dictionary having a format as shown in FIG. 33 is obtained. The eigenvectors are obtained by the number (p) of feature amounts. In FIG. 33, one line is 1
Three eigenvectors.

FIG. 30 shows the eigenvector dictionary creating means 33.
1 is a diagram for describing a configuration and processing of FIG. The operation of the eigenvector dictionary creation unit 331 will be described with reference to FIG.

[0209] When the personal characteristic data is input, it is stored in the characteristic amount storage means 332.

Next, the average value of the characteristic values is obtained by the characteristic value average calculating means 333 and stored in the coefficient storing means 336. Next, the variance-covariance matrix calculation means 334 calculates the variance-covariance matrix Cxx.

Next, the eigenvector calculation means 335 calculates the eigenvector of the variance-covariance matrix Cxx and stores it in the coefficient storage means 336. Finally, the coefficient storage means 3
36 is output and the process is terminated.

FIG. 16 is a diagram showing the configuration of an embodiment of the robot apparatus according to the present invention. Referring to FIG. 16, the robot device 201 includes a CCD camera 202
, Person detection identification means 203, dictionary data management unit 20
4, the person detection identification management unit 205, and the overall control unit 206
, A speaker 207, and a robot moving unit 208. The robot moving means 208 includes a motor 209
And wheels 210.

[0213] The person detection / identification means 203 performs person detection and identification based on the stereo image from the CCD camera 202. The person detection identification management section 205 exchanges information with the person detection identification section 203, exchanges information with the overall control section 206, and exchanges information with the dictionary data management section 204. The speaker 207 is connected to the general control unit 206 and can speak according to an instruction from the general control unit 206. Further, the overall control unit 206 sends a movement instruction to the robot moving unit 208. Robot moving means 208
Has a motor 209 and a plurality of wheels 210, and can move the robot in a free direction.

FIG. 26 is a diagram for explaining the processing of one embodiment of the robot apparatus according to the present invention. The operation of the robot apparatus 201 according to one embodiment of the present invention will be described with reference to FIGS. Robot device 20
When a person is detected, the person 1 moves in the direction of the person and approaches the person, and performs person identification when the person approaches within a predetermined distance.

The person detection / identification management unit 205 obtains the rectangular information of the detected head and the facing distance value from the head detection / tracking unit in the person detection / identification unit 203, and
(Step S71).

The overall control unit 206 refers to the facing distance value and determines whether the facing distance is shorter than a threshold value (step S73). If the distance is longer than the threshold, the robot moving means 208 is commanded to move forward in the direction of the person (step S72).

The approximate direction in which the person exists can be inferred from the coordinates of the head rectangle in the image. Step S7
Steps S73 to S73 are repeated, and when the face-to-face distance is shorter than the threshold, the overall control unit 206 acquires the person identification result from the person detection identification management unit 205 (step S74).

Next, a voice for each person is uttered from the speaker 207 to notify the interlocutor that the person has been identified (step S75).

FIG. 17 is a diagram showing the configuration of an embodiment of the robot apparatus according to the present invention. Referring to FIG. 17, the robot apparatus 221 includes a CCD camera 202, a facing distance sensor 223, a touch sensor 222, a person detection and identification unit 224, a dictionary data management unit 204, a person detection and identification management unit 205, A control unit 227, a microphone 225,
It comprises a voice recognition means 226, a speaker 207, and a robot moving means 208. The robot moving means 208 includes a motor 209 and wheels 210.

[0220] The person detection / identification means 224 performs person detection and identification based on information from the stereo image from the CCD camera 202 and the facing distance sensor 223. The person detection identification management unit 205 exchanges information with the person detection identification unit 224, exchanges information with the overall control unit 227,
The exchange of information with the dictionary data management unit 204 is performed. The speaker 207 is connected to the overall control unit 227,
The utterance can be made according to an instruction from the overall control unit 227. The overall control unit 227 sends a movement instruction to the robot moving unit 208. The robot moving means 208 has a motor 209 and a plurality of wheels 210, and can move the robot in a free direction. The touch sensor 222 is connected to the overall control unit 227, and detects the presence or absence of contact with an object from outside and the strength of the contact. The microphone 225 is connected to the voice recognition unit 226, and the voice recognition unit 226 is connected to the overall control unit 227. The voice recognition unit 226 automatically recognizes a human word from voice data from the microphone 225, and transmits a recognition result to the overall control unit.

FIG. 27 is a flowchart for explaining the processing of the robot apparatus according to one embodiment of the present invention. Referring to FIGS. 17 and 27, a robot apparatus 22 according to an embodiment of the present invention is described.
The first operation example will be described.

The robot apparatus 221 performs the sequential learning of the person image by detecting and identifying the person facing the person and updating the identification dictionary according to the reaction of the interlocutor.

The overall control unit 227 acquires the person identification result from the person detection identification management unit 205 (step S8).
1).

Next, a specific operation is performed for each identified person (step S82). The specific action refers to an entire act of saying a person's name or moving a wheel in a specific direction by a person.

Next, the process waits until the response of the interlocutor is sensed (step S83). When the response of the interlocutor is obtained, it is next determined whether the response is positive or negative (step S84). The positive response means "Yes", for example, when "Yes" is recognized by voice recognition.

The negative response means "No", for example, when "No" is recognized by voice recognition.

Also, for example, a rule of “Yes” when the touch sensor is pressed once and “No” when the touch sensor is pressed twice are determined in advance by the overall control unit, so that the reaction of the interlocutor can be performed using the touch sensor. Can be obtained.

If step S84 is a negative reaction, the operation is repeated from step S81. In the case of a positive response, the facial feature data used for identification is input to the dictionary data management unit 204 (step S8).
5).

Then, an identification dictionary is created and updated (step S86).

If there is an end instruction, the operation is ended. If not, the operation is repeated from step S81 again (step S8).
7).

FIG. 28 is a flow chart for explaining the processing of another embodiment of the robot apparatus of the present invention. With reference to FIGS. 17 and 28, an example of the operation of the robot device 221 according to another embodiment of the present invention will be described. The robot device 221 detects and identifies a facing person, acquires an image of a new person according to a command of the interlocutor, and registers a new person dictionary.

The overall control unit 227 acquires the person identification result from the person detection identification management unit 205 (step S9).
1).

Then, a specific operation is performed for each identified person (step S92). The specific action refers to an entire act of saying a person's name or moving a wheel in a specific direction by a person.

Next, it waits for the command of the interlocutor to be sensed (step S93). When the command event of the interlocutor is obtained, it is determined whether the command event is a registration command (step S94). The registration command event includes, for example, an event in which "small" has been recognized by voice recognition. In addition, by preliminarily determining in the overall control unit a rule that tapping the touch sensor once is a registration event, the registration command event can be uttered using the touch sensor.

When a registration command is received, a registration process is performed.
A new person is registered (step S95). An example of the registration processing step S95 is shown in FIG. 25 and has already been described.

Upon completion of the registration process, the process returns to step S9.
The processing is started from 1. If the registration command event has not arrived, an end determination is made (step S96).

If there is an end command, the process ends. If there is no end command, the process starts again from step S91.

FIG. 35 is a diagram showing a configuration of an embodiment of a person detection / identification system according to the related invention of the present invention.
Referring to FIG. 35, a person detection / identification system according to a related invention of the present invention includes an image acquisition unit 2 and a facing distance sensor 5.
, Person detection identification means 501, dictionary data management unit 12
, A person detection identification management unit 502, and a recording medium 503. The recording medium 503 stores a person detection / identification processing program, and may be a magnetic disk, a semiconductor memory, a CD-ROM, or another recording medium.

The person detection / identification processing program is read from the recording medium 503 by the person detection / identification means 501 and the person detection / identification management unit 502, and is read by the person detection / identification means 6 in the above-described embodiment described with reference to FIG. And the same processing as the processing by the person detection identification management unit 13 is executed.

[0240]

As described above, according to the present invention,
The following effects are obtained.

The person identification device used in the robot apparatus of the present invention has an effect that a person can be identified with an extremely high identification rate even in an environment where lighting conditions fluctuate greatly, such as a home environment. Play.

The reason is as follows. That is,
This is because, in the present invention, detection and identification are performed using not only the shading value of the image but also the movement and the facing distance information.
In addition, in the present invention, this is because an accurate frontal face is detected by the face positioning means. further,
This is because, in the present invention, the face discrimination means performs the similarity discrimination using the linear discrimination dictionary.

In the present invention, a person (user)
This is because the device or the system learns itself through the dialogue with the user to improve the identification accuracy.

Further, according to the robot apparatus of the present invention, even if an input image cannot be identified at a certain place due to a defect, after that,
By acquiring a good image by operating the device itself,
be able to.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an embodiment of a person identification device used in a robot device of the present invention.

FIG. 2 is a block diagram showing a configuration of an embodiment of a head detecting and tracking means in the person identification device used in the robot device of the present invention.

FIG. 3 is a block diagram showing the configuration of another embodiment of the head detection and tracking means in the person identification device used in the robot device of the present invention.

FIG. 4 is a block diagram showing a configuration of an embodiment of a monocular vision head rectangular coordinate detecting means in the human identification device used in the robot device of the present invention.

FIG. 5 is a diagram for explaining a head detection process in the person identification device used in the robot device of the present invention.

FIG. 6 is a diagram for explaining left and right image matching processing in the person identification device used in the robot device of the present invention.

FIG. 7 is a diagram for explaining a head tracking process in the person identification device used in the robot device of the present invention.

FIG. 8 is an explanatory diagram of a raster scan when converting a frontal face image into feature data in a person identification device used in the robot device of the present invention.

FIG. 9 is a block diagram showing a configuration of an embodiment of a front face alignment unit in the person identification device used in the robot device of the present invention.

FIG. 10 is a block diagram showing a configuration of an embodiment of a front face searching means in the person identification device used in the robot device of the present invention.

FIG. 11 is a block diagram showing the configuration of another embodiment of the front face search means in the person identification device used in the robot device of the present invention.

FIG. 12 is a diagram for explaining a head detection process and a face registration process in the person identification device used in the robot device of the present invention.

FIG. 13 is a block diagram showing a configuration of each embodiment of a face identification unit, an identification dictionary storage unit, and an identification result correction unit in the person identification device used in the robot device of the present invention.

FIG. 14 is a block diagram showing a configuration of an embodiment of a dictionary data management unit in the person identification device used in the robot device of the present invention.

FIG. 15 is a block diagram showing a configuration of each embodiment of a face identification unit, an identification dictionary storage unit, and an identification result correction unit in the person identification device used in the robot device of the present invention.

FIG. 16 is a diagram showing a configuration of an embodiment of the robot apparatus of the present invention.

FIG. 17 is a diagram showing a configuration of another embodiment of the robot apparatus of the present invention.

FIG. 18 is a flowchart illustrating a person detection process and an identification process in the person identification device used in the robot device of the present invention.

FIG. 19 is a flowchart for explaining head detection tracking processing in the person identification device used in the robot device of the present invention.

FIG. 20 is a flowchart illustrating a head detection process in the person identification device used in the robot device of the present invention.

FIG. 21 is a flowchart for explaining front face alignment processing in a person identification device used in the robot device of the present invention.

FIG. 22 is a flowchart illustrating a front face search process in the person identification device used in the robot device of the present invention.

FIG. 23 is a flowchart illustrating a front face search process in the person identification device used in the robot device of the present invention.

FIG. 24 is a flowchart for explaining face identification processing in a person identification device used in the robot device of the present invention.

FIG. 25 is a flowchart illustrating a person dictionary registration process in the person identification device used in the robot device of the present invention.

FIG. 26 is a flowchart for explaining behavior control in the robot device of the present invention.

FIG. 27 is a flowchart for explaining behavior control in the robot device of the present invention.

FIG. 28 is a flowchart for explaining behavior control in the robot device of the present invention.

FIG. 29 is a block diagram showing a configuration of an embodiment of a linear discrimination dictionary creating means in the person identification device used in the robot device of the present invention.

FIG. 30 is a block diagram showing a configuration of an embodiment of an eigenvector dictionary creating means in the person identification device used in the robot device of the present invention.

FIG. 31 is a diagram for explaining a method of creating a linear discriminant dictionary used for a person identification process used in the robot apparatus of the present invention.

FIG. 32 is a diagram for describing a face-to-face distance calculation method by stereo vision used in a person identification process used in the robot apparatus of the present invention.

FIG. 33 is an explanatory diagram of an eigenvector projection distance dictionary used for a person identification process used in the robot apparatus of the present invention.

FIG. 34 is an explanatory diagram of a linear discrimination dictionary used for a person identification process used in the robot apparatus of the present invention.

FIG. 35 is a block diagram showing a configuration of another embodiment of the person identification device used in the robot device of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Person detection identification means 2 Video acquisition means 3 Right camera 4 Left camera 5 Face-to-face distance sensor 6 Head detection tracking means 7 Front face alignment means 8 Face feature extraction means 9 Face identification means 10 Identification dictionary storage means 11 Identification result correction Means 12 Dictionary data management unit 13 Person detection identification management unit 14 Person identification device 21 Head detection unit 22 Head tracking unit 23 Head rectangle storage unit 24 Monocular head rectangular coordinate detection unit 25 Left / right image comparison unit 26 Face-to-face distance evaluation Means 27 Head detection and tracking means 28 Right camera 29 Left camera 30 Face-to-face distance sensor 31 Face-to-face distance integration means 32 Head detection and tracking means 33 Head detection means 34 CCD camera 41 Head rectangular coordinate detection means 42 Motion pixel detection means 43 People Number evaluation means 44 Head coordinate detection means 45 Head lower coordinate detection means 46 Temporal coordinate detection means 4 Moving area width 48 Integrated differential image G 49 Integrated differential image G 50 Integrated differential image upper area 51 Head detection rectangle 52 Search result 53 Head rectangular coordinate of previous frame 54 Search locus 55 Head rectangular image of previous frame 61 Front face position Matching means 62 head rectangular cutout means 63 front face search means 64 standard face dictionary data storage means 65 front face likeness determination means 66 density variance determination means 67 threshold processing means 71 front face search means 72 front face candidate extraction means 73 intermediate reduced image storage means 74 contrast correction means 75 eigenvector projection distance calculation means 76 standard face dictionary storage means 77 storage means 78 minimum projection distance determination means 79 search range end determination means 80 standard face average data storage means 81 standard face eigenvector data storage Means 82 Mean difference means 83 Vector projection calculation Output means 84 Reconstruction calculation means 85 Projection distance calculation means 86 Projection distance minimum value storage means 87 Front face density value storage means 88 Head intermediate size calculation means 89 Head rectangular image storage means 90 Image reduction means 91 Intermediate size storage means 92 Multi-resolution processing end determination means 101 Front face search means 102 Product-sum operation means 103 Maximum similarity determination means 104 Standard face dictionary data storage unit 105 Storage means 106 Maximum similarity value storage means 107 Front face gray value storage means 111 Face identification means 112 identification dictionary storage means 113 identification result correction means 114 identification result weighted average calculation means 115 feature data storage means 116 product sum operation means 117 threshold processing means 118 maximum similarity person determination means 119 registered person identification dictionary storage means 121 dictionary Data management unit 122 Individual feature data management unit 123 identification dictionary generating means 124 selector 125 person-specific feature data 126 linear discriminating dictionary creating means 127 eigenvector dictionary creating means 131 face discriminating means 132 identification dictionary storing means 133 others discriminating means 134 others discriminating dictionary storing means 151 head rectangle 152 head Rectangular image 153 Reduced head rectangular image 154 Front face search shape 155 Front face image 201 Robot device 202 CCD camera 203 Person detection / identification unit 204 Dictionary data management unit 205 Person detection / identification management unit 206 Overall control unit 207 Speaker 208 Robot moving unit 209 Motor 210 Wheel 221 Robot device 222 Touch sensor 223 Face-to-face distance sensor 224 Person detection and identification unit 225 Microphone 226 Voice recognition unit 227 Overall control unit 311 Linear discrimination dictionary creation unit 312 Collection X storage means 313 variance-covariance matrix Cxx calculation means 314 inverse matrix conversion means 315 matrix multiplication means 316 constant term calculation means 317 objective variable Y storage means 318 objective variable Y generation means 319 covariance matrix Cxy calculation means 320 coefficient storage means 331 eigenvector dictionary creation means 332 feature quantity storage means 333 feature quantity average calculation means 334 variance-covariance matrix calculation means 335 eigenvector calculation means 336 coefficient storage means 341 personal feature data X 342 objective variable Y 343 feature data average value 344 objective variable average value 401 Right camera 402 Left camera 403 Object 411 First eigenvector 412 Second eigenvector 413 Pth eigenvector 414 Average 421 Constant term 422 Class 1 identification coefficient 423 Class 2 identification coefficient 424 Scan q identification coefficients 425 product terms 501 person detecting identification processor 502 human detection identification management unit 503 recording medium

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06T 1/00 340 G06T 1/00 340A 7/20 100 7/20 100 7/60 150 7/60 150J 150P G10L 15/00 G10L 3/00 551H F-term (reference) 3C007 AS36 CS08 KS07 KS08 KT02 KT11 KV18 WA16 WB19 WB22 WC01 WC07 5B057 AA05 BA02 CA12 CA16 DA06 DA11 DB02 DC01 DC36 5D015 KK01 5L096 BA05 CA05 EA01

Claims (27)

[Claims] An image acquisition unit for acquiring an image; a head detection tracking unit for detecting a human head from the acquired image information; and a frontal face image from a partial image of the detected head. Front face alignment means for obtaining, face feature extraction means for converting a front face image into feature quantities, face identification means for identifying a person from feature quantities using an identification dictionary, and identification dictionary storage means for storing the identification dictionary And a person control device that controls the operation of the robot; a voice output unit that utters a voice according to an instruction from the general control unit; and a moving unit that moves the robot according to an instruction from the general control unit. A robot device comprising: 2. The robot apparatus according to claim 1, wherein the overall control unit controls to speak with a different voice for each person when a person identification result is obtained. 3. The system according to claim 2, wherein the overall control unit acquires a facing distance and a direction from a front object from the person identifying device, a unit acquiring a person identifying result, and Means for moving so as to approach the front object, and means for controlling when the facing distance is equal to or less than a threshold value so that the person identification result is spoken with a different voice for each person. The robot device according to claim 2, wherein 4. A video acquisition unit for acquiring an image, a head detection tracking unit for detecting a human head from the acquired image information, and a front face image from a partial image of the detected head. Front face alignment means for obtaining, face feature extraction means for converting a front face image into feature quantities, face identification means for identifying a person from feature quantities using an identification dictionary, and identification dictionary storage means for storing the identification dictionary A robot apparatus comprising: a person identification device including: a face-to-face distance sensor that measures a distance to a forward object; a touch sensor; voice input means; and voice recognition means. 5. The robot apparatus according to claim 4, wherein a process of generating a negative or positive reaction event based on the recognition result of the voice recognition and the output result of the touch sensor, A process of acquiring an image of a person and creating a dictionary in the dictionary data management unit;
A robot device comprising: 6. A robot apparatus according to claim 4, wherein a means for generating a new person registration event based on a recognition result of voice recognition and an output result of said touch sensor, Means for acquiring an image of a person and creating a dictionary in a dictionary data management unit. 7. The robot apparatus according to claim 1, wherein the human identification device outputs the identification result output from the face identification means for the past N frames (N is an integer of 2 or more). A robot apparatus further comprising a weighted average identification result correction unit. 8. The robot device according to claim 1, wherein the person identification device further includes a dictionary data management unit that generates a linear discrimination dictionary used for person identification. Robotic device to do. 9. The human identification device of the robot device according to claim 1, wherein said head detection and tracking means detects a head rectangular area from an image of one frame. A robot comprising: a head rectangle storage unit for storing a head rectangle image of a previous frame and coordinates; and a head tracking unit for performing matching processing of the head rectangle image of the previous frame with the current frame. apparatus. 10. The human identification device of the robot device according to claim 9, wherein the image acquisition unit includes two left and right imaging units, and the head detection unit that detects a rectangular head region from an image of one frame. A monocular head rectangular coordinate detecting means for detecting the head from one image; a left and right image comparing means for comparing the left and right images to calculate a facing distance value; the facing distance value and the head rectangular area A face-to-face distance evaluation means for removing erroneous detection of the head from the coordinate values of (b). 11. The human identification device of the robot device according to claim 9, further comprising a face-to-face distance sensor, wherein the head detecting means for detecting a rectangular region of the head from one frame image comprises: A monocular vision head rectangular coordinate detecting means for detecting a part, a facing distance value which is an output value of the facing distance sensor, and a facing distance evaluation means for removing false detection of a head from the coordinate values of the head rectangular area A robot device comprising: 12. The human identification device of the robot device according to claim 9, wherein the image acquisition means includes two left and right image pickup means, further comprises a facing distance sensor, and a head rectangular area from one frame image. A monocular vision head rectangular coordinate detecting means for detecting a head from one image; a left and right image matching means for calculating a facing distance value by comparing left and right images; A face-to-face distance integrating unit that calculates an integrated face-to-face distance from the face-to-face distance value calculated from the image and the output of the face-to-face distance sensor; and the integrated face-to-face distance value and the coordinate values of the head rectangular area. A face-to-face distance evaluation means for removing erroneous detection of the part; 13. The human identification device of the robot device according to claim 10, wherein the monocular vision head rectangular coordinate detecting means performs processing for generating a difference image g from a previous frame. The difference image g is stored in the past m frames (m
(Wherein is an integer of 2 or more). A robot apparatus comprising motion pixel detecting means for performing a process of integrating and generating an integrated difference image G. 14. The human identification device of a robot device according to any one of claims 10, 11 and 12, wherein the monocular visual head rectangular coordinate detecting means performs a process of generating a difference image g from a previous frame. Motion pixel detecting means for integrating the difference image g in the past m frames (where m is an integer of 2 or more) to generate an integrated difference image G; and person number evaluation means for obtaining the number of people from the integrated difference image G And a head coordinate detecting means for obtaining the Y coordinate of the head, a lower head coordinate detecting means for detecting the Y coordinate of the lower head, and a temporal coordinate detecting means for detecting the X coordinate of the temporal region. A robot device characterized by the above-mentioned. 15. The human identification device of the robot device according to claim 1, wherein the front face alignment unit uses the face distance value and the image of the head rectangle to form a front face. A robot device comprising a front face searching means for searching for a face. 16. The human identification device of the robot device according to claim 15, wherein the front face positioning means obtains a density variance of the front face image obtained by the front face searching means, and calculates the density variance by the density variance value. Density variance determining means for eliminating false detection of the front face, and comparing the distance or similarity between the patterns obtained by the front face searching means with a predetermined threshold value, thereby detecting false detection of the front face. A robot device comprising: threshold processing means for eliminating; 17. The human identification device of the robot device according to claim 15, wherein the front face searching means calculates an intermediate reduced size from the facing distance value, and the head intermediate size calculating means; Image reduction means for reducing the image of the image to the head intermediate size, means for correcting the contrast by normalizing the density, standard face eigenvector data storage means for storing eigenvector data of the front face feature amount, and the front face A standard face average data storage unit that stores an average value of the feature amount; an eigenvector projection distance calculation unit that calculates an eigenvector projection distance; and a projection distance minimum determination unit that obtains a minimum value of the eigenvector projection distance. Projection distance calculating means, average difference means, vector projection value calculating means, reconstruction calculating means, projection distance meter Robot apparatus characterized by comprising: a means. 18. The human identification device of the robot device according to claim 15, wherein the front face searching means calculates an intermediate reduced size from the facing distance value, and the head intermediate size calculating means. Image reducing means for reducing the image of the image to the head intermediate size, means for correcting the contrast by normalizing the density, standard face dictionary data storing means for storing a linear discrimination dictionary of front face feature amounts, A robot apparatus comprising: a product-sum operation unit that calculates a similarity using a discrimination dictionary; and a similarity maximum determination unit that obtains a maximum value of the calculated similarity. 19. The human identification device of a robot device according to claim 1, wherein the face identification means calculates a similarity to a registered person from a linear discrimination dictionary and feature data. Sum-of-products calculating means, maximum similarity person determining means for obtaining the maximum value of the similarity to a registered person, and comparing the maximum value of the similarity with a predetermined threshold to determine whether the person is another person. And a threshold value processing means for determining the threshold value. 20. The human identification device of the robot device according to claim 1, wherein the face identification means calculates an eigenvector projection distance from an eigenvector dictionary for identifying another person and feature data. Eigenvector others discriminating means for comparing distance values with each other, product-sum calculating means for calculating a similarity to a registered person from a linear discriminant dictionary and feature data, and a maximum value of similarity to a registered person. A robot comprising: a maximum similarity person determination unit to be obtained; and a threshold processing unit that compares the maximum value of the similarity with a predetermined threshold value to determine whether the person is another person. apparatus. 21. The personal identification device of a robot device according to claim 8, wherein the dictionary data management unit stores personal characteristic data serving as a basis of an identification dictionary. Means, selecting means for storing feature data in a specific area of the personal feature data storage means by a person ID (identification) number, and linear discriminating dictionary creating means for creating the linear discriminating dictionary. Robot device. 22. The personal identification device of the robot device according to claim 8, wherein the dictionary data management unit stores personal characteristic data based on an identification dictionary. Means, selecting means for storing feature data in a specific area of the personal feature data storage means by a person ID number, linear discriminating dictionary creating means for creating the linear discriminating dictionary, eigenvector dictionary creating means for creating an eigenvector dictionary, A robot device comprising: 23. The human identification device of the robot device according to claim 21 or 22, wherein the linear discriminant dictionary creating means calculates a variance-covariance matrix Cxx of the feature quantity X stored in the personal feature data storage means. Means for calculating a variance-covariance matrix Cxx, an inverse matrix calculating means for calculating an inverse matrix of the variance-covariance matrix Cxx, means for generating an objective variable Y from a person ID and feature data, a feature amount X and an objective variable Y A variance-covariance matrix Cxy calculating means for calculating a covariance matrix Cxy of: a matrix multiplying means for multiplying the inverse matrix of Cxx and the covariance matrix Cxy to calculate a multiplication term matrix of a linear discriminant dictionary; And a constant term calculating means for calculating a constant term vector. 24. The human identification device of the robot device according to claim 22, wherein the eigenvector dictionary creating means comprises: a feature quantity average calculating means for averaging the feature quantity X; and a variance-covariance matrix of the feature quantity X. A robot apparatus comprising: variance-covariance matrix calculation means for calculating Cxx; and eigenvector calculation means for calculating an eigenvector of the variance-covariance matrix Cxx. 25. The robot apparatus according to claim 12, wherein a coordinate Xl of the same object on a left camera image, a coordinate Xr on a right camera image, and a constant K are used to obtain Z = K / (X1-X
r) A robot apparatus provided with left and right image matching means including a process of obtaining a facing distance Z by calculation. 26. The human identification device of the robot device according to claim 12, wherein the head intermediate size calculating means is: When, (However, Hw and Hh are the horizontal size and the vertical size of the head rectangular image, RHw and RHh are the horizontal size of the front face area, the vertical size, Z is the facing distance value, e is the width of the imaging surface, W is the image size, θ is the angle of view of the camera).
A robot device comprising means for obtaining w and Mh. 27. The front face search means in the person identification device of the robot device according to claim 15, wherein: (Where V is the original image, μ is the average of the pixel values, σ is the standard deviation, and Vmax is the maximum value of the pixel values), and the robot apparatus performs contrast correction.