WO2002007095A1 - Dispositif de representation en 3d du visage et dispositif de reconnaissance peripherique comprenant ce dernier - Google Patents

Dispositif de representation en 3d du visage et dispositif de reconnaissance peripherique comprenant ce dernier Download PDF

Info

Publication number
WO2002007095A1
WO2002007095A1 PCT/JP2000/004795 JP0004795W WO0207095A1 WO 2002007095 A1 WO2002007095 A1 WO 2002007095A1 JP 0004795 W JP0004795 W JP 0004795W WO 0207095 A1 WO0207095 A1 WO 0207095A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
dimensional direction
person
feature point
tracking
Prior art date
Application number
PCT/JP2000/004795
Other languages
English (en)
Japanese (ja)
Inventor
Kentaro Hayashi
Manabu Hashimoto
Kazuhiko Sumi
Original Assignee
Mitsubishi Denki Kabushiki Kaisha
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Denki Kabushiki Kaisha filed Critical Mitsubishi Denki Kabushiki Kaisha
Priority to PCT/JP2000/004795 priority Critical patent/WO2002007095A1/fr
Publication of WO2002007095A1 publication Critical patent/WO2002007095A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the present invention relates to a technology for sequentially estimating the orientation of a person's face from an input image using an image of a person's face obtained from an imaging device as an input image, and particularly relates to a time-series method using a single imaging device.
  • the present invention relates to a face three-dimensional direction tracking device for sequentially and reliably estimating the direction of a human face from the input image obtained in step (1).
  • the present invention relates to a peripheral recognition device that can recognize and recognize dangers that the user does not notice and notify the user.
  • an image including a face region of an operating person is input using an imaging device, the position of the facial feature point of the operating person is specified from the input image, and the direction of the operating person's face is estimated using the input image.
  • a device there was a device disclosed in Japanese Patent Application Laid-Open No. 9-147711 as shown in a block diagram in FIG. 18.
  • This device uses a color video camera and a digital video camera.
  • Image input unit 101 and image input unit 101 which consist of an A / D converter that converts the image into a color image that includes the face area of the person who operates the device (for example, a personal computer).
  • Image storage unit 102 a human face region extraction unit 103 that extracts a human face region from the color image data (input image) stored in the image storage unit 102, image storage unit 10 Extraction of the human hair region from the color image stored in 2 in the person's hair region extraction unit 104 and extraction of the face region and the person's hair region extracted by the human face region extraction unit 103
  • a human head region extraction unit 105 that extracts a person's head region from the hair region extracted by the unit 104 and a facial feature point in the face region extracted by the human face region extraction unit 103
  • the first facial feature point extraction unit 106 that extracts the mouth region as the first facial feature point extraction unit 106, and the second facial feature point that extracts the eye region as the facial feature point in the extracted facial region
  • the extraction unit 107 and the head region extracted by the human head region extraction unit 105 and the first and second facial feature point extraction units 106 and 107 are extracted.
  • Direction calculation unit 108 that calculates the direction of the face from the positional relationship between the two types of facial feature points, and person operation state calculation that estimates the operation state of a person from the change state of the face direction obtained by the direction calculation unit 108
  • an output unit 110 that outputs information indicating the operation state of the person estimated by the person operation state calculation unit 109 to a personal computer or the like as a graphical user interface device. It is configured.
  • the digital image data of 8 bits each of RGB input by the image input unit 101 every several frames is temporarily stored in the image storage unit 102.
  • the human face region extraction unit 103, the human head hair region extraction unit 104, and the human head region extraction unit 105 are applied respectively. In this way, the position of the center of gravity of the face region image and the head region image of the person, and the inclination angle of the head in the projection plane are extracted.
  • the original face In addition to detecting the position of the organ inside the face that is considered to be located in front (the mouth is used as the first feature point and the eyes are used as the second feature point) on the projection plane, Calculate the position of the center of gravity.
  • the direction calculation unit 108 calculates the face direction from the barycentric position of the head calculated as described above, the tilt angle in the plane, and the barycentric position of the feature point, and furthermore, the human operation
  • the state calculation unit 109 obtains a temporal statistical distribution in the direction of the face obtained at each time, and calculates the degree of attention of the person and the state of the operated person. , Etc.).
  • This estimation result is output by the output unit 110 at regular intervals by the output unit 110 and the data in which the calculated vector in the face direction is weighted according to the degree of attention, and the data on the operation state ( Is output to a personal computer, etc.
  • the present invention has been made in order to solve the above-mentioned problems, and it is possible to stably perform three-dimensional facial expression even when feature points on the face cannot be tracked, such as when the face is turned horizontally or vertically.
  • the purpose is to provide a face three-dimensional direction tracking device that can estimate the direction. Disclosure of the invention
  • a first face three-dimensional direction tracking apparatus takes a face image of a person captured in time series as an input image and sequentially estimates the face direction of the person from the input image. hand, Estimating the three-dimensional direction of the face when the direction of the face of the person is in an angle range up to 0i based on the direction of the face of the person in the input image obtained by capturing the state in which the person is facing the front
  • a first three-dimensional direction estimating means wherein the direction of the face of the person is determined based on the direction of the face of the person in the input image obtained by capturing the state in which the person is facing the front;
  • a second three-dimensional direction estimating means for estimating a three-dimensional direction of the face when in 'an angular range of up to larger 0 3 than below theta 2 or al the 6> i,
  • Output selecting means for selecting the output of the first and second three-dimensional direction estimating means using at least the previous three-dimensional direction estimating result of the face.
  • the face can be estimated by the first three-dimensional direction estimating means, for example, when the face is turned sideways or vertically, it can be estimated by the second three-dimensional direction estimating means. . Therefore, the 3D direction of the face can be estimated over a wide range.
  • the three-dimensional direction tracking device for a second face is the three-dimensional direction tracking device for a first face
  • the second three-dimensional when the orientation of the face has been previously estimated at 0 E above selects the output of the direction estimation means, if the orientation of the face has been previously estimated is less than 0 greater than 0 2, the first three-dimensional in the case where the direction of the face estimated up to the previous is decreasing
  • a third face three-dimensional direction tracking device is the first face three-dimensional direction tracking device
  • the first three-dimensional direction estimating means uses feature points of a person's face as feature points.
  • a face feature point tracking means for detecting at least three feature point positions from the input image
  • the direction of the face of the person is estimated from the positional relationship of at least three feature points detected by the feature point tracking means of the face.
  • the direction of the face can be accurately estimated.
  • the fourth face three-dimensional direction tracking device is the third face three-dimensional direction tracking device
  • the second three-dimensional direction estimating means is that the direction of the face of the person from the input image is ⁇
  • the three-dimensional direction of the person's face is estimated using a reference image.
  • a fifth face three-dimensional direction tracking device is the third face three-dimensional direction tracking device
  • a deformation degree estimating means for estimating a deformation degree on the image
  • a normalized deformation degree detecting means for detecting the deformation degree when the image is normalized by the enlargement ratio or the reduction ratio among the deformation degrees.
  • the first three-dimensional direction estimating means includes a means for detecting the normalized deformation degree.
  • the three-dimensional direction of the face is estimated from the detected degree of deformation.
  • the sixth face three-dimensional direction tracking device is the third face three-dimensional direction tracking device
  • the apparatus further includes tracking feature point setting means for selecting a new feature point of the person according to the direction of the face of the person estimated by the first three-dimensional direction estimating means.
  • the first face three-dimensional direction estimating means 2 checks whether or not there is a new feature point that can be tracked even if the tracked feature point cannot be tracked. If the feature point is set as a new tracking target, not only will the feature point be unable to be tracked, but also the first three-dimensional direction estimation means for the first face will detect the face of the person over a wider area. Be able to estimate.
  • a seventh face three-dimensional direction tracking apparatus is the third face three-dimensional direction tracking apparatus
  • a change in the position of the feature point of the person's face obtained from the face feature point tracking means is investigated, and if the change in the position is smaller than a predetermined threshold, the time required for the change in the position is accumulated.
  • State detection means and
  • the preset time is updated to the accumulated time, and the predetermined threshold is set to the minimum value of the amount of change in the position changed for each accumulated time.
  • a stability tracking means having setting means for updating.
  • the first peripheral recognition device according to the present invention
  • An image of a face of a person captured in time series is used as an input image, and the orientation of the face of the person is sequentially estimated from the input image.
  • the input image in which the person is facing the front is captured.
  • the output of the first and second three-dimensional direction estimating means is selected using the second three-dimensional direction estimating means for estimating the three-dimensional direction of the face at the time, and at least the previous three-dimensional direction estimating result of the face.
  • 3D face tracking device with output selection means
  • An inattentive area detection means for detecting an inattentive area of the person from the three-dimensional direction of the face obtained by the output selecting means;
  • Outside information acquisition area setting means for setting an area for acquiring information on the outside world corresponding to the attention area detected by the attention area detection means;
  • Peripheral recognition means for recognizing the situation of the outside world based on the outside world information acquired by the outside world sensor;
  • Notification means for notifying according to the judgment of the danger judgment means.
  • FIG. 1 is a block diagram showing the configuration of a face three-dimensional direction tracking apparatus according to Embodiment 1 of the present invention
  • FIG. 2 explains the operation of the face three-dimensional direction tracking apparatus according to Embodiment 1 of the present invention.
  • FIG. 3 is a block diagram showing a configuration of a face feature point tracking means used in the first embodiment of the present invention.
  • FIG. 4 is a face feature point tracking means used in the first embodiment of the present invention.
  • FIG. 5 is a flow chart for explaining the operation of FIG. 5,
  • FIG. 5 is a block diagram showing a configuration of a face three-dimensional direction tracking device according to Embodiment 2 of the present invention, and
  • FIG. 6 is a face diagram according to Embodiment 2 of the present invention.
  • FIG. 1 is a block diagram showing the configuration of a face three-dimensional direction tracking apparatus according to Embodiment 1 of the present invention
  • FIG. 2 explains the operation of the face three-dimensional direction tracking apparatus according to Embodiment 1 of
  • FIG. 7 is a flowchart for explaining the operation of the three-dimensional direction tracking device.
  • FIG. 7 is a block diagram showing the configuration of the three-dimensional direction tracking device for a face according to the third embodiment of the present invention.
  • FIG. 8 is an embodiment of the present invention. 3 to explain the operation of the 3D face tracking device
  • FIG. 9 is a block diagram showing a configuration of a face three-dimensional direction tracking apparatus according to Embodiment 4 of the present invention.
  • FIG. 10 is an operation of a face three-dimensional direction tracking apparatus according to Embodiment 4 of the present invention.
  • 11 is a block diagram showing a configuration of a face three-dimensional direction tracking apparatus according to Embodiment 5 of the present invention, and
  • FIG. 12 is a face chart according to Embodiment 5 of the present invention.
  • FIG. 13 is a flowchart for explaining the operation of the three-dimensional direction tracking apparatus of FIG. 13.
  • FIG. 13 is a block diagram showing the configuration of the stability tracking means used in the fifth embodiment of the present invention.
  • FIG. FIG. 15 is a flowchart for explaining the operation of the stability tracking means used in Embodiment 5 of the present invention.
  • FIG. 15 is a flowchart for explaining the operation of the face three-dimensional direction tracking apparatus according to Embodiment 6 of the present invention.
  • FIG. 16 is a block diagram showing a configuration of a peripheral recognition device according to Embodiment 7 of the present invention.
  • FIG. 3 is a block diagram showing the configuration of FIG. BEST MODE FOR CARRYING OUT THE INVENTION
  • FIG. 1 is a block diagram showing a configuration of a face three-dimensional direction tracking apparatus according to Embodiment 1 of the present invention
  • FIG. 2 is a diagram illustrating an operation of the face three-dimensional direction tracking apparatus according to Embodiment 1 of the present invention.
  • reference numeral 1 denotes an imaging unit for inputting an image of a person's face, for example, an imaging device such as a CCD camera.
  • the three-dimensional direction estimating means 2 for the first face and the three-dimensional direction estimating means 3 for the second face use the image of the face of the person captured in time series from the imaging means 1 as an input image. Is what you do.
  • Reference numeral 2 denotes a first three-dimensional direction estimating means, which uses the input image obtained by the imaging means 1 as a reference (that is, 0) based on the orientation of a person's face in an image obtained by imaging a state in which the person is facing the front.
  • the 3D direction of the face is estimated when the direction of the face of the person (3D direction) is in the angle range from 0 to 0i.
  • 3 is a second three-dimensional direction estimating means, from the input image picture obtained by the image pickup means 1, 0 3 greater than the smaller 0 2 than the orientation (three-dimensional directions) is 0 i of the face of a person The 3D direction of the face in the range of angles
  • Reference numeral 4 denotes an output selecting means for selecting and outputting one of the outputs of the first three-dimensional direction estimating means 2 and the second three-dimensional direction estimating means 3; direction is S when 2 or less selects the output of the first three-dimensional direction estimation unit 2, the orientation of the face 6 which is previously estimated> ⁇ If this is the case, the output of the second three-dimensional direction estimating means 3 is selected, and if the previously estimated face direction is greater than 6> 2 and less than 0 i, the face direction estimated up to the previous time If the direction is decreasing, the output of the first three-dimensional direction estimating means 2 is selected, and if the direction of the face estimated up to the previous time is increasing, the second three-dimensional direction estimating means 3 is selected. Is selected.
  • Reference numeral 5 denotes a part of the first three-dimensional direction estimating means 2, which uses a characteristic portion of a person's face as a feature point and detects at least three feature point positions for each input image. This is a feature point tracking means for tracking changes in the position of the face.
  • the sixth is a part of the first three-dimensional direction estimating means 2, which is a means for estimating the three-dimensional direction of the face from the positional relationship of the feature points obtained by the feature point tracking means 5 of the face. From the positional relationship of at least three feature points obtained from the feature point tracking means 2, the three-dimensional direction of the human face in the input image is estimated.
  • Reference numeral 7 denotes a specific face direction detecting means for roughly determining in which direction the human face image obtained by the imaging means 1 faces, and more specifically, a specific direction. It detects the image of the face of the person.
  • Numeral 8 denotes a means for converting the face into a three-dimensional direction.
  • the face direction of the person detected by the specific face direction detecting means 7 and the image of the person obtained by the imaging means 1 are used to convert the person in the input image. This is to calculate the face direction more accurately.
  • the means for converting the face into the three-dimensional direction discretely estimates the three-dimensional direction of the face using, for example, a plurality of reference images whose three-dimensional directions are known from the output of the specific face direction detecting means 7 and the input image.
  • the first and second three-dimensional direction estimating means 2 and 3 of the face and the output selecting means 4 are realized by, for example, a computer.
  • the three-dimensional direction tracking device for the face configured as described above is, for example, shown in FIG.
  • the operation can be performed in the order shown in the flowchart.
  • the first three-dimensional direction estimation means 2 is 0 degrees 6
  • the three-dimensional direction is estimated from the face near the front when the angle is up to 0 degrees
  • the second 3D direction estimating means 3 is used to estimate the three-dimensional direction from the profile when the angle is in the range from 50 degrees to 90 degrees.
  • the estimation of the three-dimensional direction of the face in the case of estimating the three-dimensional direction will be described.
  • the above-described numerical values are for illustrative purposes only, and are not limited to these numerical values.
  • the face is positioned at a specific angle within the measurement range of the second three-dimensional direction estimating means 3 (for example, (70 °) Face turned sideways Images obtained by taking images in advance, etc., and the faces turned 50 °, 60 °, 80 ° and 90 ° sideways An image is generated by transforming the 70-degree face image using knowledge of the general three-dimensional structure of the face. By doing so, it is possible to reduce the time and effort required for the user to capture the state in which the user is oriented at 50 degrees, 60 degrees, 80 degrees, and 90 degrees.
  • a specific angle within the measurement range of the second three-dimensional direction estimating means 3 for example, (70 °) Face turned sideways Images obtained by taking images in advance, etc., and the faces turned 50 °, 60 °, 80 ° and 90 ° sideways
  • An image is generated by transforming the 70-degree face image using knowledge of the general three-dimensional structure of the face. By doing so, it is possible to reduce the time and effort required for the user to capture the state in which
  • step 2 temporarily store the face image obtained in step 1.
  • step 3 the profile image held in step 2 By acquiring an image and matching the input image with each of the stored profile images, if there is a profile image (face image pointing in a specific direction of 50 to 90 degrees) in the input image, it is detected I do.
  • the detected input image is matched with that of a face image oriented at 70 degrees, a face image oriented at 80 degrees, and a face image oriented at 90 degrees.
  • the face direction is interpolated and complemented with linear weighting to obtain the correct face direction (ie, to convert the face into the three-dimensional direction).
  • Steps 1 to 4 allow you to estimate the three-dimensional direction of the face when the face is facing in the horizontal direction (specific direction).
  • Steps 1 and 2 need only be executed when the face three-dimensional direction tracking apparatus of the first embodiment is operated for the first time, and once the face image is once held in step 2, it is not necessarily executed thereafter.
  • steps 5 to 8 relating to the specific operation of the first three-dimensional direction estimating means 2 will be described.
  • step 5 feature points of a person's face, such as eyes, nose, mouth, and ears, are set as feature points from an input image sequentially captured by the imaging unit 1, and the positions of the feature points are detected. I do. In general, it is necessary to detect three feature points, but if the face changes in a certain direction, such as in the horizontal or vertical direction, the face changes. It may be two that are separated in the direction in which they do.
  • Step 6 for example, a face image when the face is turned to the front is taken in advance, or when the position of the feature point corresponding to the state in which the face is turned to the front is detected in Step 5, By acquiring an image, the position of the feature point when the person is facing the front is held.
  • step 7 the difference between the position of the feature point of the face obtained in step 5 and the position of the feature point of the frontal face held in step 6 is calculated, and the position of the feature point of the current face is calculated. Calculate how much has moved from the frontal face (ie, the amount of movement between feature points).
  • step 8 various deformation amounts (translation, enlargement, distortion, etc.) on the image represented by the entire feature point are calculated from the movement amount between the feature points obtained in step 7. From the amount of deformation on these images, it can be seen that, for example, if the enlargement ratio in the horizontal direction is smaller than 1, that is, if the image is reduced, the face is in the horizontal direction.
  • the face's three-dimensional direction can be calculated from the specific magnification.
  • the 3D direction of the face can be estimated using the positions of the feature points.
  • the angle range estimated by the first three-dimensional direction estimating means 2 is close to the horizontal direction (that is, 50 degrees, which is the overlapping part of the detection ranges of the first and second three-dimensional direction estimating means 2 and 3). If a face image at a specific angle (for example, 60 degrees) is detected, the face image is used as the face image at the specific angle in step 1 described above, and the face is set at 50 degrees. , 70, 80, and 90 degrees, each face image is transformed from the 60-degree face image using knowledge of the general three-dimensional structure of the face. May be generated.
  • step 6 may be performed when the face three-dimensional direction tracking apparatus of the first embodiment is operated for the first time.In step 6, the positions of the characteristic points of the face when facing the front are temporarily held. If so, it does not need to be executed thereafter.
  • the groups of steps 1 to 4 and the groups of steps 5 to 8 can be independently executed, the groups may be executed sequentially or in parallel. If executed in parallel, The processing time can be reduced depending on the configuration of the device that performs the processing.
  • step 4 and the output of step 8 differ in the range of the face angle that can be handled by each means, the three-dimensional direction of the face can be estimated correctly by one means. It is often the case that the three-dimensional direction of the face cannot be estimated correctly using the method described in (1).
  • step 9 the output of step 4 and the output of step 8 are examined.
  • step 9 information on the output used in the previous or previous 3D estimation of the face is checked, and the output of step 4 or the output of step 8 is determined accordingly. .
  • step 4 is adopted if the immediately preceding face direction is close to the horizontal direction, otherwise the output of step 8 is adopted.
  • step 8 that is, if the previous estimated value is 50 degrees or less, the output of step 8 (that is, the first three-dimensional direction estimating means 2) is selected, and the previous estimated value is 60 degrees. If the estimated value is greater than 50 degrees and less than 60 degrees, the output of step 4 (that is, the second three-dimensional direction estimating means 3) is selected. When the estimated value is decreasing (that is, when the face is facing front), the output of step 8 (that is, the first three-dimensional direction estimating means 2) is selected, and the estimated value up to the previous time is increasing. At some point (ie, when the face is turning sideways), the output of step 4 (ie, the second three-dimensional direction estimating means 3) is selected.
  • step 10 information on the output selected in step 9 is stored for use in the next output selection.
  • the tertiary region of the face can be detected in both the frontal region where the feature points can be tracked stably and the lateral region where the feature points cannot be tracked.
  • the original direction can be estimated.
  • the three-dimensional direction of the face can be stably and reliably estimated over a wide range using a single imaging device.
  • FIG. 3 is a block diagram showing the configuration of the face feature point tracking means used in the first embodiment of the present invention
  • FIG. 4 explains the operation of the face feature point tracking means used in the first embodiment of the present invention.
  • FIG. 3 is a flowchart for performing the above.
  • 51 is included when an image of a face is taken from the input image obtained by the imaging means 1 such as a pupil of the eye (black eye) or a nostril of a nose.
  • An invariant feature point having a relatively small variation is regarded as an invariant feature point, and invariant feature point position detecting means for detecting the invariant feature point and detecting its position.
  • the invariant feature point detecting means 51 detects the invariant feature points' (pupils, nostrils), and then calculates and outputs the position of the center point (center of gravity, etc.).
  • 5 2 designates an image pattern including a characteristic part of a person's face stored in advance as a specific pattern, and a specific pattern or an image close to the specific pattern in the image captured. It is a specific pattern position detecting means for detecting a position where the specific pattern exists, and detects a position of a specific pattern on a face such as an eye or a nose.
  • the specific pattern is, for example, a region (such as a rectangle) surrounding the invariant feature points.
  • the position of the specific pattern for example, the position of the center point (such as the center of gravity) of this area is used.
  • 5 3 indicates the position of the characteristic portion of the face of the person in the image captured from the imaging means 1 by the output of the invariant feature point position detecting means 51 and the output of the specific pattern position detecting means 52.
  • An output position integrating means for detecting and storing an image of the characteristic portion of the detected face of the person as a new specific pattern.
  • the invariant feature point position detecting means 51, the specific pattern position detecting means 52, and the output position integrating means 53 are realized by, for example, a computer.
  • FIG. 4 is a diagram for explaining the operation of the face feature point tracking means 5 according to the first embodiment.
  • the invariant feature points are pupils and nostrils
  • the specific pattern is eyes and nose
  • the eyes and nose of a face are tracked.
  • step 51 it is checked whether or not a feature point position has been detected in the previous calculation, and if so, tracking from the input image based on the feature point position (eye and nose position) obtained in the previous calculation. Set the search area where the target feature point should be searched.
  • a rectangular area having a certain size surrounding the eyes and a rectangular area having a certain size surrounding the nose are set.
  • the rectangular area to be searched (referred to as a search rectangular area) is usually a rectangular area of a certain size centered on the feature point position obtained by the previous calculation.
  • the size of this rectangular area may be appropriately determined according to the time interval for capturing the input image, the time interval for detecting the feature points, the speed at which the person moves the face, and the like.
  • the feature point position has not been detected in the previous calculation That is, when detecting feature points for the first time, for example, binarizing the image using a means for binarizing the input image (not shown), detecting the position of the nostril using general knowledge of the face structure, The eye search rectangular area is set based on the position of the detected nostril.
  • step 52 a search area for searching for invariant feature points is set in the search rectangular area set in step 51.
  • a rectangular area centered on the feature point used in step 51 and smaller than the rectangular area set in step 51 is set.
  • the position of an invariant feature point usually exists within a specific pattern, and the size of the region is smaller than that of the specific pattern.
  • the entire search area set in step 51 is converted to an invariant feature point.
  • the position of the invariant feature point can be calculated in a shorter time than when the search area is used as the location.
  • the invariant feature point is a pupil
  • a shape (circle) corresponding to the pupil is set, and a rectangular area including the shape around the feature point is set as a search area for searching for the invariant feature point. It is.
  • the size of the rectangular area for searching for the invariant feature points may be appropriately determined according to the time interval for capturing the input image, the time interval for detecting the invariant feature points, the speed at which the pupil moves, and the like.
  • the search area is set in step 52 based on the relative positional relationship between the invariant feature point position and the specific pattern position obtained by the previous calculation, and the area where the invariant feature point is most likely to exist is statistically determined. Estimation using such a method may be used as a search area for searching for an invariant feature point, with a minimum rectangular area including an area having a sufficiently high probability that an invariant feature point exists.
  • step 53 the invariant feature points set in step 52 are added. Invariant feature points are detected from the search area for searching, and the positions of the invariant feature points are output.
  • invariant feature points for example, the shape of invariant feature points stored in advance (pupil shape for a pupil, nostril shape for a nostril). More specifically, for example, the invariant feature point is detected by detecting where the shape corresponding to the invariant feature point stored in advance exists in the search area set in step 52. The center position of the shape is calculated, and this is output as the position of the invariant feature point.
  • the separation factor pool (not shown) is used in step 5. If an image in the search area set in step 2 is used, it is possible to detect invariant feature points with higher accuracy.
  • the invariant feature point is the pupil, part or all of the pupil will be hidden during the movement while closing the eyes, such as blinking. Therefore, since the shape of the invariant feature point stored in advance is different from the shape of the pupil in the actual input image, the invariant feature point described above is not detected. Therefore, if no invariant feature point is detected in the search area set in step 52, it is determined that blinking is in progress, and processing such as not detecting the position of the invariant feature point is performed. By adding this, the reliability of detection in step 53 can be improved.
  • a position where an invariant feature point such as a pupil or a nostril exists can be detected from an image.
  • Step 54 specifically detects a specific pattern from the search rectangular area set in Step 51 and outputs the center position of the specific pattern.
  • the specific pattern obtained by the previous calculation is used as the template image pattern
  • the specific pattern is detected from the search area using a method such as pattern matching
  • the center position of the detected specific pattern is set to the current position.
  • the pattern match is a method of detecting a specific pattern existing in the 'search area' of the input image by finding a position where the sum of the differences between the specific pattern and the corresponding pixel value of the input image is small.
  • the position where the specific pattern of the eyes and the nose exists can be detected from the image.
  • the position of the invariant feature point obtained in step 53 and the position of the specific point and the angle obtained in step 54 should ideally coincide with each other, but this may not be the case in reality.
  • step 55 for example, the midpoint of the position obtained by step 53 and step 54 is adopted.
  • the positions of the feature points can be obtained stably and robustly unless the positions obtained from both steps 53 and 54 both have large errors.
  • step 55 the midpoint of the two feature points was adopted. Instead, it may be an internal dividing point having a weight that is a position close to the characteristic point position obtained from one of the steps.
  • the feature point position obtained in step 55 is held as feature point position information in step 56, and is prepared for feature point position detection performed at the immediately following time. In this way, it is possible to stably track the positions of the feature points on the face.
  • step 55 The output of step 55 is used in step 56, step 57, and step 59.
  • step 56 the information on the position of the feature point output in step 55 is held and updated, and is used as the position of the feature point in step 51 in the next detection.
  • the search area in step 51 can be set to an appropriate position even if the face moves.
  • step 57 based on the positions of the feature points obtained in step 55, a specific pattern (template) is obtained from the surrounding image information.
  • step 58 the acquired specific pattern is set as a new specific pattern, and the updated specific pattern is set as a specific pattern to be used for detecting the next specific pattern.
  • step 59 the feature point position obtained in step 55 is output, and the process ends.
  • the face feature point tracking operation as described above is performed at each time interval at which the imaging means 51 takes in an image in a time series, for example, at every 1/30 second.
  • the flowchart shown in FIG. 4 is an example, and another flowchart may be used as long as the input / output relationship of each step is appropriate.
  • the position of the center point of the pupil or the nostril based on the position of the invariant feature point detected by the invariant feature point position detection means 52 and the specific pattern position detection means 53
  • the output integration means 54 takes into account the detection values from both the detection means 52 and 53. The value of the feature point is adjusted so that the feature point position far from the true position is not output, and feature points on the face such as eyes and nose can be tracked robustly.
  • the invariant feature point position detecting means 52 and the specific pattern position detecting means 53 detect the invariant characteristic point position and the specific pattern position independently of each other, either one of the detecting means 5 2 (or 5 Even when the detection by 3) is not possible, the detection result by the other detection means 53 (or 52) is obtained, and this is output by the output position integration means 54 as a feature point position.
  • Feature points on the face such as eyes and nose can be tracked stably.
  • the special feature position obtained by the output position integrating means 54, the specific pattern acquired based on this characteristic point position are updated, and the next invariant characteristic point position detecting means 52 or the specific Since the specific pattern position detecting means 54 is used for detection by the pattern position detecting means 53, even if the characteristic portion of the person's face changes with time, it follows this change. Because it is possible to detect the mouth, it is possible to track the mouth bust. Furthermore, a search area is set from the image of the person's face, and the position of a specific pattern is detected from the set search area. It is also possible to shorten the detection time of the specific pattern as compared with the case of performing the operation.
  • FIGS. 3 and 4 are only examples of face feature point tracking means, and the present invention is not limited to this. Either the invariant feature point position detection means 51 or the specific pattern position detection means 52 is used. It may be one.
  • FIG. 5 is a block diagram showing the configuration of a face three-dimensional direction tracking apparatus according to Embodiment 2 of the present invention
  • FIG. 6 explains the operation of the face three-dimensional direction tracking apparatus according to Embodiment 2 of the present invention.
  • the present embodiment is different from the first embodiment in that the output selecting means 4 is disposed at a stage preceding the first three-dimensional direction estimating means 2 and the second three-dimensional direction estimating means 3.
  • 6 0 ° 0 i is next as an example, 0 2 5 0 ° horizontal, 0 3 is 9 0 degrees laterally, the first three-dimensional directions estimation means 2 Is in the angle range from 0 to 60 degrees, the 3D direction is estimated from the face near the front, and the second 3D direction estimation means 3 is in the angle range from 50 to 90 degrees
  • the estimation of the three-dimensional direction of the face when estimating the three-dimensional direction from the profile at the time will be described.
  • step 11 it is held by step 10 described later.
  • step 10 Using the previously obtained 3D direction information of the face, if the previous face direction is close to horizontal, go to step 1; otherwise, go to step 5. That is, in this embodiment, if the previous estimated value is 50 degrees or less, the process proceeds to step 5 (that is, the first three-dimensional direction estimating means 2), and if the previous estimated value is 60 degrees or more. Goes to step 1 (i.e., the second three-dimensional direction estimating means 3) . If the previous estimated value is larger than 50 degrees and smaller than 60 degrees, the estimated value up to the previous time tends to decrease.
  • step 5 that is, the first three-dimensional direction estimating means 2
  • step 1 the second three-dimensional direction estimating means 3
  • step 4 and step 8 are retained in step 10 and used as information for the next three-dimensional direction estimation and output as the current estimation result.
  • FIG. 7 is a block diagram showing a configuration of a face three-dimensional direction tracking device according to a third embodiment of the present invention
  • FIG. 8 illustrates an operation of the face three-dimensional direction tracking device according to a third embodiment of the present invention.
  • reference numeral 21 denotes a degree-of-deformation estimating means, which is based on the positions of the feature points obtained from the face feature point tracking means 5 and the positions of the feature points on the reference image obtained in advance. Estimate the degree of deformation.
  • Reference numeral 22 denotes a storage area for storing the positions of the feature points on the reference image.
  • Reference numeral 23 denotes a normalized deformation degree detecting means for detecting, from the deformation degrees estimated by the deformation degree estimating means 21, the deformation degree when the image is normalized at the enlargement ratio or the reduction ratio.
  • Numeral 24 denotes a three-dimensional direction estimating means for estimating the three-dimensional direction of the face by using the degree of deformation detected by the normalized deformation degree detecting means 23.
  • step 21 the degree of deformation on the image from the reference image to the input image is estimated using the movement amount between the feature points obtained in step 7.
  • step 22 an amount indicating the enlargement or reduction of the image is extracted from the degree of deformation on the image obtained in step 21, and the other degree of deformation is multiplied by the reciprocal of the enlargement or reduction ratio. To normalize.
  • step 23 the three-dimensional direction of the face is calculated from the degree of deformation normalized in step 22.
  • FIG. 9 shows a configuration of a face three-dimensional direction tracking apparatus according to Embodiment 4 of the present invention.
  • FIG. 10 is a flowchart for explaining the operation of the face three-dimensional direction tracking apparatus according to Embodiment 4 of the present invention.
  • reference numeral 31 denotes a part position predicting means, and an output of the means 6 for estimating the three-dimensional direction of the face from the positional relationship of the feature points (that is, the face estimated by the first three-dimensional direction estimating means 2) Based on the three-dimensional direction, the position of the new feature point on the face is predicted.
  • Reference numeral 32 denotes a new feature point setting means for setting a new feature point based on the part position obtained by the part position prediction means 31.
  • the feature points of the face to be tracked are eyes and nose.
  • the shape of the head can be approximated by an elliptical sphere model, and the positions of the feature points other than the eyes and nose that are currently handled, such as the ears and mouth, are determined. Predict.
  • step 32 it is determined whether or not there is a trackable feature point at the predicted position of the new feature point portion obtained in step 31. If so, it is determined as a new tracking feature point.
  • the first three-dimensional direction estimating means 2 has a new feature point that can be tracked even when the feature point to be tracked cannot be tracked. If the feature points cannot be tracked, the number of occurrences will decrease, and the means for estimating the three-dimensional direction of the first face will be It is possible to estimate the direction of a person's face in a wider range.
  • Embodiment 5 Embodiment 5.
  • FIG. 11 is a block diagram showing a configuration of a face three-dimensional direction tracking apparatus according to Embodiment 5 of the present invention
  • FIG. 12 is a diagram illustrating the operation of a face three-dimensional direction tracking apparatus according to Embodiment 5 of the present invention.
  • FIG. 3 is a flowchart for performing the above.
  • reference numeral 41 denotes stability tracking means. Based on information obtained from the face feature point tracking means 3, it is determined whether the position of the face feature point has little or no change. Is what you do.
  • the face is updated sequentially based on the determination result of the stability tracking means 41. By doing so, more stable position information of the feature points of the frontal face can be obtained.
  • step 41 it is determined whether or not the tracking is the first, and if it is the first, the process proceeds to step 43 and the feature point position obtained in step 5 is set as the feature point position of the frontal face. Hold.
  • step 7 calculate the amount of movement between feature points using the feature point positions of the front face held in step 43.
  • step 42 determines whether the face is stable. If stable, go to step 4 3 and go to step 5.
  • the obtained feature point position is updated and held as a new front face feature point position. Then, the process proceeds to step 7 to calculate the amount of movement between the feature points using the feature point positions of the front face updated in step 43.
  • step 7 If it is not stable, proceed to step 7 without updating the feature points of the frontal face.
  • FIG. 13 is a block diagram showing the configuration of the stability tracking means used in the fifth embodiment of the present invention
  • FIG. 14 explains the operation of the stability tracking means used in the fifth embodiment of the present invention. It is a flowchart for the.
  • reference numeral 401 denotes a motion width detecting means corresponding to the motion state detecting means
  • 402 denotes a reference motion width
  • 4003 denotes a reference tracking time
  • 404 denotes a stable tracking determining means.
  • reference numeral 406 is a reference tracking time setting means, both of which are realized by a combination, and these constitute a stability tracking means for tracking the stability of the face.
  • the reference movement width setting means 405 and the reference tracking time setting means 406 constitute a setting means.
  • the stability tracking determination means 4 04 detects the movement width of the feature point position by the movement width detection means 401 using the output of the output position integration means 4, the reference movement width 8 and the reference tracking time 4 03. Then, based on the detection result, the stable tracking determination means 404 determines whether or not the tracking is stable, and outputs the result.
  • the reference movement width setting means 405 updates the reference movement width 402
  • the reference tracking time setting means 406 updates the reference tracking time 9.
  • step 401 it is determined whether or not to use the stability tracking means for the first time, and if so, in step 402, the reference tracking time is set to a predetermined fixed value. Then, the reference motion width is set to a predetermined specific fixed value. Then, the change of the position of the feature point is examined for a time corresponding to the reference tracking time.
  • the movement width of the position of the feature point is set to a value smaller than, for example, the reference movement width.
  • step 4003 the position of the feature point is obtained from the position of the feature point obtained by the previous calculation and the position of the feature point obtained by this calculation. Calculate the movement width of. At this time, for example, if the movement width is defined as the relative distance of the feature point position from the previous feature point ⁇ standing, the movement width of the feature point position is the length of the vector representing the difference between the positions.
  • step 404 it is checked whether or not the movement width of the feature point position set in the above step 402 or step 403 is smaller than the reference movement width.
  • step 405 the tracking time is set to 0 in step 405
  • the fact that the tracking is not stable (not stable) is output in step 406, and the process returns to step 401.
  • step 407 is needed to change the position of the feature point in step 404.
  • the accumulated time that is, the difference between the time when the previous feature point position was calculated and the time when the current feature point position was calculated is accumulated in the tracking time.
  • step 408 it is checked in step 408 whether this tracking time is longer than the reference tracking time.
  • step 406 a message indicating that the tracking is not stable tracking is output in step 406, and the process returns to step 401.
  • step 408 If it is determined in step 408 that the tracking time is longer than the reference tracking time, the tracking time used in step 409 is set as a new reference tracking time, and in step 410 this updated reference tracking time is used. Hold tracking time. Next, the smallest change in the position (movement width) of the feature point corresponding to each time interval accumulated as the tracking time in step 411 is set as a new reference movement width. The updated reference motion width is retained, a message indicating that the tracking is stable is output in step 410, and the process returns to step 410.
  • the above embodiments are estimated angle range of the first three-dimensional direction estimation unit 2 estimates the angle range and the second three-dimensional direction estimation means 3 is partially overlapped (S 2 ⁇ ⁇ ), yet face Has been described in the horizontal (left-right) direction,
  • e ⁇ i in the configuration of the first embodiment and the face moves three-dimensionally in both vertical and horizontal directions (up, down, left, and right) will be described.
  • FIG. 15 is a flowchart for explaining the operation of the face three-dimensional direction tracking apparatus according to Embodiment 6 of the present invention, and has the same configuration as that of FIG.
  • the first three-dimensional direction estimating means 2 uses the input image obtained by the imaging means 1 as a reference for the direction of the human face in an image obtained by imaging a state in which the person is facing the front (that is, 0 )
  • the orientation of the person's face (3D direction) is in the range of 0 to 0 i in the horizontal direction (rotation in the horizontal plane), and in the range of 0 to 0 in the vertical direction (rotation in the vertical plane).
  • the output selecting means 4 predicts the next face direction using the three-dimensional direction estimation result of the face up to the previous time, and if the predicted face direction is 6> i or less and ⁇ i or less, Selects the output of the first three-dimensional direction estimation means 2 and the predicted face direction is greater than 0 or If it is larger than 0 or larger than ⁇ i, the output of the second three-dimensional direction estimating means 3 is selected.
  • the three-dimensional direction tracking apparatus for a face configured as described above can be operated, for example, in the order shown in the flowchart of FIG.
  • the face is set to a specific angle within a measurement range of the second three-dimensional direction estimating means 3 (for example, 70 °)
  • the face image when turned sideways is acquired by taking a picture in advance, and the face image when the face is turned 60 °, 80 °, and 90 ° sideways is The 70-degree profile image is transformed using the knowledge of the general three-dimensional structure of the face.
  • the face when the face is turned upward and downward at a specific angle (for example, 50 degrees) within the measurement range of the second three-dimensional direction estimating means 3 is set.
  • Images obtained by taking images in advance, etc., and the respective face images when the face is turned up and down by 40 degrees and 60 degrees are described above using knowledge of the general three-dimensional structure of the face.
  • the upper and lower face images of 50 degrees are transformed and generated. By doing so, it is possible to reduce the time and effort required for the user to image the face 60, 80, or 90 degrees horizontally or at 40 or 60 degrees vertically. Becomes possible. Or take a face image with the face turned 60, 70, 80, or 90 degrees horizontally, or with the face turned 40, 50, or 60 degrees up or down. If this is obtained, an accurate face image will be obtained, and more accurate detection will be possible.o
  • step 72 the face image obtained in step 71 is temporarily stored.
  • step 73 the profile and upper and lower face images stored in step 72 are acquired, and the input image is matched with each of the stored profile and upper and lower face images to thereby obtain a profile in the input image. If there is an image (a face image pointing in a specific direction between 60 and 90 degrees horizontally) or an upper and lower face image (a face image pointing in a specific direction between 40 and 60 degrees above or below) Detect it.
  • step 74 for example, the detected input image and the side face image oriented horizontally by 70 degrees, the profile image oriented sideways by 80 degrees, and the profile image oriented sideways by 90 degrees Using that matching score, the face direction is interpolated and complemented with linear weighting using each matching score to obtain the correct face direction (that is, the face is converted to the three-dimensional direction).
  • the top and bottom faces are interpolated and complemented as in the case of the profile face to obtain the correct face orientation.
  • Steps 7 1 and 7 2 may be executed when the face three-dimensional direction tracking apparatus of the sixth embodiment is operated for the first time, and after the face image is once held in step 72, Need not be performed.
  • step 75 from the input images sequentially captured by the imaging means 1, characteristic portions of a person's face such as eyes, nose, mouth, and ears are set as feature points, and the positions of the feature points are determined. To detect.
  • step 76 for example, a face image was taken in advance when the face was turned to the front, and in step 75, the positions of the feature points corresponding to the state in which the face was turned to the front were detected By acquiring the face image at that time, the position of the feature point in a state where the person is facing the front is held.
  • step 77 the difference between the position of the feature point of the face obtained in step 75 and the position of the feature point of the frontal face held in step 76 is calculated. Calculate how much the position of the point has moved from the frontal face (ie, the amount of movement between feature points).
  • step 78 various deformation amounts (translation, enlargement, distortion, etc.) on the image represented by the entire feature point are calculated from the movement amount between the feature points obtained in step 77. From the amount of deformation on these images, it can be seen that, for example, if the magnification in the horizontal direction is smaller than 1, that is, if the image is reduced, the face is horizontal. The three-dimensional direction of the face can be calculated from the specific magnification.
  • the feature points on the face can be tracked, and the three-dimensional direction of the face can be estimated using the positions of the feature points.
  • the face image is converted to the above-described step 71.
  • a 0-degree face image may be generated by deformation.
  • the upper and lower faces are close to upward or downward (for example, If the face image is detected (up or down by 40 degrees), the face image is transformed to generate the respective face images when the image is turned up or down by 50 or 60 degrees. May be.
  • step 76 may be performed when the face three-dimensional direction tracking apparatus of the sixth embodiment is operated for the first time.In step 76, the position of the feature point of the face when facing the front is temporarily held. After that, you don't have to 'execute.
  • each group may be executed sequentially or in parallel. May be.
  • the processing time can be reduced depending on the configuration of the apparatus that executes the processing.
  • step 74 and the output of step 78 differ from each other in the range of face angles that can be handled by each means. However, situations can often occur in which the other means cannot correctly estimate the 3D direction of the face.
  • step 79 one of the output of step 74 and the output of step 78 is selected.
  • step 79 information on the three-dimensional direction estimation result of the face up to the previous time is checked, and it is determined which of the output of step 74 and the output of step 78 is to be output.
  • the next three-dimensional direction of the face is predicted using, for example, Kalman filter, and the predicted face direction is calculated. If both the vertical and horizontal directions are within the estimation range of the first three-dimensional direction estimating means 2, the output of the three-dimensional direction estimating means 2 is selected, and at least one of the vertical and horizontal directions is the first three-dimensional direction estimating means 2. If it is out of the estimation range, the output of the three-dimensional direction estimation means 2 is selected. In this embodiment, in other words, if the predicted face orientation is less than 60 degrees horizontally and less than 40 degrees vertically, the output of the first three-dimensional direction estimation means 2 is selected and the prediction is performed.
  • the second three-dimensional direction estimation is performed if the orientation of the detected face is greater than 60 degrees horizontally or greater than 40 degrees vertically, or if it is greater than 60 degrees horizontally and greater than 40 degrees vertically.
  • step 80 the information of the output selected in step 79 is held for use in the next output selection.
  • the three-dimensional direction of the face can be stably and reliably estimated over a wide range using a single imaging device.
  • FIG. 16 is a block diagram showing the configuration of the peripheral recognition device according to the seventh embodiment of the present invention.
  • FIG. 17 is a flowchart for explaining the operation of the peripheral recognition device according to the seventh embodiment of the present invention. It is.
  • the peripheral recognition device is for detecting a danger area around the vehicle from an inattentive area of a person operating a moving vehicle, for example, and assisting the operation. From the output of the 3D direction tracking device for one of the faces shown in 6 above, the area that does not enter the field of view of the person (blind spot: The area is called), and if there is something that could cause an accident in this area, this is reported to the person.
  • reference numeral 61 denotes a face three-dimensional direction tracking device, which is shown in any one of the first to fifth embodiments.
  • Reference numeral 62 denotes an inattentive area detection means, which detects an inattentive area of a person based on the three-dimensional direction of the face obtained from the face three-dimensional direction tracking device 61.
  • Numeral 63 denotes an external world information acquisition area setting means, which sets an area for acquiring external world information corresponding to the inattentive area detected by the inattentive area detecting means 62.
  • Numeral 64 denotes an external sensor, which acquires external information of the area set by the external information acquisition area setting means 63.
  • Reference numeral 65 denotes peripheral recognition means for recognizing the state of the outside world based on the outside world information acquired by the outside world sensor 64.
  • Reference numeral 66 denotes danger determination means, which determines danger based on the recognition result recognized by the peripheral recognition means 65.
  • Reference numeral 67 denotes a notification means, which notifies, for example, when it is determined to be dangerous according to the determination of the risk determination means 66.
  • the inattentive area detection means 62, the external world information acquisition area setting means 63, the peripheral recognition means 65, and the danger determination means 66 are realized by, for example, a combination.
  • a laser radar or a millimeter-wave radar is used as the external field sensor 64, and an in-vehicle display alarm or the like is used as the notification means 67, for example.
  • the peripheral recognition device sets an area for acquiring external world information corresponding to the inattentive area based on the face direction sequentially estimated by the face three-dimensional direction tracking apparatus 61, and there is no danger in the set external world information. It is configured to determine whether or not there is, and to notify the user based on the determination result. It will be possible to detect the danger at a short time, and it will be possible to perform certain tasks when operating a moving vehicle more safely.
  • FIG. 17 is a flowchart for explaining a specific operation of the peripheral recognition device of FIG.
  • step 61 information on the orientation of a person's face estimated by the face three-dimensional direction tracking device 61 is obtained.
  • the method of estimating the face direction is the same as that described in each of the first to fifth embodiments.
  • step 62 the inattention region of the person is detected from the orientation of the face acquired in step 61.
  • step 63 an area for acquiring outside world information is set in the detected inattention area, and outside world information in the set area is acquired and recognized using the outside world sensor 64.
  • step 64 it is checked whether there is any dangerous external information that may cause an accident.
  • step 64 If it is determined in step 64 that there is no dangerous object, the process waits until the next three-dimensional direction tracking device 61 detects a new face direction without doing anything.
  • step 64 If it is determined in step 64 that there is a dangerous thing, the outside world information is acquired in step 65 in more detail, and it is determined again whether there is a dangerous thing based on this information.
  • step 65 If it is determined in step 65 that there is no dangerous object, the process waits until the next three-dimensional direction tracking device 61 estimates a new face direction without doing anything. If it is determined in step 65 that a dangerous thing is recommended, in step 66, the operator is notified that there is a dangerous thing.
  • step 65 external information corresponding to the inattentive area is acquired and recognized from the face direction sequentially estimated by the face three-dimensional direction tracking device 61, and there is a danger in the external information.
  • the system is configured to judge whether or not to do so and notify the user based on the judgment result, so that the user can quickly detect the danger in the inattention area, so that the vehicle can be operated more safely. It is possible to perform such tasks as performing.
  • the present invention is used for sequentially estimating the direction of a person's face from an image of the person's face sequentially taken in by an imaging means, and further estimated by a three-dimensional direction tracking device for this face It is used for assisting person's work by recognizing dangerous objects in the person's inattentiveness region based on the face direction and alerting them.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un dispositif de représentation en 3D du visage, caractérisé en ce qu'il comprend un premier dispositif de déduction de direction en 3D (2) destiné à déduire la direction en 3D du visage d'une personne lorsque l'orientation du visage est comprise dans un angle inférieur ou égal à Υ1 par rapport à l'orientation du visage dans une image d'entrée prise lorsque la personne regarde vers l'avant, un second dispositif de déduction de direction en 3D (3) destiné à déduire la direction en 3D du visage lorsque l'orientation du visage est comprise dans un angle compris entre υ2 qui est un angle inférieur ou égal à Υ1 et υ3 qui est un angle supérieur à υ1, ainsi qu'un dispositif de sélection de sortie (4) destiné à sélectionner les sorties des premier ou second dispositif de déduction de direction en 3D en fonction des résultats de la prédiction précédente de la direction en 3D du visage, de façon à déduire dans l'ordre l'orientation du visage de la personne à partir de l'image d'entrée saisie en série chronologique. FIG. 1 1 DISPOSITIF D'IMAGERIE 7 DISPOSITIF DE DETECTION D'ORIENTATION DU VISAGE SPECIFIQUE 8 DISPOSITIF DE CONVERSION DE DIRECTION 3D DU VISAGE 5 DISPOSITIF DE REPRODUCTION DE POINT PRECIS DE VISAGE 6 DISPOSITIF DE DEDUCTION DE DIRECTION 3D DU VISAGE SUR LA BASE DE RELATION DE POSITION DE POINT PRECIS 4 DISPOSITIF DE SELECTION DE SORTIE
PCT/JP2000/004795 2000-07-17 2000-07-17 Dispositif de representation en 3d du visage et dispositif de reconnaissance peripherique comprenant ce dernier WO2002007095A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2000/004795 WO2002007095A1 (fr) 2000-07-17 2000-07-17 Dispositif de representation en 3d du visage et dispositif de reconnaissance peripherique comprenant ce dernier

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2000/004795 WO2002007095A1 (fr) 2000-07-17 2000-07-17 Dispositif de representation en 3d du visage et dispositif de reconnaissance peripherique comprenant ce dernier

Publications (1)

Publication Number Publication Date
WO2002007095A1 true WO2002007095A1 (fr) 2002-01-24

Family

ID=11736267

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2000/004795 WO2002007095A1 (fr) 2000-07-17 2000-07-17 Dispositif de representation en 3d du visage et dispositif de reconnaissance peripherique comprenant ce dernier

Country Status (1)

Country Link
WO (1) WO2002007095A1 (fr)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7146028B2 (en) 2002-04-12 2006-12-05 Canon Kabushiki Kaisha Face detection and tracking in a video sequence
WO2009091029A1 (fr) * 2008-01-16 2009-07-23 Asahi Kasei Kabushiki Kaisha Dispositif, procédé et programme d'estimation de posture de visage
WO2010010926A1 (fr) * 2008-07-24 2010-01-28 国立大学法人静岡大学 Procédé de suivi de points caractéristiques et dispositif de suivi de points caractéristiques
US7860340B2 (en) 2004-11-04 2010-12-28 Nec Corporation Three-dimensional shape estimation system and image generation system
US8331631B2 (en) 2006-02-08 2012-12-11 Fujifilm Corporation Method, apparatus, and program for discriminating the states of subjects

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0371273A (ja) * 1989-08-10 1991-03-27 Nippon Telegr & Teleph Corp <Ntt> 適応的頭部位置検出装置
JPH07129770A (ja) * 1993-10-28 1995-05-19 Mitsubishi Electric Corp 画像処理装置
JPH10274516A (ja) * 1997-03-31 1998-10-13 Victor Co Of Japan Ltd 顔の方向検出装置
EP0990416A1 (fr) * 1998-10-01 2000-04-05 Mitsubishi Denki Kabushiki Kaisha Système de classement pour les directions de vue d'une personne

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0371273A (ja) * 1989-08-10 1991-03-27 Nippon Telegr & Teleph Corp <Ntt> 適応的頭部位置検出装置
JPH07129770A (ja) * 1993-10-28 1995-05-19 Mitsubishi Electric Corp 画像処理装置
JPH10274516A (ja) * 1997-03-31 1998-10-13 Victor Co Of Japan Ltd 顔の方向検出装置
EP0990416A1 (fr) * 1998-10-01 2000-04-05 Mitsubishi Denki Kabushiki Kaisha Système de classement pour les directions de vue d'une personne

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7146028B2 (en) 2002-04-12 2006-12-05 Canon Kabushiki Kaisha Face detection and tracking in a video sequence
US7860340B2 (en) 2004-11-04 2010-12-28 Nec Corporation Three-dimensional shape estimation system and image generation system
US8331631B2 (en) 2006-02-08 2012-12-11 Fujifilm Corporation Method, apparatus, and program for discriminating the states of subjects
WO2009091029A1 (fr) * 2008-01-16 2009-07-23 Asahi Kasei Kabushiki Kaisha Dispositif, procédé et programme d'estimation de posture de visage
JP4728432B2 (ja) * 2008-01-16 2011-07-20 旭化成株式会社 顔姿勢推定装置、顔姿勢推定方法、及び、顔姿勢推定プログラム
US8401251B2 (en) 2008-01-16 2013-03-19 Asahi Kasei Kabushiki Kaisha Face pose estimation device, face pose estimation method and face pose estimation program
WO2010010926A1 (fr) * 2008-07-24 2010-01-28 国立大学法人静岡大学 Procédé de suivi de points caractéristiques et dispositif de suivi de points caractéristiques
JPWO2010010926A1 (ja) * 2008-07-24 2012-01-05 国立大学法人静岡大学 特徴点追跡方法及び特徴点追跡装置
JP5429885B2 (ja) * 2008-07-24 2014-02-26 国立大学法人静岡大学 特徴点追跡方法及び特徴点追跡装置

Similar Documents

Publication Publication Date Title
US10600207B2 (en) Posture state estimation apparatus and posture state estimation method
US9031327B2 (en) Information processing device, method, and program that recognizes a predetermined part of a body
JP5517858B2 (ja) 画像処理装置、撮像装置、画像処理方法
JP5598751B2 (ja) 動作認識装置
JP5147761B2 (ja) 画像監視装置
JP5001930B2 (ja) 動作認識装置及び方法
EP2704056A2 (fr) Appareil et procédé de traitement d&#39;images
CN106603968B (zh) 信息处理装置及信息处理方法
JP2005056387A (ja) 画像処理装置、撮像装置、画像処理方法
CN114842397A (zh) 一种基于异常检测的实时老人跌倒检测方法
CN113508420A (zh) 物体追踪装置以及物体追踪方法
JP2010191793A (ja) 警告表示装置及び警告表示方法
US11887331B2 (en) Information processing apparatus, control method, and non-transitory storage medium
JP4821355B2 (ja) 人物追跡装置、人物追跡方法および人物追跡プログラム
JP2019212148A (ja) 情報処理装置及び情報処理プログラム
JP2012226645A (ja) 画像処理装置および方法、記録媒体並びにプログラム
JP4203279B2 (ja) 注目判定装置
JP2024098114A (ja) 情報処理装置、情報処理方法、及びプログラム
KR101542206B1 (ko) 코아스-파인 기법을 이용한 객체 추출과 추적 장치 및 방법
JP6384167B2 (ja) 移動体追跡装置及び移動体追跡方法、並びにコンピュータ・プログラム
JP6798609B2 (ja) 映像解析装置、映像解析方法およびプログラム
WO2002007095A1 (fr) Dispositif de representation en 3d du visage et dispositif de reconnaissance peripherique comprenant ce dernier
US20220138458A1 (en) Estimation device, estimation system, estimation method and program
JP2011233072A (ja) ロボット、位置推定方法及びプログラム
JP2022019988A (ja) 情報処理装置、ディスプレイ装置、及び制御方法

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): JP US

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE

121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase