JP2010128961A - Image monitoring apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image monitoring apparatus capable of searching a face position from an image in a short period of time. <P>SOLUTION: An ECU 20 for searching a face position of a driver from a captured image that is imaged by an image sensor 10 includes a face position early stage finding part 21 for finding (searching) the face position of the driver from the whole captured image acquired by the image sensor 10 and preparing a template, a face position tracking part 22 for tracking the face position of the driver in the captured image by template matching and updating the template, a face search position setting part 23 for setting a search area for searching the face position in the captured image, and a face position history recording part 24 for recording a search history of the face position and preparing a two-dimensional frequency histogram H (x, y). When fails in tracking the face position, the ECU 20 searches the face position in order of coordinates (x, y), the number of searching times of which is large with reference to the two-dimensional frequency histogram H (x, y). <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image monitoring apparatus that searches a face position of a target person from a captured image acquired by an image pickup apparatus and monitors the state of the target person.

  2. Description of the Related Art Conventionally, an image monitoring device that searches a driver's face position from a captured image captured by an imaging device mounted on a vehicle and monitors the state of the driver is known. This image monitoring device identifies a driver's face position by matching an arbitrary region of a captured image with a template, and once the face position is searched, it is searched A face position is searched in a small search range set based on the face position, and the face position is tracked.

By the way, if the face position is tracked using such a captured image, the face of the driver may be hidden (lost) because the face of the driver is hidden by an obstacle such as a hand. Therefore, in Patent Document 1, when the face position cannot be searched in the search range, the search range is increased as the face moving speed increases, thereby preventing the face position from being lost.
JP 2001-195582 A

  However, since the technique described in Patent Document 1 changes the search range according to the moving speed of the face, there is a problem that it cannot be handled when the driver's face is not moving. In addition, the technique described in Patent Document 1 has a problem that the search range may not be sufficiently suppressed because the search range may be too large depending on the moving speed of the face.

  Therefore, an object of the present invention is to provide an image monitoring apparatus that can search a face position from a captured image in a short time.

  The image monitoring apparatus according to the present invention searches for the face position of the target person in the picked-up image by sequentially matching a predetermined cutout area and the template in the picked-up image acquired by the image pickup means, and monitors the state of the target person The monitoring device records a face position search history and re-searches the face position based on the search history when the face position search fails.

  According to the image monitoring apparatus of the present invention, when the search for the face position fails, the face position is re-searched based on the face position search history, so that an area having a high probability of the face position is preferentially used. To explore. For this reason, even if the face position search fails, the face position can be searched from the captured image in a short time.

  In this case, in the re-search, it is preferable to search for the face position in descending order of the number of searches in the search history. According to this image monitoring apparatus, by re-searching the face position in the descending order of the number of searches, it is possible to search for the face position in descending order of the possibility that the face position exists. A position search can be performed.

  Also, the imaging device is mounted on a vehicle, records a face position search history corresponding to the vehicle direction instruction information or direction change operation information, and the re-search is a direction instruction or direction change operation when the search fails. It is preferable to search for face positions in descending order of the number of searches in the corresponding search history. According to this image monitoring apparatus, in view of the fact that the driver tends to perform a similar operation such as tilting a face when performing a direction change operation of the vehicle, the driver's face position searched in the past and By learning in association with direction indication or direction change operation, the search range of the face position can be narrowed down by the direction indication information or direction change operation information, so the face position search from the captured image in a shorter time It can be performed.

  Further, the re-search is preferably performed outward from the face position searched immediately before the search failed within the search result range including the face position recorded as the search history. According to this image monitoring device, the range to be re-searched is the search result range including the face position searched in the past, and by re-searching outward from the face position searched immediately before the search failed, To search the face position from the captured image in a shorter time in order to narrow down the search range to the range where the face position is likely to exist and to search for the face position in the order that the face position is more likely to exist. Can do.

  According to the present invention, it is possible to search for a face position from a captured image in a short time.

[First Embodiment]
DESCRIPTION OF EMBODIMENTS Hereinafter, a preferred embodiment of an image monitoring apparatus according to the present invention will be described in detail with reference to the drawings. An image monitoring apparatus according to the present invention searches for a driver's face position from a captured image acquired from an image sensor mounted on a vehicle and monitors the state of the driver. For example, the driver's face position The driver's snoozing situation is monitored by detecting the opening / closing situation of the driver's bag. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

  FIG. 1 is a diagram showing a block configuration of an image monitoring apparatus according to the present embodiment. As shown in the figure, an image monitoring apparatus 1 according to the present embodiment is mounted on a vehicle, and includes an image sensor 10 that captures an image of a driver's face, and an ECU (Electronic Control Unit) 20 that searches for the position of the driver's face. And.

  The image sensor 10 includes a camera using a semiconductor element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The image sensor 10 is built in a steering column of a vehicle and is operated while sitting in a driver's seat. It is an imaging device which images a person's face. Then, the image sensor 10 transmits the captured image to the ECU 20. The image sensor 10 may be configured with an infrared camera in order to capture the driver's face even at night.

  The ECU 20 is electrically connected to the image sensor 10. When the captured image transmitted from the image sensor 10 is acquired, the ECU 20 searches for the driver's face position from the captured image and monitors the driver's state. Part. Therefore, the ECU 20 functions as a face position initial finding unit 21, a face position tracking unit 22, a face search position setting unit 23, and a face position history recording unit 24. Note that the ECU 20 is mainly configured by a computer including a CPU, a ROM, and a RAM, for example.

  The face position initial finding unit 21 finds (searches) the driver's face position from the entire captured image acquired by the image sensor 10 using a known technique such as a neural network. This face position indicates the center position of the found face with coordinate values (x, y) in the entire captured image. When the face position initial finding unit 21 finds the driver's face position from the captured image, the face position initial finding unit 21 creates a driver face template based on the found face position. This template is created by cutting out a rectangular area having a constant vertical and horizontal width around the face position (x, y) from the captured image. In order to improve the processing speed and adaptability of template matching, which will be described later, it is desirable that the template be a rectangular area that is approximately the same size as the driver's face or slightly smaller than the driver's face.

  The face position tracking unit 22 tracks (tracks) the driver's face position in the captured image using a known method such as template matching after the face position is found by the face position initial finding unit 21. This template matching is a technique for sequentially searching out a cutout area having the same shape as a template from a captured image, comparing the cutout area with the template, and searching for a cutout area matching (matching) the template. The template matching method may be any of SAD (Sum of Absolute Difference), SSD (Sum of Square Difference), and normalized correlation. Then, the face position tracking unit 22 searches the driver's face position by template matching in a predetermined search region in the captured image, and tracks the driver's face position. Further, when the face position tracking unit 22 searches for the driver's face position, the face position tracking unit 22 updates the driver's face template in the searched face position extraction region.

  The face search position setting unit 23 sets a search area for searching for a face position in a captured image. That is, the face search position setting unit 23 sets a predetermined rectangular area centered on the previously searched face position as a search area. This search range is a range larger than the rectangular region of the template, and is a size that does not decrease the search speed. As a result, the face position tracking unit 22 searches for the face position in the search region set by the face search position setting unit 23.

  The face position history recording unit 24 records a face position search history in the face position initial finding unit 21 and the face position tracking unit 22. That is, when the face position history recording unit 24 searches for a face position in the face position initial finding unit 21 and the face position tracking unit 22, the face position coordinate value (x, y) is recorded as a search history. Then, the face position history recording unit 24 creates a two-dimensional frequency histogram H (x, y) from this search history.

  FIG. 2 is a diagram showing an example of a two-dimensional frequency histogram. As shown in FIG. 2, the two-dimensional frequency histogram H (x, y) represents the captured image on the xy coordinate axes, and the value of the corresponding coordinate (x, y) is as many as the number of face positions searched. Are added (+1 count up). Therefore, by referring to the two-dimensional frequency histogram H (x, y), it is possible to easily search for a frequently searched position where the face position is searched.

  Further, the face position history recording unit 24 performs a first in first out (FIFO) process on the data used for the two-dimensional frequency histogram H (x, y), and a predetermined time (for example, 5 to 10 minutes) has elapsed since the search. The deleted data is deleted (-1 countdown) from the two-dimensional frequency histogram H (x, y).

  Further, the face position history recording unit 24 smoothes the created two-dimensional frequency histogram H (x, y) to create a two-dimensional frequency histogram H ′ (x, y).

  Next, the processing operation of the image monitoring apparatus 1 according to the first embodiment will be described with reference to FIG. FIG. 3 is a flowchart showing the processing operation of the image monitoring apparatus according to the first embodiment. Note that the processing described below is performed by the processing of the ECU 20 of the image monitoring apparatus 1, and the following processing is repeatedly performed each time a captured image is transmitted from the image sensor 10 to the ECU 20.

  First, when the ignition of the vehicle is turned on by the driver, the image sensor 10 starts imaging at an interval of 30 f / S (30 frames per second) or 60 f / S. Then, the ECU 20 starts the following process when the captured image of the driver's face captured by the image sensor 10 is acquired (step S1).

  When acquiring the captured image, the ECU 20 determines whether or not the captured image is acquired for the first time in step S1 (step S2). That is, immediately after the ignition of the vehicle is turned on and the processing of the image monitoring apparatus 1 is started, it is determined that the captured image is acquired for the first time, and the captured image is acquired for the second time or later. If the face position has already been searched (or found) in the process of (2), it is determined that the captured image is not acquired for the first time.

  When it is determined that the captured image is acquired for the first time (step S2: YES), the ECU 20 performs initial discovery (search) of the driver's face position from the acquired captured image (step S3). In the initial discovery of the face position, the driver's face is searched using a known method such as a neural network using the entire captured image acquired in step S1 as a search range, and the center coordinates of the face in the captured image are represented by the face position (x , Y).

  Next, the ECU 20 creates a template for tracking the driver's face position (step S4). The template is created by first filtering the captured image (grayscale image) acquired in step S1 with a 3 × 3 horizontal Sobel filter to generate a horizontal Sobel filter image. 4 is a diagram illustrating a captured image, FIG. 5 is a diagram illustrating an example of a horizontal Sobel filter, and FIG. 6 is a diagram illustrating a horizontal Sobel filter image. As shown in FIG. 5, the horizontal Sobel filter generates an image with emphasized edges by weighting and summing nine pixels centered on the target pixel. Then, when the captured image of the grayscale image shown in FIG. 4 is filtered by the 3 × 3 horizontal Sobel filter shown in FIG. 5, a horizontal Sobel filter image with the edge shown in FIG. 6 emphasized is generated. Is done. In addition, the robustness with respect to the change of an optical environment etc. improves by performing the filter process by a horizontal Sobel filter.

  Once the horizontal Sobel filter image is generated, the horizontal Sobel filter image is then smoothed with a 3 × 3 smoothing filter to generate a horizontal Sobel filter smoothed image. FIG. 7 is a diagram illustrating an example of a smoothing filter, and FIG. 8 is a diagram illustrating a horizontal Sobel filter smoothed image. As shown in FIG. 7, the smoothing filter generates an image in which noise and the like are smoothed by performing the same weighting and summing on nine pixels centered on the target pixel. When the horizontal Sobel filter image shown in FIG. 6 is smoothed by the 3 × 3 smoothing filter shown in FIG. 7, the horizontal Sobel filter smoothed by the noise shown in FIG. 8 is smoothed. An image is generated.

  Once the horizontal Sobel filter smoothed image is generated, a template cut-out process is performed. FIG. 9 is a diagram for explaining processing for cutting out a template from a horizontal Sobel filter smoothed image. As shown in FIG. 9, in the template cut-out process, a rectangular area A having a constant vertical and horizontal width centered on the coordinates (x, y) of the face position is cut out from the horizontal Sobel filter smoothed image. The cut-out rectangular area A is set as a template T.

  When step S4 ends or when it is determined in step S2 described above that the captured image acquisition is not the first time (step S2: NO), the ECU 20 performs tracking of the face position by template matching (step S5). FIG. 10 is a diagram for explaining the tracking of the face position. As shown in FIG. 10, in tracking of the face position, first, a rectangular area within a predetermined range centering on the face position searched last time is set as a search area B. Next, in this search area B, a rectangular area having a constant vertical and horizontal width centered on the face position searched last time is sequentially cut out as a cut-out area C. Then, template matching of the cutout region C is sequentially performed using the template T created in step S4 or the template T updated in step S9 described later. When a cutout area C having a template matching value within a predetermined threshold is detected, this cutout area C is set as the driver's face position.

  Next, the ECU 20 determines whether or not the tracking is successful (step S6). In step S6, whether or not the tracking is successful is determined by whether or not a cutout region in which the template matching value in step S5 is within a predetermined threshold is detected.

  When it is determined that the tracking is successful (step S6: YES), the ECU 20 records the center coordinates (x, y) of the clipped area cut out in step S5, that is, the coordinates (x, y) of the driver's face position. (Step S7).

  Then, the ECU 20 performs the count-up process of the two-dimensional frequency histogram H (x, y) and the FIFO process based on the face position coordinates (x, y) recorded in step S7 (step S8). That is, in step S8, the value of the two-dimensional frequency histogram H (x, y) corresponding to the coordinate (x, y) of the face position recorded in step S7 is incremented by 1, while the value is recorded in step S7. The value of the two-dimensional frequency histogram H (x, y) corresponding to the coordinate (x, y) of the face position after a predetermined time has elapsed is counted down by −1. As described above, by performing FIFO processing on the data of the two-dimensional frequency histogram H (x, y) and searching for the face position with the latest data, tracking with higher accuracy can be performed.

  Next, the ECU 20 updates the template (step S9). The template updated in step S9 is the template created in step S4 or the template updated in the previous step S9. The template is updated by cutting out a rectangular area having a constant width and width centered on the coordinates (x, y) of the face position recorded in step S7 from the captured image acquired in step S1. Update the template. By updating the template in this way, it is possible to improve the robustness to changes in the driver's facial expression and changes in the light environment.

  On the other hand, when it is determined that the tracking has failed in step S6 described above (step S6: NO), the ECU 20 performs a smoothing process on the two-dimensional frequency histogram H (x, y) as a stage before performing the face position search. It performs (step S10). As the smoothing filter for performing the smoothing process, the 3 × 3 smoothing filter shown in FIG. 7 is used. Then, using the 3 × 3 smoothing filter, smoothing processing is performed on the two-dimensional frequency histogram H (x, y) to create a two-dimensional frequency histogram H ′ (x, y). As described above, the smoothing process of the two-dimensional frequency histogram H (x, y) is performed before the search for the face position, thereby reducing the error of the tracked face position (x, y) and the influence of noise. be able to.

  Next, the ECU 20 refers to the two-dimensional frequency histogram H ′ (x, y) smoothed in step S10, and coordinates (x, y) having a large number of counts in the two-dimensional frequency histogram H ′ (x, y). ) In order, the driver's face position is searched (step S11). In step S11, first, the coordinates of the two-dimensional frequency histogram H '(x, y) are sorted (rearranged) in descending order of the count value. Then, template matching is performed in order from the coordinate with the highest count value. At this time, as in step S5, template matching is performed in a predetermined search area centered on the coordinates (x, y) of the previously searched face position. If the face position is not searched in this search area, the template is searched. Expand the range of matching. Note that the template used in template matching is the template created in step S4 or the template updated in step S9.

  And after step S9 or step S11 mentioned above is complete | finished, a process is once complete | finished and a process is repeated from step S1 again.

  Thus, according to the image monitoring apparatus 1 according to the first embodiment, when face position tracking fails, the number of past searches is large with reference to the two-dimensional frequency histogram H (x, y). By searching for the face position in order from the coordinates, it is possible to search for the face position in descending order of the possibility that the face position exists, so that the face position can be searched from the captured image in a shorter time.

[Second Embodiment]
Next, a second embodiment will be described. The image monitoring apparatus according to the second embodiment is obtained by adding a steering angle detection sensor and a winker state detection sensor to the image monitoring apparatus according to the first embodiment, and other configurations are the same. For this reason, below, only a different part from the image monitoring apparatus which concerns on 1st Embodiment is demonstrated.

  FIG. 11 is a diagram illustrating a block configuration of an image monitoring apparatus according to the second embodiment. As shown in the figure, the image monitoring apparatus 101 according to the second embodiment further includes a steering angle detection sensor 30 and a blinker state detection sensor 40 in addition to the image sensor 10 and the ECU 20.

  The steering angle detection sensor 30 is a sensor that detects the steering angle θ of the steering operated by the driver as the driver's direction change operation information. For example, the steering angle detection sensor 30 may detect the steering angle θ in units of 1 ° or may be detected in units of a predetermined angle width. Then, the steering angle detection sensor 30 transmits the detected steering angle θ to the ECU 20.

  The winker state detection sensor 40 is a sensor that detects a winker state φ indicating the state of the winker operated by the driver as the direction instruction operation information of the driver. The winker state φ is information indicating that the winker has been operated to the right or left. Then, the blinker state detection sensor 40 transmits the detected blinker state φ to the ECU 20.

  The face position history recording unit 24 of the ECU 20 associates the face position searched from the captured image with the steering angle θ and the winker state φ at the time when the captured image is captured, and the two-dimensional frequency histogram H (x, y). Create Specifically, a two-dimensional frequency histogram H (x, y) is created for each combination of the steering angle θ and the winker state φ, and when the face position is searched from the captured image, the captured image is captured. The value of the coordinate (x, y) of the face position is incremented by one with respect to the two-dimensional frequency histogram H (x, y) corresponding to the combination of the steering angle θ and the winker state φ at that time. The two-dimensional frequency histogram H (x, y) may be provided every time the steering angle θ changes by 1 °, or may be provided every time the steering angle θ changes by a predetermined range such as 30 °, 60 °, and 90 °. Alternatively, the steering angle θ may be provided separately on the + (plus) side and the − (minus) side.

  Next, the processing operation of the image monitoring apparatus 101 according to the second embodiment will be described with reference to FIG. FIG. 12 is a flowchart illustrating the processing operation of the image monitoring apparatus according to the second embodiment. Note that the processing described below is performed by the processing of the ECU 20 of the image monitoring apparatus 101, and the following processing is repeatedly performed each time a captured image is transmitted from the image sensor 10 to the ECU 20.

  First, when the ignition of the vehicle is turned on by the driver, steps S1 to S6 are performed as in the first embodiment.

  If it is determined in step S6 that the tracking has been successful (step S6: YES), the ECU 20 determines the center coordinates of the clip region cut out in step S5, that is, the coordinates (x, y) of the driver's face. In addition to recording, the steering angle θ detected by the steering angle detection sensor 30 and the winker state φ detected by the winker state detection sensor 40 are detected and recorded (step S21).

  Next, the ECU 20 performs a count-up process of the two-dimensional frequency histogram H (x, y) based on the face position coordinates (x, y), the steering angle θ, and the winker state φ recorded in step S21. FIFO processing is performed (step S22). That is, in step S22, a two-dimensional frequency histogram H (x, y) corresponding to the combination of the steering angle θ and the winker state φ recorded in step S21 is selected, and the selected two-dimensional frequency histogram H (x, y). The value corresponding to the coordinate (x, y) of the face position at +1 is counted up. On the other hand, the value of the two-dimensional frequency histogram H (x, y) corresponding to the coordinate (x, y) of the face position that has passed for a predetermined time after recording in step S21 is counted down by -1. At this time, the value corresponding to the coordinates (x, y) of the face position after a predetermined time is counted down by −1 from all the two-dimensional frequency histograms H (x, y).

  Next, the ECU 20 updates the template (step S9) and once ends the process.

  On the other hand, when it is determined that the tracking has failed in step S6 described above (step S6: NO), the ECU 20 first times when the captured image acquired in step S1 is captured (or when tracking fails in step S6). ) Is detected and recorded (step S23).

  Then, the ECU 20 selects a two-dimensional frequency histogram H (x, y) corresponding to the combination of the steering angle θ and the winker state φ detected in step S23, and this selected 2 is used as a previous stage for searching for the face position. A smoothing process is performed on the dimensional frequency histogram H (x, y) to create a two-dimensional frequency histogram H ′ (x, y) (step S10).

  Next, the ECU 20 refers to the two-dimensional frequency histogram H ′ (x, y) smoothed in step S10, and coordinates (x, y) having a large number of counts in the two-dimensional frequency histogram H ′ (x, y). ) To search for the driver's face position (step S11). That is, in step S11, the face position is searched with reference to the two-dimensional frequency histogram H (x, y) corresponding to the combination of the steering angle θ and the winker state φ detected in step S23.

  When the search for the face position in step S11 ends, the process is once ended, and the process is repeated again from step S1.

  Thus, according to the image monitoring apparatus 101 according to the second embodiment, the driver tends to perform a similar operation such as tilting his / her face when performing a direction change operation of the vehicle. In view of this, the coordinates (x, y) of the driver's face position searched in the past are recorded in the two-dimensional frequency histogram H (x, y) in association with the combination of the steering angle θ and the winker state φ. Thus, since the search range of the face position can be narrowed down by the steering angle θ and the winker state φ, the face position can be searched from the captured image in a shorter time.

[Third Embodiment]
Next, a third embodiment will be described. The image monitoring apparatus according to the third embodiment has basically the same configuration as the image monitoring apparatus according to the first embodiment, and only the processing operation is different. For this reason, below, only a different part from the image monitoring apparatus which concerns on 1st Embodiment is demonstrated.

  FIG. 13 is a diagram illustrating a block configuration of an image monitoring apparatus according to the third embodiment. As shown in the figure, the image monitoring apparatus 201 according to the third embodiment includes an image sensor 10 and an ECU 20, as with the image monitoring apparatus 1, and the ECU 20 includes a face position initial finding unit 21, It functions as a face position tracking unit 22, a face search position setting unit 23, and a face position history recording unit 24.

  Then, when the searched face position is recorded, the face position history recording unit 24 of the ECU 20 creates a two-dimensional search result matrix H (x, y) instead of the two-dimensional frequency histogram H (x, y). FIG. 14 is a diagram illustrating an example of a two-dimensional search performance matrix. As shown in FIG. 14, the two-dimensional search performance matrix H (x, y) represents the captured image with the xy coordinate axes, and changes the value of the coordinates where the face position is searched to 1. For this reason, the position where the face position has been searched can be easily searched by referring to the two-dimensional search result matrix H (x, y). In FIG. 14, the coordinate area that is blacked out indicates 1, and the coordinate area that is not blacked out indicates 0.

  Next, the processing operation of the image monitoring apparatus 201 according to the third embodiment will be described with reference to FIG. FIG. 15 is a flowchart illustrating the processing operation of the image monitoring apparatus according to the third embodiment. Note that the processing described below is performed by the processing of the ECU 20 of the image monitoring apparatus 201, and the following processing is repeatedly performed each time a captured image is transmitted from the image sensor 10 to the ECU 20.

  First, when the ignition of the vehicle is turned on by the driver, steps S1 to S6 are performed as in the first embodiment.

  If it is determined in step S6 that the tracking is successful (step S6: YES), the ECU 20 records the coordinates (x, y) of the driver's face position (step S7).

  Next, the ECU 20 performs a rewrite process of the two-dimensional search result matrix H (x, y) and a FIFO process based on the coordinates (x, y) of the face position recorded in step S31 (step S31). . That is, in step S31, if the value of the two-dimensional search result matrix H (x, y) corresponding to the coordinate (x, y) of the face position recorded in step S7 is 0, this value is changed to 1. . If the value of the two-dimensional search result matrix H (x, y) has already been changed to 1, this value is not particularly changed. On the other hand, the value of the two-dimensional search performance matrix H (x, y) corresponding to the coordinates (x, y) of the face position that has passed for a predetermined time after recording in step S7 is changed to zero.

  Next, the ECU 20 updates the template (step S9) and once ends the process.

  On the other hand, when it is determined that the tracking has failed in step S6 described above (step S6: NO), the ECU 20 refers to the two-dimensional search result matrix H (x, y) and calculates the search result range (step S32). ). FIG. 16 is a diagram showing a search result range in the two-dimensional search result matrix H (x, y) shown in FIG. As shown in FIG. 16, the search result range D calculates the maximum value and the minimum value of the x coordinate and the y coordinate that are changed to 1 in the two-dimensional search result matrix H (x, y). A rectangular range surrounding the maximum value and the minimum value is set as a search result range D. In other words, the search result range D is a rectangular range including the coordinates searched once even before being deleted by the FIFO. In addition, this search performance range D is good also as the range of the minimum circle | round | yen including all the coordinates changed into 1 in the two-dimensional search performance matrix H (x, y), for example.

Then, the ECU 20 searches for the driver's face in the search result range D calculated in step S32 (step S33). FIG. 17 is a diagram for explaining a driver's face search using a two-dimensional search result matrix. As shown in FIG. 17, in step S33, the position of the face is directed from the inside toward the outside in order from the inside of the search result range with the coordinates (x ₀ , y ₀ ) of the face position searched immediately before the tracking failure as the center. Search for. That is, the face is searched in order from the coordinates having a distance close to the coordinates (x ₀ , y ₀ ) within the search result range.

  When the search for the face position in step S33 is finished, the process is once ended, and the process is repeated from step S1 again.

As described above, according to the image monitoring apparatus 201 according to the third embodiment, when tracking fails, the search range is the search result range D including the face position searched in the past, and immediately before the search fails. By searching outward from the face position (x ₀ , y ₀ ) searched for in ( ₂ ), the search range is narrowed down to a range where the face position is likely to exist, and there is a possibility that more face positions exist Since the face positions are searched in descending order, the face position can be searched from the captured image in a shorter time.

  The preferred embodiment of the present invention has been described above, but the present invention is not limited to the above embodiment. For example, the coordinates indicating the face position may indicate one pixel of the captured image, or may indicate a plurality of pixel ranges (predetermined areas).

  In the third embodiment, the search range is narrowed down using the search result range D. However, for example, a search result range is prepared for each combination of the steering angle θ and the winker state φ, and tracking fails. The search range may be narrowed down by the search result range corresponding to the combination of the steering angle θ and the winker state φ at the time.

It is the figure which showed the block configuration of the image monitoring apparatus which concerns on 1st Embodiment. It is the figure which showed an example of the two-dimensional frequency histogram. It is a flowchart which shows the processing operation of the image monitoring apparatus which concerns on 1st Embodiment. It is the figure which showed the captured image. It is a figure which shows an example of a horizontal Sobel filter. It is the figure which showed the horizontal Sobel filter image. It is the figure which showed an example of the smoothing filter. It is the figure which showed the horizontal Sobel filter smoothed image. It is a figure for demonstrating the process which cuts out a template from a horizontal Sobel filter smoothed image. It is a figure for demonstrating the tracking of a face position. It is the figure which showed the block configuration of the image monitoring apparatus which concerns on 2nd Embodiment. It is a flowchart which shows the processing operation of the image monitoring apparatus which concerns on 2nd Embodiment. It is the figure which showed the block configuration of the image monitoring apparatus which concerns on 3rd Embodiment. It is the figure which showed an example of the two-dimensional search performance matrix. It is a flowchart which shows the processing operation of the image monitoring apparatus which concerns on 3rd Embodiment. It is the figure which showed the search performance range in the two-dimensional search performance matrix shown in FIG. It is a figure for demonstrating the search of a driver | operator's face using a 2-dimensional search performance matrix.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image monitoring apparatus, 10 ... Image sensor, 20 ... ECU, 21 ... Face position initial discovery part, 22 ... Face position tracking part, 23 ... Face search position setting part, 24 ... Face position history recording part, 30 ... Steering angle Detection sensor, 40 ... Winker state detection sensor, 101 ... Image monitoring device, 201 ... Image monitoring device, A ... Rectangular region, B ... Search region, C ... Clipping region, D ... Search result range, T ... Template.

Claims (4)

An image monitoring apparatus that searches for a face position of a target person in the picked-up image by sequentially matching a predetermined cutout region and a template in the picked-up image acquired by the image pickup means, and monitors the state of the target person.
Record the search history of the face position,
An image monitoring apparatus, wherein when the search for the face position fails, the face position is searched again based on the search history.   The image monitoring apparatus according to claim 1, wherein the re-searching searches for the face position in descending order of the number of searches in the search history. The imaging device is mounted on a vehicle,
The search history of the face position is recorded corresponding to the direction instruction information or the direction change operation information of the vehicle,
The image monitoring apparatus according to claim 1, wherein the re-searching searches the face positions in descending order of the number of searches in the search history corresponding to a direction instruction or a direction changing operation when the search fails. . The re-search is performed outward from a face position searched immediately before the search has failed within a search result range including the face position recorded as the search history. Image monitoring device.