JP2006268565A - Occupant detector and occupant detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately detect an occupant even when the shape of a photographed surface has a shape similar to that of the head part of the occupant, even when a moving object except the occupant is included in a photographing area, or even when a face area image of the occupant is partially hidden behind an obstacle. <P>SOLUTION: A photograph part 2 photographs an original image including the vehicle occupant, a computing part 3 generates an edge image from the original image to extract an area put between extracted edges as a candidate area. Then, the computing part 3 integrates the extracted candidate area and an adjacent pixel matching a predetermined condition together, and integrates and updates the candidate areas to store it in a storage part 4. The computing part 3 calculates characteristic quantity of the updated candidate area and extracts a target candidate area having high probability of a face area from the candidate area. In the target candidate area, the computing part 3 and the storage area 4 determine movement quantity and front-back relationship on the screen in a fixed time for extracting the face area. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an occupant detection device and an occupant detection method for detecting the position and state of an occupant in a vehicle by image processing.

Conventionally, two images of the interior space including the occupant are provided with a time difference, and edge images of the two captured images are created, and the position of the occupant is based on the difference image shape of the two edge images. There is known an occupant detection device that detects the state of the vehicle (for example, see Patent Document 1).
JP 2000-113164 A

  However, since the conventional occupant detection device is configured to detect the position and state of the occupant from the difference image shape of the edge image, the shape on the imaging surface has the same shape as the head of the occupant, and When a subject other than the moving occupant is present in the imaging region, the subject may be erroneously detected as the occupant. In addition, when the subject exists between the head of the occupant and the imaging surface, the subject partially hides the occupant's head in the captured image, so that the subject can be recognized as the head region of the occupant. A characteristic shape may not be obtained and an occupant may not be detected.

  The present invention has been made in view of the above points, and the shape on the imaging surface has the same shape as the head of an occupant, and a subject other than the moving occupant is present in the imaging region. Another object of the present invention is to provide an occupant detection device and an occupant detection method capable of accurately detecting an occupant even when an image of the occupant's face region is partially hidden by an obstacle.

  In order to solve the above-described problem, an occupant detection device and an occupant detection method according to the present invention generate an edge image from an in-vehicle image, extract an area between the edge images as a candidate area, and are adjacent to the candidate area. It is determined whether or not the difference value between the pixel value of the pixel and the pixel value of the candidate area is within a predetermined range, and if the difference value is within the predetermined range, the pixels adjacent to the candidate area are integrated into the candidate area to be a candidate The area is updated, and it is determined whether or not the candidate area is an image area corresponding to the face of an occupant in the vehicle.

  According to the occupant detection device and the occupant detection method according to the present invention, the shape on the imaging surface has the same shape as the occupant's head, and a subject other than the moving occupant exists in the imaging region. Even when the image of the occupant's face area is partially hidden by an obstacle, the occupant can be accurately detected.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[Configuration of passenger detection device]
First, the configuration of an occupant detection device according to an embodiment of the present invention will be described with reference to FIG.

  As shown in FIG. 1, an occupant detection device 1 according to an embodiment of the present invention is mounted on a vehicle and mainly includes an imaging unit 2, a calculation unit 3, and a storage unit 4. The imaging unit 2 is configured by a photographing device such as a far-infrared camera (IR camera), and is installed at a position such as a map lamp unit or a center cluster unit as shown in FIG. The installation location of the imaging unit 2 may be an arbitrary position as long as it can capture a passenger's face.

  The calculation unit 3 is configured by a well-known information processing device, and an internal CPU (Central Processing Unit) executes a program to realize a face area extraction process described later. The calculation unit 3 functions as an edge image generation unit, a candidate region extraction unit, a candidate region update unit, and a region type determination unit according to the present invention. The storage unit 4 is configured by a storage device such as a RAM (Random access memory), and temporarily stores candidate area and update candidate area data (pixel values) extracted by a face area extraction process described later.

  The occupant detection device 1 having such a configuration performs the face area extraction process shown below, so that the shape on the imaging surface is the same as that of the occupant's head and other than the moving occupant The occupant is accurately detected even when the subject is present in the imaging region or when the image of the occupant's face region is partially hidden by an obstacle. The operation of the occupant detection device 1 when executing this face area extraction process will be described below with reference to the flowchart shown in FIG.

[Facial area extraction processing]
The flowchart shown in FIG. 3 starts when the vehicle engine starts or when the occupant detection device 1 is turned on, and proceeds to step S1.

  In the process of step S1, the calculation unit 3 initializes variables used in the subsequent process. When the initialization is completed, the process proceeds to the next step S2.

  In the process of step S2, the calculation part 3 acquires the far-infrared image containing the passenger | crew's face in a vehicle interior from the imaging part 2 as an original image. After obtaining the original image, the process proceeds to the next step S3.

In the process of step S3, the calculation unit 3 functions as an edge image generation unit according to the present invention, and generates an edge image of the original image acquired in step S2. Specifically, as illustrated in FIG. 4, when the original image 41 includes the subjects 42 and 43 and the background 44, the calculation unit 3 uses the Sobel filter to convert the original image 41 in the horizontal direction. A vertical sobel image is generated by extracting the temperature gradient, and an edge image is generated by converting the pixel value Si of the pixel i in the vertical sobel image into a label image Ei according to the following formula 1. Since the original image 41 is a far-infrared image, the pixel value Si corresponds to a temperature gradient, THP in Equation 1 is a predetermined threshold value that is a positive value, and THM is a predetermined threshold value that is a negative value. It is.

  That is, when the temperatures of the subjects 42 and 43 are higher than the background 44, the pixel values corresponding to the subjects 42 and 43 are higher than the pixel values of the background 44. When the temperature of the subject 43 is higher than the temperature of the subject 42, the pixel value corresponding to the subject 43 is higher than the pixel value of the subject 42. Therefore, when edge images are generated for these images, an edge is extracted at the boundary between the background 44 and the subjects 42 and 43, and an edge image 51 as shown in FIG. 5 is generated. Edges 52 and 53 in the edge image 51 are positive edges of the label image Ei = 1, and edges 54 and 55 are negative edges of the label image Ei = −1. The other image area is defined as label image Ei = 0. Note that the method of generating the edge image 51 is not limited to the above method, and for example, a filter other than a Sobel filter may be used. Further, the pixel value of the edge image may be a value other than temperature as long as the positive / negative of the edge strength is determined. When the edge image 51 is detected, the process proceeds to the next step S4.

  In the process of step S4, the calculation unit 3 functions as a candidate area extraction unit according to the present invention, and extracts and extracts an image area sandwiched between edges in the edge image 51 obtained by the process of step S3 as a candidate area. The candidate area data is stored in the storage unit 4. Specifically, the calculation unit 3 scans the edge image 51 obtained in the process of step S3 in the horizontal direction, and calculates pixel values on a line ending from a positive value pixel to a negative value pixel on the scanning line. 1 and other pixel values are converted to 0. That is, as shown in FIG. 6, the arithmetic unit 3 sets the pixel value of the region A on the line L, which starts from the positive edge 52 and ends at the negative edge 55, on the line L obtained by scanning the edge image 51 in the horizontal direction. And Similarly, the calculation unit 3 sets the pixel value in the range A on the line L starting from the positive edge 53 and ending at the negative edge 54 to 1. The calculation unit 3 converts the pixel values on the other lines to 0. As a result, as shown in FIG. 7, images of candidate areas 72 and 73 having a pixel value of 1 and a background area 74 having a pixel value of 0 are obtained. In this processing, the calculation unit 3 identifies the identification numbers n (n = 1, 2, 3,..., N: N is a candidate in the candidate regions 72 and 73 and the background region 74 extracted for the processing described later. The total number of areas) is allocated and temporarily stored in the storage unit 4. When extraction of the candidate area is completed, the process proceeds to the next step S5.

In the process of step S5, the calculation unit 3 functions as a candidate area update unit according to the present invention, determines whether a pixel adjacent to the candidate area is a pixel having the same value as the pixel value of the candidate area, If the pixels have the same value, the pixels are integrated into the candidate area to be noticed, and the candidate area data stored in the storage unit 4 is updated. Specifically, the calculation unit 3 calculates an average value TAn of pixel values Ti (i is a pixel existing in the candidate area n) of pixels belonging to the candidate area with the identification number n in the original image. Next, the calculation unit 3 calculates the absolute value of the difference between the pixel value TAn of the pixel belonging to the candidate area with the identification number n and the pixel value Tj of the pixel adjacent to the candidate area with the identification number n to the following Equation 2. It is determined whether or not it is equal to or less than a predetermined threshold THA as shown, and if the absolute value of the difference is equal to or less than the predetermined threshold, the pixel Tj adjacent to the candidate area with the identification number n is set as a candidate area to be noted. Integrate. Note that the predetermined threshold THA is preferably smaller than the absolute values of the thresholds THP and THM set in Equation 1 in order to avoid integrating pixels across edges. When the integration of the pixels is completed, the process proceeds to the next step S6.

  In the process of step S6, the calculation unit 3 determines whether or not the newly adjacent pixel value j satisfies Expression 2 for the candidate area n with the identification number n updated in Step S5. If not satisfied, the update is terminated and the process proceeds to the next step S7. If it satisfies, the process returns to step S5. Here, the processing from step S4 to step S6 will be described in detail with reference to FIGS. 7, 8, and 9. The candidate regions 72 and 73 obtained by the processing of step S4 shown in FIG. When the original image 41 shown in FIG. 8 is superimposed, a superimposed image 81 shown in FIG. 8 is obtained. At this time, the candidate area 72 is a part of the subject 42 shown in FIG. Therefore, an area having a pixel value equal to that of the candidate area 72 is extracted in step S5, and the area having the same pixel value is integrated in the candidate area 72 in step S6, thereby superimposing the object 42 as shown in FIG. Thus, the updated candidate area 94 is obtained. This is the candidate region extraction processing step, and after performing simple region extraction based on edges in steps S1 to S6, if the pixel values of adjacent pixels are close, the pixels are the same The region is updated as belonging to the region, and the final region extraction is performed. Further, according to this processing, it is determined which region the occupant's image region is included in comparison with a general region dividing method that divides the region for all pixels on the screen, for example, the region expanding method. The number of times is reduced, and the calculation load can be greatly reduced.

  In the processing from the next step S7 to step S14, the calculation unit 3 and the storage unit 4 function as region type determination means according to the present invention, and extract one corresponding to the occupant's face region from among a plurality of candidate regions. . Specifically, in the process of step S7, the calculation unit 3 calculates the shape feature amount of each extracted candidate region. For example, the calculation unit 3 calculates three candidate shape feature amounts: the circularity of the candidate region, the area of the candidate region, and the aspect ratio of the rectangular region that circumscribes the candidate region. Note that the shape feature amount is not limited to the above three, and may be any value as long as the shape feature amount can appropriately represent the shape of the subject to be detected. When the shape feature amount of each candidate area is calculated, the process proceeds to the next step S8.

  In the process of step S8, the calculation unit 3 determines an area with a high probability that the candidate area of interest does not correspond to the face area of the passenger based on the shape feature amount of the candidate area calculated in step S7. Specifically, the calculation unit 3 proceeds to the next step S9 if the shape feature values of the candidate region to be noticed all satisfy a condition that it is within a predetermined threshold range, and even if there is one shape feature value of the candidate region to be noticed. If it is not within the predetermined threshold range, it is determined that the candidate region of interest is other than the occupant's face, and the process returns to step S2.

  In the process of step S9, the calculation unit 3 calculates the pixel value feature amount of the candidate region of interest, and determines whether the pixel value feature amount is within a predetermined range. Specifically, the calculation unit 3 calculates a variance value of pixel values in the target candidate region as a pixel value feature amount. Here, the reason why the variance value is applied as the pixel value feature amount is that, for example, when the candidate region other than the face region is a warm can juice drink or the like, the variation in the pixel value is small because the temperature change in the candidate region is small. On the other hand, in the face area, due to the influence of eyes, mouth, and nose, temperature unevenness occurs in the candidate area, so that the dispersion tends to increase. Therefore, if the variance value is within a predetermined threshold range, the candidate area can be determined to be an area other than the face area. If the pixel value feature amount of the candidate area to be noticed is within the predetermined threshold range, the process proceeds to step S10, and if the pixel value feature quantity of the candidate region to be noticed is not within the predetermined threshold range, the process returns to step S2. In the present embodiment, the variance value in the candidate region is calculated as the pixel feature amount. However, the calculation unit 3 may calculate the regularity of the pixel value distribution in the region as the pixel feature amount. Specifically, the temperature distribution in the face region is generated by the eyes, mouth, and nose, and the pixel value distribution on the image is regular, so prepare an average template of pixel value distribution in advance. The pixel feature amount is calculated using the correlation value with the template as an index of regularity.

  In the process of step S10, the calculation unit 3 stores the image after the processing steps up to step S9 in the storage unit 3. If stored, the process proceeds to the next step S11.

  In the process of step S11, the calculation unit 3 calculates the movement amount on the image at two different times of the candidate region of interest, and determines whether or not the movement amount between the two times is within a predetermined range. If it is within the predetermined range, it is determined as a face region, and if it is not within the predetermined range, it is determined as a region other than the face region. Here, the basis of this determination processing will be described with reference to FIG. When the subject 102 is imaged to the same size as the occupant's face 101, the distance between the subject 102 and the imaging surface 104 is the distance between the occupant's face 101 and the imaging surface 104, depending on the mounting position of the camera 103. It becomes about half. When the subject 102 is moved by the occupant's hand at this distance, movement on the imaging surface 104 occurs, but the amount of movement is generally displayed when the seated occupant's face 101 is moved. It becomes larger than the movement amount generated in This is because, in the case of an occupant seated in real space, the speed of moving the hand can be generally faster than the speed of moving the face, and the face is farther away from the imaging surface 104 than the hand. If the movement speed in the real space is the same because it exists, the movement amount of the face area in the image is smaller. Therefore, according to this determination process, it can be accurately determined whether or not the candidate region corresponds to the face region.

  In the process of step S12, when there are a plurality of candidate areas, the calculation unit 3 obtains the positional relationship in the front-rear direction of the plurality of candidate areas. Specifically, the calculation unit 3 performs tracking processing of each region, and determines that the region where the shape of the region does not change is present in the foreground when the candidate regions overlap on the screen. When the context is obtained, the process proceeds to the next step S13.

  In the process of step S13, the calculation unit 3 extracts the candidate area determined as the rear area in the process of step S12 as a face area. Here, with reference to FIG. 10, the basis for determining the rear area as the face area will be described. Since the subject 102 exists at a distance closer to the imaging surface 104 than the subject 101, a phenomenon occurs in which the occupant's face 101 is hidden by the subject 102 and the shape on the imaging surface 104 changes. If the subject 102 is a warm drink or the like held by the occupant in the hand, the subject 102 is less likely to move backward than the occupant's face region 101. Can be considered. When the calculation unit 3 extracts the candidate area as the face area, the process proceeds to the next step S14.

  In the process of step S14, the calculation unit 3 determines whether or not to end the face area extraction process. If the face area extraction process is to be terminated, the face area extraction process is terminated. If the face area extraction process is not to be terminated, the process returns to step S2.

  In the face area extraction process in the present embodiment, a far-infrared image is used as the original image acquired by the imaging means. This is because when extracting a face area, the temperature difference inside the face area is small compared to the temperature difference occurring inside the face area and the temperature difference between the face area and the background. Since the nose, mouth, and other contours do not generate edges, only the facial contours are generated as edges, and the entire face region can be efficiently extracted. In other words, when the temperature change due to the part of a subject is small, it is possible to capture the subject as a surface, and even if a pattern exists on the subject surface, it can capture the pattern like visible light. Therefore, the far-infrared image is suitable when only the contour of the subject is desired.

  As is clear from the above description, in the occupant detection device 1 according to the embodiment of the present invention, the imaging unit 2 captures an original image including a vehicle occupant, the calculation unit 3 generates an edge image from the original image, A region between the extracted edges is extracted as a candidate region. Next, the calculation unit 3 integrates the extracted candidate regions and adjacent pixels that meet a predetermined condition, integrates and updates the candidate regions, and stores them in the storage unit 4. Next, the calculation unit 3 calculates the feature amount of the updated candidate area, and extracts a candidate area to be noticed having a high probability of being a face area from the candidate areas. Then, the calculation unit 3 and the storage unit 4 determine the amount of movement and the front-rear relationship on the screen for a certain time with respect to the candidate region of interest, and extract a face region. Therefore, according to the occupant detection device 1 according to the embodiment of the present invention, the shape on the imaging surface has the same shape as the head of the occupant, and a subject other than the moving occupant exists in the imaging region. Even when the image of the occupant's face area is partially hidden by an obstacle, the occupant can be accurately detected.

  As mentioned above, although the Example which applied the invention made | formed by this inventor was demonstrated, this invention is not limited by the description and drawing which make a part of indication of this invention by this Example. That is, it is added that all other examples, examples, operation techniques and the like made by those skilled in the art based on the form of the above examples are included in the scope of the present invention.

1 is a block diagram of an occupant detection device according to an embodiment of the present invention. It is a figure which shows the example of attachment of an imaging part. It is a flowchart figure which shows the flow of the face area extraction process which concerns on the Example of this invention. It is a figure which shows the original image imaged by the imaging part. It is a figure which shows the edge image obtained from the present image shown in FIG. It is a figure for demonstrating the method of candidate area | region extraction. It is a figure which shows the candidate area | region obtained from the edge image shown in FIG. It is the figure which displayed the candidate area | region and the original image superimposed. It is a figure which shows the updated candidate area | region. It is a figure for demonstrating the method of calculating | requiring the positional relationship of the front-back direction of a some candidate area | region.

Explanation of symbols

1: Occupant detection device 2: Imaging unit 3: Calculation unit 4: Storage unit
41: Original image
72, 73: Candidate areas
94: Updated candidate area

Claims (8)

An imaging means for capturing an image in the vehicle;
Edge image generation means for generating an edge image of an image captured by the imaging means;
Candidate area extraction means for extracting an image area sandwiched between edge images generated by the edge image generation means as candidate areas;
When the difference value between the pixel value of the pixel adjacent to the candidate area extracted by the candidate area extraction means and the pixel value of the candidate area is within a predetermined range, and the difference value is within the predetermined range Candidate area updating means for updating the candidate area by integrating the pixels adjacent to the candidate area into the candidate area;
An occupant detection apparatus comprising: an area type determination unit that determines whether or not the candidate area processed by the candidate area update unit is an image area corresponding to an occupant's face in a vehicle.   The area type determining means is configured to determine whether the candidate area is the face of an occupant in the vehicle when at least one of the aspect ratio, circularity, and size of the candidate area after processing by the candidate area updating means is within a predetermined range. The occupant detection device according to claim 1, wherein the occupant detection device is determined to be an image region corresponding to.   The area type determination means calculates the amount of movement of the candidate area after processing by the candidate area update means, and if the movement amount continues outside the predetermined threshold range for a predetermined time, the candidate area is other than the occupant's face The occupant detection device according to claim 1, wherein the occupant detection device is determined to be an image region.   When there are a plurality of candidate areas after processing by the candidate area updating means, the area type determining means images the occupant from the front, and subjects corresponding to the candidate areas for each of the plurality of candidate areas 2. The distance between the image capturing unit and the image capturing unit is compared, and it is determined that the candidate region corresponding to the subject having the longest distance from the image capturing unit is an image region corresponding to the face of an occupant in the vehicle. The occupant detection device according to any one of claims 3 to 3.   The region type determination unit is an image region corresponding to a passenger's face in a vehicle when the uniformity of pixel values in the candidate region after processing by the candidate region update unit is within a predetermined threshold range. The occupant detection device according to any one of claims 1 to 4, wherein the occupant detection device is determined.   The region type determination unit is an image region other than the occupant's face when the shape or position of the candidate region processed by the candidate region update unit does not change for a predetermined time or more and a predetermined value or more. The occupant detection device according to any one of claims 1 to 5, wherein the occupant detection device is determined.   The occupant detection device according to any one of claims 1 to 6, wherein the image captured by the imaging unit is a far-infrared image. A generation step for generating an edge image from an image in the vehicle;
An extraction step of extracting a region sandwiched between the edge images as a candidate region;
A determination step of determining whether or not a difference value between a pixel value of a pixel adjacent to the candidate region and a pixel value of the candidate region is within a predetermined range;
When the difference value is within a predetermined range, an update step of updating the candidate area by integrating pixels adjacent to the candidate area into the candidate area;
Determining whether the candidate area after the determination and updating step is an image area corresponding to the face of the passenger in the vehicle.

Images