JP5298111B2 - Occupant detection device - Google Patents

Description

  The present invention relates to an occupant detection device.

2. Description of the Related Art Conventionally, for example, an apparatus that detects an occupant's head position from an image obtained by imaging with an imaging device and determines the occupant's seating state from the head position is known (for example, see Patent Document 1).
In addition, conventionally, for example, an apparatus that detects an occupant's head center-of-gravity position and a shoulder width from an image obtained by imaging by an imaging device and determines whether the occupant is an adult or a child is known ( For example, see Patent Document 2).

JP 2001-331790 A JP 2007-198929 A

By the way, according to the apparatus according to the above-described prior art, the head position fluctuates as the occupant's posture changes, and thus there is a possibility that an adult and a child cannot be accurately distinguished.
Moreover, it may be difficult to detect the shoulder width due to the operating state of the occupant's arm, the imaging direction of the imaging device, etc., and it is possible to accurately determine whether the occupant is an adult or a child. The problem that it is not possible arises.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide an occupant detection device capable of accurately discriminating between an adult and a child.

  In order to solve the above-described problems and achieve the object, the occupant detection device according to the first aspect of the present invention captures an imaging region in the passenger compartment and outputs an imaging result (for example, in the embodiment). A distance image generating means (for example, in the embodiment) that generates a distance image having distance information of the imaging region in a three-dimensional space based on the imaging result output from the imaging means. Based on the processing unit 23) and the distance image, volume determination means that estimates the volume of the occupant region and determines whether the volume is equal to or greater than a predetermined volume threshold (for example, the volume determination unit 56 in the embodiment). ) And a knee determination means (for example, a knee in the embodiment) for determining whether or not there is a passenger's knee in the passenger compartment below a predetermined position in the vehicle vertical direction based on the distance image. Part determination unit 57) and the volume determination means When the volume of the occupant area is determined to be equal to or greater than the predetermined volume threshold, and the knee determination means determines that the knee is present below the predetermined position, the An occupant who determines that the occupant is an adult and determines that the occupant is a child when the knee determination unit determines that the knee does not exist below a predetermined position in the vehicle vertical direction. Determination means (for example, an occupant determination unit 58 in the embodiment).

  Further, in the occupant detection device according to the second aspect of the present invention, the knee determination means is a fixed length inverse in the reverse direction of the normal vector in the three-dimensional space for each of the plurality of pixels constituting the distance image. Internal coordinate setting means (for example, step S11 in the embodiment) that calculates a vector and uses the position coordinates in the three-dimensional space designated by the inverse vector as internal coordinates, and a plurality of the three-dimensional space For each unit space, a score value relating to the total number of the internal coordinates included in the unit space is calculated, and the score value of the unit space including the internal coordinates corresponding to each of the plurality of pixels Score image generation means (for example, step S12 in the embodiment) that generates a score image shown corresponding to each pixel, and knee part extraction means (for example, implementation) that extracts the knee based on the score value Comprising the step S61) and in state.

  Furthermore, in the occupant detection device according to the third aspect of the present invention, a head region extraction means (for example, a head region determination unit 54 in the embodiment) that extracts the head region of the occupant based on the distance image is provided. The volume determination means estimates the volume of the occupant area including the head area.

According to the occupant detection device according to the first aspect of the present invention, the size of only the head is smaller than the size of the entire body between the adult and the child, and the position of the head of the adult sitting on the seat In some cases, the head position of the child standing on the seat is almost the same.
At this time, it is easier to detect the knee part in the seated adult than the knee part is detected in the standing child, and the volume of the adult sitting on the seat and the volume of the child standing on the seat are the predetermined volume threshold. Even if it is determined as above, whether or not the occupant is an adult or a child by determining whether or not the knee exists below a predetermined position in the vehicle vertical direction. Can be easily and accurately determined.

  Furthermore, according to the occupant detection device according to the second aspect of the present invention, a score value for each unit space is calculated by a voting method using internal coordinates based on a fixed-length inverse vector set for each pixel, and the score value is Since the occupant's knee is extracted according to the spherical index, the detection accuracy can be improved while preventing an increase in calculation load.

  Further, according to the occupant detection device according to the third aspect of the present invention, the position of the head of the adult seated on the seat and the position of the head of the child standing on the seat substantially coincide, and further, the seat on the seat Even if it is determined that the volume of the adult and the volume of the child standing on the seat are approximately the same, it is determined whether or not the knee exists below the predetermined position in the vehicle vertical direction. By doing so, it can be easily and accurately determined whether the occupant is an adult or a child.

1 is a configuration diagram of an occupant detection device according to an embodiment of the present invention. 1 is a configuration diagram of an occupant detection device according to an embodiment of the present invention. It is a figure which shows the example of the kernel of the partial differential filter which concerns on embodiment of this invention. It is a figure which shows typically the correspondence of observation pixel coordinate P _{(v, u)} , head vector H _{(v, u),} and internal coordinate I _{(v, u)} concerning embodiment of this invention. It is a figure which shows an example of the voxel space V which concerns on embodiment of this invention by a sheet coordinate system. It is a figure which shows the example of the correspondence of each pixel (v, u) of the distance image which concerns on embodiment of this invention, internal coordinate I _{(v, u),} and the voxel space V. It is a figure which shows an example of the distance image which concerns on embodiment of this invention, an example of distribution of the score value per voxel space V in a three-dimensional space, and an example of the score image S. It is a figure which shows an example of the binarized image B which concerns on embodiment of this invention, and an example of the area division | segmentation image E. It is a figure which shows the example of two image area | regions Ea and Eb divided | segmented by the area | region division part which concerns on embodiment of this invention, and the image area | region F0 which two image area | region Ea and Eb were couple | bonded by the area | region coupling | bond part. . It is a figure which shows the example of two image area | regions Ea and Eb divided | segmented by the area | region division part which concerns on embodiment of this invention, and the image area | region F0 which two image area | region Ea and Eb were couple | bonded by the area | region coupling | bond part. . An example of the region combined image F generated by the region combining unit according to the embodiment of the present invention, an example of the region combined image F processed by the spherical shape determining unit, and the region combined image F processed by the volume determining unit It is a figure which shows an example. It is a figure which shows an example of the maximum value of each axis | shaft in the camera coordinate system of the image area | region Fm which comprises the area | region combined image F which concerns on embodiment of this invention. It is a figure which shows an example of the correspondence of normal vector N _{(v, u)} and front area | region SA in the sheet coordinate system which concerns on embodiment of this invention. It is a figure which shows an example of front area SA extracted from the distance image which concerns on embodiment of this invention. It is a figure which shows the example of the side view image which looked at the passenger | crew area | region which concerns on embodiment of this invention from the vehicle left-right direction, the luminance image of a passenger | crew area, and the score image of a passenger | crew area. It is a flowchart which shows operation | movement of the passenger | crew detection apparatus which concerns on embodiment of this invention. It is a flowchart which shows the process of the head area | region detection shown in FIG. It is a flowchart which shows the process of the internal coordinate setting shown in FIG. It is a flowchart which shows the process of the score image generation shown in FIG. It is a flowchart which shows the process of head region determination shown in FIG. It is a flowchart which shows the process of the knee part determination shown in FIG.

  Hereinafter, an occupant detection device according to an embodiment of the present invention will be described with reference to the accompanying drawings.

  The occupant detection device 10 according to the present embodiment includes a monocular distance image sensor 11 and a control device 12 as shown in FIGS. 1 and 2, for example, and is an airbag (for example, mounted on a vehicle 13). , A front airbag) 14 constitutes a part of an airbag system 15 that controls the deployment of the airbag.

  The monocular distance image sensor 11 is a monocular imaging device, and is disposed, for example, in the center of the front portion of the roof of the vehicle 13 (for example, the position between the left and right sun visors), and at least the seat 16 (for example, the interior of the vehicle interior). A passenger seat), a passenger seated on the seat 16, a child seat installed on the seat 16, and the like are set as imaging targets, and these imaging targets are set to be imaged from the front side obliquely upward.

  The monocular distance image sensor 11 includes a light emitting unit 21, a light receiving unit 22, and a processing unit 23, for example.

  For example, the light emitting unit 21 diffuses light emitted from the light emitter 31 toward the imaging target, such as a light emitter 31 such as an LED that emits light in the infrared region, a drive circuit 32 that controls the light emission of the light emitter 31. A scattering plate 33 is provided.

  The drive circuit 32 is controlled according to, for example, a light emission control command signal output from the light receiving unit 22 and a light amount control command signal output from the processing unit 23.

The light emission control command signal output from the light receiving unit 22 synchronizes the operation of the light emitting unit 21 and the operation of the light receiving unit 22, and the light emission pulse output from the light emitting unit 21 and the light emission pulse is an object to be imaged. In order to be able to detect the phase difference (time difference) from the received light pulse that is reflected and input to the light receiving unit 22, the light emission timing of the light emission pulse is instructed.
The light amount control command signal output from the processing unit 23 instructs the light emission amount of the light emission pulse output from the light emitting unit 21.

  For example, the light receiving unit 22 receives a light receiving pulse formed by reflecting a light emission pulse emitted from the light emitting unit 21 toward the imaging target, reflected by the imaging target, a filter 37, and the light received through the lens 36 and the filter 37. And a light receiving element 38 such as a CMOS sensor for detecting a pulse.

The light receiving unit 22 outputs a light emission control command signal to the light emitting unit 21 in accordance with the control signal output from the processing unit 23, and outputs image data as a detection result of the light reception pulse by the light receiving element 38.
This image data is composed of pixels in a two-dimensional array corresponding to the position at which the light receiving pulse is detected in the light receiving element 38 (that is, the position of the reflection point of the light emission pulse on the imaging target). Information on the amount of light (that is, the number of pulses) of the received light pulse detected in step 1 and information on the phase difference (time difference) between the light emission pulse and the light reception pulse.

  The processing unit 23 outputs a light amount control command signal to the light emitting unit 21 and also outputs control signals to instruct the light receiving unit 22 to perform various operations. For example, the distance image generating unit 41 and the luminance image generating unit 42. And is configured.

  The distance image generation unit 41 is based on the image data output from the light receiving unit 22, and information on the phase difference (time difference) between the light emission pulse and the light reception pulse and the speed of the light emission pulse and the light reception pulse (that is, the light intensity) Speed), information on the distance from the monocular distance image sensor 11 to the reflection point to be imaged in a three-dimensional space is generated. Then, a distance image indicating the distance from the monocular distance image sensor 11 to the imaging target is generated for each pixel of the two-dimensional array, and this distance image is output to the control device 12.

  The luminance image generation unit 42 generates a luminance image indicating the light amount (that is, the number of pulses) of the received light pulse detected by the light receiving element 38 for each pixel of the two-dimensional array based on the image data output from the light receiving unit 22. Then, this luminance image is output to the control device 12.

  The control device 12 includes, for example, an image processing unit 51, an internal coordinate setting unit 52, a score image generation unit 53, a head region determination unit 54, a sheet region detection unit 55, a volume determination unit 56, and a knee portion. The determination unit 57, an occupant determination unit 58, and a drive control unit 59 are provided.

  First, based on the distance image output from the monocular distance image sensor 11, the image processing unit 51 determines the position of the reflection point of the imaging target in the three-dimensional space corresponding to the distance information for each pixel of the two-dimensional array. Three X images, Y images, and Z images (X coordinate, Y coordinate, Z coordinate) described in the coordinate system and corresponding to each pixel of the two-dimensional array are shown in the camera coordinate system. XYZ image) is generated.

The camera coordinate system is, for example, an XYZ coordinate system in which the light receiving axis orthogonal to the imaging surface of the light receiving element 38 of the monocular distance image sensor 11 is the Z axis.
These XYZ images are subjected to image processing such as smoothing and noise removal with reference to the luminance image.

Further, the image processing unit 51 converts each axis component (X coordinate, Y coordinate, Z coordinate) in the camera coordinate system into the seat coordinate system, that is, the vehicle longitudinal direction as the T axis, the vehicle lateral direction as the B axis, and the vehicle vertical direction. Converted into each axis component (T coordinate, B coordinate, H coordinate) in the TBH coordinate system as the H axis, a predetermined structure (for example, pillar, door, dashboard, center) stored in advance in the vehicle interior Data associated with the console).
Then, each axis component (T coordinate, B coordinate, H coordinate) in the sheet coordinate system after this data removal is converted into each axis component (X coordinate, Y coordinate, Z coordinate) in the camera coordinate system, and this Three X images, Y images, and Z images (XYZ images) based on the converted axis components (X coordinate, Y coordinate, Z coordinate) are output.

  Based on the three XYZ images output from the image processing unit 51, the internal coordinate setting unit 52 generates a fixed-length inverse vector in the reverse direction of the normal vector in the three-dimensional space for each of the plurality of pixels constituting the distance image. The position coordinates in the three-dimensional space specified by the inverse vector are calculated and set as internal coordinates.

  First, the internal coordinate setting unit 52 convolves the three XYZ images output from the image processing unit 51 with a partial differential filter such as a Sobel filter including a kernel shown in FIGS. By performing the partial differential in the horizontal direction and the partial differential in the vertical direction for each pixel by calculation, a horizontal partial differential image (X horizontal partial differential image Xu, Y horizontal partial differential image Yu, Z is obtained for each of the three XYZ images. Lateral partial differential image Zu) and vertical partial differential image (X vertical partial differential image Xv, Y vertical partial differential image Yv, Z vertical partial differential image Zv) are calculated.

Then, the internal coordinate setting unit 52, for example, a normal vector N ₍ for each pixel (v, u) of the distance image specified by the vertical pixel v and the horizontal pixel u described as shown in the following formula (1). _{v, u)} are _converted into pixel values (X lateral partial differential image Xu _{(v, u)} , Y lateral partial differential image Yu _{(v, u)} , Z lateral partial differential image Zu _{(v, u).} ) And pixel values of each vertical partial differential image (X vertical partial differential image Xv _{(v, u)} , Y vertical partial differential image Yv _{(v, u)} , Z vertical partial differential image Zv _{(v, u)} ), and camera It is calculated based on the unit vector (i, j, k) of each axis (X axis, Y axis, Z axis) in the coordinate system.

Then, the internal coordinate setting unit 52 performs normalization by dividing the normal vector N _{(v, u)} by the pixel area | N _{(v, u)} |, for example, as shown in the following formula (2). The inverse vector E is calculated by inverting the sign.

Then, the internal coordinate setting unit 52 multiplies the inverse vector E _{(v, u)} by a predetermined head thickness HeadThick (for example, 90 mm, etc.) with respect to the thickness of the head of the human body, for example, as shown in the following formula (3). Then, a fixed-length head vector H _{(v, u)} is calculated.

Then, as shown in FIG. 4 and the following mathematical formulas (4) and (5), for example, the internal coordinate setting unit 52 observes pixel coordinates P _{(in the} camera coordinate system) indicating the position of each pixel (v, u) in the distance image. _{v, u)} and the head vector H _{(v, u)} are combined to calculate the internal coordinates I _{(v, u)} .

Then, the internal coordinate setting unit 52 calculates each axis component (X internal coordinate, Y internal coordinate, Z internal coordinate) of the internal coordinate I _{(v, u)} corresponding to each pixel (v, u) of the distance image. Three X internal coordinate images IX, Y internal coordinate images IY, and Z internal coordinate images IZ shown in association with the pixel values for each pixel (v, u) are generated.

The score image generation unit 53 calculates a score value related to the total number of internal coordinates I _{(v, u)} included in the unit space for each of a plurality of unit spaces constituting the three-dimensional space, and each pixel ( Corresponding to the pixel value for each pixel (v, u), with the score value of the unit space including the internal coordinates I _{(v, u)} corresponding to each _{v, u)} as a spherical index of a predetermined size A score image S shown is generated.

  First, as shown in FIG. 5, for example, the score image generation unit 53 has, for example, a predetermined length as a plurality of unit spaces (three-dimensional coordinate space) constituting a three-dimensional space including the imaging region of the monocular distance image sensor 11. A plurality of cubes (voxel spaces) V having a side length (for example, 50 mm) are set.

Then, the score image generation unit 53, for example, as shown in FIGS. 6A and 6B, includes a voxel including an internal coordinate I _{(v, u)} corresponding to each pixel (v, u) of the distance image. For the space V, the pixel area of each pixel (v, u) is added as a score value.
Thus, for example, when the addition of the score values for each voxel space V is completed for all the pixels (v, u) of the distance image L as shown in FIG. 7A, for example, as shown in FIG. A distribution of score values in units of voxel space V in a three-dimensional space is obtained.

Then, for example, as shown in FIG. 7C, the score image generation unit 53 scores the voxel space V including the internal coordinates I _{(v, u)} corresponding to each pixel (v, u) of the distance image. A score image S is generated that indicates the value corresponding to the pixel value for each pixel (v, u).

  The head region determination unit 54, for each of the three internal coordinate images IX, IY, and IZ generated by the internal coordinate setting unit 52, for example, a Sobel filter including a kernel shown in FIGS. The horizontal partial differential image and the vertical partial differential image are calculated by performing the partial differential in the horizontal direction and the partial differential in the vertical direction for each pixel by the convolution calculation using the partial differential filter.

Then, the head region determination unit 54 synthesizes the three internal edge images DX, DY, and DZ by synthesizing the lateral partial differential image and the vertical partial differential image for each of the three internal coordinate images IX, IY, and IZ. Generate.
Then, for example, as shown in the following formula (6), the three inner edge images DX, DY, DZ are synthesized, and further divided by the pixel area | N _{(v, u)} | to perform normalization. An internal edge image D in which edges are extracted from the distribution of a plurality of internal coordinates I _{(v, u) in} a three-dimensional space is generated.

  Then, the head region determination unit 54 sets the region where the pixel value is less than the predetermined threshold EdgeThreshold in the internal edge image D to “1” and the pixel value is equal to or greater than the predetermined threshold EdgeThreshold, for example, as shown in Equation (7) below. By performing a binarization process for setting the area to “0”, for example, a binarized image B as shown in FIG. 8A is generated.

  The head region determination unit 54 performs a plurality of different image regions En (n is, for example, as illustrated in FIG. 8B by performing a labeling process such as 8-neighbor labeling on the binarized image B. An area-divided image E made up of an arbitrary natural number) is generated.

Then, the head region determination unit 54 refers to the three internal coordinate images IX, IY, and IZ generated by the internal coordinate setting unit 52 while referring to the three internal coordinate images IX, IY, and IZ. Image regions whose difference in average value of _{(v, u)} is equal to or less than a predetermined difference are combined into a single image region to generate a region combined image F.
Note that image regions that are not combined with each other in the region divided image E are included in the region combined image F without change.

For example, when an edge that is equal to or greater than a predetermined threshold EdgeThreshold is extracted due to the brim of the hat worn by the occupant in the inner edge image D, for example, as shown in FIGS. 9A and 10A, The head region determination unit 54 divides the passenger's head region into two image regions Ea and Eb.
On the other hand, since the two image regions Ea and Eb are head regions that are single human body parts, the internal coordinates I _{(v, u)} in the image regions Ea and Eb are substantially equal, and the two image regions Ea. , Eb, the difference between the average values of the internal coordinates I _{(v, u)} is equal to or smaller than a predetermined difference. For example, as shown in FIG. 9B and FIG. Regions Ea and Eb are combined into a single image region F0.

  Then, the head region determination unit 54 refers to the score image S generated by the score image generation unit 53, and in the region combined image F, the average value of the score values among a plurality of image regions is a predetermined threshold average. Image areas below the value are removed.

  For example, as shown in FIG. 11A, the head region determination unit 54 has a predetermined degree of spherical shape of a predetermined size from a region combined image F composed of a plurality of image regions F1,. Lower image areas, for example, the image area F3 corresponding to the occupant's torso and the image areas F4 and F5 corresponding to the left and right thighs of the occupant are removed, for example after removal as shown in FIG. The region combined image F is generated.

Then, the head region determination unit 54 refers to the XYZ image generated by the image processing unit 51, and in the region combined image F, the volume of the plurality of image regions is equal to or higher than the upper limit volume value or lower than the lower limit volume value. Remove image area.
The upper limit volume value and the lower limit volume value are threshold values for the volume of the head of the human body, and are stored in advance as the volume of the head of the human body is less than the upper limit volume value and larger than the lower limit volume value.

For example, as shown in FIG. 12, the head region determination unit 54 determines the maximum value of each axis in the camera coordinate system for each of a plurality of image regions Fm (m is an arbitrary natural number) constituting the region combined image F ( XMAX, YMAX, ZMAX) and the minimum value (XMIN, YMIN, ZMIN) are calculated.
Then, as shown in the following formula (8), the volume vol is calculated from the maximum value (XMAX, YMAX, ZMAX) and the minimum value (XMIN, YMIN, ZMIN) of each axis, and this volume vol is the head of the human body. It is judged whether it is more than the upper limit volume value which deviates from the predetermined range with respect to the volume, or below the lower limit volume value.

  Accordingly, the head region determination unit 54 determines that the volume is greater than or equal to the upper limit volume value or less than or equal to the lower limit volume value from the region combined image F configured by the plurality of image regions F1 and F2 as illustrated in FIG. For example, the image region F1 corresponding to the headrest 16a of the seat 16 is removed, and a region combined image F after removal as shown in FIG. 11C is generated.

  The head region determination unit 54 refers to the score image S generated by the score image generation unit 53, and the image having the highest average score value among the plurality of image regions constituting the region combined image F. The region is determined to be a head region where the occupant's head is present.

For example, the sheet region detection unit 55 performs various processes such as appropriate feature amount extraction, template matching, and comparison with a sheet model on the XYZ image or the distance image, thereby the region where the sheet 16 exists. Is detected.
For example, as shown in FIGS. 13 and 14, the seat region detection unit 55 determines that the region where the normal vector N _{(v, u)} is inclined in the vehicle left-right direction (B direction) is a predetermined threshold angle or less. Extract from the distance image as SA.

Then, the sheet region detection unit 55 removes, for example, the region combined image F formed from the region divided image E by the head region determination unit 54 (that is, an image region having an average score value equal to or less than a predetermined threshold average value). The maximum value and the minimum value of the B axis in the sheet coordinate system of the front area SA for each image area from which the front area SA is extracted among a plurality of image areas constituting the area combined image F) before being processed From this, the position (for example, the center position) and the length (B direction width) in the vehicle left-right direction (B direction) are calculated.
The front area SA in which the position in the B direction is within the predetermined position range and the length in the B direction (B direction width) is within the predetermined length range is detected as the headrest area in which the headrest 16a of the seat 16 is present. The position of the headrest 16a is calculated based on the headrest area.

Then, the seat region detection unit 55 is centered on the position of the headrest 16a in a TH image in the seat coordinate system (that is, a side view image when the imaging region of the monocular distance image sensor 11 is viewed from the vehicle left-right direction). The waist region of the seat 16 is searched between two different arcs.
Then, from the thickness of the waist region, the predetermined seat thickness stored in advance is subtracted from the vehicle rear side, or the predetermined human body thickness stored in advance is subtracted from the vehicle front side to calculate the position of the waist. Then, the front shape of the seat 16 is calculated based on the position of the headrest 16a and the position of the waist.
Then, the area on the vehicle rear side of the calculated frontal shape is detected as the area of the seat 16, and the area of the seat 16 is removed from the XYZ image (or the distance image corresponding to the XYZ image).

The volume determination unit 56 configures, for example, a sheet region removal image based on the sheet region removal image obtained by removing the region of the sheet 16 from the XYZ image (or the distance image corresponding to the XYZ image) by the sheet region detection unit 55. A region (occupant region) including a head region detected by the head region determination unit 54 based on the total number of unit spaces including at least one data point among a plurality of unit spaces (for example, the voxel space V). Estimate volume.
Then, it is determined whether or not the estimated volume of the occupant area is equal to or greater than a predetermined volume, and the determination result is output.
The predetermined volume with respect to the volume of the occupant region is set to be a lower limit value of a volume range including, for example, the volume of an adult male or the volume of a child standing on the seat 16.

  When the volume determination unit 54 determines that the volume of the occupant region is equal to or greater than the predetermined volume, the knee determination unit 57 refers to the score image S and, for example, based on the seat region removal image, the vehicle vertical direction It is determined whether or not the occupant's knee exists below the predetermined position.

For example, as shown in FIG. 15A, the knee determination unit 57 sets the predetermined position in the vehicle vertical direction to a position at least above the knee V of the adult seated on the seat 16.
Then, in a region below the height Hth of the predetermined position (for example, the height from the floor in the vehicle interior) Hth, the number of pixels having a score value equal to or greater than a predetermined value is calculated, and the number of pixels is a predetermined threshold value. It is determined whether it is above.
When it is determined that the number of pixels is equal to or greater than the predetermined threshold, it is determined that the occupant's knee is present. On the other hand, when the number of pixels is determined to be less than the predetermined threshold, It is determined that there is no knee.

  For example, as shown in FIGS. 15A to 15C, the occupant determination unit 58 is located below the height Hth of a predetermined position (for example, the height from the floor in the passenger compartment) in the knee determination unit 57. When it is determined that the occupant's knee V is present in the region, it is determined that there is an adult seated on the seat 16 in the occupant region. On the other hand, if it is determined that the occupant's knee V is not present in the region below the height Hth of the predetermined position, it is determined that there is a child standing on the seat 16 in the occupant region.

  The drive control unit 59 is based on the determination result of the occupant determination unit 58 and the detection result signal output from the collision sensor 70 such as an acceleration sensor that detects sudden deceleration of the vehicle 13. Controls the deployment of the bag (eg, front airbag) 14.

  The occupant detection device 10 according to the present embodiment has the above-described configuration. Next, the operation of the occupant detection device 10 will be described.

First, for example, in step S01 shown in FIG. 16, as preprocessing for the distance image output from the monocular distance image sensor 11, XYZ images in the camera coordinate system are generated, and luminance images are referred to for these XYZ images. Then, image processing such as smoothing and noise removal is performed.
Then, the image-processed XYZ image is temporarily converted into an image in the seat coordinate system to remove data related to a predetermined structure (eg, pillar, door, dashboard, center console, etc.) in the passenger compartment. The image after the removal is converted again into an XYZ image in the camera coordinate system.

  Next, in step S02, a head region detection process for detecting a head region where the head of the occupant is present is performed based on the distance image, as will be described later.

Next, in step S03, a sheet area detection process for detecting an area where the sheet 16 is present is performed based on the distance image.
In this step S03, for example, a region where the normal vector N _{(v, u)} is inclined in the vehicle left-right direction (B direction) is equal to or smaller than a predetermined threshold angle is defined as the front region SA, and the position in the B direction is within the predetermined position range. Further, the front area SA in which the length in the B direction (B direction width) is within a predetermined length range is detected as the headrest area where the headrest 16a of the seat 16 is present.

And even if the angle of the seat back 16b changes, the seat 16 changes the position of the waist from the position of the headrest 16a according to the relative positional relationship between the position of the headrest 16a and the position of the waist. The position is calculated, and the front shape of the seat 16 is calculated based on the position of the headrest 16a and the position of the waist.
Then, the area on the vehicle rear side of the calculated frontal shape is detected as the area of the seat 16, and the area of the seat 16 is removed from the XYZ image (or the distance image corresponding to the XYZ image).

Next, in step S04, the head detected by the head region detection unit 52 based on the sheet region removal image obtained by removing the region of the sheet 16 from the XYZ image (or the distance image corresponding to the XYZ image). The volume of the area (occupant area) including the area is estimated.
And it is determined whether the volume of the estimated passenger | crew area | region is more than predetermined volume.

  Next, in step S05, when it is determined that the volume of the occupant area is equal to or greater than the predetermined volume, for example, referring to the score image S, based on the seat area removal image, from a predetermined position in the vehicle vertical direction. Also, a knee determination process is performed to determine whether the occupant's knee exists on the lower side.

Next, in step S06, when it is determined in the knee determination process that the occupant's knee exists in a region below the height of a predetermined position (for example, the height from the floor in the passenger compartment). It is determined that there is an adult seated on the seat 16 in the passenger area.
On the other hand, when it is determined in the knee determination process that the occupant's knee does not exist in the region below the height Hth at the predetermined position, there is a child standing on the seat 16 in the occupant region. Judge that.

  Next, in step S07, a determination result based on an occupant determination process as to whether an adult or a child is present in the occupant area, and a collision sensor 70 such as an acceleration sensor that detects sudden deceleration of the vehicle 13, for example, are output. Based on the signal of the detection result, deployment of an airbag (for example, a front airbag) 14 mounted on the vehicle 13 is controlled, and the process proceeds to the end.

Hereinafter, the head region detection process in step S02 described above will be described.
First, in step S11 shown in FIG. 17, for example, a fixed-length inverse vector is calculated in the direction opposite to the normal vector in the three-dimensional space for each of the plurality of pixels constituting the distance image corresponding to the XYZ image. An internal coordinate setting process is performed in which the position coordinates in the three-dimensional space designated by the vector are internal coordinates.

  Next, in step S12, a score value related to the total number of internal coordinates included in the unit space is calculated for each of a plurality of unit spaces constituting the three-dimensional space, and the internal coordinates corresponding to each pixel of the distance image are calculated. A score image generation process is performed for generating a score image S indicating the score value of a unit space including a corresponding to the pixel value of each pixel as a spherical index of a predetermined size.

  Next, in step S13, a head region determination that determines a head region in which the occupant's head is present from among a plurality of image regions, based on the distribution of a plurality of internal coordinates in the three-dimensional space and the score image S. Perform the following process.

The internal coordinate setting process in step S11 described above will be described below.
First, for example, in step S21 shown in FIG. 18, the horizontal partial differentiation and the vertical partial differentiation are performed on each of the three XYZ images by a convolution operation using a partial differential filter such as a Sobel filter. By performing the above, a horizontal partial differential image and a vertical partial differential image are calculated for every three XYZ images.

Next, in step S22, using the above mathematical formula (1), the lateral partial differential image and the vertical partial differential image, and the unit vector of each axis (X axis, Y axis, Z axis) in the camera coordinate system. Then, a normal vector N _{(v, u)} for each pixel (v, u) of the distance image specified by the vertical pixel v and the horizontal pixel u is calculated.

Next, in step S23, using the above formulas (2) and (3), an inverse vector having a fixed length in the reverse direction of the normal vector N _{(v, u)} , that is, a head vector H _{(v, u).} Is calculated.

Next, in step S24, using the above formulas (4) and (5), the position of each pixel (v, u) of the distance image and the head of the observation pixel coordinate P _{(v, u)} indicating the camera coordinate system are used. The internal coordinate I _{(v, u)} is calculated by combining with the part vector H _{(v, u),} and the process proceeds to return.

The score image generation process in step S12 described above will be described below.
First, for example, in step S31 shown in FIG. 19, a predetermined length (for example, 50 mm) is used as a plurality of unit spaces (three-dimensional coordinate space) that constitute a three-dimensional space including the imaging region of the monocular distance image sensor 11. A plurality of cubes (voxel spaces) V having a side length are set.

Next, in step S32, the pixel area of each pixel (v, u) with respect to the voxel space V including the internal coordinates I _{(v, u)} corresponding to each pixel (v, u) of the distance image. Is added as a score value.

Next, in step S33, the score value of the voxel space V including the internal coordinates I _{(v, u)} corresponding to each pixel (v, u) of the distance image is obtained for each pixel (v, u). A score image S shown corresponding to the pixel value is generated, and the process proceeds to return.

Below, the process of head region determination in step S13 mentioned above is demonstrated.
First, for example, in step S41 shown in FIG. 20, each axis component (X internal coordinate, Y internal coordinate, Z internal coordinate ₎ of the internal coordinate I _{(v, u)} corresponding to each pixel (v, u) of the distance image. ) For each of the three X internal coordinate images IX, Y internal coordinate images IY, and Z internal coordinate images IZ shown corresponding to the pixel values for each pixel (v, u), for example, a partial differential filter such as a Sobel filter By performing a convolution operation in accordance with the above, a horizontal partial differential image and a vertical partial differential image are calculated by performing a partial differential in the horizontal direction and a partial differential in the vertical direction for each pixel.

Next, in step S42, three internal edge images DX, DY, and DZ are generated by synthesizing the lateral partial differential image and the vertical partial differential image for each of the three internal coordinate images IX, IY, and IZ. .
Then, by synthesizing the three internal edge images DX, DY, and DZ, and dividing by the pixel area | N _{(v, u)} |, normalization is performed, whereby a plurality of internal coordinates I _(v, An inner edge image D in which edges are extracted from the distribution of _u) is generated.
Then, in the internal edge image D, binarization is performed by performing binarization processing in which an area where the pixel value is less than the predetermined threshold EdgeThresh is “1” and an area where the pixel value is equal to or greater than the predetermined threshold EdgeThresh is “0”. An image B is generated.

Next, in step S43, the binarized image B is subjected to a labeling process such as 8-neighbor labeling to generate a region divided image E composed of a plurality of different image regions.
Next, in step S44, an average value of the internal coordinates I _{(v, u)} is calculated for each of a plurality of image areas constituting the area divided image E.
Next, in step S45, image regions in which the difference between the average values of the internal coordinates I _{(v, u)} is larger than a predetermined difference are image regions that are independent from each other, while the internal coordinates I _{(v, u).} the difference between the average value _of) the image region among more than a predetermined difference and combined into a single image area to produce an area bonded image F.

Next, in step S46, the average value of the score values is calculated for each of a plurality of image regions constituting the region combined image F.
Next, in step S47, image areas whose average score value is equal to or less than a predetermined threshold average value are removed from a plurality of image areas constituting the area combined image F.

Next, in step S48, the maximum value and the minimum value of each axis in the camera coordinate system are calculated for each of a plurality of image regions constituting the region combined image F, and each axis is calculated using the above equation (8). The volume of the image area is calculated from the maximum value and the minimum value.
Next, in step S49, an image area whose volume is greater than or equal to the upper limit volume value or less than the lower limit volume value out of the predetermined range is removed from the plurality of image areas constituting the area combined image F.

Next, in step S50, an area in which the normal vector N _{(v, u)} for each pixel (v, u) of the distance image is tilted in the vehicle left-right direction (B direction) is equal to or smaller than a predetermined threshold angle. An area SA is extracted from the distance image.

Next, in step S51, the maximum value of the B axis in the sheet coordinate system of the front area SA for each image area from which the front area SA is extracted among the plurality of image areas constituting the area combined image F. And the width dimension (B direction width) of the vehicle left-right direction is calculated from the minimum value.
Next, in step S52, an image area having a front area SA whose width in the B direction is equal to or larger than a predetermined threshold is removed from a plurality of image areas constituting the area combined image F. Then, referring to the score image S, in the region combined image F at this time point, the image region having the highest average score value among the plurality of image regions is the head region where the occupant's head exists. Judge that there is.

Hereinafter, the knee determination process in step S05 described above will be described.
First, for example, in step S61 shown in FIG. 21, the score value is greater than or equal to a predetermined value in a region below the height Hth of a predetermined position in the vehicle vertical direction (for example, the height from the floor in the vehicle interior). The number of pixels is calculated.
Next, in step S62, it is determined whether or not the number of pixels is equal to or greater than a predetermined threshold value.
If this determination result is "YES", the process proceeds to step S63, and in this step S63, it is determined that there is an adult seated on the seat 16 in the passenger area, and the process proceeds to return.
On the other hand, if this determination is “NO”, the flow proceeds to step S 64, and in this step S 64, it is determined that there is a child standing on the seat 16 in the passenger area, and the flow proceeds to return.

As described above, according to the occupant detection device 10 according to the present embodiment, the size of only the head is smaller than the size of the entire body between the adult and the child, and the head of the adult sitting on the seat 16 is smaller. The position of the part and the position of the child's head that has stood up on the seat 16 may substantially coincide.
At this time, it is easier to detect a knee part in a seated adult than in a child who has stood up, and it is determined whether or not the knee part exists below a predetermined position in the vehicle vertical direction. By doing so, it can be easily and accurately determined whether the occupant is an adult or a child.

  Further, a score value for each unit space is calculated by a voting method using internal coordinates based on a fixed-length inverse vector set for each pixel (v, u) of the distance image, and the score value becomes a spherical index. Accordingly, since the knee part of the occupant is extracted, detection accuracy can be improved while preventing an increase in calculation load.

  Further, the position of the head of the adult seated on the seat 16 and the position of the head of the child standing on the seat 16 substantially coincide, and further, the volume of the adult seated on the seat 16 and the child standing on the seat 16 Even if it is determined that the volume of the occupant is comparable, whether or not the occupant is an adult by determining whether or not the knee portion exists below a predetermined position in the vehicle vertical direction, Alternatively, it is possible to easily and accurately determine whether the child is a child.

  In the above-described embodiment, the knee determination unit 57 determines adults and children based on the number of pixels. However, the determination is not limited to this, and determination is performed based on other indicators such as a pixel area. Also good.

DESCRIPTION OF SYMBOLS 10 Occupant detection apparatus 11 Monocular distance image sensor 12 Control apparatus 22 Light-receiving part (imaging means)
23 processing unit (distance image generating means)
52 Internal coordinate setting unit 53 Score image generating unit 54 Head region determining unit (head region extracting means)
55 Sheet region detection unit 56 Volume determination unit (volume determination means)
57 Knee determination unit (knee determination means)
58 Crew determination unit (Crew determination means)
59 Drive control unit step S11 Internal coordinate setting unit step S12 Score image generation unit step S61 Knee part extraction unit

Claims (3)

Imaging means for imaging an imaging area in the passenger compartment and outputting an imaging result;
A distance image generating means for generating a distance image having information on the distance of the imaging region in a three-dimensional space based on the imaging result output from the imaging means;
Based on the distance image, a volume determination means that estimates the volume of the occupant region and determines whether the volume is equal to or greater than a predetermined volume threshold;
Knee determining means for determining whether or not there is a passenger's knee in the passenger compartment below a predetermined position in the vehicle vertical direction based on the distance image;
When the volume determination means determines that the volume of the occupant area is equal to or greater than the predetermined volume threshold value, the knee determination means determines that the knee is present below the predetermined position. In this case, it is determined that the occupant is an adult, and the occupant is a child when the knee determination unit determines that the knee does not exist below a predetermined position in the vehicle vertical direction. An occupant detection device comprising: an occupant determination means for determining The knee determination means includes
For each of a plurality of pixels constituting the distance image, a fixed-length inverse vector is calculated in the reverse direction of the normal vector in the three-dimensional space, and the position coordinates in the three-dimensional space designated by the inverse vector are internally Internal coordinate setting means as coordinates,
The unit space that calculates a score value related to the total number of the internal coordinates included in the unit space for each of the plurality of unit spaces constituting the three-dimensional space, and includes the internal coordinates corresponding to the plurality of pixels. Score image generation means for generating a score image indicating the score value of each of the plurality of pixels in correspondence with each other,
The occupant detection device according to claim 1, further comprising knee extraction means for extracting the knee based on the score value. A head region extracting means for extracting a head region of the occupant based on the distance image;
The occupant detection device according to claim 1, wherein the volume determination unit estimates a volume of the occupant region including the head region.