JP2008002827A - Occupant detector - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method for controlling light intensity of each light-emitting element in a photographic device by illuminating with a plurality of light-emitting elements, and to provide a monitoring system of this method and an occupant detection system. <P>SOLUTION: An image is photographed by illuminating each of parts of a photographing region, assigned to the plurality of light-emitting elements 14 (S4); the amount reflecting the brightness of each part of the image, corresponding to the light-emitting elements is obtained (S10); to each image part, corresponding to the comparison of the amount and a prescribed threshold value, the driving amount of the light-emitting element, corresponding to the image part, is changed so as to widen the dynamic range by the prescribed variation (S16, S20). If the amount is the number of pixels, the brightness of which in the image part concerned is higher than the saturation threshold, and the amount is larger than the threshold to the number of pixels, the driving amount is reduced by the variation; and if the amount is the number of pixels, the brightness of which in the image part concerned is lower than the lowest threshold, the amount is larger than threshold to the number of pixels, the driving amount is increased by the variation. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an occupant detection device that detects the state of a passenger based on an image taken inside a vehicle under illumination by a light emitting element, and more particularly, such an occupant detection device can obtain a good image. In particular, the present invention relates to a technique for controlling the light intensity of a light emitting element.

  In a system that detects an occupant based on a photographed image, it is important to obtain an image with a stable brightness while minimizing the influence of ambient brightness. Therefore, in dark conditions at night, the aperture is widened and auxiliary light is projected to ensure the brightness necessary for shooting. Under bright daytime conditions, the auxiliary light is not used and the aperture is narrowed. The method can be considered. In such a method, when the light of the headlight of the oncoming vehicle is inserted into the passenger compartment at night, auxiliary light is necessary immediately before the light is inserted, but the aperture is reduced at the moment when the light of the headlight is inserted. There is a need. Similarly, when morning and evening oblique light enters the passenger compartment, it is necessary to instantaneously reduce the aperture. However, it is difficult to obtain an image with stable brightness by changing the aperture in response to such a sudden change in brightness or turning on / off the auxiliary light.

In view of this point, Patent Literature 1 proposes an occupant detection system that can always detect an occupant state based on an image with stable brightness without being affected by changes in ambient brightness. . In this system, a near-infrared light as auxiliary light is projected onto a predetermined area including a passenger seat by an auxiliary light projector (corresponding to an LED or a light emitting element described later), and a near-infrared light is projected by a photographing device. An image obtained by cutting visible light having a shorter wavelength than the light is taken. The passenger state in the vehicle seat is detected by performing image processing on the image.
JP 2004-144512 A (summary, FIG. 6)

  Even in the system of Patent Document 1, it is possible to obtain an image with relatively stable brightness by suppressing the influence of visible light contained in light such as sunlight, headlights, and street lights. However, it is difficult to say that it is sufficient to shoot by cutting visible light on the shorter wavelength side than near infrared light while projecting constant near infrared light. Saturation may occur, and conversely, the shaded area may be dark and information may not be obtained. Also, when an object approaches the lens of the camera, it also approaches a light emitting element such as an LED, and the light is strongly reflected from the approaching object, and that part saturates. However, there is a limit to the gain adjustment, and it may not be able to cope with it.

  Therefore, the present invention suppresses the influence of local changes in the illuminance of the imaging region when detecting the seating state of the occupant based on the image captured by illuminating the predetermined imaging region with a plurality of light emitting elements. It is an object of the present invention to provide an occupant detection device capable of controlling the light intensity of the plurality of light emitting elements.

The present invention has been made in view of the above problems, and an imaging device (for example, included in a camera 12 described later) disposed toward a passenger in a vehicle interior, and an illumination region in the imaging region of the imaging device. A plurality of light emitting elements that emit near-infrared light so that they are different from each other (for example, 14 described later), and control means that controls the light intensity of the light emitting elements based on the image of the illumination region (for example, described later) There is provided an occupant detection device including an occupant detection ECU 20).
According to the occupant detection device configured as described above, it is possible to control the light intensities of the plurality of light emitting elements that emit near-infrared light so that the illumination regions are different from each other in the imaging region.

The control means may be configured to reduce the light intensity of the light emitting element corresponding to the one illumination region when one luminance distribution of the plurality of illumination regions exceeds a predetermined threshold value.
According to the occupant detection device configured in this way, it is possible to prevent information loss due to being too bright.

The control unit may be configured to stop the light emission of the light emitting element corresponding to the illumination region when one luminance distribution of the plurality of illumination regions exceeds a threshold value greater than the predetermined threshold value.
According to the occupant detection device configured in this way, it is possible to prevent information loss due to being too bright.

The control means may be configured to increase the light intensity of the light emitting element corresponding to the one illumination region when one luminance distribution of the plurality of illumination regions is below a predetermined threshold.
According to the occupant detection device configured in this way, it is possible to prevent information loss due to being too dark.

The predetermined threshold may be set for each illumination region.
According to the occupant detection device configured as described above, the light intensity of the light emitting element can be optimally controlled for each illumination region.

The light emitting elements are arranged such that the respective light emitting areas overlap each other by a predetermined amount, and when the light intensity of the light emitting elements corresponding to one of the plurality of light emitting areas is changed, the control means You may comprise so that the light intensity of each light emitting element corresponding to the illumination region to overlap may be adjusted according to each overlap amount.
According to the occupant detection device configured as described above, the illuminance of each illumination region can be adjusted according to the degree of overlap with the surrounding illumination regions.

  According to the present invention, in an occupant detection device that detects a seating state of an occupant based on an image captured by illuminating a predetermined imaging area with a plurality of light emitting elements, the influence of local changes in illuminance in the imaging area is suppressed. Thus, the light intensity of the plurality of light emitting elements can be controlled.

Hereinafter, the present invention will be described in detail with reference to embodiments of the present invention and the accompanying drawings.
In addition, when showing the same element in several drawing, the same referential mark is attached | subjected.
FIG. 1 is a block diagram conceptually showing the configuration of an occupant detection system that shoots while illuminating a shooting area with a plurality of light emitting elements according to an embodiment of the present invention. In FIG. 1, an occupant detection system 1 includes a camera unit 10 that captures an imaging region and an occupant detection ECU 20 that detects an occupant from a captured image from the camera unit 10. FIG. 2 is a diagram illustrating the configuration of the camera unit 10 of FIG. 1 and the shooting area that is the shooting target of the camera unit 10. 2, (a) is a front view of the camera unit 10, (b) is a side view as seen from the direction of arrow A in (a), and (c) is a plan view as seen from the direction of arrow B in (a). It is. (D) will be described later. As shown in FIGS. 2A to 2C, the camera unit 10 has a trapezoidal cross section in the horizontal direction, and includes a pair of cameras 12-1 and 12-2 on the front surface. A total of six light emitting elements 14-1 to 14-6 are provided, one pair on the side. In the present embodiment, the light emitting element is described as an LED. FIG. 2D is a diagram illustrating a state in which the light L1 to L6 from the six LEDs 14-1 to 14-6 are irradiated to the imaging regions of the cameras 12-1 and 12-2. In FIG. 2D, the portions (indicated by circles) irradiated with the lights L1 to L6 are also represented by the same symbols L1 to L6 as the lights of (b) and (c). For convenience, the shooting area is a plane parallel to the front of the camera unit 10 (the face provided with the cameras 12-1 and 12-2), and the shooting area is divided into six equal parts according to the number of LEDs 14-1 to 14-6. Each of them was used as a measurement area. The measurement areas illuminated on the LEDs 14-1 to 14-6 are denoted as R1 to R6, respectively.

The LEDs 14-1 to 14-6 mainly emit light in a wavelength region of 700 nm to 1000 nm, for example, and the cameras 12-1 and 12-2 can range from visible light of about 700 nm to near infrared light of about 1000 nm. It is preferable to have sensitivity only in the wavelength region. For this reason, it is preferable that the cameras 12-1 and 12-2 include an image sensor such as a CMOS or a CCD having sufficient sensitivity in a wavelength region of 1000 nm from visible light and a filter that cuts a wavelength of less than 700 nm. The camera unit 10 configured in this manner is arranged at a position where the seat portion in the vehicle interior is a photographing area, and the visual field of the cameras 12-1 and 12-2 is obstructed, for example, in the vicinity of the map lamp of the vehicle. It is preferable to arrange at a position where there is little possibility that an object will be placed.
The occupant detection ECU 20 is one of various ECUs (electronic control units) used in a vehicle on which the occupant detection system 1 of the present invention is mounted. Like other ECUs, a CPU or MPU, RAM, non-volatile (not shown) In addition to the memory 30 and the like, an interface for communicating with the cameras 12-1 and 12-2 and the LEDs 14-1 to 14-6 and other ECUs are used to form an in-vehicle network (CAN = Controller Area Network). It is a kind of computer equipped with an interface. In other words, the occupant detection ECU 20 includes interfaces to the cameras 12-1 and 12-2, the LEDs 14-1 to 14-6 and the CAN, and can store and execute an LED dimming unit (program) 32 described later. As long as it is a computer, it may be an apparatus having an arbitrary specification.

  As shown in FIG. 1, the occupant detection ECU 20 includes a nonvolatile memory 30 that stores various programs necessary for processing (referred to as “xx section” for system components) and data. The nonvolatile memory 30 includes an LED dimming unit 32 that controls the light intensity of the LEDs 14-1 to 14-6, an imaging unit 40 that performs imaging using the cameras 12-1 and 12-2, and a captured image. A feature extraction unit 42 that extracts the included features, a position calculation unit 44 that determines the presence area of the target using the results of the feature extraction unit 42, and various determinations regarding each occupant based on the processing results of the position calculation unit 44 An occupant determination unit 46 is stored as a program. The occupant determination unit 46 includes an occupant presence / absence determination unit 50 that determines the presence or absence of an occupant, an occupant head position determination unit 52 that determines the position of the occupant's head, and an occupant body type determination unit 54 that determines the occupant's body shape. Hereinafter, the LED dimming unit 32 will be described in detail.

FIG. 3 is a flowchart showing a dimming process flow of the LED dimming unit 32 executed by a CPU (not shown) of the occupant detection ECU 20. When the LED dimming unit 32 is called, first, in step S2, the loop index j is set to 0, and in step S4, a captured image obtained by capturing the imaging region by operating the cameras 12-1 and 12-2 is captured. In step S6, the corresponding pixels of the two images by the camera 12-1 and the camera 12-2 are averaged, an image composed of the pixels of the average value is synthesized, and the subsequent processing is performed on this image.
Next, in step S8, the measurement region index i (i = 1, 2,..., 6) is set to an initial value 1. In step S10, a luminance histogram is created for the image in the measurement region Ri. FIG. 4 is an example of a luminance histogram for the measurement region Ri. In the histogram of FIG. 4, the number of luminance levels is 256 (level 0 to 255), and a threshold B1 on the low luminance side and a threshold B2 on the high luminance side are set in order to determine the brightness situation of the measurement region Ri. Yes. The number of pixels whose luminance is equal to or less than the threshold value B1 is equal to the area of the shaded portion on the left side and is represented by n1i. The number of pixels whose luminance is greater than or equal to the threshold B2 is equal to the area of the hatched portion on the right side, and this is represented by n2i. For detecting a portion that is too bright and causing saturation, the threshold B2 on the high luminance side is set to, for example, about level 254, and for detecting a portion that is too dark and information is lost, the threshold B1 on the low luminance side is set. For example, it is preferable to set the luminance level to about 10. However, in FIG. 4, for convenience of drawing, the two threshold values B1 and B2 are shown at a luminance level close to the middle.

  Returning now to FIG. In determination step S12, it is determined whether or not the number of pixels n2i having a luminance equal to or higher than the threshold B2 on the high luminance side is greater than a predetermined threshold N2i. Here, the threshold value N2i is a value that is judged to be too bright to obtain a good image when the number of pixels n2i having a luminance equal to or higher than the threshold value B2 exceeds the value, and is preferably obtained by experiments. Accordingly, when the number of pixels n2i having a luminance equal to or higher than the threshold B2 is larger than the threshold N2i in the determination step S12 (Yes), the drive amount Vi of the LED 14-i is decreased by a predetermined change amount Δ in the drive amount reduction step S16. However, when such control is repeated, even if the driving amount of the LED 14-i is reduced to the lower limit (here, 0), the number of pixels n2i having a luminance equal to or higher than the threshold B2 may still exceed the threshold N2i. In such a case, it is determined whether or not Vi−Δ becomes negative in step S14 prior to step S16 so that the drive amount of the LED 14-i is not set to 0 or less in step S16. If it is not negative (ie, No), the process proceeds to step S16. If it is negative (Yes), it is meaningless to continue the current control loop any more. The drive amount of the LED 14-i is set to 0, which is the lower limit value, and the process proceeds to judgment step S26.

  On the other hand, when the number of pixels n2i having a luminance equal to or higher than the threshold B2 on the high luminance side is less than the predetermined threshold N2i in the determination step S12 (in the case of No), the threshold B1 is equal to or lower than the threshold B1 on the low luminance side in the determination step S18. It is determined whether the number of luminance pixels n1i is larger than a predetermined threshold value N1i. Here, the threshold value N1i is a value that is determined to be too bright to obtain a good image when the number of pixels n1i having a luminance equal to or less than the threshold value B1 exceeds the value, and is preferably obtained by experiment. Therefore, if the number of pixels n1i having a luminance equal to or lower than the threshold B1 on the low luminance side is not larger than the predetermined threshold N1i (No), neither the image is too bright nor too dark, and the process proceeds to the determination step S26 without doing anything. If the number n1i of pixels having a luminance equal to or lower than the threshold B1 on the low luminance side is larger than the predetermined threshold N1i (Yes), step S19 is executed for the same reason as in the determination step S14. That is, when the drive amount Vi of the LED 14-i corresponding to the measurement region Ri is increased by Δ, it is determined whether or not the upper limit value Vmax of the drive amount Vi is exceeded (that is, whether Vi + Δ> Vmax is satisfied). . If it exceeds (in the case of Yes), the drive amount Vi is set to the upper limit value Vmax in step S21, and the process proceeds to determination step S26. On the other hand, if the drive amount Vi does not exceed the upper limit value Vmax even if the drive amount Vi is increased by Δ in the determination step S19 (in the case of No), the drive amount Vi is changed to a predetermined change amount in the drive amount increase step S20. Increase by Δ and proceed to step S22.

When the drive amount decrease step S16 or the drive amount increase step S20 is executed, the luminance of the adjacent region is adjusted in the subsequent step S22. For this reason, the designer of the occupant detection system 1 needs to create data as shown in FIG. 5 in advance and store it in the nonvolatile memory 30. In the example of FIG. 5, a set of block numbers of two adjacent blocks (measurement areas) is used as the ID (identifier) of a record constituting the table, and the interdependency between the blocks of the set is determined for each ID. Is set. The interdependency is expressed as a percentage of Δ that should change the LED driving amount corresponding to the other block when the driving amount of the LED corresponding to one block of the set is changed by Δ, for example. Is. For example, in FIG. 2D, when the LED irradiation range is viewed, the irradiation range pairs (L1, L4), (L2, L3), (L3, L6) and (L4, L5) do not overlap at all. Therefore, also in the table of FIG. 5, the mutual block degree of the adjacent block numbers (1, 4), (2, 3), (3, 6) and (4, 5) is set to 0%. In FIG. 2 (d), there is a slight overlap in the LED irradiation range groups (L1, L2) and (L5, L6), so in FIG. 5, the adjacent block numbers (1, 2) and (5, 6 ) Is set to 5% interdependency.
Here, an example is given and the process in step S22 is demonstrated. For example, when i = 3, that is, when the drive amount of the LED 14-3 corresponding to the measurement region R3 is decreased by Δ in step S16, the other block of the group including the block number 3 in the table of FIG. Thus, only (Δ × interdependency) may be adjusted. First, there are five blocks 1, 2, 4 to 6 that form a pair with (adjacent to) block number 3. Of these, the blocks 2 and 6 have 0% interdependency and need not be adjusted. Since the interdependencies of the blocks 1 and 5 are 10%, the driving amount of the corresponding LEDs 14-1 and 14-5 needs to be reduced by Δ × 10%. Since the remaining block 4 has an interdependency of 15%, the driving amount of the corresponding LED 14-4 needs to be reduced by Δ × 15%. For easy understanding, FIG. 5 includes a record with 0% interdependency. However, a record with 0% interdependency may be omitted.

In this way, after the brightness adjustment of the adjacent area in step S22 is completed, the loop index j is incremented in step S24 to indicate that the drive amount has been changed for the measurement area Ri, and the process proceeds to determination step S26. .
As described above, when the drive amount Vi is changed for an arbitrary measurement region Ri, the loop index j is incremented regardless of the change direction (that is, whether the drive amount is increased or decreased). This represents the number of measurement regions in which the drive amount Vi has been changed in one control loop (however, changes in step S15 and later-described step S21 are excluded). Therefore, if j = 0, it is understood that the drive amount was not changed (change by S16 or S20) for any measurement region Ri in the control loop. Therefore, when j becomes 0 from a value other than 0 in the process of control, all the measurement areas R1, R2,..., RM (M is the number of measurement areas, and is 6 in the present example. ) Has converged to a predetermined threshold range. This can be used to determine the end of the control loop as will be described later.

As described above, after the completion of step S15, S21 or S24, or after the No branch of the determination step S18, by executing the determination step S26, the measurement region index i becomes the number M of measurement regions (in the present example, It is determined whether or not M = 6) has been reached. If i = M is not satisfied (in the case of No), the measurement area index i is incremented in step S28 and the process returns to step S10 in order to perform processing for the next measurement area. If i = M in the determination step S26 (in the case of Yes), it means that M, that is, six measurement regions have been exhausted, and therefore in a further determination step S30, it is determined whether or not the loop index j is 0. . If the index j is not 0 (in the case of No), as can be seen from the above description, there is a drive amount Vi that has not converged within a predetermined range. Therefore, in order to continue the control, step S2 is performed. Return. If it is determined in step S30 that j is 0 (Yes), all drive amounts V1, V2,..., VM (V6 in the present example) have converged to their predetermined ranges. Therefore, the LED dimming process ends.
As described above, according to the occupant detection system of the present invention, the driving amount of each light emitting element is measured by measuring the luminance for each measurement region that substantially matches the illumination part of the light emitting element without using an illuminance sensor or the like. By adjusting, a good image can be obtained over the entire imaging region.
As described above, the seating state of the occupant and the like can be detected by the feature extraction unit 42, the position calculation unit 44, and the occupant determination unit 46 based on the image captured by controlling the light intensity of each light emitting element.
In the above description, the occupant detection system has been described. However, according to the present invention, the light intensity of each light emitting element can be obtained so that a good or appropriate photographed image can be obtained in an apparatus that illuminates and photographs with a plurality of light emitting elements. A method of controlling can be obtained.

The above is merely an example of an embodiment for explaining the present invention. Accordingly, it is easy for those skilled in the art to make various changes, modifications, or additions to the above-described embodiments in accordance with the technical idea or principle of the present invention.
For example, a single threshold value may be used without changing the threshold value N1i for the number of pixels having a luminance equal to or lower than the threshold value B1 on the low luminance side for each measurement region, and the threshold value for the number of pixels having a luminance value equal to or higher than the threshold value B2 on the high luminance side. Obviously, a single threshold may be used without changing N2i for each measurement region.

  In the above-described embodiment, the low-brightness side threshold B1 and the high-brightness side threshold B2 are used. However, both thresholds may be equally set to B1 = B2. In this case, for example, the threshold value N2i with respect to the number of pixels having a luminance equal to or higher than the threshold value B2 on the high luminance side may be Np / 2 (Np is the total number of pixels in the measurement region Ri).

In the above-described embodiment, the control is given priority to the former among the control when it is too bright and the control when it is too dark. That is, in the determination step S12, when the number of pixels n2i having a luminance equal to or higher than the threshold B2 on the high luminance side is larger than the predetermined threshold N2i (in the case of Yes), a process of unconditionally reducing the drive amount is performed (S16). However, depending on the settings of the threshold values B1 and B2 and the threshold values N1i and N2i of the number of pixels, the condition of S12 that the number of pixels n2i having the luminance higher than the threshold B2 on the high luminance side is larger than the predetermined threshold N2i, There may be a case where both of the conditions of S18 that the number n1i of pixels having a luminance equal to or less than the threshold value B1 is greater than the predetermined threshold value N1i are satisfied. For example, in a situation where there are almost half dark parts and half bright parts in one measurement region Ri, the probability of occurrence increases as the threshold values N1i and N2i of the number of pixels are smaller than Np / 2. In other words, in the above-described embodiment, control in the case of being too bright is also given priority in such a case.
However, in the case of Yes in the determination step S12, instead of executing S16 unconditionally, it is determined whether or not the number n1i of pixels having a luminance equal to or lower than the threshold B1 on the low luminance side is larger than the predetermined threshold N1i. In some cases, it may be configured to perform another process.
Moreover, it is also possible to replace the step groups S12 to S16 and the step groups S18 to S21 so that priority is given to the control in the case of being too dark.
In the example shown in FIG. 3, control is performed both when the image is too bright and when the image is too dark. However, only one of the controls may be performed.
Note that any of current, voltage, and power may be used as the driving amount.

It is a block diagram which shows notionally the structure of the passenger | crew detection system which image | photographs, illuminating an imaging | photography area | region with several light emitting element by one Embodiment of this invention. FIG. 2 is a diagram illustrating the configuration of the camera unit 10 of FIG. 1 and the shooting area that is the shooting target of the camera unit 10. It is a flowchart which shows the flow of a process of the LED light control part 32 which is a program which the passenger | crew detection ECU20 of FIG. 1 stores and executes. It is an example of the histogram of the brightness | luminance with respect to measurement area | region Ri. It is a figure of the interdependence table | surface used in the brightness | luminance adjustment (step S22) of an adjacent area | region.

Explanation of symbols

1 occupant detection system 10 camera unit 12 camera 14 light emitting element 20 occupant detection ECU
30 Nonvolatile memory

Claims (6)

An image sensor arranged toward the passenger in the passenger compartment;
A plurality of light emitting elements that emit near-infrared light so that the illumination areas are different from each other in the imaging area of the imaging element;
An occupant detection device comprising: control means for controlling light intensity of the light emitting element based on an image of the illumination area.   2. The occupant detection according to claim 1, wherein the control means reduces the light intensity of the light emitting element corresponding to the one illumination region when one luminance distribution of the plurality of illumination regions exceeds a predetermined threshold value. apparatus.   3. The occupant according to claim 2, wherein the control unit stops light emission of the light emitting element corresponding to the illumination region when one luminance distribution of the plurality of illumination regions exceeds a threshold value greater than the predetermined threshold value. Detection device.   4. The control device according to claim 1, wherein the control means increases the light intensity of the light emitting element corresponding to the one illumination region when one luminance distribution of the plurality of illumination regions falls below a predetermined threshold value. 5. The occupant detection device according to one item.   The occupant detection device according to any one of claims 1 to 4, wherein the control unit sets the predetermined threshold value for each illumination region. Each light emitting element is arranged so that the respective illumination areas overlap by a predetermined amount,
When the light intensity of the light emitting element corresponding to one of the plurality of illumination areas is changed, the control means changes the light intensity of each light emitting element corresponding to the illumination area overlapping with the one illumination area. The occupant detection device according to any one of claims 1 to 5, wherein the occupant detection device is adjusted according to a lap amount.

Images