JP6149826B2 - Imaging apparatus and scene determination method - Google Patents

Description

  The present invention relates to an imaging apparatus and a scene determination method.

  In an imaging apparatus, control is generally performed so that a state (for example, luminance) of a video signal generated by capturing an image of a subject does not change as much as possible even when an external situation (for example, brightness) changes.

JP 2013-150153 A

  One of the imaging devices is a so-called in-vehicle camera that is installed in a vehicle and images the outside. When a vehicle is traveling on a highway in the daytime, for example, it becomes dark rapidly when the vehicle enters the tunnel, and brightens rapidly when the vehicle leaves the tunnel. When the in-vehicle camera images a subject, the brightness may change abruptly.

  Here, the in-vehicle camera is taken as an example, but the same applies to the case where the outside is imaged from inside the vehicle with a general video camera. Note that the tunnel is an example of a specific state where light is temporarily blocked in a situation where the imaging device is photographing a subject in the daytime.

  In order to control the imaging device so as not to change the state of the video signal to be output as much as possible, control with higher responsiveness than before is necessary, but it is not preferable to deteriorate the stability of the control. Therefore, it is required to accurately determine that the state is a specific state such as a tunnel without deteriorating the stability of the control, and to perform control so as not to change the state of the output video signal.

  In order to meet such a demand, the present invention accurately determines whether or not a specific state in which light is temporarily blocked in a situation where a subject is photographed in the daytime, and outputs an image signal. An object of the present invention is to provide an imaging apparatus capable of stably controlling the state of the above. It is another object of the present invention to provide a scene determination method capable of accurately determining a specific state where light is temporarily blocked at night, daytime, and daytime.

  In order to solve the above-described problems of the related art, the present invention provides a first detection unit that captures an image of a subject, a sensor that detects brightness, and a period in which the sensor detects predetermined brightness. A first detector for detecting whether or not to continue for a period of time; and detecting whether or not a period of time detected by the sensor as having a predetermined brightness is continued for a second period shorter than the first period of time. The imaging unit is daytime and the light is temporarily based on the second detection unit, the first detection result by the first detection unit, and the second detection result by the second detection unit. And a determination unit that determines that the subject is being imaged in a specific state blocked by the imaging apparatus.

  In addition, in order to solve the above-described problems of the related art, the present invention detects whether or not the period detected by the sensor for detecting brightness as having a predetermined brightness continues for the first time, It is detected whether or not the period detected by the sensor as having a predetermined brightness continues for a second time shorter than the first time, and the period detected as having the predetermined brightness is as described above. If it does not continue for the first time, it is determined that it is nighttime, and the period in which the predetermined brightness is detected continues for the first time, and the period in which the predetermined brightness is detected is detected. If it continues for the second time, it is determined that it is daytime, and the period in which the predetermined brightness is detected continues for the first time, and the period in which the predetermined brightness is detected is detected. If it does not continue for the second time, it is daytime and light is temporarily blocked Provides a scene determination method characterized by determining that the a particular state.

  According to the imaging apparatus of the present invention, in a situation where a subject is being photographed in the daytime, it is accurately determined whether or not it is in a specific state where light is temporarily blocked, and the state of the video signal to be output is determined. It can be controlled stably. According to the scene determination method of the present invention, it is possible to accurately determine a specific state where light is temporarily blocked at night, daytime, and daytime.

It is a block diagram which shows the imaging device of one Embodiment. It is a block diagram which shows the automatic exposure control model in the imaging device of one Embodiment. It is a circuit diagram which shows the specific structure of the infrared sensor 6 in FIG. It is a block diagram which shows the functional internal structure of the scene determination part 103 in FIG. FIG. 5 is a waveform diagram for explaining the operation of the scene determination unit 103 shown in FIG. 4. FIG. 5 is a diagram for explaining scene determination by the scene determination unit 103 shown in FIG. 4, which is a scene determination criterion according to the scene determination method of the embodiment. It is a figure for demonstrating the operation | movement of multiple exposure.

  Hereinafter, an imaging apparatus and a scene determination method according to an embodiment will be described with reference to the accompanying drawings. The imaging device of this embodiment is assumed to be an in-vehicle camera.

  In FIG. 1, input light Lin is input to the imaging unit 2 from the outside via a lens 1. Although one lens 1 is shown here, the imaging apparatus actually has a plurality of lenses. The imaging unit 2 includes an imaging device 21 and an exposure setting circuit 22 for setting how to perform exposure in the imaging device 21.

  The image sensor 21 is, for example, a CMOS (Complementary Metal Oxide Semiconductor) sensor. The image sensor 21 may be a CCD (Charge Coupled Device) sensor.

  The image sensor 21 photoelectrically converts the input light Lin and outputs, for example, RGB signals. The RGB signals are input to the video signal processing unit 3. The video signal processing unit 3 generates a luminance signal (Y signal) and a color signal (C signal) based on the RGB signals, and performs various signal processing. The video signal processing unit 3 supplies the video signal to the display unit 4.

  The video signal processing unit 3 may supply the Y signal and the C signal to the display unit 4, or return the Y signal and the C signal that have undergone the signal processing to the RGB signal, and the RGB signal to the display unit 4. You may supply. The video signal processing unit 3 may supply a composite video signal to the display unit 4 or supply a digital video signal conforming to the HDMI (registered trademark) standard (HDMI: High-Definition Multimedia Interface) to the display unit 4. May be.

  The video signal processing unit 3 includes a multiple exposure combining unit 31. The operation of the multiple exposure combining unit 31 will be described in detail later.

  The specific configuration or operation of the video signal processing unit 3 is not particularly limited. Instead of the display unit 4, a recording unit for recording the video signal output from the video signal processing unit 3 may be provided.

  The Y signal is input to the exposure evaluation value generation unit 5. The exposure evaluation value generation unit 5 generates an average value of the luminance of each frame as an exposure evaluation value based on the input Y signal, and supplies it to the control unit 10.

  The exposure evaluation value generation unit 5 may simply average the luminance within one frame to obtain an exposure evaluation value. The exposure evaluation value generation unit 5 may divide one frame into a plurality of regions, obtain an average value in each region, weight the average value of the central region, and then average it to obtain an exposure evaluation value.

  The exposure evaluation value generated by the exposure evaluation value generation unit 5 is input to the control unit 10. The control unit 10 can be configured by a CPU. The control unit 10 includes an exposure control unit 101, a light source determination unit 102, and a scene determination unit 103 as functional internal configurations.

  The exposure control unit 101 controls the exposure setting circuit 22 so that the exposure evaluation value is constant. The exposure control unit 101 sets a shutter speed as an exposure setting in the image sensor 21.

  The exposure control unit 101 supplies a control value for setting a shutter speed such as 1/60 seconds or 1/250 seconds to the exposure setting circuit 22. The exposure setting circuit 22 operates the image sensor 21 so as to capture an image at the shutter speed indicated by the input control value.

  FIG. 2 shows an automatic exposure control model in the imaging apparatus described above. The control unit 10 holds a target luminance value as a control target value. The control target value may be a predetermined gray level.

  A virtual subtracter 104 in the control unit 10 subtracts the exposure evaluation value output from the exposure evaluation value generation unit 5 from the control target value. A difference value between the control target value and the exposure evaluation value is input to the exposure control unit 101. The exposure control unit 101 controls the shutter speed set by the exposure setting circuit 22.

  A virtual adder 23 in the imaging unit 2 applies an imaging process with the shutter speed set by the exposure setting circuit 22 to the input light Lin. The output of the adder 23 is input to the exposure evaluation value generation unit 5 and supplied to the display unit 4.

  The automatic exposure control model shown in FIG. 2 constitutes a feedback control loop. Therefore, the exposure evaluation value always matches the control target value. Even if the input light Lin becomes a disturbance and the luminance value fluctuates, the exposure evaluation value is controlled to be constant.

  As described above, the imaging apparatus illustrated in FIG. 1 performs control so that the state (in this case, luminance) of the video signal displayed on the display unit 4 does not change as much as possible even if the brightness changes as an external situation.

  Returning to FIG. 1, the infrared sensor (IR sensor) 6 detects infrared rays. The detected value detected by the IR sensor 6 is converted into a digital value by the A / D converter 7. The digital value output from the A / D converter 7 is referred to as IRout. The digital value IRout here is not binary but multivalued. The digital value IRout is input to the control unit 10 and the band pass filter 8.

  As shown in FIG. 3, the IR sensor 6 can be composed of a phototransistor 6pt. The power supply voltage Vcc is supplied to the collector terminal of the phototransistor 6pt, and the emitter terminal is grounded via the resistor R6. The A / D converter 7 is also supplied with the power supply voltage Vcc at one end and grounded at the other end.

  The band pass filter 8 extracts a frequency component of 110 Hz included in the input digital value IRout. The detection unit 9 detects the presence or absence of a frequency component of 110 Hz. A detection value indicating a detection result by the detection unit 9 is input to the control unit 10. For example, the detection value may be 1 when the detection unit 9 detects that a frequency component of 110 Hz is present, and may be 0 when it is detected that there is no frequency component.

  When the detection unit 9 detects that a frequency component of 110 Hz is present, the light detected by the IR sensor 6 includes flicker. When the detection unit 9 does not detect that there is a frequency component of 110 Hz, the light detected by the IR sensor 6 does not include flicker.

  The light source determination unit 102 receives the digital value IRout from the A / D converter 7 and the detection value from the detection unit 9. The light source determination unit 102 determines that the input digital value IRout is equal to or greater than a predetermined value and that there is infrared light, and determines that the light source is a light bulb when the detection value determines that flicker is included.

  The light source determination unit 102 determines that the input digital value IRout is less than a predetermined value and that there is no infrared ray, and determines that the flicker is included according to the detection value, it determines that the light source is a fluorescent lamp.

  The light source determination unit 102 determines that the input digital value IRout is equal to or greater than a predetermined value and that there is an infrared ray, and determines that the light source is the sun when it is determined that the flicker is not included according to the detection value.

  A determination value indicating the determination result of the light source by the light source determination unit 102 is input to the video signal processing unit 3. Since the color temperature of light differs depending on the type of light source, it is preferable to adjust the white balance of the video signal generated by the video signal processing unit 3 depending on the type of light source.

  The video signal processing unit 3 adjusts the white balance of the video signal according to the input determination value. The imaging apparatus shown in FIG. 1 performs control so that the state of the video signal displayed on the display unit 4 (here, white balance) does not change as much as possible even when the light source changes as an external situation.

  Next, a specific configuration and operation of the scene determination unit 103 will be described with reference to FIGS. As shown in FIG. 4, the scene determination unit 103 includes a long time constant integration unit 1031, a short time constant integration unit 1032, comparison units 1033 and 1034, and a determination unit 1035 as functional internal configurations.

  In FIG. 4, the digital value IRout output from the A / D converter 7 is input to the long time constant integration unit 1031 and the short time constant integration unit 1032.

  FIG. 5A shows an example of the waveform of the digital value IRout. It is assumed that the power supply of the imaging device is turned on at time t0 and the scene determination unit 103 starts scene determination at time t1. It is assumed that the vehicle has passed through the tunnel at time t2 to t3. N1 and N2 in the digital value IRout are noise components generated in the valleys of the building.

  The long time constant integrating unit 1031 generates the first integrated value by integrating the input digital value IRout with the first time constant. The first time constant is, for example, 30 minutes. The short time constant integrating unit 1032 generates the second integrated value by integrating the input digital value IRout with the second time constant. The second time constant is, for example, 1 second.

  The comparison unit 1033 compares the first integrated value with a predetermined threshold value TH1, and if the first integrated value is smaller than the threshold value TH1, the first value is low, and if the first integrated value is greater than or equal to the threshold value TH1, the first value is high. The detection value D1 is output. FIG. 5B shows an example of the first detection value D1 binarized by the comparison unit 1033.

  The comparison unit 1034 compares the second integrated value with a predetermined threshold value TH2, and if the second integrated value is smaller than the threshold value TH2, it is low, and if the second integrated value is greater than or equal to the threshold value TH1, the second value that is high. The detection value D2 is output. FIG. 5C shows an example of the second detection value D2 binarized by the comparison unit 1034. In FIG. 5C, the period during which the second detection value D2 is low may be temporally shifted from the time t2 to t3, but here, the temporal shift is ignored.

  The determination unit 1035 determines a scene based on the first detection value D1 and the second detection value D2. As shown in FIG. 6, if the first detection value D1 is low (L), the determination unit 1035 causes the imaging device (imaging unit 2) to operate at night regardless of the value of the second detection value D2. It is determined that the subject is being photographed.

  If both the first and second detection values D1 and D2 are high (H), the determination unit 1035 determines that the imaging device is photographing the subject during the daytime.

  If the first detection value D1 is H and the second detection value D2 is L, the determination unit 1035 is imaging the subject in the daytime, but the vehicle travels inside the tunnel. It is determined that The situation in which the first detection value D1 is H and the second detection value D2 is L is that the imaging device is imaging a subject in a specific state during the daytime when light is temporarily blocked. Is the situation.

  In the present embodiment, the scene determination unit 103 determines a scene using a digital value IRout based on a detection value detected by the IR sensor 6 provided to detect the type of light source. The scene determination unit 103 may determine a scene using a digital value based on the detected brightness value detected by the illuminance sensor instead of the IR sensor 6.

  The scene determination unit 103 may determine a scene using a digital value based on a brightness detection value detected by an image sensor having a wide dynamic range instead of the IR sensor 6.

  The scene determination unit 103 may determine a scene using a digital value based on a brightness detection value detected by an external brightness detection sensor such as the IR sensor 6, an illuminance sensor, or an image sensor. Note that it is preferable to use the IR sensor 6 because the use of the IR sensor 6 makes it possible to both detect the type of light source and discriminate the scene.

  In FIG. 4, the long time constant integration unit 1031 and the comparison unit 1033 detect whether or not the period detected by the sensor as having a predetermined brightness continues for 30 minutes, which is the first time. The short time constant integration unit 1032 and the comparison unit 1034 detect whether or not the period detected by the sensor as having a predetermined brightness continues for the second time of 1 second.

  Instead of the long time constant integration unit 1031 and the comparison unit 1033, and the short time constant integration unit 1032 and the comparison unit 1034, a counter that counts time may be used.

  The scene determination unit 103 detects that there is a predetermined brightness by a first detection unit that detects whether or not the period of time detected by the sensor as having a predetermined brightness continues for the first time. And a second detection unit that detects whether or not the period continues for a second time shorter than the first time.

  When the scene determination unit 103 determines whether the shooting state is night, daytime, or inside the tunnel during the daytime, it is preferable to control the exposure as follows.

  The exposure control unit 101 supplies a relatively fast shutter speed control value to the exposure setting circuit 22 if the determination result is daytime. The exposure control unit 101 supplies a control value of a relatively slow shutter speed to the exposure setting circuit 22 if the determination result is nighttime. In this way, the luminance can be increased even at night, and the luminance of the video signal generated by imaging the subject can be controlled so as not to change as much as possible.

  The exposure control unit 101 supplies a relatively slow shutter speed control value to the exposure setting circuit 22 if the determination result is daytime and inside the tunnel. In this way, even when the light is temporarily blocked and darkened, the luminance can be increased, and the luminance of the video signal generated by imaging the subject can be controlled so as not to change as much as possible.

  When the determination result is inside the tunnel at night or in the daytime, the exposure control unit 101 does not simply slow down the shutter speed to increase the brightness, but causes the exposure setting circuit 22 to execute multiple exposure to widen the dynamic range. You may do it.

  Specifically, the exposure control unit 101 controls the exposure setting circuit 22 so that the image sensor 21 captures images alternately at a relatively slow shutter speed and a relatively fast shutter speed. The relatively slow shutter speed is 1/60 seconds, for example, and the relatively fast shutter speed is 1/250 seconds, for example.

  Then, as shown in FIG. 7, the long time exposure frames FLE1, FLE2,... And the short time exposure frames FSE1, FSE2,.

  The multiple exposure combining unit 31 combines the long exposure frame FLE1 and the short exposure frame FSE1 to generate a combined frame FMX1. The multiple exposure combining unit 31 repeatedly outputs the combined frame FMX1 at the timing of the next frame.

  The multiple exposure combining unit 31 combines the long exposure frame FLE2 and the short exposure frame FSE2 to generate a combined frame FMX2. The multiple exposure composition unit 31 repeatedly outputs the composite frame FMX2 at the timing of the next frame. Thereafter, the multiple exposure combining unit 31 repeats the same operation.

  If the determination result is daytime, the exposure control unit 101 controls the exposure setting circuit 22 to execute normal exposure instead of multiple exposure.

  At this time, the exposure control unit 101 can make substantially normal exposure by making the shutter speed set as a relatively slow shutter speed the same as the shutter speed set as a relatively fast shutter speed. .

  If the multiple exposure combining unit 31 combines two frames shot at the same shutter speed to generate combined frames FMX1, FMX2,..., It is equivalent to normal exposure instead of multiple exposure. In this way, the operation in the multiple exposure combining unit 31 can be made common regardless of whether the determination result is daytime or at night or in the tunnel at daytime.

  The present invention is not limited to the embodiment described above, and various modifications can be made without departing from the scope of the present invention.

2 Imaging unit 31 Multiple exposure composition unit 6 Infrared sensor (sensor)
101 Exposure control unit 103 Scene determination unit
1031 Long time constant integration unit (first detection unit)
1032 Short time constant integration unit (second detection unit)
1033 Comparison unit (first detection unit)
1034 Comparison unit (second detection unit)
1035 Judgment part

Claims (6)

An imaging unit for imaging a subject;
A sensor for detecting brightness;
A first detection unit for detecting whether or not a period detected by the sensor as having a predetermined brightness continues for a first time;
A second detector for detecting whether or not a period of time detected by the sensor as having a predetermined brightness continues for a second time shorter than the first time;
Based on the first detection result by the first detection unit and the second detection result by the second detection unit, the imaging unit is daytime and the light is temporarily blocked. A determination unit for determining that the subject is being imaged;
An imaging apparatus comprising: The first detection unit includes:
A first integrating unit that integrates the output of the sensor with a first time constant to generate a first integrated value;
A first comparator that binarizes the first integrated value based on a comparison with a first threshold;
Have
The second detection unit includes:
A second integration unit that integrates the output of the sensor with a second time constant shorter than the first time constant to generate a second integrated value;
A second comparison unit that binarizes the second integrated value based on a comparison with a second threshold;
The imaging apparatus according to claim 1, wherein:   The imaging apparatus according to claim 1, further comprising an exposure control unit that controls exposure in the imaging unit according to a determination result by the determination unit. When the determination unit determines that the imaging unit is imaging the subject in the specific state, the exposure control unit performs a long exposure with a first shutter speed by the imaging unit. , Alternately performing short-time exposure shorter than the first shutter speed,
The imaging apparatus according to claim 3, further comprising a multiple exposure composition unit that synthesizes the long exposure frame by the long exposure and the short exposure frame by the short exposure to generate a composite frame.   The imaging apparatus according to claim 1, wherein the sensor is an infrared sensor provided to detect a type of a light source. Detecting whether or not the period detected by the sensor for detecting brightness as having a predetermined brightness continues for the first time;
Detecting whether the period of time detected by the sensor as having a predetermined brightness continues for a second time shorter than the first time;
If the period detected as having the predetermined brightness does not continue for the first time, it is determined that it is nighttime;
If the period detected as having the predetermined brightness continues for the first time, and if the period detected as having the predetermined brightness continues for the second time, it is determined as daytime,
The period detected when the predetermined brightness is present continues for the first time, and the period detected when the predetermined brightness is detected does not continue for the second time. A scene determination method, characterized in that it is determined that the specific state is temporarily blocked.

Images