JP4962132B2 - Imaging device - Google Patents

Description

  The present invention relates to signal processing of an imaging apparatus.

  As a background art regarding the wide dynamic range function, for example, there is JP-A-2002-314885 (Patent Document 1). In this publication, it is stated that “It is possible to pick up images using high-speed and low-speed electronic shutter processing and perform signal processing on the obtained video signal to pick up an image for a very wide range of light”. As a solution, “The imaging output of the CCD camera unit 302 is obtained as first and second image signals with different exposure times by an electronic shutter circuit. The first and second image signals are branched into two systems. The characteristics are controlled and combined, and the integrated value circuit 315, the peak value detection circuit 316, and the microcomputer circuit 318 detect the screen characteristics and control the shutter timing so as to obtain an image with a wide luminance dynamic range. " Are listed.

Japanese Patent Laid-Open No. 2002-314885

  In recent years, an imaging apparatus using a solid-state imaging device has been used for many applications such as a video camera and a surveillance camera. In general, in a camera, in order to capture an image with an appropriate amount of light, the amount of light is adjusted by an electronic shutter function of a solid-state image sensor. The electronic shutter function here refers to adjusting a charge accumulation period within a field period by applying a shutter pulse in a solid-state imaging device capable of sweeping out signal charges accumulated in each photoelectric conversion element by applying a shutter pulse. This means that the exposure time can be varied. If the shutter pulse interval is shortened, the exposure time is shortened, and if the shutter pulse interval is lengthened, the exposure time is lengthened.

  And for a subject with a wide range of brightness, which is a mixture of low brightness subjects and high brightness subjects, images taken with short shutter pulse intervals and long shutter pulse intervals during the field image period or frame image period. A wide dynamic range is achieved by capturing and combining both images.

  By the way, in recent years, lighting and information display devices using LEDs (Light Emitting Diodes), which are excellent in energy saving and long life of devices, have been rapidly increasing, and they are repeatedly turned on and off at the commercial frequency of AC power supply or twice that. ing.

As described above, the electronic shutter function of the image pickup apparatus using the solid-state image pickup device has the time to take an image,
Since it has a time during which it does not image and the light emission timing of the LED information display device also has a time during which it emits light and a time during which it does not emit light, the imaging period by the electronic shutter and the light emission period of the LED are the combinations shown in FIG. When it becomes, it will image the light extinction state of LED.

  When such a conventional shutter function is used, for example, in a surveillance camera installed in the elevator car, there is a problem that the display information of the floor number displayed on the LED information display device in the elevator, which is important information, cannot be photographed. Arise. By extending the exposure time, it is possible to reliably shoot the information on the LED information display device, but the dynamic range at the time of imaging is insufficient for surrounding images that occupy the majority of the screen, and whiteout It is inappropriate for surveillance camera video. Although the conventional wide dynamic range function is effective in ensuring image quality for a subject with a mixture of low-brightness and high-brightness subjects, the subject that repeats high-speed flashing at the local position of the screen, which is the subject of the present invention, is ensured. It is not effective for shooting in the lighting state. In this regard, the above-described Patent Document 1 does not consider these problems.

  SUMMARY OF THE INVENTION An object of the present invention is to solve the above-described problem and to provide an imaging apparatus that can suitably photograph a subject.

In order to solve the above-described object, for example, the configuration described in the claims is adopted. For example, an imaging unit that optically forms light from a subject, a solid-state imaging device that electrically converts an optical image obtained from the imaging unit and outputs it as a video signal, and a solid-state imaging device An imaging device driving unit that controls the first shutter speed and the second shutter speed by switching between the first shutter speed and the second shutter speed, and is set to be longer than the blinking cycle of the object that repeats blinking included in the subject A first image storage means for storing a video signal of one frame shot at the first shutter speed, a second image storage means for storing a video signal of one frame shot at the second shutter speed; In the image comparison means and the image comparison means for reading out the video signal of each frame from the first and second image storage means and dividing it into a plurality of areas and comparing the signals in the same area of each frame, Based on the signal amount, a position information storage unit that stores position information of an area that is determined to be a blinking object, and an area that is determined to be a blinking object that is stored in the position information storage unit And an image composition means for replacing the video signal of the area stored in the second image storage means with the video signal of the area stored in the first image storage means.

  According to the imaging apparatus of the present invention, it is possible to provide an imaging apparatus that can suitably photograph a subject.

  Embodiments of an image pickup apparatus according to the present invention will be described below with reference to the drawings.

  First, an installation example of a surveillance camera attached in an elevator car according to the present invention will be described with reference to FIG.

  The surveillance camera 100 mounted in the elevator cage 420 shoots the LED information display device 410, the floor display unit 411, and the door 430 within the angle of view. The floor number display unit 411 in the LED information display device 410 notifies the elevator user of the floor number in which the elevator cage 420 is located in response to a signal from the LED information display device 410.

  Next, the configuration of the LED information display device 410 according to the present invention will be briefly described with reference to FIG.

  The LED display device 410 includes a floor display unit 411, a power supply circuit unit 501, and an LED control circuit 503. The LED display is repeatedly turned on and off as shown in FIG. 2 to save energy and extend the life of the device. In order to realize the intermittent drive for turning on and off, the power supply circuit unit 501 is provided with a virtual switch 502 that switches ON / OFF according to the input voltage by using an AC voltage input power supply 504. The floor display unit 411 outputs the current floor to the floor display unit 411 according to the control of the LED control circuit 503 based on the signal from the saddle position sensor 505 during the period when the virtual switch 502 is ON. The power supply circuit 501, the virtual switch 502, and the LED control circuit 503 may be outside the LED information display device 410.

  Next, the configuration of an embodiment of the surveillance camera 100 that implements the present invention will be described with reference to FIG. In FIG. 1, 101 is an imaging means. 102 electrically converts optical information, and uses a frame accumulation method (a method in which the signal accumulation time of one pixel of the solid-state imaging device is 1/30 seconds) or a field accumulation method (a signal accumulation in one pixel of the solid-state imaging device). This is a solid-state imaging device that reads out signal charges and outputs them as video signals by a method having a time of 1/60 seconds. Reference numeral 103 denotes signal amplification means. Reference numeral 104 denotes analog signal processing means. Reference numeral 105 denotes A / D conversion means. Reference numeral 106 denotes digital signal processing means. Reference numeral 107 denotes D / A conversion means. Reference numeral 108 denotes video output means. Reference numeral 109 denotes an image sensor driving means. 110 is a signal processing microcomputer. Reference numeral 111 denotes signal selection means. Reference numerals 121 and 122 denote image storage means capable of storing one frame of video. 123 is a comparison means. Reference numeral 124 denotes position information storage means capable of storing position information. 125 is a synthesis means. Reference numeral 140 denotes a control controller outside the monitoring camera 100.

  Hereinafter, the operation of the monitoring camera 100 will be described. An optical image obtained by the imaging means 101 such as a lens for optically imaging light from the subject is photoelectrically converted by the solid-state imaging device 102, the output is amplified by the signal amplifying means 103, and analog signal processing is performed as a video signal. Supplied to means 104. The analog signal processing unit 104 performs gamma correction, gain characteristic correction, and the like on the video signal output from the signal amplification unit 103.

  The video signal output from the analog signal processing means 104 is A / D converted by the A / D conversion means 105, and image storage means for each frame based on information from the signal processing microcomputer 110 sent to the signal selection means 111. 121, stored in the image storage means 122. The video signal stored in the image storage unit 121 and the image storage unit 122 is used to determine the area of the subject blinking at a high speed like the floor display unit 411 captured by the solid-state image sensor 102 by the comparison unit 123, and the position thereof Information is stored in the position information storage means 124. As the position information, instead of inputting the information from the comparison unit 123, a signal from the controller 140 provided outside the monitoring camera 100 can be stored.

  The video signals stored in the image storage unit 121 and the image storage unit 122 are synthesized based on the position information stored in the position information storage unit 124 by the synthesis unit 125 and are converted into digital signals as one-frame video signals. It is supplied to the processing means 106. The digital signal processing means 106 performs various signal processing such as contour correction and noise reduction on the digital video signal, D / A conversion is performed by the D / A conversion means 107, and the signal is encoded by the output processing means 108. Output to the outside. Note that the D / A conversion circuit 107 may be omitted and output as digital. The output video signal is connected to an external device (not shown) such as an external monitor or a recording device.

  At this time, the luminance level information of the video signal is transmitted from the digital signal processing means 106 to the signal processing microcomputer 110 for each frame.

The signal processing microcomputer 110 is based on the luminance level information of the video signal from the digital signal processing means 106 and the information on the presence / absence of the object that repeatedly blinks from the position information storage means 124.
The image sensor driving means 109 is controlled to drive the solid-state image sensor 102 at a predetermined electronic shutter speed.

  Further, the solid-state imaging device 102, the signal amplifying unit 103, the analog signal processing unit 104, the A / D conversion unit 105, the digital signal processing unit 106, the D / A conversion unit 107, the video output unit 108, the image sensor driving unit 109, the signal The processing microcomputer 110, the signal selection unit 111, the image storage unit 121, the image storage unit 122, the comparison unit 123, the image storage unit 124, and the synthesis unit 125 may be combined into one chip or in any combination.

  Next, FIG. 5 shows the influence of the shutter speed on an image frame including a subject that repeatedly blinks at high speed and surrounding subjects, and FIG. 6 shows a method for detecting an image frame in a lighting state of a subject that blinks at high speed. A method of synthesizing the image frame of the lighting state of the subject and the image frames of the surrounding subjects will be described with reference to FIG.

  FIG. 5 shows, for example, the floor display unit 411 and the door 430 that the image state of the subject that repeats blinking and the surrounding subject are different depending on the shutter speed.

  FIG. 5A shows an image when the shutter speed is shot at a sufficiently high speed (for example, 1/60 seconds) that allows shooting when an object that repeatedly blinks at high speed is turned on. At this time, the floor display unit 411 is photographed while the LED is lit, and the door 430 is also photographed.

  FIG. 5B shows an image taken at a shutter speed of 1/120 seconds. At this time, when the floor display unit 411 and the door 430 are in the shooting state, the exposure time is ½ compared to when the shutter speed is 1/60 seconds. When shooting with B) set to the same value, the amount of video signal obtained is halved.

  FIG. 5C shows an image taken at a shutter speed of 1/480 seconds. In the floor display unit 411, the light emission timing of the LED and the shutter timing of the solid-state imaging device are completely deviated from each other so that the LED lighting state cannot be photographed, and the signal amount is very small in the portion where the LED lighting state cannot be photographed. In addition, since the exposure time of the photographing state of the door 430 is 1/8 compared to when the shutter speed is 1/60 seconds, the amplification factor by the signal amplification unit 103 is set to the same value (A) and (C). The amount of signal is 1/8.

  That is, when shooting a subject that repeatedly blinks at high speed, the light emission timing of the LED and the shutter timing of the solid-state image sensor may be completely shifted. In such a case, even if the exposure time is 1 / N, the signal amount Does not become 1 / N, and the signal amount becomes very small. Therefore, in such a case, it is necessary to detect whether or not the LED is lit.

  Next, a method of detecting an image frame in a lighting state of a subject that repeatedly blinks at high speed using the above characteristics will be described using the flowchart of FIG.

  First, in step S100, a subject is photographed at a shutter speed 1 / N1. The shutter speed is made sufficiently long, such as 1/60 seconds, in which the lighting state of the LED or the like can be reliably photographed according to the characteristics of the subject such as the LED that blinks at high speed. High-speed blinking of LEDs or the like often repeats blinking and extinguishing at a period synchronized with the commercial power supply (frequency of the AC power supply), or one cycle or half cycle of the commercial power supply (twice the frequency of the AC power supply). Therefore, the shutter speed is set longer than the blinking cycle of the object that repeats blinking in synchronization with the commercial power supply, or set longer than the blinking cycle of the object that repeats blinking in one cycle or half cycle of the commercial power supply. The subject can be suitably imaged.

  Next, in step S101, the video signal captured in step S100 is stored in the image storage unit 121 as Y1. Y1 is arbitrarily divided into n areas, Y1_1 to Y1_n.

  In step S102, the subject is photographed at a shutter speed 1 / N2. The shutter is shot at a high shutter speed at which there is a high possibility that no LED will appear, for example, 1/4000 seconds.

  In step S103, the captured video signal is stored in the image storage unit 122 as Y2. Y2 is also divided into n areas, Y2_1 to Y2_n.

  In step S104, a counter for counting the number of areas is reset.

  In step S105, the video signal division area Y1_k stored in the image storage unit 121 and the video signal division area Y2_k stored in the image storage unit 122 are compared. If the ratio of (Y1_k signal amount) / (Y2_k signal amount) and N2 / N1 is greater than or equal to the threshold value 1 / M, it is determined that the area k is an LED unit that repeatedly blinks at high speed, and the process proceeds to step S106. On the other hand, if the ratio of (Y1_k signal amount) / (Y2_k signal amount) and N2 / N1 is smaller than the threshold value 1 / M, it is determined that the area k is a normal subject, and the process proceeds to step S107.

  In step S106, information indicating that the area k is a subject that blinks at high speed, such as an LED, is stored in the position information storage unit 124.

  In step S107, information indicating that the area k is a normal subject is stored in the position information storage unit 124.

  In step S108, the counter k is increased to k + 1.

  In step S109, it is determined whether or not area k is the final area. When k ≦ n, the area in which information is written in S106 or S107 is not final, so the process returns to step S105, and the area k + 1 is determined. When k = n + 1, since the area where the information is written in S106 or S107 is the last, the process proceeds to step S110 and ends.

  However, even when exposure is performed with a high-speed shutter, the LED unit may be exposed depending on the timing. Therefore, the procedure from the flowcharts S100 to S110 in FIG. 6 is repeated several times, and the LED unit is determined even once in S105. In S106, the area written as the LED unit in the storage unit 124 is not written in the storage unit 124 as the normal unit in S107.

  Thus, by reading the position information of the n areas stored in the position information storage means 124, it is possible to determine the position information of the subject that repeatedly blinks at high speed.

  When the surveillance camera is fixed, the LED unit is always arranged at a fixed position, so that the information data written in the position information storage unit 124 need not be updated. Further, the position information of the subject may be specified by the control controller 140 installed outside the monitoring camera 100 without using the flowchart of FIG.

  Next, an image synthesis method of an image of a lighting state of a subject that repeatedly blinks at high speed and an image of surrounding subjects will be described with reference to FIGS. 1 and 7.

  The signal processing microcomputer 110 first determines from the subject position information stored in the position information storage means 124 whether there is an area that repeatedly blinks at high speed.

When there is no area that repeats flashing at high speed, the signal processing microcomputer 110 drives the solid-state imaging device 102 at a shutter speed that optimizes exposure according to the luminance signal of the entire screen obtained from the digital signal processing means 106. Then, the image sensor driving means 109 is operated.
The video signal whose signal charge has been read by the solid-state imaging device 102 by the frame accumulation method is A / D converted by the A / D converter 105 and then directly supplied to the digital signal processor 106 by the signal selector 111. The digital signal processing means 106 performs various signal processing such as contour correction and noise reduction on the digital video signal, then D / A conversion is performed by the D / A conversion means 107, and the signal is encoded by the output processing means 108. Output to the outside.

  On the other hand, when there is an area where the blinking is repeated at high speed, the signal processing microcomputer 110 has a sufficiently high shutter speed (for example, 1/60 second) shutter speed and a digital signal that can capture an image of an object that blinks at high speed. An image sensor driving unit that drives the solid-state image sensor 102 by repeating a shutter speed (for example, 1/480 seconds) optimized for exposure in accordance with the luminance signal of the entire screen obtained from the processing unit 106 for each frame. 109 is operated.

  The video signal whose signal charge has been read out by the solid-state imaging device 102 by the frame accumulation method is subjected to A / D conversion by the A / D conversion means 105, and is then distributed frame by frame by the signal selection means 111, and the shutter speed 1 / The video signal shot in 60 seconds is stored in the image storage unit 121, and the video signal whose exposure is optimized according to the luminance signal of the entire screen (taken at a shutter speed of 1/480 seconds) is stored in the image storage unit 122. Is done. Further, the amplification factor of the amplification means 103 is changed as necessary so that the luminance of the two frames of video is the same.

  FIG. 7A shows a video signal shot at a shutter speed 1/60 seconds stored in the image storage unit 121, and FIG. 7B shows a shutter speed 1/480 stored in the image storage unit 122. This is a video signal shot in seconds.

  As can be seen from FIG. 7A, when shooting at a shutter speed of 1/60 seconds, it is possible to shoot an area (such as an LED) that repeatedly blinks at a high speed, but the exposure level for other areas. Is not appropriate, resulting in inappropriate images such as overexposure.

  On the other hand, as can be seen from FIG. 7B, when shooting at a shutter speed of 1/480 seconds, it is not possible to shoot an area (such as an LED) that repeatedly blinks at high speed, but for other areas, Since the exposure level is appropriate, an appropriate image is obtained.

  Next, on the basis of the subject position information stored in the position information storage means 124, the combining means 125 determines an area from the image storage means 121 that repeatedly blinks at a high speed from a video shot at a shutter speed of 1/60 seconds. Read and replace the signal portion of the same area read from the image storage means 122. The video signal of one frame synthesized from the two frames by the synthesizing unit 125 is supplied to the digital signal processing unit 106, and various kinds of processing such as contour correction and noise reduction for the digital video signal by the digital signal processing unit 106 are performed. After the signal processing is performed, D / A conversion is performed by the D / A conversion unit 107, and the signal is encoded by the output processing unit 108 and output to the outside.

  Thus, the video signal obtained by combining the video signal in FIG. 7A and the video signal in FIG. 7B is the video signal in FIG.

  By controlling as described above, subjects that blink at high speeds while maintaining the appropriate exposure level for the surrounding image, even if the subject is mixed with lighting that repeatedly blinks at high speeds, such as LED information display devices, when shooting the subject Can also shoot.

Configuration diagram of surveillance camera in the present invention Comparison of LED emission timing and CCD imaging timing Surveillance camera and subject in the present invention Configuration diagram of LED information display device Images taken at each shutter speed Flow chart of area determination means that blinks at high speed Composite of video information of 2 frames

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Surveillance camera 101 ... Lens 102 ... Solid-state image sensor 103 ... Signal amplification means 104 ... Analog signal processing means 105 ... A / D conversion means 106 ..Digital signal processing means 107... D / A conversion means 108... Video output means 109... Image sensor driving means 110. 121 ... Image storage means, 122 ... Image storage means, 123 ... Comparison means, 124 ... Image storage means, 125 ... Composition means, 140 ... Control controller, 410 ... LED Information display device, 411, floor display, 420, elevator cage, 430, door, 501, power supply circuit, 502, switch, 503, LED control circuit, 504 ... power, 505 ... basket position sensor

Claims (6)

Imaging means for optically imaging light from a subject, a solid-state imaging device that electrically converts an optical image obtained from the imaging means and outputs it as a video signal, and first for the solid-state imaging device a shutter speed of the imaging apparatus including the imaging device driving means for controlling by switching a second shutter speed, and
First image storage means for storing a one-frame video signal imaged at the first shutter speed set longer than a blinking cycle of an object that repeats blinking included in the subject;
Second image storage means for storing a one-frame video signal imaged at the second shutter speed;
Image comparison means for reading out the video signal of each frame from the first and second image storage means and dividing it into a plurality of areas, and comparing the signals of the same area of each frame;
Oite the image comparison unit, on the basis of a signal amount of the signal, a position information storage means for storing position information of the area is determined to be subject to repeat blinking,
In an area that is determined to be a blinking object that is stored in the position information storage unit , the video signal of the area stored in the second image storage unit is stored in the first image storage unit. imaging apparatus characterized by comprising an image combining means for replacing a manner that the video signal of the area, a. The imaging device according to claim 1,
2. The imaging apparatus according to claim 1, wherein the first shutter speed is a fixed shutter speed, and the second shutter speed is an arbitrary shutter speed. The imaging device according to claim 2,
Means for amplifying the video signal output from the solid-state imaging device is provided, and the ratio of the amplification factor of the video signal photographed at an arbitrary shutter speed to the amplification factor of the video signal photographed at a fixed shutter speed is determined by an arbitrary shutter speed. An image pickup apparatus having a fixed shutter speed ratio relative to The imaging device according to claim 2,
The imaging apparatus according to claim 1, wherein the fixed shutter speed is set longer than a blinking cycle of an object that repeatedly blinks in synchronization with a commercial power source included in the subject. The imaging device according to claim 2,
The imaging apparatus according to claim 1, wherein the fixed shutter speed is set to be longer than a blinking cycle of an object that repeats blinking in one cycle or half cycle of a commercial power source included in the subject. The imaging apparatus according to claim 2, wherein
The image pickup apparatus, wherein the signal is a luminance level.

Images