JP6324090B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus. For example, the present invention relates to an exposure control that changes signal levels such as backlight correction and blackout correction, and an imaging apparatus that performs various image corrections.

  As a background field of the present technology, for example, there is JP-A-2008-131530 (Patent Document 1). The purpose of this publication is to “provide an imaging apparatus and an image processing apparatus having a simple configuration for executing gradation correction in accordance with the subject luminance level of an image”. In the configuration for correcting an image shot by exposure control that adjusts the tone, the tone correction is performed by selecting data corresponding to the subject brightness of the shot image from a limited number of tone correction characteristic data corresponding to the brightness. Specifically, the correction processing is performed by applying a gradation correction characteristic set corresponding to a region including the average luminance of the main subject of the photographed image.With this configuration, for example, a digital signal processing unit ( It is possible to execute the correction process quickly and efficiently without increasing the circuit scale of the DSP).

JP 2008-131530 A

  In Patent Document 1, exposure control for improving whiteout of an image is performed by feedback control, and it cannot follow when a high-brightness object moves within and outside the angle of view at high speed. However, there is a problem that visibility cannot be improved and image quality is deteriorated due to overcorrection.

In order to solve the above object, the configuration described in the claims is adopted.
The present application includes a plurality of means for solving the above-mentioned object. To give an example, an image for outputting an image signal obtained by performing signal level correction processing on the image signal output from the image pickup means and the image pickup means. The degree of gradation collapse in a predetermined luminance range in the image signal output from the signal correction means and the image pickup means is evaluated, and it is determined that there is no gradation collapse according to the evaluation result and that there is no gradation collapse. And a gradation collapse suppression control means for switching the exposure amount of the image pickup means and the input / output characteristics of the signal level correction processing of the image signal correction means.

  According to the present invention, it is possible to stably provide a video with improved visibility and image quality appropriately according to the presence or absence of a high-luminance subject with high responsiveness to subject variations.

1 is a schematic diagram showing an image pickup apparatus according to a first embodiment of the present invention. The figure which shows an example of the process sequence regarding the exposure control of the imaging device which concerns on 1st Example of this invention. The figure which shows an example of an operation | movement when the high-intensity photographic subject to which a whiteout of exposure control of the imaging device which concerns on 1st Example of this invention does not exist within an angle of view. The figure which shows an example of an operation | movement when the high-intensity photographic subject to which whitening of exposure control of the imaging device which concerns on 1st Example of this invention exists in an angle of view The figure which shows an example of the signal level correction process of the imaging device which concerns on 1st Example of this invention. The figure which shows an example of operation | movement when the high-intensity photographic subject to which a whiteout of the signal level correction | amendment intensity parameter control processing of the imaging device which concerns on 1st Example of this invention does not exist in an angle of view. The figure which shows an example of an operation | movement when the high-intensity subject to which a white-out of the signal level correction | amendment intensity | strength parameter control processing of the imaging device which concerns on 1st Example of this invention exists in an angle of view. The figure which shows an example of the process sequence regarding the control switching process of the imaging device which concerns on 1st Example of this invention. The figure which shows an example of the determination method of the brightness | luminance threshold value 12 in the control switching process of the imaging device which concerns on 1st Example of this invention. The figure which shows an example of the determination method of the period threshold value M1 in the control switching process of the imaging device which concerns on 1st Example of this invention. The figure which shows an example of the determination method of the high-intensity subject determination threshold th2 in the control switching process of the imaging device which concerns on 1st Example of this invention. The figure which shows an example of the process sequence regarding the 1st example of exposure control of the imaging device which concerns on 2nd Example of this invention. The figure which shows an example of operation | movement at the time of applying the exposure control of the imaging device which concerns on 1st Example of this invention. An example of operation when applying the first example of exposure control of the image pickup apparatus according to the second embodiment of the present invention The figure which shows an example of the process sequence regarding the 2nd example of exposure control of the imaging device which concerns on 2nd Example of this invention. The figure which shows an example of operation | movement at the time of applying the 2nd example of exposure control of the imaging device which concerns on 2nd Example of this invention.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a schematic diagram showing an imaging apparatus according to the first embodiment of the present invention. In FIG. 1, 0101 is an imaging unit, 0102 is a camera signal processing unit, 0103 is an image signal correction unit, 0104 is a detection unit, 0105 is a gradation collapse suppression control unit, 0105_1 is a control switching unit, 0105_2 is an exposure control unit, and 0105_3. Is an image signal correction control unit.

  In the image pickup apparatus shown in FIG. 1, the image pickup unit 0101 appropriately includes a lens group including a zoom lens and a focus lens, an image pickup element such as an iris, a shutter, a CCD or a CMOS, a CDS, an AGC, and an AD converter. The optical image received by the imaging device by exposure based on exposure conditions such as iris aperture, sensor accumulation time, shutter speed, and AGC gain amount is photoelectrically converted and output as an image signal.

  The camera signal processing unit 0102 performs various camera image processing such as digital gain processing, demosaicing processing, luminance signal and color signal generation processing, and noise correction processing on the image signal output from the imaging unit 0101, and outputs it as a video signal.

  The image signal correction unit 0103 is a signal level that is a signal value for each pixel of the video signal based on a predetermined input / output characteristic and a correction intensity parameter acquired from the image signal correction control unit 0105_3 in the video signal output from the camera signal processing unit 0102. Is corrected and output as a final video signal. An example of this process will be described later with reference to FIG.

  The detection unit 0104 calculates a feature quantity such as histogram data, an average value, and a maximum value of signal values in a predetermined region from the image signal output from the imaging unit 0101, and outputs it as detection data.

  The control switching unit 0105_1 evaluates the degree of gradation collapse such as whiteout or blackout in the image signal based on the detection data output from the detection unit 0104, and performs normal control and suppression of gradation collapse according to the evaluation result. The control status to be executed is output by switching which control is prioritized. Here, the normal control refers to controlling the exposure condition of the imaging unit 0101 and the signal level correction process of the image signal correction unit 0103 so that the subject of interest has brightness and image quality with which high visibility can be obtained. In addition, the control that prioritizes the suppression of gradation loss is that the high-brightness subject that blows off in normal control or the low-brightness subject that loses black is imaged so that high visibility is obtained. This means that the exposure conditions of the unit 0101 and the signal level correction processing of the image signal correction unit 0103 are controlled. Note that a whiteout subject refers to a portion where the luminance is the maximum value (255 or the like) in the image signal or a portion near the maximum value, and the degree of whiteout is, for example, the maximum luminance for the pixel of the image signal. Refers to the percentage of pixels near the value or maximum value. Similarly, a subject that is black-crushed refers to a portion where the luminance in the image signal is a minimum value (0 or a black level corresponding to the luminance when the image sensor is shielded) or a portion near the minimum value. The degree of indicates, for example, the ratio of pixels having a minimum value or near the minimum value with respect to the pixels of the image signal. Pixels in the vicinity of the maximum value / minimum value can be determined, for example, depending on whether the luminance is above / below a predetermined threshold value. The evaluation of the degree of gradation collapse is, for example, determining whether or not the degree of gradation collapse is equal to or greater than a threshold value.

  In the following, for the sake of simplicity of explanation, a case will be described as an example where the gradation collapse is whiteout in the image signal. For example, the control switching unit 0105_1 sequentially evaluates the degree of whiteout in the image signal from the detection data output from the detection unit 0104 when a moving image is captured by normal control, and the high-brightness subject that is whiteout. Is determined to occur within the angle of view, the control is switched to giving priority to suppression of whiteout. In addition, when taking an image with control giving priority to suppression of overexposure, if it is determined that there is no high brightness subject within the angle of view that will be overexposure even after returning to normal control, normal control is performed. Switch to. An example of this control will be described later with reference to FIG.

  When performing normal control based on the control state output from the control switching unit 0105_1, the exposure control unit 0105_2 is a brightness that allows the subject of interest to obtain high visibility while referring to the detection data output from the detection unit 0104. The exposure of the imaging unit 0101 is controlled so as to be the same. Also, in the case of control giving priority to suppression of overexposure, the exposure target is set to be higher than that of normal control so that a high brightness subject that is overexposure becomes bright enough to obtain high visibility under normal control. The exposure of the imaging unit 0101 is controlled so that becomes smaller. An example of this control will be described later with reference to FIG.

  The image signal correction control unit 0105_3 performs signal level correction processing with a predetermined characteristic that is a predetermined reference in the case of normal control based on the control state output by the control switching unit 0105_1. Determine the correction strength parameter to do. In the case of control giving priority to suppression of whiteout, in consideration of the fact that the imaging unit 0101 performs imaging with an exposure target smaller than the normal control, the signal level correction in the dark part is particularly corrected compared to the normal control. The correction intensity parameter is determined so as to increase the amount. An example of this control will be described later with reference to FIG.

  As a result, when there is no high-luminance subject within the angle of view, an image is generated so that the subject of interest has the optimal brightness and image quality. When there is a high-luminance subject within the angle of view, the high-luminance subject is overexposed. In order to generate a video that suppresses the visibility and improves the visibility of the subject of interest other than the high-intensity subject by the signal level correction process, It is possible to provide a video with adaptively improved visibility and image quality.

  Although the above has been described using overexposure as an example of suppression of gradation crushing, the same applies to the case of suppressing blackout. In other words, the control switching unit 0105_1 switches to control giving priority to suppression of black crushing when a low-luminance subject that is blacked out by normal control is generated within the angle of view, and the exposure control unit 0105_2 controls black crushing suppression. In the prioritized control, the exposure of the imaging unit 0101 is controlled so that the exposure target is larger than that in the normal control, and the image signal correction control unit 0105_3 is set so that the signal level correction amount particularly in the bright part is larger than that in the normal control. Then, the correction intensity parameter of the image signal correction control unit 0105_3 is determined. As a result, when there is a low-luminance subject within the angle of view, it is possible to improve visibility by suppressing black crushing of the low-luminance subject, and to generate a video that maintains the visibility of the subject of interest by signal level correction processing. Is possible. If both whiteout and blackout occur in the video, for example, it may be decided in advance which control to control overexposure or blackout is prioritized, Alternatively, it may be configured such that the user can select which one is prioritized and input from a user input unit (not shown).

  Although the imaging apparatus shown in FIG. 1 is described by focusing attention on the processing unit directly related to the present invention, other high image quality processing and signal format conversion processing that are important in actual image processing, The compression / decompression process or the like may be performed before, after, or during each process.

  1 is executed by an application on the CPU for a personal computer, a microcomputer, a DSP, a dedicated LSI, or the like for an embedded device, except for the imaging unit 0101. . Signal processing such as histogram data acquisition and signal level correction processing is performed on a DSP or dedicated LSI, and control processing such as control switching processing, exposure control, and correction intensity parameter control of signal level correction processing is performed by a microcomputer. Therefore, cost and performance may be optimized.

  FIG. 2 is a diagram showing an example of exposure control according to the first embodiment of the present invention. In the present invention, the exposure control is performed by the exposure control unit 0105_2.

  FIG. 2A is a diagram illustrating an example of a processing sequence related to exposure control of the imaging apparatus according to the first embodiment of the present invention. In the processing sequence related to the exposure control shown in FIG. 2A, in ST0201, an exposure target value g_e is set such that the subject of interest is bright enough to obtain high visibility. For example, g_e can be determined as follows. A subject located near the center of the angle of view is regarded as an important subject, and a predetermined area set near the center of the angle of view is defined as a subject of interest, and the exposure is controlled to control the exposure within the predetermined area. Assuming that the visibility is best when the average brightness reaches a certain brightness, assuming that this state is the exposure target, the target value of the average brightness in the predetermined area becomes the exposure target value g_e. This exposure target value g_e is stored in advance in an unillustrated storage unit such as a nonvolatile memory as an adjustment value, and is set by reading the g_e from the storage unit by the exposure control unit 0105_2 in ST0201.

  In ST0202, the current control state is acquired from the control switching unit 0105_1. If the current control is control giving priority to whiteout suppression, the process proceeds to ST0203, and if it is normal control, the process proceeds to ST0204.

  In ST0203, the exposure target value is reduced by dividing the exposure target value g_e by the predetermined exposure correction amount X. For example, X measures in advance a representative illuminance ratio between the approximate brightness of a subject having significant information, such as a display on an electronic bulletin board or a license plate with a light source, and the approximate brightness of the subject of interest among high-intensity subjects. decide. This exposure correction amount X is stored in advance in a storage unit (not shown) such as a nonvolatile memory as an adjustment value, and exposure control unit 0105_2 reads g_e from the storage unit in ST0203, thereby exposing exposure target value g_e. Correction using the correction amount X is performed.

  In ST0204, the exposure detection value det_e is compared with the exposure target value g_e. The exposure detection value det_e is an average luminance within a predetermined area set near the center of the angle of view as described above, and is acquired from the detection unit 0104 as detection data. As a result of the comparison, if the exposure detection value det_e is substantially equal to the exposure target value g_e, the process proceeds to ST0205, if it is significantly larger, proceeds to ST0206, and if it is significantly smaller, the process proceeds to ST0207. In determining whether they are substantially equal, significantly larger, or significantly smaller, for example, two threshold values are provided, and are significantly larger when the difference between the exposure detection value det_e and the exposure target value g_e is larger than the larger threshold value. A method of determining that the value is significantly smaller when the threshold value is smaller than the smaller threshold value, or approximately equal when the value is between the two threshold values is conceivable, but the present embodiment is not limited to this method.

  In ST0205, it is determined that the proper exposure has already been reached, and the exposure control is stopped.

  In ST0206, it is determined that the current exposure amount is larger than the target value and brighter than the appropriate exposure, and exposure control is performed so that the exposure amount becomes smaller. For example, the exposure amount is reduced by reducing the AGC gain, shortening the sensor accumulation time, or reducing the iris aperture.

  In ST0207, it is determined that the current exposure amount is smaller than the target value and darker than the appropriate exposure, and exposure control is performed so that the exposure amount is increased. For example, the exposure amount is increased by increasing the gain of AGC, increasing the sensor accumulation time, or increasing the aperture of the iris.

  FIG. 2B shows an example of the operation in the case where there is no overexposed high-luminance subject in the field of view of the exposure control of the image pickup apparatus according to the first embodiment of the present invention. As shown in FIG. 2B, normal exposure control is selected when there is no high-brightness photographic subject in the field of view. Therefore, exposure is controlled so that the brightness of the vehicle that is the subject of interest is optimized, and high visibility can be obtained.

  FIG. 2C shows an example of an operation in the case where a high-luminance subject with overexposure in the exposure control of the imaging apparatus according to the first embodiment of the present invention exists within the angle of view. As shown in FIG. 2C, when a high-luminance subject that is over-exposed exists within the angle of view, exposure control giving priority to over-exposure suppression is selected. Therefore, the exposure is controlled so that the exposure target becomes smaller than the normal control, and the visibility of the oncoming vehicle that is a high-luminance subject is improved.

  Thus, the exposure control unit 0105_2 can control the exposure so that the brightness of the subject of interest is appropriate when performing normal control. It is possible to control the exposure so that the brightness of the other subjects is as close as possible as possible, while preventing the high-brightness subject having, from being overexposed. In addition, when a moving image is captured and a whiteout subject is generated within the angle of view and the control is switched from normal control to whiteout suppression priority control, the exposure target value g_e is corrected by the known exposure correction amount X. , Feedforward control in which the required exposure control amount is uniquely determined. That is, it is possible to immediately determine how much the AGC gain, sensor accumulation time, etc. should be changed, and to perform exposure control at the time of switching at high speed and stably. The same applies when switching from whiteout suppression priority control to normal control.

  In the above example, the subject of interest is a predetermined area near the center of the screen, but this need not be the case. For example, if there is a possibility that a camera is installed with a depression angle and an important subject appears at the bottom of the screen, a predetermined area at the bottom of the screen may be set as the subject of interest. Alternatively, a specific subject such as a person's face or a vehicle may be detected from the image by image recognition, and a local region including the specific subject may be set as the subject of interest. As a result, it is possible to prioritize the subject that the user wants to see according to the application and to make the exposure appropriate.

  In the above example, the average luminance in the predetermined area is used as the exposure detection value det_e. However, as an example of general exposure control, a method using a weighted average of average luminance in a plurality of predetermined areas, or a maximum value is used. There are known methods using a combination of other feature quantities such as the minimum value and the like. Of course, these methods may be combined.

  In the above example, only the presence / absence of the division of the high correction amount X with respect to the exposure target value g_e is switched between the normal control and the control giving priority to suppression of whiteout, but the exposure control method itself is switched. It doesn't matter. For example, in the normal control, the exposure is controlled so that the exposure detection value det_e and the exposure target value g_e coincide with each other as in the above example, and in the control giving priority to the whiteout suppression, the maximum value of a predetermined region in the image signal is By controlling the exposure until it becomes smaller than a desired value, it is possible to generate an image with no whiteout.

  Further, when comparing the exposure detection value det_e and the exposure target value g_e, hysteresis values α and β may be provided in the exposure target value g_e to compare the magnitude relationship between det_e and g_e + α and g_e−β. As a result, even when the exposure amount detection value det_e varies due to the influence of noise or the like, the exposure control can be operated stably without malfunctioning due to the influence of variation.

  FIG. 3 is a diagram illustrating an example of signal level correction processing of the imaging apparatus according to the first embodiment of the present invention. In the present invention, the signal level correction process is performed by the image signal correction unit 0103.

  In FIG. 3, 0301 is a representative subject area existing in a dark part in the video signal, and 0302 is a representative subject area existing in a bright part in the video signal. The image signal correction unit 0103 has input / output characteristics that have different correction characteristics in the dark part and the bright part in advance, and applies corresponding input / output characteristics depending on whether the target pixel exists in the dark part or the bright part. Correct the signal level. Since the area indicated by 0301 is a subject area existing in a dark part, the visibility of a dark subject can be improved by performing correction using the input / output characteristics for raising the signal. In addition, since the area indicated by 0302 is a subject area existing in a bright part, the visibility of a bright subject can be improved by performing correction with input / output characteristics that sink signals. Here, the input / output characteristics of the dark part and the input / output characteristic of the bright part are corrected according to the correction intensity parameter acquired from the image signal correction control unit 0105_3. Therefore, when the correction intensity parameter is small, the correction can be weakened by bringing the input and output characteristics close to each other.

  As a result, the gradation can be adaptively corrected according to the brightness of the subject to improve the visibility in the blackout area and the overexposure area, and the correction strength parameter can be given as an input to correct it as necessary. It is possible to change presence or absence and strength.

  The determination of whether the target pixel exists in the dark part or the bright part is performed by, for example, calculating the low frequency component of the luminance signal from the average value of the luminance of the local region including the target pixel, and calculating the calculated low frequency component. This can be done by threshold processing.

  FIG. 4 is a diagram illustrating an example of signal level correction intensity parameter control processing of the imaging apparatus according to the first embodiment of the present invention. In the present invention, the signal level correction intensity parameter control processing is performed by the image signal correction control unit 0105_3.

  FIG. 4A shows an example of an operation in the case where there is no overexposed high-luminance subject in the angle of view in the signal level correction intensity parameter control processing of the imaging apparatus according to the first embodiment of the present invention. As shown in FIG. 4A, the exposure control unit 0105_2 selects normal exposure control when there is no overexposed high-luminance subject within the angle of view. Therefore, since the video signal output from the camera signal processing unit 0102 is already a video having high visibility, the image signal correction control unit 0105_3 determines the correction strength parameter so that the degree of signal level correction is reduced. Therefore, it is possible to suppress unnaturalness of the video and noise enhancement that occur when the signal level of the corrected video is corrected more than necessary.

  FIG. 4B shows an example of the operation when a high-intensity subject with whiteout in the signal angle of the signal level correction intensity parameter control processing of the imaging apparatus according to the first embodiment of the present invention exists within the angle of view. As shown in FIG. 4B, when a high-intensity subject that is whiteout exists within the angle of view, the exposure control unit 0105_2 selects exposure control that prioritizes whiteout suppression. Therefore, the video signal output from the camera signal processing unit 0102 improves the visibility of the oncoming vehicle, which is a high-luminance subject, while the preceding vehicle in the dark portion becomes darker and the visibility is reduced. Therefore, the image signal correction control unit 0105_3 determines the correction intensity parameter so that the degree of correction of the signal level in the dark part is particularly increased. As a result, the corrected image can be generated with high visibility for both the oncoming vehicle that is a high-luminance subject and the preceding vehicle that is not the high-luminance subject.

  As described above, the image signal correction control unit 0105_3 can generate a high-quality video by reducing the correction strength of the signal level to suppress video breakdown and noise enhancement when performing normal control. In the control that prioritizes whiteout suppression, it is possible to improve the visibility of a subject of interest other than a high-luminance subject whose visibility is reduced due to exposure control that suppresses whiteout. In addition, even when a moving image is captured and a whiteout subject occurs within the angle of view, and the normal control is switched to the whiteout suppression priority control, the signal level correction intensity is switched at the same timing as the exposure control switching. It is possible to prevent oversight due to a decrease in visibility.

  FIG. 5 is a diagram illustrating an example of a processing sequence related to control switching processing of the imaging apparatus according to the first embodiment of the present invention. In the present invention, the control switching process is performed by the control switching unit 0105_1. It is assumed that the control switching process is repeatedly performed in the control switching unit 0105_1 at regular or variable time intervals, for example, in units of frames.

  In the processing sequence related to the control switching process shown in FIG. 5, in ST0501, it is determined by referring to a memory or the like whether the current control state is normal control or control giving priority to whiteout suppression, and the current control is normal control. If YES, the process proceeds to ST0502, and if the control is given priority to whiteout suppression, the process proceeds to ST0509.

  In ST0502, the number N1 of pixels whose signal value exceeds the luminance threshold value l1 in a predetermined region in the image signal is obtained as detection data from the detection unit 0104, and the process proceeds to ST0503. The luminance threshold value l1 is determined in advance near the saturation of the signal value, for example. As a result, N1 is the area of a high-luminance subject that is overexposed in an image captured by normal control.

  In ST0503, N1 is compared with a high brightness subject determination threshold th1. If N1 is greater than or equal to th1, it is determined that a high-intensity subject that is whiteout has occurred, and if ST1504 is less than th1, it is determined that there is no high-luminance subject that is whiteout and ST0507 is reached. move on. The high-brightness subject determination threshold th1 is determined in advance to a value that exceeds a typical size of a point light source or the like so that the influence of a high-brightness subject that is unlikely to contain significant information such as a point light source can be excluded. As a result, only a high-luminance subject that is likely to contain significant information such as an electronic bulletin board can be determined.

  In ST0504, the counter cnt1 for counting the length of the period in which it is determined that the high-luminance subject has continuously occurred is incremented, and the process proceeds to ST0505.

  In ST0505, cnt1 is compared with the period threshold value M1. If cnt1 is a value equal to or greater than M1, it is determined that there is a high-luminance subject that is whiteout for a certain period, and the process proceeds to ST0506. If cnt1 is less than M1, the process proceeds to ST0508. An example of a method for determining M1 will be described later with reference to FIG.

  In ST0506, the control state of the next frame is switched to control giving priority to whiteout suppression, and the process is recorded in a memory or the like and the process ends. As a result, when it is determined that a high-brightness subject that causes whiteout has occurred during normal control, it is possible to immediately switch to control that gives priority to whiteout suppression and improve the visibility of the high-brightness subject. It becomes possible.

  In ST0507, cnt1 is initialized to 0, and the process proceeds to ST0508.

  In ST0508, the control state of the next frame shift is maintained as normal control, and the process ends.

  In ST0509, the detection unit 0104 obtains the number of pixels N2 whose signal value exceeds the luminance threshold value l2 in a predetermined region in the image signal as detection data, and proceeds to ST0510. The luminance threshold value l2 is set in advance as a threshold value for determining the luminance of the subject that is predicted to be whitened when normal exposure is performed. An example of a method for determining the luminance threshold value l2 will be described later with reference to FIG.

  In ST0510, N2 is compared with high-luminance subject determination threshold th2. When N2 is less than th2, it is determined that the high-luminance subject that has been blown out when the normal control is returned is lost, and when ST1 is N1 or more, when N1 is greater than or equal to th1, it is blown out when the normal control is restored. It is determined that a high brightness subject exists, and the process proceeds to ST0514. The high brightness subject determination threshold th2 is determined based on the same standard as the high brightness subject determination threshold th1, and an example of the determination method will be described later with reference to FIG.

  In ST0511, the counter cnt2 for counting the length of the period in which it is determined that the high-luminance subject has continuously disappeared is incremented, and the process proceeds to ST0512.

  In ST0512, cnt2 is compared with period threshold value M2. If cnt2 is a value equal to or greater than M2, it is determined that a high-intensity subject that is overwhelmed when returning to normal control for a certain period of time continues, and in ST0513, if cnt1 is lower than M1, Proceed to ST0515. The period threshold value M2 is determined on the same basis as the period threshold value M1.

  In ST0513, the control state of the next frame shift is switched to normal control, recorded in a memory or the like, and the process ends. As a result, when control giving priority to whiteout suppression is performed, if it is determined that there is no high-intensity subject that is whiteout when returning to normal control, the control is immediately switched to normal control, and the brightness of the subject of interest is adjusted. It is possible to return to a proper and high-quality video.

  In ST0514, cnt2 is initialized to 0, and the process proceeds to ST0515.

  In ST0515, the control state of the next frame shift is held with the control giving priority to whiteout suppression, and the process ends.

  This makes it possible to immediately switch between control that prioritizes improving the visibility of a high-brightness subject and control that prioritizes the image quality of the subject of interest, depending on the presence or absence of a high-brightness subject that is whiteout within the angle of view. it can. In particular, when switching from a control that prioritizes whiteout suppression to normal control, the conventional feedback control that gradually increases the exposure target while observing the degree of whiteout may result in reduced responsiveness or exposure control. In some cases, overcorrection may occur before the image converges, and whiteout may occur. However, in the control based on this embodiment, switching is performed immediately after ensuring that whiteout will not occur even if the control is returned to normal control. Can do. Therefore, even when a high-brightness object moving at high speed, such as a vehicle with a lamp, enters and exits the angle of view, it provides stable and stable images with high responsiveness and optimal visibility for each scene. Is possible.

  FIG. 6 is a diagram illustrating an example of a method for determining the luminance threshold value l2 in the control switching process of the imaging apparatus according to the first embodiment of the present invention.

  FIG. 6A shows an exposure control that prioritizes whiteout suppression as a first example of a video image when priority is given to whiteout suppression in the image pickup apparatus according to the first embodiment of the present invention. An example of a scene in which an oncoming vehicle is present within the angle of view as a high-luminance subject.

  In exposure control that prioritizes whiteout suppression, considering that the exposure amount is 1 / X compared to when normal exposure control is applied, it is for high-luminance subject determination in control that prioritizes whiteout suppression. The luminance threshold value l2 is set assuming that the luminance threshold value l1 for determining a high-luminance subject in normal control is 1 / X. As shown in FIG. 6A, the high-intensity subject that is blown out by the normal exposure control is distributed in a range higher than 12 in the histogram obtained by the exposure control giving priority to the whiteout suppression. Therefore, if the number of pixels having a signal value equal to or greater than the luminance threshold value l2 is large, it can be determined that whiteout occurs when returning to normal control.

  In FIG. 6B, exposure control giving priority to whiteout suppression is applied as a second example of a video image when priority is given to whitening suppression in the image pickup apparatus according to the first embodiment of the present invention. An example of a scene in which a high brightness subject does not exist within the angle of view is shown.

  As shown in FIG. 6B, when there is no high-intensity subject that is whiteout in the angle of view in normal exposure control, the histogram obtained by exposure control giving priority to whiteout suppression has a range higher than l2. There is no frequency. Therefore, if the number of pixels having a signal value equal to or greater than the luminance threshold value l2 is small, it can be determined that whiteout does not occur even when the normal control is restored.

  In this way, by determining the brightness threshold in the control giving priority to whiteout suppression from the brightness threshold value of normal control and the exposure correction amount, when control giving priority to whiteout suppression is performed, the control is returned to the normal control. Therefore, it is possible to perform stable switching control such as returning to normal control only when whiteout does not occur by accurately determining whether or not whiteout occurs.

  FIG. 7 is a diagram illustrating an example of a method for determining the period threshold value M1 in the control switching process of the imaging apparatus according to the first embodiment of the present invention.

  In FIG. 7, the horizontal axis indicates the shooting time, and the vertical axis indicates the number N1 of high-luminance subject pixels calculated at each time. In this example, the subject fluctuates according to the shooting time, and the high-brightness subject does not exist within the angle of view until time t1 and during the period from t2 to t3, and the period from time t1 to t2 disappears in a very short time. It is assumed that the flash light to be turned on within the angle of view, and a high-luminance subject that does not disappear in a minute time such as an oncoming vehicle exists within the angle of view during the period after t3. At this time, as described with reference to FIG. 5, the control switching unit 0105_1 increments the counter cnt1 in accordance with the period when the high-luminance subject pixel number N1 continuously exceeds the high-luminance subject determination threshold th1. When cnt1 exceeds the period threshold value M1, the control is switched from normal control to control giving priority to whiteout suppression. Therefore, by setting the period threshold value M1 to a value sufficiently larger than t2-t1, the control is not switched in the presence or absence of a momentary high-intensity subject such as flash light, and the control is performed when an oncoming vehicle or the like occurs. It is possible to improve the stability of control such that switching is performed. By similarly setting the period threshold value M2 when the control giving priority to the whiteout suppression is performed, the control stability is improved such that the control is not switched when the light is momentarily lost. It becomes possible.

  FIG. 8 is a diagram illustrating an example of a method for determining the high brightness subject determination threshold th2 in the control switching process of the imaging apparatus according to the first embodiment of the present invention.

In FIG. 8, the horizontal axis indicates the number N1 or N2 of high-luminance subject pixels, and the vertical axis indicates a determination result as to which control the control switching unit 0105_1 switches to in the next control. As shown in FIG. 8, for example, the high-luminance subject determination threshold th2 in the control giving priority to whiteout suppression is set smaller than the high-luminance subject determination threshold th1 in normal control. As a result, when the number of high-luminance subject pixels is in the vicinity of the range between th1 and th2, control that prioritizes whiteout suppression from normal control is relatively easy to shift, and control that prioritizes whiteout suppression. Therefore, it becomes relatively difficult to shift to normal control. By setting th1 and th2 in such a manner as described above, when the detection data of the number of high-luminance subject pixels varies from time to time due to the influence of flicker, noise, etc., the control switching is suppressed from being repeated alternately. Can be stabilized.
As described above, according to the present embodiment, an image having improved visibility and image quality adapted to each scene with high responsiveness and stability to changes in the presence / absence of a high-luminance subject within the shooting angle of view. It becomes possible to provide.

  FIG. 9 is a diagram showing a first example of exposure control according to the second embodiment of the present invention. In the present invention, the first example of exposure control is performed by the exposure control unit 0105_2.

  FIG. 9A is a diagram showing an example of a processing sequence relating to a first example of exposure control of the imaging apparatus according to the second embodiment of the present invention, and FIG. 9B is a first embodiment of the present invention. FIG. 9C shows an example of the operation when the exposure control of the image pickup apparatus according to the present invention is applied, and FIG. 9C shows the operation when the first example of the exposure control of the image pickup apparatus according to the second embodiment of the present invention is applied. It is an example.

  In the processing sequence relating to the first example of the exposure control shown in FIG. 9A, in ST0901, the current control state is acquired from the control switching unit 0105_1, and the current control state is a control giving priority to whiteout suppression. If so, the process proceeds to ST0902, and if it is normal control, the process proceeds to ST0906.

  In ST0902, the number of pixels Nd1 whose signal value is lower than the luminance threshold value d1 in a predetermined region in the image signal is acquired as detection data from the detection unit 0104, and the process proceeds to ST0903. The luminance threshold value d1 is determined in advance near the black level of the signal value, for example. As a result, Nd1 is the area of the subject that is crushed black in the image captured by the control that prioritizes whiteout suppression.

  In ST0903, the blackout pixel number Nd1 is compared with the blackout determination threshold thd1. If Nd1 is significantly larger than thd1, it is determined that black crushing exists, and the process proceeds to ST0904. If Nd1 is significantly smaller than thd1, it is determined that there is no black crushing, and the process proceeds to ST0905. When Nd1 is substantially the same as thd1, it is determined that there is a possibility that blackout may occur if the exposure correction amount is further increased, and the process proceeds to ST0906. The thd1 is determined based on a typical ratio of low-luminance subjects that naturally exist within the angle of view, such as black patterns and shadows. In determining whether they are substantially equal, significantly larger, or significantly smaller, for example, two threshold values are provided in the same manner as described above, and the difference between the blackout pixel number Nd1 and the blackout determination threshold thd1 is larger than the threshold value. It is possible to consider a method such as judging that it is significantly larger if it is larger, significantly smaller if it is smaller than the smaller threshold, or approximately equal if it is between two thresholds, but this embodiment is limited to this method. Is not to be done.

  In ST0904, the exposure correction amount X is decreased and the process proceeds to ST0906. Thereby, when black crushing has occurred, the effect of suppressing whiteout can be suppressed and black crushing can be improved.

  In ST0905, the exposure correction amount X is increased and the process proceeds to ST0906. As a result, when blackout does not occur, the effect of suppressing whiteout can be strengthened to improve the visibility of a high-luminance subject.

  In ST0906, exposure control is performed using the controlled exposure correction amount X. The operation of the exposure control is as described with reference to FIG. 2. When normal control is performed, exposure control is performed with reference to a predetermined exposure target value g_e, and in the control giving priority to whiteout suppression, it is determined in advance. Since the exposure target value g_e is corrected using the controlled exposure correction amount X, exposure control is performed.

  As shown in FIG. 9B, when the exposure control according to the first embodiment of the present invention is applied, a strong whiteout suppression effect can be obtained, but the illuminance between the high brightness subject and the subject of interest other than the high brightness subject. If the difference is very large, the subject of interest other than the high-luminance subject may be blacked out. As shown in FIG. 9C, when the exposure control according to the first embodiment of the present invention is applied, the exposure correction amount X is determined by determining the presence or absence of black crushing with reference to the distribution of pixels below the luminance threshold value d1. Because of the control, when the exposure target is lowered too much and the blackout becomes severe, it is possible to generate an image that suppresses the whiteout and the blackout with the best balance by suppressing the effect of whiteout suppression.

In the above example, when comparing the blackout pixel count Nd1 with the blackout determination threshold thd1, hysteresis values γ and Δ are provided for the blackout determination threshold thd1, and the magnitude relationship between Nd1 and thd1 + γ and thd1−Δ. May be compared. Thereby, even when the blackout pixel number Nd1 varies due to the influence of noise or the like, the exposure control can be operated stably without malfunctioning due to the influence of variation.
Further, when controlling the exposure correction amount X, limit processing may be performed by providing a maximum value and a minimum value in the control range of the exposure correction amount X. As a result, it is possible to determine and guarantee the best effect and the lowest effect of whiteout suppression in advance.

  FIG. 10 is a diagram showing a second example of exposure control according to the second embodiment of the present invention. In the present invention, the second example of exposure control is performed by the exposure control unit 0105_2.

  FIG. 10A is a diagram showing an example of a processing sequence relating to a second example of exposure control of the imaging apparatus according to the second embodiment of the present invention, and FIG. 10B is a second embodiment of the present invention. It is an example of operation | movement at the time of applying the 2nd example of exposure control of the imaging device which concerns on this.

  In the processing sequence relating to the second example of exposure control shown in FIG. 10A, in ST1001, the current control state and the next control state are acquired from the control switching unit 0105_1, and the current control state is normal. If it is a control and the next control state is a control that gives priority to whiteout suppression, that is, if the timing is switched from normal control to control that gives priority to whiteout suppression, ST1002 should be the other timing. If so, proceed to ST1008.

  In ST1002, the number of pixels Nd2 whose signal value falls below the luminance threshold d2 and the number of pixels Nd3 below the luminance threshold d3 in the predetermined area in the image signal are acquired from the detection unit 0104 as detection data, and the process proceeds to ST1003. The luminance threshold value d2 and the luminance threshold value d3 are set so that d2 <d3. As a result, Nd2 is the area of the subject that is predicted to be blacked out when control is performed with priority given to whiteout suppression when the exposure correction amount X is large. Nd2 is the area of the subject that is predicted to be blacked out when the control is performed with priority given to whiteout suppression when the exposure correction amount X is slightly large.

  In ST1003, the first blackout prediction pixel number Nd2 is compared with the blackout determination threshold thd2. If Nd2 is greater than or equal to thd2, it is predicted that blackout will occur when the exposure correction amount X is large and switching to control that prioritizes whiteout suppression, and if Nd2 falls below thd2, the process proceeds to ST1004.

  In ST1004, the second predicted blackout pixel number Nd3 is compared with the blackout judgment threshold thd3. If Nd3 is greater than or equal to thd3, it is predicted that blackout will occur when the exposure correction amount X is slightly larger when switching to control that prioritizes whiteout suppression, and if Nd3 falls below thd3, the process proceeds to ST1007. Thd2 and thd3 may be determined in the same manner as thd1.

  In ST1005, the exposure correction amount X is set to X2, and the process proceeds to ST1008.

  In ST1006, X1 is set as the exposure correction amount X, and the flow proceeds to ST1008.

  In ST1007, the exposure correction amount X is set to X0, and the process proceeds to ST1008.

  In ST1008, exposure control is performed using the controlled exposure correction amount X. The operation of the exposure control is as described with reference to FIG. 2. When normal control is performed, exposure control is performed with reference to a predetermined exposure target value g_e, and in the control giving priority to whiteout suppression, it is determined in advance. The exposure control is performed after the exposure target value g_e is corrected using the controlled exposure correction amount X. Therefore, the values of X0 to X2 set as the exposure correction amount X in ST1005 to ST1007 are used as the initial value of the exposure correction amount X when the control giving priority to whiteout suppression is started. Here, X0 to X2 are reference exposure correction amounts determined in advance so as to satisfy the relationship of X0> X1> X2. For this reason, as shown in FIG. 10B, when it is predicted that blackout will occur when switching to control that prioritizes whiteout suppression when the number of pixels below the luminance threshold d2 is large and the exposure correction amount X is large. Can reduce the exposure correction amount X in advance to switch to control giving priority to whiteout suppression, and immediately generate an image that suppresses whiteout suppression and suppresses whiteout and blackout in the most balanced manner. it can. In addition, when it is predicted that blackout will not occur when switching to control that prioritizes whiteout suppression even if the exposure correction amount X is large, the effect of whiteout suppression can be achieved by increasing the exposure correction amount X in advance. Can be generated.

  In addition, after determining the initial value of the exposure correction amount X when switching from normal control to control giving priority to skip suppression in the above example, the first example of exposure control shown in FIG. 10 is further combined. The exposure correction amount X may be corrected. As a result, even when the brightness of the shooting scene changes and the degree of blackout changes after the control is switched, an image with the most balanced suppression of whiteout and blackout is generated according to the shooting scene at that time. be able to.

  As described above, according to the present embodiment, the whiteout and blackout are suppressed in a well-balanced manner by adapting to each scene with high responsiveness and stability with respect to changes in the presence or absence of a high-luminance object within the shooting angle of view. Video can be provided. In addition, this invention is not limited to an above-described Example, Various modifications are included.

  For example, in the imaging apparatus shown in FIG. 1, the image signal correction unit 0103 is configured to input a video signal output from the camera signal processing unit 0102 and perform signal level correction processing. The configuration may be between the unit 0101 and the camera signal processing unit 0102. In this case, since the image signal correction unit 0103 performs processing on, for example, a RAW image signal, level correction processing with a high degree of freedom can be performed without being affected by various nonlinear signal processing performed in the camera signal processing unit. It becomes possible.

  Further, in the above-described embodiments, the description has been given by taking the whiteout as an example of the gradation collapse, but the present invention can also be applied to other cases such as a black collapse or an intermediate gradation collapse.

  Further, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described.

  Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment.

  The present invention can be used for an imaging device, a PC application connected to the imaging device, an image processing device, etc., for example, in consumer, surveillance, in-vehicle, or business use.

0101 Imaging unit 0102 Camera signal processing unit 0103 Image signal correction unit 0104 Detection unit 0105 Gradation collapse suppression control unit 0105_1 Control switching unit 0105_2 Exposure control unit 0105_3 Image signal correction control unit

Claims (8)

Imaging means;
Image signal correcting means for performing signal level correction processing on the image signal output from the imaging means and outputting a level-corrected image signal;
The degree of gradation collapse in a predetermined luminance range in the image signal output by the image pickup means is evaluated for each frame, and it is determined that there is no gradation collapse and no gradation collapse according to the evaluation result. A gradation collapse suppression control means for determining whether or not to switch control of input / output characteristics of the exposure amount of the imaging means and signal level correction processing of the image signal correction means;
With
The gradation collapse suppression control means is
By controlling the exposure amount of the image pickup means and the input / output characteristics of the signal level correction processing of the image signal correction means, the control that gives priority to the suppression of gradation collapse and the normal control that does not prioritize the suppression of gradation correction are performed.
When the normal control is performed, the evaluation value is equal to or greater than a predetermined first determination threshold based on a combination of evaluation values of the degree of gradation collapse calculated for each frame within a predetermined period. If the length of the period is equal to or higher than a predetermined first period threshold value, possess litho determines gradation collapse, and implement a control giving priority to the gradation collapse suppression,
When the control is performed with priority given to the suppression of gradation collapse, the evaluation value is set to a predetermined second value based on a combination of evaluation values of the degree of gradation collapse calculated for each frame within a predetermined period. When the length of the period that is equal to or less than the determination threshold is equal to or greater than a predetermined second period threshold, it is determined that there is no gradation collapse and the normal control is performed. apparatus. The imaging apparatus according to claim 1,
The gradation collapse suppression control means as a control giving priority to the gradation collapse suppression when assessed gradation collapse there, the exposure amount of the imaging unit as compared with a case of evaluating that there is no collapse gradation Control to increase / decrease a predetermined exposure correction amount in a direction to suppress gradation collapse,
An imaging apparatus characterized by the above. The imaging device according to claim 2,
The gradation collapse suppression control means, wherein the control giving priority to gradation collapse suppression, there is no collapse tone input and output characteristics of the signal level correction processing of the image signal correction means when assessed gradation collapse there Control so that the degree of level correction is greater than
An imaging apparatus characterized by the above. In the imaging device according to any one of claims 2 to 3,
The gradation-shattering suppression control means calculates the number of pixels within a predetermined luminance range in the image signal as an evaluation value of the degree of gradation-shattering when control that gives priority to gradation-shattering suppression is not performed. , initiating a control giving priority to the gradation collapse suppressing determines that the gradation collapse there is greater than a predetermined threshold value,
An imaging apparatus characterized by the above. The imaging device according to claim 4,
The gradation-shattering suppression control means, when performing control giving priority to gradation-shattering suppression, according to an exposure correction amount that increases or decreases the predetermined luminance range by control giving priority to gradation-shattering suppression. the corrected brightness range provided, is calculated as the evaluation value of the degree of collapse gradations the number of pixels in the luminance range that the correction in the image signal, the it is determined that there is no collapse gradation is smaller than a predetermined threshold value Ending control that prioritizes gradation loss suppression,
An imaging apparatus characterized by the above. The imaging apparatus according to claim 1,
The gradation collapse suppression control means, the method determines the presence or absence of gradation collapse, be different between the case of ending the case of starting the control giving priority to the gradation collapse suppression,
An imaging apparatus characterized by the above. The imaging apparatus according to claim 1,
The gradation collapse suppression control means evaluates the degree of gradation collapse in a second predetermined luminance range when performing control giving priority to gradation collapse suppression, and according to the evaluation, An image pickup apparatus characterized by controlling an exposure correction amount that is increased or decreased by control giving priority to the suppression of collapse. The imaging device according to claim 3,
The gradation collapse suppression control means, when starting control that prioritizes the gradation collapse suppression, performs gradation collapse in a second predetermined luminance range when the control that prioritizes gradation gradation suppression is performed in advance. Controlling the exposure correction amount to be increased or decreased by the control giving priority to the suppression of gradation collapse according to the estimation.
An imaging apparatus characterized by the above.

Images