TW201704836A - Exposure control system and method thereof - Google Patents

Abstract

An exposure control system is provided. The exposure control system comprises: an image capturing unit configured to capture a long-exposure image and a short-exposure image with a first exposure value and a second exposure value, respectively; and a processor, configured to calculate histograms of the long-exposure image and the short-exposure image, and calculate an exposure ratio according to the calculated histograms, the first and second exposure values, wherein when the exposure ratio is smaller than a first threshold, the processor switches a current exposure mode to a low dynamic range mode, wherein when the exposure ratio is larger than a second threshold, the processor switches the current exposure mode to a high dynamic range mode, wherein when the exposure ratio is between the first threshold and the second threshold, the processor does not switch the current exposure mode.

Description

Exposure control system and method thereof

The present invention relates to image processing, and more particularly to an exposure control system and method thereof for adjusting a current exposure mode in accordance with a long exposure image and a short exposure image.

The more common camera or camera on the market, the internal photosensitive element can capture a dynamic range of about 1000:1 (meaning the brightness ratio of the brightest pixel to the darkest pixel). The sensory components of professional cameras or cameras often have a dynamic range of no more than 10 ⁴ :1. However, the dynamic range of common indoor scenes is mostly over 2000:1, and outdoor scenes are higher than 10 ⁶ :1. In other words, the photosensitive element of the above camera or camera can only capture part of the information of the scene.

In general, the conventional exposure control system adjusts the average brightness of the entire picture to the brightness set by the user, but this is not good for images with dynamic images with high dynamic range. Therefore, there is a need for an exposure control system and method thereof to solve the above problems.

The invention provides an exposure control system comprising: an image The capturing unit is configured to capture a long exposure image and a short exposure image for a scene with a first exposure value and a second exposure value, and a processor for separately calculating the long exposure image and the short exposure image a histogram, and calculating an exposure ratio according to the histogram of the long exposure image and the short exposure image, the first exposure value, and the second exposure value, wherein when the exposure ratio is less than a first threshold, the processor is to expose the exposure control system One of the current exposure modes is switched to a low dynamic range mode, and the first exposure value is set equal to the second exposure value, wherein when the exposure ratio is greater than a second threshold, the processor switches the current exposure mode of the exposure control system to one a high dynamic range mode, and combining the long exposure image and the short exposure image into a high dynamic range image, and the second threshold is greater than the first threshold, wherein when the exposure ratio is between the first threshold and the second threshold, the processor Do not switch the current exposure mode.

The present invention further provides an exposure control method for an exposure control system, wherein the exposure control system includes an image capture unit, the method comprising: using the image capture unit with a first exposure value and a second exposure value Obtaining a long exposure image and a short exposure image for a scene; calculating a histogram of the long exposure image and the short exposure image, and according to the long exposure image and the histogram of the short exposure image, the first exposure value, And calculating, by the second exposure value, an exposure ratio; when the exposure ratio is less than a first threshold, switching a current exposure mode of one of the exposure control systems to a low dynamic range mode, and setting the first exposure value to be equal to the first a second exposure value; when the exposure ratio is greater than a second threshold, switching the current exposure mode of the exposure control system to a high dynamic range mode, and synthesizing the long exposure image and the short exposure image into a high dynamic range An image, wherein the second threshold is greater than the first threshold; and when the exposure ratio is between the first threshold and the second threshold, the current exposure mode is not switched

100‧‧‧Exposure Control System

110‧‧‧Image capture unit

120‧‧‧ processor

130‧‧‧ memory unit

140‧‧‧ storage unit

210-240‧‧‧ Curve

S410-S450‧‧‧Steps

1 is a block diagram showing an exposure control system 100 in accordance with an embodiment of the present invention.

2A and 2B are diagrams showing histograms for long exposure images and short exposure images, respectively, in accordance with an embodiment of the present invention.

3A and 3B are diagrams showing histogram luminance intervals for long exposure images and short exposure images, respectively, according to another embodiment of the present invention.

Figure 4 is a flow chart showing an exposure control method in accordance with an embodiment of the present invention.

The above described objects, features and advantages of the present invention will become more apparent from the description of the appended claims.

1 is a block diagram showing an exposure control system 100 in accordance with an embodiment of the present invention. As shown in FIG. 1 , the exposure control system 100 includes an image capturing unit 110 , a processor 120 , a memory unit 130 , and a storage unit 140 . The image capturing unit 110 is configured to capture an image of a scene. The storage unit 140 stores high dynamic range imaging or the like for image processing. For example, the memory unit 130 is a volatile memory such as a dynamic random access memory (DRAM). The storage unit 140 is a non-volatile memory such as a hard disk, a flash memory, or the like. The processor 120 loads the code of the high dynamic range imaging module stored in the storage unit 140 into the memory unit 130, and performs corresponding image processing on the image captured by the image capturing unit 110.

The image capturing unit 110 has a multi-exposure function, that is, the image captured by the image sensor in the image capturing unit 110 can be exposed with different exposure values to generate two An image with different exposure values, an increase in exposure time or an increase in sensitivity will increase the exposure value.

2A and 2B are diagrams showing histograms for long exposure images and short exposure images, respectively, in accordance with an embodiment of the present invention. In a scene in a particular location, its dynamic range does not change much over time. Therefore, the present invention first estimates the dynamic range of the scene by using the brightness of the first x percent dark spot and the first percent y bright spot in the initially captured image, but the ratio of the above x and y can be applied with the application. Adjusted for the difference between the sensor and the sensor. Because the dynamic range that the sensor can capture is limited, in order for the sensor to capture the brightness information of the above two points, the exposure control system 100 allows the first x percent of the dark spots to be captured by the long exposure image. . Similarly, the first y percent of the highlights will be captured by the short exposure image. Furthermore, the exposure control system 100 takes into account the continuity and convergence speed of the motion image when generating a high dynamic range image, and designs the dark point of the first X percent and the highlight of the previous y percent to fall. Within the specified range. When the exposure value converges, the exposure control system 100 can estimate the dynamic range of the scene based on the exposure ratio.

In the present embodiment, the initial long exposure time T _{L, INIT} and the initial short exposure time T _{S, INIT} are not particularly required, as long as T _{L, INIT} exposure time is greater than or equal to T _{S, INIT} . For example, in one embodiment, the initial long exposure time T _{L, INIT} may be 100 ms, the initial short exposure time T _{S , INIT} may be 51 ms. In the setting of the high dynamic range imaging module of the present invention, the number of pixels in the long exposure image that is lower than a first predetermined brightness value (for example, 10) needs to be less than one of the total number of pixels (for example, 2%, That is, the aforementioned x=2). The processor 120 first calculates, according to the histogram of the first image, the number of pixels in the histogram whose pixel brightness is less than the first predetermined brightness value.

As shown in FIG. 2A, if the histogram for the relatively lower luminance pixel in the initial long exposure image is represented by curve 210, and the luminance is lower than a first predetermined luminance value (for example, 10) in the initial long exposure image. The number of pixels is greater than the first ratio (eg 2%). In detail, in the curve 210, the number of pixels with brightness below 10 is 4%, which is greater than the first ratio (such as 2%), so the brightness in the long exposure image that is not consistent with the preset is lower than the first predetermined brightness value. The condition that the number of pixels is smaller than the first ratio (the number of pixels whose luminance is lower than 10 is only 2%).

Therefore, the processor 120 determines that there are too many pixels with low brightness in the initial long exposure image, and the curve 210 needs to be moved or expanded to the right, so that the brightness in the adjusted histogram curve 220 is lower than the first predetermined brightness value (for example, 10). The number of pixels is less than the first ratio (for example, 2%). For example, the processor 120 can increase the exposure value of the long exposure image (for example, the exposure time or the sensitivity can be adjusted), so that the number of pixels with low brightness is relatively reduced, so that the brightness in the long exposure image conforms to the preset is lower than the first. The number of pixels of the predetermined brightness value is less than the condition of the first ratio.

In other words, if the sensor of the image capturing unit 110 has a linear camera opto-electronic conversion function, when the exposure value increases, the brightness of the pixel captured by the image capturing unit 110 will also follow. Linear increase. For convenience of explanation, the above exposure value only considers the exposure time, and assumes that the number of pixels below the luminance 10 in the curve 210 is smaller than the first ratio (2%), and the number of pixels below the luminance 20 in the curve 220 is smaller than the first ratio (2) %). At this time, the processor 120 calculates the target long exposure time T _L =T _{L , INIT} *(20/10)=200 ms.

In the setting of the high dynamic range imaging module of the present invention, the number of pixels in the short exposure image that is higher than a second predetermined brightness value (for example, 1020) needs to be less than one second of the total number of pixels (for example, 2%, That is, the aforementioned y=2). The processor 120 first calculates, according to the histogram of the first image, the number of pixels whose pixel brightness in the histogram is less than the second predetermined brightness value.

As shown in FIG. 2B, if the histogram for the relatively lower luminance pixel in the initial image is represented by the curve 230, and the luminance in the initial image is higher than the second predetermined luminance value (for example, 1020), for example, 4%) is greater than the second ratio (2%). At this time, the processor 120 determines that the number of pixels with high brightness in the initial short exposure image is too large, and the curve 230 needs to be moved or expanded to the left, so that the brightness in the adjusted histogram curve 240 is higher than the second predetermined brightness value (for example, The number of pixels of 1020) is less than the second ratio (for example, 2%). For example, the processor 120 can reduce the exposure value of the short exposure image (for example, the exposure time or the sensitivity), so that the number of pixels with high brightness is relatively reduced, so that the brightness in the short exposure image is higher than the second in the preset short exposure image. The number of pixels of the predetermined brightness value is less than the condition of the second ratio.

Similarly, if the sensor of the image capturing unit 110 has a linear camera photoelectric conversion function, when the exposure value decreases, the brightness of the pixel captured by the image capturing unit 110 also decreases linearly. For convenience of explanation, the above exposure values only consider the exposure time, and it is assumed that the number of pixels having a luminance of 1020 or more in the curve 230 is smaller than the first ratio (2%), and the number of pixels having a luminance of 1000 or more in the curve 240 is smaller than the first ratio (2) %). At this time, the processor 120 calculates the target long exposure time T _S =T _{S, INIT} *(1000/1020)=50 ms.

After separately adjusting the exposure time T _L of the long exposure image and the exposure time T _S of the short exposure image, the processor 120 calculates the exposure ratio R=T _L /T according to the target long exposure time T _L and the target short exposure time T _S . _S , and can produce a high dynamic range image by adjusting the long exposure image and the short exposure image.

In practical applications, not all actual scenes must use the above high dynamic range image. For example, if the dynamic range of the actual scene is not large (low dynamic range scene), then a single image can be taken with the automatic exposure system built into the camera today. The automatic exposure system built into the camera today usually calculates the average brightness value or median of the single image after shooting, and will fall within a specific range, such as brightness 128, and then according to the relevant parameters (aperture, shutter and sensitivity). Take this single image.

The processor can determine the current dynamic range or the low dynamic range according to the exposure ratio R generated by the adjusted exposure time T _L and T _S , and then adopt a high dynamic range mode respectively (ie, through the above process to generate a high dynamic range) Image, or take a low dynamic range mode (ie direct shooting of a single image directly using the parameters of the camera's built-in auto exposure system).

In detail, if the exposure ratio R is less than or equal to a first threshold (for example, 2), it indicates that the target long exposure time T _{L is} closer to the target short exposure time Ts, and even the target long exposure time T _{L is} smaller than the target short exposure time Ts. (If the exposure ratio R is less than 1), it means that the dynamic range of the long-exposure image and the short-exposure image taken is limited, and the image capturing unit 110 uses the low dynamic range mode to cover the dynamic range in the scene at this time. Therefore, if the exposure ratio R is less than the first threshold, the processor 120 determines that the current scene is a low dynamic range scene, and enters the low dynamic range mode at this time, and then adopts the camera built-in automatic exposure system to capture a single image.

In contrast, if the exposure ratio R is greater than a second threshold (for example, 3), the second threshold is greater than or equal to the first threshold, which represents a large difference between the exposure time of the long exposure image and the short exposure image, indicating that the long exposure is taken. The dynamic range covered by the image and the short exposure image is large, so that the image capturing unit 110 needs to use the high dynamic range mode to cover the dynamic range in the current scene. Therefore, if the exposure ratio R is greater than the second threshold, the processor 120 determines that the current scene is a high dynamic range scene, and enters the high dynamic range mode at this time, and further generates a high exposure image and a short exposure image respectively by the above process. Dynamic range image.

In the actual application, since the image capturing unit 110 continuously captures the video image, although the dynamic range of the video image captured by the same scene is usually fixed, the front and back frames of the video image may be due to scene change. The histogram of the image after the capture is greatly changed. In this case, if you switch between the low dynamic range mode and the high dynamic range mode, it is likely to cause flicker or 影像 of the image. (artifact).

To solve the above problem, the processor 120 threshold threshold may set the second threshold to be greater than the first threshold, for example, set the first threshold to 2, and set the second threshold to 3, thus defining the second threshold and the first threshold. An interval. When the exposure ratio R is less than or equal to the first threshold, the processor 120 switches the current exposure mode to the low dynamic range mode. When the exposure ratio R is greater than or equal to the second threshold, the processor 120 switches the current exposure mode to the high dynamic mode. When the exposure ratio R is between the first threshold and the second threshold, the processor 120 does not switch the current exposure mode. The pattern is generated according to the previous image generation, so that the number of fast switching in the low dynamic range mode and the high dynamic range mode can be avoided. It should be noted that the first threshold and the second threshold may be adjusted according to actual conditions. In addition, the current exposure mode does not cause the image to flicker due to switching between the low dynamic range mode and the high dynamic range mode because the picture pixels are in the vicinity of a certain boundary condition or noise is generated.

It should be noted that, in view of the time sequence of the video images, the processor 120 calculates a histogram of the Nth image and calculates the target exposure time of the long exposure image and the short exposure image of the Nth image. When the image capturing unit 110 captures the N+1th image, the target exposure time of the calculated Nth image is used as the current exposure of the long exposure image and the short exposure image of the (N+1)th image. time. Similarly, the current exposure time of the long exposure image and the short exposure image of the (N+1)th image is used to calculate the target exposure time of the long exposure image and the short exposure image of the N+2 image, respectively.

3A and 3B are diagrams showing histogram luminance intervals for long exposure images and short exposure images, respectively, according to another embodiment of the present invention. In another embodiment, when the processor 120 calculates the target exposure time of the long exposure image and the short exposure image, the processor 120 separately counts the number of pixels in different luminance intervals in the histogram. As shown in FIG. 3A, when the processor 120 calculates the target exposure time of the long exposure image, the processor 120 separately counts the brightness intervals 0~10, 0~20, 0~40, 0~80, 0~160. The number of pixels accumulated, and so on. For example, if the number of pixels (for example, 4%) in the interval 0-20 is greater than the first ratio (for example, 2%), the processor 120 can also expand the luminance interval of the 3A image to the right, for example, the pixel of the luminance interval 0-20. Expand to the brightness interval 0~40 respectively. Therefore, the number of accumulated pixels below the luminance 20 can be controlled to be smaller than the first ratio. In short, in this embodiment, if the initial long exposure time T _{L, INIT} is, for example, 100 ms, the target long exposure time T _L = T _{L , INIT} * 2 = 200 ms.

As shown in FIG. 3B, when the processor 120 is calculating the target exposure time of the short exposure image, the processor 120 separately counts the number of pixels in the brightness interval 1000~1023, 950~1023, 910~1023, and so on. For example, if the number of pixels (for example, 4%) in the interval 1000~1023 is greater than the second ratio (for example, 2%), the processor 120 can also expand the luminance interval of the 3B image to the left, for example, the pixel of the brightness interval 1000~1023. Expand to the brightness interval 950~1023. Therefore, the number of pixels above 1000 in brightness can be controlled to be less than the second ratio. Similarly, the processor 120 calculates the target short exposure time according to the corresponding adjustment ratio. It should be noted that, for convenience of description, the brightness interval in the above embodiment is calculated in a geometric progression. The brightness interval of the present invention may be adjusted according to actual conditions, and is not limited to the above numerical values, and is not limited to the geometric progression. As long as the statistical stability of the strategy can be determined by exposure to achieve anti-flickering purposes.

Figure 4 is a flow chart showing an exposure control method in accordance with an embodiment of the present invention. In step S410, the image capturing unit 110 captures a long exposure image and a short exposure image for a scene by using a first exposure value and a second exposure value. It should be noted that the processor 120 updates the first exposure value and the second exposure value after each screen is processed, and the image capturing unit 110 captures the next one by using the updated first exposure value and the second exposure value. Long exposure images and short exposure images.

In step S420, the processor 120 calculates a histogram of the long exposure image and the short exposure image, and calculates an exposure ratio according to the histogram of the long exposure image and the short exposure image, the first exposure value, and the second exposure value. Exposure ratio calculation For the formula, refer to the embodiments of Figures 2A-2B and Figures 3A-3B.

In step S430, when the exposure ratio is less than a first threshold, the processor 120 switches the current exposure mode of one of the exposure control systems 100 to a low dynamic range mode. It should be noted that when the exposure ratio is less than the first threshold, it indicates that the target long exposure time is close to the target short exposure time, that is, the dynamic range of the long exposure image and the short exposure image taken at the same time is limited, and the image capture is performed at this time. Unit 110 uses the low dynamic range mode to cover the dynamic range in the scene at this time.

In step S440, when the exposure ratio is greater than a second threshold, the processor 120 switches the current exposure mode of the exposure control system 100 to a high dynamic range mode, and combines the long exposure image and the short exposure image into a high dynamic. a range image, wherein the second threshold is greater than the first threshold. It should be noted that if the calculated exposure ratio R is large, that is, the exposure time between the long exposure image and the short exposure image is large, indicating that the dynamic range of the long exposure image and the short exposure image taken at the same time is large. . At this time, the image capturing unit 110 needs to use the high dynamic range mode to cover the dynamic range in the scene at this time.

At step S450, when the exposure ratio is between the first threshold and the second threshold, the processor 120 does not switch the current exposure mode of the exposure control system 100. It should be noted that the current exposure mode of the exposure control system 100 does not cause the image pixels to be switched between the low dynamic range mode and the high dynamic range mode because the picture pixels are in the vicinity of a certain boundary condition or there is noise generation. Flashing.

The method of the present invention, or a specific type or part thereof, may be included in a physical medium such as a floppy disk, a compact disc, a hard disk, or any Other machines are readable (e.g., computer readable) storage media, wherein when the code is loaded and executed by a machine, such as a computer, the machine becomes a device or system for participating in the present invention. The method, system and apparatus of the present invention may also be transmitted in a coded form via some transmission medium, such as a wire or cable, optical fiber, or any transmission type, wherein the code is received and loaded by a machine, such as a computer. And when executed, the machine becomes a device or system for participating in the present invention. When implemented in a general purpose processor, the code in conjunction with the processor provides a unique means of operation similar to application specific logic.

The present invention has been disclosed in the above preferred embodiments, and is not intended to limit the scope of the present invention. Any one of ordinary skill in the art can make a few changes without departing from the spirit and scope of the invention. The scope of protection of the present invention is therefore defined by the scope of the appended claims.

S410-S450‧‧‧Steps

Claims (10)

An exposure control system includes: an image capturing unit that captures a long exposure image for a scene with a first exposure value, and captures a short exposure image for the scene with a second exposure value; and a processor Calculating a target long exposure time and a target short exposure time according to the long exposure image and the histogram of the short exposure image, the first exposure value, and the second exposure value, and according to the target long exposure time and the target The short exposure time calculates an exposure ratio, wherein when the exposure ratio is less than a first threshold, the processor switches the current exposure mode of one of the exposure control systems to a low dynamic range mode, wherein when the exposure ratio is greater than one When the threshold is two, the processor switches the current exposure mode of the exposure control system to a high dynamic range mode, and combines the long exposure image and the short exposure image into a high dynamic range image, and the second threshold Greater than the first threshold, wherein the processor does not switch the current exposure mode when the exposure ratio is between the first threshold and the second threshold. The exposure control system of claim 1, wherein the exposure time of one of the first exposure values is greater than or equal to one of the exposure times of the second exposure value. The exposure control system of claim 1, wherein the processor calculates a first pixel number less than a first brightness in a histogram of the long exposure image, When the number of the first pixels is greater than a first ratio, the processor further calculates a first predetermined ratio to adjust a histogram of the long exposure image, so that the adjusted histogram of the long exposure image is smaller than the first The brightness of one of the second pixels is less than the first ratio, wherein the processor further adjusts the first exposure value according to the predetermined ratio to calculate the target long exposure time. The exposure control system of claim 1, wherein the processor calculates a first pixel number in a first luminance interval in a histogram of the long exposure image, wherein the first pixel number is greater than a first ratio, the processor further calculating a first predetermined ratio to adjust a histogram of the long exposure image, such that the adjusted number of second pixels in the histogram of the long exposure image is in the first brightness interval The first ratio is smaller than the first ratio, wherein the processor further adjusts the first exposure value according to the predetermined ratio to calculate the target long exposure time. The exposure control system of claim 3, wherein the processor calculates a third pixel number greater than a second brightness in a histogram of the short exposure image, wherein the third pixel number is greater than one In a second ratio, the processor further calculates a second predetermined ratio to adjust a histogram of the short exposure image, such that the adjusted fourth image of the short exposure image is greater than the second brightness. The second ratio, The processor further adjusts the second exposure value according to the second predetermined ratio to calculate the target short exposure time. The exposure control system of claim 4, wherein the processor calculates a third pixel number in a second luminance interval in the histogram of the short exposure image, wherein the third pixel number is greater than a second ratio, the processor further calculating a second predetermined ratio to adjust a histogram of the short exposure image, such that the adjusted number of fourth pixels in the histogram of the short exposure image is in the second brightness interval Less than the second ratio, wherein the processor further adjusts the second exposure value according to the second predetermined ratio to calculate the target short exposure time. An exposure control method for an exposure control system, wherein the exposure control system includes an image capture unit, the method comprising: using the image capture unit to capture a long exposure image of a scene with a first exposure value and a second exposure value captures a short exposure image of the scene; calculating a target long exposure time and a target according to the long exposure image and the histogram of the short exposure image, the first exposure value, and the second exposure value a short exposure time, and calculating an exposure ratio according to the target long exposure time and the target short exposure time; when the exposure ratio is less than a first threshold, switching one of the exposure control systems to a low dynamic range mode When the exposure ratio is greater than a second threshold, switching the current exposure mode of the exposure control system to a high dynamic range mode, and the length The exposure image and the short exposure image are combined into a high dynamic range image, wherein the second threshold is greater than the first threshold; and when the exposure ratio is between the first threshold and the second threshold, the current is not switched Exposure mode. The method of claim 7, wherein the exposure time of one of the first exposure values is greater than or equal to one of the exposure times of the second exposure value. The method of claim 7, further comprising: calculating a number of first pixels in the histogram of the long exposure image that is less than a first brightness or in a first brightness interval; when the first number of pixels When the ratio is greater than a first ratio, a first predetermined ratio is calculated to adjust a histogram of the long exposure image, such that the adjusted histogram of the long exposure image is smaller than the first brightness or is in the first brightness interval. The number of two pixels is less than the first ratio; and the first exposure value is adjusted according to the predetermined ratio to calculate the target long exposure time. The method of claim 7, further comprising: calculating a third pixel number greater than a second brightness or a second brightness interval in a histogram of the short exposure image; When the second ratio is greater than a second ratio, a second predetermined ratio is calculated to adjust the histogram of the short exposure image, such that the adjusted histogram of the short exposure image is greater than the second brightness or is in the second brightness interval. The number of four pixels is less than the second ratio; The second exposure value is adjusted according to the second predetermined ratio to calculate the target short exposure time.