JP2009038814A - Imaging system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging system which obtains a stable gradation conversion picture by performing gradation conversion while automatically and optimally controlling calculation timing of a gradation conversion curve in accordance with a scene change or a photographing condition change without being affected by a screen position or a slight screen change of a subject. <P>SOLUTION: An imaging system includes: selection means 113 for selecting one image at time intervals based on the quantity of information and a scene change, from an image group of an M-bit gradation width from an imaging system; detection means 114 for detecting a change in characteristic information between selected images; calculation means 115 for calculating a gradation conversion curve of the selected image on the basis of a result of the detection; and gradation conversion means 110 for gradation conversion upon the image group of the M-bit gradation width into an image group of an N-bit gradation width (M, N are natural numbers, M≥N), on the basis of a detection result in the detection means 114, while using a novel gradation conversion curve calculated by the calculation means 115 when a change in the characteristic information is detected, or using a current gradation conversion curve when the change in the characteristic information is not detected. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging system in which the gradation width of an imaging system that obtains images in a time series is wider than the gradation width in the output system, and appropriately updates the gradation conversion curve that appropriately converts the gradation width. The present invention relates to an imaging system that obtains a high-quality output image.

  For example, in a digital video camera, in order to prevent deterioration in image quality due to digit loss in digital signal processing, the gradation width of the image in the input and processing system is set to the gradation width of the final output image (usually 8 bits). More generally, it is set to about 10 to 12 bits. In this case, it is necessary to perform gradation conversion so as to match the gradation width of the output system. There has also been proposed a method of generating a wide dynamic range image with a wider gradation width by combining a plurality of images having different exposure amounts. In this case as well, it is necessary to perform gradation conversion so that the obtained wide dynamic range image matches the gradation width of the output system.

  As a method of performing the above-described gradation conversion, conventionally, gradation conversion is performed using a fixed gamma curve adapted to a standard scene, an applicable gradation conversion curve based on a histogram, or the like. As disclosed, a method has been proposed in which an image is captured at regular intervals, and gradation conversion is performed based on luminance statistics of a specific region (for example, the center of the image) of the captured image.

JP 2000-307896 A

  However, when the gradation conversion is fixedly performed by gamma conversion or the like, appropriate gradation conversion cannot be performed according to the changing scene, and the gradation conversion is applied based on a histogram or the like. When the conversion curve is used, there is a problem that appropriate gradation conversion cannot be performed unless the gradation conversion curve is calculated at an appropriate timing in accordance with a scene change at the time of shooting.

  Further, as disclosed in Patent Document 1, when an image is captured at regular intervals and tone conversion is performed based on the luminance statistics of the specific area, a tone conversion curve is calculated without scene change. Because even if it is not necessary to calculate, the calculation is wasteful, and even if a scene change occurs, the gradation conversion curve is not calculated if the timing is not correct, so appropriate gradation conversion according to the scene change There is a problem that can not be done.

  In addition, since a specific area such as the center of the image is set, there is a problem that it is not possible to cope with detection of scene changes in other areas, and the gradation conversion curve is regularly updated, so the screen is stable. There is also the problem of impairing sex.

  Recently, video distribution via a network has also been carried out. In this case, the image size, frame rate, etc. will change depending on the delivery conditions. However, the above-described conventional technology automatically adjusts the timing for calculating the gradation conversion curve in response to changes in the image size and frame rate. Therefore, there is a problem that appropriate gradation conversion cannot be performed.

  Accordingly, an object of the present invention made in view of such a point is that the timing for calculating the gradation conversion curve for converting the gradation width of the image obtained in time series is influenced by the screen position of the subject and the minute screen change. It is an object of the present invention to provide an imaging system that can automatically and optimally adjust according to scene changes and changes in shooting conditions, always perform appropriate gradation conversion, and obtain a stable gradation conversion screen.

The invention of the imaging system according to claim 1 that achieves the above object is characterized in that an image group having continuous M-bit gradation width from the imaging system in time series is represented by N-bit gradation width (M and N are natural numbers M In an imaging system that converts to an image group of ≧ N) and outputs it to a subsequent processing system,
From the image group having the M-bit gradation width, an information amount is obtained from the image size and the number of frames of the image group photographed per unit time by the imaging system so that the load on the subsequent processing system does not increase. Selecting means for selecting one image at long time intervals as the amount of information increases;
Detecting means for detecting a change in characteristic information between images sequentially selected by the selecting means;
Calculation means for calculating a gradation conversion curve of the image selected by the selection means based on the detection result of the detection means;
Based on the detection result of the detection means, when a change in the characteristic information is detected, a new gradation conversion curve calculated by the calculation means is used. Gradation conversion means for converting the image group having the M-bit gradation width into the image group having the N-bit gradation width, respectively, using the gradation conversion curve of
The selection means is characterized in that an image is selected at a time interval of 0 when the detection means detects a scene change due to a change in specific information.

Furthermore, the invention of the imaging system according to claim 2 which achieves the above object is characterized in that an image group having continuous M-bit gradation width from the imaging system in an N-bit gradation width (M and N are natural numbers) of the output system. In an imaging system that converts the image group to M ≧ N) and outputs the image group to a subsequent processing system,
From the image group having the M-bit gradation width, an information amount is obtained from the image size and the number of frames of the image group photographed per unit time by the imaging system so that the load on the subsequent processing system does not increase. Selecting means for selecting one image at long time intervals as the amount of information increases;
Detecting means for detecting a change in characteristic information between images sequentially selected by the selecting means;
Calculation means for selectively calculating a gradation conversion curve of the image selected by the selection means based on the detection result of the detection means;
Recording means for recording a plurality of sets of characteristic information of the image selected by the selection means and the gradation conversion curves calculated by the calculation means for the image;
When a change in the characteristic information is detected based on the detection result of the detection means, a new gradation conversion curve calculated by the calculation means or a past gradation conversion recorded in the recording means Tone conversion means for converting the image group having the M-bit gradation width into the image group having the N-bit gradation width using the current gradation conversion curve when no change in the characteristic information is detected using a curve And having
The selection means is characterized in that an image is selected at a time interval of 0 when the detection means detects a scene change due to a change in specific information.

The invention according to claim 3 is the imaging system according to claim 1 or 2,
The detecting means is
As the characteristic information, luminance calculation means for calculating an average luminance level of the image selected by the selection means,
Determining means for determining the presence or absence of a scene change based on a change in average brightness level between sequential images calculated by the brightness calculating means;
It is characterized by having.

The invention according to claim 4 is the imaging system according to claim 1 or 2,
The detecting means is
As the characteristic information, a motion vector calculation unit that calculates a motion vector from two sequential images selected by the selection unit;
Determining means for determining the presence or absence of a scene change based on the motion vector calculated by the motion vector calculating means;
It is characterized by having.

The invention according to claim 5 is the imaging system according to claim 1 or 2,
The detecting means is
Histogram calculation means for calculating a histogram from the image selected by the selection means as the characteristic information;
Determining means for determining the presence or absence of a scene change based on a change in histogram between sequential images calculated by the histogram calculating means;
It is characterized by having.

The invention according to claim 6 is the imaging system according to claim 2,
The detecting means is
Similarity calculating means for calculating the similarity between the characteristic information of the image selected by the selecting means and a plurality of characteristic information recorded in the recording means;
When the similarity calculated by the similarity calculating unit satisfies a predetermined condition, a reading unit that reads out from the recording unit a gradation conversion curve that forms a pair with the similar characteristic information;
It is characterized by having.

The invention according to claim 7 is the imaging system according to claim 1 or 2,
The detecting means is
As a change in the characteristic information, a photographing condition detecting unit that detects a photographing condition of an exposure condition or a focusing condition of the image selected by the selecting unit;
Determination means for determining the presence or absence of a scene change based on a change in shooting conditions between sequential images detected by the shooting condition detection means;
It is characterized by having.

  According to the first aspect of the present invention, when the selection unit extracts an image at an appropriate timing based on the information amount and the scene change, and when the detection unit detects a change in the characteristic information of the extracted sequential images. The gradation conversion curve is calculated by the calculation means based on the image extracted by the selection means, and an image group having an M-bit gradation width is set to N bits by the gradation conversion means using the newly calculated gradation conversion curve. When converted to an image group of gradation width and no change in the characteristic information is detected by the detection means, the image group of M bit gradation width is N-bit gradation width on the gradation conversion curve currently used in the gradation conversion means. Since it is converted into an image group, a tone conversion curve can be set automatically at an appropriate timing in accordance with a scene change without unnecessary calculation, and accordingly, the tone change is always appropriately performed with respect to the scene change. Can, it is possible to obtain a stable gradation conversion screen with high quality.

  According to the second aspect of the present invention, the selection unit extracts an image at an appropriate timing based on the information amount and the scene change, the change in the characteristic information of the extracted sequential images is detected by the detection unit, and the When the characteristic information corresponding to the characteristic information of the image is not recorded in the recording means, the gradation conversion curve is calculated by the calculation means based on the image extracted by the selection means, and the newly calculated gradation The conversion curve and the characteristic information of the image that characterizes the scene are recorded as a set in the recording unit, and an image group having an M-bit gradation width is formed by the gradation conversion unit using the newly calculated gradation conversion curve. It is converted into an image group having an N-bit gradation width. Further, when the change of the characteristic information is detected by the detecting means and the characteristic information corresponding to the characteristic information is recorded in the recording means, a gradation conversion curve corresponding to the characteristic information recorded in the recording means is obtained. When the gradation conversion unit converts the image group having the M-bit gradation width into the image group having the N-bit gradation width, and the change of the characteristic information is not detected by the detection unit, the gradation conversion curve is calculated by the calculation unit. Instead, the image group having the M-bit gradation width is converted into the image group having the N-bit gradation width by the gradation conversion curve currently used in the gradation converting means. Therefore, a tone conversion curve can be set automatically at an appropriate timing according to the scene change, and a tone conversion curve can be quickly and appropriately converted to a scene change without unnecessary calculation, and stable with high quality. A gradation conversion screen can be obtained.

  According to the third aspect of the present invention, since the scene change is detected from the time-series change in the luminance value of the selected image, the scene change can be detected with a small amount of calculation and a low-cost processing system.

  According to the invention of claim 4, since a scene change is detected based on a time-series change of a motion vector, gradation conversion with emphasis on the main subject becomes possible, and a high-quality image can be obtained. .

  According to the invention of claim 5, since the scene change is detected from the time-series change of the histogram, the scene change can be detected with a relatively small amount of calculation and a low-cost processing system. Further, by placing emphasis on the intermediate gradation excluding the dark part and the bright part, the scene change detection ability can be improved.

  According to the invention of claim 6, when a scene change is detected, the similarity with the past characteristic information recorded in the recording means is obtained, and when the similarity is high, the past characteristic information Since gradation conversion is performed using past gradation conversion curves recorded in pairs, it is possible to save the trouble of calculating gradation conversion curves in the case of similar scenes. Is possible.

  According to the seventh aspect of the present invention, since a scene change is detected based on a change in the shooting condition of the image exposure condition or the focusing condition, gradation conversion is automatically performed at an appropriate timing according to the change of the shooting condition. A curve can be set.

  Hereinafter, an embodiment of an imaging system according to the present invention will be described with reference to FIGS.

  1 to 6 show a first embodiment of the present invention. FIG. 1 is a block diagram showing a configuration of the entire imaging system. FIG. 2 is a block diagram showing an example of the configuration of a selection unit shown in FIG. FIG. 3 is an explanatory diagram of a time setting function in the selection unit, FIG. 4 is a block diagram illustrating a configuration of an example of the detection unit illustrated in FIG. 1, and FIG. 5 is a block diagram illustrating a configuration of an example of the calculation unit illustrated in FIG. FIG. 6 is a block diagram showing a configuration of an example of the gradation converting unit shown in FIG.

  In the imaging system shown in FIG. 1, an image such as a moving image is captured by an imaging system having a lens system 100, an aperture 101, a low-pass filter 102, and a single-plate CCD 103, and an analog video signal output from the CCD 103 of this imaging system is captured. The digital signal is converted by the A / D converter 104 and transferred to the photometric evaluation unit 106, the in-focus detection unit 107 and the signal processing unit 109 via the image buffer 105.

  The photometric evaluation unit 106 is connected to the diaphragm 101 and the CCD 103, and the focal point detection unit 107 is connected to the AF motor 108. The signal processing unit 109 is connected to an output unit 112 such as a monitor or a video recorder via a gradation conversion unit (gradation conversion unit) 110 and a D / A converter 111, and a selection unit (selection unit) 113. Then, it connects with the detection part (detection means) 114 and the calculation part (calculation means) 115, respectively. Further, the gradation conversion unit 110 is connected with a calculation unit 115 and a standard gradation curve ROM (standard gradation curve recording means) 116, respectively.

  The photometric evaluation unit 106, the in-focus detection unit 107, the signal processing unit 109, the gradation conversion unit 110, the output unit 112, the selection unit 113, the detection unit 114, and the calculation unit 115 are further added to the control unit 117 such as a microcomputer. The controller 117 is further connected to an external I / F unit 118 having a power switch, a shutter button, and an interface for switching various modes during photographing. Here, the control unit 117 constitutes a control means for controlling the entire operation, and also constitutes an initialization detection means, a photographing condition detection means, and a reset means. 1, 2, and 4 to 6, a signal line indicated by a bold solid line indicates a video signal line, a signal line indicated by a thin solid line indicates a control signal line, and a signal line indicated by a broken line indicates the others. The data line is shown.

  In the imaging system shown in FIG. 1, the user designates imaging conditions such as the image size and the number of frames via the external I / F unit 118, and then starts imaging by pressing the shutter button. The video signal obtained from the CCD 103 upon the start of photographing is converted into a digital signal by the A / D converter 104, transferred to the image buffer 105, and stored. In the present embodiment, the gradation width of the digitized signal is, for example, 12 bits.

  The video signal transferred to the image buffer 105 is transferred to the photometric evaluation unit 106 and the in-focus detection unit 107, respectively. The photometric evaluation unit 106 calculates a luminance level (exposure condition) in the image from the video signal, and controls the electronic shutter speed of the diaphragm 101 and the CCD 103 so as to achieve appropriate exposure based on the luminance level. The focus detection unit 107 detects the edge strength (focusing condition) in the image from the video signal, and controls the AF motor 108 so that the edge strength is maximized, that is, the focused image is obtained. To do. The exposure conditions calculated by the photometric evaluation unit 106 and the shooting conditions such as the focus condition detected by the focus detection unit 107 are transferred to the control unit 117.

  The signal processing unit 109 reads a single-panel video signal on the image buffer 105 under the control of the control unit 117, and performs known interpolation processing, white balance processing, enhancement processing, and the like based on the video signal. The three-plate video signal is generated, and the generated video signal is selected by the selection unit 113 at predetermined time intervals and transferred to the detection unit 114.

  The detection unit 114 calculates predetermined characteristic information from the selected video signal (image), compares the calculated characteristic information with characteristic information that has already been calculated from the previously selected image, and determines the scene change. The presence or absence is detected, and the result is transferred to the control unit 117.

  The control unit 117 calculates a new gradation conversion curve based on the image selected by the selection unit 113 in the calculation unit 115 when a scene change is detected based on the signal from the detection unit 114. Control to transfer to the gradation conversion unit 110.

  On the other hand, when no scene change is detected, the tone conversion unit 110 continues to use the tone conversion curve currently used without calculating a new tone conversion curve. To control. When the control unit 117 detects an initialization state such as when the power is turned on, the gradation conversion unit 110 controls the gradation conversion curve stored in the standard gradation curve ROM 116 to be read.

  The tone conversion unit 110 converts the signal from the signal processing unit 109 to match the tone width of the output system based on the tone conversion curve obtained as described above. In this embodiment, the gradation width of the output system is, for example, 8 bits. The digital video signal output from the gradation conversion unit 110 is converted into an analog video signal by the D / A converter 111 and output to the output unit 112 such as a monitor or a video recorder.

  Next, detailed configurations and operations of the respective units of the selection unit 113, the detection unit 114, the calculation unit 115, and the gradation conversion unit 110 will be described.

  FIG. 2 is a block diagram illustrating an exemplary configuration of the selection unit 113. The selection unit 113 includes a time setting unit (time setting unit) 200, a function ROM 201, an image reading unit 202, a reduction unit (reduction unit) 203, and a reduced image buffer 204. The time setting unit 200 is connected to the control unit 117, the function ROM 201, and the image reading unit 202. The image reading unit 202 is further connected to the control unit 117, the signal processing unit 109, and the reduction unit 203. The reduction unit 203 is Further, the reduced image buffer 204 is further connected to the detection unit 114 and the calculation unit 115, respectively.

  Information from the photometric evaluation unit 106, the in-focus detection unit 107, and the external I / F unit 118 is transferred to the time setting unit 200 via the control unit 117. The time setting unit 200 determines a time interval for selecting an image from a function set in the function ROM 201 based on information such as the image size and the number of frames obtained from the external I / F unit 118.

  FIG. 3 shows an example of a time setting function set in the function ROM 201. In this time setting function, the time interval is set longer as the image size or the number of frames increases so that the load on the processing system in the subsequent stage does not increase.

  If the number of frames is 0, it means still image shooting. In this case, the time interval is 0, and all images are selected. Also, the time setting unit 200 determines that a scene change has occurred when the photometry and focusing conditions from the photometry evaluation unit 106 and the focus detection unit 107 change suddenly, and sets the time interval to zero. In addition, it is also possible to detect a scene change using information such as white balance, lens zoom position, camera movement, and the like. Furthermore, if the scene change amount exceeds a predetermined value and there is a high possibility that the image will fail in the current gradation conversion curve, the standard gradation curve ROM 116 is not waiting for calculation of a new gradation conversion curve. It is also possible to switch to a tone conversion curve. Also, it is possible to expand such as preparing a plurality of the above time setting functions and switching them according to the purpose.

  The image reading unit 202 reads a signal from the signal processing unit 109 at a predetermined time interval based on control by the time setting unit 200 and transfers the signal to the reduction unit 203. The reduction unit 203 reduces the image at a predetermined reduction rate, for example, 1/8, and transfers the image to the reduced image buffer 204 for storage. Note that the reduction ratio in the reduction unit 203 need not be fixed, but may be variable. For example, the amount of information within a unit time can be obtained from the reduced image size and the selected time interval, and the reduction rate can be controlled so that this amount of information is not more than a certain value.

  The reduced image buffer 204 is a ring-shaped buffer capable of recording a plurality of reduced images, and is configured to overwrite an old image when the buffer is full. The reduced image in the reduced image buffer 204 is transferred to the detection unit 114 or the calculation unit 115.

  FIG. 4 is a block diagram illustrating an exemplary configuration of the detection unit 114. The detection unit 114 includes a luminance calculation unit (luminance calculation unit) 300, a luminance value buffer 301, and a determination unit (determination unit) 302. The luminance calculation unit 300 is connected to the control unit 117 and the selection unit 113, and is connected to the determination unit 302 via the luminance value buffer 301. The determination unit 302 is further connected to the control unit 117.

  The luminance calculation unit 300 takes in the reduced image from the selection unit 113 based on the control by the control unit 117, and calculates the average luminance value of the reduced image. The average luminance value is transferred to the luminance value buffer 301 and stored. The luminance value buffer 301 is a ring-shaped buffer that records the average luminance value, and is configured to overwrite the old average luminance value when the buffer is full.

  The determination unit 302 monitors a time-series change in the average luminance value from the luminance value buffer 301, and determines that a scene change has occurred when a change equal to or greater than a predetermined threshold has occurred continuously a predetermined number of times. This determination result is transferred to the control unit 117.

  FIG. 5 is a block diagram illustrating an exemplary configuration of the calculation unit 115. The calculation unit 115 includes a luminance calculation unit (separation unit) 400, a proper exposure extraction unit (extraction unit) 401, an edge extraction unit (feature amount calculation unit) 402, a histogram generation unit (histogram generation unit) 403, and a conversion curve calculation. Part (gradation conversion curve calculation means) 404. The luminance calculation unit 400 is connected to the control unit 117 and the selection unit 113, connected to the edge extraction unit 402 via the appropriate exposure extraction unit 401, and connected to the conversion curve calculation unit 404 via the histogram creation unit 403. To do. The appropriate exposure extraction unit 401 is further connected to the control unit 117, the histogram creation unit 403 is further connected to the appropriate exposure extraction unit 401 and the edge extraction unit 402, and the conversion curve calculation unit 404 is further connected to the control unit 117 and the floor. Each of them is connected to the tone conversion unit 110.

  When the scene change is detected by the detection unit 114 under the control of the control unit 117, the luminance calculation unit 400 takes in the reduced image from the selection unit 113 and calculates the luminance. The luminance signal calculated by the luminance calculation unit 400 is compared with a predetermined threshold value related to the dark part and the bright part by the appropriate exposure extraction unit 401 (for example, for a 12-bit gradation, the dark part is “128” and the bright part is “3968”). A luminance signal that is not less than the dark portion threshold value and not more than the bright portion threshold value is extracted as an appropriate exposure area and transferred to the edge extracting unit 402 and the histogram creating unit 403, respectively.

  The edge extraction unit 402 performs known edge detection on the luminance signal from the appropriate exposure extraction unit 401, extracts pixels having edge strengths equal to or greater than a predetermined threshold as edge portions, and sends the information to the histogram creation unit 403. Forward.

  The histogram creation unit 403 creates a histogram of the luminance signal at the edge portion in the appropriate exposure area from the luminance signal from the luminance calculation section 400 based on the information on the appropriate exposure area and the information on the edge part. Transfer to the conversion curve calculation unit 404.

  The conversion curve calculation unit 404 obtains a gradation conversion curve by accumulating the histogram from the histogram creation unit 403 and transfers it to the gradation conversion unit 110.

  FIG. 6 is a block diagram illustrating an exemplary configuration of the gradation conversion unit 110. The gradation conversion unit 110 includes a Y / C separation unit 500, a luminance signal buffer 501, a color difference signal buffer 502, a luminance correction unit 503, a gradation conversion curve reading unit (transfer means) 504, a correction luminance signal buffer 505, and a color difference correction. A unit 506 and a Y / C synthesis unit 507.

  The Y / C separation unit 500 is connected to the signal processing unit 109 and to the luminance signal buffer 501 and the color difference signal buffer 502, respectively. The luminance signal buffer 501 is further connected to the color difference correction unit 506 and is connected to the Y / C synthesis unit 507 via the luminance correction unit 503 and the corrected luminance signal buffer 505. The color difference signal buffer 502 is further connected to the Y / C synthesis unit 507 via the color difference correction unit 506. The corrected luminance signal buffer 505 is further connected to the color difference correction unit 506, and the Y / C synthesis unit 507 is further connected to the D / A converter 111. Further, the gradation conversion curve reading unit 504 is connected to the calculation unit 115 and the standard gradation curve ROM 116 and is also connected to the luminance correction unit 503. Further, the Y / C separation unit 500, the luminance correction unit 503, the gradation conversion curve reading unit 504, the color difference correction unit 506, and the Y / C composition unit 507 are connected to the control unit 117, respectively.

  The video signal from the signal processing unit 109 is separated into a luminance signal and a color difference signal by the Y / C separation unit 500, and the luminance signal is stored in the luminance signal buffer 501 and the color difference signal is stored in the color difference signal buffer 502. In addition, the gradation conversion curve reading unit 504, when the initialization state such as power-on is detected under the control of the control unit 117, the standard gradation stored in the standard gradation curve ROM 116. When a conversion curve is read and a scene change is detected by the detection unit 114, a new gradation conversion curve from the calculation unit 115 is read.

  The luminance signal stored in the luminance signal buffer 501 is transferred to the luminance correction unit 503 and the color difference correction unit 506, respectively. The luminance correction unit 503 converts the luminance signal from the luminance signal buffer 501 to the gradation width of the output system based on a predetermined gradation conversion curve read by the gradation conversion curve reading unit 504, which is 8 bits in this embodiment. The converted luminance signal is transferred to the Y / C synthesis unit 507 and the color difference correction unit 506 via the corrected luminance signal buffer 505, respectively.

  Further, the color difference signal stored in the color difference signal buffer 502 is transferred to the color difference correction unit 506. The color difference correction unit 506 calculates a correction coefficient for correcting the color difference signal based on the luminance signal before and after gradation conversion from the luminance signal buffer 501 and the corrected luminance signal buffer 505 and the theoretical limit model in which a color can exist. The color difference signal from the color difference signal buffer 502 is corrected by the calculated correction coefficient, and the corrected color difference signal is transferred to the Y / C synthesis unit 507.

  The Y / C combining unit 507 combines the luminance signal after gradation conversion from the corrected luminance signal buffer 505 and the corrected color difference signal from the color difference correcting unit 506, and the combined video signal is D / A. Transfer to converter 111.

  As described above, in the present embodiment, with respect to an image captured in time series, the selection unit 113 selects an image at a time interval suitable for the imaging conditions such as the image size and the number of frames, and the selected image is selected. Based on the characteristics of the image, the detection unit 114 detects the presence or absence of a scene change. When a scene change is detected from the image itself or its photometric information or focus information, the calculation unit 115 calculates a new gradation conversion curve. When a scene change is detected, the tone conversion unit uses the tone conversion curve newly calculated by the calculation unit 115. When no scene change is detected, the tone conversion unit uses the current tone conversion curve. Since the gradation characteristics of each image are converted at 110, a gradation conversion curve can be calculated at an appropriate timing, and a high-quality image can be obtained. In addition, since the image is reduced when detecting a scene change from the selected image, the processing system can be reduced in size and cost, and a minute change in the image can be absorbed by the reduction process. Therefore, a stable image can be obtained. In addition, since the scene change is detected not from the specific area but from the entire image, the scene change can be detected regardless of the subject position. In addition, since the standard gradation curve ROM 116 for storing the standard gradation curve is provided, it is possible to cope with the initialization situation.

  FIGS. 7 to 10 show a second embodiment of the present invention, FIG. 7 is a block diagram showing a configuration of the entire imaging system, and FIG. 8 is a block diagram showing an example of the configuration of the detection unit shown in FIG. FIG. 9 is a diagram for explaining calculation of histogram similarity in the detection unit shown in FIG. 8, and FIG. 10 is a block diagram showing a configuration of an example of the recording unit shown in FIG.

  The imaging system according to the present embodiment is different from the imaging system according to the first embodiment in that a first image buffer 600, a second image buffer 601 and a synthesizing unit (synthesizing unit) 602 are provided instead of the image buffer 105. The recording unit 603 is added to the detection unit 114, the calculation unit 115, the control unit 117, and the gradation conversion unit 110, and the detection unit 114 has a configuration different from that of the first embodiment. Since the other configuration is the same as that of the first embodiment, the same reference numerals are given to components performing the same operation, and detailed description thereof will be omitted.

  In FIG. 7, the first image buffer 600 and the second image buffer 601 are connected to the A / D converter 104 and connected to the signal processing unit 109 via the synthesis unit 602, and the first image buffer 600 is connected to the A / D converter 104. Further, the photometric evaluation unit 106 and the in-focus detection unit 107 are connected. 7, 8, and 10, a signal line indicated by a thick solid line indicates a video signal line, a signal line indicated by a thin solid line indicates a control signal line, and a signal line indicated by a broken line indicates other data lines. Is shown.

  In the imaging system shown in FIG. 7, the user designates imaging conditions such as the image size and the number of frames via the external I / F unit 118, and then starts imaging by pressing the shutter button. The video signal obtained from the CCD 103 upon the start of photographing is converted into a digital signal by the A / D converter 104, transferred to the first image buffer 600 and stored as a long exposure image. In the present embodiment, the gradation width of the digitized signal is, for example, 12 bits as in the first embodiment. The video signal stored in the first image buffer 600 is transferred to the photometric evaluation unit 106 and the in-focus detection unit 107, and the exposure condition and the in-focus condition are calculated.

  Next, a second image is taken under an exposure condition that is a predetermined exposure ratio, for example, 1/8 with respect to the exposure condition calculated by the photometric evaluation unit 106, and the video signal is converted into an A / D converter. The digital signal is converted at 104 and transferred to the second image buffer 601 and stored as a short exposure image.

  The compositing unit 602 sequentially reads the long-exposure image stored in the first image buffer 600 and the short-exposure image stored in the second image buffer 601. First, a predetermined threshold ( For example, in the case of 12-bit gradation, “3890”) is left as an appropriate exposure area, and then a short-time exposure image corresponding to an area other than the appropriate exposure area is read and the exposure ratio is corrected (for example, When the short-time exposure is set to an exposure ratio of 1/8 with respect to the long-time exposure, it is corrected to 8 times), and the image is combined with the image of the appropriate exposure area for long-time exposure. The video signal synthesized by the synthesis unit 602 is transferred to the signal processing unit 109, where it is processed in the same manner as in the first embodiment to generate a three-plate video signal.

  The video signal generated by the signal processing unit 109 is selected at a predetermined time interval by the selection unit 113 and transferred to the detection unit 114 as in the first embodiment.

  As in the first embodiment, the detection unit 114 calculates predetermined characteristic information from the selected video signal (image), and has already been calculated from the calculated characteristic information and the previously selected image. The presence / absence of a scene change is detected by comparing with the characteristic information. If a scene change is detected here, similar characteristic information is recorded with reference to past characteristic information recorded in the recording unit 603. It is determined whether or not it has been performed, and those results are transferred to the control unit 117.

  When a scene change is detected based on the signal from the detection unit 114 and there is characteristic information similar to the recording unit 603, the control unit 117 calculates a gradation conversion curve corresponding to the similar characteristic information. The gradation conversion unit 110 is controlled to read from the recording unit 603. Further, when a scene change is detected and there is no characteristic information similar to the recording unit 603, the calculation unit 115 calculates a new gradation conversion curve from the image selected by the selection unit 113 and performs gradation conversion. At the same time, the new gradation conversion curve calculated by the calculation unit 115 and the image characteristic information calculated by the detection unit 114 are transferred to the recording unit 603 and stored as a set. To do.

  On the other hand, if no scene change is detected, the tone conversion curve of the currently used tone conversion curve is not calculated by the calculation unit 115, as in the first embodiment. The gradation conversion unit 110 is controlled so as to continue use. When the control unit 117 detects an initialization state such as when the power is turned on, the gradation conversion unit 110 controls the gradation conversion curve stored in the standard gradation curve ROM 116 to be read.

  The tone conversion unit 110 converts the signal from the signal processing unit 109 to match the tone width of the output system based on the tone conversion curve obtained as described above. In this embodiment, the gradation width of the output system is, for example, 8 bits. The digital video signal output from the gradation conversion unit 110 is converted into an analog video signal by the D / A converter 111 and output to the output unit 112 such as a monitor or a video recorder.

  FIG. 8 is a block diagram illustrating an exemplary configuration of the detection unit 114 illustrated in FIG. 7. The detection unit 114 includes a histogram calculation unit (histogram calculation unit) 700, a histogram buffer 701, a determination unit (determination unit) 702, a similarity calculation unit (similarity calculation unit) 703, and a gradation conversion curve designation unit (readout). Means) 704. The histogram calculation unit 700, the determination unit 702, the similarity calculation unit 703, and the gradation conversion curve designation unit 704 are connected to the control unit 117, respectively. In addition, the histogram calculation unit 700 is further connected to the selection unit 113 and is also connected to the recording unit 603, the determination unit 702, and the similarity calculation unit 703 via the histogram buffer 701. The determination unit 702 is further connected to the similarity calculation unit 703, the similarity calculation unit 703 is further connected to the recording unit 603 and the gradation conversion curve designation unit 704, and the gradation conversion curve designation unit 704 is further connected to the recording unit. Connect to 603.

  In the detection unit 114 illustrated in FIG. 8, the histogram calculation unit 700 takes in a reduced image from the selection unit 113 based on the control of the control unit 117 and calculates a histogram of the reduced image. This histogram is transferred to the histogram buffer 701 and stored. The histogram buffer 701 is a ring-shaped buffer capable of recording a histogram, and is configured to overwrite an old histogram when the buffer is full.

  The determination unit 702 monitors the change in the shape of the histogram from the histogram buffer 701, determines that a change in shape greater than or equal to a predetermined value has occurred, and determines that a scene change has occurred when the situation continues for a predetermined number of times. The determination result is transferred to the control unit 117.

  If it is determined that a scene change has occurred, the control unit 117 controls the similarity calculation unit 703 to record the histogram on the histogram buffer 701 that has been determined that the scene change has occurred and the recording unit 603. The degree of similarity with the histogram is calculated.

  FIG. 9 is a diagram for explaining the similarity calculation method. FIG. 9A shows a histogram on the histogram buffer 701 and a histogram having a predetermined allowable range for the histogram, for example, a histogram having a range of ± 20%. FIG. 9B shows a past histogram to be compared recorded in the recording unit 603, and FIG. 9C shows a histogram on the histogram buffer 701 for a certain gradation value and its histogram. An allowable range and a past histogram value to be compared on the recording unit 603 are shown.

  The similarity calculation unit 703 sequentially determines whether the past histogram value on the recording unit 603 is included in the allowable range while changing the gradation value. Here, when the total number of gradation values included in the allowable range is a predetermined ratio or more, for example, 70% or more with respect to all gradation values, it is determined that the histograms of both are high in similarity, otherwise In this case, it is determined that the degree of similarity is low, and the result is transferred to the control unit 117. It should be noted that the allowable range used in determining the similarity of the histogram does not need to be constant for all gradations. For example, it is set so that it is wide or neglected in a dark part and a bright part, and narrow in a halftone. It is also possible to do. In this case, the criteria for determination in the halftone where the main subject is likely to be strict, and a tone conversion curve having a higher similarity is selected.

  If it is determined that the similarity is high, the control unit 117 controls the gradation conversion curve designating unit 704 to select a gradation conversion curve paired with the histogram from the recording unit 603. The information is transferred to the control unit 117. On the other hand, when it is determined that the similarity is low with respect to all the histograms recorded in the recording unit 603, the control unit 117 causes the calculation unit 115 to calculate a new gradation conversion curve. Control.

  FIG. 10 is a block diagram illustrating a configuration example of the recording unit 603 illustrated in FIG. The recording unit 603 includes a characteristic recording unit 800, a gradation conversion curve recording unit 801, a monitoring unit (monitoring unit) 802, a history management unit (history management unit) 803, and a discard control unit (discarding unit) 804. Yes. The characteristic recording unit 800 and the gradation conversion curve recording unit 801 constitute a characteristic recording unit / tone conversion curve recording unit. The monitoring unit 802, the history management unit 803, and the discard control unit 804 are connected to the control unit 117, respectively. The characteristic recording unit 800 is connected to the detection unit 114, the monitoring unit 802, and the discard control unit 804, and the monitoring unit 802 is further connected to the gradation conversion curve recording unit 801 and the discard control unit 804, and the discard control unit 804 is connected. Are further connected to a gradation conversion curve recording unit 801 and a history management unit 803. The gradation conversion curve recording unit 801 is further connected to the calculation unit 115, the history management unit 803, and the gradation conversion unit 110, respectively.

  In the recording unit 603 shown in FIG. 10, the characteristic recording unit 800 records the image characteristic information from the detection unit 114, and the gradation conversion curve recording unit 801 records the gradation conversion curve from the calculation unit 115 as a set. The monitoring unit 802 monitors the remaining amount of the characteristic recording unit 800 and the gradation conversion curve recording unit 801, and notifies the control unit 117 when the remaining amount is less than the predetermined remaining amount.

  On the other hand, the history management unit 803 manages the number of times the gradation conversion curve on the gradation conversion curve recording unit 801 has been read by the gradation conversion unit 110 as history information. The control unit 117 activates the discard control unit 804 upon receiving a notification when the remaining amount from the monitoring unit 802 is equal to or less than the predetermined remaining amount. Based on the history information from the history management unit 803, the discard control unit 804 selects a tone conversion curve with the least number of times read by the tone conversion unit 110, and a characteristic paired with the tone conversion curve. Information is selected, the selected gradation conversion curve and characteristic information are discarded from the gradation conversion curve recording unit 801 and the characteristic recording unit 800, respectively, and the history information of the history management unit 803 is discarded, and the remaining amount is determined. Recover.

  As described above, in this embodiment, an image group having a wide dynamic range exceeding the gradation width of the CCD 103 is obtained, and the gradation conversion curve is calculated based on the scene change. A high-quality output image can be obtained. Also, once calculated characteristic information of the image and the gradation conversion curve are recorded as a set, and when a scene change occurs, the degree of similarity with the recorded characteristic information is determined, and the degree of similarity is determined. If it is high, the gradation conversion curve recorded in correspondence with the characteristic information is used, so that it is not necessary to perform useless calculation and it is possible to quickly adapt to the scene change. Furthermore, when recording the characteristic information and the gradation conversion curve, the history information used in the past is preferentially erased so that it can be efficiently recorded even with a small memory amount. .

  In addition, this invention is not limited only to the said embodiment, Many deformation | transformation or a change is possible. For example, in the first embodiment, the detection unit 114 is configured to detect the presence / absence of a scene change based on the luminance value. However, as in the second embodiment, the presence / absence of a scene change is detected based on a histogram. Alternatively, as shown in FIG. 11, it is also possible to detect the presence or absence of a scene change based on a motion vector.

  That is, the detection unit 114 shown in FIG. 11 includes a first image buffer 310, a second image buffer 311, a motion vector calculation unit (motion vector calculation unit) 312, a motion vector buffer 313, and a determination unit (determination unit) 314. Have. The motion vector calculation unit 312 and the determination unit 314 are respectively connected to the control unit 117, and the first image buffer 310 and the second image buffer 311 are respectively connected to the selection unit 113 and to the motion vector calculation unit 312. . The motion vector calculation unit 312 is further connected to the determination unit 314 via the motion vector buffer 313. In FIG. 11, a signal line indicated by a bold solid line indicates a video signal line, a signal line indicated by a thin solid line indicates a control signal line, and a signal line indicated by a broken line indicates other data lines.

  In the detection unit 114 illustrated in FIG. 11, the reduced images output in time series from the selection unit 113 are sequentially stored in the first image buffer 310 and the second image buffer 311, and the first image buffer 310 and the second image are displayed. A motion vector is calculated for each block by a known motion detection process in which the motion vector calculation unit 312 divides the image into predetermined blocks and performs matching between two reduced images that are stored in the buffer 311 and move back and forth in time series. The detection result is transferred to the motion vector buffer 313 and stored.

  The determination unit 314 reads motion vector information from the motion vector buffer 313, determines that a scene change has occurred when a motion vector greater than a predetermined value is detected in a predetermined number of blocks, and the determination result is used as a control unit 117. Forward to.

  Also, when detecting the presence or absence of a scene change based on the histogram in this way, when calculating the gradation conversion curve based on the histogram in the calculation unit 115, the region corresponding to the block in which the motion vector is detected is displayed. Processing such as multiplying a weighting factor to give a gradation width to the region with emphasis is also possible.

  Furthermore, the detection unit 114 can be configured to detect the presence or absence of a scene change using a change in focus information in addition to the luminance value, histogram, or motion vector.

  In the second embodiment, a wide dynamic range is obtained based on two images, a long exposure image and a short exposure image having a predetermined exposure ratio with respect to the exposure conditions obtained from the long exposure image. Although an image is generated, the second embodiment can be effectively applied to a case where an image having a wide dynamic range is generated from an image group of arbitrary different exposure, and the first embodiment. Similarly to the above, the present invention can be effectively applied to a case where gradation conversion is performed on a normal image.

  In the second embodiment, the detection unit 114 is configured to detect the presence / absence of a scene change based on a histogram. However, as in the first embodiment, the presence / absence of a scene change is detected based on a luminance value. 11 or configured to detect the presence / absence of a scene change based on a motion vector as shown in FIG. 11, or to detect the presence / absence of a scene change using a change in focus information. It can also be configured.

1 is a block diagram illustrating an overall configuration of an imaging system according to a first embodiment of the present invention. It is a block diagram which shows the structure of an example of the selection part shown in FIG. It is explanatory drawing of the function for time setting in the selection part shown in FIG. It is a block diagram which shows the structure of an example of the detection part shown in FIG. It is a block diagram which shows the structure of an example of the calculation part shown in FIG. It is a block diagram which shows the structure of an example of the gradation conversion part shown in FIG. It is a block diagram which shows the whole structure of the imaging system in 2nd Embodiment of this invention. It is a block diagram which shows the structure of an example of the detection part shown in FIG. It is a figure for demonstrating the similarity calculation of the histogram in the detection part shown in FIG. It is a block diagram which shows the structure of an example of the recording part shown in FIG. It is a block diagram which shows the modification of a detection part.

Explanation of symbols

100 Lens system 101 Aperture 102 Low pass filter 103 CCD
DESCRIPTION OF SYMBOLS 104 A / D converter 105 Image buffer 106 Photometry evaluation part 107 Focus detection part 108 AF motor 109 Signal processing part 110 Gradation conversion part 111 D / A converter 112 Output part 113 Selection part 114 Detection part 115 Calculation part 116 Standard gradation curve ROM
117 Control Unit 118 External I / F Unit 600 First Image Buffer 601 Second Image Buffer 602 Composition Unit 603 Recording Unit

Claims (7)

An image group having continuous M-bit gradation width from the imaging system is converted to an output image group having N-bit gradation width (M and N are natural numbers M ≧ N) and output to a subsequent processing system. In the imaging system to
From the image group having the M-bit gradation width, an information amount is obtained from the image size and the number of frames of the image group photographed per unit time by the imaging system so that the load on the subsequent processing system does not increase. Selecting means for selecting one image at long time intervals as the amount of information increases;
Detecting means for detecting a change in characteristic information between images sequentially selected by the selecting means;
Calculation means for calculating a gradation conversion curve of the image selected by the selection means based on the detection result of the detection means;
Based on the detection result of the detection means, when a change in the characteristic information is detected, a new gradation conversion curve calculated by the calculation means is used. Gradation conversion means for converting the image group having the M-bit gradation width into the image group having the N-bit gradation width, respectively, using the gradation conversion curve of
The imaging system according to claim 1, wherein the selection means selects an image at a time interval of 0 when the detection means detects a scene change due to a change in specific information. An image group having continuous M-bit gradation width from the imaging system is converted to an output image group having N-bit gradation width (M and N are natural numbers M ≧ N) and output to a subsequent processing system. In the imaging system to
From the image group having the M-bit gradation width, an information amount is obtained from the image size and the number of frames of the image group photographed per unit time by the imaging system so that the load on the subsequent processing system does not increase. Selecting means for selecting one image at long time intervals as the amount of information increases;
Detecting means for detecting a change in characteristic information between images sequentially selected by the selecting means;
Calculation means for selectively calculating a gradation conversion curve of the image selected by the selection means based on the detection result of the detection means;
A recording unit that records a plurality of sets of characteristic information of the image selected by the selection unit and the gradation conversion curve calculated by the calculation unit for the image;
When a change in the characteristic information is detected based on the detection result of the detection means, a new gradation conversion curve calculated by the calculation means or a past gradation conversion recorded in the recording means Tone conversion means for converting the image group having the M-bit gradation width into the image group having the N-bit gradation width using the current gradation conversion curve when no change in the characteristic information is detected using a curve And having
The imaging system according to claim 1, wherein the selection means selects an image at a time interval of 0 when the detection means detects a scene change due to a change in specific information. The imaging system according to claim 1 or 2,
The detecting means is
As the characteristic information, luminance calculation means for calculating an average luminance level of the image selected by the selection means,
Determining means for determining the presence or absence of a scene change based on a change in average brightness level between sequential images calculated by the brightness calculating means;
An imaging system comprising: The imaging system according to claim 1 or 2,
The detecting means is
A motion vector calculating means for calculating a motion vector from two sequential images selected by the selecting means as the characteristic information;
Determining means for determining the presence or absence of a scene change based on the motion vector calculated by the motion vector calculating means;
An imaging system comprising: The imaging system according to claim 1 or 2,
The detecting means is
Histogram calculation means for calculating a histogram from the image selected by the selection means as the characteristic information;
Determining means for determining presence or absence of a scene change based on a change in histogram between sequential images calculated by the histogram calculating means;
An imaging system comprising: The imaging system according to claim 2,
The detecting means is
Similarity calculating means for calculating the similarity between the characteristic information of the image selected by the selecting means and a plurality of characteristic information recorded in the recording means;
When the similarity calculated by the similarity calculating unit satisfies a predetermined condition, a reading unit that reads out from the recording unit a gradation conversion curve that forms a pair with the similar characteristic information;
An imaging system comprising: The imaging system according to claim 1 or 2,
The detecting means is
As a change in the characteristic information, a photographing condition detecting unit that detects a photographing condition of an exposure condition or a focusing condition of the image selected by the selecting unit;
Determination means for determining the presence or absence of a scene change based on a change in shooting conditions between sequential images detected by the shooting condition detection means;
An imaging system comprising:

Images