JP2010276968A - Image display and image display method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display and an image display method at low cost, performing high-speed display without deteriorating gray-scale characteristics of a multi-gray-scale image. <P>SOLUTION: An output image H1 corresponding to a low gray-scale range of an input image A, and an output image H2 corresponding to a multi-gray-scale range thereof are extracted from the input image A. They are displayed at a high speed as a sub-frame, and thereby, substantial high-quality image is displayed without deteriorating the gray-scale characteristics of the input image A. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image display apparatus that displays a multi-tone image such as an HDR (High Dynamic Range) image and a control method thereof.

  In recent years, high-quality images are required not only for medical diagnosis and exhibition at museums, but also for photo viewing at ordinary homes, and there is a growing need to provide images with more tones than before. ing. In order to respond to such needs, an imaging device capable of capturing a multi-gradation image than before, a transmission device or recording / reproducing device capable of handling multi-gradation image data than before, and a conventional device A display device capable of multi-gradation display is required.

  In the imaging system, a camera that improves the dynamic range (DR) of the image sensor itself of the camera, a camera that acquires images in a plurality of luminance ranges corresponding to the same scene by a plurality of imaging systems with different sensitivities, etc. are announced. (For example, refer to Patent Document 1).

  On the other hand, in the display system, multi-grayscale luminance control that is higher than conventional ones can be performed and display with a large DR can be achieved by the following method. For example, in a liquid crystal display, there are a method of controlling the aperture ratio of a liquid crystal device with higher accuracy than before, a method of controlling the amount of light for each area of the screen using projection means such as an LED for a backlight, etc. Patent Document 2).

  However, the DR of light in an actual living space or in the natural world is very large, and further multi-gradation and higher DR are required for both the imaging system and the display system. Particularly in the display system, display performance that can cope with the progress of the imaging system is required.

  A general electronic image is displayed at a frame frequency of 60 Hz to 50 Hz. In the case of such a frequency, when different images are displayed by dividing the frame into a plurality of subframes, the images of the individual subframes are not recognized by the human eye, and one frame image obtained by adding them is added. Recognized as This is because the human eye cannot respond to the frequency at which the subframe switches. Such a phenomenon is called the integral effect of the human eye. By utilizing this human eye integration effect, an image that is originally intended to be displayed is divided into a plurality of images and displayed in respective sub-frames, so that the composite image is substantially displayed on the human eye. The

  From the viewpoint of brightness control of the display, not only the method of controlling the light intensity of each frame, but also a method of performing control to add brightness in the time direction using the above-described integration effect of the human eye is widely used. ing. For example, there is a method of displaying RGB components of the same frame in order and combining them so as to be seen as seen in some micro display projectors (for example, see Patent Document 3). Further, as seen in a plasma display, there is a method in which one field period is divided into a plurality of subfields, and gradation is expressed by a selected combination of the subfields.

  For example, in order to improve the moving image visibility of a display, there is a method of displaying a high-luminance image in one subframe and a low-luminance image in the other subframe in a double-speed display device (for example, a patent) References 4 and 5). According to such a display method, since the display image of each subframe is time-integrated with the eyes, not only the original image is substantially reproduced but also an effect close to black insertion is obtained at low luminance. , Motion blur is improved.

US6909461B1 US6891672B2 Special registration 36600610 JP 2006-343706 A JP 2004-240317 A

  As described above, in order to display an HDR image, further multi-gradation is required for both the imaging system and the display system. In particular, in a display system, DR enlargement and multi-gradation are realized by controlling the luminance level direction of the device, such as increasing the accuracy of light emission intensity control and increasing the accuracy of liquid crystal aperture ratio control. However, in this case, there is a problem that the cost of the display system is unavoidable due to the high accuracy of the device and the complicated structure.

  Further, many existing transmission systems and processing systems are premised on handling data with a predetermined number of bits (for example, 8 bits). Therefore, if processing is performed using other numbers of bits, an existing system that is generally inexpensive cannot be used, which also increases costs.

  Further, when performing the double speed display using the eye integration effect as described above, for example, in the case of performing the double speed display on an 8-bit gradation display, a theoretical 9-bit image display capability can be obtained. However, in the conventional double-speed display technology (Patent Documents 4 and 5), the bit depth of the image to be displayed is the same as the bit depth of the actually displayed image, and the display gradation is improved. It was not a thing. That is, although the 9-bit display is possible in the configuration of the display system, the actually displayed image is expressed as an 8-bit image instead of a 9-bit gradation expression. Therefore, in the conventional double speed display system, for example, when there is a change of one gradation in the original 9-bit image (512 gradations), this cannot be expressed.

  The present invention has been made to solve the above-described problems, and provides an image display apparatus and an image display method that can perform double-speed display at low cost without impairing the gradation characteristics of a multi-gradation image. For the purpose.

  As a means for achieving the above object, an image display apparatus of the present invention comprises the following arrangement. That is, an input unit that acquires an input image, a clipping unit that generates M output images by cutting out M gradation ranges from the input image, and the M output images as M subframes. Display means for displaying at M-times speed.

  Further, the image processing apparatus includes an input unit that acquires M images obtained by shooting the same scene with different exposure levels, and a display unit that displays the M images as M sub-frames at an M-times speed. .

  According to the present invention having the above-described configuration, it is possible to perform double speed display at a low cost without impairing the gradation characteristics of the multi-gradation image.

The block diagram which shows the structure of the HDR image display system in 1st Embodiment, The figure which shows the relationship between the input image A and output image H1, H2 in 1st Embodiment. The figure which shows the relationship between the input image A in 1st Embodiment, output image H1, H2, and the synthesized image Hdisp. The block diagram which shows the structure of the HDR image display system in 2nd Embodiment, The figure which shows the relationship between the input image A and output image H1, H2 in 2nd Embodiment. The figure which shows the relationship between the input image A in 2nd Embodiment, the output images H1, H2, and the synthesized image Hdisp. The block diagram which shows the structure of the image display system in 3rd Embodiment, The figure which shows the relationship between the input image A in 3rd Embodiment, and the output images H1-H4. The figure which shows the relationship between the input image A in 3rd Embodiment, the output images H1-H4, and the synthesized image Hdisp. The figure which shows the relationship between the input image A and output image H1, H2 in 4th Embodiment. The figure which shows the relationship between the input image A in 4th Embodiment, output image H1, H2, and the synthesized image Hdisp. The figure which shows the relationship between the input image A and output image H1, H2 in 5th Embodiment. The figure which shows the relationship between the input image A in 5th Embodiment, output image H1, H2, and the synthesized image Hdisp. The figure which shows the relationship between the input image A and output image H1, H2 in 6th Embodiment. The figure which shows the relationship between the input image A in 6th Embodiment, output image H1, H2, and the synthesized image Hdisp. It is a block diagram which shows the structure of the HDR image display system in 7th Embodiment.

  Hereinafter, the present invention will be described in detail based on preferred embodiments with reference to the accompanying drawings. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

<First Embodiment>
In the display system of this embodiment, multi-gradation input image data is divided into a plurality of image data for each gradation level, and these are sequentially displayed within one frame. Control to see the added image.

  In the present embodiment, M pieces of image data each corresponding to a predetermined gradation width are cut out from multi-tone input image data, each of the M pieces of image data is converted into a predetermined number of gradations, and M-times speed is obtained. Are sequentially displayed on the image display device. In the following description, it is assumed that the input image is A, the extracted M images are B1 to BM, and the output image after gradation conversion is H1 to HM.

  Hereinafter, a case where M = 2, the input image A is 12 bits deep, and the output images H1 and H2 are 8 bits deep will be described as an example.

  FIG. 1 shows a block configuration example of an HDR image display system in the present embodiment. The HDR camera (high dynamic range camera) 101 in the figure outputs a 12-bit image A. The image data cutout unit 102 generates partial gradation images B 1 and B 2 having the same gradation resolution from the 12-bit input image A acquired from the HDR camera 101. Here, the partial gradation image B1 is an image having the same number of pixels as the input image A. In the input image A, for pixels corresponding to pixel data of 0 to 1433, pixel data of the same pixel in the partial gradation image B1 is set to a value obtained by converting the corresponding A value according to the rules. In addition, for pixels whose pixel data exceeds 1433 in the input image A, the maximum value is set for the pixel data of the same pixel in the partial gradation image B1. The partial gradation image B2 is also an image having the same number of pixels as the input image A. In the input image A, for pixels corresponding to pixel data 1434 to 4095, pixel data of the same pixel in the partial gradation image B2 is set to a value obtained by converting the value of the corresponding input image A according to the rules. In addition, for pixels whose pixel data is less than 1434 in the input image A, the minimum value is set for the pixel data of the same pixel in the partial gradation image B2.

  The gradation converters A103 and B104 respectively perform gradation conversion so that the partial gradation images B1 and B2 can be accommodated in 256 gradations (8 bits), and output the converted 8-bit images H1 and H2. The output images H1 and H2 are sent to the switch 107 via transmission paths (or processing systems) 105 and 106 corresponding to 8 bits.

  The switch 107 is switched at 120 Hz to send the output images H1 and H2 alternately to the double-speed display device 108 as sequential data. The double-speed display device 108 has a frame frequency of 60 Hz and displays output images H1 and H2 every 1/120 second.

  Here, the relationship between the input image A and the output images H1 and H2 will be described with reference to FIG. As shown in FIG. 2, in the input image A, th is a threshold for gradation division, the output image H1 corresponds to the gradations 0 to th of the input image A, and the output image H2 is the gradation th of the input image A. Corresponds to ~ 4095. In the present embodiment, th = 1433.

  Here, the relationship between the input image A, the output images H1, H2, and the composite image Hdisp will be described with reference to FIG. The composite image Hdisp is the sum of the output images H1 and H2 displayed in the subframe for double-speed display, and is an image that is substantially displayed by the above-described human eye integration effect.

  In FIG. 3, th = 1433 corresponds to about 35% as the luminance level of the input image A. Since the output image H1 corresponds to 0 to 1433 gradations of the input image A, a polygonal line such as P0-P1-P4 is drawn on the graph in the figure. Since the output image H2 corresponds to the 1434 to 4095 gradations of the input image A, a straight line such as P2-P4 is drawn on the graph in the figure. The composite image Hdisp = H1 + H2 draws a polygonal line such as P0-P1-P3 on the graph in the figure.

  In the present embodiment, the input image A has a 12-bit depth, and the output images H1 and H2 both have an 8-bit depth, but the threshold th for the input image A does not divide the gradation into two equal parts. Therefore, the ratio between the number of gradations of the output images H1 and H2 and the number of gradations corresponding to the input image A is different. That is, the gradation conversion ratios are different in the gradation conversion units A103 and B104 corresponding to the output images H1 and H2, respectively. This can be seen from the fact that the output image H1 (P0-P1) and the output image H2 (P2-P4) have different slopes as shown in FIG.

  In the present embodiment, by setting the threshold th to be equivalent to 35% luminance, the gradation range corresponding to the output image H1 is set to be slightly narrow, the gradation range corresponding to the output image H2 is set to be slightly wide, and the composite image Hdisp is P0. Draw a convex line like -P1-P3. As a result, the composite image is adjusted so that the contrast is enhanced (specifically, the gradation characteristics are adjusted), and when the contrast is slightly enhanced in this way, it is closer to the actual appearance. Become.

  In the present embodiment, as described above, 8-bit output images H1 and H2 are created from the input image A having a 12-bit depth, and these are sequentially displayed on the double-speed display device 108 in units of 8 bits. Then, due to the integration effect of the human eye, the double-speed display device 108 substantially performs gradation display equivalent to 9 bits. In the present embodiment, the adjustment of the gradation characteristics, that is, the extraction of the image data with a predetermined gradation width is performed before the bit depth conversion in the gradation conversion units A103 and B104. There is no loss of information.

  The image processing in the present embodiment described above can be expressed by the following equation.

B2 = A-1433 (1-1)
B1 = A-B2 (1-2)
H2 = F256_1434 (B2) (1-3)
H1 = F256_2662 (B1) (1-4)
These equations (1-1), (1-2), (1-3), and (1-4) are defined for every pixel data.

  Here, the partial gradation images B1 and B2 do not have negative values, and if the right sides of the expressions (1-1) and (1-2) are negative in the calculation, the partial gradation images B1 and B2 Is set to 0.

  The function Fn1_n2 (X) defines that the image data X defined by the gradation number n2 is converted into image data having the gradation number n1 (assigned to the n1 gradation).

  In the present embodiment, an effect can be obtained by performing the same processing on each component of RGB at a minimum. Accordingly, it is assumed that all the embodiments perform the same processing for each RGB component. However, when any color correction is performed on an image (composite image Hdisp) that is visible as a result of the integration effect of human eyes, processing with different parameters for each color may be performed within the scope of the present invention. .

  As described above, according to this embodiment, display with the maximum number of gradations that can be expressed in the double-speed display device can be performed without impairing the gradation characteristics. That is, since the display gradation of the M × speed display device can be utilized efficiently, a multi-gradation image photographed by an HDR camera or the like can be provided to the user with high quality.

  Moreover, since an existing transmission system or processing system (for example, limited to 8-bit processing, for example) that is generally inexpensive can be used, the cost can be suppressed.

  In addition, this embodiment is more effective in the following points with respect to the display system which performs the double-speed display shown in the conventional example. For example, in a display system (M = 2) that performs 8-bit double-speed display, if the original image is 9 bits in the above-described conventional example, missing gradation information occurs in the image that is actually displayed at double-speed. It was. However, according to the present embodiment, two 8-bit images can be created with respect to the original 9-bit image so that gradation information is not lost, and these can be sequentially displayed. Therefore, according to the present embodiment, an image representation having 9-bit gradation both formally and substantially is made.

  In the present embodiment, even if M = 2 and the input image exceeds 9 bits, a gradation range that is meaningful as an image is selected from the original multi-bit image, and the gradation range is selected. Allocating to two 256 gradations (8-bit gradation), it is possible to display sequentially.

  As described above, according to the present embodiment, the gradation characteristics are already adjusted for the input image having a gradation of more than 9 bits while minimizing the lack of gradation information and contributing to the actual expression. A 9-bit image can be finally displayed.

  In addition, the present embodiment can be generalized to M-times speed display including M subframes. Therefore, the present embodiment can be applied to M 8-bit images, and is different from the conventional double-speed display system in that the larger M is, the more the multi-bit input image can be handled.

  In this respect, it can be seen that the present embodiment is more useful, unlike the double-speed display system shown in the conventional example.

<Second Embodiment>
Hereinafter, a second embodiment according to the present invention will be described. Also in the second embodiment, M pieces of image data are cut out from the input image A and sequentially displayed on an M-times speed image display device. The input image is A, and the cut out M pieces of output images are H1 to HM. Will be described.

  The second embodiment shows an example in which M = 2, the input image A has a 9-bit depth, and gradation conversion with bit depth conversion is not performed.

  FIG. 4 shows a block configuration example of the HDR image display system in the second embodiment. In FIG. 4, the same reference numerals are given to the same configurations as in FIG. 1 of the first embodiment described above.

  The HDR camera 201 in FIG. 4 outputs a 9-bit image A. The image data cutout unit 202 extracts 8-bit output images H1 and H2 from the 9-bit input image A. The output images H1 and H2 are sent to the switch 107 via transmission paths (or processing systems) 105 and 106 corresponding to 8 bits.

  The switch 107 sends the output images H1 and H2 to the double-speed display device 108 as sequential data by switching at 120 Hz. The double-speed display device 108 has a frame frequency of 60 Hz and displays output images H1 and H2 every 1/120 second.

  Here, the relationship between the input image A and the output images H1 and H2 in the second embodiment will be described with reference to FIG. As shown in FIG. 5, the threshold value th for gradation division of the input image A in the second embodiment is 255. Therefore, the output image H1 corresponds to the gradations 0 to 255 of the input image A, and the output image H2 is input. This corresponds to the gradations 256 to 511 of the image A. In the second embodiment, since the number of gradations of both the output images H1 and H2 is 1: 1 with respect to the number of gradations corresponding to the input image A, gradation conversion with bit depth conversion is performed. It is unnecessary.

  Here, the relationship between the input image A, the output images H1, H2, and the composite image Hdisp in the second embodiment will be described with reference to FIG. The composite image Hdisp is the sum of the output images H1 and H2 displayed in the subframe for double-speed display, and is an image that is substantially displayed by the above-described human eye integration effect.

  In FIG. 6, th = 255 corresponds to exactly 50% as the luminance level of the input image A. Since the output image H1 corresponds to 0 to 255 gradations of the input image A, a polygonal line such as P0-P1-P4 is drawn on the graph in the figure. Since the output image H2 corresponds to 256 to 511 gradation levels of the input image A, a straight line such as P2-P4 is drawn on the graph in the figure. The composite image Hdisp = H1 + H2 draws a straight line such as P0-P1-P3 on the graph in the figure.

  In the second embodiment, since the composite image Hdisp draws a straight line like P0-P1-P3, it can be seen that no adjustment is made to the entire gradation characteristic.

  The image processing in the second embodiment described above can be expressed by the following equation.

H2 = A-255 (2-1)
H1 = A-H2 (2-2)
These equations (2-1) and (2-2) are defined for every pixel data.

  Here, the output images H1 and H2 do not have negative values, and when the right side of the expressions (2-1) and (2-2) is negative in the calculation, 0 is set as the output images H1 and H2. Is done.

  As described above, according to the second embodiment, by setting the input image A to 9-bit gradation, the 9-bit gradation is maintained without changing the gradation as in the first embodiment. In this way, double speed display can be performed.

<Third Embodiment>
The third embodiment according to the present invention will be described below. Also in the third embodiment, M pieces of image data are cut out from the input image A and sequentially displayed on an M-times speed image display device. The input image is A, and the cut out M pieces of output images are H1 to HM. Will be described.

  In the third embodiment, taking M = 4 as an example, a generalized concept of M, that is, a process of sequentially displaying M output images cut out from multiple images on an M-times-speed display device will be described. For the sake of simplicity, it is assumed that tone conversion with bit depth conversion is not performed in the third embodiment.

  FIG. 7 shows a block configuration example of the HDR image display system in the third embodiment. The HDR camera 301 in the figure outputs a 10-bit image A. The image data cutout unit 302 extracts 8-bit output images H1, H2, H3, and H4 from the 10-bit input image A. The output images H1, H2, H3, and H4 are sent to terminals 1, 2, 3, and 4 of the changeover switch 307 via transmission paths (or processing systems) 303, 304, 305, and 306 corresponding to 8 bits.

  The changeover switch 307 is sequentially switched in the order of terminal 1 → terminal 2 → terminal 3 → terminal 4 → terminal 1 in 1 cycle 1/60 seconds. As a result, data in which output images H 1, H 2, H 3, and H 4 are sequentially continuous is constructed and sent to the quadruple speed display device 308. Since the quadruple speed display device 308 has a frame frequency of 60 Hz, the output images H1 to H4 are sequentially displayed every subframe of 1 / (4 × 60) seconds.

  Here, the relationship between the input image A and the output images H1 to H4 in the third embodiment will be described with reference to FIG. In the third embodiment, since the input image A has a 10-bit depth (1024 gradations), the output images H1 to H4 are each cut out so as to correspond to the gradation range obtained by equally dividing the gradation of the input image A into four. It is. That is, as shown in FIG. 8, the third embodiment has th11, th12, th13, and th14 as the gradation division threshold values of the input image A, which are 255, 511, 767, and 1023, respectively. Therefore, the output image H1 corresponds to the gradations 0 to 255 of the input image A, the output image H2 corresponds to the gradations 256 to 511 of the input image A, and the output image H3 corresponds to the gradations 512 to 767 of the input image A. Correspondingly, the output image H4 corresponds to the gradations 768 to 1023 of the input image A. In the third embodiment, for any of the output images H1 to H4, the number of gradations is 1: 1 with respect to the number of gradations corresponding to the input image A, so gradation conversion with bit depth conversion is not performed. It is unnecessary.

  Here, the relationship between the input image A, the output images H1 to H4, and the composite image Hdisp in the third embodiment will be described with reference to FIG. The composite image Hdisp is a sum of the output images H1 to H4 displayed in the subframe of quadruple speed display, and is an image substantially displayed by the above-described human eye integration effect.

  In FIG. 9, since the output image H1 corresponds to 0 to 255 gradations of the input image A, a polygonal line such as P0-P1-P7 is drawn on the graph in the figure. Since the output image H2 corresponds to the 256 to 511 gradations of the input image A, a broken line such as P0-P2-P3-P7 is drawn on the graph in the figure. Since the output image H3 corresponds to 512 to 764 gradations of the input image A, a polygonal line such as P0-P4-P5-P7 is drawn on the graph in the figure. Since the output image H4 corresponds to the 765 to 1023 gradations of the input image A, a broken line such as P0-P6-P7 is drawn on the graph in the figure. In the third embodiment, the gradient portions generated on the graph for each of the output images H1 to H4 all have the same gradient. The composite image Hdisp = H1 + H2 + H3 + H4 draws a straight line such as P0-P1-P8 on the graph in the figure.

  The image processing in the third embodiment described above can be expressed by the following equation.

H1 = A-th13 (3-1)
H2 = A-H1-th12 (3-2)
H3 = A-H1-H2-th11 (3-3)
H4 = A-H1-H2-H3 (3-4)
However, th13 = 764, th12 = 511, th11 = 255, and these expressions (3-1), (3-2), (3-3), and (3-4) are for all pixel data. Defined in

  Here, the output images H1 to H4 have no negative value, and if the right side of the expressions (3-1) to (3-4) is negative in the calculation, 0 is set as the output images H1 to H4. Is done.

  As described above, according to the third embodiment, by setting M = 4, the display capability can be four times the display capability of each subframe. That is, if the display capability of each subframe is 8 bits, a total display capability of 10 bits can be shown by the integration effect of the human eye. That is, if the original image (input image A) is 10 bits as in the third embodiment, M = 4 and the gradation range of the image is equally divided into four without dropping gradation information. Can be displayed.

  In the third embodiment, if the value of M is further increased, that is, if a display device with a higher refresh rate is used, it is apparent that the gradation display capability is improved. In addition, by making M a generalized format, it is possible to deal with a multi-tone image having a larger bit depth as the input image A.

  Further, in the second and third embodiments, the example in which the number of gradations of each of the plurality of output images is equal is described in order not to perform the gradation conversion accompanying the bit depth conversion. The present invention is applicable as long as the total is equal to or less than the number of gradations of the input image A.

<Fourth embodiment>
The fourth embodiment according to the present invention will be described below. Also in the fourth embodiment, M pieces of image data are cut out from the input image A and are sequentially displayed on an M-times speed image display device. The input image is A, the cut out M images are B1 to BM, The output image after gradation conversion will be described as H1 to HM.

  In the fourth embodiment, an example is shown in which the gradation characteristics are adjusted by making the luminance range of the output images H1 and H2 with respect to the input image A fixed without being fixed, depending on the adjustment contents.

  The configuration of the HDR image display system in the fourth embodiment is the same as that in FIG. 1 shown in the first embodiment described above, except that the image cutout unit 102 and the accompanying gradation conversion unit A103 and gradation conversion unit B104. The operation is different.

  Here, the relationship between the input image A and the output images H1 and H2 in the fourth embodiment will be described with reference to FIG. In the fourth embodiment, the input image A has a 12-bit depth (4095 gradations), and the output images H1 and H2 have an 8-bit depth. As shown in FIG. 10, the output image H1 corresponds to the first gradation th1 to the second gradation th2 in the input image A, and the output image H2 corresponds to the third gradation th3 to the fourth gradation th4 of the input image A. Correspond.

  The values of th1 to th4 are determined according to what adjustment is performed on the input image A. The fourth embodiment is characterized in that no adjustment is applied to a portion close to black in the input image A, and the high luminance portion is controlled so as not to be processed.

  For this purpose, first, th1 = 0 is set. Then, in order to make maximum use of the limited display gradation in accordance with the actual image, the gradation range of the original image where there is no actual data is excluded from the processing target. For this purpose, th4 is set to a value smaller than the maximum gradation of the input image A, that is, 3480 instead of 4095 (about 85% in actual luminance). Further, since it is closer to the actual appearance when the contrast is slightly enhanced, the gradation range corresponding to the output image H1 is slightly narrowed and the gradation range corresponding to the output image H2 is set slightly wider. Further, the respective gradation ranges are adjacent to each other. As a result, th2 = 1433 (about 35% in luminance level) and th3 = th2 + 1 = 1434.

  Here, the relationship between the input image A, the output images H1, H2, and the composite image Hdisp in the fourth embodiment will be described with reference to FIG. The composite image Hdisp is the sum of the output images H1 and H2 displayed in the subframe for double-speed display, and is an image that is substantially displayed by the above-described human eye integration effect.

  In FIG. 11, since the output image H1 corresponds to 0 (th1) to th2 of the input image A, a polygonal line such as P0-P1-P4 is drawn on the graph in the figure. Since the output image H2 corresponds to th3 to th4 of the input image A, a straight line such as P2-P4 is drawn on the graph in the figure. The composite image Hdisp = H1 + H2 draws a polygonal line such as P0-P1-P3 on the graph in the figure.

  In the fourth embodiment, since the high-luminance portion of the input image A is discarded, the broken line of Hdisp reaches the maximum value before the input level reaches 4095. The composite image Hdisp has a convex polygonal line such as P0-P1-P3. In the fourth embodiment, the contrast of the input image A is close to that of the actual eye in the fourth embodiment. This is because th2 (th3) is set to emphasize a little.

  The image processing in the fourth embodiment described above can be expressed by the following equation.

Btop = A−th3 (4-1)
B2 = A-th2-Btop (4-2)
B1 = A-th1-Btop-B2 (4-3)
H2 = F256_ (th4-th3) (B2) (4-4)
H1 = F256_ (th2-th1) (B1) (4-5)
However, th1 = 0
th2 = 1433 (about 35% in actual brightness)
th3 = th2 + 1 = 1434
th4 = 3480 (about 85% in actual brightness)
These equations (4-1), (4-2), (4-3), (4-4), and (4-5) are defined for every pixel data.

  Here, Btop is a value indicating the maximum luminance range of the input image A, and indicates the lower limit boundary luminance of the maximum luminance range. Here, the maximum luminance range is a gradation range excluded in the fourth embodiment.

  Further, the partial gradation images B1 and B2 do not have negative values, and if the right side of the expressions (4-2) and (4-3) is negative in the calculation, the partial gradation images B1 and B2 are obtained. 0 is set.

  The function Fn1_n2 (X) defines that the image data X defined by the gradation number n2 is converted into image data having the gradation number n1 (assigned to the n1 gradation).

  As described above, according to the fourth embodiment, the luminance range of the output images H1 and H2 with respect to the input image A is adjusted so as not to be adjusted for a portion close to black in the input image A, and the high luminance portion. Can be set so that they are not processed.

<Fifth Embodiment>
The fifth embodiment according to the present invention will be described below. Also in the fifth embodiment, M pieces of image data are cut out from the input image A, and are sequentially displayed on an M-times speed image display device. The input image is A, the cut out M pieces of images are B1 to BM, The output image after gradation conversion will be described as H1 to HM.

  Also in the HDR image display system of the fifth embodiment, the luminance range of the output images H1 and H2 with respect to the input image A can be set flexibly as in the fourth embodiment described above.

  Here, the relationship between the input image A and the output images H1 and H2 in the fifth embodiment will be described with reference to FIG. In the fifth embodiment, the input image A has a 12-bit depth (4095 gradations), and the output images H1 and H2 have an 8-bit depth. As shown in FIG. 12, the output image H1 corresponds to the first gradation th1 to the second gradation th2 in the input image A, and the output image H2 corresponds to the third gradation th3 to the fourth gradation in the input image A. Corresponds to gradation th4.

  In the fifth embodiment, in order to correct this by taking as an example the case where the input image A was captured in a slightly overexposed state and slightly blacked and the sensor is saturated near the maximum gradation. An example of performing the adjustment will be shown.

  In the fifth embodiment, the values of th1 to th4 are determined as follows. First, in order to eliminate the above-described black floating, th1 is set to a value larger than the minimum gradation of the input image A, that is, a value larger than zero. Here, th1 = 204 (approximately 5% in actual luminance).

  Then, th4 is set larger than the maximum gradation (4095) of the input image in order to suppress the multi-gradation side and display. Here, th4 = 4709 (115% in actual luminance). Further, since it is closer to the actual appearance when the contrast is slightly enhanced, the gradation range corresponding to the output image H1 is slightly narrowed and the gradation range corresponding to the output image H2 is slightly widened. Further, the respective gradation ranges are adjacent to each other. Thereby, th2 = 1842 (about 45% in luminance level) and th3 = th2 + 1 = 1842.

  Here, the relationship between the input image A, the output images H1, H2, and the composite image Hdisp in the fifth embodiment will be described with reference to FIG. The composite image Hdisp is the sum of the output images H1 and H2 displayed in the subframe for double-speed display, and is an image that is substantially displayed by the above-described human eye integration effect.

  In FIG. 13, since the output image H1 corresponds to th1 to th2 of the input image A, a polygonal line such as P0-P1-P4 is drawn on the graph in the figure. Since the output image H2 corresponds to th3 to th4 of the input image A, a straight line such as P2-P5 is drawn on the graph in the figure. The composite image Hdisp = H1 + H2 draws a polygonal line such as P0-P1-P3 on the graph in the figure.

  According to FIG. 13, since Hdisp does not occur while the luminance of the input image A is equal to or less than th1, black float is corrected. Further, since the output image H2 is set not to reach 255 gradations even when the luminance of the input image A reaches the maximum value, the broken line of Hdisp does not reach the maximum value of 4095. , 511 which is the maximum value of Hdisp is not reached. Accordingly, the saturation state of the multi-gradation part can be relaxed.

  The image processing in the fifth embodiment can be expressed by the following equation.

B2 = A-th3 (5-1)
B1 = A-th1-B2 (5-2)
H2 = F256_ (th4-th3) (B2) (5-3)
H1 = F256_ (th2-th1) (B1) (5-4)
However, th1 = 204 (approx. 5% in actual brightness)
th2 = 1842 (about 45% in actual brightness)
th3 = th2 + 1 = 1843
th4 = 4709 (115% in actual brightness)
These equations (5-1), (5-2), (5-3), and (5-4) are defined for every pixel data.

  In addition, the partial gradation images B1 and B2 have no negative value, and if the right side of the expressions (5-1) and (5-2) is negative in the calculation, the partial gradation images B1 and B2 are obtained. 0 is set.

  The function Fn1_n2 (X) defines that the image data X defined by the gradation number n2 is converted into image data having the gradation number n1 (assigned to the n1 gradation).

  As described above, according to the fifth embodiment, the luminance range of the output images H1 and H2 with respect to the input image A is set so as to eliminate black floating and further reduce the saturation state of the multi-gradation part. Can do.

<Sixth Embodiment>
The sixth embodiment according to the present invention will be described below. Also in the sixth embodiment, M pieces of image data are cut out from the input image A and sequentially displayed on an M-times speed image display device. The input image is A, the cut out M images are B1 to BM, The output image after gradation conversion will be described as H1 to HM.

  Also in the HDR image display system of the sixth embodiment, the luminance ranges of the output images H1 and H2 with respect to the input image A can be set flexibly as in the fourth embodiment described above.

  Here, the relationship between the input image A and the output images H1 and H2 in the sixth embodiment will be described with reference to FIG. In the sixth embodiment, the input image A has a 12-bit depth (4095 gradations), and the output images H1 and H2 have an 8-bit depth. As shown in FIG. 14, the output image H1 corresponds to the first gradation th1 to the second gradation th2 in the input image A, and the output image H2 corresponds to the third gradation th3 to the fourth gradation in the input image A. Corresponds to gradation th4.

  In the sixth embodiment, in order to particularly enhance the contrast of the input image A, the values of th1 to th4 are determined so that the gradation regions corresponding to the output images H1 and H2 overlap. First, in order to overlap the corresponding gradations, th3 <th2. Since no adjustment is performed for the low gradation portion, th1 = 0 is set. Also, in order to make the best use of the limited gradation display capability by excluding the gradation area where there is no actual data, th4 is 3480 (actual brightness) instead of the maximum gradation 4095 of the input image A. About 85%).

  Here, the relationship between the input image A, the output images H1, H2, and the composite image Hdisp in the sixth embodiment will be described with reference to FIG. The composite image Hdisp is the sum of the output images H1 and H2 displayed in the subframe for double-speed display, and is an image that is substantially displayed by the above-described human eye integration effect.

  In FIG. 15, since the output image H1 corresponds to 0 (th1) to th2 of the input image A, a polygonal line such as P0-P2-P5 is drawn on the graph in the figure. Since the output image H2 corresponds to th3 to th4 of the input image A, a straight line such as P3-P5 is drawn on the graph in the figure. The composite image Hdisp = H1 + H2 draws a polygonal line such as P0-P6-P1-P4 on the graph in the figure.

  In the sixth embodiment, since the high luminance portion of the input image A is discarded, the broken line of Hdisp reaches the maximum value before the input level reaches 4095. In addition, it can be seen that the contrast is enhanced in this range when the composite image Hdisp draws a convex polygonal line in P6-P1-P4.

  The image processing in the sixth embodiment can be expressed by the following equation.

B2 = A-th3 (6-1)
Btmp = A−th2 (6-2)
B1 = A-th1-Btmp (6-3)
H2 = F256_ (th4-th3) (B2) (6-4)
H1 = F256_ (th2-th1) (B1) (6-5)
However, th1 = 0
th2 = 1637 (about 40% in actual brightness)
th3 = 613 (about 15% in actual brightness)
th4 = 3480 (approx. 85% in actual brightness)
These equations (6-1), (6-2), (6-3), (6-4), and (6-5) are defined for every pixel data.

  Btmp is temporary data necessary for calculating the partial gradation image B1.

  Also, Btmp and partial gradation images B1 and B2 do not have negative values, and when the right side of equations (4-1), (4-2), and (4-3) is negative in the calculation, 0 is set for each.

  The function Fn1_n2 (X) defines that the image data X defined by the gradation number n2 is converted into image data having the gradation number n1 (assigned to the n1 gradation).

  As described above, according to the sixth embodiment, the luminance range of the output images H1 and H2 with respect to the input image A can be set so as to enhance the contrast.

<Seventh embodiment>
The seventh embodiment according to the present invention will be described below. In the seventh embodiment, an example is shown in which an HDR camera that outputs M-times-speed image data is used on the premise that M subframes are sequentially displayed as M-times-speed display. Hereinafter, a case where M = 2 will be described as an example.

  FIG. 16 shows a block configuration example of an HDR image display system in the seventh embodiment. The double speed HDR camera 501 in the figure includes two photoelectric conversion units 505 and 506. The images H1 and H2 generated by the two photoelectric conversion units 505 and 506 are alternately switched at 120 Hz by the switch 504 and are alternately output to the 8-bit transmission path (or processing system) 502 as sequential data. The output images H1 and H2 are sent to the double speed display device 503. The double speed display device 503 alternately displays the output image H1 of the photoelectric conversion unit 505 and the output image H2 of the photoelectric conversion unit 506 at 120 Hz. At this time, in the double speed display device 503, the sum of the output images H1 and H2 can be seen by the integration effect of the human eye.

  Here, in the seventh embodiment, the division of roles of the output images H1 and H2 with respect to the finally observed output image, that is, individual characteristics is not particularly limited. However, considering the physical characteristics of a general imaging camera, it is desirable that one is an image taken with a higher exposure level and the other is an image taken with a lower exposure level. As a result, images having different exposure levels in the same scene can be obtained as the output images H1 and H2.

  Moreover, although the specific structure of the photoelectric conversion parts 505 and 506 is not limited, generally the following structures can be considered. For example, there is a configuration in which two area sensors are provided and these are optically combined. In addition, there is a configuration in which one system of photoelectric conversion units is photographed while switching exposure conditions at high speed. Further, there is a configuration in which an image based on two types of exposure conditions is read from one area sensor by a specific pixel arrangement. The double-speed HDR camera 501 in the seventh embodiment may have any of these configurations.

  As described above, according to the seventh embodiment, M types of images photographed on the imaging device side based on predetermined M types of exposure conditions are sequentially displayed in M frames on the M double speed display device. At this time, by controlling the exposure conditions, it is possible to perform more effective M-times display using the integration effect of the human eye, as in the first to sixth embodiments described above.

<Other embodiments>
The present invention can take the form of, for example, a system, apparatus, method, program, or storage medium (recording medium). Specifically, the present invention may be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, an imaging device, a web application, a printer, etc.), or applied to a device composed of a single device. May be.

  In addition, the present invention can realize the same processing as in the first to seventh embodiments with a computer program. In this case, each of the components including FIG. 1 may be functioned by a function or a subroutine executed by the CPU. Further, the computer program is usually stored in a computer-readable storage medium such as a CD-ROM, and is set in a reading device (CD-ROM drive or the like) included in the computer and copied or installed in the system. Become executable. Therefore, it is obvious that such a computer readable storage medium is also within the scope of the present invention.

Claims (17)

An input means for acquiring an input image;
Clipping means for cutting out M gradation ranges from the input image to generate M output images;
Display means for displaying the M output images as M sub-frames at M-times speed;
An image display device comprising: Furthermore, it has a gradation conversion means for gradation-converting each of the M output images,
2. The image display apparatus according to claim 1, wherein the display unit displays M output images converted by the gradation conversion unit as M sub-frames at M times speed.   The image display apparatus according to claim 2, wherein the gradation converting unit performs conversion so that each of the M output images has the same number of gradations.   The image display apparatus according to claim 1, wherein the M gradation ranges are adjacent to each other and have the same size. M = 2.
The M gradation ranges are a range from the first gradation to a second gradation larger than the first gradation, and a third gradation from the third gradation larger than the third gradation. Range up to 4 gradations,
4. The image display device according to claim 1, wherein the fourth gradation is larger than the second gradation. 5.   The image display apparatus according to claim 5, wherein the fourth gradation is smaller than a maximum gradation of the input image.   The image display apparatus according to claim 5, wherein the first gradation is larger than a minimum gradation of the input image.   The image display apparatus according to claim 5, wherein the fourth gradation is larger than a maximum gradation of the input image.   The image display apparatus according to claim 5, wherein the second gradation is adjacent to the third gradation.   6. The image display device according to claim 5, wherein the second gradation is larger than the third gradation.   The image display apparatus according to claim 10, wherein the first gradation is smaller than the third gradation. Input means for acquiring M images of the same scene taken at different exposure levels;
Display means for displaying the M images as M sub-frames at M-times speed;
An image display device comprising:   The image display apparatus according to claim 1, wherein the input unit inputs an image captured by a high dynamic range camera. An input step for obtaining an input image;
A step of cutting out M gradation ranges from the input image to generate M output images;
A display step of displaying the M output images as M sub-frames at M-times speed;
An image display method characterized by comprising: An input step of acquiring M images of the same scene taken with different exposure levels;
A display step of displaying the M images as M sub-frames at M times speed;
An image display method characterized by comprising:   The program for functioning a computer as each means in the image display apparatus of any one of Claims 1 thru | or 13.   The computer-readable recording medium which recorded the program of Claim 16.

Images