JP6250970B2 - Image processing apparatus, imaging apparatus, image processing method, program, and recording medium - Google Patents

Description

  The present invention relates to an image processing device, an imaging device, an image processing method, a program, and a recording medium.

  Currently, a technique for expanding the dynamic range of image data is known. Patent Document 1 proposes a control method for calculating a luminance conversion function for dynamic range correction and converting the luminance of the entire image in order to make a specific image included in a recorded image a desired brightness. ing.

  In addition, as an imaging device such as a digital video camera or a digital camera, a device equipped with a display device such as a liquid crystal display or an organic EL display (panel) or an electronic viewfinder (EVF) is widely used for confirming a subject image being shot. doing. Such an image pickup apparatus displays a subject image being photographed by a user on a display device, and this display is generally performed with an image quality when image data is recorded.

JP 2010-193513 A

  The user of the imaging apparatus may intentionally record a dark image or a bright image. For example, when recording a dark image, the dark image is also displayed on the display device of the imaging device. Therefore, it is difficult for the user to check the gradation of the subject image being shot. Even if the luminance of the display device is increased, the gradation reproducibility is not improved, so it is still difficult to confirm the gradation.

  Further, Patent Document 1 is not intended to confirm the subject image being shot. Even if Patent Document 1 is used for confirming the subject image being shot, the luminance of the entire image is converted, so that the displayed image may be significantly different from the actually recorded image. is there.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a technique that makes it easy for a user to check the gradation of an image being shot.

In order to solve the above problems, the present invention provides a developing unit that generates a developed image by developing image data generated by an imaging unit, and an acquiring unit that acquires a partial image from a specific region of the developed image. , An enlargement unit that enlarges the dynamic range of the luminance of the acquired partial image, a superimposing unit that superimposes the partial image with the expanded dynamic range on the developed image, and a developed image on which the partial image is superimposed is a first image. a and output means for outputting to the display device, the image processing apparatus Ru wherein the development with the developing means includes a gamma correction corresponding to a gamma characteristic of the second display device, the image processing apparatus, Recording means for recording the developed image; and inverse gamma correction means for performing reverse gamma correction corresponding to the gamma characteristic of the second display device on the developed image. For example, the acquisition unit may provide an image processing apparatus, characterized by acquiring the partial image from the inverse gamma corrected developed image.

  Other features of the present invention will become more apparent from the accompanying drawings and the following description of the preferred embodiments.

  According to the present invention, it is possible to make it easy for the user to check the gradation of the image being shot.

1 is a block diagram showing a configuration of an image processing apparatus 10 according to a first embodiment. The block diagram which shows the structure of the image processing apparatus 20 which concerns on 2nd Embodiment. The figure which shows the process which expands the dynamic range of the whole image. 3 is a flowchart showing image processing by the image processing apparatus 10; 5 is a flowchart showing image processing by the image processing apparatus 20; (A) a diagram showing development gamma and reverse development gamma, (b) a diagram showing display gamma and reverse display gamma, (c) a diagram showing an example of a gamma curve for dynamic range expansion, and (d) a diagram for dynamic range expansion. The figure which shows the other example of a gamma curve. Explanatory drawing of the process which superimposes the partial image in which the dynamic range was expanded on a development image.

  Embodiments of the present invention will be described below with reference to the accompanying drawings. The technical scope of the present invention is determined by the claims, and is not limited by the following individual embodiments. In addition, not all combinations of features described in the embodiments are essential to the present invention.

[First Embodiment]
FIG. 1A is a block diagram illustrating a configuration of an image processing apparatus 10 according to the first embodiment. In FIG. 1A, a sensor 100 is an image sensor such as a CCD or a CMOS. The sensor 100 photoelectrically converts a captured image formed by adjusting the amount of incident light and focusing by a photographing lens group (not shown), and outputs a digital signal obtained by analog / digital conversion. In each pixel of the image sensor, one of R (red), G (green), and B (blue) color filters is arranged in a predetermined arrangement, for example, a Bayer arrangement or a honeycomb arrangement. An RGB image signal is output for each color. The sensor 100 may output RGB image signals by performing pixel addition processing for adding several pixels to reduce the output image as one pixel for high-speed output. When the image processing apparatus 10 includes the sensor 100, the image processing apparatus 10 can operate as an imaging apparatus such as a digital camera or a digital video camera. However, the image processing apparatus 10 may not include the sensor 100. In this case, the image processing apparatus 10 acquires RGB image signals from an external sensor.

  The development processing unit 101 receives an RGB image signal (image data), and performs RGB offset adjustment, gain adjustment, and development gamma correction processing. In the development gamma correction process, an image input value is converted into an output value as in the development gamma correction curve shown in FIG. The development gamma correction curve is a curve for generating a recorded image desired by the user in consideration of characteristics of the sensor 100 and a photographing lens group (not shown). By changing the curve, it is possible to generate a recorded image for display on a TV monitor, or a recorded image that reproduces the texture and gradation of a movie film. In other words, the development processing unit 101 performs gamma correction corresponding to the gamma characteristic of the display device that is to display the recorded image. Next, the development processing unit 101 converts the RGB image signal into a luminance signal (Y) and a color difference signal (Cb, Cr) and outputs them. The development processing unit 101 also performs lens distortion correction processing, image stabilization processing, noise reduction processing, and the like of the imaging apparatus.

  The encoding unit 102 receives the YCbCr image signal (development image), compresses and encodes it in accordance with the recording format, and outputs it to the recording unit 103. The recording unit 103 records the compression-coded image on a recording medium (not shown).

  The reverse development gamma unit 104 performs processing of the reverse characteristic gamma (the reverse development gamma in FIG. 4A) of the development gamma correction in the development processing unit 101, and converts the developed image into an image having linear characteristics. In other words, the reverse development gamma unit 104 performs reverse gamma correction corresponding to the gamma characteristic of the display device that is to display the recorded image.

  The display gamma unit 105 applies a gamma correction (shown in FIG. 4B) corresponding to the gamma characteristic of the display device for EVF display (display gamma in FIG. 4B) to the developed image processed by the reverse development gamma. Process with reverse display gamma). As a result, the developed image is converted into an image having nonlinear characteristics.

  The image cutout unit 106 cuts out and acquires an image of a specified specific area from the developed image processed by the reverse development gamma, and transmits the cut image to the dynamic range expansion unit 107. The specific area is designated through the area designation unit 113. The area specifying unit 113 includes, for example, a user interface such as a panel. Alternatively, the area specifying unit 113 may detect a face or an object from the developed image and specify a specific area based on the detection result.

  The dynamic range expansion unit 107 obtains an average value of the luminance of the cut out partial images. Next, the dynamic range expansion unit 107 calculates a gamma curve that can expand the dynamic range of the luminance up to the specified expansion range with the average value of the luminance as the center value, and expands the dynamic range.

  The display gamma processing selection unit 108 converts the image to be displayed on the EVF into an output image from the development processing unit 101 and an output image from the display gamma unit 105 (inverse gamma correction is performed once by the reverse development gamma unit 104 and gamma correction is performed by the display gamma unit 105. Image).

  The superimposing unit 109 superimposes the partial image whose dynamic range has been expanded by the dynamic range expanding unit 107 on the image selected by the display gamma processing selection unit 108. At that time, the superimposing unit 109 may superimpose the partial image on a specific area where the image cutout unit 106 has acquired the partial image, or may superimpose the partial image on a different area. The display output unit 110 outputs the image superimposed by the superimposing unit 109 to a display device for EVF display. At that time, the display output unit 110 converts the format of the output image in accordance with the reception format of the display device panel. When an image is displayed on an external monitor, the display output unit 110 outputs an image signal whose format has been converted in accordance with an image signal transmission format such as HDMI (registered trademark), SDI, component or composite to the outside of the image processing apparatus 10. To do.

  Here, as a reference, processing for expanding the dynamic range of the entire image will be described with reference to FIG. In the image cutout unit 201, a partial image such as a face region is cut out from the input image 200. The luminance conversion function calculation unit 202 obtains a luminance range for expansion of the dynamic range and a conversion function. The dynamic range expansion process 203 converts the input image 200 according to the obtained function and generates a display image 204.

  Next, image processing by the image processing apparatus 10 will be described with reference to FIG. 3A. In the present embodiment, a scenario will be described as an example in which a user confirms gradation on a liquid crystal panel while shooting a moving image by the image processing apparatus 10 operating as a digital video camera. However, the image processing of this flowchart can be executed even before the start of photographing as long as the image processing apparatus 10 and the liquid crystal panel are activated. It can also be executed when shooting still images instead of moving images. Unless otherwise specified, the processing of each step in this flowchart can be performed by a CPU (not shown) of the image processing apparatus 10 executing a program stored in a ROM (not shown).

  In S301, the development processing unit 101 performs the development gamma correction processing described above with reference to FIG. 4A, converts the RGB image signal into a luminance signal (Y) and a color difference signal (Cb, Cr), and outputs the converted signal. .

  In S302, the reverse development gamma unit 104 performs a reverse gamma characteristic calculation process for development gamma correction, and converts the developed image into an image having a linear characteristic. In S303, the area specifying unit 113 specifies a specific area of the developed image, and the image cutout unit 106 cuts out a partial image from the specified specific area.

  In step S304, the dynamic range expansion unit 107 calculates the average value of the luminances of the partial images cut out in step S303. In S305, the dynamic range expansion unit 107 generates a gamma curve for dynamic range expansion centering on the average value of the luminance calculated in S304 as in the examples shown in FIGS. 4C and 4D. The dynamic range expansion gamma curve can arbitrarily change the input range width and the output range width (dynamic range expansion range) around the average value of luminance. These range widths are set by a control block such as a CPU, for example. For example, the dynamic range expansion unit 107 generates a gamma curve for gamma correction (reverse display gamma in FIG. 4B) corresponding to the gamma characteristic of the display device for EVF, the average value of the expansion range of the dynamic range and the luminance. Deforms based on As a result, a dynamic range expansion curve (FIG. 4C or FIG. 4D) can be generated. The gamma curve for expanding the dynamic range may be generated as shown in the examples shown in FIGS. 4C and 4D with the average value of each RGB component of the image as the center. In S306, the dynamic range expansion unit 107 performs dynamic range expansion processing based on the gamma curve generated in S305.

  In step S <b> 307, the image processing apparatus 10 determines whether the display gamma processing selection unit 108 has selected the output image of the display gamma unit 105. If the display gamma processing selection unit 108 has selected the output image of the display gamma unit 105, the process proceeds to S308. Otherwise, the process proceeds to S309.

  In S308, the display gamma unit 105 performs gamma correction (FIG. 4B) corresponding to the gamma characteristic of the display device for EVF display (display gamma in FIG. 4B) for the developed image processed by the reverse development gamma. (b) Process by reverse display gamma).

  In step S309, the superimposing unit 109 superimposes the partial image whose dynamic range has been expanded in step S306 on the developed image selected by the display gamma processing selecting unit 108. For example, as illustrated in FIG. 5A, the superimposing unit 109 superimposes the partial image on the specific area where the image clipping unit 106 has acquired the partial image. Alternatively, the superimposing unit 109 may superimpose the partial image in an area different from the specific area as illustrated in FIG. In this case, the user can confirm by comparing the original image with the image whose dynamic range is expanded. 5A and 5B, the superimposing unit 109 may enlarge the partial image and superimpose it on the developed image. 5A and 5B, unlike the case of FIG. 2, the dynamic range is expanded only for the partial image, not the entire developed image. In the case of FIG. 5B, the superimposing unit 109 may superimpose a frame line indicating a specific area from which the image cutout unit 106 has acquired the partial image on the developed image.

  In S310, the display output unit 110 outputs the developed image on which the partial image is superimposed to the display device for EVF display. As a result, an image is displayed on the display device as shown in FIGS. 5 (a) and 5 (b). In the case of the configuration of the present embodiment, processing from S301 to S310 is performed within one frame time, and it is possible to perform shooting assistance display for gradation confirmation in real time. Thereafter, the process returns to S301, and the same process is performed for the next frame.

  If the EVF display device (first display device) and the recorded image display device (second display device) have different gamma characteristics, the output image of the development processing unit 101 is directly used as the EVF display device. If displayed, it may not be displayed with proper gradation. According to the image processing shown in FIG. 3A, the partial images shown in FIGS. 5A and 5B are subjected to the reverse development gamma processing in S302, and in S306, the dynamic considering the gamma characteristics of the display device for EVF. It is an image that has undergone range expansion processing. Therefore, even if the EVF display device and the recorded image display device have different gamma characteristics, the user can easily confirm the gradation of the partial image.

  Note that the image processing apparatus 10 may be configured to control synchronization of the driving cycle of the sensor 100 and the panel driving cycle. This configuration may be a configuration in which the cycle control and the delay time control can be controlled by the sensor 100 and the panel, respectively. Specifically, the delay amount of the synchronization signal between the sensor 100 and the panel is measured, and the delay amount of the synchronization signal of the panel drive cycle is adjusted within a predetermined range with the sensor drive cycle as a reference.

  The image processing apparatus 10 also has a control block such as a CPU for controlling each block, and this control block performs setting adjustment such as development processing, reverse development gamma processing, area designation processing, display gamma processing, and the like. .

  The development processing unit 101 may receive a YCbCr image signal instead of an RGB image signal. In this case, the development processing unit 101 converts the YCbCr image signal into an RGB image signal and then performs development processing. Alternatively, the development process with the RGB image signal may be omitted.

  As described above, according to the present embodiment, the image processing apparatus 10 acquires a partial image from a specific area of the developed image, expands the dynamic range of the partial image, and superimposes it on the developed image. Thereby, the user can easily confirm the gradation of the image being shot.

[Second Embodiment]
In the first embodiment, the number of specific areas from which partial images are acquired is one. In the second embodiment, a configuration will be described in which there are a plurality of specific regions and partial images are acquired from each. Hereinafter, the points different from the first embodiment will be mainly described.

  FIG. 1B is a block diagram illustrating a configuration of an image processing apparatus 20 according to the second embodiment. In FIG. 1B, the same reference numerals are assigned to the same blocks as in FIG. 1A.

  The memory 1 121 temporarily stores the developed image after the reverse development gamma processing. The area designating unit 122 designates a plurality of specific areas in the developed image. The image cutout unit 123 cuts out a partial image from each of a plurality of specific areas. The dynamic range expansion unit 124 performs a dynamic range expansion process on each partial image. The memory 2 125 temporarily stores a plurality of partial images with an expanded dynamic range. The superimposing unit 126 superimposes each of the plurality of partial images stored in the memory 2 125 on the developed image.

  Next, image processing by the image processing apparatus 20 will be described with reference to FIG. 3B. In FIG. 3B, the same steps as those in FIG. 3A are denoted by the same reference numerals. Unless otherwise specified, the processing of each step in this flowchart can be performed by the CPU (not shown) of the image processing apparatus 20 executing a program stored in a ROM (not shown). In the present embodiment, the developed image that has been subjected to the reverse development gamma processing in S <b> 302 is stored in the memory 1 121.

  In step S <b> 401, the image processing apparatus 20 determines whether the processing has been completed for all of the plurality of areas specified by the area specifying unit 122. If the process has been completed for all areas, the process proceeds to S307. Otherwise, the process proceeds to S402.

  In S402 to S405, the same processing as S303 to S306 in FIG. 3A is performed on one of the plurality of areas designated by the area designating unit 122, and the finally obtained partial image is stored in the memory 2125. Stored. By repeating the processing from S 401 to S 405, partial images corresponding to each of the plurality of areas specified by the area specifying unit 122 are stored in the memory 2 125.

  In step S <b> 406, the superimposing unit 126 superimposes each of the plurality of partial images stored in the memory 2 125 on the developed image selected by the display gamma processing selection unit 108. At that time, as illustrated in FIG. 5C, the superimposing unit 126 superimposes each partial image on the specific area where the image clipping unit 123 has acquired each partial image. Alternatively, the superimposing unit 126 may superimpose each partial image on an area different from the specific area. In this case, the superimposing unit 126 may superimpose a frame line indicating a specific area where the partial image is acquired by the image cutout unit 123 on the developed image.

  Note that the memory 1 121 and the memory 2 125 may be implemented by physically separate memories, or may be implemented by a configuration in which one memory is used by address control.

  As described above, according to the present embodiment, the image processing apparatus 20 acquires partial images from each of a plurality of specific regions of the developed image, expands the dynamic range of each partial image, and superimposes the developed images on the developed image. . Thereby, the user can easily confirm the gradation of the image being shot.

[Other Embodiments]
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

Claims (13)

Developing means for generating a developed image by developing image data generated by the imaging means;
Obtaining means for obtaining a partial image from a specific region of the developed image;
Expansion means for expanding the dynamic range of the luminance of the acquired partial image;
Superimposing means for superimposing the partial image with the expanded dynamic range on the developed image;
Output means for outputting the developed image on which the partial image is superimposed to the first display device;
An image processing apparatus Ru provided with,
The development by the developing means includes gamma correction corresponding to the gamma characteristic of the second display device,
The image processing apparatus includes:
Recording means for recording the developed image;
Reverse gamma correction means for performing reverse gamma correction corresponding to the gamma characteristic of the second display device on the developed image;
Further comprising
The acquisition unit acquires the partial image from the developed image subjected to the inverse gamma correction.
An image processing apparatus. The image processing apparatus according to claim 1, wherein the enlarging unit expands the dynamic range around an average value of luminance of the partial images. The expansion means uses a gamma curve obtained by transforming a gamma curve for gamma correction corresponding to the gamma characteristic of the first display device based on the expansion range of the dynamic range and the average value. The image processing apparatus according to claim 2, wherein the dynamic range is expanded. The gamma correction part which performs the gamma correction corresponding to the gamma characteristic of the said 1st display apparatus with respect to the said reverse-gamma-corrected developed image is further provided. The Claim 1 thru | or 3 characterized by the above-mentioned. Image processing apparatus. The development apparatus according to claim 1, further comprising: a selection unit that selects a development image generated by the development unit or the reverse gamma corrected and the gamma corrected development image as the development image on which the partial image is superimposed by the superimposition unit. Item 5. The image processing apparatus according to Item 4 . The superposition unit, an image processing apparatus according to any one of claims 1 to 5, characterized in that superimposes the partial image in the specific area of the developed image. The superposition unit, an image processing apparatus according to any one of claims 1 to 5, characterized in that superimposes the partial image area different from the specific area of the developed image. The image processing apparatus according to any one of claims 1 to 7, further comprising a specifying means for specifying the specific area. The acquisition unit, an image processing apparatus according to any one of claims 1 to 8, characterized in that to obtain a partial image from each of a plurality of specific areas. An image processing apparatus according to any one of claims 1 to 9 ,
The imaging means;
An imaging apparatus comprising: An image processing method by an image processing apparatus,
A developing step in which the developing unit of the image processing apparatus generates a developed image by developing the image data generated by the imaging unit;
An acquisition step in which the acquisition unit of the image processing apparatus acquires a partial image from a specific region of the developed image;
An enlarging step in which the enlarging means of the image processing device expands the dynamic range of the luminance of the acquired partial image;
A superimposing step in which the superimposing unit of the image processing apparatus superimposes the partial image with the expanded dynamic range on the developed image;
An output step in which the output means of the image processing device outputs the developed image on which the partial image is superimposed to the first display device;
Equipped with a,
The development by the development step includes gamma correction corresponding to the gamma characteristic of the second display device,
The image processing method includes:
A recording step in which the recording means of the image processing apparatus records the developed image; and
A reverse gamma correction step in which a reverse gamma correction unit of the image processing apparatus performs a reverse gamma correction corresponding to the gamma characteristic of the second display device on the developed image;
Further comprising
In the obtaining step, the partial image is obtained from the developed image subjected to the inverse gamma correction.
An image processing method. The program for functioning a computer as each means of the image processing apparatus of any one of Claim 1 thru | or 9 . A computer-readable recording medium storing a program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 9 .

Images