JP2019205063A - Imaging apparatus and control method and program thereof - Google Patents

Abstract

To provide a technique capable of reducing impact on the lightness of the other image signal even if the lightness of one image signal is changed, when handling multiple image signals having different gradation characteristics.SOLUTION: An imaging apparatus has setting means for setting a first exposure reference value in a first image signal, and a second exposure reference value in a second image signal having gradation characteristics different from those of the first image signal, brightness control means for controlling the brightness of the first image signal on the basis of the first exposure reference value thus set, control means for controlling gray scale conversion characteristics, and gray scale conversion means for converting an image signal, subjected to brightness change by the brightness control means, into a second image signal by using gradation conversion characteristics. The control means finds gradation conversion characteristics for restraining brightness change of the second image signal according to the discrepancy of the first and second exposure reference values.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an imaging apparatus, a control method thereof, and a program.

  In recent years, with the expansion of the displayable luminance range, Rec. ITU-R BT. Devices such as displays and cameras corresponding to HDR (High Dynamic Range) standards such as 2100 are increasing. On the other hand, since there are many devices that only support the conventional SDR (standard dynamic range) standard, the video producer may be forced to create both the video corresponding to the HDR standard and the video corresponding to the SDR standard. is there. In order to reduce such a burden of video production, a video conversion device having a function of inputting an HDR signal, converting it into an SDR signal, and outputting it has been proposed (Patent Document 1). In Patent Document 1, a code value of an input HDR signal is converted into a luminance value using an HDR EOTF, and the converted luminance value is further converted into another luminance value related in advance, and then an SDR EOTF is used. A technique for converting to an SDR code value is proposed. Note that EOTF (Electro-Optical Transfer Functions) is also referred to as an electro-optical transfer function.

Japanese Patent No. 5948618

  However, in the technique proposed in Patent Document 1, since the conversion between luminance values is related in advance, when the HDR brightness is changed, the SDR brightness also changes in conjunction. Since HDR has an expanded dynamic range, there is a large room for allowing a change in brightness during shooting according to the intention of the photographer to express. On the other hand, since the SDR has relatively little room for allowing the brightness to change, when the SDR brightness is linked to the HDR, an unintended whiteout may occur on the SDR side.

  The present invention has been made in view of the above problems, and its object is to handle the brightness of one image signal even when the brightness of one image signal is changed when handling a plurality of image signals having different gradation characteristics. It is to realize a technology capable of reducing the influence exerted.

  In order to solve this problem, for example, an image processing apparatus of the present invention has the following configuration. That is, setting means for setting a first exposure reference value in the first image signal and a second exposure reference value in the second image signal having gradation characteristics different from those of the first image signal are set. Further, brightness control means for controlling the brightness of the first image signal based on the first exposure reference value, control means for controlling gradation conversion characteristics, and brightness control by the control of the brightness control means. Gradation conversion means for converting the changed image signal into the second image signal using the gradation conversion characteristics, and the control means includes the first exposure reference value and the second exposure signal. The gradation conversion characteristic that suppresses the change in the brightness of the second image signal according to the difference in the exposure reference value is obtained.

  According to the present invention, when a plurality of image signals having different gradation characteristics are handled, even if the brightness of one image signal is changed, the influence on the brightness of the other image signal can be reduced. Become.

1 is a block diagram illustrating a functional configuration example of a digital camera as an example of an imaging apparatus according to Embodiment 1. FIG. FIG. 2 is a block diagram illustrating an example functional configuration of an image processing unit, an image conversion unit, a system control unit, and an operation unit in the first embodiment. Table showing an example of a three-dimensional lookup table in the first embodiment 7 is a flowchart showing a series of operations related to conversion characteristic calculation processing in the first embodiment. FIG. 6 is a diagram for explaining gradation conversion characteristics set in the first gradation conversion unit in the first embodiment. FIG. 9 is a block diagram illustrating an example of functional configurations of an image processing unit, a system control unit, and an operation unit in the second embodiment. 7 is a flowchart showing a series of operations related to conversion characteristic calculation processing in the second embodiment.

<Embodiment 1>
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. In the following, an example in which a digital camera capable of converting the gradation of an image signal is used as an example of an imaging apparatus will be described. However, this embodiment can be applied not only to a digital camera but also to other devices that can convert the gradation of an image signal. These devices may include, for example, a mobile phone including a smartphone, a game machine, a tablet terminal, a clock-type or glasses-type information terminal, a medical device, a monitoring system, an in-vehicle system device, and the like.

(Configuration of digital camera)
FIG. 1 is a block diagram illustrating a functional configuration example of a digital camera as an example of the imaging apparatus of the present embodiment. One or more of the functional blocks shown in FIG. 1 may be realized by hardware such as an ASIC or a programmable logic array (PLA), or may be realized by a programmable processor such as a CPU or MPU executing software. May be. Further, it may be realized by a combination of software and hardware. Therefore, in the following description, even when different functional blocks are described as the operation subject, the same hardware can be realized as the subject.

  The photographing lens 102 is a lens group including a zoom lens and a focus lens, and forms a subject image. An aperture 103 is an aperture used for light amount adjustment. An ND 104 is an ND filter used for dimming. The imaging unit 105 is an imaging device configured by a CCD, a CMOS device, or the like that converts an optical image into an electrical signal. The imaging unit 105 also has functions such as accumulation control using an electronic shutter, change of analog gain, and readout speed. The A / D converter 106 converts an analog signal into a digital signal. The A / D converter 106 is used to convert an analog image signal output from the imaging unit 105 into a digital image signal. The barrier 101 covers the imaging system including the imaging lens 102 of the digital camera 100, thereby preventing the imaging system including the imaging lens 102, the aperture 103, the ND 104, and the imaging unit 105 from being dirty or damaged.

  The image processing unit 115 performs various image processing such as color conversion processing, gamma correction, and digital gain addition on the data from the A / D converter 106 or the data from the memory control unit 116. In addition, predetermined calculation processing is performed using the captured image data, and the calculation result is transmitted to the system control unit 114. Based on the transmitted calculation result, the system control unit 114 performs exposure control, ranging control, white balance control, and the like. As a result, TTL (through-the-lens) AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, and the like are performed. Details of the image processing unit 115 will be described later.

  Output data from the A / D converter 106 is directly written into the memory 119 via the image processing unit 115 and the memory control unit 116 or via the memory control unit 116. The memory 119 stores an image signal captured by the imaging unit 105 and converted into a digital signal by the A / D converter 106 and an image signal to be displayed on the display unit 118. The memory 119 has a storage capacity sufficient to store a moving image and sound for a predetermined time.

  The memory 119 also serves as an image display memory (video memory). The D / A converter 117 converts the image display data stored in the memory 119 into an analog signal and supplies the analog signal to the display unit 118. Thus, the display image data written in the memory 119 is displayed on the display unit 118 via the D / A converter 117. The display unit 118 includes a display such as an LCD, for example, and performs display according to the analog signal from the D / A converter 117 on the display. A digital signal once A / D converted by the A / D converter 106 and stored in the memory 119 is converted into an analog signal by the D / A converter 117, and sequentially transferred to the display unit 118 for display as an electronic viewfinder. It can function and display a through image.

  The nonvolatile memory 113 is an electrically erasable / recordable memory, and for example, an EEPROM is used. The nonvolatile memory 113 stores constants, programs, and the like for operating the system control unit 114. Here, the program is a program for executing various flowcharts described later in the embodiment of the present invention.

  The system control unit 114 includes, for example, a CPU or MPU, and develops and executes a program recorded in the non-volatile memory 113 in the system memory 121, thereby controlling the entire digital camera 100 and controlling the operation of conversion processing described later. To do. Note that the system control unit 114 may also have the function of the image processing unit 115.

  The system memory 121 includes, for example, a RAM, and develops constants and variables for operation of the system control unit 114, a program read from the nonvolatile memory 113, and the like. The system control unit 114 also performs display control by controlling the memory 119, the D / A converter 117, the display unit 118, and the like.

  The system timer 120 is a time measuring unit that measures the time used for various controls and the time of a built-in clock. A mode switch 109, a recording switch 108, and an operation unit 107 are operation means for inputting various operation instructions to the system control unit 114.

  The mode switch 109 switches the operation mode of the system control unit 114 to any one of a moving image recording mode, a still image recording mode, a reproduction mode, and the like. As modes included in the moving image recording mode and the still image recording mode, there are an auto shooting mode, an auto scene discrimination mode, a manual mode, various scene modes for shooting settings for each shooting scene, a program AE mode, a custom mode, and the like. The mode changeover switch 109 can be directly switched to one of these modes included in the moving image shooting mode. Alternatively, after the mode changeover switch 109 temporarily switches to the moving image shooting mode, it may be switched to any of these modes included in the moving image shooting mode using another operation member. The recording switch 108 switches between a shooting standby state and a shooting state. The system control unit 114 starts a series of operations from reading a signal from the imaging unit 105 to writing moving image data to the recording medium 123 by the recording switch 108. The power switch 110 is a switch for switching on / off the power of the digital camera 100.

  Each operation member of the operation unit 107 is appropriately assigned a function for each scene by selecting and operating various function icons displayed on the display unit 118, and functions as various function buttons. Examples of the function buttons include an end button, a return button, an image advance button, a jump button, a narrowing button, and an attribute change button. For example, when a menu button is pressed, various setting menu screens are displayed on the display unit 118. The user can make various settings intuitively using the menu screen displayed on the display unit 118, the four-way key in the four directions of up / down / left / right, and the SET button. The operation unit 107 operates as a setting unit that sets an exposure reference value as will be described later.

  The power control unit 111 includes a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, and the like, and detects whether or not a battery is installed, the type of battery, and the remaining battery level. The power supply control unit 111 controls the DC-DC converter based on the detection result and an instruction from the system control unit 114, and supplies a necessary voltage to each unit including the recording medium 123 for a necessary period. The power supply unit 112 includes, for example, a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li ion battery, an AC adapter, or the like. The I / F 122 is an interface with a recording medium 123 such as a memory card or a hard disk, or an external output device. In the example illustrated in FIG. 1, a state in which the recording medium 123 is connected is illustrated. The recording medium 123 is a recording medium for recording a captured image or video, and is composed of, for example, a semiconductor memory or a magnetic disk.

  Next, a functional configuration example of the image processing unit 115, the image conversion unit 124, the system control unit 114, and the operation unit 107 in the present embodiment will be described with reference to FIG.

  Each block included in the image processing unit 115 and the image conversion unit 124 can acquire all data in the imaging apparatus through the system control unit 114.

  An exposure reference value setting unit 206 included in the operation unit 107 sets an exposure reference value of a signal output from the image processing unit 115. On the other hand, an exposure reference value setting unit 209 included in the operation unit 107 sets an exposure reference value of a signal output from the image conversion unit 124. The exposure reference value represents the correspondence between the brightness of the subject (reflectance of the subject) and the luminance (display luminance) output from the display device.

  A brightness control unit 207 included in the system control unit 114 controls the brightness of a signal output from the image processing unit 115 based on the exposure reference value set by the exposure reference value setting unit 206. The system control unit 114 controls the imaging unit 105, the diaphragm 103, the ND 104, the gain adjustment unit 201, the gradation conversion unit 203, and the like in order to control the brightness of the signal output from the image processing unit 115. The conversion characteristic calculation unit 208 uses the exposure reference values set by the exposure reference value setting unit 206 and the exposure reference value setting unit 209 to calculate the gradation conversion characteristics set in the gradation conversion unit 204.

  The gain adjustment unit 201 performs gain adjustment and white balance correction by individually applying a gain to each of the RGB signals with respect to the signal output from the A / D converter 106. The matrix calculation unit 202 corrects the color tone by applying a matrix to the RGB signal output from the gain adjustment unit 201. Thereby, the difference in spectral sensitivity and the like of the imaging unit 105 is corrected, and color reproduction suitable for the digital camera 100 is realized.

  A gradation conversion unit 203 included in the image processing unit 115 performs gradation conversion by applying common gradation conversion characteristics to the RGB signals output from the matrix calculation unit 202. The output from the gradation conversion unit 203 is output to the recording medium 123 through the image conversion unit 124 and the I / F 122.

  The gradation conversion unit 204 included in the image conversion unit 124 applies the gradation conversion characteristics set by the system control unit 114 to each of the RGB signals input from the image processing unit 115 and has different gradation characteristics. Generate a signal. In the generation of the signal, gradation characteristic conversion is often performed with reference to a one-dimensional lookup table, but there is also a case of gradation characteristic conversion by calculation.

  The 3DLUT processing unit 205 performs color conversion with reference to, for example, a three-dimensional lookup table as shown in FIG. 3, and outputs the converted image signal. When the same RGB value as the input RGB value exists in the conversion source table, it is converted as it is into the conversion destination RGB value. If the input RGB value does not exist in the conversion source table, a plurality of neighboring points including the input RGB value are extracted from the conversion source table, and the conversion destination RGB value is obtained by interpolation and converted.

(Conversion characteristics calculation process)
Next, the conversion characteristic calculation process for determining the gradation conversion process in the image conversion unit 124 will be described with reference to FIG. Note that this processing is realized by the system control unit 114 controlling each unit of the digital camera 100 by developing and executing a program recorded in the nonvolatile memory 113 in the system memory 121. Further, unless otherwise specified, each step of this process is executed by the system control unit 114. This process is started, for example, when the system control unit 114 receives an operation instruction from the user via the operation unit 107.

  In the present embodiment, an example of calculating characteristics of conversion processing from an HDR signal (also referred to as a first image signal) to an SDR signal (also referred to as a second image signal) will be described. However, conversion from an SDR signal to an HDR signal, conversion between HDR signals, conversion between SDR signals, and the like may also be included. Further, not only the HDR signal and the SDR signal, but also a Log signal having logarithmic gradation characteristics may be included. In this embodiment, it is assumed that a 3DLUT for performing only color correction processing excluding tone conversion is fixedly set in the 3DLUT processing unit 205 in advance. However, the 3DLUT setting is linked to the processing of this embodiment. The form which changes may also be included.

  In S <b> 41, the exposure reference value setting unit 206 sets an exposure reference value (also referred to as a first exposure reference value) of the HDR signal generated by the image processing unit 115 in the system control unit 114. In the present embodiment, for example, when a subject with a reflectance of 18% is photographed with appropriate exposure, the brightness of the subject is set by how many nits the brightness is displayed from the display device. However, it is not always necessary to use the reflectance of 18% as a reference, and any numerical value may be used as a reference. Further, not only the setting by the user via the exposure reference value setting unit 206, but the system control unit 114 may automatically set a predetermined value or the like.

  In S <b> 42, the exposure reference value setting unit 209 sets an exposure reference value (also referred to as a second exposure reference value) of the SDR signal output from the image conversion unit 124 in the system control unit 114. As in S41, in the present embodiment, for example, when a subject with a reflectance of 18% is photographed with appropriate exposure, the brightness of the subject is displayed by the number of nits displayed from the display device. However, it is not always necessary to use the reflectance of 18% as a reference, and any numerical value may be used as a reference. Further, the setting is not limited to the user setting via the exposure reference value setting unit 209, and the system control unit 114 may automatically set a predetermined value or the like. In the present embodiment, the exposure reference value for the SDR signal is set so that a subject with a reflectance of 18% is displayed with a brightness of 18 nits.

  In S43, the system control unit 114 (brightness control unit 207) controls the brightness of the signal generated by the image processing unit 115 based on the exposure reference value of the HDR signal set in S41. Here, in the present embodiment, a case will be considered in which a state in which an object with a reflectance of 18% is displayed at a brightness of 18 nits is changed to a state in which an object with a reflectance of 18% is displayed at a brightness of 36 nits. As a method for controlling the brightness, for example, (i) a method of changing a gradation conversion characteristic set in the gradation conversion unit 203 and (ii) a gradation conversion characteristic set in the gradation conversion unit 203 are changed. There is a method of changing the input value to the gradation conversion unit 203 without any change.

  First, with reference to FIG. 5A, (i) a method of changing the gradation conversion characteristics set in the gradation conversion unit 203 will be described. FIG. 5A shows the gradation conversion characteristics set in the gradation conversion unit 203. In FIG. 5A, an input value 501 represents an input value corresponding to a subject having a reflectance of 18%. The output value 504 represents an output value corresponding to a brightness of 18 nits, and the output value 505 represents an output value corresponding to a brightness of 36 nits. The input maximum value 506 represents the maximum value of the input value.

  The first gradation conversion characteristic 502 is a gradation conversion characteristic for displaying an object having a reflectance of 18% with a brightness of 18 nits. That is, in the HDR signal, when a subject with a reflectance of 18% is displayed with a brightness of 18 nit, the first gradation conversion characteristic 502 is set in the gradation conversion unit 203. Since the gradation conversion characteristics can be defined even when the input maximum value 506 is exceeded, the output value when the input maximum value 506 is exceeded is indicated by a dotted line.

  The second gradation conversion characteristic 503 is a gradation conversion characteristic for displaying an object having a reflectance of 18% with a brightness of 36 nits. Therefore, in the HDR signal, if the second gradation conversion characteristic 503 is set for the gradation converting unit 203, an object with a reflectance of 18% is displayed with a brightness of 36 nits.

  For this reason, the system control unit 114 changes the conversion characteristics (first gradation conversion characteristic 502, second gradation conversion characteristic 503, etc.) stored in the system memory 121 in advance according to the exposure reference value set in S41. The gradation conversion characteristic to be selected and set in the gradation conversion unit 203 is changed. Alternatively, the system control unit 114 may calculate a gradation conversion characteristic corresponding to the setting based on the reference gradation conversion characteristic.

  Next, (ii) a case where the brightness of a signal generated by the image processing unit 115 is controlled by changing an input value to the gradation conversion unit 203 will be described. FIG. 5B shows the gradation conversion characteristics set in the gradation conversion unit 203 as in FIG. As in the case of changing the gradation conversion characteristics set in the gradation conversion unit 203, when an object with a reflectance of 18% is set to be displayed with a brightness of 18 nit, the input value 501 has a reflectance of 18%. Is an input value corresponding to the subject.

  When changing the input value to the gradation converting unit 203 to set the display brightness corresponding to the subject with 18% reflectance to 36 nits, at least one of the aperture 103, the ND 104, the imaging unit 105, and the gain adjusting unit 201 is controlled. . As a result, the input value corresponding to the reflectance of 18% is moved to the input value 507. That is, changing the input value to the gradation conversion unit 203 corresponds to changing the appropriate exposure of the digital camera 100. The input value 507 is associated with an output value 505 that is an output value corresponding to a brightness of 36 nits by the first gradation conversion characteristic 502. For this reason, when an object with a reflectance of 18% is photographed under this condition, it is displayed with a brightness of 36 nits.

  As described above, the brightness of the signal generated by the image processing unit 115 based on the exposure reference value of the HDR signal set in S41 by either the method (i) or (ii) described above. To control.

In S44, the system control unit 114 (conversion characteristic calculation unit 208) is included in the gradation conversion characteristics set in the gradation conversion unit 203 from the difference between the exposure reference value set in S41 and the exposure reference value set in S42. Calculate the gain. For example, it is assumed that the exposure reference value is set so that a subject with a reflectance of 18% is displayed at 36 nits in S41, and the subject with a reflectance of 18% is set to be displayed at 18 nits in S42.
At this time, the desired gain k is
k = 18 nit / 36 nit = 0.5 Formula (1)
It becomes.

  In S45, the system control unit 114 (conversion characteristic calculation unit 208) uses the gradation conversion characteristic set in the gradation conversion unit 203 in S43, the gain calculated in S44, and the gradation conversion characteristic of the SDR signal. The gradation conversion characteristics set in the gradation conversion unit 204 are calculated. Here, an EOTF (electro-optical transfer function) representing the relationship between the output IRE (the luminance signal level with white being 0 and black being 100) and the reflectance corresponding to the gradation conversion characteristics controlled in S43 is defined as EOTFHDR. Further, EOTF representing the relationship between the output IRE corresponding to the SDR signal and the reflectance is defined as EOTFFSDR. At this time, the relationship between the HDR output IRE x and the SDR output IRE f (x) is expressed by the following equation using the gain k obtained in S44.

  In Expression (2), for example, when the exposure reference value of the HDR signal is increased, the gain k according to the difference in the exposure reference value between the signals is obtained by the brightness of the HDR signal obtained in S43 (in this case, 0 <k <1) is multiplied. That is, in Expression (2), the change in the brightness of the SDR signal is suppressed by the gain k (the effect of the change in the HDR signal is canceled by the gain k). Therefore, the brightness of the SDR signal is kept substantially constant according to the exposure reference value of the SDR signal regardless of the brightness of the HDR signal. In S43, whether the reflection of the exposure reference value is performed by either (i) changing the gradation conversion characteristic or (ii) changing the input value is the same as the output IRE, the expression (3) ) Based gradation conversion characteristics are the same in both cases.

  In S <b> 46, the system control unit 114 sets the gradation conversion characteristics obtained in S <b> 45 in the gradation conversion unit 204. At this time, since the gradation conversion characteristic obtained in S45 is the conversion characteristic between the output IREs, it is necessary to set the conversion characteristic between the output values. Since the relationship between the output value and the output IRE is predetermined, the conversion may be executed based on, for example, a table. Thereby, since the difference in the exposure reference value of the HDR signal can be canceled out by the gradation converting unit 204, the brightness of the SDR signal is substantially constant (that is, the brightness of the SDR signal is shot with appropriate exposure set to SDR). Is equivalent to the Thereafter, the system control unit 114 ends this process.

  As described above, according to the present embodiment, even when the exposure reference value of the HDR signal is changed by a user operation or the like, the brightness fluctuation of the SDR signal generated by converting the HDR signal is changed. Can be reduced and the brightness can be maintained. Therefore, when the photographer changes the brightness of the HDR signal for the purpose of expression, it is possible to prevent the brightness information of the SDR signal from collapsing due to the brightness of the SDR signal changing in conjunction with the brightness. In other words, when dealing with a plurality of image signals having different gradation conversion characteristics, it is possible to reduce the influence on the brightness of the conversion destination image signal even if the brightness of the conversion source image signal is changed. Become. Also, if the exposure reference value for the SDR signal is changed, the brightness of the SDR signal can be changed independently of the HDR signal. That is, the user can take a picture without being aware of the relationship between the HDR signal and the SDR signal.

<Embodiment 2>
Next, Embodiment 2 will be described. In the first embodiment, when the SDR signal is generated based on the HDR signal, the brightness of the SDR signal is determined based on the exposure reference value of the SDR signal by the conversion characteristic calculation process regardless of the exposure reference value of the HDR signal. To keep on. In the second embodiment, the HDR signal and the SDR signal are generated based on the output from the A / D converter 106, respectively. In this case, the brightness of the SDR signal is kept at the brightness based on the exposure reference value of the SDR signal regardless of the exposure reference value of the HDR signal. The digital camera 100 of the present embodiment is the same as that of the first embodiment except that the system control unit 114 and the image processing unit 115 are partially different in configuration. For this reason, the same reference numerals are assigned to the same components, and redundant description is omitted, and differences will be described mainly. In the present embodiment, since both the HDR signal and the SDR signal are generated in the image processing unit 115, the image conversion unit 124 does not perform the conversion process.

  First, a functional configuration example of the system control unit 114 and the image processing unit 115 according to the present embodiment will be described with reference to FIG. Each block of the image processing unit 115 can acquire all data in the imaging apparatus through the system control unit 114.

  A brightness control unit 608 of the system control unit 114 controls the brightness of the image signal output from the gradation conversion unit 203 based on the exposure reference value set by the exposure reference value setting unit 206. In order to control the brightness, at least one of the imaging unit 105, the diaphragm 103, the ND 104, the gain adjustment unit 201, and the gradation conversion unit 203 is controlled. The brightness control unit 609 controls the brightness of the SDR signal output from the gradation conversion unit 612 based on the exposure reference value set by the exposure reference value setting unit 206 and the exposure reference value setting unit 209. In order to control this brightness, at least one of the gain adjustment unit 610 and the gradation conversion unit 612 is controlled.

  The gain adjustment unit 610, matrix calculation unit 611, and gradation conversion unit 612 have the same configuration as the gain adjustment unit 201, matrix calculation unit 202, and gradation conversion unit 203. In the present embodiment, an HDR signal is generated by the system of the gain adjustment unit 201, the matrix calculation unit 202, and the gradation conversion unit 203. On the other hand, an SDR signal is generated by the system of the gain adjustment unit 610, the matrix calculation unit 611, and the gradation conversion unit 612.

(Conversion characteristics calculation process)
Next, with reference to FIG. 7, a conversion characteristic calculation process for the gradation conversion process (brightness control) in the gradation conversion unit 612 will be described. In the present embodiment, a case where an HDR signal is output from the gradation conversion unit 203 and an SDR signal is output from the gradation conversion unit 612 will be described as an example. However, the HDR signal and the SDR signal may be reversed, or both may be an HDR signal or an SDR signal. Further, not only the HDR signal and the SDR signal, but also a Log signal having logarithmic gradation characteristics may be included. Note that this processing is realized by the system control unit 114 controlling each unit of the digital camera 100 by developing and executing a program recorded in the nonvolatile memory 113 in the system memory 121. Further, unless otherwise specified, each step of this process is executed by the system control unit 114. This process is started, for example, when the system control unit 114 receives an operation instruction from the user via the operation unit 107.

  In S41 to S42, the exposure reference value setting unit 206 and the exposure reference value setting unit 209 set exposure reference values for HDR and SDR, respectively, as in the first embodiment.

  In S71, the system control unit 114 (brightness control unit 608) controls the brightness of the HDR signal based on the exposure reference value for the HDR signal set in S41. The brightness control method includes (i) a method of changing the brightness by controlling the gain adjustment unit 201 and the gradation conversion unit 203, and (ii) an A / A by controlling the imaging unit 105, the aperture 103, and the ND 104. There is a method of changing the brightness of the signal output from the D converter 106. The system control unit 114 executes any one of these methods in the same manner as the processing described in S43.

  In S44, as in the first embodiment, the system control unit 114 is included in the gradation conversion characteristics set in the gradation conversion unit 612 from the difference between the exposure reference value set in S41 and the exposure reference value set in S42. Calculate the gain.

  In S72, the system control unit 114 controls the brightness of the SDR signal based on the exposure reference value set in S41 and S42. The control method differs depending on how the brightness of the HDR signal is controlled in S71.

  First, the case where the brightness of the HDR signal is controlled by controlling the imaging unit 105, the diaphragm 103, and the ND 104 in S71 will be described. When the brightness of the HDR signal is controlled using the imaging unit 105, the diaphragm 103, and the ND 104, the brightness of the signal output from the A / D converter 106 changes, so that the signal is input to the gain adjustment unit 610. The brightness of the signal will change. In order to obtain a brightness based on the exposure reference value set by the exposure reference value setting unit 209, the gain obtained in S44 may be used. The system control unit 114 may set the gain in the gain adjustment unit 610 or may set the gradation conversion characteristic based on the gradation characteristic of the SDR signal and the gain in the gradation conversion unit 612.

  Next, when the brightness of the HDR signal is controlled by controlling the gain adjustment unit 201 and the gradation conversion unit 203, the signal input to the gain adjustment unit 610 is the HDR signal set by the exposure reference value setting unit 206. It does not depend on the exposure reference value. Therefore, similarly to the case where the brightness of the HDR signal is controlled in S71, the gain adjustment unit 610 or the gradation conversion unit 612 is controlled with reference to only the exposure reference value of the SDR signal set by the exposure reference value setting unit 209. do it. Thereafter, the system control unit 114 ends this process.

  As described above, according to the present embodiment, even in the configuration in which the HDR signal and the SDR signal are generated in each system, the brightness of the SDR signal is substantially reduced based on the exposure target value of the SDR signal regardless of the exposure target value of the HDR signal. Can be kept constant. For this reason, as in the first embodiment, when the photographer changes the brightness of the HDR signal for the purpose of expression, the brightness information of the SDR signal is prevented from collapsing due to the change of the brightness of the SDR signal. be able to. In other words, when a plurality of image signals having different gradation characteristics are handled, even if the brightness of one image signal is changed, the influence on the brightness of the other image signal can be reduced. Further, since the brightness of the SDR signal can be changed independently of the HDR signal by changing the exposure reference value of the SDR signal, the photographer takes a picture without being aware of the relationship between the HDR signal and the SDR signal. be able to.

(Other embodiments)
In the first embodiment, the reflection of the exposure reference value of the HDR signal (that is, the brightness control of the HDR signal) is changed by changing the gradation conversion characteristic set in the gradation conversion unit 203 or the input value to the gradation conversion unit 203 An example in which any of the above is performed has been described. However, it is not always necessary to use either one, and both may be combined as long as the display luminance is correct.

  In the first embodiment, when the HDR signal is converted to generate the SDR signal, the control that can maintain the brightness of the SDR signal even if the exposure reference value of the HDR signal is changed has been described. This control is effective when shooting with an HDR signal more important than an SDR signal. On the other hand, there is a case where it is desired to take an image taking the SDR signal more important than the HDR signal. In such a case, it is necessary to change the order of signal processing so that the SDR signal is converted to generate the HDR signal. Specifically, the brightness of the HDR signal and the SDR signal can be maintained by changing the processing order while maintaining the exposure reference value of the HDR signal and the exposure reference value of the SDR signal. Further, by performing the processes of S44 to S46, the exposure reference values of the HDR signal and the SDR signal can be controlled independently without affecting each other.

  The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 107 ... Operation part, 114 ... System control part, 115 ... Image processing part, 124 ... Image conversion part, 203 ... Gradation conversion part, 204 ... Gradation conversion part, 207 ... Brightness control part, 208 ... Conversion characteristic calculation part

Claims (13)

Setting means for setting a first exposure reference value in the first image signal and a second exposure reference value in the second image signal having a gradation characteristic different from that of the first image signal;
Brightness control means for controlling the brightness of the first image signal based on the set first exposure reference value;
Control means for controlling gradation conversion characteristics;
Gradation conversion means for converting an image signal whose brightness has been changed by the control of the brightness control means into the second image signal using the gradation conversion characteristics;
The control means obtains the gradation conversion characteristic that suppresses a change in brightness of the second image signal according to a difference between the first exposure reference value and the second exposure reference value. An imaging device.   The gradation converting unit converts the first image signal whose brightness is changed by the control of the brightness control unit into the second image signal using the gradation conversion characteristic. The imaging apparatus according to claim 1. The first image signal has a first gradation characteristic, and the second image signal has a second gradation characteristic,
The control means calculates a gain according to the difference from the first exposure reference value and the second exposure reference value, and calculates the first gradation characteristic, the second gradation characteristic, and the gain. The imaging apparatus according to claim 2, wherein the gradation conversion characteristic is obtained based on the image quality.   The gradation converting means converts an image signal picked up with brightness changed under the control of the brightness control means into the second image signal using the gradation conversion characteristic. The imaging device according to claim 1. The first image signal has a first gradation characteristic, and the second image signal has a second gradation characteristic,
The control means calculates a gain according to the difference from the first exposure reference value and the second exposure reference value, and based on the second gradation characteristic and the gain, the gradation conversion characteristic The imaging device according to claim 4, wherein the imaging device is obtained.   The said control means calculates | requires the said gradation conversion characteristic using the value which multiplied the said gain to the output of the electro-optical transfer function corresponding to a said 2nd gradation characteristic, The Claim 3 or 5 characterized by the above-mentioned. The imaging device described.   The brightness control means controls the brightness of the first image signal by changing the first gradation characteristic based on the first exposure reference value. The imaging apparatus according to 3 or 5.   The imaging apparatus according to claim 1, further comprising an exposure control unit that changes an appropriate exposure of the imaging apparatus based on the first exposure reference value.   9. The imaging apparatus according to claim 1, wherein each exposure reference value set by the setting unit represents correspondence between brightness of a subject and luminance output from a display device. 10. .   10. The imaging apparatus according to claim 1, wherein the first image signal is an HDR signal, and the second image signal is an SDR signal. 11.   10. The imaging apparatus according to claim 1, wherein one of the first image signal and the second image signal is a Log signal having logarithmic gradation characteristics. 11. A setting step in which the setting means sets a first exposure reference value in the first image signal and a second exposure reference value in the second image signal having gradation characteristics different from the first image signal;
A brightness control step, wherein the brightness control means controls the brightness of the first image signal based on the set first exposure reference value;
A control step in which the control means controls the gradation conversion characteristics;
A gradation conversion unit that converts an image signal whose brightness has been changed by the control of the brightness control process into the second image signal using the gradation conversion characteristic; ,
In the control step, the gradation conversion characteristic that suppresses a change in brightness of the second image signal according to a difference between the first exposure reference value and the second exposure reference value is obtained. A method for controlling the imaging apparatus.   The program for functioning a computer as each means of the imaging device of any one of Claim 1 to 11.

Images