JP2021118457A - Imaging device and control method thereof, program and storage medium - Google Patents

Abstract

To provide an imaging device capable of precisely overlaying display corresponding to luminance value in exposure assist display, when image conversion according to user designated color conversion is performed between a block for detecting a luminance value from an image and a block for outputting a display image.SOLUTION: An imaging device according to the embodiment includes: setting means for setting a predetermined conversion characteristic designated by a user, for color conversion of a first image signal in which a subject is imaged by imaging means; image processing means for outputting a second image signal by performing image processing, including conversion based on the predetermined conversion characteristic, to the first image signal; conversion means for converting luminance value, based on image conversion characteristic by the image processing means by acquiring the image conversion characteristic by the image processing means, between a first luminance value corresponding to the first image signal and a second luminance value corresponding to the second image signal; and overlay means for overlaying a predetermined color pattern to the second image signal based on either the first luminance value or the second luminance value, obtained by conversion by the conversion means.SELECTED DRAWING: Figure 2

Description

The present invention relates to an image pickup apparatus, a control method thereof, a program, and a storage medium.

Conventionally, in the process of video production, there is a process called color grading in which the gradation and color of an image captured by an imaging device such as a video camera are corrected according to the intention of the creator. In this process, the user uses color grading software that operates on a PC, for example, to adjust the gradation and tint, or to overexpose unnecessary parts, so that the recorded image is suitable for viewing. Finish to. The conversion characteristics for color grading can be written out in a format such as 3DLUT (3D look-up table). Therefore, the user can perform the same color grading process on another image, or perform the same process with software different from that at the time of color grading.

In recent years, some imaging devices are known to be capable of reading a user-specified 3DLUT into the imaging device itself, and it is possible to directly record an image to which conversion processing by 3DLUT is applied (this function). Is also called a user LUT function). When recording with an imaging device using the user LUT function, it is difficult to make adjustments by color grading, so it is necessary to shoot with the expected exposure at the time of shooting. Since it is necessary for the user to determine the exposure after grasping the brightness of the face, the overexposure state of the high-brightness part, the blackout state of the low-brightness part, etc., the assist function for visualizing the brightness value is important. Become.

As an assist function for visualizing the brightness value in the image pickup device, by superimposing a color pattern according to the brightness value on the captured image, multiple areas such as overexposure area, proper brightness of face, and underexposure area can be grasped at the same time. (Patent Document 1).

JP-A-2009-130464

In the technique proposed in Patent Document 1, the state of the brightness values in a plurality of regions can be grasped by the assist function, but the assist function when recording an image to which the conversion process by the LUT specified by the user is applied by the imaging device. Is not considered. Generally, when the exposure assist function is realized, a color pattern corresponding to the brightness value detected by the detection block of the image pickup apparatus is superimposed on the image signal (display image) to be grasped as the assist function. However, depending on the circuit configuration of the image pickup apparatus, the positions on the circuit of the detection block that detects the luminance value and the block that outputs the image signal to be grasped as the exposure assist function may be different. That is, an image processing block that performs conversion processing by a user-specified 3DLUT may be sandwiched between the blocks. When the characteristics of the image signal are converted by such an image processing block, the luminance value detected by the detection block and the luminance value of the image signal that the user originally wants to grasp are different, and the correct luminance value is found in an appropriate region. It may not be possible to superimpose the display corresponding to.

The present invention has been made in view of the above problems. The purpose is to display the display corresponding to the luminance value in the exposure assist display when the image conversion is performed by the color conversion specified by the user between the block that detects the luminance value from the image and the block that outputs the display image. It is to realize a technology that can be superimposed with high accuracy.

In order to solve this problem, for example, the image pickup apparatus of the present invention has the following configuration. That is, the setting means for setting a predetermined conversion characteristic specified by the user for performing color conversion on the first image signal in which the subject is imaged by the image pickup means, and the first image signal. An image processing means that performs image processing including conversion based on a predetermined conversion characteristic and outputs a second image signal, and an image conversion characteristic by the image processing means is acquired to correspond to the first image signal. A conversion means for converting the brightness value between the first brightness value and the second brightness value corresponding to the second image signal based on the conversion characteristics of the image by the image processing means, and the conversion means. It is characterized by having a superimposing means for superimposing a predetermined color pattern on the second image signal based on the first luminance value or the second luminance value obtained by being converted by. And.

According to the present invention, when image conversion is performed by a user-specified color conversion between a block that detects a luminance value from an image and a block that outputs a display image, a display corresponding to the luminance value is displayed in the exposure assist display. Can be superimposed more accurately.

Block diagram showing a functional configuration example of the image pickup apparatus according to the first embodiment A block diagram showing an example of the internal function configuration of the image processing unit according to the first embodiment. The figure which shows a part of the table of 3DLUT in Embodiment 1. The figure which shows an example of the assist function in Embodiment 1. A flowchart showing a series of operations of the luminance value calculation process in the first embodiment. The figure which shows a part of the lookup table after conversion of the input value in Embodiment 1. The figure which shows a part of the lookup table after sorting in Embodiment 1. The figure which shows the conversion example of the reference luminance in Embodiment 1. A flowchart showing a series of operations of the luminance value calculation process in the second embodiment. The figure which shows a part of the lookup table after sorting in Embodiment 3. A block diagram showing a configuration of an imaging device and a display device according to the fourth embodiment.

(Embodiment 1)
Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are given the same reference numbers, and duplicate explanations are omitted.

Hereinafter, as an example of the image pickup apparatus, a case where a digital camera having an exposure assist function is used will be described as an example. However, this embodiment is applicable not only to a digital camera but also to other devices having an exposure assist function. These devices may include, for example, smartphones, game consoles, tablet terminals, clock-type and eyeglass-type information terminals, medical devices, surveillance camera systems, and the like.

(Digital camera configuration)
FIG. 1 is a block diagram showing a functional configuration example of a digital camera 100 as an example of the image pickup apparatus of the present embodiment. One or more of the functional blocks shown in FIG. 1 may be realized by hardware such as an ASIC or a programmable logic array (PLA), or may be realized by a programmable processor such as a CPU or MPU executing software. You may. It may also be realized by a combination of software and hardware. Therefore, in the following description, the same hardware can be realized as the main body even if different functional blocks are described as the main body of operation.

An example of the functional configuration of the digital camera 100 will be described with reference to FIG. The photographing lens 102 is a lens group including a zoom lens and a focus lens, and forms a subject image. The diaphragm 103 is a diaphragm used for adjusting the amount of light. The ND 104 is an ND filter used for dimming. The image pickup unit 105 includes an image pickup device composed of a CCD, a CMOS element, or the like that converts an optical image into an electric signal. Further, the imaging unit 105 also has functions such as control of accumulation by an electronic shutter, analog gain, and change of reading speed. The A / D converter 106 converts an analog signal into a digital signal. The A / D converter 106 is used to convert an analog signal output from the imaging unit 105 into a digital signal. The barrier 101 covers the imaging system including the photographing lens 102 of the digital camera 100 to prevent the imaging system including the photographing lens 102, the aperture 103, the ND 104, and the imaging unit 105 from being soiled or damaged.

The image processing unit 115 performs color conversion processing, gamma correction, addition of digital gain, and the like on the data from the A / D converter 106 or the data from the memory control unit 116. Further, a predetermined calculation process is performed using the captured image data, and the calculation result is transmitted to the system control unit 114. Based on the transmitted calculation result, the system control unit 114 performs exposure control, distance measurement control, white balance control, and the like. As a result, TTL (through-the-lens) AF (autofocus) processing, AE (autoexposure) processing, AWB (auto white balance) processing, and the like are performed. Details of the image processing unit 115 will be described later. A part or all of the functions included in the image processing unit 115 may be realized by the system control unit 114.

The output data from the A / D converter 106 is written directly to the memory 119 via the image processing unit 115 and the memory control unit 116, or via the memory control unit 116. The memory 119 stores image data captured by the imaging unit 105 and converted into digital data by the A / D converter 106, and image data to be displayed on the display unit 118. The memory 119 has a storage capacity sufficient to store moving images and sounds for a predetermined time.

Further, the memory 119 also serves as a memory (video memory) for displaying an image. The D / A converter 117 converts the image display data stored in the memory 119 into an analog signal and supplies it to the display unit 118. In this way, the image data for display written in the memory 119 is displayed by the display unit 118 via the image superimposition unit 125 and the D / A converter 117 that superimpose the icon, character information, and the like on the image.

The image superimposing unit 125 also realizes an assist function (exposure assist function) for superimposing an image of a specific pattern on a part of the image based on the image information of the image processing unit 115 described later. The display unit 118 displays on a display such as an LCD according to the analog signal from the D / A converter 117. The digital signal once A / D converted by the A / D converter 106 and stored in the memory 119 is analog-converted by the D / A converter 117 and sequentially transferred to the display unit 118. By displaying the transferred analog signal, the display unit 118 functions as an electronic viewfinder and can display a through image.

The non-volatile memory 113 is a memory that can be electrically erased and recorded, and for example, EEPROM is used. The non-volatile memory 113 stores constants, programs, and the like for the operation of the system control unit 114. The program is a computer program for executing various flowcharts described later in the present embodiment.

The system control unit 114 controls the entire digital camera 100. By expanding and executing the program recorded in the non-volatile memory 113 described above in the system memory 121, each process according to the embodiment described later is realized. The system memory 121 includes, for example, a RAM, and temporarily stores constants, variables, programs read from the non-volatile memory 113, and the like for operation of the system control unit 114. The system control unit 114 also controls the display by controlling the memory 119, the image superimposition unit 125, the D / A converter 117, the display unit 118, and the like.

The system timer 120 is a time measuring unit that measures the time used for various controls and the time of the built-in clock. The mode changeover switch 109, the recording switch 108, and the operation unit 107 are operation means for inputting various operation instructions to the system control unit 114.

The mode changeover switch 109 switches the operation mode of the system control unit 114 to any one of a moving image recording mode, a still image recording mode, a playback mode, and the like. Modes included in the moving image recording mode and the still image recording mode include an auto shooting mode, an auto scene discrimination mode, a manual mode, various scene modes for shooting settings for each shooting scene, a program AE mode, a custom mode, and the like. By operating the mode changeover switch 109, it is possible to directly switch to any of these modes included in the moving image shooting mode. Alternatively, after switching to the moving image shooting mode once with the mode changeover switch 109, the mode may be switched to any of these modes included in the moving image shooting mode by using another operating member. The recording switch 108 switches between a shooting standby state and a shooting state. The system control unit 114 starts a series of operations from reading a signal from the imaging unit 105 to writing moving image data to the recording medium 123 by the recording switch 108.

The power switch 110 is a switch for turning on / off the power of the digital camera 100. Each operation member of the operation unit 107 is assigned a function as appropriate for each scene by selecting and operating various function icons superimposed on the image superimposing unit 125 and displayed on the display unit 118, and acts as various function buttons. .. The function buttons include, for example, an end button, a back button, an image feed button, a jump button, a narrowing down button, an attribute change button, and the like. For example, when the menu button is pressed, various settable menu screens are superimposed on the image superimposing unit 125 and displayed on the display unit 118. The user can intuitively make various settings by using the menu screen displayed on the display unit 118 and the cross keys and SET buttons in four directions of up, down, left, and right.

The power supply control unit 111 includes a battery detection circuit, a DC-DC converter, a switch circuit for switching a block to be energized, and the like, and detects whether or not a battery is installed, the type of battery, and the remaining battery level. Further, the power supply control unit 111 controls the DC-DC converter based on the detection result and the instruction of the system control unit 114, and supplies a necessary voltage to each unit including the recording medium 123 for a necessary period.

The power supply unit 112 may be composed of, for example, a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li ion battery, an AC adapter, or the like, and supplies power to each part of the digital camera 100. do. The I / F 122 is an interface with a recording medium 123 such as a memory card or a hard disk, or an external output device. In the example shown in FIG. 1, the state at the time of connection with the recording medium 123 is shown. The recording medium 123 is a recording medium such as a memory card for recording a captured image, and is composed of a semiconductor memory, a magnetic disk, or the like. The image conversion unit 124 performs gradation conversion, color conversion, and the like on the signal input from the image processing unit 115.

(Structure of image processing unit 115)
Next, an example of the functional configuration of the image processing unit 115 according to the present embodiment will be described with reference to FIG. The gain correction unit 20 applies a predetermined gain to the RGB signal output from the A / D converter 106 and amplifies it. The WB correction unit 21 corrects the white balance by applying a gain to the R signal and the B signal among the signals output from the gain correction unit 20. The matrix calculation unit 22 corrects the color tone by applying a predetermined matrix to the signal output from the WB correction unit 21. The gradation conversion unit 23 performs gradation conversion on the signal output from the matrix calculation unit 22. The output from the gradation conversion unit 23 is output to the image conversion unit 124 and the recording medium 123.

Here, the image signal corresponding to the input of the matrix calculation unit 22 (that is, the output of the WB correction unit 21) is referred to as a first image signal. The image information detection unit 25 acquires a luminance value (first luminance value) as each pixel value of the first image signal. In the present embodiment, the image signal targeted in the assist function described later is an image signal (second image signal) output from the 3DLUT processing unit 24, and the brightness value used as a reference in the assist function is the first reference brightness. Let it be a value. This reference luminance value corresponds to a reference luminance value that serves as a reference for overexposure (also referred to as saturation) in the second image signal. Alternatively, the reference brightness value is a reference brightness value that is a reference immediately before overexposure, a reference brightness value that is a reference of proper brightness of a face, a reference brightness value that is a reference immediately before blackout, and a reference brightness that is a reference of blackout. Corresponds to one of the values. The image processing unit 115 converts the first luminance value (detected by the image information detecting unit 25) into a second luminance value corresponding to the second image signal and compares it with the first reference luminance value. do. Alternatively, the image processing unit 115 converts the first reference luminance value into the second reference luminance value corresponding to the first image signal (conversion by the matrix calculation unit 22, the gradation transformation unit 23, and the 3DLUT processing unit 24). It may be compared with the first luminance value (corresponding to the inverse transformation). Based on such a comparison result, the image superimposing unit 125 superimposes a pattern described later on the display image. In the following description, the latter process, that is, the process of converting the first reference luminance value to the second reference luminance value will be described as an example.

The 3DLUT processing unit 24 performs color conversion using the 3DLUT (3D look-up table) illustrated in FIG. The 3DLUT shown in FIG. 3 is a table that shows conversion characteristics for color grading, for example, and converts the RGB value of the input value 31 which is the input value of the 3DLUT processing unit 24 into the RGB value of the output value 32. In the example shown in FIG. 3, for the sake of explanation, the RGB value of the input value and the RGB value of the output value (corresponding to the RGB value of the input value) are shown side by side. In the LUT, since the RGB values of the input values are separated at equal intervals, only the output values may be listed as the actual data, but even such a table is substantially the same as the 3D LUT shown in FIG. It is the same. If the same RGB value as the input RGB value is in the conversion source table (that is, it is described in the LUT as the input value), it is directly converted to the RGB value of the conversion destination (the RGB value described in the LUT as the output value). To output). On the other hand, when the input RGB value does not exist in the conversion source table, the input RGB value is interpolated from the points around the input RGB value. This interpolation can be performed, for example, by extracting a tetrahedron containing RGB values and using a known method called tetrahedron interpolation, but other interpolation methods may also be used.

Next, an example of the assist function realized by the image superimposing unit 125 according to the embodiment will be described with reference to FIG. The example shown in FIG. 4 shows an example in which a pattern of a specific color is superimposed on the image signal according to the image 41 according to the range of the luminance value of the image 42. According to this superimposition method, the user can simultaneously grasp the overexposure area, the area immediately before the overexposure, the appropriate brightness of the face, the area immediately before the underexposure, the underexposure area, and the like at the time of shooting. ..

As shown in FIG. 4, the image superimposition unit 125 sets each region, for example, the overexposed region (region 43) is yellow, the appropriate region (region 44) is, for example, green, and the underexposed region (region 45) is, for example, blue. Superimpose different colors on the image. By doing so, the user can easily grasp a plurality of luminance ranges. Although the example shown in FIG. 4 shows an example in which three regions are displayed, the pattern may be superimposed on a plurality of regions, not limited to the three regions.

Consider a case where a color pattern corresponding to each of the luminance ranges shown in FIG. 4 is superimposed on the luminance of the image output from the 3D LUT processing unit 24. If the image output from the 3DLUT processing unit 24 can be acquired and the brightness value can be acquired by detecting each pixel value of the image itself, the image superimposing unit 125 has a color corresponding to each luminance range. Overlay patterns. On the other hand, when the image output from the 3D LUT processing unit 24 cannot be directly detected (the second image signal shown in FIG. 2 cannot be directly obtained), the second image signal corresponds to the pixel value of the second image signal. It is necessary to generate a brightness value. In this case, that is, the case where the second image signal cannot be directly detected will be described. For example, if the image processing unit 115 can acquire information from the first image signal shown in FIG. 2, the image in each unit existing between the first image signal and the generation of the second image signal. The conversion is applied to the first luminance value detected by the image information detection unit 25. That is, the image processing unit 115 applies the image conversion by the matrix calculation unit 22, the gradation conversion unit 23, and the 3DLUT processing unit 24 to the first luminance value, and corresponds to the pixel value of the second image signal. Convert to a second brightness value. On that basis, the image superimposing unit 125 can realize the function by superimposing the pattern on the corresponding region defined by the first reference luminance value. Alternatively, the image processing unit 115 generates a second reference luminance value by applying the inverse transformation of the image transformation in the matrix calculation unit 22, the gradation transformation unit 23, and the 3DLUT processing unit 24 to the first reference luminance value. May be good. In this case, the assist function can be realized by superimposing patterns based on the first luminance value.

(A series of operations of the brightness value calculation process for the assist function)
Next, with reference to FIG. 5, a series of operations of the luminance value calculation process for the assist function will be described. In the following description, a case realized by the image processing unit 115 will be described as an example, unless otherwise specified. However, this process may be realized by the system control unit 114 expanding and executing the program stored in the non-volatile memory 113 in the working area of the system memory 121. Further, in the present embodiment, the user can set an arbitrary (user-specified) 3DLUT to the 3DLUT processing unit 24 via the operation unit 107, and the set 3DLUT is stored in, for example, the recording medium 123. Further, in the following example, the case where the range of the signal is 0 to 1 (the unit may use% as an example) in all of the A / D converter 106 to the 3DLUT processing unit 24 will be described as an example.

The luminance value calculation process may be executed in a state where the image processing unit 115 inputs an image signal of an image of a subject from the image pickup unit 105, but it is also executed before the image signal is input from the image pickup unit 105. be able to. In this case, the assist function that processes the image signal at the stage when the image signal is input from the image pickup unit 105 may be realized.

In the following description, the input of the matrix calculation unit 22 is (R1, G1, B1), the input of the gradation conversion unit 23 is (R2, G2, B2), and the input of the 3DLUT processing unit 24 is (R3, G3, B3). It is expressed as. At this time, in the present embodiment, the matrix calculation unit 22 and the gradation conversion unit 23 perform calculations according to Equations 1 and 2, respectively.

In step S51, the image processing unit 115 reads the 3DLUT data specified by the user from, for example, the recording medium 123. In step S52, the image processing unit 115 analyzes the 3DLUT data read in step S51 and obtains the input signal of the matrix calculation unit 22 required to generate the output signal from the 3DLUT processing unit 24. Here, an example of searching for an input value that serves as a reference luminance value for use in determining overexposure is shown. In step S52, the input value of the 3DLUT data shown in FIG. 3 is converted into the input value of the matrix calculation unit 22. From Equations 1 and 2, the relationship between the input values (R1, G1, B1) of the matrix calculation unit 22 and the input values (R3, G3, B3) of the 3DLUT processing unit 24 can be expressed by Equation 3.

That is, in order to convert the input value (R3, G3, B3) of the 3DLUT processing unit 24 into the input value (R1, G1, B1) of the matrix calculation unit 22, the calculation shown in Equation 4 may be performed.

FIG. 6 shows the result of converting the input value of the 3D LUT into the input signal of the matrix calculation unit 22 (that is, the equivalent of the first image signal) by this calculation. In addition, in this embodiment, in order to realize the assist function based on the luminance value, in FIG. 6, the input value (R1, G1, B1) of the matrix calculation unit 22 and the output value (R4, G4) of the 3DLUT processing unit 24 are shown. , B4) The brightness values Y1 and Y4 calculated from) are also shown.

In step S53, the image processing unit 115 sorts the 3DLUT data generated in step S52 based on the output luminance value, for example, by the value of Y4. FIG. 7 shows an example of sorted 3DLUT data.

In step S54, the image processing unit 115 extracts the output value corresponding to the maximum input luminance value. In the example shown in FIG. 7, a plurality of rows of the output value Y4 have the maximum value 1 in the data as the output luminance value, and the corresponding input luminance values Y1 also have the plurality of values. Therefore, in order to display the assist function in the correct area, it is necessary to convert the reference luminance value (for example, 1 corresponding to overexposure in Y4).

In step S55, the image processing unit 115 determines whether or not it is necessary to convert the reference brightness value from the first reference brightness value to the second reference brightness value. In the example shown in FIG. 7, the average value obtained from a plurality of input luminance values (values of Y1) in which the output value of Y4 is 1, is about 87%. In this case, the image processing unit 115 determines that it is necessary to convert the reference luminance value when 1 in Y4 and the average value in Y1 are different, and proceeds to step S56. On the other hand, when the image processing unit 115 determines that it is not necessary to convert the reference luminance value, the image processing unit 115 ends this series of operations without performing step S56.

In step S56, the image processing unit 115 converts the reference luminance value as shown in row 81 of FIG. That is, the reference luminance value in Y1 is set as the calculated average value, and the region of the luminance value overexposed is set to 100 from the average value. Although an example in which the average value of a plurality of input luminance values is used is shown here, another method may be used as long as the method uses a value representing a plurality of input luminance values. For example, data close to the maximum value, the minimum value, the median value, the mode value, and the achromatic color of the plurality of input luminance values may be used.

Similarly, the image processing unit 115 extracts an output value corresponding to the reference luminance value with respect to the luminance value in the middle of the blackout in the row 83 and the middle of the row 82 (that is, during the blackout and saturation) (at this time, extraction). The accuracy may be improved by extracting multiple luminance values before and after the output value.) Then, the image processing unit 115 converts the reference luminance value as shown in rows 82 and 83 based on the relationship between the output value corresponding to each reference luminance value and the corresponding input value. After that, when the image superimposition unit 125 inputs an image signal of the subject imaged from the image pickup unit 105, the image superimposition unit 125 images the corresponding color pattern based on the converted reference luminance, as in the assist function shown in FIG. Superimposes on and ends this series of operations.

As described above, in the present embodiment, when the brightness value of the image displayed as the assist function cannot be directly detected, the conversion characteristics of the image processing block (matrix calculation unit 22, gradation conversion unit 23, 3DLUT processing unit 24) are obtained. did. Then, according to the conversion characteristic, the reference luminance value corresponding to the output value from the 3DLUT processing unit 24 is converted into the luminance value corresponding to the input value of the matrix calculation unit 22. By doing so, when the image conversion is performed by the color conversion specified by the user between the block that detects the luminance value from the image and the block that outputs the display image, the exposure assist display corresponds to the luminance value. It is possible to superimpose the display more accurately.

(Embodiment 2)
Next, the second embodiment will be described. In the first embodiment, an example is shown in which an analysis is performed on the 3DLUT at the time of reading the 3DLUT applied by the 3DLUT processing unit 24, and the conversion characteristics obtained by the analysis are used in the subsequent processing. In the example shown in the first embodiment, when there are a plurality of input values to be converted into a specific output value depending on the characteristics of the read 3DLUT and the input value has a wide range, the user does not necessarily accurately grasp the display of the assist function. It may not be possible. Therefore, in the present embodiment, a control example that realizes the assist function by converting the input value into the output value is shown. Further, although the reference luminance value is changed in the first embodiment, the present embodiment shows a method of converting the luminance value detected by the image information detection unit 25. That is, in the present embodiment, the operation of the luminance value calculation process for the assist function is different from that of the first embodiment, but the configuration of the digital camera 100 is the same as or substantially the same as that of the first embodiment. Therefore, the configurations that are the same as or substantially the same as the configurations shown in the first embodiment will be given the same reference numbers, and the description thereof will be omitted, and the differences will be mainly described.

(A series of operations of the brightness value calculation process for the assist function)
A series of operations of the luminance value calculation process for the assist function according to the present embodiment will be described with reference to FIG. In the present embodiment, as in the first embodiment, the luminance value is detected from the input image of the matrix calculation unit 22, and the target of the assist function is the output image of the 3DLUT processing unit 24.

In step S91, the image information detection unit 25 detects the signal value for each pixel of the first image signal which is the input image. Next, in step S92, the image processing unit 115 performs conversion by the matrix calculation unit 22 and the gradation conversion unit 23 represented by the equation 3 on the detected signal value, and inputs the input value to the 3DLUT processing unit 24. (That is, (R3, G3, B3)) is calculated.

In step S93, the image processing unit 115 applies the conversion by 3DLUT equivalent to the processing by the 3DLUT processing unit 24 to the input value of the 3DLUT processing unit 24 to generate the output value of the 3DLUT processing unit 24. In step S94, the image processing unit 115 calculates the luminance value (that is, corresponding to the second luminance value) from the output values (R, G, B) of the 3D LUT processing unit 24.

In step S95, the image superimposing unit 125 superimposes a pattern of a specific color on the display image after comparing the calculated luminance value with the reference luminance value. In the present embodiment, by repeating this series of operations for each pixel signal, it is possible to more accurately superimpose the display corresponding to the luminance value in the exposure assist display. The image superimposition unit 125 ends this series of operations when the display image and the pattern are superimposed on the final pixel signal.

At the time of reading the 3DLUT, a new 3DLUT may be generated by synthesizing all the processes of steps S92 to S94. In that case, data is prepared for the number of reference points so that each RGB value becomes the value of the reference point of 3DLUT, and the values are subjected to the processing of steps S92 to S94 to obtain the brightness for each reference point. Calculate the value. If a new 3DLUT whose brightness value is the output value is generated and applied to the brightness value of the input image of the matrix calculation unit 22, the processing of steps S92 and S94 can be omitted, and as a result, the calculation amount can be reduced. It is possible to reduce it.

(Embodiment 3)
Next, the third embodiment will be described. In the first and second embodiments, the case where the brightness value detected from the input image of the matrix calculation unit 22 is used for the assist function for the output image from the 3DLUT processing unit 24 has been described. On the other hand, in the present embodiment, the brightness value detected from the output image of the 3DLUT processing unit 24 is used for the assist function for the input image of the matrix calculation unit 22. Further, in this embodiment, for convenience of explanation, a 3DLUT having different conversion characteristics from the 3DLUT used in the first and second embodiments is used. As described above, in the present embodiment, the operation of the luminance value calculation process for the assist function and the 3D LUT are different from the above-described embodiment, but the configuration of the digital camera 100 is the same as or substantially the same as that of the embodiment. Therefore, the configurations that are the same as or substantially the same as the configurations shown in the above-described embodiments will be given the same reference numbers, the description will be omitted, and the differences will be mainly described.

FIG. 10 shows a 3D LUT according to this embodiment. Similar to the first embodiment, this 3DLUT is obtained by analyzing the processing in the matrix calculation unit 22, the gradation conversion unit 23, and the 3DLUT processing unit 24 when reading the 3DLUT data. This 3DLUT converts the input value into the input signal of the matrix calculation unit 22, generates 3DLUT data which is the output of the 3DLUT processing unit 24 as the output value, calculates the value of Y from both input and output, and Y of the output. It is sorted using.

In the example shown in FIG. 10, the Y value of the input value has reached 1, while the Y value of the output value has not reached 1. Therefore, it is necessary to convert the reference luminance value in order to display the assist function in the correct area. In the example of FIG. 10, for example, the converted luminance value corresponding to the input value (the bottom row of the table shown in FIG. 10) in which Y (in the overexposed state) is 1 is about 90%. be. According to this conversion characteristic, if the detected luminance value is 90% or more, it can be determined that the whiteout is overexposed. Therefore, the reference luminance value for determining the overexposure is converted to 90%.

Similarly, the reference luminance value of the blackout and the intermediate luminance can be converted by obtaining the luminance value after the 3D LUT processing unit 24 corresponding to the corresponding input luminance value. At this time, a plurality of output values before and after corresponding to the reference luminance value may be further utilized, and data close to the average value, the median value, the maximum value, the minimum value, the minimum value, the mode value, and the achromatic color may be used. By doing so, it becomes possible to more accurately superimpose the display corresponding to the luminance value in the exposure assist display.

(Embodiment 4)
Next, the fourth embodiment will be described. In the first and second embodiments, the case where the target of the assist function is an image signal inside the digital camera 100 has been described as an example. On the other hand, in the present embodiment, the case where the brightness level to be displayed by the external display device 1101 of the digital camera 100 is targeted for the assist function will be described. In the present embodiment, the reference luminance value for superimposing the image as the assist function is treated as the display luminance level of the display device 1101.

The configuration of the digital camera according to this embodiment is shown in FIG. In the present embodiment, the image output from the image processing unit 115 is converted by the image conversion unit 124 instead of the image superimposition unit 125 and displayed on the external display device 1101. Therefore, the operation of the present embodiment is the same as that of the first embodiment except for the operation by the image conversion unit 124. Therefore, the operation by the image conversion unit 124 will be mainly described, and the description of the configuration and operation that are the same as or substantially the same as those of the first embodiment will be omitted.

First, it is necessary to estimate or acquire the conversion characteristic in which the signal value is converted into the display luminance level in the display device 1101. Therefore, for example, if the display luminance level with respect to the signal value is uniquely determined (by a predetermined function or the like), the conversion characteristic is used. Further, in the case of a signal format in which the display luminance level fluctuates depending on the ability of the display device 1101, the conversion characteristic is determined by acquiring (or estimating from the information that can be obtained) the information of the ability of the display device 1101.

The image processing unit 115 applies the conversion characteristics of the display device 1101 obtained as described above to the matrix calculation unit 22, the gradation conversion unit 23, and the 3DLUT processing unit 24 used for the analysis in the first and second embodiments. Further, the added conversion characteristics are obtained. By doing so, it is possible to obtain the conversion characteristic from the luminance value detected by the image information detection unit 25 to the luminance level displayed by the display device 1101.

When image conversion is performed by the image conversion unit 124 instead of the display device 1101, the conversion characteristics of the image conversion unit 124 may be considered instead of the conversion characteristics of the display device 1101. Alternatively, when the image conversion unit 124 further performs image conversion in addition to the display device 1101, the image processing unit 115 obtains the conversion characteristics in consideration of the conversion characteristics of the image conversion unit 124. Then, the image processing unit 115 can convert the reference luminance value by using the obtained conversion characteristic.

As described above, in the present embodiment, by further considering the conversion characteristics of the display device 1101, even when the external display device 1101 is used, the display corresponding to the luminance value is more accurate in the exposure assist display. It becomes possible to superimpose well.

In the above-described embodiment, a case where a 3D LUT that can be arbitrarily set by the user for color conversion is used has been described as an example. However, instead of 3DLUT, other conversion processing such as 1DLUT (one-dimensional lookup table), 2DLUT (two-dimensional lookup table), CDL (Color Decision List), matrix calculation, or the like may be used.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The invention is not limited to the above embodiments, and various modifications and modifications can be made without departing from the spirit and scope of the invention. Therefore, a claim is attached to make the scope of the invention public.

115 ... Image processing unit, 25 ... Image information detection unit, 22 ... Matrix calculation unit, 23 ... Gradation conversion unit, 24 ... 3DLUT processing unit, 125 ... Image superimposition unit, 118 ... Display unit, 124 ... Image conversion unit, 1101 … Display device

Claims (13)

A setting means for setting a predetermined conversion characteristic specified by the user for performing color conversion on the first image signal obtained by capturing the subject by the imaging means, and a setting means.
An image processing means that outputs a second image signal by performing image processing including conversion based on the predetermined conversion characteristic on the first image signal.
By acquiring the conversion characteristics of the image by the image processing means, the first luminance value corresponding to the first image signal and the second luminance value corresponding to the second image signal are described as described above. A conversion means that converts the luminance value based on the conversion characteristics of the image by the image processing means, and
It has a superimposing means for superimposing a predetermined color pattern on the second image signal based on the first luminance value or the second luminance value obtained by being converted by the converting means. An imaging device characterized by this. The conversion means converts the first luminance value, which becomes saturated after the image processing, to the maximum value of the second luminance value, based on the conversion characteristics of the image by the image processing means. Alternatively, it is configured to convert the first luminance value, which will be blacked out after the image processing, to the minimum value of the second luminance value.
The imaging apparatus according to claim 1, wherein the superimposing means superimposes the color pattern based on the second luminance value obtained by being converted by the converting means. The first conversion means saturates the reference luminance value, which is a reference for saturation in the second image signal, after performing the image processing, based on the conversion characteristics of the image by the image processing means. It is configured to be converted into a luminance value, or a reference luminance value that is a reference for blackout in the second image signal is converted into the first luminance value that will be blackened after the image processing is performed. Being done
The imaging apparatus according to claim 1, wherein the superimposing means superimposes the color pattern based on the reference luminance value and the second luminance value. It further has an output means for outputting a third image signal for display on an external display device.
The conversion means further acquires the conversion characteristics for displaying the image signal on the external display device, and the conversion characteristics of the image for displaying the image signal on the external display device and the image by the image processing means. The luminance value is converted between the first luminance value corresponding to the first image signal and the third luminance value corresponding to the third image signal by using the conversion characteristic of. The image pickup apparatus according to any one of claims 1 to 3, wherein the image pickup apparatus is characterized by Claim 1 is characterized in that the conversion means acquires the conversion characteristics of an image by the image processing means by applying the image processing means to the first image signal captured by the imaging means. 4. The image pickup apparatus according to any one of 4. The conversion means obtains the first image signal captured by the imaging means without applying the image processing means, and the first image signal is based on the conversion characteristics of the image by the image processing means. Any one of claims 1 to 4, wherein the luminance value is converted between the first luminance value corresponding to the above and the second luminance value corresponding to the second image signal. The imaging apparatus according to. The conversion means sets the second luminance value corresponding to the second image signal to the first brightness value corresponding to the first image signal based on the inverse conversion of the conversion characteristics of the image by the image processing means. The image pickup apparatus according to claim 6, further comprising converting into a luminance value of. When the conversion means has a plurality of the first luminance values that will be saturated after the image processing, or the first luminance values that will be blacked out after the image processing. One of claims 1 to 7, wherein any one of the average value, the median value, the mode value, the maximum value, and the minimum value in each of the plurality of luminance values is used as a new first luminance value. The imaging apparatus according to item 1. Based on the conversion characteristics of the image by the image processing means, the conversion means will have the first luminance value that will be saturated after the image processing and the blackout after the image processing. It is characterized in that a predetermined number of luminance values between the first luminance value and the first luminance value are converted into the second luminance value as a reference luminance value as a reference for superimposing the pattern. The imaging device according to any one of claims 1 to 8. Claim 1 is characterized in that the predetermined conversion characteristic is composed of any one of a three-dimensional lookup table, a two-dimensional lookup table, a one-dimensional lookup table, a CDR (Color Decision List), and a matrix operation. 9. The imaging apparatus according to any one of 9. A setting process for setting a predetermined conversion characteristic specified by the user for performing color conversion on the first image signal obtained by capturing the subject by the imaging means, and a setting process.
An image processing step of performing image processing including conversion based on the predetermined conversion characteristics on the first image signal and outputting the second image signal.
By acquiring the conversion characteristics of the image by the image processing, the image is set between the first luminance value corresponding to the first image signal and the second luminance value corresponding to the second image signal. A conversion means that converts the brightness value based on the conversion characteristics of the image by processing, and
It has a superimposing means for superimposing a predetermined color pattern on the second image signal based on the first luminance value or the second luminance value obtained by being converted by the converting means. A control method for an imaging device, characterized in that. A program for causing a computer to function as each means of the imaging device according to any one of claims 1 to 10. A storage medium that stores a program for causing a computer to function as each means of the imaging device according to any one of claims 1 to 10.

Images