WO2020071108A1

WO2020071108A1 - Image processing device and image processing method

Info

Publication number: WO2020071108A1
Application number: PCT/JP2019/036388
Authority: WO
Inventors: 康一郡司
Original assignee: キヤノン株式会社
Priority date: 2018-10-04
Filing date: 2019-09-17
Publication date: 2020-04-09

Abstract

An image processing device according to the present invention has a generation means for generating output image data on the basis of target image data, an acquisition means for acquiring feature values from the output image data, and an output means for outputting the output image data and feature information based on the feature values, wherein when the image area of the output image data includes a first area, which is an image area of the target image data, and a second area, which is an image area of prescribed image data, the acquisition means acquires the feature values of the first area.

Description

Image processing apparatus and image processing method

The present invention relates to an image processing device and an image processing method.

(2) An imaging device such as a digital camera or a digital video camera can perform imaging (imaging and recording of an image) while displaying an image (imaging image) obtained by imaging on an EVF (electronic viewfinder). For example, a display panel included in the imaging device or a display device (external device) connected to the imaging device is used as an EVF, and a photographer checks a captured image displayed on the EVF to check various feature amounts of the captured image. I do.

特徴 The feature quantity that the photographer wants to check includes the luminance value (luminance level) of the captured image. In recent years, photographing and display in HDR (High Dynamic Range), which is a relatively wide dynamic range (luminance range), have been in full swing, and standardization and commercialization related to HDR have been promoted. For example, in standards such as HDR10 +, MaxCLL (Maximum Content Content Light Level) indicating the maximum luminance value of a scene for each scene, and MaxFALL (Maximum Frame Average Level) such as MaxFALL indicating the maximum value of the average luminance value of a frame for each scene. Additional information is defined. In the additional information, information (MaxCLL and MaxCLL values) may dynamically change for each scene. In MaxCLL and MaxFALL, one frame can be treated as one scene. In other words, MaxCLL can indicate the maximum luminance value of a frame for each frame, and MaxFALL can indicate the average luminance value of a frame for each frame.

The additional information can be transmitted from the device to another device by communication conforming to the HDMI standard or the like. For example, the additional information can be transmitted from the imaging device to the display device. The display device can easily adjust the display luminance (the luminance on the display surface) using the additional information as the luminance evaluation value for display. However, a predetermined image may be added to the edge of the captured image, and additional information that does not match the intention of the photographer may be generated. Specifically, additional information that does not match the intention of the photographer may be generated from an image in which a black band image (a band-shaped black image) is added to the top and bottom and left and right of the captured image. A state in which black bands are added above and below the image is called "letterbox", and a state in which black bands are added to the left and right of the image is called "pillar box".

Patent Literature 1 discloses a display device that acquires a feature amount from an image by excluding an area of a predetermined image from an area for acquiring a feature amount, and controls light emission luminance of a backlight light source based on the acquired feature amount. Have been.

JP 2007-140483 A

However, according to the technology disclosed in Patent Document 1, the display device needs to be provided with a function of acquiring a feature amount of an image, and when additional information (MaxCLL or MaxFALL) input to the display device is used, the photographer is Display luminance that does not meet the intended purpose is realized.

The object of the present invention is to provide a technique capable of more reliably realizing a display luminance intended by a photographer or the like.

A first aspect of the present invention provides:
Generating means for generating output image data based on the target image data;
Acquiring means for acquiring a feature amount from the output image data;
Output means for outputting the output image data and feature information based on the feature amount,
Has,
When the image area of the output image data includes a first area that is an image area of the target image data and a second area that is an image area of predetermined image data, the acquisition unit may include the first area. An image processing apparatus is characterized by acquiring the characteristic amount of (1).

A second aspect of the present invention provides:
A generating step of generating output image data based on the target image data;
An obtaining step of obtaining a feature amount from the output image data,
An output step of outputting the output image data and feature information based on the feature amount,
Has,
In the case where the image area of the output image data includes a first area that is an image area of the target image data and a second area that is an image area of predetermined image data, in the acquiring step, the first area This is an image processing method characterized by acquiring the characteristic amount of (1).

A third aspect of the present invention is a program for causing a computer to function as each unit of the above-described image processing apparatus.

According to the present invention, it is possible to more reliably realize the display luminance intended by the photographer or the like.

FIG. 2 is a block diagram illustrating a configuration example of an imaging device according to the first to fourth embodiments. 9 is a flowchart illustrating an example of a processing flow of the imaging apparatus according to the first to fourth embodiments. FIG. 9 is a diagram illustrating an example of display image data and the like according to the first to fourth embodiments.

<Example 1>
Hereinafter, Embodiment 1 of the present invention will be described. Although an example in which the image processing apparatus according to the present embodiment is an imaging apparatus will be described, the image processing apparatus may be a personal computer (PC) or the like.

FIG. 1A is a block diagram illustrating a configuration example of an imaging device according to the present embodiment.

The lens group 100 includes at least one lens and guides light from a subject to the imaging sensor unit 101. The lens group 100 is configured to be able to control the amount of light incident on the image sensor 101 from the lens group 100, the state of focusing, and the like.

The image sensor 101 converts the light incident from the lens group 100 into image data, and outputs (transmits) the image data to the development processing unit 102. Specifically, the image sensor 101 includes an image sensor such as a CCD or a CMOS, an A / D converter that converts an analog signal into a digital signal, and the like. The image sensor converts light incident from the lens group 100 and imaged by the image sensor into an analog signal (photoelectric conversion). Then, the A / D converter converts the analog signal obtained by the image sensor into a digital signal (the image data).

画素 Each pixel of the image sensor includes an R sub-pixel having a red color filter, a G sub-pixel having a green color filter, and a B sub-pixel having a blue color filter. In each pixel of the image sensor, the arrangement of the R sub-pixel, the G sub-pixel, and the B sub-pixel is a predetermined arrangement. Specifically, in each pixel of the image sensor, one R sub-pixel, one B sub-pixel, and two G sub-pixels are arranged in a mosaic pattern. Such an array is called a “Bayer array” or the like, and the image data output from the imaging sensor unit 101 (A / D converter) is also Bayer array image data (Bayer image data).

(4) The development processing unit 102 performs a development process on the Bayer image data output from the imaging sensor unit 101, and outputs the image data after the development process to the display image generation unit 103. The development processing includes offset adjustment for adding an offset value to a gradation value (R value, G value, B value, etc.), gain adjustment for multiplying a gradation value by a gain value, gamma conversion for converting a gradation characteristic, and the like. . The conversion characteristics (gamma value, gamma curve, and the like) of the gamma conversion are determined in consideration of the characteristics of the lens group 100, the imaging sensor unit 101, and the like. By changing the conversion characteristics of the gamma conversion, image data for broadcasting or image data for theater (for example, image data reproducing the texture and gradation of a movie film) can be generated. The development process includes a process of converting Bayer image data (RGB image data in which each pixel includes one R sub-pixel, one B sub-pixel, and two G sub-pixels) into YCbCr image data. The YCbCr image data is image data in which each pixel value includes a luminance value (Y value) and a color difference value (Cb value and Cr value). The development processing includes correction processing for correcting image distortion due to distortion of the lens group 100 (lens), image stabilization processing for reducing vibration of an image (subject shown in the image) due to vibration of the imaging device, and noise reduction of the image. It also includes noise reduction processing to reduce noise.

The image data output from the development processing unit 102 may not be YCbCr image data. For example, the development process includes a debayer process, and the development processing unit 102 performs the debayer process to convert the Bayer image data into RGB images each including one R subpixel, one G subpixel, and one B subpixel. The data may be converted into image data and output. RGB image data in which each pixel includes one R sub-pixel, one G sub-pixel, and one B sub-pixel may be obtained (generated) by converting YCbCr image data. An RGB value (R value, G value, and B value) can be calculated from the YCbCr value (Y value, Cb value, and Cr value), and a YCbCr value can be calculated from the RGB value.

The image data output from the imaging sensor unit 101 and the development processing unit 102 is captured image data representing a subject, and is image data to be processed by the imaging device (target image data). Note that the target image data is not limited to the captured image data. For example, the target image data may be CG (computer graphic) image data.

The display image generation unit 103 generates display image data (output image data) based on the YCbCr image data output from the development processing unit 102, and outputs the display image data to the feature amount acquisition unit 104 and the IF processing unit 106. . The display image data is image data to be displayed on the display surface. More specifically, the display image generation unit 103 converts the resolution (image size) of the YCbCr image data into the resolution of the display surface, and converts the gradation value (Y value, Cb value, Cr value, etc.) of the YCbCr image data. Adjust the data size (bit width). The display image generation unit 103 also combines the image data representing the predetermined graphic image with the YCbCr image data so that the predetermined graphic image is superimposed on the image represented by the YCbCr image data. The predetermined graphic image is, for example, an image in which the photographing assist information is represented by figures or characters. Further, when the aspect ratio of the YCbCr image data is different from the aspect ratio of the display surface, for example, the display image generation unit 103 determines the YCbCr image data so that the aspect ratio of the display image data matches the aspect ratio of the display surface. Is added. Through these processes, display image data is generated.

FIG. 3A shows an example of display image data to which predetermined image data has been added. In FIG. 3A, the additional image (image represented by predetermined image data) is a black band image (band-shaped black image), and black bands are formed above and below the target image (image represented by YCbCr image data). An image has been added. The state shown in FIG. 3A is called "letterbox" or the like. A black band image may be added to the left and right of the target image, and such a state is called a “pillar box” or the like. Note that the additional image need not be a black band image or an image added to adjust the aspect ratio of display image data. The shape and color of the additional image are not particularly limited. The additional image may be an image on which a picture is drawn.

The feature amount acquiring unit 104 acquires a feature amount from the display image data generated by the display image generating unit 103, and outputs the feature amount to the additional information generating unit 105. Although the feature amount is not particularly limited, in the present embodiment, MaxCLL (Maximum Content Content Light Level) indicating the maximum luminance value of the scene for each scene, and MaxFALL (Maximum Frame Average) indicating the maximum value of the average luminance value of the frame for each scene Light @ Level) is acquired as a feature amount. For this reason, the feature amount (the value of MaxCLL or MaxCLL) may dynamically change for each scene. In MaxCLL and MaxFALL, one frame can be treated as one scene. In other words, MaxCLL can indicate the maximum luminance value of a frame for each frame, and MaxFALL can indicate the average luminance value of a frame for each frame. In the present embodiment, MaxCLL indicating the maximum luminance value of a frame for each frame and MaxFALL indicating the average luminance value of the frame for each frame are acquired as feature amounts.

In the prior art, the entire image area of the display image data (the entire image area including the image area of the target image and the image area of the black band image) is set as the average luminance value (MaxFALL) of the display image data shown in FIG. ) Is obtained. For this reason, the obtained average luminance value differs from the average luminance value of the target image, and differs from the value intended by the photographer (user of the image processing apparatus). A broken line in FIG. 3B indicates MaxFALL obtained from the display image data in FIG. 3A in the related art.

Therefore, in the present embodiment, the feature amount obtaining unit 104 obtains the average luminance value of the image area of the target image as the average luminance value (MaxFALL) of the display image data without considering the image area of the black band image. . Thereby, MaxFALL indicating the average luminance value intended by the photographer (user of the image processing apparatus) can be obtained. The solid line in FIG. 3B indicates MaxFALL obtained from the display image data in FIG. 3A in the present embodiment. According to the additional image, MaxFALL (broken line) of the related art indicates an average luminance value lower than MaxFALL (solid line) of the present embodiment. For this reason, in the related art, display luminance lower than the display luminance intended by the photographer is realized based on MaxFALL (broken line). In this embodiment, since MaxFALL indicating the average luminance value intended by the photographer can be obtained, display luminance intended by the photographer can be realized based on MaxFALL (solid line). Since the display image data is image data generated by the imaging device, the imaging device can individually determine the image area of the black band image and the image area of the target image.

In this embodiment, since the color of the additional image is black, the additional image does not affect the maximum luminance value (MaxCLL) of the display image data. For this reason, the feature amount acquiring unit 104 may acquire the maximum luminance value of the entire image area of the display image data as the maximum luminance value (MaxCLL) of the display image data, or may acquire the maximum luminance value of the image area of the target image. May be obtained. FIG. 3C shows MaxFALL and MaxCLL obtained from the display image data of FIG. 3A in the present embodiment. When the additional image includes a color other than black (such as white), the additional image may affect the maximum luminance value (MaxCLL) of the display image data. In that case, the feature amount acquiring unit 104 may acquire the maximum luminance value of the image area of the target image as the maximum luminance value (MaxCLL) of the display image data without considering the image area of the additional image. As a result, MaxCLL indicating the maximum luminance value intended by the photographer can be more reliably obtained, and the display luminance intended by the photographer can be more reliably realized.

The additional information generation unit 105 generates additional information to be added to the display image data based on the feature amounts (MaxCLL and MaxFALL) acquired by the feature amount acquisition unit 104. Then, the additional information generation unit 105 outputs the additional information to the IF processing unit 106. For example, the additional information is information based on the HDMI standard or the like, and includes feature information indicating MaxCLL, MaxFALL, and the like. Further, the additional information includes area information indicating an image area in which the feature amount (MaxCLL or MaxFALL) has been acquired. In this embodiment, the area information indicates whether MaxFALL is obtained from the entire image area of the display image data or only from the image area of the target image. Note that the area information need not be included in the additional information.

The IF processing unit 106 adds the display image data generated by the display image generation unit 103 and the additional information generated by the additional information generation unit 105 to the display device 107 (external device) connected to the imaging device. Output. Specifically, the display unit 109 is connected to the imaging device by a connection method based on the HDMI standard or the like. Then, the IF processing unit 106 generates a signal in a format compliant with the HDMI standard or the like as a signal including the display image data and the additional information, and outputs the generated signal to the display device 107. The display image data and the additional information may be output separately. The display image data and the additional information may be recorded in a storage device without being output to the display device 107.

The display device 107 can be used as an electronic viewfinder (EVF) of the imaging device. The display device 107 extracts display image data and additional information from a signal received from the imaging device, and displays an image based on the display image data on a display surface at a display luminance based on the additional information (MaxCLL, MaxFALL, or the like). The display luminance is the luminance on the display surface. When the display device 107 is a liquid crystal display device, the display luminance can be adjusted by adjusting the light emission luminance of the backlight unit, the transmittance of the liquid crystal panel, and the like. Specifically, the display brightness can be adjusted by adjusting the voltage and current supplied to the backlight unit, the voltage and current supplied to the liquid crystal panel, and the like. When the display device 107 is an organic EL display device or a plasma display device, the display luminance can be adjusted by adjusting the emission luminance of the display panel (organic EL panel or plasma panel). Specifically, the display luminance can be adjusted by adjusting the voltage or current supplied to the display panel.

FIG. 2A is a flowchart illustrating an example of a processing flow of the imaging apparatus according to the present embodiment.

In step S100, the image sensor 101 starts imaging. In step S101, the development processing unit 102 performs a development process. In step S102, the display image generation unit 103 generates display image data from the image data after the development processing. The image area of the display image data includes at least the image area of the target image. When the state of the display image data is a letter box, a pillar box, or the like, the image area of the display image data further includes an image area of a black band image.

In step S103, the feature amount acquiring unit 104 determines whether or not the image area of the display image data generated in step S102 includes the image area of the black band image. If the image area of the black band image exists (for example, if the state of the display image data is letter box or pillar box), the process proceeds to step S104, and if not, the process proceeds to step S106.

In step S104, the feature amount acquiring unit 104 generates the luminance value of each pixel in the image area (target area) of the target image without acquiring the luminance value in the image area of the black belt image in step S102. Acquire (extract) from display image data. In step S105, the feature amount obtaining unit 104 calculates a feature amount (MaxCLL or MaxFALL) using each luminance value obtained in step S104. For example, MaxCLL indicates the maximum luminance value of the frame (maximum luminance value of each pixel) for each frame, and MaxFALL indicates the average luminance value of the frame (average luminance value of each pixel) for each frame. Show.

In step S106, the feature amount acquiring unit 104 acquires (extracts) the luminance value of each pixel in the entire image area (the entire image area of the display image data) from the display image data generated in step S102. In step S107, the feature amount obtaining unit 104 calculates a feature amount (MaxCLL or MaxFALL) using each of the luminance values obtained in step S106.

In step S108, the additional information generation unit 105 generates additional information based on the feature amounts (MaxCLL and MaxFALL) calculated in step S105 or step S107.

In step S109, IF processing section 106 adds the additional information generated in step S108 to the display image data generated in step S102, and outputs the display image data to display device 107. Then, the display device 107 displays an image on the display surface based on the display image data output from the IF processing unit 106 at a display luminance based on the additional information (MaxCLL, MaxFALL, or the like) output from the IF processing unit 106 ( Start of display).

As described above, according to the present embodiment, in the imaging device (image processing device) separate from the display device, a feature amount that matches the intention of the photographer (user of the image processing device) is acquired, and Feature information that matches the intention of the user is generated. This makes it possible to more reliably realize the display luminance intended by the photographer. For example, even if the display device does not have a function of acquiring a suitable feature amount, it is possible to realize a display luminance that matches the intention of the photographer based on the feature information generated by the imaging device.

<Example 2>
Hereinafter, a second embodiment of the present invention will be described. In the following, points (configuration, processing, and the like) different from the first embodiment will be described in detail, and description of the same points as the first embodiment will be omitted.

FIG. 1B is a block diagram illustrating a configuration example of the imaging apparatus according to the present embodiment. The imaging device according to the present embodiment is not connected to a display device (external device), and includes a display processing unit 108 instead of the IF processing unit 106 of the first embodiment (FIG. 1A). Furthermore, the imaging device according to the present embodiment includes the display unit 109.

The display processing unit 108 generates control information for controlling display brightness and the like based on the additional information (MaxCLL, MaxFALL, or the like) generated by the additional information generation unit 105. The control information can also be said to be “information based on the feature amount acquired by the feature amount acquiring unit 104”. Then, the display processing unit 108 outputs (transmits) the display image data generated by the display image generation unit 103 and the control information generated based on the additional information to the display unit 109. For example, the display unit 109 is connected to the display processing unit 108 by a connection method compliant with the MIPI standard or the like, and the display processing unit 108 generates a signal of a format compliant with the MIPI standard or the like as a control information signal. Output.

The display unit 109 can be used as an electronic viewfinder (EVF) or the like. The display unit 109 displays an image based on the display image data output from the display processing unit 108 on a display surface at a display luminance based on the control information (MaxCLL, MaxFALL, or the like) output from the display processing unit 108. When the display unit 109 is a combination of a liquid crystal panel and a backlight unit, the display luminance can be adjusted by adjusting the light emission luminance of the backlight unit, the transmittance of the liquid crystal panel, and the like. When the display unit 109 is a display panel such as an organic EL panel or a plasma panel, the display luminance can be adjusted by adjusting the light emission luminance of the display panel.

処理 The processing flow of the imaging apparatus according to the present embodiment is the same as that of the first embodiment (FIG. 2A). However, in step S109, the display processing unit 108 generates control information based on the additional information generated in step S108, and outputs display image data (step S102) and control information to the display unit 109. Then, the display unit 109 displays an image based on the display image data output from the display processing unit 108 on the display surface at a display luminance based on the control information (MaxCLL, MaxFALL, or the like) output from the display processing unit 108 ( Start of display).

As described above, according to the present embodiment, the display luminance intended by the photographer (user of the image processing apparatus) can be more reliably realized by the imaging apparatus (image processing apparatus) alone.

<Example 3>
Hereinafter, a third embodiment of the present invention will be described. In the following, differences (configuration, processing, etc.) from the first and second embodiments will be described in detail, and description of the same points as the first and second embodiments will be omitted.

FIG. 1C is a block diagram illustrating a configuration example of the imaging apparatus according to the present embodiment. The imaging apparatus according to the present embodiment has a configuration in which the configuration of the first embodiment (FIG. 1A) and the configuration of the second embodiment (FIG. 1B) are combined. Specifically, the display image generation unit 103 outputs the display image data to the feature amount acquisition unit 104, the IF processing unit 106, and the display processing unit 108. Then, the additional information generation unit 105 outputs the additional information to the IF processing unit 106 and the display processing unit 108.

FIG. 2B is a flowchart illustrating an example of a processing flow of the imaging apparatus according to the present embodiment.

In step S200, the image sensor 101 starts imaging. In step S202, the development processing unit 102 performs a development process. With one type of development processing, image data common to the display device 107 and the display unit 109 may or may not be obtained as image data after development processing. The development process for the display device 107 and the development process for the display unit 109 are separately performed, and the image data for the display device 107 and the image data for the display unit 109 are separately obtained as the image data after the development process. You may be.

In step S202, the display image generation unit 103 generates display image data from the image data after the development processing. In the present embodiment, the display image generation unit 103 individually generates display image data for the display device 107 and display image data for the display unit 109. For this reason, a black band image exists in only one of the two display image data, or a black band image exists in both of the two display image data. In the present embodiment, it is assumed that the display image data for the display device 107 is image data (image data having a black band image; letterbox image data) as shown in FIG. Assume that the display image data for the display unit 109 is image data (image data having no black band image; image data having only the target image) as shown in FIG. A black band image may not exist in the display image data for the display device 107, and a black band image may exist in the display image data for the display unit 109. Note that the display device 107 and the display unit 109 may generate common display image data.

In step S203, the feature amount acquiring unit 104 determines whether or not the image area of the display image data generated in step S202 includes the image area of the black band image. The determination in step S203 is made individually for the display image data for the display device 107 and the display image data for the display unit 109. Steps S204 and S205 are performed on the display image data having the black belt image area, and steps S206 and S207 are performed on the display image data having no black belt image area. In the present embodiment, since the display image data for the display device 107 is the image data of FIG. 3A, the processes of steps S204 and S205 are performed on the display image data for the display device 107. Then, since the display image data for the display unit 109 is the image data of FIG. 3D, the processes of steps S206 and S207 are performed on the display image data for the display unit 109.

In step S204, the feature amount obtaining unit 104 generates the luminance value of each pixel in the entire image area (the entire image area including the image area of the target image and the image area of the black band image) in step S202. Acquire (extract) from display image data. In step S205, the feature amount obtaining unit 104 calculates a feature amount (MaxCLL or MaxFALL) using each luminance value obtained in step S206. Specifically, the feature amount acquiring unit 104 acquires the feature amount of the image area of the target image using the luminance value of each pixel in the image area of the target image, as in the first and second embodiments. Further, the feature amount obtaining unit 104 obtains a feature amount of the entire image area using the luminance value of each pixel in the entire image area.

In step S206, the feature amount obtaining unit 104 obtains (extracts) the luminance value of each pixel in the entire image area (the entire image area of the display image data) from the display image data generated in step S202. In step S207, the feature amount obtaining unit 104 calculates a feature amount (MaxCLL or MaxFALL) using each of the luminance values obtained in step S206. Specifically, similarly to the first and second embodiments, the feature amount obtaining unit 104 obtains the feature amounts of the entire image area using the luminance values of each pixel in the entire image area.

In step S208, the additional information generation unit 105 generates additional information based on the feature amounts (MaxCLL and MaxFALL) calculated in step S105 or step S107. As described above, in the present embodiment, the processing of steps S204 and S205 is performed on the display image data for the display device 107. For this reason, in the additional information for the display device 107, the area information indicating the image area of the target image is associated with the feature amount of the image area of the target image. Are associated with each other. Steps S206 and S207 are performed on the display image data for the display unit 109. Therefore, the additional information for the display unit 109 includes information indicating that the feature amount is not that of the image including the black band image.

In step S209, the IF processing unit 106 and the display device 107 display an image based on the display image data generated in step S202 with the display luminance based on the additional information generated in step S208, as in the first embodiment. It is displayed on the display surface of the device 107. As in the second embodiment, the display processing unit 108 and the display unit 109 display an image based on the display image data generated in step S202 with a display luminance based on the additional information generated in step S208. To be displayed.

In the present embodiment, as the feature amount (MaxCLL or MaxFALL) for the display device 107, the feature amount of the image area of the target image and the image area including the entire image area (the image area of the target image and the image area of the black belt image) are used. Are obtained. Therefore, when displaying an image on the display device 107, the above two feature amounts can be switched and used. If the feature amount of the image area of the target image is used, the display luminance intended by the photographer can be realized as in the first embodiment. If the feature amount (MaxFALL) of the entire image area is used, the display luminance is reduced by the black band image, so that the power consumption of the display device 107 can be reduced.

As described above, according to this embodiment, since both the processing of the first embodiment and the processing of the second embodiment are performed, both the effects of the first embodiment and the effects of the second embodiment can be obtained. Further, since the additional information for the display device 107 and the additional information for the display unit 109 are generated by the same device, the appearance of the image displayed on the display device 107 and the appearance of the image displayed on the display unit 109 are changed. Can be close to each other. When displaying an image, the feature amount of the target image and the feature amount of the image including the target image and the black band image can be switched and used, so that the display brightness that matches the intention of the photographer (user of the image processing apparatus) And display brightness for reducing power consumption can be suitably switched.

Note that, as shown in FIG. 1D, a pair for the display device 107 and a pair for the display unit 109 may be separately provided as a pair of the feature amount acquisition unit 104 and the additional information generation unit 105.

<Example 4>
Hereinafter, a fourth embodiment of the present invention will be described. In the following, differences (configuration, processing, etc.) from the first to third embodiments will be described in detail, and description of the same points as the first to third embodiments will be omitted.

ブロック A block diagram illustrating a configuration example of the imaging apparatus according to the present embodiment is the same as FIG.

(E), (F), and (G) of FIG. 3 show examples of display image data to which predetermined image data generated by the display image generation unit 103 is added. In FIG. 3E, the additional image (the image represented by the predetermined image data) is a character image (a time code image representing the shooting time or the reproduction time), and the target image (the image represented by the YCbCr image data). A character image is added to the lower right of the parentheses. A character image may be surrounded by a frame as shown in FIG. 3E, but may be only a character as shown in FIG. 3F. The additional image in FIG. 3G is a frame image, and indicates, for example, an area to be actually recorded in the display image data. The shape and color of the additional image (character image or frame image) and the superimposition position in the display image data are not particularly limited. The additional image may be an image in which characters and patterns are drawn.

In the present embodiment, the feature amount obtaining unit 104 sets the average luminance value of the target image image area as the average luminance value (MaxFALL) or the maximum luminance value (MaxCLL) of the display image data without considering the image area of the character image. Get value and maximum brightness value. This makes it possible to obtain MaxFALL indicating the average luminance value intended by the photographer (user of the image processing apparatus) and the maximum luminance value (MaxCLL), thereby realizing the display luminance intended by the photographer. When the average luminance value and the maximum luminance value of the entire image area of the display image data including the image area of the character image are obtained, the character image is black (when the luminance value is low) or white (when the luminance value is high). ), The display luminance contrary to the photographer's intention is realized. Since the display image data is image data generated by the imaging device, the imaging device can individually determine the image area of the character image and the image area of the target image. Since the determination method can be determined not only by the area information but also by the pixel value and the gradation value of the image area of the character image, the image area of the additional image includes the unique pixel value and the gradation value specified by the photographer. It can also be an image area. If the target image includes a pixel value or a gradation value designated by the photographer as an image area of the additional image, the specified pixel value or the gradation value is not included in the target image. It is necessary to change the value to a value of ± 1 of the designated value) to obtain a unique pixel value or gradation value.

FIG. 2C is a flowchart illustrating an example of a processing flow of the imaging apparatus according to the present embodiment.

(4) In step S300, the imaging sensor unit 101 starts imaging. In step S301, the developing unit 102 performs a developing process. In step S302, the display image generation unit 103 generates display image data from the image data after the development processing. The image area of the display image data includes at least the image area of the target image. When a character image is superimposed upon generation of display image data, the image area of the display image data further includes the image area of the character image. In the present embodiment, an example of an image on which a character image is superimposed is described. However, the image may be a frame image shown in FIG. And so on.

In step S303, the feature amount acquiring unit 104 determines whether the image area of the display image data generated in step S302 includes the image area of the character image. If the image area of the character image exists, the process proceeds to step S304; otherwise, the process proceeds to step S306.

In step S304, the feature amount acquiring unit 104 displays the luminance value of each pixel in the image area (target area) of the target image without acquiring the luminance value in the image area of the character image in the display generated in step S302. Acquire (extract) from image data. In step S305, the feature amount obtaining unit 104 calculates a feature amount (MaxCLL or MaxFALL) using each of the luminance values obtained in step S304. For example, MaxCLL indicates the maximum luminance value of the frame (maximum luminance value of each pixel) for each frame, and MaxFALL indicates the average luminance value of the frame (average luminance value of each pixel) for each frame. Show.

In step S306, the feature amount obtaining unit 104 obtains (extracts) the luminance value of each pixel in the entire image area (the entire image area of the display image data) from the display image data generated in step S302. In step S307, the feature amount obtaining unit 104 calculates a feature amount (MaxCLL or MaxFALL) using each of the luminance values obtained in step S306.

In step S308, the additional information generation unit 105 generates additional information based on the feature amounts (MaxCLL and MaxFALL) calculated in step S305 or step S307.

In step S309, the IF processing unit 106 outputs the display image data generated in step S302 to the display device 107 with the additional information generated in step S308 added. Then, the display device 107 displays an image on the display surface based on the display image data output from the IF processing unit 106 at a display luminance based on the additional information (MaxCLL, MaxFALL, or the like) output from the IF processing unit 106 ( Start of display).

The configuration example of the imaging apparatus in the case where there is the image area of the character image described in the present embodiment is an example, and the display brightness that matches the intention of the photographer can be realized even with the configurations of the second and third embodiments.

Each block of the first to fourth embodiments (FIGS. 1A to 1D) may or may not be individual hardware. The functions of two or more blocks may be realized by common hardware. Each of a plurality of functions of one block may be realized by individual hardware. Two or more functions of one block may be realized by common hardware. Also, each block may or may not be realized by hardware. For example, the device may have a processor and a memory in which a control program is stored. Then, the functions of at least some of the blocks of the device may be realized by the processor reading out the control program from the memory and executing the control program.

The first to fourth embodiments (including the above-described modified examples) are merely examples, and the configuration obtained by appropriately modifying or changing the configuration of the first to fourth embodiments within the scope of the present invention. Are also included in the present invention. A configuration obtained by appropriately combining the configurations of the first to fourth embodiments is also included in the present invention.

<Other Examples>
The present invention supplies a program for realizing one or more functions of the above-described embodiments to a system or an apparatus via a network or a storage medium, and one or more processors in a computer of the system or the apparatus read and execute the program. This processing can be realized. Further, it can be realized by a circuit (for example, an ASIC) that realizes one or more functions.

The present invention is not limited to the above embodiment, and various changes and modifications can be made without departing from the spirit and scope of the present invention. Therefore, the following claims are appended to make the scope of the present invention public.

This application claims priority based on Japanese Patent Application No. 2018-188954 filed on Oct. 4, 2018 and Japanese Patent Application No. 2019-054487 filed on Mar. 22, 2019. Yes, and the entire contents thereof are incorporated herein.

# 103: Display image generation unit # 104: Feature acquisition unit # 105: Additional information generation unit # 106: IF processing unit # 108: Display processing unit

Claims

Generating means for generating output image data based on the target image data;
Acquiring means for acquiring a feature amount from the output image data;
Output means for outputting the output image data and feature information based on the feature amount,
Has,
When the image area of the output image data includes a first area that is an image area of the target image data and a second area that is an image area of predetermined image data, the obtaining unit may include the first area. An image processing apparatus characterized by acquiring a feature amount of (1).
The image processing apparatus according to claim 1, wherein the feature amount is an average luminance value.
The image processing apparatus according to claim 1, wherein the feature amount is a maximum luminance value.
4. The image processing apparatus according to claim 1, wherein the output unit outputs the output image data and the characteristic information to a display device.
The image processing device is an imaging device,
The image processing apparatus according to any one of claims 1 to 4, wherein the target image data is captured image data.
The image processing apparatus according to any one of claims 1 to 5, wherein the predetermined image data is image data added to adjust an aspect ratio of the output image data.
The image processing apparatus according to claim 1, wherein the predetermined image data is image data added to indicate shooting information or reproduction information of the output image data.
When the image area of the output image data includes a first area, which is an image area of the target image data, and a second area, which is an image area of predetermined image data, the obtaining unit includes: The image processing apparatus according to any one of claims 1 to 7, further acquiring a feature amount of the entire image area.
The image processing apparatus according to any one of claims 1 to 8, wherein the output unit further outputs area information indicating an image area in which the feature amount has been acquired.
A generating step of generating output image data based on the target image data;
An obtaining step of obtaining a feature amount from the output image data,
An output step of outputting the output image data and feature information based on the feature amount,
Has,
In the case where the image area of the output image data includes a first area that is an image area of the target image data and a second area that is an image area of predetermined image data, in the acquiring step, the first area An image processing method characterized by acquiring a feature amount of (1).
A program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 9.