JP2023172980A - Image display device and image display method - Google Patents

Abstract

To provide an image display device and an image display method for properly adjusting image quality regarding a specific color.SOLUTION: An image display device includes: an image acquisition unit for acquiring an input image; an object detection unit which detects a predetermined object from the input image; a size acquisition unit which determines a size of the predetermined object; a determination unit which determines whether to adjust image quality regarding a specific color of the predetermined object on the basis of the size of the predetermined object; an image quality adjustment unit which adjusts image quality of a region which is at least a part of the input image regarding the specific color; and a display control unit which controls the input image with adjusted image quality to be displayed on a display panel.SELECTED DRAWING: Figure 3

Description

The present invention relates to an image display device, an image display method, and the like.

2. Description of the Related Art Conventionally, methods are known for adjusting the image quality of specific colors in an image. For example, Patent Document 1 discloses a method of performing correction suitable for a specific color area when the proportion of pixels of a specific color is large, based on the result of counting the number of pixels that include a specific color such as the skin color of a face. has been done.

Japanese Patent Application Publication No. 2004-297617

The method disclosed in Patent Document 1 determines whether or not to perform correction based on pixel information of a specific color such as skin color. However, in situations where part of the face is hidden by a mask, sunglasses, etc., or the face is blurred, the number of pixels for skin color decreases. Therefore, with the method of Patent Document 1, even when the proportion of the face in the input image is large, there is a possibility that suitable correction cannot be performed on the skin color of the face.

According to some aspects of the present disclosure, it is possible to provide an image display device, an image display method, and the like that appropriately adjust image quality regarding a specific color.

One aspect of the present disclosure includes: an image acquisition unit that acquires an input image; an object detection unit that detects a predetermined object from the input image; a size acquisition unit that determines the size of the predetermined object; a determination unit that determines whether or not to adjust image quality with respect to a specific color that the predetermined object has based on the size; and when the determination unit determines that the image quality is to be adjusted, with respect to the specific color; An image display device including: an image quality adjustment unit that adjusts the image quality of at least a part of the input image; and a display control unit that controls displaying the input image with the image quality adjusted on a display panel. do.

Another aspect of the present disclosure is to obtain an input image, detect a predetermined object from the input image, determine the size of the predetermined object, and determine whether the predetermined object is determined based on the size of the predetermined object. If it is determined that the image quality is to be adjusted, the image quality of at least a part of the input image is adjusted with respect to the specific color, and the image quality is adjusted. The present invention relates to an image display method that controls displaying the adjusted input image on a display panel.

1 is an example of the appearance of a television receiver that is an example of an image display device. 1 is a configuration example of a television receiving device that is an example of an image display device. It is an example of a structure of an image display device. It is a flow chart explaining processing of an image display device. This is an example of object detection results. 3 is a flowchart illustrating a size index value acquisition process. This is an example of the relationship between size statistics and size index values. 7 is a flowchart illustrating processing for determining whether or not image quality adjustment is necessary for a specific color. 7 is a flowchart illustrating processing for determining whether or not image quality adjustment is necessary for a specific color. This is an example of time-series changes in index values. This is an example of time-series changes in hysteresis processing results. This is an example of time-series changes in chattering filter processing results. It is an example of a structure of an image display device. It is a flow chart explaining processing of an image display device. FIG. 3 is a diagram illustrating input and output of scene determination. This is an example of a scene determination result (scene index value) and an object detection result. It is a flow chart explaining acquisition processing of an index value.

This embodiment will be described below with reference to the drawings. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant description will be omitted. Note that this embodiment described below does not unduly limit the content described in the claims. Furthermore, not all of the configurations described in this embodiment are essential configuration requirements of the present disclosure.

1. First Embodiment 1.1 System Configuration FIG. 1 is a diagram showing an example of the configuration of a television receiving device 10, which is an example of an image display device 100 according to this embodiment. The television receiving device 10 is, for example, a device that receives broadcast waves of television broadcasting and displays images based on the received broadcast waves on a display panel 16. Note that FIG. 1 is an example of the external configuration of the television receiving device 10, and various modifications can be made to the specific shape.

FIG. 2 is a diagram showing an example of the hardware configuration of the television receiving device 10. As shown in FIG. The television receiving device 10 includes a processor 11, a tuner 12, a communication interface 13, a memory 14, an operation interface 15, and a display panel 16. However, the configuration of the television receiving device 10 is not limited to that shown in FIG. 2, and various modifications such as omitting some configurations or adding other configurations are possible.

The processor 11 can be a variety of processors, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a DSP (Digital Signal Processor). Further, the processor 11 may include hardware such as an ASIC (application specific integrated circuit) and an FPGA (field programmable gate array). The processor 11 is connected to each part of the television receiving device 10 and controls each part.

The tuner 12 includes an interface that receives television broadcast waves of a specific frequency, a circuit that processes the received broadcast waves, and the like. For example, the interface here is a terminal for connecting an antenna cable. Further, the circuits and the like here may include an RF (Radio Frequency) circuit, a decoding circuit that performs decoding, an A/D conversion circuit that performs analog/digital conversion, and the like. The tuner 12 receives a signal corresponding to a television broadcast wave from an antenna, and outputs a video signal based on the signal to the processor 11. The video signal here is, for example, a set of multiple images acquired in time series.

The communication interface 13 is an interface that performs communication according to a communication method such as IEEE802.11, and in a narrow sense, it is a communication chip for performing communication according to the communication method. For example, the television receiving device 10 communicates with a public communication network such as the Internet via the communication interface 13. Specifically, the television receiving device 10 may perform a process of connecting to a content server via a public communication network and receiving video content such as a movie from the content server.

The memory 14 is a work area for the processor 11 and stores various information. The memory 14 may be a semiconductor memory such as an SRAM (Static Random Access Memory) or a DRAM (Dynamic Random Access Memory), a register, or a magnetic storage device such as an HDD (Hard Disk Drive). Alternatively, it may be an optical storage device such as an optical disk device.

The operation interface 15 is an interface used when a user operates the television receiving device 10, and may be a button provided on the casing of the television receiving device 10, or an interface used for communication with a remote control (for example, It may also be an infrared receiving device).

The display panel 16 is a display that displays images. The display panel 16 may be, for example, a liquid crystal display, an organic EL display, or another type of display.

FIG. 3 is a diagram showing a configuration example of the image display device 100 according to the present embodiment. The image display device 100 includes an image acquisition section 110, an object detection section 120, a size acquisition section 130, a determination section 150, an image quality adjustment section 160, and a display control section 170. However, the configuration of the image display device 100 is not limited to that shown in FIG. 3, and modifications such as addition or omission of configurations are possible.

The image display device 100 of this embodiment corresponds to, for example, the television receiving device 10 shown in FIG. 2. For example, each part of the image display device 100 may be realized by the processor 11 of the television receiving device 10. For example, the memory 14 stores programs and various data. More specifically, the memory 14 stores computer-readable instructions, and when the processor 11 executes the instructions, the functions of each part of the image display device 100 shown in FIG. 3 are realized as processing. Ru. Each section of the image display device 100 includes an image acquisition section 110, an object detection section 120, a size acquisition section 130, a determination section 150, an image quality adjustment section 160, and a display control section 170. Further, each section of the image display device 100 may include a scene acquisition section 140, which will be described later using FIG. 10. The instructions here may be instructions of an instruction set that constitutes a program, or instructions that instruct a hardware circuit of the processor 11 to operate.

The image acquisition unit 110 acquires an input image. The input image here represents an image to be displayed on the display panel 16. For example, the input image may be an image included in a television broadcast video signal acquired via the antenna and tuner 12. The video signal here is Rec., which is a standard for encoding etc. in high-definition television broadcasting. 709 (BT.709). Rec. In 709, for example, RGB color space parameters are defined.

An example in which the input image corresponds to a television broadcast video signal will be described below. However, the method of this embodiment is not limited to this. For example, the input image may be an image included in a video signal that the communication interface 13 obtains from a content server. Further, the television receiving device 10 may be connected to a reproduction device for media such as a BD (Blu-ray Disc, registered trademark), and an image included in a video signal read from the media may be used as an input image.

Furthermore, the object detection unit 120 detects a predetermined object from the input image. The predetermined object here is, for example, a person's face. However, the predetermined object may be other objects such as a blue sky or a green landscape. Note that the green scenery represents, for example, a scenery with plants such as plants.

For example, when the target is a person's face, the object detection unit 120 detects parts included in the face such as eyes, mouth, nose, etc., and extracts the person's face from the input image based on the types and positional relationships of the detected parts. may be detected. For example, the object detection unit 120 may identify the area corresponding to the face by detecting the outline of the person's face based on the positional relationship of the parts. In object detection, processing is performed to identify not only the presence or absence of the object, but also the position and range in which a predetermined object exists in the input image. Furthermore, the results of machine learning may be used when detecting objects from images. For example, a convolutional neural network (CNN) may be used that sets a detection window in a part of the input image and determines whether or not there is a face in the area included in the detection window. By repeating the process while changing the size and shape of the detection window, it becomes possible to detect a predetermined object from the input image. Note that various methods using neural networks, such as YOLO (You Only Look Once), which are suitable for real-time detection, are known, and these can be widely applied in this embodiment. In addition, the object detection method is not limited to the above example, and a wide range of known methods can be applied.

The size acquisition unit 130 acquires the size of a predetermined object detected by the object detection unit 120. For example, when a rectangular area D1 or the like that includes a predetermined object is detected as a result of object detection, as will be described later using FIG. The size can be obtained based on the product of the horizontal length (number of pixels) and the horizontal length (number of pixels).

The determination unit 150 determines whether or not to adjust the image quality regarding a specific color based on the size of a predetermined object. A specific determination method will be described later using FIGS. 6 to 8B and the like.

When it is determined that the image quality should be adjusted, the image quality adjustment unit 160 adjusts the image quality of at least a part of the input image with respect to a specific color that a predetermined object has. For example, memory colors, which are colors that people remember as images of certain objects, are known. The memory color may differ from the actual color of the object. In this case, if the image is displayed faithfully to the actual color, the user who views the image may feel uncomfortable because the color is different from the memory color. Therefore, the image quality adjustment unit 160 may perform image quality adjustment processing on a specific color included in a predetermined object so as to bring the tint of the specific color closer to the memory color. For example, since it is known that the saturation of the memory color of a person's skin color is lower than the actual color, the image quality adjustment unit 160 may perform image quality adjustment processing to lower the saturation of the skin color area. Furthermore, it is known that the saturation of the memory color of blue skies and green scenery is higher than the actual color, so the image quality adjustment unit 160 adjusts the saturation of the blue region corresponding to the sky and the green region corresponding to the green scenery. You may perform adjustment processing to increase the image quality. Further, image quality adjustment is not limited to saturation adjustment, but may also include brightness and hue adjustment. Further, the adjustment of image quality regarding a specific color in this embodiment is not limited to adjustment based on memory colors.

Further, the color space determined by the above-mentioned television broadcasting standards may be narrower than the color space that can be expressed by the display panel 16. In this case, if the input image is displayed as is, even if the display panel 16 has the ability to express vivid colors, it may not be able to demonstrate its ability. Therefore, the image quality adjustment unit 160 may perform image quality adjustment processing to increase at least one of brightness and saturation on the input image, regardless of the object detection result. For example, the image quality adjustment unit 160 may adjust the image quality of the input image by combining both an adjustment process related to a specific color of a predetermined object and an adjustment process that does not depend on the object.

The display control unit 170 performs control to display on the display panel 16 the input image whose image quality has been adjusted by the image quality adjustment unit 160. For example, the display control unit 170 outputs an image signal and a timing control signal that instructs the control timing of a drive circuit included in the display panel 16 to the display panel 16. Note that the image quality adjustment for the specific color may be omitted depending on the determination by the determination unit 150, and the image to be displayed here may be an image on which no adjustment for the specific color has been performed.

Note that although an example in which the image display device 100 is the television receiving device 10 has been described above, the present invention is not limited to this. For example, the image display device 100 may correspond to a device such as a set-top box. In this case, the display control performed by the display control unit 170 may be a control for outputting an input image whose image quality has been adjusted to a device having the display panel 16 (television receiving device 10, display, etc.).

In the method of this embodiment, as described above, the image quality regarding a specific color of a predetermined object is adjusted based on the result of object detection. Various methods of object detection can be considered as described above, but even when a part of a given object is hidden, the given object can be detected with high accuracy, so it is possible to appropriately determine whether or not image quality adjustment is necessary for a particular color. becomes possible. For example, when the predetermined object is a face, even if part of the face is hidden by a mask, sunglasses, etc. and the number of skin-colored pixels decreases, it can be determined that image quality adjustment is necessary if the size of the face is large. As a result, compared to the conventional method that uses the number of pixels of a color corresponding to a specific color, a region in which pixels of the color are continuous, etc., omissions in image quality adjustment can be suppressed.

Further, part or all of the processing performed by the image display device 100 of this embodiment may be realized by a program. The processing performed by the image display device 100 is, for example, the processing performed by the processor 11 of the television receiving device 10.

The program according to this embodiment can be stored in a non-transitory information storage device (information storage medium), which is a computer-readable medium, for example. The information storage device can be realized by, for example, an optical disk, a memory card, an HDD, or a semiconductor memory. The semiconductor memory is, for example, a ROM. The image display device 100, processor 11, etc. perform various processes of this embodiment based on programs stored in the information storage device. That is, the information storage device stores programs for making the computer function as each part of the image display device 100. A computer is a device that includes an input device, a processing section, a storage section, and an output section. Specifically, the program according to this embodiment is a program for causing a computer to execute each step described later using FIG. 4 and the like.

For example, the program according to this embodiment causes a computer to function as the image acquisition section 110, object detection section 120, size acquisition section 130, determination section 150, image quality adjustment section 160, and display control section 170 of the image display device 100.

Further, the method of this embodiment may be applied to an image display method including the following steps. The image processing method includes the steps of acquiring an input image, detecting a predetermined object from the input image, determining the size of the predetermined object, and determining the specificity of the predetermined object based on the size of the predetermined object. a step of determining whether or not to adjust image quality related to color; and, if it is determined that image quality is to be adjusted, a step of adjusting image quality of at least a part of the input image with respect to a specific color; The method includes a step of controlling display of an input image on a display panel. For example, the image display method may include steps described below using FIGS. 4, 6, 8A, 8B, etc.

1.2 Details of Processing FIG. 4 is a flowchart illustrating the processing of the image display device 100 according to the present embodiment. The processing shown in FIG. 4 may be performed on images of each frame, for example, when video information that is a set of time-series images is acquired.

First, in step S101, the image acquisition unit 110 acquires an input image. For example, the image acquisition unit 110 acquires an image of one given frame from the video information acquired via the tuner 12 or the like, and outputs it to the object detection unit 120.

In step S102, the object detection unit 120 performs object detection processing to detect a predetermined object in the input image acquired from the image acquisition unit 110. As mentioned above, this process may be a process of detecting the outline of a given object based on the structures included in the object (eyes, nose, etc.), or may be a process of applying machine learning such as CNN or YOLO. The process may be performed using a different object detection method, or may be a process using another object detection method.

FIG. 5 is a diagram illustrating an example of a detection result obtained by object detection processing by the object detection unit 120. Image IM in FIG. 5 corresponds to the input image. In the example of FIG. 5, the input image (image IM) includes two human faces F1 and F2. In this case, the object detection unit 120 obtains, for example, a rectangular area D1 that includes the face F1 and a rectangular area D2 that includes the face F2 as the detection results of the object detection process. For example, the detection result is information that specifies a rectangular area, and may be a set of coordinates of the reference point of the rectangular area (for example, upper left coordinates and lower right coordinates), or may be a set of coordinates of the reference point of the rectangular area (for example, It may be a set of the upper left coordinate) and the length in the vertical and horizontal directions, or it may be other information. If a plurality of predetermined objects are detected, information specifying a rectangular area for each predetermined object is obtained. For example, the detection result may include the number of detected predetermined objects and information specifying a rectangular area corresponding to each predetermined object.

Note that the detection result is not limited to a rectangular area. For example, when the object detection process is a process of detecting the outline of a face from structures such as eyes and nose, the object detection unit 120 may obtain an area surrounded by the outline as a detection result. Furthermore, in the process using a neural network, the probability of whether or not each pixel of the input image belongs to a face may be determined for each pixel of the input image, instead of performing the process for each detection window. In this case, the detection result may be a set of pixels whose probability of belonging to a face is equal to or higher than a predetermined threshold, and the shape is not limited to a rectangle. For example, the detection result may include the number of detected predetermined objects and information specifying the area corresponding to each predetermined object. The object detection unit 120 outputs the detection result to the size acquisition unit 130.

In step S103, the size acquisition unit 130 determines the size of the predetermined object detected by the object detection unit 120. When a plurality of objects are detected as shown in FIG. 5, the object detection unit 120 calculates the size of each object. The size here is information representing the size, and may be the product of the length in the vertical direction and the length in the horizontal direction of the rectangular area, or may be the total number of pixels included in the detection result. The size acquisition unit 130 outputs the determined size to the determination unit 150. For example, the size acquisition unit 130 may output the number of detected predetermined objects and the size of each predetermined object to the determination unit 150.

In step S104, the determination unit 150 determines whether or not to adjust the image quality regarding the specific color included in the predetermined object, based on the size output from the size acquisition unit 130.

For example, when the size of a predetermined object is larger than a predetermined size, the determination unit 150 may determine that image quality regarding a specific color should be adjusted. In this way, it is possible to determine whether or not to adjust the specific color based on the size of the predetermined object itself, regardless of the number of pixels corresponding to the specific color. Therefore, if a predetermined object is captured in a large size in the input image, even if a part of the skin color area is blocked by a mask or the like, the specific color caused by the predetermined object is adjusted. Therefore, for example, when displaying an image in which a person's face is displayed in a large size, it is possible to increase the probability that image quality adjustment for skin color will be performed. For example, since it is possible to prevent a specific color included in a large displayed object from deviating from the memory color, it is possible to prevent a user from feeling uncomfortable when viewing the displayed image. On the other hand, if the predetermined object is small, the effect of the predetermined object on the user is small, and thus adjustment of image quality can be omitted.

For example, the determination unit 150 may determine an index value based on the size of a predetermined object, and determine whether or not to adjust the image quality regarding a specific color based on the index value and a given threshold value. In this way, it becomes possible to appropriately determine whether the size is large or not. Furthermore, by adjusting the method of calculating the index value and the threshold value, it becomes possible to flexibly change the criteria for determining whether or not to adjust the image quality regarding a specific color. For example, the process in step S104 may include a process of calculating an index value, and a process of determining whether adjustment of image quality regarding a specific color is necessary based on the index value. Examples of each process will be described below.

FIG. 6 is a flowchart illustrating a process for determining a size index value, which is an index value related to size, in the determination process in step S104. First, in step S201, the determination unit 150 determines whether the number of detected predetermined objects is one or more.

If the number of detected predetermined objects is 1 or more (step S201: Yes), the determination unit 150 calculates size statistics based on the size of each predetermined object output from the size acquisition unit 130 in step S202. Calculate the amount.

For example, the determination unit 150 may determine the size of the largest predetermined object among the detected one or more predetermined objects as the size statistic. Alternatively, the determination unit 150 may determine the total value, average value, median value, etc. of the sizes of one or more predetermined objects as the size statistics. At this time, the predetermined objects used to calculate the total value etc. may be all of the detected predetermined objects, or may be the top n objects in terms of size (n is an integer of 2 or more). good. Here, n may be a fixed value or a value dynamically determined depending on the number of detected predetermined objects. Alternatively, the determination unit 150 may determine the minimum value among the top n sizes of one or more predetermined objects as the size statistic.

Further, the determination unit 150 may obtain the ratio of the above-mentioned maximum value to the entire input image as a size statistic. For example, the determination unit 150 may determine the size of the input image based on the resolution of the input image. The size of an input image is the product of the number of pixels in the vertical direction and the number of pixels in the horizontal direction. For example, when the maximum size of a detected predetermined object is B and the size of the input image is A, the value C representing the size statistics may be C=(B/A)×100. . Moreover, B here may be replaced with a value such as a total value, an average value, a median value, or a minimum value, as described above.

Note that the determination unit 150 may calculate the index value based on the position of a predetermined object detected from the input image. For example, the determination unit 150 may use the position of a predetermined object to calculate the size statistics described above. For example, the determining unit 150 may calculate the size statistics described above by multiplying the size of each predetermined object determined by the size acquiring unit 130 by a weight according to the position. For example, the determination unit 150 may use a weight that increases as the position of the predetermined object approaches the center of the input image, and decreases as the position of the predetermined object moves away from the center. For example, even if the objects have the same size, an object closer to the center of the input image is evaluated to be relatively large in size, and an object far from the center is evaluated to be relatively small in size. In this way, in calculating the size statistics, it is possible to relatively increase the contribution of objects that tend to attract the user's attention, so that, for example, it is possible to suppress omissions in adjusting the image quality of objects that tend to attract the user's attention.

After calculating the size statistics, in step S203, the determination unit 150 calculates a size index value based on a comparison process between the size statistics and a given threshold TH. The size index value here is an index representing the magnitude of the size, and is information that, for example, the larger the size, the larger the value. Further, the threshold value here may be, for example, a value greater than 0 and less than or equal to 100.

For example, if the size statistic is greater than or equal to the threshold TH (step S203: Yes), the determination unit 150 sets the size index value to 100 in step S204. On the other hand, if the size statistic is less than the threshold TH (step S203: No), in step S205, the determination unit 150 determines a size index value based on interpolation processing.

FIG. 7 is a diagram showing an example of the relationship between size statistics, threshold value TH, and size index value. For example, as shown in FIG. 7, in step S205, the determination unit 150 may obtain the size index value by linear interpolation. However, FIG. 7 is an example of the relationship between the size statistics, the threshold TH, and the size index value, and the size index value may be determined based on other relationships. For example, interpolation processing using a nonlinear function may be performed in a range where the size statistics are less than the threshold TH.

If the number of detected predetermined objects is 0 (step S201: No), the size index value is set to 0 in step S206.

As described above, when a plurality of predetermined objects are detected from the input image, the determination unit 150 may calculate the index value based on the size of at least one predetermined object. For example, as described above, the maximum size of an object may be used for calculating the index value, or the average value of the sizes of two or more objects may be used for calculating the index value. In this way, even when there are a plurality of predetermined objects, it is possible to appropriately evaluate the size of the predetermined object in the input image. In other words, regardless of the number of predetermined objects included in the input image, it is possible to appropriately quantify the influence that a predetermined object has on a user.

Note that, as shown in FIG. 7, in this embodiment, conversion processing from size statistics to size index values may be performed. The size statistic has a maximum value of 100, for example, but in actual input images, there are not many cases in which a person's face covers the entire image. For example, the above-mentioned B is likely to have a smaller value than A, and the probability that the size statistic will be 100 or a value in the vicinity is low. In this regard, for the size index value in the example of FIG. 7, values around 100, which is the maximum value, are also appropriately used. That is, by converting to a size index value, it becomes possible to adjust the degree of variation in values within a numerical range (for example, from 0 to 100). In particular, as will be described later in the second embodiment, when comparing the size index value with other index values such as the scene index value, the magnitude relationship is It becomes possible to compare them appropriately. However, the determination unit 150 is not prevented from using the size statistics themselves as the size index value. In this case, steps S203 to S205 in FIG. 6 can be omitted, and the size statistics obtained in step S202 directly serve as the size index value.

FIG. 8A is a flowchart illustrating the process of determining whether image quality adjustment is necessary for a specific color based on the index value, among the determination processes in step S104. The processing here may be processing based on the index value and a given threshold value TH0. The index value is the size index value described above. Furthermore, although the given threshold TH0 here is different information from the threshold TH used for comparison with the size statistics, this does not prevent the threshold TH0 and the above-mentioned threshold TH from being set to the same value. .

As shown in FIG. 8A, first in step S301, it is determined whether the size index value is equal to or greater than a threshold value TH0. In the example of FIG. 7, the size index value is a value between 0 and 100, and TH0 may be set within a range between 0 and 100.

If the size index value is equal to or greater than the threshold value TH0 (step S301: Yes), in step S302, the determination unit 150 determines to perform image quality adjustment regarding the specific color. If the size index value is less than the threshold TH0 (step S301: No), in step S303, the determination unit 150 determines not to perform image quality adjustment for the specific color. In this way, it is possible to determine whether or not image quality adjustment for a specific color is necessary based on the magnitude of the size index value.

Furthermore, the process of determining whether image quality adjustment is necessary for a specific color based on the index value is not limited to the process shown in FIG. 8A. FIG. 8B is a flowchart illustrating another process of determining whether or not to adjust the image quality regarding a specific color based on the index value and a given threshold value, among the determination processes in step S104.

As shown in step S401, the determination unit 150 may perform filter processing on the index value to suppress time-series fluctuations. The determination unit 150 determines whether or not to adjust the image quality based on the index value after the filter processing. By suppressing time-series fluctuations in the size index value, frequent changes in the determination result can be suppressed, for example, even if the size index value fluctuates minutely in the vicinity of the threshold value. Since it is possible to suppress frequent changes in the image quality of a specific color, it is possible to suppress the sense of discomfort given to the user.

The filter processing here includes hysteresis processing, which performs threshold judgment using multiple threshold values that differ depending on the direction of change in the index value, and chattering filter processing, which changes the value on the condition that the same value is obtained a predetermined number of times or more. It may also include. In this way, time-series fluctuations in index values can be appropriately suppressed. Note that an example will be described below in which the chattering filter process is a process in which a value is changed on the condition that the same value is continuously acquired a predetermined number of times or more.

9A to 9C are diagrams illustrating a specific example of the filtering process in step S401. FIG. 9A is a diagram illustrating an example of time-series changes in index values. The horizontal axis in FIG. 9A represents time, and the vertical axis represents index values. The index value in this embodiment is the size index value described above. In FIG. 9A, timings t1 to t11 are timings at which size index values are calculated, and correspond to frames in a video signal, for example. In the following, the processing result of the hysteresis processing will be referred to as a first processing result.

For example, in hysteresis processing, when the current first processing result is 0, the determination unit 150 determines whether or not to increase the first processing result to 1 based on comparison processing with a relatively large threshold TH1. do. For example, the determination unit 150 outputs 1 as the first processing result when it is determined that the size index value is equal to or greater than the threshold value TH1. Further, when it is determined that the size index value is less than the threshold value TH1, the determination unit 150 continues to output 0 as the first processing result.

Further, when the current first processing result is 1, the determining unit 150 determines whether or not to reduce the first processing result to 0 based on a comparison process with a threshold TH2, which is smaller than the threshold TH1. For example, the determination unit 150 outputs 0 as the first processing result when it is determined that the size index value is less than the threshold value TH2. Furthermore, when it is determined that the size index value is equal to or greater than the threshold value TH2, the determination unit 150 continues to output 1 as the first processing result.

To summarize the above, the determination unit 150 outputs 1 as the first processing result when the size index value is greater than or equal to the threshold TH1, and outputs 0 as the first processing result when the size index value is less than the threshold TH2. However, when the size index value is greater than or equal to the threshold value TH2 and less than the threshold value TH1, the same value as the first processing result one timing ago is continued to be output.

FIG. 9B is a diagram showing a processing result of performing hysteresis processing using threshold values TH1 and TH2 on the index values shown in FIG. 9A. At timing t1, the size index value is less than the threshold value TH2, so the determination unit 150 outputs 0 as the first processing result. The same applies to timings t2 and t3, and the determination unit 150 outputs 0 as the first processing result.

At timing t4, the size index value is greater than or equal to the threshold value TH2 and less than TH1. In this case, the determination unit 150 outputs 0 as the first processing result because the output of the first processing result one timing ago is continued.

At timing t5, the size index value is determined to be greater than or equal to the threshold value TH1. Therefore, the determination unit 150 outputs 1 as the first processing result.

At timing t6, the size index value is greater than or equal to the threshold value TH2 and less than TH1. In this case, the determination unit 150 outputs 1 as the first processing result because the output of the first processing result from one timing ago is continued. The same applies to timing t7.

At timing t8, it is determined that the size index value is less than the threshold value TH2. Therefore, the determination unit 150 outputs 0 as the first processing result.

The same applies below, and the determination unit 150 outputs 1 as the first processing result at timings t9 and t10, and outputs 0 as the first processing result at timing t11.

As shown in FIG. 9A, although the degree of variation in the size index value is large in this example, the variation in value is suppressed by performing hysteresis processing. For example, in the example of FIG. 9A, there are many variations in value that cross the threshold TH1 or TH2, but as shown in FIG. 9B, the variation in value in the result of the hysteresis process is suppressed to three times, t5, t8, and t11.

FIG. 9C is a diagram showing the processing result of chattering filter processing performed on the processing result of the hysteresis processing shown in FIG. 9B. Hereinafter, for convenience of explanation, the processing result of the chattering filter processing will also be referred to as the second processing result. For example, the determination unit 150 changes the value of the second processing result to the input value on the condition that an input value different from the second processing result one timing before has been acquired a predetermined number of times in succession. In other words, even if an input value different from the second processing result is obtained, if the consecutive number of times is less than the predetermined number, the input value will not be reflected in the second processing result. FIG. 9C shows an example in which three times is set as the predetermined number of times, but the value of the predetermined number of times can be modified in various ways. Further, it is assumed that the initial value of the second processing result is 0.

As shown in FIG. 9B, from timing t1 to t4, the value of the first processing result, which is the input value, is 0, and the value of the second processing result one timing before is 0. Therefore, as shown in FIG. 9C, the determination unit 150 continues to output 0 as the second processing result.

At timing t5, the value of the first processing result, which is the input value, is 1, which is different from 0, which is the value of the second processing result one timing before. Therefore, the determining unit 150 determines whether the consecutive number of times is equal to or greater than a predetermined number of times. Here, the consecutive number is 1, which is smaller than the predetermined number of 3, so the input value is not reflected. As a result, as shown in FIG. 9C, the determination unit 150 continues to output 0 as the second processing result.

At timing t6, the value of the first processing result, which is the input value, is 1, which is different from 0, which is the value of the second processing result one timing before. In this case, the number of consecutive times increases to 2, but since it is smaller than the predetermined number of 3, the input value is not reflected. As a result, as shown in FIG. 9C, the determination unit 150 continues to output 0 as the second processing result.

At timing t7, the value of the first processing result, which is the input value, is 1, which is different from 0, which is the value of the second processing result. In this case, since the number of consecutive operations increases to 3, the number of consecutive operations becomes 3 or more, which is the predetermined number of operations, and the input value is reflected in the second processing result. As a result, as shown in FIG. 9C, the determination unit 150 outputs 1 as the second processing result.

After timing t8, since the second processing result one timing before is 1, a comparison is made between the consecutive number of times the input value is 0 and the predetermined number of times. For example, 0 is acquired as the input value at t8 and t11, but the consecutive number of times is 1 in both cases, which is smaller than the predetermined number of times of 3, so the input value is not reflected. As a result, as shown in FIG. 9C, the determination unit 150 continues to output 1 as the second processing result from timing t8 to t11.

As can be seen from the comparison between FIG. 9B and FIG. 9C, by performing chattering filter processing, short-term value fluctuations are no longer reflected in the processing results, making it possible to further suppress time-series fluctuations in index values. Become.

Returning to FIG. 8B, the explanation will be continued. In step S402, the determining unit 150 determines whether the second processing result, which is the value after filtering, is 1. If the second processing result is 1 (step S402: Yes), in step S403, the determining unit 150 determines to perform image quality adjustment regarding the specific color. If the second processing result is 0 (step S402: No), in step S404, the determination unit 150 determines that the image quality regarding the specific color is not to be adjusted. In this way, it is possible to suppress time-series fluctuations in the size index value and then determine whether or not image quality adjustment for a particular color is necessary.

By the processing described above using FIGS. 6 to 8B, the processing of the determination unit 150 shown in step S104 of FIG. 4 is completed. The determination unit 150 outputs the determination result of whether or not to adjust the image quality regarding the specific color to the image quality adjustment unit 160.

In step S105, the image quality adjustment unit 160 determines whether a determination result indicating that the image quality regarding the specific color is to be adjusted is obtained. If the determining unit 150 determines to adjust the image quality regarding the specific color (step S105: Yes), in step S106, the image quality adjusting unit 160 adjusts the image quality regarding the specific color with respect to the input image. For example, when the predetermined object is a person's face, the memory color is lighter than the actual color, so the image quality adjustment unit 160 performs processing to reduce the saturation of pixels of a color corresponding to the skin color. Furthermore, if the predetermined object is a blue sky or green scenery, the memory color is darker than the actual color, so the image quality adjustment unit 160 performs processing to increase the saturation of blue and green pixels.

Note that in step S106, the image quality adjustment unit 160 adjusts the image quality of the area corresponding to the predetermined object detected by the object detection unit 120 in the input image, and also adjusts the image quality of the other area without adjusting the image quality of the other area. good. In this way, it is possible to limit the image quality correction target for a specific color to a predetermined object. For example, when an image includes an object whose skin color is different from that of a person's skin, the image quality of the object is not adjusted, so that it is possible to prevent the user from feeling uncomfortable.

Note that the area corresponding to the predetermined object here is, for example, the rectangular area D1 or the rectangular area D2 that are the detection results shown in FIG. 5, but they do not have to match exactly. For example, the image quality adjustment regarding a specific color for the face F1 may be performed for a region from which a part of the rectangular region D1 is excluded. For example, the area to be adjusted may be an area that includes a predetermined percentage or more of D1. Alternatively, the area to be adjusted may include an area near the rectangular area D1. For example, the area to be adjusted may be an area in which the ratio of areas other than the rectangular area D1 to the area is less than a predetermined ratio.

Alternatively, the image quality adjustment unit 160 may adjust the image quality of an area of the input image that includes an area corresponding to the predetermined object detected by the object detection unit 120 and an area not corresponding to the predetermined object. The area targeted for image quality adjustment here may be an area having a size equal to or larger than a predetermined ratio with respect to the size of the entire input image. For example, the image quality adjustment unit 160 may adjust the image quality of a specific color for the entire input image. In this way, it is not necessary to strictly set the area to be subjected to image quality adjustment, so that the processing load can be reduced.

Furthermore, when the determination unit 150 determines that the image quality adjustment for the specific color is not to be performed (step S105: No), the image quality adjustment unit 160 skips the image quality adjustment for the specific color shown in step S106.

Although not shown in FIG. 4, the image quality adjustment unit 160 may perform processing to increase at least one of the brightness and saturation of the input image without being limited to a specific color. In this way, it becomes possible to express vivid colors regardless of the standards of television broadcasting. For example, when adjusting the image quality for a specific color, the image quality adjustment unit 160 may perform both the image quality adjustment for the specific color and the image quality adjustment not limited to the specific color. Furthermore, when not adjusting the image quality for a specific color, the image quality adjustment unit 160 may only perform image quality adjustment not limited to the specific color.

In step S107, the display control unit 170 performs control to display the input image after image quality adjustment on the display panel 16. The input image after image quality adjustment here is, in a narrow sense, an input image on which the image quality regarding a specific color has been adjusted. However, the input image after image quality adjustment may be an input image that has undergone both image quality adjustment related to a specific color and image quality adjustment not limited to a specific color, or may be an input image in which only image quality adjustment not limited to a specific color has been performed. It may be an input image.

2. Second Embodiment FIG. 10 is a diagram showing a configuration example of an image display device 100 according to this embodiment. When compared with the configuration shown in FIG. 3, the configuration is such that a scene acquisition unit 140 is added. The scene acquisition unit 140 determines the scene of the input image output from the image acquisition unit 110, and acquires a scene determination result. For example, as described below, the scene acquisition unit 140 may determine, for each of a plurality of candidate scenes, the probability that the input image corresponds to the candidate scene. The candidate scenes here include a person's face, a blue sky, a green scenery, an animation, and the like.

The image acquisition unit 110 is similar to the first embodiment except that it outputs the input image to the object detection unit 120 and the scene acquisition unit 140. The determining unit 150 determines whether image quality adjustment is necessary for a specific color based on the scene determination result in the scene acquiring unit 140 in addition to the size index value. Details of the processing in the determination unit 150 will be described later. The object detection section 120, size acquisition section 130, image quality adjustment section 160, and display control section 170 are the same as those in the first embodiment.

FIG. 11 is a flowchart illustrating the processing of the image display device 100 according to this embodiment. The processing shown in FIG. 11 may be performed on images of each frame, for example, when video information that is a set of time-series images is acquired.

First, in step S501, the image acquisition unit 110 acquires an input image. In step S502, the object detection unit 120 performs object detection processing to detect a predetermined object in the input image acquired from the image acquisition unit 110. In step S503, the size acquisition unit 130 determines the size of the predetermined object detected by the object detection unit 120. The processing in steps S501 to S503 is similar to steps S101 to S103 in FIG. For example, in step S503, the size acquisition unit 130 outputs the number of detected predetermined objects and the size of each predetermined object to the determination unit 150.

In step S504, the scene acquisition unit 140 determines the scene of the input image and acquires the scene determination result. For example, the scene acquisition unit 140 may perform scene determination using a classification model acquired by machine learning. However, the process described below is an example of scene determination, and scene determination may be performed using other machine learning such as SVM (Support Vector Machine), or by a method different from machine learning. good. Note that the processes shown in steps S502 and S503 and the process shown in step S504 may be executed in parallel or sequentially.

FIG. 12 is a diagram illustrating input and output in scene determination. The scene determination may be performed using, for example, the classification model using the CNN described above. The input of the CNN is the input image output from the image acquisition unit 110. Note that the input is not limited to the input image itself, and may be the result of some preprocessing performed on the input image.

The output of the CNN may be, for example, the probability for each of a plurality of candidate scenes. The candidate scenes here include, for example, the above-mentioned human face, blue sky, green scenery, and animation. Further, the candidate scenes are not limited to these, and can include various scenes such as buildings, food, animals, etc. For example, CNN calculates the probability that the input image scene is a person's face, the probability that the input image scene is a blue sky, the probability that the input image scene is a green landscape, and the probability that the input image scene is an animation. Outputs four values of likelihood. Each value may be a numerical value of 0 or more and 100 or less, for example. Hereinafter, the probability that it is a human face is called the face scene judgment value, the probability that it is a blue sky is called the blue sky scene judgment value, the certainty that it is a green scenery is called the green scene judgment value, and the certainty that it is an animation is called the animation scene judgment value. write.

For example, CNN is obtained by machine learning based on training data. The training data here is data in which the scene classification results are added as correct data to the learning images. The scene classification result is obtained, for example, by input by a user who viewed the learning image. For example, if the user determines that the learning image is an image of a person's face, the correct data includes a face scene determination value of 100, a blue sky scene determination value, a green scene determination value, and an animation scene determination value. Data with a value of 0 is assigned. Furthermore, one image may correspond to a plurality of scenes, such that a blue sky and a person's face are included in one learning image. In this example, data in which the face scene determination value and the blue sky scene determination value are 100, and the green scene determination value and the animation scene determination value are 0 is assigned as correct data. Note that methods for generating trained models for image classification are widely known, and these methods can be widely applied to this embodiment.

For example, the image display device 100 includes a storage unit (not shown), and the storage unit stores a CNN that is a trained model. Note that the storage unit here may be the memory 14 of the television receiving device 10 shown in FIG. 2. The scene acquisition unit 140 reads the CNN from the storage unit and inputs the input image to the CNN to obtain four values: a face scene determination value, a blue sky scene determination value, a green scene determination value, and an animation scene determination value. The scene acquisition unit 140 outputs at least one of these values to the determination unit 150 as a scene determination result.

The determination unit 150 determines whether or not to adjust the image quality regarding a specific color included in a predetermined object based on the size output from the size acquisition unit 130 and the scene determination result output from the scene acquisition unit 140. judge. For example, the determination unit 150 may obtain a size index value based on the size and a scene index value based on the scene determination result, and calculate the index value based on the size index value and the scene index value. The determination unit 150 determines whether image quality adjustment is necessary for a specific color based on the index value. In this way, it is possible to determine whether or not image quality adjustment is necessary based on different information: the object detection result and the scene determination result. Therefore, it becomes possible to improve the determination accuracy.

At this time, the scene acquisition unit 140 may obtain the probability that the input image is a scene corresponding to a predetermined object as a scene determination result. Then, the determination unit 150 may obtain the probability that the scene corresponds to a predetermined object as a scene index value. For example, when the predetermined object is a person's face, the scene acquisition unit 140 acquires information including at least a face scene determination value as a scene determination result, and the determination unit 150 acquires the face scene determination value as a scene index value. do. Further, when the predetermined object is a blue sky, the scene acquisition unit 140 acquires information including at least a blue sky scene determination value as a scene determination result, and the determination unit 150 acquires the blue sky scene determination value as a scene index value. good. Similarly, when the predetermined object is a green scene, the scene acquisition unit 140 acquires information including at least a green scene determination value as a scene determination result, and the determination unit 150 uses the green scene determination value as a scene index value. get. In this way, it becomes possible to use the result of object detection regarding the predetermined object and the result of scene determination regarding the predetermined object for calculating the index value. Since different determination results are used for the same object, it is possible to improve the accuracy of determining whether or not image quality adjustment is necessary.

Here, the differences between object detection and scene determination will be explained. For example, object detection is a process that determines the specific position and size of a given object in addition to whether it exists, and in some cases includes the process of detecting detailed shapes such as parts such as eyes and nose. . Therefore, it is possible to obtain more detailed information regarding a predetermined object than by scene determination, and the detection accuracy of the predetermined object is high. On the other hand, scene determination is a process of determining how well the input image as a whole matches the characteristics of the candidate scene. Therefore, although the specific number, size, position, etc. of objects cannot be determined, determinations regarding various candidate scenes can be made as shown in FIG. 12.

FIG. 13 is a diagram illustrating an example of the relationship between the input image format, the face scene determination value, and the object detection result. As described above, the face scene determination value is a numerical value between 0 and 100 that represents the probability that the input image is a scene that includes a person's face. The object detection result indicates whether or not an object was detected from the input image.

As shown in FIG. 13, when an image is captured in a normal manner without an object blocking the face, the face scene determination value becomes a relatively high value, and a face is detected as a result of object detection. Therefore, in this case, both the size index value based on the object detection result and the scene index value (face scene determination value) become information that reflects the predetermined object (person's face) included in the input image.

In addition, if the nose or mouth is blocked by a mask, etc., if the person is facing diagonally, if the eyes are blocked by goggles, etc., or if subtitles are displayed around the face, etc. As shown, the face scene judgment value may decrease. This is thought to be because some parts of the face are occluded and other information such as subtitles is mixed, causing the input image to deviate from a state that seems to be a scene including a face. In this case, the scene index value does not appropriately reflect the predetermined object (person's face) included in the input image, and if only the scene index value is used, it may be determined that image quality adjustment regarding skin color is not necessary. be. On the other hand, in object detection, even if some parts are missing, the remaining parts can be detected, so the facial area can be detected appropriately. The same holds true when the shape of parts on the image is distorted because the face is not facing forward, or when other information such as subtitles is mixed, and object detection is possible. Therefore, when an input image in which a person's face is displayed in these manners is targeted, by using the size index value based on the object detection result, it is possible to prevent the image quality from being omitted from adjustment. The face scene judgment values shown in Fig. 13 are just examples, and the face scene judgment values are does not necessarily decrease. That is, even if any of these cases applies, a scene including a person's face may be appropriately detected by scene determination.

Further, if the person's face in the image is blurred because the person is out of focus, for example, parts cannot be detected appropriately, and a predetermined object may be determined not to be detected. In this case, if only the size index value based on the object detection result is used, it may be determined that image quality adjustment regarding skin color is unnecessary. On the other hand, although fine structures may be lost when the image is out of focus, the overall tendency of the input image is that there is little deviation from the focused state. In scene determination, the probability that the input image as a whole is a scene including a face is determined, so even if the image is out of focus, if the image includes a person, the face scene determination value is likely to be determined to be high. Therefore, by using the scene index value, it is possible to suppress the omission of image quality adjustment. Note that Figure 13 shows that there is a possibility that a face cannot be detected by object detection processing in the case of "out of focus"; even in the case of "out of focus", the face of a person may be properly detected by object detection processing. may be detected.

As can be seen from the example in FIG. 13, the preferred facial aspects are different for object detection and scene determination. Therefore, by using both the size index value based on object detection and the scene index value based on scene determination, it is possible to appropriately determine whether image quality adjustment is necessary for a specific color, regardless of the aspect of the face in the input image. becomes possible.

For example, in step S505 in FIG. 11, the determination unit 150 determines the size index value using the same method as in the first embodiment, and acquires the scene index value based on the output from the scene acquisition unit 140. Then, the determination unit 150 may obtain the maximum value of the size index value and the scene index value as an index value used to determine whether or not image quality adjustment regarding a specific color is necessary. For example, in the example shown in Figure 12, the size index value is likely to be used as the index value in situations such as covering the nose/mouth, looking diagonally downward, blindfolding, subtitles under the face, etc., and the scene index value is likely to be used in cases where the face is out of focus. The value becomes more likely to be used as an index value. In this way, even if the aspect of the input image changes, information suitable for the aspect is used as the index value, so that it is possible to suppress the omission of adjustment of the image quality regarding a specific color.

The processing after the index value is calculated may be the same as in the first embodiment. For example, as shown in FIG. 8A, the determination unit 150 may determine whether to perform image quality adjustment for a specific color based on a comparison process between the index value and the threshold value TH0. Alternatively, as shown in FIG. 8B, the determination unit 150 performs processing to suppress time-series changes in index values (step S401), and then determines whether or not to perform image quality adjustment for a specific color based on the processed values. It may be determined whether For example, the determination unit 150 obtains a size index value and a scene index value for each of a plurality of frames, and determines whichever is larger as the index value for the frame. Then, by performing the processing described above using FIGS. 9A to 9C on the determined time-series index values, index values whose time-series changes are suppressed may be determined.

The processing in steps S506 to S508 is the same as steps S105 to S107 in FIG. 4, so a detailed explanation will be omitted.

Further, when performing scene determination, the determination unit 150 may perform scene determination regarding a plurality of candidate scenes in step S505, and in step S507, the image quality adjustment unit 160 may perform image quality adjustment regarding a specific color according to the result of the scene determination. A specific example will be explained below.

In the above description, an example has been described in which image quality adjustment regarding a person's skin color is performed using a size index value based on a detection result of a person's face and a face scene determination value that is a scene index value. In this case, the image quality adjustment that makes the blue region of the blue sky and the green region of the green scenery a specific color may be omitted. However, as described above using FIG. 12, the scene determination result does not include the probability that the scene includes a blue sky (blue sky scene determination value) or the probability that the scene includes a green landscape (green scene determination value). But that's fine. Since the memory color of a blue sky or a green scenery is more vivid than the actual color, it is useful to adjust the image quality of the blue of the blue sky or the green of a green scenery. Therefore, it may be determined whether image quality adjustment for a specific color is necessary based on the blue sky scene determination value or the green scene determination value. For example, image quality adjustment may be performed to increase the saturation of the blue region of the blue sky when the blue sky scene determination value is large, and to increase the saturation of the green region of the green scenery when the green scene determination value is large. In this way, it is possible to determine whether image quality adjustment is necessary even for a specific color included in an object that is not a target of object detection.

The same applies when a predetermined object changes. For example, when the predetermined object is a blue sky, the determination unit 150 determines whether to increase the saturation of the blue region based on the larger of the size index value based on the blue sky detection result and the blue sky scene determination value. The determining unit 150 may use the face scene determination value and the green scene determination value as they are for the person's face and the green scene to determine whether image quality adjustment is necessary for the skin color area and the green area.

Further, there may be two or more predetermined objects. For example, the object detection unit 120 may perform both human face detection processing and blue sky detection processing. In this case, the determination unit 150 determines whether or not to lower the saturation of the skin color area based on the size index value and face scene determination value based on the detection result of the person's face. The determination unit 150 also determines whether to increase the saturation of the blue region based on the size index value and the blue sky scene determination value based on the blue sky detection result. Further, the determination unit 150 may use the green scene determination value as is for the green scene to determine whether or not image quality adjustment is necessary for the green region of the green scene.

Naturally, the object detection unit 120 may perform object detection processing using each of a person's face, a blue sky, and a green scenery as predetermined objects. In this case, the determination unit 150 determines whether or not image quality adjustment is necessary for all of the skin color region, blue region, and green region, using both the object detection result and the scene determination result.

As described above, in this embodiment, the predetermined object may be a person's face, a blue sky, a green landscape, or any other object. . Further, the object to be subjected to image quality adjustment regarding a specific color is not limited to one, and for example, as described above, two or more of a person's face, a blue sky, and a green scenery may be the objects. At this time, for objects to be subjected to object detection processing, the object detection results and scene determination results are used together, making it possible to improve the accuracy of determining whether or not image quality adjustment is necessary.

3. Third Embodiment In the second embodiment, when a size index value based on an object detection result and a scene index value based on a scene determination result are obtained, the larger value is used as the index value. . However, the method of obtaining the index value is not limited to this. A specific example will be explained below. Note that among the processes shown in FIG. 11, the processes excluding step S505 are the same as those in the second embodiment.

FIG. 14 is a flowchart illustrating the index value acquisition process in this embodiment. It is assumed that the determination unit 150 has acquired the size index value and the scene index value before the processing shown in FIG. 14 .

In step S601, the determination unit 150 weights the size index value using a first weight. Further, in step S602, the determination unit 150 weights the scene index value using the second weight. The weighting here refers to a process in which the size index value is multiplied by a first weight, and a process in which the scene index value is multiplied by a second weight, but other weighting processes may be performed. Moreover, the first weight and the second weight may be the same value or may be different values. The determination unit 150 of this embodiment calculates an index value based on the weighted size index value and the weighted scene index value, as described later in step S605.

In this way, it becomes possible to correct the degree of contribution of each of the size index value and scene index value to the determination of whether image quality adjustment is necessary. For example, the first weight may be a value greater than or equal to 0 and less than or equal to 1, and the closer the value is to 0, the smaller the value of the size index value becomes, and therefore the degree of contribution of the size index value becomes smaller. Further, the second weight may be a value of 0 or more and 1 or less, and the closer the value is to 0, the smaller the scene index value becomes, and therefore the smaller the contribution of the scene index value. Further, at least one of the first weight and the second weight may have a value of 1 or more. According to the method of this embodiment, it becomes possible to flexibly determine which of the size index value and the scene index value is more important.

For example, the first weight may be a larger value than the second weight. In this case, it becomes possible to carry out processing in which the size index value is given more importance than the scene index value. For example, as shown in FIG. 13, various aspects of a predetermined object (for example, a person's face) in an input image can be considered, but object detection may cover a wider range than scene determination. That is, in determining whether image quality adjustment is necessary for a specific color, the size index value based on object detection may be more reliable information than the scene index value. Therefore, by making the first weight larger than the second weight, processing that emphasizes more reliable information can be realized. However, the processing of this embodiment is not limited to this, and the values of the first weight and the second weight may be the same, or the second weight may be larger than the first weight.

Further, as shown in FIG. 12, the scene acquisition unit 140 may acquire a scene determination result indicating the probability that the input image is an animation. The scene determination result here is, for example, the above-mentioned animation scene determination value. The determination unit 150 may perform weighting processing on at least one of the size index value and the scene index value based on the probability that the input image is an animation. For example, the determination unit 150 may perform weighting processing using a third weight such that when the anime scene determination value is large, at least one of the size index value and the scene index value is smaller than when the value is small. . Here, the weight that reduces the index value may be rephrased as the weight that determines that the image quality of a specific color is not adjusted. Note that in FIG. 14, an example is shown in which the determination unit 150 performs weighting processing on the size index value in step S603 and weighting processing on the scene index value in step S604, but if either one of steps S603 and S604 is May be omitted.

For example, the determination unit 150 may set a weight of 0 or more and less than 1 as the third weight. For example, the third weight may be 0 when the animation scene determination value is greater than or equal to a given threshold, and may be 1 when it is less than the given threshold. In this case, when it is determined that there is a high probability that the input image is an animation, at least one of the size index value and the scene index value is set to zero. In this way, for example, it is possible to suppress image quality adjustment from being performed on a skin color area in an animation. In animation, adjusting the image quality of pixels corresponding to skin color may give the user a sense of discomfort; therefore, by using the third weight, the occurrence of the sense of discomfort can be suppressed.

Note that the determination unit 150 may perform weighting by multiplying both the size index value and the scene index value by a third weight. In this way, since both index values become small, it becomes difficult to perform image quality adjustment regarding a specific color. However, the processing in this embodiment is not limited to this, and the determination unit 150 may perform weighting by multiplying either the size index value or the scene index value by a third weight.

Further, the third weight is not limited to two values of 0 and 1. For example, the third weight may be 1 when the anime scene determination value is 0, 0 when the anime scene determination value is 100, and linear interpolation may be performed in a range where the anime scene determination value is greater than 0 and less than 100. In this way, flexible weighting can be performed depending on the size of the animation scene determination value. Note that the range of the third weight is not limited to this, and when the animation scene determination value is 100, a value larger than 0 may be set. For example, by setting the value of the third weight to 0.5 when the animation scene determination value is 100, the third weight may be set in a range of 0.5 or more and 1 or less. Further, the upper limit value of the third weight may be changed by setting the third weight when the animation scene determination value is 0 to a value smaller than 1. Moreover, the interpolation process in the range where the animation scene determination value is greater than 0 and less than 100 is not limited to linear interpolation, and interpolation process using a nonlinear function may be performed.

Note that in FIG. 14, the process related to the size index value (steps S601 and S603) and the process related to the scene index value (steps S602 and S604) may be executed in parallel or sequentially. After the weighting process is completed, in step S605, the determination unit 150 obtains an index value for determining whether image quality adjustment is necessary for a specific color based on the weighted size index value and scene index value. Specifically, the determination unit 150 compares the weighted size index value and the weighted scene index value, and obtains the larger value as the index value. The processing after obtaining the index value is the same as in the first embodiment and the second embodiment, and the processing shown in FIG. 8A or the processing shown in FIG. 8B may be performed.

Furthermore, an example has been described above in which the third weight applied to at least one of the size index value and the scene index value is adjusted depending on the probability that it is an animation. However, in this embodiment, other image quality adjustments may be performed based on the probability that the image is an animation. For example, the image quality adjustment unit 160 may change the specific content of image quality adjustment regarding a specific color based on the probability that it is an animation. For example, in the above example, the image quality adjustment for a specific color is an adjustment that brings the expression of the specific color closer to the memory color. On the other hand, the image quality adjustment unit 160 adjusts the expression of the specific color closer to the memory color if, for example, the probability that it is an animation is less than a predetermined value, and if the probability that it is an animation is more than a predetermined value. may perform adjustment to bring the specific color closer to a color different from the memory color. In a broader sense, the image quality adjustment unit 160 may determine a target color for image quality adjustment regarding a specific color based on the probability that it is an animation (anime scene determination value). Here, the target color is information specified by a set of values of brightness, saturation, and hue. In this way, it becomes possible to realize image quality adjustment suitable for both cases where the input image (video signal) is an animation and cases where the input image is not an animation. Further, the animation scene determination value may be used to determine a target color in image quality adjustment of a color different from the specific color. In addition, various modifications can be made to the specific processing.

Although this embodiment has been described in detail as above, those skilled in the art will easily understand that many modifications can be made without substantively departing from the novelty and effects of this embodiment. . Therefore, all such modifications are intended to be included within the scope of the present disclosure. For example, a term that appears at least once in the specification or drawings together with a different term with a broader or synonymous meaning may be replaced by that different term anywhere in the specification or drawings. Furthermore, all combinations of this embodiment and modifications are also included within the scope of the present disclosure. Further, the configuration and operation of the image display device, the television receiving device, etc. are not limited to those described in this embodiment, and various modifications are possible.

DESCRIPTION OF SYMBOLS 10... Television receiving device, 11... Processor, 12... Tuner, 13... Communication interface, 14... Memory, 15... Operation interface, 16... Display panel, 100... Image display device, 110... Image acquisition part, 120... Object detection part , 130... Size acquisition section, 140... Scene acquisition section, 150... Judgment section, 160... Image quality adjustment section, 170... Display control section, D1, D2... Rectangular area, F1, F2... Face, IM... Image, t1 to t11 …timing

Claims (13)

an image acquisition unit that acquires an input image;
an object detection unit that detects a predetermined object from the input image;
a size obtaining unit that obtains the size of the predetermined object;
a determination unit that determines whether to adjust image quality with respect to a specific color of the predetermined object, based on the size of the predetermined object;
an image quality adjustment unit that adjusts the image quality of at least a part of the input image with respect to the specific color when the determination unit determines that the image quality is to be adjusted;
a display control unit that controls displaying the input image with the image quality adjusted on a display panel;
An image display device including: In claim 1,
The determination unit includes:
An image display device that determines to adjust the image quality regarding the specific color when the size of the predetermined object is larger than a predetermined size. In claim 1,
The determination unit includes:
An image display device that calculates an index value based on the size of the predetermined object, and determines whether or not to adjust the image quality regarding the specific color based on the index value and a given threshold. In claim 3,
The determination unit includes:
The image display device calculates the index value based on the size of at least one of the predetermined objects when a plurality of the predetermined objects are detected from the input image. In claim 4,
The determination unit includes:
An image display device that calculates the index value based on the position of the predetermined object detected from the input image. In any one of claims 3 to 5,
including a scene acquisition unit that determines a scene of the input image and acquires a scene determination result;
The determination unit includes:
An image display device that obtains a size index value based on the size and a scene index value based on the scene determination result, and calculates the index value based on the size index value and the scene index value. In claim 6,
The scene acquisition unit includes:
An image display device that obtains, as the scene determination result, a probability that the input image is a scene corresponding to the predetermined object. In claim 6,
The determination unit includes:
The size index value is weighted with a first weight, the scene index value is weighted with a second weight, and based on the weighted size index value and the weighted scene index value, the An image display device for determining index values. In claim 6,
The scene acquisition unit includes:
obtaining the scene determination result indicating the probability that the input image is an animation;
The determination unit includes:
An image display device that performs weighting processing on at least one of the size index value and the scene index value based on the probability that the input image is the animation. In any one of claims 3 to 5,
The determination unit includes:
Performing filter processing to suppress time-series fluctuations on the index value,
The image quality adjustment section includes:
An image display device that adjusts the image quality based on the index value after the filter processing. In any one of claims 1 to 5,
The image quality adjustment section includes:
An image display device that adjusts the image quality of an area corresponding to the predetermined object detected by the object detection unit in the input image, and does not adjust the image quality of other areas. In any one of claims 1 to 5,
The image quality adjustment section includes:
An image display device that adjusts the image quality of an area of the input image that includes an area corresponding to the predetermined object detected by the object detection unit and an area that does not correspond to the predetermined object. Get the input image,
detecting a predetermined object from the input image;
determining the size of the predetermined object;
Based on the size of the predetermined object, determine whether to adjust image quality regarding a specific color of the predetermined object;
If it is determined that the image quality is to be adjusted, adjusting the image quality of at least a partial area of the input image with respect to the specific color;
performing control to display the input image with the image quality adjusted on a display panel;
Image display method.

Images