WO2021014714A1 - Image processing device, image display device, image processing method, and program - Google Patents

Abstract

Provided are an image processing device and the like for performing color component adjustment processing with greater versatility. The image processing device is provided with: a reference image generation unit that uses an input image as a basis to generate a reference image in which the color components of at least some of the pixels of the input image are changed; a color difference information calculation unit that calculates color difference information for each pixel between the input image and the reference image; and a color component adjustment unit that uses the color difference information as a basis to generate an output image in which the color components of each of the pixels of the input image are adjusted.

Description

Image processing equipment, image display equipment, image processing methods, and programs

The present disclosure relates to an image processing device, an image display device, an image processing method, and a program. The present application claims priority based on Japanese Patent Application No. 2019-133404 filed in Japan on July 19, 2019, the contents of which are incorporated herein by reference.

Patent Document 1 below discloses an image processing method capable of improving the image quality by restoring the brightness and color even for an input face image whose brightness and color have deteriorated due to shooting in low illuminance or the like.

Japanese Unexamined Patent Publication No. 2005-316743 (published on November 10, 2005)

In the process described in Patent Document 1, color adjustment can be performed only for an object assumed in advance, such as a face area in an image, and the versatility is low. Therefore, one aspect of the present disclosure is, for example, to provide an image processing apparatus or the like that performs more versatile color component adjustment processing.

The image processing apparatus according to one aspect of the present disclosure is, for example, a reference image that generates a reference image in which the color components of at least some of the pixels included in the input image are changed based on the input image. A color difference information calculation unit that calculates color difference information based on the color difference between the generation unit, the plurality of pixels included in the input image, and the plurality of pixels included in the reference image, and the color difference information calculation unit included in the input image. It is provided with a color component adjusting unit that generates an output image in which the color components of a plurality of pixels are adjusted by using the color difference information.

The control method of the image processing device according to one aspect of the present disclosure is, for example, to generate a reference image in which the color components of at least a part of the plurality of pixels included in the input image are changed based on the input image. The reference image generation step, the color difference information calculation step for calculating the color difference information based on the color difference between the plurality of pixels included in the input image and the plurality of pixels included in the reference image, and the input image It includes a color component adjusting step of generating an output image in which the color components of the plurality of pixels included are adjusted by using the color difference information.

In the control program according to one aspect of the present disclosure, for example, a reference image in which the color components of at least a part of the plurality of pixels included in the input image are changed is provided to an image processing device based on the input image. A reference image generation function to be generated, a color difference information calculation function for calculating color difference information based on the color difference between the plurality of pixels included in the input image and the plurality of pixels included in the reference image, and the input image. The output image in which the color components of the plurality of pixels included in the above are adjusted is realized with the color component adjusting function of generating the output image by using the color difference information.

It is a block diagram which shows an example of the main part structure of the image processing apparatus which concerns on Embodiment 1. FIG. It is a block diagram which shows an example of the main part structure of the reference image generation part provided in the said image processing apparatus. It is a block diagram which shows an example of the main part structure of the color difference information calculation part provided in the said image processing apparatus. It is a block diagram which shows an example of the main part structure of the color component adjustment part provided in the said image processing apparatus. It is a flowchart which shows an example of the processing executed by the said image processing apparatus. It is a flowchart which shows an example of the processing executed by the reference image generation part provided in the said image processing apparatus. It is a flowchart which shows an example of the processing executed by the color difference information calculation unit provided in the image processing apparatus. It is a flowchart which shows an example of the processing executed by the color component adjustment part provided in the said image processing apparatus. It is a block diagram which shows an example of the main part structure of the reference image generation part provided in the image processing apparatus which concerns on Embodiment 2. FIG. It is a block diagram which shows an example of the main part structure of the reference image generation part provided in the image processing apparatus which concerns on Embodiment 3. It is a block diagram which shows an example of the main part structure of the image processing apparatus which concerns on Embodiment 4. It is a block diagram which shows an example of the main part structure of the color component adjustment part provided in the said image processing apparatus. It is a block diagram which shows an example of the main part structure of the reference image generation part provided in the image processing apparatus which concerns on Embodiment 5. It is a block diagram which shows an example of the main part structure of the color difference information calculation part provided in the image processing apparatus which concerns on Embodiment 6. It is a block diagram which shows an example of the main part structure of the color component adjustment part provided in the said image processing apparatus. It is a block diagram which shows an example of the main part structure of the color difference information calculation part provided in the image processing apparatus which concerns on Embodiment 7. It is a block diagram which shows an example of the main part structure of the color component adjustment part provided in the said image processing apparatus. It is a schematic diagram which shows typically the image display apparatus provided with the image processing apparatus which concerns on each embodiment.

<Overview of image processing equipment>
In order to display an image beautifully on an image display device (for example, a liquid crystal television, a smartphone, etc.), it is preferable to adjust the color of the image. However, if the saturation of all the pixels included in the image is uniformly emphasized, an image that may give an unnatural impression to the viewer may be generated. This is because the change in impression according to the saturation enhancement depends on the frequency of contact of the viewer with the object in the image.

Therefore, the image processing apparatus according to one aspect of the present disclosure generates, for example, a reference image in which the color components of at least a part of the plurality of pixels included in the input image are changed based on the input image. You can. Next, the image processing device may calculate color difference information based on the color difference between the plurality of pixels included in the input image and the plurality of pixels included in the reference image, for example. Then, the image processing apparatus may generate, for example, an output image in which the color components of a plurality of pixels included in the input image are adjusted based on the color difference information, using the color difference information.

Therefore, the above image processing device can realize more versatile color component adjustment than the conventional image processing device.

<Embodiment 1>
The first embodiment (Embodiment 1) will be described with reference to FIGS. 1 to 8. In each drawing, the same or equivalent elements are designated by the same reference numerals, and duplicate description will be omitted.

(Overall configuration of image processing device 100)
FIG. 1 is a block diagram showing an example of a main configuration of the image processing apparatus 100. As illustrated in FIG. 1, the image processing device 100 is a device that processes an input input image 1a to generate an output image, and is, for example, a television, a personal computer, a smartphone, a tablet terminal, or a recording device (for example,). , Hard disk recorder), etc. The image processing device 100 includes, for example, a control unit 110.

The control unit 110 has a function of comprehensively controlling various functions of the image processing device 100, for example, an image conversion unit 101, an image storage unit 102, a reference image generation unit 103, a color difference information calculation unit 104, and a color component adjustment. Part 105 may be included.

In the image conversion unit 101, for example, when the first color gamut expressing the input image 1a and the second color gamut used in the processing of the image processing apparatus 100 are different, the input image 1a is the second color gamut. The input image 1a may be converted so that the input image 1a can be expressed by, and the converted input image 1b may be output to the image storage unit 102.

The input image 1a and the input image 1b may be a still image or a moving image. Details of the processing executed by the image conversion unit 101 will be described later.

The image storage unit 102 may store the input image 1b input from the image conversion unit 101, for example. The image storage unit 102 can output the input image 1b to the reference image generation unit 103, the color difference information calculation unit 104, and the color component adjustment unit 105 at the timing of each processing. As a result, the image processing device 100 can process the input image 1b at an appropriate timing.

For example, the reference image generation unit 103 converts the reference image 1c in which the color components of at least some of the pixels included in the input image 1b input from the image storage unit 102 are changed into the input image 1b. Based on this, the generated reference image 1c may be output to the color difference information calculation unit 104.

As will be described later, the reference image generation unit 103 generates a color-reduced image 2a (see FIG. 2) in which the color components of the plurality of pixels included in the input image 1b are reduced, and then includes the color-reduced image 2a. A colored image in which a color component is added to each of the plurality of pixels may be generated as a reference image 1c. The details of the process executed by the reference image generation unit 103 will be described later.

As described above, the image processing apparatus 100 generates, for example, a color-reduced image 2a by reducing the color component of the input image 1b, and then outputs a colored image in which the color component is added to the color-reduced image 2a as a reference image 1c. You can do it. As a result, the image processing apparatus 100 can generate a reference image 1c in which the color components of at least a part of the plurality of pixels included in the input image 1b are changed.

In the following, the color components of the plurality of pixels included in the input image 1b may be, for example, components other than the components corresponding to the luminance in each pixel value. For example, in the YCbCr color space, the Cb and Cr components are "color components", in the HSL color space, the H and S components are "color components", and in the CIELAB color space, the a ^* and b ^* components. Is the "color component". However, the definition of color components is not limited to these examples.

The color difference information calculation unit 104 includes, for example, a plurality of pixels included in the input image 1b input from the image storage unit 102 and a plurality of pixels included in the reference image 1c input from the reference image generation unit 103. The color difference information 1d based on the color difference may be calculated. The color difference information calculation unit 104 may output the calculated color difference information 1d to the color component adjustment unit 105. Details of the processing executed by the color difference information calculation unit 104 will be described later.

For example, the color component adjusting unit 105 generates an output image 1e in which the color components of a plurality of pixels included in the input image 1b are adjusted, using the color difference information 1d input from the color difference information calculation unit 104, and outputs the output image 1e. Image 1e may be output. For example, the color component adjusting unit 105 may increase the saturation of the plurality of pixels as the distance (color difference value) corresponding to each of the plurality of pixels included in the input image 1b increases. Details of the processing executed by the color component adjusting unit 105 will be described later.

(Structure of reference image generation unit 103)
FIG. 2 is a block diagram showing an example of the configuration of a main part of the reference image generation unit 103 included in the image processing device 100. As illustrated in FIG. 2, the reference image generation unit 103 may include a color reduction unit 221 and a coloring unit 222. Further, the coloring unit 222 may include, for example, a first deep learning device 223 and a storage unit 224.

The color reduction unit (reference image generation unit) 221 may generate, for example, a color reduction image 2a in which the color components of a plurality of pixels included in the input image 1b input from the image storage unit 102 are reduced. In the first embodiment, an example in which the color reduction unit 221 generates the color reduction image 2a by setting the color components of the plurality of pixels included in the input image 1b to zero will be described. That is, in the first embodiment, the color reduction image 2a is a monochrome image.

The color reduction unit 221 may be generated as a color reduction image 2a by, for example, linearly or non-linearly converting the luminance values of a plurality of pixels included in the input image 1b by any method. In this case, the color reduction unit 221 executes an operation so that only the luminance value is generated and deletes the color component, so that the process is equivalent to the process of making the color component zero.

The coloring unit (reference image generation unit) 222 may generate, for example, a coloring image in which color components are added to a plurality of pixels included in the color reduction image 2a as the reference image 1c. In the first embodiment, an example in which the coloring unit 222 adds a color component to the color-reduced image 2a by using the first deep learning device (first learned model) 223 defined by the first model parameter 225. To explain. That is, the coloring unit 222 adds a color component to the monochrome image by the first deep learning device 223.

The first deep learner 223 may be, for example, a neural network (for example, a multi-layer perceptron (including a deep model)). Further, the first model parameter 225 may be, for example, data including the connection weight of the neural network constituting the first deep learner 223.

The first model parameter 225 is optimized as follows, for example. First, the image processing device 100 collects color source images in which various subjects are captured by an appropriate method (for example, randomly) from an arbitrary information source (for example, the Internet). Next, the image processing device 100 generates a color-reduced image 2a by a process executed by the color-reducing unit 221 on the color source image. As a result, a combination in which the color-reduced image 2a is input data and the color source image is correct data is obtained as teacher data. Then, the image processing device 100 optimizes the first model parameter 225 by training the first deep learning device 223 using the teacher data.

Here, when the above information source is the Internet, the teacher data used for learning of the first deep learning device 223 can be created from images having a narrow color gamut that already exists in many places in the world. Therefore, the image processing device 100 can easily prepare the teacher data. Since the ability to prepare a large amount of teacher data means that the learning accuracy of the deep learning device can be improved, the image processing apparatus 100 can perform color component adjustment processing with higher accuracy.

In the process of generating the color-reduced image 2a in the teacher data, the image processing device 100 may, for example, convert the color gamut of the color source image and generate the color-reduced image 2a from the converted image. Also, it is preferable that the teacher data is diverse. If there is enough teacher data randomly sampled from the population (for example, all the images in the world), it can be assumed that the set of teacher data reflects the nature of the population.

The accuracy of the color that humans memorize for an object depends on the frequency of personal contact with the object, but if the above diversity is properly secured, the color of the universally distributed object will be. It is reproduced with high accuracy in the reference image 1c. In other words, the image processing device 100 may hardly emphasize the color of the object reflected in the input image 1b, for example.

On the other hand, the colors of objects that are not universally distributed (for example, objects that are unevenly distributed depending on the individual environment) do not have to be reproduced with high accuracy in the reference image 1c. Further, the color of an object having a large color variation (for example, an object whose color changes depending on the season and the weather, such as the color of a mountain and the sky) does not have to be reproduced with high accuracy in the reference image 1c. In other words, the image processing device 100 may, for example, greatly emphasize the color of the object reflected in the input image 1b and generate the adjusted image as the output image 1e.

Humans feel uncomfortable when the colors of familiar (universally distributed) objects are emphasized, but it is difficult for humans to feel uncomfortable even when the colors of unfamiliar objects and objects with many color variations are emphasized. The image processing device 100 utilizes this property of human beings to distinguish between a universally distributed object, a non-universally distributed object, and an object having a large color variation, and inputs so as to emphasize only the latter color. Adjust image 1a.

As a result, the image processing device 100 can realize more versatile color component adjustment than the conventional image processing device that can adjust the color only for the object assumed in advance.

As illustrated in FIG. 2, the first model parameter 225 is stored in the storage unit 224. Further, for example, before the first model parameter 225 is incorporated into the image processing device 100, the first deep learning device 223 is trained in advance outside the image processing device 100 (a device other than the image processing device 100). May be generated by The storage unit 224 only needs to be a storage device capable of reading and writing arbitrary data, and its configuration and form are not limited. The first model parameter 225 may be incorporated into the first deep learning device 223 from the storage unit 224, for example, during the operation of the first deep learning device 223.

(Structure of Color Difference Information Calculation Unit 104)
FIG. 3 is a block diagram showing a main configuration of the color difference information calculation unit 104 included in the image processing device 100. As illustrated in FIG. 3, the color difference information calculation unit 104 may include a first color space conversion unit 321, a second color space conversion unit 322, a color difference calculation unit 323, and a color difference normalization unit 324.

In the first embodiment, the color difference information calculation unit 104 determines the pixel value of each pixel of the input image 1b and each pixel of the reference image 1c at a position corresponding to each pixel of the input image 1b in a predetermined color space. An example will be described in which a plurality of color difference values, which are distances from the pixel values of the above, are calculated, and the plurality of color difference values are included in the color difference information 1d and output.

The first color space conversion unit 321 may, for example, convert the pixel value of each pixel of the reference image 1c into a pixel value in a predetermined color space. Further, the second color space conversion unit 322 may, for example, convert the pixel value of each pixel of the input image 1b into the pixel value in a predetermined color space. Here, the predetermined color space in the first color space conversion unit 321 and the predetermined color space in the second color space conversion unit 322 may be the same color space or different color spaces. May be good. Further, the transformation may be a linear transformation or a non-linear transformation appropriately set according to the purpose.

The predetermined color space may be, for example, a uniform color space. In the first embodiment, it is assumed that the predetermined color space is the CIELAB color space, which is one of the uniform color spaces. However, the predetermined color space may be a CIELUV color space or a color space with higher uniformity accuracy such as CIE2000. Further, the predetermined color space may be a color space other than the uniform color space (for example, YCbCr color space).

The color difference calculation unit (color difference information calculation unit) 323 corresponds to, for example, the pixel values of a plurality of pixels included in the input image 1b and the pixel values of the plurality of pixels included in the reference image 1c corresponding to the plurality of pixels. The distances may be calculated in a predetermined color space, and the distances may be included as a part of the color difference information 1d. The color difference calculation unit 323 may use the distance as the color difference value of the pixel, for example.

Here, the above distance may be any distance such as the Euclidean distance, the Manhattan distance, and the squared distance. Further, since the color difference calculation unit 323 can calculate a plurality of color difference values, for example, the color difference values may be calculated for all the pixels of the input image 1b and the reference image 1c. Details of the processing executed by the color difference calculation unit 323 will be described later.

The color difference normalization unit 324 may, for example, normalize a plurality of color difference values obtained by the color difference calculation unit 323, and output the plurality of color difference values after normalization in the color difference information 1d. For example, the color difference normalization unit 324 performs an operation of zeroing a color difference value equal to or less than a predetermined value among a plurality of color difference values and subtracting a predetermined value from a color difference value larger than the predetermined value among the plurality of color difference values. The result of is included in the color difference information 1d and output.

(Effect of color difference value of color difference information calculation unit 104)
As described above, in the reference image 1c generated by the reference image generation unit 103, the colors of the universally distributed objects are reproduced with high accuracy. Therefore, the color difference value of the universally distributed object portion generated by the color difference information calculation unit 104 is a relatively small value. Further, in the reference image 1c generated by the reference image generation unit 103, the colors of objects that are not universally distributed and objects that have a large color variation do not have to be reproduced with high accuracy. Therefore, the color difference value of the part of the object that is not universally distributed and the object that has a large color variation generated by the color difference information calculation unit 104 is a relatively large value.

Therefore, the image processing device 100 can distinguish between the part of the object that is universally distributed in the input image 1b and the part of the object that is not universally distributed and the part of the object having a large color variation depending on the magnitude of the color difference value. It works.

(Structure of color component adjusting unit 105)
FIG. 4 is a block diagram showing the configuration of the color component adjusting unit 105. As illustrated in FIG. 4, the color component adjusting unit 105 may include a third color space conversion unit 421, a saturation adjusting unit 422, and a color space inverse conversion unit 423.

The third color space conversion unit (color component adjustment unit) 421 may generate a second image by, for example, converting the input image 1b into an image in a color space having saturation as an axis component. Here, the color space may be, for example, an HSL color space, an HSV color space, or the like.

The saturation adjusting unit (color component adjusting unit) 422 may generate an output image 1e based on the converted image, for example. Specifically, the saturation adjustment unit 422 colors each pixel of the second image input from the third color space conversion unit 421 according to the color difference information 1d input from the color difference information calculation unit 104, for example. A third image with adjusted degree components may be generated.

The saturation adjustment unit 422 may adjust the saturation of each pixel of the second image according to the magnitude of the color difference value included in the color difference information 1d, for example. Specifically, the saturation adjusting unit 422 may adjust the saturation so that the adjusted saturation is in the range of 0.0 to 1.0. Alternatively, the saturation adjusting unit 422 may emphasize the saturation more strongly, for example, as the pixel has a larger color difference value. Then, the saturation adjustment unit 422 may output, for example, an image in which the HSL pixel value of each pixel is (H, S2, L) as the above-mentioned third image.

The color space inverse conversion unit (color component adjustment unit) 423 converts, for example, a third image input from the saturation adjustment unit 422 into an image in the color space of the input image 1b to generate an output image 1e, and the output image 1e is generated. The output image 1e may be output.

The color space inverse conversion unit 423 converts, for example, a third image in which the HSL pixel value of each pixel is (H, S2, L) into an image in the RGB color space which is the color space of the input image 1b. , The output image 1e may be generated. That is, while the third color space conversion unit 421 converts the image from the RGB color space to the HSL color space, the color space inverse conversion unit 423 may execute the reverse conversion.

Here, when the third color space conversion unit 421 converts the input image 1b into a relative color space having saturation as an axis component and then adjusts the color component, the color space inverse conversion unit 423 is easy. Can be converted back to. Therefore, the image processing device 100 can simplify the above adjustment / inverse conversion process.

As described above, the color component adjusting unit 105 converts, for example, the input image 1b into a second image in the color space having saturation as an axis component, and converts each pixel of the second image into color difference information 1d. A third image whose saturation component is adjusted accordingly may be generated, and the third image may be inversely converted into the color space of the input image 1b to generate the output image 1e. That is, the color component adjusting unit 105 may perform the saturation adjustment processing of each pixel of the input image 1b as the color component adjusting processing of each pixel of the input image 1b.

(Image color gamut and its conversion)
It is preferable that the color gamut targeted by the color component adjustment treatment is sufficiently wide. This is because the wider the color gamut, the greater the degree of freedom in the color component adjustment process. That is, the image processing apparatus 100 may convert the input image 1b into an image having a wide color gamut, for example, in the initial stage of a series of processing. As a result, the image processing apparatus 100 can prevent the pixel value of each pixel included in the image from deviating from the value outside the color range in the processing after the conversion processing. For example, the color gamut targeted by each process included in the image processing apparatus 100 is defined by the ITU-R recommendation BT. It may be the color gamut specified in 2020 (hereinafter referred to as Rec.2020).

For example, the input image 1b is the ITU-R recommendation BT. It is an image in an RGB color space conforming to the color gamut defined in 709 (hereinafter referred to as Rec.709), and each process included in the image processing apparatus 100 is a Rec. Consider the case of processing in an RGB color space compliant with 2020. In this case, the image conversion unit 101 converts the image into an image in the XYZ color space, and rec. Converts to an image in an RGB color space compliant with 2020.

For example, when the input image 1b is an image in a color space other than the RGB color space (for example, YCbCr color space) and the image is converted into an image in the RGB color space, the image conversion unit 101 executes the same processing. By doing so, the image can be converted.

If the color gamut of the input image 1a and the target color gamut of each process included in the image processing device 100 are the same, the image conversion unit 101 does not need to execute the color gamut conversion process. Therefore, the image conversion unit 101 may send the input image 1a to the image storage unit 102 without converting the input image 1a. Further, when the conversion process becomes unnecessary, for example, the color gamut of the input image 1a is determined in advance to one type, the image processing device 100 does not have to include the image conversion unit 101.

When the color gamut of the input image 1a is wider than the target color gamut for each process included in the image processing device 100, it is necessary to perform a process of fitting an image having a wide color gamut into a narrow color gamut. Various well-known prior arts can be applied to the treatment. In the following, unless otherwise specified, the image converted by the image conversion unit 101 is described in Rec. An image in an RGB color space compliant with 2020.

(Effects related to saturation adjustment of color component adjusting unit 105)
As described above, the color difference value generated by the color difference information calculation unit 104 is a relatively small value in the universally distributed object portion. Therefore, the color component adjusting unit 105 hardly changes the saturation of the pixel corresponding to the object which is uncomfortable for human beings when the color component changes. On the other hand, the color difference value generated by the color difference information calculation unit 104 is a relatively large value in the portion of the object that is not universally distributed and the object that has a large color variation. For this reason, the color component adjusting unit 105 greatly changes the saturation of the pixels corresponding to the object with less discomfort. That is, the color component adjusting unit 105 increases the saturation of each pixel of the input image 1b as the color difference value with respect to the pixel increases.

As a result, the image processing device 100 can execute the color component adjusting process on the input image 1a so as not to give a human discomfort to any object in the image.

(Processing executed by the image processing device 100)
FIG. 5 is a flowchart showing an example of processing executed by the image processing apparatus 100. As illustrated in FIG. 5, the image conversion unit 101 converts the input image 1a into an image in the color gamut used in the processing of the image processing device 100 (S501), if necessary.

The reference image generation unit 103 generates a reference image 1c in which the color components of at least a part of the pixels included in the input image 1b are changed based on the input image 1b (S502). The color difference information calculation unit 104 calculates the color difference information 1d based on the color difference between the plurality of pixels included in the input image 1b and the plurality of pixels included in the reference image 1c (S503). The color component adjusting unit 105 generates an output image 1e in which the color components of the plurality of pixels included in the input image 1b are adjusted by using the color difference information 1d (S504), and outputs the output image 1e. (S505).

(Process executed by the reference image generation unit 103)
FIG. 6 is a flowchart showing an example of processing executed by the reference image generation unit 103 included in the image processing device 100. FIG. 6 is a flowchart illustrating an example of step S502 included in the flowchart illustrated in FIG. 5 in more detail.

As illustrated in FIG. 6, a color-reduced image 2a in which the color components of the plurality of pixels included in the input image 1b are reduced is generated (S601). Then, the coloring unit 222 generates a colored image in which a color component is added to each of the plurality of pixels included in the reduced color image 2a as the reference image 1c (S602).

(Process executed by the color difference information calculation unit 104)
FIG. 7 is a flowchart showing an example of the processing executed by the color difference information calculation unit 104 provided in the image processing apparatus 100. FIG. 7 is a flowchart illustrating an example of step S503 included in the flowchart illustrated in FIG. 5 in more detail.

As illustrated in FIG. 7, the first color space conversion unit 321 converts the pixel value of each pixel of the reference image 1c into the pixel value in a predetermined color space (S701). The second color space conversion unit 322 converts the pixel value of each pixel of the input image 1b into the pixel value in a predetermined color space (S702).

The color difference calculation unit 323 calculates the distance between the pixel values in the predetermined color space at the same pixel position for each pixel of the input image 1b and the reference image 1c, and sets this as a plurality of color difference values ( S703). That is, the color difference calculation unit 323 corresponds to the pixel values of the plurality of pixels included in the input image 1b and the pixel values of the plurality of pixels included in the reference image 1c corresponding to the plurality of pixels. Are calculated in a predetermined color space.

The color difference normalization unit 324 includes a plurality of color difference values after normalization in the color difference information 1d and outputs them (S704). At this time, the color difference calculation unit 323 may, for example, normalize a plurality of color difference values obtained by the color difference calculation unit 323. The processes of S701 and S702 may be performed in a different order or in parallel.

(Process executed by the color component adjusting unit 105)
FIG. 8 is a flowchart showing an example of processing executed by the color component adjusting unit 105 included in the image processing apparatus 100. FIG. 8 is a flowchart illustrating an example of step S504 included in the flowchart illustrated in FIG. 5 in more detail.

As illustrated in FIG. 8, the third color space conversion unit 421 converts the input image 1b into an image in the color space whose axis component is saturation (S801). The saturation adjustment unit 422 generates an image (third image) in which the saturation components of a plurality of pixels included in the converted image (second image) are adjusted according to the color difference information 1d (third image). S802). The color space inverse conversion unit 423 generates an output image 1e by inversely converting the third image into the color space of the input image 1b (S803).

(Effect of image processing device 100)
As described above, human beings feel uncomfortable when the colors of familiar objects are emphasized, but they are less likely to feel uncomfortable when the colors of unfamiliar objects and objects with variations in color are emphasized. The image processing apparatus 100 utilizes this property of human beings to learn universally distributed objects, non-universally distributed objects, and objects having color variations, and emphasizes only the latter color. Adjust the input image 1a.

As a result, the image processing device 100 can realize more versatile color component adjustment than the conventional image processing device that can adjust the color only for the object assumed in advance in the image.

<Embodiment 2>
A second embodiment (Embodiment 2) will be described with reference to FIGS. 1, 2, and 9. In each drawing, the same or equivalent elements are designated by the same reference numerals, and duplicate description will be omitted.

(Structure of reference image generation unit 903)
The control unit 110 included in the image processing device 100 may include, for example, the reference image generation unit 903 instead of the reference image generation unit 103.

FIG. 9 is a block diagram showing an example of the configuration of the main part of the reference image generation unit 903. As illustrated in FIG. 9, the reference image generation unit 903 includes a color reduction unit 921 in place of the color reduction unit 221. Further, the storage unit 224 stores the first model parameter 925 in place of the first model parameter 225.

The color reduction unit (reference image generation unit) 921 may generate, for example, a color reduction image 2a in which the color components of a plurality of pixels included in the input image 1b input from the image storage unit 102 are reduced.

Here, in the first embodiment, an example in which the color reduction unit 221 generates a color reduction image 2a (monochrome image) by setting the color components of a plurality of pixels included in the input image 1b to zero has been described. On the other hand, in the second embodiment, an example in which the color reduction unit 921 generates the color reduction image 2a by making the color components of the plurality of pixels included in the input image 1b larger than zero will be described. That is, the color reduction image 2a generated by the color reduction unit 921 is an image in which the color is reduced so that the color component remains. In other words, the color reduction unit 921 leaves a clue of the color component for the coloring unit 222 to color in the color reduction image 2a.

The first model parameter 925 is optimized in the same way as the first model parameter 225. However, the first model parameter 925 is optimized using the color reduction image 2a generated by the color reduction unit 921 as input data. That is, the first model parameter 925 optimizes the combination of the color-reduced image 2a in which the clue for coloring is left and the color source image as teacher data.

Therefore, the image processing device 100 can correctly color the first deep learning device 223 defined by the first model parameter 925. That is, the image processing device 100 can reduce the probability of executing the color component adjusting process that makes humans feel uncomfortable, and increase the versatility of the color component adjusting process.

<Embodiment 3>
A third embodiment (Embodiment 3) will be described with reference to FIGS. 1, 2, and 10. In each drawing, the same or equivalent elements are designated by the same reference numerals, and duplicate description will be omitted.

(Structure of reference image generation unit 1003)
The control unit 110 included in the image processing device 100 may include, for example, the reference image generation unit 1003 instead of the reference image generation unit 103.

FIG. 10 is a block diagram showing an example of the configuration of the main part of the reference image generation unit 1003. As illustrated in FIG. 10, the reference image generation unit 1003 includes a color reduction unit 1021 in place of the color reduction unit 221.

The color reduction unit (reference image generation unit) 1021 may generate, for example, a color reduction image 2d in which the color components of a plurality of pixels included in the input image 1b input from the image storage unit 102 are reduced. The color reduction unit 1021 includes a second deep learner (second trained model) 1026 defined by the second model parameter 1028 and a storage unit 1027. The storage unit 1027 stores the second model parameter 1028. In addition, the storage unit 224 stores the first model parameter 1025.

The second deep learner 1026 may be, for example, a neural network (for example, a multi-layer perceptron (including a deep model)). Further, the second model parameter 1028 may be, for example, data including the connection weight of the neural network constituting the second deep learner 1026.

The second model parameter 1028 is optimized as follows, for example. First, the image processing device 100 collects color source images (input image 1a) in which various subjects are captured by an appropriate method (for example, randomly) from an arbitrary information source (for example, the Internet).

Next, the image processing device 100 converts the color gamut of the color source image by a process executed by the image conversion unit 101. Next, the image processing device 100 generates a color-reduced image 2a in which the color source image is reduced by a process executed by the color-reducing unit 221. In the third embodiment, the reduced color image 2a may be a monochrome image as in the first embodiment.

As a result, a combination in which the converted color source image is input data and the color reduction image 2a is correct data is obtained as teacher data. Then, the image processing device 100 optimizes the second model parameter 1028 by training the second deep learning device 1026 using the teacher data.

The first model parameter 1025 is optimized as follows, for example. First, the image processing device 100 converts the color gamut of the color source image by a process executed by the image conversion unit 101. Next, the image processing device 100 generates a color-reduced image 2d obtained by reducing the color of the converted color source image by a process executed by the second deep learning device 1026 defined by the second model parameter 1028.

As a result, a combination in which the color-reduced image 2d is input data and the color source image after conversion is correct data is obtained as teacher data. Then, the image processing device 100 optimizes the first model parameter 1025 by training the first deep learning device 223 using the teacher data.

Here, the image processing device 100 may proceed with the optimization so that the second deep learning device 1026 can completely infer the monochrome image, for example, or the optimization may proceed before the inference becomes possible. You may intentionally stop the conversion on the way. In the following, the explanation will be continued assuming the latter case.

When learning (optimization of the second model parameter 1028) is stopped in the middle, the second deep learning device 1026 generates an image in which the color is reduced so that the color component of the input image 1b remains. However, familiar (universally distributed) objects undergo color reduction even if the learning is shallow, and unfamiliar objects and objects with variations in color retain their original color components.

Note that the object for which the second deep learning device 1026 could not completely reduce the color (the original color component remains) is an object for which it is difficult for the first deep learning device 223 to add the color component. To do. This is because the first model parameter 1025 and the second model parameter 1028 are parameters optimized using the teacher data generated from the same color source image.

In other words, the second deep learning device 1026 leaves clues for color components for coloring in the color-reduced image 2d according to the distribution of objects. Therefore, the image processing device 100 can correctly color the first deep learning device 223 defined by the first model parameter 1025. That is, the image processing device 100 can reduce the probability of executing the color component adjusting process that makes humans feel uncomfortable, and further enhance the versatility of the color component adjusting process.

<Embodiment 4>
A fourth embodiment (Embodiment 4) will be described with reference to FIGS. 11 and 12. In each drawing, the same or equivalent elements are designated by the same reference numerals, and duplicate description will be omitted.

(Overall configuration of image processing device 1100)
FIG. 11 is a block diagram showing an example of the main configuration of the image processing apparatus 1100. As illustrated in FIG. 11, the image processing device 1100 is a device that processes the input input image 1a to generate the output image 1e, and includes, for example, a control unit 1110.

The control unit 1110 has a function of comprehensively controlling various functions of the image processing device 1100, and replaces the color component adjusting unit 105 (see FIG. 1) included in the control unit 110 (see FIG. 1) with a color component adjusting unit. Including part 1105. Further, the control unit 1110 may further include, for example, a user adjustment amount input unit 1106.

The user adjustment amount input unit 1106 may acquire, for example, an input related to the user adjustment amount 11a. Here, the user adjustment amount 11a is, for example, a color component input via an input device (for example, a keyboard, a remote controller displayed on the screen, a volume controller, etc.) connectable to the image processing device 1100 so as to be input. It may be an amount for adjusting the adjustment amount. The user adjustment amount 11a may be, for example, a real number within the range of 0 to 2, or a discrete value of strong (representing 2 in real number), medium (representing 1), and weak (representing 0). There may be.

FIG. 12 is a block diagram showing an example of the configuration of the main part of the color component adjusting unit 1105. The color component adjusting unit 1105 may further include, for example, a color difference information adjusting unit 1224 in addition to each unit provided by the color component adjusting unit 105.

The color difference information adjustment unit (color component adjustment unit) 1224 further uses the user adjustment amount 11a to adjust the color components of a plurality of pixels included in the input image 1b to generate the output image 1e. The color difference information adjustment unit 1224 outputs, for example, a value obtained by multiplying the color difference information 1d (color difference value) input from the color difference information calculation unit 104 by the user adjustment amount 11a to the saturation adjustment unit 422 as the adjusted color difference value. You can.

With the above configuration, the image processing device 1100 allows the user to fine-tune the color component. Therefore, the image processing apparatus 1100 can further enhance the versatility of the color component adjusting process.

<Embodiment 5>
A fifth embodiment (Embodiment 5) will be described with reference to FIGS. 1 and 13. In each drawing, the same or equivalent elements are designated by the same reference numerals, and duplicate description will be omitted.

(Structure of reference image generation unit 1303)
The control unit 110 included in the image processing device 100 may include, for example, the reference image generation unit 1303 instead of the reference image generation unit 103.

FIG. 13 is a block diagram showing an example of the configuration of the main part of the reference image generation unit 1303. As illustrated in FIG. 13, the reference image generation unit 1303 further includes a model parameter replacement unit 1329. The model parameter replacement unit 1329 may replace, for example, the first model parameter 225 with a model parameter (for example, model parameter 1330, model parameter 1331) different from the first model parameter 225.

When the user inputs an instruction / operation for exchanging the model parameters to the image processing device 100, the model parameter exchanging unit 1329 is, for example, a model parameter storage server communicably connected to the image processing device 100 via the network 1308. The different model parameters are obtained from 1309 and the first model parameters 225 are replaced with the different model parameters. The different model parameters are parameters optimized with teacher data different from the teacher data used for optimization with the first model parameter 225. The method of accumulating the different model parameters is not limited to cloud storage such as the model parameter storage server 1309, and may be, for example, a local disk.

Where human preference for color depends on the personal environment, the image processing apparatus 100 with the reference image generator 1303 has a variety of teacher data (eg, according to predetermined criteria) from the first model parameter 225. By replacing the model parameters optimized using the teacher data that includes the classified images as correct answer data, the teacher data that includes only the images that meet the predetermined criteria as correct answer data, etc.), the user can change the tendency of color component adjustment. Can be changed to suit your taste.

When the reference image generation unit 1303 further includes the color reduction unit 1021, the model parameter replacement unit 1329 may replace, for example, the second model parameter 1028 with a model parameter different from the second model parameter 1028.

With the above configuration, the image processing device 100 can realize a color component adjusting process that suits the user's taste. Therefore, the image processing device 100 can further enhance the versatility of the color component adjusting process.

<Embodiment 6>
A sixth embodiment (Embodiment 6) will be described with reference to FIGS. 1 and 14 to 15. In each drawing, the same or equivalent elements are designated by the same reference numerals, and duplicate description will be omitted.

(Structure of color difference information calculation unit 1404)
The control unit 110 provided in the image processing device 100 may include, for example, the color difference information calculation unit 1404 instead of the color difference information calculation unit 104, and the color component adjustment unit 1505 instead of the color component adjustment unit 105.

FIG. 14 is a block diagram showing an example of the configuration of the main part of the color difference information calculation unit 1404. As illustrated in FIG. 14, the color difference information calculation unit 1404 includes a saturation difference calculation unit 1425 in place of the color difference calculation unit 323 and a saturation difference normalization unit 1426 in place of the color difference normalization unit 324. You can.

The saturation difference calculation unit (color difference information calculation unit) 1425 is used, for example, for the first saturation value calculated for each of the plurality of pixels included in the input image 1b and for the plurality of pixels included in the reference image 1c. A plurality of saturation difference values indicating the difference from the second saturation value calculated in each case may be calculated. Here, the plurality of saturation difference values may take either a positive value or a negative value.

The saturation difference normalization unit (color difference information calculation unit) 1426, for example, normalizes the plurality of saturation difference values, and uses the normalized plurality of saturation difference values as a part of the color difference information 1d. It may be included and output. For example, the saturation difference normalization unit 1426 may normalize the saturation difference value based on a value obtained by multiplying the saturation difference value by an arbitrary constant.

FIG. 15 is a block diagram showing an example of the configuration of the main part of the color component adjusting unit 1505. As illustrated in FIG. 15, the color component adjusting unit 1505 may include a saturation adjusting unit 1522 instead of the saturation adjusting unit 422.

The saturation adjustment unit (color component adjustment unit) 1522 may generate an output image 1e based on, for example, the converted image. Specifically, the saturation adjustment unit 1522 has, for example, a color difference for each pixel of the second image (for example, the input image 1b converted to the HSL color space) input from the third color space conversion unit 421. A third image in which the saturation component is adjusted according to the color difference information 1d input from the information calculation unit 1404 may be generated.

At this time, the saturation adjustment unit 1522 adjusts the saturation of each pixel of the second image output by the third color space conversion unit 421 according to the saturation difference value included in the color difference information 1d. You can. Specifically, the saturation adjustment unit 1522 may change the saturation of the plurality of pixels according to the positive / negative of the saturation difference value corresponding to each of the plurality of pixels included in the second image ( For example, if the saturation difference value is positive, the saturation may be increased, and if the saturation difference value is negative, the saturation may be decreased).

That is, the color component adjusting unit 1505 adjusts the saturation by increasing or decreasing the saturation of the plurality of pixels included in the input image 1b, respectively, and generates the third image. As a result, the image processing apparatus 100 can realize a wide range of color component adjustment processing, so that the versatility of the color component adjustment processing can be further enhanced.

<Embodiment 7>
A seventh embodiment (Embodiment 7) will be described with reference to FIGS. 1, 2, 4, 16, and 17. In each drawing, the same or equivalent elements are designated by the same reference numerals, and duplicate description will be omitted.

(Structure of color difference information calculation unit 1604)
The control unit 110 provided in the image processing device 100 may include, for example, the color difference information calculation unit 1604 instead of the color difference information calculation unit 104, and the color component adjustment unit 1705 instead of the color component adjustment unit 105.

FIG. 16 is a block diagram showing an example of the configuration of the main part of the color difference information calculation unit 1604. As illustrated in FIG. 16, the color difference information calculation unit 1604 replaces the first color space conversion unit 321 with the fourth color space conversion unit 1627, and replaces the second color space conversion unit 322 with the fifth. The color space conversion unit 1628 may be provided with a hue difference calculation unit 1629 instead of the color difference calculation unit 323, and a hue difference normalization unit 1630 instead of the color difference normalization unit 324.

The fourth color space conversion unit 1627 and the fifth color space conversion unit 1628 may, for example, convert the reference image 1c and the input image 1b into images in a color space having hue as an axis component, respectively. Here, the color space may be, for example, an HSL color space, an HSV color space, or the like. The transformation may be a linear transformation or a non-linear transformation appropriately set according to the purpose.

The hue difference calculation unit 1629, for example, has a first hue value calculated for each of a plurality of pixels included in the input image 1b and a second hue calculated for each of the plurality of pixels included in the reference image 1c. A plurality of hue difference values indicating the difference from the values may be calculated. In other words, the hue difference calculation unit 1629 may calculate the hue difference between the reference image 1c and the input image 1b.

The hue difference normalization unit 1630 may, for example, normalize a plurality of hue difference values calculated by the hue difference calculation unit 1629, and include the plurality of normalized hue difference values in the color difference information 1d for output. For example, when the hue difference normalization unit 1630 is a value outside the predetermined range (for example, when it is larger than 6 °), the hue difference value may be replaced with zero and output.

That is, if the hue difference value of a certain pixel is out of the predetermined range, it is interpreted that the reliability of coloring by the deep learning device 223 is low, and the image processing apparatus 100 does not use the coloring for hue adjustment. As a result, the image processing apparatus 100 can further enhance the versatility of the color component adjusting process.

FIG. 17 is a block diagram showing an example of the configuration of the main part of the color component adjusting unit 1705. As illustrated in FIG. 17, the color component adjusting unit 1705 may include, for example, a sixth color space conversion unit 1721, a hue adjusting unit 1724, and a color space inverse conversion unit 1723.

The sixth color space conversion unit (color component adjustment unit) 1721 may generate a second image by, for example, converting the input image 1b into an image in a color space having a hue as an axis component. Here, the color space may be, for example, an HSL color space, an HSV color space, or the like.

The hue adjustment unit (color component adjustment unit) 1724 may change the hues of the plurality of pixels included in the input image 1b according to the plurality of hue difference values input from the hue difference information calculation unit 1604, for example. Specifically, when the sixth color space conversion unit 1721 converts the input image 1b, the hue adjustment unit 1724 may generate the output image 1e based on the converted image. At this time, the hue adjusting unit 1724 may generate an image in which the hue components of a plurality of pixels included in the converted image are adjusted according to the color difference information 1d, for example.

The color space inverse conversion unit 1723 may, for example, inversely convert the image generated by the hue adjustment unit 1724 into the color space of the input image 1b.

The calculated hue difference for the pixels corresponding to the universally distributed object is smaller than the hue difference of other objects (objects that are not universally distributed, objects with large color variation, etc.). In order to adjust the color component (specifically, the hue) based on the hue difference, the color component adjusting unit 1705 reduces the adjustment amount of the former color component and increases the latter.

As a result, the image processing device 100 can realize more versatile color component adjustment than the conventional image processing device that can adjust the color only for the object assumed in advance in the image.

<Embodiment 8>
An eighth embodiment (Embodiment 8) will be described with reference to FIG. In each drawing, the same or equivalent elements are designated by the same reference numerals, and duplicate description will be omitted.

FIG. 18 is a schematic view schematically showing an image display device 1801 provided with an image processing device according to each embodiment. The image display device 1801 may be any device as long as it can display an arbitrary image, and may be, for example, a liquid crystal display, an organic EL display, or the like. The image display device 1801 displays the output image 1e output by the image processing device on the display surface 1802, and presents the output image 1e to the user.

<Example of realization by software>
The control block provided in the image processing device according to each aspect of the present disclosure may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by a CPU (Central Processing Unit) or the like. It may be realized by software using a controller (processor). That is, each image processing device is a CPU that executes instructions of a control program that is software that realizes each function, the control program, and a ROM (Read Only Memory) in which various data are readablely recorded by a computer (or CPU). ) Or a storage device (these are referred to as "recording media"), a RAM (Random Access Memory) for deploying the control program, and the like. Then, when the computer (or CPU) reads the control program from the recording medium and executes it, an example of the object according to each aspect of the present disclosure is achieved. As the recording medium, a "non-temporary tangible medium", for example, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. Further, the control program may be supplied to the computer via an arbitrary transmission medium (communication network, broadcast wave, etc.) capable of transmitting the control program. Each aspect of the present disclosure can also be realized in the form of a data signal embedded in a carrier wave, in which the control program is embodied by electronic transmission.

The above control program can be implemented in any programming language. For example, the control program can be implemented using a script language such as ActionScript or JavaScript (registered trademark), an object-oriented programming language such as Objective-C or Java (registered trademark), or a markup language such as HTML5. In addition, an information processing terminal (for example, a smartphone or a personal computer) having each part that realizes each function realized by the control program and a server having each part that realizes the remaining functions different from the above-mentioned functions are also available. It falls within the scope of this disclosure.

The present disclosure is not limited to the above-described embodiment, and various modifications can be made within the scope of the claims, and the embodiment obtained by appropriately combining the technical means disclosed in the different embodiments. Is also included in the technical scope of the present disclosure. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment.

Claims (20)

1. A reference image generation unit that generates a reference image in which the color components of at least some of the pixels included in the input image are changed based on the input image.
   A color difference information calculation unit that calculates color difference information based on the color difference between the plurality of pixels included in the input image and the plurality of pixels included in the reference image.
   An image processing device including a color component adjusting unit that generates an output image in which the color components of the plurality of pixels included in the input image are adjusted by using the color difference information.
2. The reference image generation unit generates a color-reduced image in which the color components of the plurality of pixels included in the input image are reduced, and then adds a color component to each of the plurality of pixels included in the color-reduced image. The image processing apparatus according to claim 1, wherein the image is generated as the reference image.
3. The image processing device according to claim 2, wherein the reference image generation unit uses a first trained model defined by a first model parameter to add a color component to the color-reduced image.
4. The image processing apparatus according to claim 3, further comprising a model parameter replacement unit that replaces the first model parameter with a model parameter different from the first model parameter.
5. The reference image generation unit is characterized in that, among the plurality of pixels included in the input image, an image in which the value of the color component of the pixel having a color component is set to zero or more is generated as the color reduction image. The image processing apparatus according to any one of claims 2 to 4.
6. The reference image generation unit according to any one of claims 2 to 5, wherein the reference image generation unit generates the color-reduced image by using the second trained model defined by the second model parameter. Image processing device.
7. The color difference information calculation unit determines the distance between the pixel values of the plurality of pixels included in the input image and the pixel values of the plurality of pixels included in the reference image corresponding to the plurality of pixels. The image processing apparatus according to any one of claims 1 to 6, wherein the distance is calculated in each color space and the distance is included as a part of the color difference information.
8. The image according to claim 7, wherein the color component adjusting unit increases the saturation of the plurality of pixels as the distance corresponding to each of the plurality of pixels included in the input image increases. Processing equipment.
9. The color difference information calculation unit has a first saturation value calculated for each of the plurality of pixels included in the input image and a second saturation value calculated for each of the plurality of pixels included in the reference image. The image processing apparatus according to any one of claims 1 to 8, wherein a plurality of saturation difference values indicating a difference from the saturation value are included as a part of the color difference information.
10. The image processing apparatus according to claim 9, wherein the color component adjusting unit changes the saturation of the plurality of pixels included in the input image according to the plurality of saturation difference values.
11. Claims 1 to 10 are characterized in that the color component adjusting unit converts the input image into an image in a color space having saturation as an axis component, and generates the output image based on the converted image. The image processing apparatus according to any one of the above.
12. The color component adjusting unit generates an image in which the saturation components of a plurality of pixels included in the converted image are adjusted according to the color difference information, and the generated image is used in the color space of the input image. The image processing apparatus according to claim 11, wherein the output image is generated by inverse conversion.
13. The color difference information calculation unit has a first hue value calculated for each of the plurality of pixels included in the input image and a second hue calculated for each of the plurality of pixels included in the reference image. The image processing apparatus according to any one of claims 1 to 12, wherein a plurality of hue difference values indicating a difference from the values are included as a part of the color difference information.
14. The image processing apparatus according to claim 13, wherein the color component adjusting unit changes the hues of the plurality of pixels included in the input image according to the plurality of hue difference values.
15. The color component adjusting unit according to claims 1 to 14, wherein the color component adjusting unit converts the input image into an image in a color space having a hue as an axis component, and generates the output image based on the converted image. The image processing apparatus according to any one item.
16. The color component adjusting unit generates an image in which the hue components of a plurality of pixels included in the converted image are adjusted according to the color difference information, and reverses the generated image to the color space of the input image. The image processing apparatus according to claim 15, wherein the output image is generated by conversion.
17. Further provided with a user adjustment amount input unit for acquiring input related to the user adjustment amount,
    Claims 1 to 16 are characterized in that the color component adjusting unit further adjusts the color components of the plurality of pixels included in the input image by further using the user adjustment amount to generate the output image. The image processing apparatus according to any one of the above.
18. An image display device including the image processing device according to any one of claims 1 to 17.
19. A reference image generation step of generating a reference image in which the color components of at least some of the pixels included in the input image are changed based on the input image, and
    A color difference information calculation step for calculating color difference information based on the color difference between the plurality of pixels included in the input image and the plurality of pixels included in the reference image.
    A control method of an image processing apparatus including a color component adjusting step of generating an output image in which the color components of the plurality of pixels included in the input image are adjusted by using the color difference information.
20. For image processing equipment
    A reference image generation function that generates a reference image in which the color components of at least some of the pixels included in the input image are changed based on the input image.
    A color difference information calculation function that calculates color difference information based on the color difference between the plurality of pixels included in the input image and the plurality of pixels included in the reference image.
    A control program that realizes a color component adjusting function that generates an output image in which the color components of the plurality of pixels included in the input image are adjusted by using the color difference information.

Images