JP6753145B2 - Image processing equipment, image processing methods, image processing systems and programs - Google Patents

Description

The present invention relates to an image processing apparatus, an image processing method, an image processing system, and a program.

In recent years, the number of users who shoot and browse digital images is increasing due to the spread of devices such as digital cameras, smartphones, and tablets. At this time, the shooting environment varies depending on the influence of the illumination light and the like, and the shooting target also varies. Therefore, the image taken after shooting may not be what the user intended, and therefore it is common practice to adjust the color tone and the like of the shot image.
In addition, even if the same item is photographed, the color reproduction will be different if the shooting environment is different, even if the model characteristics of multiple digital cameras used and the same digital camera are used, and the image will be photographed under specific conditions. The color may be adjusted (for example, the image of one condition is adjusted to the image of the other condition).

In Patent Document 1, an image to which the ID of the digital camera that captured the image is given is input, color adjustment is performed on the input image, and color adjustment is performed with the original image before color adjustment. Create a correction profile based on the color difference from the later adjusted image, store the created correction profile in association with the digital camera ID, and then enter the image with the digital camera ID. By attaching the correction profile created at the time of creation to the image and performing color matching at the time of printing or display, the same color adjustment as when creating the correction profile for any image is performed. Image processing devices capable of this are described.

Further, in Patent Document 2, a correction profile created based on a color difference between an adjusted image in which an arbitrary color adjustment is performed on an image and an original image before color adjustment is captured as an image. The profile is stored in association with the camera ID, the correction profile is searched using the digital camera ID included in the image data when the image is input, and the profile selected by the user is selected from the searched correction profiles. An image processing device attached to the input image is described.

Further, Patent Document 3 includes a conversion step of converting an input of an RGB signal or a CMY signal into a CIEXYZ signal, and an inverse conversion step of returning a CIEXYZ signal to an RGB signal or a CMY signal, and the matrix used in the conversion step is included. Describes the color correction method given by the sum of the first matrix and the second matrix, which are independent of each other, obtained based on two vectors obtained by dividing the coloring vector into a direction parallel to the achromatic vector and a direction perpendicular to the achromatic vector. Has been done.

Japanese Unexamined Patent Publication No. 2004-304537 Japanese Unexamined Patent Publication No. 2004-304539 Japanese Unexamined Patent Publication No. 9-139855

For example, in the conventional method of photographing a color patch of a color sample and adjusting the color based on the color patch, when an article such as clothing is photographed, it is used for the color material and the clothing used for the color patch. Since the color characteristics such as the spectral reflectance are different from those of the existing color material, the accuracy of the color adjustment may be lowered due to this. Further, when the number of images for which color adjustment is performed increases, the amount of work tends to be enormous. Furthermore, the result of color adjustment tends to vary due to differences in the skill and sensitivity of the user (retoucher) who performs color adjustment.
In addition, when an article such as clothing is photographed due to differences in the characteristics of digital cameras and differences due to the influence of the environment, spectral reflection between the color material used for the color patch and the color material used for the clothing Since the color characteristics such as the reflectance are different, the accuracy of the color adjustment is lowered due to this, and the shooting result is liable to vary.
An object of the present invention is to provide an image processing apparatus or the like capable of creating a conversion relationship in which the accuracy of color adjustment does not easily decrease even when the color characteristics of articles are different or the shooting conditions are different.

The invention according to claim 1 is a plurality of first color data which are color data in a region of interest of a first image representing a color correction target, and a second image different from the first image. A color data extraction unit that extracts a plurality of second color data paired with the first color data corresponding to the region of interest, and the extracted first color data and the second color data. A color conversion model creation unit that creates a color conversion model that expresses the relationship between the two, and a conversion relationship creation unit that creates a conversion relationship that reproduces the color adjustment of the first image based on the color conversion model. The color conversion model creating unit is an image processing apparatus characterized in that the color conversion model is created so that the relationship between the first color data and the second color data is a non-linear monotonic increase function. is there.
The invention according to claim 2 is a plurality of first color data which are color data in a region of interest of a first image representing a color correction target, and a second image different from the first image. A color data extraction unit that extracts a plurality of second color data paired with the first color data corresponding to the region of interest, and the extracted first color data and the second color data. A color conversion model creation unit that creates a color conversion model that expresses the relationship between the two, and a conversion relationship creation unit that creates a conversion relationship that reproduces the color adjustment of the first image based on the color conversion model. The color conversion model creating unit is an image processing apparatus characterized in that a weight is set for a pair of the first color data and the second color data to create the color conversion model.
In the invention according to claim 3 , the color conversion model creating unit sets the weight so that the relationship between the first color data and the second color data becomes a monotonically increasing function, and performs the color conversion. The image processing apparatus according to claim 2 , wherein a model is created.
According to a fourth aspect of the invention, the color transformation model creating section, an image processing apparatus according to claim 2 or 3, characterized in that setting the weighting according to the size of the region of interest.
The invention according to claim 5 is a plurality of first color data which are color data in a region of interest of a first image representing a color correction object, and a second image different from the first image. A color data extraction step of extracting a plurality of second color data paired with the first color data corresponding to the region of interest, and the extracted first color data and the second color data. A color conversion model creation step of creating a color conversion model representing the relationship of the above, and a conversion relationship creation step of creating a conversion relationship for reproducing the color adjustment of the first image based on the color conversion model are included. In the color conversion model creation step, the color conversion model is created so that the relationship between the first color data and the second color data is a non-linear monotonic increase function, or the first color data and the first color data are combined. It is an image processing method that creates the color conversion model by setting a weight for the pair with the second color data, or at least one of them.
The invention according to claim 6 includes a photographing device for photographing an imaged object and an image processing device for performing color adjustment on a first image photographed by the photographing device, and the image processing device is the first. Corresponding to a plurality of first color data which are color data in the attention region of one image and the attention region in a second image different from the first image, and paired with the first color data. A color data extraction unit that extracts a plurality of second color data, and a color conversion model creation unit that creates a color conversion model that represents the relationship between the extracted first color data and the second color data. A conversion relationship creating unit that creates a conversion relationship that reproduces the color adjustment of the first image based on the color conversion model, and the color conversion model creating unit includes the first color data and the above. Create the color conversion model so that the relationship with the second color data is a non-linear monotonic increase function, or set weights for the pair of the first color data and the second color data. An image processing system that creates or at least one of the color conversion models .
The invention according to claim 7 tells a computer that a plurality of first color data which are color data in a region of interest of a first image representing a color correction object and a second image different from the first image. A color data extraction function that extracts a plurality of second color data pairs with the first color data corresponding to the region of interest in the image, and the extracted first color data and the second color data. A color conversion model creation function that creates a color conversion model that represents the relationship with color data, and a conversion relationship creation function that creates a conversion relationship that reproduces the color adjustment of the first image based on the color conversion model. The color conversion model creation function creates the color conversion model so that the relationship between the first color data and the second color data is a non-linear monotonic increase function, or the first color conversion model is created. It is a program that creates the color conversion model by setting weights for the pair of the color data of the above and the second color data, or at least one of them.

According to the invention of claim 1 , when the color adjustment is performed using the created conversion relationship, a gradation step is less likely to occur.
According to the invention of claim 2 , it is possible to create a color conversion model having smoother characteristics.
According to the invention of claim 3 , it is possible to more easily create a color conversion model that becomes a monotonically increasing function.
According to the invention of claim 4 , it is possible to create a conversion relationship in which the accuracy of color adjustment is improved for more important colors.
According to the fifth aspect of the present invention, it is possible to provide an image processing method capable of creating a conversion relationship in which the accuracy of color adjustment is less likely to decrease even when the color characteristics of articles are different or the shooting conditions are different. In addition, it is possible to create a conversion relationship that matches the color characteristics of the article. Further, when color adjustment is performed using the created conversion relationship, at least one of the two is less likely to occur and the color conversion model having smoother characteristics can be created.
According to the sixth aspect of the present invention, it is possible to provide an image processing system capable of creating a conversion relationship in which the accuracy of color adjustment is less likely to decrease even when the color characteristics of the object to be photographed are different or the imaging conditions are different. In addition, it is possible to create a conversion relationship that matches the color characteristics of the article. Further, when color adjustment is performed using the created conversion relationship, at least one of the two is less likely to occur and the color conversion model having smoother characteristics can be created.
According to the invention of claim 7 , even when the color characteristics of the articles are different or the shooting conditions are different, a function capable of creating a conversion relationship in which the accuracy of color adjustment is unlikely to decrease can be realized by a computer. In addition, it is possible to create a conversion relationship that matches the color characteristics of the article. Further, when color adjustment is performed using the created conversion relationship, at least one of the two is less likely to occur and the color conversion model having smoother characteristics can be created.

It is a figure which shows the configuration example of the image processing system in this embodiment. It is a block diagram which shows the functional structure example of the image processing apparatus in this embodiment. (A)-(b) is a figure which showed the example of the pair of the 1st color data and the 2nd color data. FIG. 1A is a diagram showing a case where the color space is divided into quadrants and a pair of the first color data and the second color data is extracted from the divided quadrants. FIG. (B) is a diagram showing a case where the color space is divided by hue and a pair of the first color data and the second color data is extracted from the divided color regions. FIG. 3C is a diagram showing a case where the color range is divided by lightness and saturation and a pair of the first color data and the second color data is extracted from the divided color areas. It is a figure which showed the example of the case where the histogram is created about the pixel in a small area. It is a figure which showed the case of extracting the color data of different brightness. It is a figure which showed an example of a color conversion model. (A) to (b) are a case where a color conversion model is created so that the relationship between the first color data and the second color data becomes a monotonous increase function, and a color conversion model so as not to have a monotonous increase function. It is a figure which compared the case where was created. It is a flowchart explaining the operation of an image processing apparatus. It is a figure which showed the hardware configuration of an image processing apparatus.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

<Explanation of the entire image processing system>
FIG. 1 is a diagram showing a configuration example of the image processing system 1 according to the present embodiment.
As shown in the figure, the image processing system 1 of the present embodiment displays an image based on the image processing device 10 that adjusts the color of the original image captured by the camera 40 and the image data output by the image processing device 10. A display device 20, an input device 30 for the user to input various information to the image processing device 10, and a camera 40 for photographing the image target S and generating image data for color adjustment by the image processing device 10. To be equipped.

The image processing device 10 is, for example, a so-called general-purpose personal computer (PC). Then, the image processing device 10 is designed to perform color adjustment and the like by operating various application software under the control of the OS (Operating System).

The display device 20 displays an image on the display screen 21. The display device 20 includes, for example, a liquid crystal display for a PC, a liquid crystal television, a projector, or the like, which has a function of displaying an image by additive color mixing. Therefore, the display method in the display device 20 is not limited to the liquid crystal method. In the example shown in FIG. 1, the display screen 21 is provided inside the display device 20, but when a projector is used as the display device 20, the display screen 21 is a screen or the like provided outside the display device 20. It becomes.

The input device 30 is composed of a keyboard, a mouse, and the like. The input device 30 starts and ends application software for performing color adjustment, and as will be described in detail later, when performing color adjustment, the user inputs an instruction for performing color adjustment to the image processing device 10. Used for.

The camera 40 is an example of a photographing device, and includes, for example, an optical system that converges incident light and an image sensor that is an imaging means that detects the light converged by the optical system.
The optical system is composed of a single lens or a combination of a plurality of lenses. In the optical system, various aberrations are removed by the combination of lenses and the coating applied to the lens surface. The image sensor is configured by arranging image pickup elements such as CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor).

The image processing device 10 and the display device 20 are connected via a DVI (Digital Visual Interface). In addition, instead of DVI, connection may be made via HDMI (registered trademark) (High-Definition Multimedia Interface), DisplayPort, or the like.
Further, the image processing device 10 and the input device 30 are connected via, for example, USB (Universal Serial Bus). In addition, instead of USB, it may be connected via IEEE1394, RS-232C or the like.
Further, the image processing device 10 and the camera 40 are connected by wire in the illustrated example, and are connected via, for example, USB, IEEE1394, RS-232C. As a result, the image data of the image taken by the camera 40 by wire is sent to the image processing device 10. However, the present invention is not limited to this, and a wireless connection such as a wireless LAN (Local Area Network) or Bluetooth (Bluetooth) (registered trademark) may be used. Further, the image data may be passed to the image processing device 10 via a memory card such as an SD card without connecting the image processing device 10 and the camera 40.

In such an image processing system 1, the user first photographs the image target S with the camera 40. The image taken by the camera 40 is an original image as a first image, and this image data is sent to the image processing device 10. Then, the display device 20 displays the original image which is an image before the color processing is performed. Next, when the user inputs an instruction for performing color adjustment to the image processing device 10 using the input device 30, the image processing device 10 performs color adjustment on the original image. The result of this color adjustment is reflected in, for example, an image displayed on the display device 20, and an image after color adjustment as a second image different from the first image is redrawn and displayed on the display device 20. To. In this case, the user can interactively perform the color adjustment while looking at the display device 20, and the color adjustment work can be performed more intuitively and more easily.
Further, as the second image, in addition to the image after color adjustment by the image processing device 10, another image may be taken by another camera having different model characteristics from the camera 40, that is, different shooting conditions. In this case, the image taken by the camera 40 can be seen as the first image, and the other images taken by the cameras with different shooting conditions can be seen as the second image.

Further, the display device 20 creates a conversion relationship in which the color of the original image can be adjusted based on the result of the color adjustment to obtain the image after the color adjustment. In this color adjustment, for example, when the first color data and the second color data are RGB data including Red (R), Green (G), and Blue (B), the first color data is (R). _{a, G a, B a)} , the second color data _{(R b,} when G _{b, B} b) _{to, (R a, G a,} B a) → (R b, G b, B b) and It is a process to do. By using this conversion relationship, it is possible to reproduce the same color adjustment as the color adjustment performed earlier. This conversion relationship is also called a profile, and can be created as, for example, a three-dimensional LUT (Look up Table). However, it is not limited to this. For example, the conversion relationship, _{R a} → _R b, may be a _{G a} → _G b, 1-dimensional LUT to _{B a} → _{B b.} The conversion relationship _may be a multi-dimensional matrix to _{(R a, G a, B} a) → (R b, G b, B b). The process of creating this conversion relationship will be described later.

The image processing system 1 in the present embodiment is not limited to the embodiment shown in FIG. For example, a tablet terminal can be exemplified as the image processing system 1. In this case, the tablet terminal is provided with a touch panel, and the touch panel displays an image and inputs a user's instruction such as a touch. That is, the touch panel functions as the display device 20 and the input device 30. As the camera 40, a camera built in the tablet terminal can be used. Similarly, a touch monitor can be used as a device that integrates the display device 20 and the input device 30. This uses a touch panel as the display screen 21 of the display device 20. In this case, the image is displayed on the touch monitor based on the image data output by the image processing device 10. Then, the user inputs an instruction for performing color adjustment by touching the touch monitor or the like.

<Explanation of image processing device>
Next, the image processing device 10 will be described.

FIG. 2 is a block diagram showing a functional configuration example of the image processing device 10 according to the present embodiment. Note that FIG. 2 shows a selection of various functions of the image processing apparatus 10 that are related to the present embodiment. In this embodiment, an example is shown in which a color-adjusted image obtained by adjusting the color of the original image is used as the second image.
As shown in the figure, the image processing device 10 of the present embodiment includes an image data acquisition unit 11 that acquires image data, a user instruction reception unit 12 that receives user instructions, and a color adjustment unit 13 that adjusts the color of an image. A color data extraction unit 14 for extracting predetermined image data, a color conversion model creation unit 15 for creating a color conversion model, a conversion relationship creation unit 16 for creating a conversion relationship, and an output unit 17 are provided.

The image data acquisition unit 11 acquires the image data of the original image before the color adjustment is performed. This image data is in a data format for displaying on the display device 20, and is, for example, the RGB data described above. The image data acquisition unit 11 may acquire image data in another data format and color-convert the image data to obtain RGB data or the like.

The user instruction receiving unit 12 receives an instruction by the user regarding color adjustment input by the input device 30. Specifically, the user instruction receiving unit 12 receives an instruction for designating an area in the image for which color adjustment is performed as user instruction information. Further, the user instruction receiving unit 12 receives an instruction for color adjustment for this area as user instruction information.

The color adjustment unit 13 adjusts the color of the image data of the original image based on the user instruction information received by the user instruction reception unit 12. The user adjusts the color of the selected area by adjusting the hue, saturation, and brightness, for example.

The color data extraction unit 14 is different from the plurality of first color data, which is the color data in the region of interest of the original image (first image) representing the color correction target, and the original image (first image). A plurality of second color data paired with the first color data are extracted corresponding to the region of interest in the second image. Here, the color data extraction unit 14 performs a plurality of first color data, which is color data in the region of interest of the original image, and a plurality of second color data after color adjustment is performed on the first color data. And extract. This is because the color data extraction unit 14 uses a plurality of first color data, which is color data in the region of interest of the original image (first image) representing the color correction target, and the original image (first image). On the other hand, in the region of interest of the second image, which is the image after the color adjustment, the second color data after the color adjustment is extracted from the first color data can be paraphrased.
Here, the region of interest is an region in an image that requires color adjustment, and is, for example, a display region of the article if the image was used to sell the article. That is, the color of the article is required to reproduce the color more strictly and to match the color of the actual article with the color of the article displayed as an image. Therefore, it is likely to be the target of the area for color adjustment. On the other hand, the background area of the article is less likely to be requested in this way and is less likely to be the target of the area for color adjustment. The region of interest can be designated by the user as user instruction information, for example.
In this case, the color data extraction unit 14 has the first color data which is the image data of the original image in the area where the article is displayed and the second color data which is the image data after color adjustment in the same area (same pixel position). Extract with color data. At this time, the first color data and the second color data are paired and extracted in a one-to-one correspondence.

3A to 3B are diagrams showing an example of a pair of the first color data and the second color data.
Here, FIG. 3A shows an example of the original image before color adjustment and the first color data which is the image data thereof. Here, the original image is an image of a blouse, and the color data in the small regions shown in 1 to 5 are shown as RGBa1 to RGBa5. In this case, the blouse is a single blue color, and RGBa1 to RGBa5 are all RGB data representing blue.
Further, FIG. 3B shows an example of the image after color adjustment and the second color data which is the image data thereof. Here, the color data in the small regions shown in 1 to 5 similar to FIG. 3A are shown as RGBb1 to RGBb5.

In order to extract the pair of the first color data and the second color data, for example, the following methods (1) to (3) can be used.

(1) The color space is divided by quadrants and hues, and a pair of the first color data and the second color data is extracted in the separated color area.
FIG. 4A is a diagram showing a case where the color space is divided into quadrants and a pair of the first color data and the second color data is extracted from the divided quadrants.
In the example shown in FIG. 4 (a), the L ^* a ^* b ^* color space is divided by the a ^* axis and the b ^* axis, whereby the L ^* a ^* b ^* color space is divided into 4 of the first to fourth quadrants. An example of dividing by two quadrants is shown. And here, the case where the color data indicated by the four black dots in the fourth quadrant is extracted is shown. In this case, the separated color regions are each of the four quadrants, and the color data is extracted from one of the color regions.

Further, FIG. 4B is a diagram showing a case where the color space is divided by hue and a pair of the first color data and the second color data is extracted from the divided color regions. It is assumed that the area inside the dotted line in FIG. 4B is the color gamut of the display device 20.
In the example shown in FIG. 4 (b), an example in which the L ^* a ^* b ^* color space is divided into six regions 1 to 6 for each hue is shown. And here, the case where the color data indicated by the five black dots in the area 3 and the area 5 is extracted is shown. In this case, there are six separated color regions, and color data is extracted from two of the color regions. That is, color data is extracted from a plurality of color regions.

(2) The color range is divided by lightness and saturation, and a pair of the first color data and the second color data is extracted in the separated color area.
FIG. 4C is a diagram showing a case where the color range is divided by lightness and saturation and a pair of the first color data and the second color data is extracted from the divided color areas. Note that FIG. 4C is a cross-sectional view of FIG. 4B cut perpendicular to the paper surface, and it is assumed that the area inside the dotted line is the color gamut of the display device 20.
In the example shown in FIG. 4C, an example in which the L ^* a ^* b ^* color space is divided into four areas 1 to 4 for each lightness and saturation is shown. Here, the case where the color data indicated by the four black dots in each of the regions 1 to 4 is extracted is shown. In this case, there are four separated color regions, and color data is extracted from all the color regions. That is, color data is extracted from a plurality of color regions. Note that the color data may be extracted from any one of the color regions as described with reference to FIG. 4A.

(3) A pair of the first color data and the second color data is extracted within the area in the image or the range on the color space specified by the user.
In this case, the user specifies, for example, the display area of the blouse of FIG. 3 as the area in the image. It also specifies the blue area on the color space.

Further, it is preferable that the pair of the first color data and the second color data is extracted within a predetermined small area. For example, the selected pixel is centered and one pixel around it is set as a small area. In this case, the small area is composed of 3 pixels × 3 pixels = 9 pixels. Then, the representative color is determined from the average value, the mode value, the peak of the histogram, etc. of the 9 pixels, and is used as the first color data and the second color data. By setting the representative color as the first color data and the second color data, it is possible to reduce the variation in the color adjustment at the time of the color adjustment using the conversion relationship described later.

FIG. 5 is a diagram showing an example when a histogram is created for the pixels in the small area.
In the figure, the horizontal axis represents RGB data or L ^* a ^* b ^* data, and the vertical axis represents the number of pixels (shown as Count).
At this time, when there are a plurality of histogram peaks as shown in the circle of FIG. 5, a plurality of sets of the first color data and the second color data may be determined as representative colors from one small area. .. At this time, as a reference for the representative color, for example, the number of pixels of the pixel having the pixel value having a peak may be 20% or more of the total number of pixels in this small region.

When color data is extracted from a plurality of color regions, color data having a hue, saturation, or lightness that are significantly different from each other may be extracted.
FIG. 6 is a diagram showing a case where color data having different brightness is extracted.
In the illustrated example, the original image is an image including a white blouse and a black skirt, and the case where the color data in the small region shown by 1 to 8 in the original image is extracted is shown. In this case, the pair of the first color data and the second color data is extracted from the small area shown by 1 to 4 from the white blouse, and the pair from the small area shown by 5 to 8 is extracted from the black skirt. The case where the pair of the 1st color data and the 2nd color data is extracted is shown. That is, color data having greatly different brightness is extracted. In this case, both the display area of the white blouse and the display area of the black skirt are the areas of interest, and it is necessary to extract the pair of the first color data and the second color data from both. Is.

In this case, it is preferable to determine the number of color data to be extracted according to the size of the region of interest (area, number of pixels, occupancy rate). In the case of FIG. 6, since the sizes of the areas of interest of both the white blouse and the black skirt are almost the same, the same number of color data are extracted. That is, the number of color data to be extracted is 1: 1 for each region of interest. If the area of interest of the white blouse and the area of interest of the black skirt are 2: 1, the number of color data to be extracted is 2: 1.

The color conversion model creation unit 15 creates a color conversion model representing the relationship between the first color data and the second color data extracted by the color data extraction unit 14.

FIG. 7 is a diagram showing an example of a color conversion model.
Here, the horizontal axis represents the first color data which is the color data before the color adjustment, and the vertical axis represents the second color data which is the color data after the color adjustment. The first color data and the second color data are RGB data, and in the figure, the first color data is shown as RGBa, and the second color data is shown as RGBb.

The black circle is a plot of the first color data and the second color data extracted by the color data extraction unit 14, and here, the first color data extracted by the color data extraction unit 14 and the first color data. It shows that there were 12 pairs with the second color data.

The solid line is the relationship between the first color data and the second color data, and represents the color conversion model created by the color conversion model creation unit 15. That is, it can be said that the color conversion model is a function expressing the relationship between the first color data and the second color data. If this function is f, it can also be expressed as RGBb = f (RGBa). This color conversion model can be created by a known method. However, it is preferable to use a method having high fitting performance for non-linear characteristics such as a weighted regression model or a neural network. However, the characteristic is not limited to the non-linear characteristic, and the linear characteristic using the Property model may be used.

Further, it is preferable that the color conversion model creating unit 15 creates a color conversion model so that the relationship between the first color data and the second color data becomes a non-linear monotonic increase function.
8 (a) to 8 (b) show the case where the color conversion model is created so that the relationship between the first color data and the second color data becomes a monotonous increase function, and the case where the colors do not become a monotonous increase function. It is the figure which compared the case where the conversion model was created.
In the figure, the curve drawn by the solid line represents the color conversion model. The thick line in this color conversion model is a case where the color conversion model is created so that the relationship between the first color data and the second color data becomes a monotonically increasing function, and the same color conversion as in FIG. It is a model. The thin line is a case where the color conversion model is created so that the relationship between the first color data and the second color data does not become a monotonically increasing function. Here, the monotonous increase function means that the slope of the tangent line of the solid line representing the color conversion model is 0 or more, and there may be a place where the slope is 0. That is, it is a monotonous increasing function in a broad sense.

In the color conversion model shown by the thick line, the slope of the tangent line is not less than 0 (negative) and is always 0 or more.
On the other hand, in the color conversion model shown by the thin line, the slope of the tangent line may be less than 0 (negative). That is, there is a place where RGBb decreases as RGBa increases. When color adjustment is performed using the conversion relationship created by using this color conversion model, a gradation step may occur in the image after color adjustment. If a color conversion model is created so that the relationship between the first color data and the second color data becomes a monotonically increasing function, it is possible to reduce the occurrence of gradation steps and reduce the variation in color adjustment.

In order for the color conversion model creation unit 15 to create a color conversion model as shown by a thick line, for example, the pair of the first color data and the second color data shown as Pr1, Pr2 and Pr3 is removed. Can be considered.

Further, the color conversion model creation unit 15 may create a color conversion model by setting weights for the pair of the first color data and the second color data.
In this case, the color conversion model creation unit 15 can create a color conversion model by setting weights so that the relationship between the first color data and the second color data becomes a monotonically increasing function. That is, a small weight is set for the pair of the first color data and the second color data shown as Pr1 and Pr2, and a large weight is set for the other pairs. For example, when the pair of the first color data and the second color data has a large difference from the average characteristic difference direction, a small weight is set and the difference from the average characteristic difference direction is small. Sets a large weight. The weight may be set to 0 for the pair of the first color data and the second color data shown as Pr1, Pr2, and Pr3. In this case, the result is the same as removing them.

The usage of weights is not limited to this. For example, the color conversion model creation unit 15 may set the weight according to the size of the region of interest.
For example, in the example shown in FIG. 6, the number of color data to be extracted is determined according to the size of the region of interest, but the number of color data to be extracted is the same for each region of interest, and the size of the region of interest is large. Correspondingly, a larger weight may be set as the region of interest is larger, and a smaller weight may be set as the region of interest is smaller. If the area of interest of the white blouse and the area of interest of the black skirt are 2: 1, the weight is also 2: 1.

Returning to FIG. 2, the conversion relationship creation unit 16 creates a conversion relationship that reproduces the color adjustment of the original image based on the color conversion model created by the color conversion model creation unit 15. This conversion relationship reproduces the result of the color adjustment performed by the user on the original image. That is, when the color adjustment is performed on the original image using this conversion relationship, the same color adjustment as the color adjustment previously performed by the user can be performed again to obtain the image after the color adjustment.

When the conversion relationship is a three-dimensional LUT, representative pixel values are selected for each of R, G, and B. For example, when each of the data of R, G, and B is represented by an 8-bit gradation, the pixel value is an integer of 0 to 255, and this is divided into, for example, eight. Then, RGB data that can be represented by each pixel value divided into 8 is used as a grid point (so-called 9 grid points). In this case, grid ^points is 9 3 = 729. Then, for each of the grid points, the correspondence relationship between the first color data and the second color data is calculated based on the color conversion model. A three-dimensional LUT expresses this correspondence by a LUT (Look Up Table). As a result, 3-dimensional LUT, the input values for each lattice point _{(R a, G a, B} a) - the output value _{(R b, G b, B} b) are described as the corresponding relationship.

At this time, it is preferable that the conversion relationship creating unit 16 creates a conversion relationship in which the color included in the region of interest is adjusted in color but the other colors are not adjusted in color. For example, in the case of the image shown in FIG. 3, the conversion relationship creating unit 16 creates a conversion relationship in which the blue color region is color-adjusted but the other colors are not color-adjusted. Further, in the case of the image shown in FIG. 6, the conversion relationship creating unit 16 creates a conversion relationship in which the white and black color regions are color-adjusted but the other colors are not color-adjusted. When the conversion relationship is three-dimensional LUT, the 3-dimensional LUT, the input values of the grid points close to the color included in the target area _{(R a, G a, B} a) and the output value _{(R b, G b,} B Although the value different from _b), the input values of the other grid points _{(R a, G a, B} a) and the output value _{(R b,} G _b, the same value and the _{B b).}

The output unit 17 outputs image data after color adjustment and conversion-related data. The image data after color adjustment is output to the display device 20, and the display device 20 displays the image after color adjustment based on the image data. Further, the conversion-related data is stored in, for example, the image processing device 10, and the color adjustment can be performed using this conversion-related data. Further, the image is output to an external device different from the image processing device 10, and the external device can perform color adjustment using this conversion relationship.

Next, the operation of the image processing device 10 will be described.
FIG. 9 is a flowchart illustrating the operation of the image processing device 10. The operation of the image processing device 10 described below can also be regarded as an image processing method performed by the image processing device 10.
First, the image data acquisition unit 11 acquires the image data of the original image (step 101).
The image data of the original image is output from the output unit 17 to the display device 20 (step 102). As a result, the original image is displayed on the display screen 21 of the display device 20.

Next, the user instruction receiving unit 12 receives the area in the image for which color adjustment is performed and the color adjustment instruction as user instruction information (step 103).
Then, the color adjustment unit 13 adjusts the color of the image data of the original image based on the user instruction information (step 104).
The color-adjusted image data is output from the output unit 17 to the display device 20 (step 105). As a result, the color-adjusted image is displayed on the display screen 21 of the display device 20.

Next, the color data extraction unit 14 obtains a plurality of first color data which are color data in the region of interest of the original image and a plurality of second color data after color adjustment is performed on the first color data. (Step 106: color data extraction step). This can be done by the method described in FIGS. 3-6.

Then, the color conversion model creation unit 15 creates a color conversion model that non-linearly represents the relationship between the first color data and the second color data extracted by the color data extraction unit 14 (step 107: color conversion model creation). Process). This can be done by the method described with reference to FIGS. 7-8.

Further, the conversion relationship creation unit 16 creates a conversion relationship that reproduces the color adjustment of the original image performed by the color adjustment unit 13 based on the color conversion model created by the color conversion model creation unit 15 (step 108: conversion relationship). Creation process). It is created, for example, in a three-dimensional LUT, as described above.
Then, the output unit 17 outputs conversion-related data (step 109).

In the present embodiment, it is preferable that the conversion relationship creating unit 16 creates a conversion relationship for each group predetermined for the original image.
This group is divided into those that are expected to have similar color characteristics such as spectral reflectance of articles.
For example, groups are divided into categories of goods. Specifically, the articles are classified according to whether they are clothing, printed matter, furniture, stationery, and the type of coloring material. Then, for articles having the same color characteristics of the articles, color adjustment is performed with the same conversion relationship, and for articles with different color characteristics of the articles, color adjustment is performed with different conversion relationships. This improves the accuracy of color adjustment. Further, for the same reason, the group may be divided according to the main color of the article and the pattern of the article. Further, the group may be further divided according to shooting conditions such as the presence / absence of a flash, illumination light, exposure, background, shooting mode, camera type (manufacturer, model number, etc.) and setting conditions of the camera 40. At this time, as the shooting conditions, for example, when the item is clothing, whether it is worn by a person, worn by a mannequin, or taken by placing it on a table (such as by placing it on a table). Conditions such as may be added. Further, the group may be divided according to the user (retoucher) who performs the color adjustment.

Further, at this time, it is preferable that the conversion relationship creation unit 16 creates a conversion relationship by using a plurality of original images for each predetermined group. That is, if the results of color adjustment using a plurality of original images are obtained, it is possible to create conversion relationships corresponding to various colors for each group. Further, even when an original image having only similar colors is used, a plurality of color adjustment results can be obtained for this color, so that the accuracy of the conversion relationship for performing the color adjustment is similarly improved.

Further, in the present embodiment, the conversion relationship creating unit 16 creates a conversion relationship in which the color included in the region of interest is color-adjusted, but the other colors are not color-adjusted. This makes it possible to improve the accuracy of color adjustment. If a conversion relationship for performing color adjustment for all color areas is created, the accuracy of color adjustment tends to decrease. In the present embodiment, the accuracy of color adjustment is improved as compared with the case where the conversion relationship is created by one type of conventional image. That is, conventionally, since a conversion relationship for performing color adjustment for all color regions with one type of image has been created, the accuracy of color adjustment can be improved in the present embodiment as compared with this.

As described above, in the present embodiment, by using the conversion relationship created for each group, even if the color characteristics of the object to be photographed are different, the conversion relationship of the group that matches the conversion relationship is selected to adjust the color with higher accuracy. Can be done.

In addition, by preparing this conversion relationship in advance and using the prepared conversion relationship, it is difficult for differences in color adjustment due to differences in the skill and sensitivity of the user (retoucher) who performs color adjustment to occur, and in this respect as well, color adjustment Accuracy is improved.

In the above-mentioned example, an image taken by the camera 40 is used as the original image, but the original image is not particularly limited. For example, an image read by a scanner may be used as the original image. Further, commercially available image data or image data distributed on the Internet or the like may be used as the original image as it is.
Further, in the above-mentioned example, the spectral reflectance of the article is illustrated as a color characteristic, but the characteristic is not limited to this as long as it is a characteristic that affects when photographing or the like. For example, the uneven structure on the surface of the article, the presence or absence of gloss, the texture, and the like can be mentioned.

<Hardware configuration example of image processing device>
Next, the hardware configuration of the image processing device 10 will be described.
FIG. 10 is a diagram showing a hardware configuration of the image processing device 10.
The image processing device 10 is realized by a personal computer or the like as described above. As shown in the figure, the image processing device 10 includes a CPU (Central Processing Unit) 91 as a calculation means, a main memory 92 as a storage means, and an HDD (Hard Disk Drive) 93. Here, the CPU 91 executes various programs such as an OS (Operating System) and application software. The main memory 92 is a storage area for storing various programs and data used for executing the various programs, and the HDD 93 is a storage area for storing input data for various programs and output data from various programs.
Further, the image processing device 10 includes a communication interface (communication I / F) 94 for communicating with the outside.

<Program description>
The processing performed by the image processing apparatus 10 according to the present embodiment described above is prepared as, for example, a program such as application software.

Therefore, in the processing performed by the image processing apparatus 10 in the present embodiment, a plurality of first color data, which is color data in the region of interest of the first image representing the color correction target, and the first image are stored in the computer. A color data extraction function that extracts a plurality of second color data pairs with the first color data corresponding to a region of interest in a second image different from the above, and the extracted first color data and the second A color conversion model creation function that creates a color conversion model that represents the relationship with the color data of the first image, and a conversion relationship creation function that creates a conversion relationship that reproduces the color adjustment of the first image based on the color conversion model. It can be regarded as a program to be realized.

The program that realizes the present embodiment can be provided not only by communication means but also by storing it in a recording medium such as a CD-ROM.

Although the present embodiment has been described above, the technical scope of the present invention is not limited to the scope described in the above embodiment. It is clear from the description of the scope of claims that the above-described embodiment with various modifications or improvements is also included in the technical scope of the present invention.

1 ... Image processing system, 10 ... Image processing device, 11 ... Image data acquisition unit, 12 ... User instruction reception unit, 13 ... Color adjustment unit, 14 ... Color data extraction unit, 15 ... Color conversion model creation unit, 16 ... Conversion Relationship creation unit, 17 ... Output unit

Claims (7)

Corresponding to a plurality of first color data which are color data in the attention region of the first image representing the color correction target and the attention region in the second image different from the first image, the first A color data extraction unit that extracts one color data and a plurality of second color data pairs,
A color conversion model creation unit that creates a color conversion model that represents the relationship between the extracted first color data and the second color data, and
A conversion relationship creating unit that creates a conversion relationship that reproduces the color adjustment of the first image based on the color conversion model, and
With
The color conversion model creating unit is an image processing apparatus characterized in that the color conversion model is created so that the relationship between the first color data and the second color data is a non-linear monotonic increase function. Corresponding to a plurality of first color data which are color data in the attention region of the first image representing the color correction target and the attention region in the second image different from the first image, the first A color data extraction unit that extracts one color data and a plurality of second color data pairs,
A color conversion model creation unit that creates a color conversion model that represents the relationship between the extracted first color data and the second color data, and
A conversion relationship creating unit that creates a conversion relationship that reproduces the color adjustment of the first image based on the color conversion model, and
With
The image processing device is characterized in that the color conversion model creating unit creates the color conversion model by setting weights for a pair of the first color data and the second color data. The color conversion model creating unit is characterized in that the color conversion model is created by setting the weights so that the relationship between the first color data and the second color data becomes a monotonically increasing function. The image processing apparatus according to claim 2 . The image processing apparatus according to claim 2 or 3 , wherein the color conversion model creating unit sets the weight according to the size of the region of interest. Corresponding to a plurality of first color data which are color data in the attention region of the first image representing the color correction target and the attention region in the second image different from the first image, the first A color data extraction step for extracting one color data and a plurality of second color data pairs,
A color conversion model creation step of creating a color conversion model representing the relationship between the extracted first color data and the second color data, and
A conversion relationship creation process for creating a conversion relationship that reproduces the color adjustment of the first image based on the color conversion model, and
Including
In the color conversion model creation step, the color conversion model is created so that the relationship between the first color data and the second color data is a non-linear monotonic increase function, or the first color data and the first color data are combined. An image processing method in which a weight is set for a pair with the second color data to create the color conversion model, or at least one of them . A shooting device that shoots the subject and
An image processing device that adjusts the color of the first image shot by the shooting device, and
With
The image processing device is
Corresponding to the plurality of first color data which is the color data in the attention region of the first image and the attention region in the second image different from the first image, the first color data A color data extraction unit that extracts a plurality of paired second color data,
A color conversion model creation unit that creates a color conversion model that represents the relationship between the extracted first color data and the second color data, and
A conversion relationship creating unit that creates a conversion relationship that reproduces the color adjustment of the first image based on the color conversion model, and
With
The color conversion model creating unit creates the color conversion model so that the relationship between the first color data and the second color data is a non-linear monotonic increase function, or the first color data and the first color data. An image processing system that creates the color conversion model by setting weights for a pair with the second color data, or at least one of them . On the computer
Corresponding to a plurality of first color data which are color data in the attention region of the first image representing the color correction target and the attention region in the second image different from the first image, the first A color data extraction function that extracts one color data and a plurality of second color data pairs,
A color conversion model creation function that creates a color conversion model that represents the relationship between the extracted first color data and the second color data, and
A conversion relationship creation function that creates a conversion relationship that reproduces the color adjustment of the first image based on the color conversion model, and
Realized,
The color conversion model creation function creates the color conversion model so that the relationship between the first color data and the second color data is a non-linear monotonic increase function, or the first color data and the first color data. A program that creates the color conversion model by setting weights for a pair with the second color data, or at least one of them .

Images