JP2019208125A - Image editing device, image editing method, and image editing program - Google Patents

Abstract

To provide an image editing device, an image editing method, and an image editing program capable of more flexibly editing a color of an image.SOLUTION: The image editing device includes: a decomposition unit for decomposing an image obtained by photographing an object into a reflectance image representing a reflectance of the object and a shadow image representing a shadow of the object; a selection unit for selecting a plurality of representative colors on the basis of the number of pixels colors included in the reflectance image or the number of pixels of a similar color of the colors; a first generation unit for generating a first change image in which one of the colors included in the reflectance image is changed from the reflectance image, on the basis of the change of any of the representative colors to a different color; and a second generation unit for generating a composite image by composing the first change image and the shadow image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image editing apparatus, an image editing method, and an image editing program.

  2. Description of the Related Art Conventionally, an image editing apparatus that accepts designation of an image area and edits the image by changing the color of the designated area has been used. The area can be freely specified, and the color of the specified area can be freely changed.

  Non-Patent Document 1 below discloses a technique for editing a color of an entire image by extracting a “palette color” that is a color representing an image by applying the k-means method and changing the palette color. ing. Here, the palette color is changed so as not to change the order of brightness of the palette colors. As a result, for example, when the color of the shadow portion is changed to a bright color, the shadow portion is prevented from becoming brighter than the surroundings and resulting in an unnatural image.

Huiwen Chang, Ohad Fried, Yiming Liu, Stephen DiVerdi, and Adam Finkel-stein, "Palette-based Photo Recoloring," ACM Transactions on Graphics (Proc. SIGGRAPH) 34 (4), July 2015.

  By the technique described in Non-Patent Document 1, it is not necessary to specify an image area, and a color representing the image is extracted, so that even a person unfamiliar with image editing can easily edit the image. become able to. However, as in the technique described in Non-Patent Document 1, if there is a limitation in the order of the brightness of the palette colors, when the dark color of the object itself instead of the shadow is changed to a bright color, the bright color from before the change is changed. It becomes brighter and whiteout may occur in the image. Conversely, when the object itself is changed from a bright color to a dark color, the dark color before the change becomes darker, and the image may be blacked out. For this reason, the technique described in Non-Patent Document 1 has substantially limited editing of image colors.

  Therefore, the present invention provides an image editing apparatus, an image editing method, and an image editing program that can edit the color of an image more flexibly.

  An image editing apparatus according to an aspect of the present invention includes an image obtained by photographing an object, a decomposition unit that decomposes the image into a reflectance image that represents the reflectance of the object and a shadow image that represents the shadow of the object, and the reflectance image. On the basis of the number of pixels of a plurality of colors or the number of pixels of similar colors of a plurality of colors, based on the selection unit for selecting a plurality of representative colors and changing one of the plurality of representative colors to a different color, From the reflectance image, a first generation unit that generates a first modified image in which any one of a plurality of colors included in the reflectance image is modified, a first modified image, and a shadow image are combined to generate a composite image A second generating unit.

  According to this aspect, the image is decomposed into the reflectance image and the shadow image, the first modified image is generated based on the change in the representative color of the reflectance image, and the first modified image and the shadow image are synthesized to form the composite image. Thus, the color of the image can be edited independently of the shadow, and the color of the image can be edited more flexibly. For example, even if the bright color before the change is changed to brighter, it is possible to prevent whiteout from occurring in the image. Further, even if the bright color before the change is changed to dark, it is possible to prevent blackout from occurring in the image.

  In the above aspect, a determination unit that determines the number of the plurality of representative colors so that the plurality of colors included in the reflectance image is represented by the plurality of representative colors while suppressing an increase in the number of the plurality of representative colors. Further, it may be provided.

  When the number of multiple representative colors in the reflectance image is increased, the multiple representative colors can better represent the multiple colors included in the reflectance image, resulting in fine adjustments. However, it becomes difficult to balance the colors of the images, and editing time is required. On the other hand, if the number of the plurality of representative colors of the reflectance image is reduced, the editing time of the color of the image is shortened. However, it is difficult to better represent the plurality of colors included in the reflectance image by the plurality of representative colors. Become. According to this aspect, the number of the representative colors is appropriately determined by the determining unit, and even an inexperienced person can easily edit the image.

  In the above aspect, the first generation unit may generate the first changed image from the reflectance image based on changing the plurality of representative colors independently of each other.

  According to this aspect, by generating the first changed image based on the plurality of representative colors of the reflectance image being changed independently of each other, for example, the representative colors are set so as to maintain the order of the luminances of the plurality of representative colors. Compared with the case of changing, the color of the image can be edited more flexibly.

  In the above aspect, the first generation unit generates a second change image in which the color of the shadow image is changed from the shadow image based on the change in the color of the shadow image, and the second generation unit includes the first change image and The second changed image may be combined to generate a combined image.

  According to this aspect, the second changed image is generated based on the change in the color of the shadow image, and the combined image is generated by combining the first changed image and the second changed image, so that the color of a part of the image It is possible to reproduce the change in lighting by changing the brightness and color of the entire image.

  The said aspect WHEREIN: You may further provide the display part which displays a reflectance image, a shadow image, and an image, a 1st change image, a 2nd change image, and a synthesized image so that comparison is possible.

  According to this aspect, it is possible to grasp at a glance what kind of change has been made by displaying the images before and after the color change in a comparable manner, and assisting in the smooth editing of the color of the image. Can do.

  In the above aspect, the display unit updates and displays the first modified image and the composite image when the first modified image is generated by at least the first generating unit, and the second modified image is generated. In this case, the second modified image and the composite image may be updated and displayed.

  According to this aspect, when the representative color of the reflectance image is changed, the first changed image and the composite image are updated and displayed in conjunction with the change, and how the image is changed by changing the representative color. You can see at a glance whether it changes. In addition, when the color of the shadow image is changed, the second changed image and the composite image are updated and displayed in conjunction with the change, and it is possible to see at a glance how the image changes due to the change of the shadow image. I can grasp. Thereby, it can assist so that the edit of the color of an image may become smoother.

  An image editing method according to another aspect of the present invention includes an image obtained by photographing an object, decomposed into a reflectance image representing the reflectance of the object and a shadow image representing the shadow of the object, and included in the reflectance image. Based on the selection of a plurality of representative colors based on the number of pixels of a plurality of colors or the number of similar colors of a plurality of colors, and the reflection based on changing one of the plurality of representative colors to a different color. Generating a first changed image in which any one of a plurality of colors included in the reflectance image is changed from the rate image, synthesizing the first changed image and the shadow image, and generating a synthesized image; including.

  According to this aspect, the image is decomposed into the reflectance image and the shadow image, the first modified image is generated based on the change in the representative color of the reflectance image, and the first modified image and the shadow image are synthesized to form the composite image. Thus, the color of the image can be edited independently of the shadow, and the color of the image can be edited more flexibly.

  An image editing program according to another aspect of the present invention provides a computer provided in an image editing apparatus to convert an image obtained by capturing an object into a reflectance image representing the reflectance of the object and a shadow image representing the shadow of the object. A separation unit for separation, a selection unit for selecting a plurality of representative colors based on the number of pixels of a plurality of colors included in the reflectance image, or the number of similar colors of the plurality of colors, or a plurality of representative colors are different. A first generation unit that generates a first changed image in which any one of a plurality of colors included in the reflectance image is changed from the reflectance image based on the change to the color, a first changed image, and a shadow image And function as a second generation unit that generates a composite image.

  According to this aspect, the image is decomposed into the reflectance image and the shadow image, the first modified image is generated based on the change in the representative color of the reflectance image, and the first modified image and the shadow image are synthesized to form the composite image. Thus, the color of the image can be edited independently of the shadow, and the color of the image can be edited more flexibly.

  According to the present invention, an image editing apparatus, an image editing method, and an image editing program capable of editing the color of an image more flexibly are provided.

It is a figure which shows the functional block of the image editing apparatus which concerns on embodiment of this invention. It is a figure which shows the physical structure of the image editing apparatus which concerns on this embodiment. It is an example of the image edited by the image editing apparatus which concerns on embodiment of this invention. It is an example of the reflectance image decomposed | disassembled from the image by the image editing apparatus which concerns on embodiment of this invention. It is an example of the shadow image decomposed | disassembled from the image by the image editing apparatus which concerns on embodiment of this invention. It is an example of the 1st change image produced | generated by the image editing apparatus which concerns on embodiment of this invention. It is an example of the 2nd change image produced | generated by the image editing apparatus which concerns on embodiment of this invention. It is an example of the synthesized image produced | generated by the image editing apparatus which concerns on embodiment of this invention. It is a flowchart of the 1st example of the image editing process performed by the image editing apparatus which concerns on embodiment of this invention. It is an example of the image edit screen displayed by the image editing apparatus which concerns on embodiment of this invention. It is a flowchart of the 2nd example of the image editing process performed by the image editing apparatus which concerns on embodiment of this invention. It is an example of the image edited by the image editing apparatus of a prior art example. It is an example of the image edited with the image editing apparatus of the prior art example.

  Hereinafter, an embodiment according to an aspect of the present invention (hereinafter referred to as “the present embodiment”) will be described with reference to the drawings. In addition, in each figure, what attached | subjected the same code | symbol has the same or similar structure.

  First, the editing of the color of an image by the conventional image editing apparatus and the problem will be described. FIG. 12 shows an example of an image IMG1 edited by the conventional image editing apparatus. The image IMG1 is an image obtained by photographing objects such as an ocher bed, two brown cushions placed on the bed, a gray carpet, and a vermilion sofa.

  The image editing apparatus of the conventional example weights the color of the pixel of the image IMG1 according to the number of times the color is used and plots it in a color space such as Lab space, and the plotted distribution is, for example, 16 × 16 × 16. To discretize. Then, the point having the largest weight is selected as the first representative color. Then, after performing a process of reducing the weight for a point that is close to the first representative color in the color space, the point having the second largest weight is selected as the second representative color. In the conventional image editing apparatus, five representative colors are selected in this way.

  The representative colors extracted from the image IMG1 by the image editing apparatus of the conventional example are ocher color C1, gray C2, dark gray C3, vermilion C4, and brown C5 in descending order of luminance. Here, the ocher C1 is the color of the bed, the gray C2 is the color of the carpet, the vermillion C4 is the color of the sofa, and the brown C5 is the color of the cushion.

  Here, dark gray C3 is the color of the part of the carpet where the shadows overlap. As described above, in the image editing apparatus of the conventional example, the bright portion color and the dark portion color of the same target may be selected as different representative colors. Therefore, when lightening the dark gray C3, it is necessary to lighten the gray C2 and the ocher C1 at the same time. Conversely, when darkening the gray C2, the dark gray C3, vermilion C4, and brown C5 are not darkened any more. And inconvenience arises.

  The image editing apparatus of the conventional example accepts a change of any one of the representative colors of ocher C1, gray C2, dark gray C3, vermillion C4, and brown C5, and keeps the order of luminance of these colors so that all Change the representative color. Here, an example in which vermilion C4 is changed to white will be described.

  FIG. 13 shows an example of an image IMG11 edited by a conventional image editing apparatus. When receiving an instruction to change the vermilion C4 to the white C14, the image editing apparatus of the conventional example has higher luminance than the white C14 for the ocher C1, gray C2, and dark gray C3, which are brighter than the vermilion C4. Thus, the ocher color C1 is changed to the bright yellow color C11, the gray C2 is changed to the bright cream color C12, and the dark gray C3 is changed to the cream color C13. The brown C5 is not changed as it is.

  In the image IMG11 edited by the image editing apparatus of the conventional example, the ocher color C1, gray C2, and dark gray C3 selected as the representative colors are changed to yellow C11, light cream color C12, and cream color C13. Flying has occurred. In addition, by changing the vermilion color C4 to white C14, a part of the color of the sofa is changed to white C14, but the shadow portion remains dark vermilion close to brown C5.

  As described above, the image editing apparatus of the conventional example changes the representative color so that the order of the luminances of the plurality of representative colors is maintained. A color with high brightness will become brighter, and whiteout may occur in the image. In addition, when a bright color is changed to a dark color, a dark color before the change is further darkened, and blackout may occur in the image. For this reason, the image editing apparatus of the conventional example is substantially limited in editing the color of the image.

  FIG. 1 is a diagram showing functional blocks of an image editing apparatus 10 according to an embodiment of the present invention. The image editing apparatus 10 includes a decomposition unit 11, a selection unit 12, a first generation unit 13, a second generation unit 14, a determination unit 15, a storage unit 16, an input unit 10e, and a display unit 10f.

The decomposition unit 11 decomposes an image obtained by capturing an object into a reflectance image that represents the reflectance of the object and a shadow image that represents the shadow of the object. The decomposition unit 11 may decompose the image into a reflectance image and a shadow image based on an existing algorithm. The decomposition unit 11 is, for example, S.I. Bell, two others, “Intrinsic Images in the Wild”, using the techniques described in ACM Transactions on Graphics (SIGGRAPH 2014), splitting the image into a reflectance image and a shading image You can do it. The decomposition of the image by the decomposition unit 11 may be performed for each pixel having an RGB value, and the RGB value I _i (i represents any of R, G, and B) of the image is the RGB value R of the reflectance image. and _i, may be decomposed into a product I _i = R _i × S _i between the RGB value S _i of the shaded image. However, the shaded image may be a grayscale image.

  The selection unit 12 selects a plurality of representative colors based on the number of pixels of a plurality of colors included in the reflectance image or the number of pixels of a similar color of the plurality of colors included in the reflectance image. The selection unit 12 weights the color of the pixel of the reflectance image according to the number of pixels in which the color is used and plots the color in a color space such as Lab space, and the plotted distribution is, for example, 16 × 16 × 16. It is possible to discretize and select the point with the largest weight as the first representative color. Further, the “similar color” may be a color in the vicinity of a plurality of colors included in the reflectance image in the color space, and the selection unit 12 averages the distribution that is discretized into 16 × 16 × 16, The “similar color” having the largest weight after averaging may be selected as the first representative color. In addition, after performing the process of reducing the weight for a point that is close to the first representative color in the color space, the point having the second largest weight may be selected as the second representative color. The number of representative colors selected by the selection unit 12 may be determined by the determination unit 15.

  The first generation unit 13 generates a first changed image obtained by changing any of the plurality of colors included in the reflectance image from the reflectance image based on changing any of the plurality of representative colors to a different color. Generate. For example, the first generation unit 13 receives an instruction to change any of the plurality of representative colors to a different color by the input unit 10e, and changes the instructed color among the plurality of colors included in the reflectance image. As described above, the first modified image may be generated from the reflectance image.

The second generation unit 14 combines the first changed image and the shadow image to generate a combined image. The second generation unit 14 calculates the RGB value I1 of the composite image by the product of the RGB value R1 _i (i represents any of R, G, and B) of the first changed image and the RGB value S _i of the shadow image. _i = R1 _i × S _i may be calculated.

  In this way, the image is decomposed into a reflectance image and a shadow image, a first modified image is generated based on a change in the representative color of the reflectance image, and a composite image is generated by combining the first modified image and the shadow image. Thus, the color of the object in the image can be edited independently of the shadow, and the color of the image can be edited more flexibly. For example, even if the color that was dark before the change is changed to be brighter, whiteout is prevented from occurring in the image. Further, even if the bright color before the change is changed to dark, it is possible to prevent blackout from occurring in the image.

  The determination unit 15 determines the number of representative colors such that the plurality of colors included in the reflectance image are represented by the plurality of representative colors while suppressing an increase in the number of representative colors. For example, the determination unit 15 may determine the number of representative colors so that the BIC (Bayesian Information Criterion) is minimized. That is, the number of a plurality of representative colors is represented as k, the number of pixels of the reflectance image × 3 (three colors of RGB) is represented as n, and the likelihood of a model representing the reflectance image with a plurality of representative colors is represented as L. At this time, the determination unit 15 may determine the number of representative colors such that BIC = k × ln (n) −2 × ln (L) is minimized.

  When the number of multiple representative colors in the reflectance image is increased, the multiple representative colors can better represent the multiple colors included in the reflectance image, resulting in fine adjustments. However, it becomes difficult to balance the colors of the images, and editing time is required. On the other hand, if the number of the plurality of representative colors of the reflectance image is reduced, the editing time of the color of the image is shortened. However, it is difficult to better represent the plurality of colors included in the reflectance image by the plurality of representative colors. Become. The determining unit 15 appropriately determines the number of representative colors, and even an unaccustomed person can easily edit an image.

  Note that the selection unit 12 and the determination unit 15 use a so-called x-means method to cluster a plurality of colors included in the reflectance image, determine a clustering number using a BIC, and select a plurality of representative colors. You can do it.

  The first generation unit 13 may generate the first changed image from the reflectance image based on changing the plurality of representative colors independently of each other. For example, the first generation unit 13 receives an instruction to change any one of the plurality of representative colors to a different color by the input unit 10e, and relates to the representative color related to the change among the plurality of colors included in the reflectance image. The first changed image may be generated from the reflectance image so that only the color is changed.

  Compared with the case where the representative color is changed to maintain the order of the luminance of the representative colors, for example, by generating the first changed image based on the fact that the representative colors of the reflectance image are changed independently of each other. Thus, the color of the image can be edited more flexibly.

  The first generation unit 13 may generate a second changed image in which at least one of hue, saturation, and brightness of the shadow image is changed from the shadow image based on the change of the color of the shadow image. Then, the second generation unit 14 may generate a combined image by combining the first changed image and the second changed image.

  A second changed image is generated based on the change in the color of the shaded image, and the first changed image and the second changed image are combined to generate a combined image, thereby not only changing the color of a part of the image. The lighting change can be reproduced by changing the brightness and color of the entire image.

  FIG. 2 is a diagram illustrating a physical configuration of the image editing apparatus 10 according to the present embodiment. The image editing apparatus 10 includes a CPU (Central Processing Unit) 10a corresponding to a calculation unit, a RAM (Random Access Memory) 10b corresponding to a storage unit, a ROM (Read only Memory) 10c corresponding to a storage unit, and a communication unit. 10d, an input unit 10e, a display unit 10f, and a measurement unit 10g. Each of these components is connected to each other via a bus so that data can be transmitted and received. In this example, the case where the image editing apparatus 10 is configured by one computer will be described. However, the image editing apparatus 10 may be realized by combining a plurality of computers. The configuration illustrated in FIG. 2 is an example, and the image editing apparatus 10 may have a configuration other than these, or may not have a part of these configurations.

  The CPU 10a is a control unit that performs control related to execution of a program stored in the RAM 10b or the ROM 10c, and calculates and processes data. The CPU 10a is an arithmetic unit that executes a program (an image editing program) that decomposes an image into a reflectance image and a shadow image, and generates a composite image based on color editing related to the reflectance image and the shadow image. The CPU 10a receives various data from the input unit 10e and the communication unit 10d, and displays a calculation result of the data on the display unit 10f or stores it in the RAM 10b or the ROM 10c.

  The RAM 10b can rewrite data in the storage unit, and may be composed of, for example, a semiconductor storage element. The RAM 10b may store data such as an image editing program executed by the CPU 10a, an image to be edited, and a generated composite image. These are examples, and the RAM 10b may store data other than these, or some of them may not be stored.

  The ROM 10c is capable of reading data out of the storage unit, and may be composed of, for example, a semiconductor storage element. The ROM 10c may store, for example, an image editing program and data that is not rewritten.

  The communication unit 10d is an interface that connects the image editing apparatus 10 to other devices. The communication unit 10d may be connected to a communication network such as the Internet.

  The input unit 10e receives data input from the user and may include, for example, a keyboard and a touch panel.

  The display unit 10f visually displays a calculation result by the CPU 10a, and may be configured by an LCD (Liquid Crystal Display), for example. The display unit 10f may display an image to be edited, a reflectance image and a shadow image decomposed from the image, and the generated first modified image, second modified image, and composite image.

  The image editing program may be provided by being stored in a computer-readable storage medium such as the RAM 10b or the ROM 10c, or may be provided via a communication network connected by the communication unit 10d. In the image editing apparatus 10, various operations described with reference to FIG. 1 are realized by the CPU 10 a executing the image editing program. In addition, these physical structures are illustrations, Comprising: It does not necessarily need to be an independent structure. For example, the image editing apparatus 10 may include an LSI (Large-Scale Integration) in which a CPU 10a, a RAM 10b, and a ROM 10c are integrated.

  FIG. 3 is an example of an image IMG1 edited by the image editing apparatus 10 according to the embodiment of the present invention. The image IMG1 is an image obtained by photographing objects such as an ocher bed, two brown cushions placed on the bed, a gray carpet, and a vermilion sofa. The image shown in the figure is the same as the image IMG1 edited by the conventional image editing apparatus shown in FIG. The decomposition unit 11 of the image editing apparatus 10 decomposes the image IMG1 into a reflectance image that represents the reflectance of the object and a shadow image that represents the shadow of the object.

  FIG. 4 is an example of the reflectance image IMG2 decomposed from the image IMG1 by the image editing apparatus 10 according to the embodiment of the present invention. The reflectance image IMG2 is an image representing the reflectance of an object such as an ocher bed, two brown cushions placed on the bed, a gray carpet, and a vermilion sofa, and does not show the shadow of the object. Different from the image IMG1.

  In this example, the selection unit 12 of the image editing apparatus 10 selects vermilion C21, ocher C22, and gray C23 as a plurality of representative colors of the reflectance image. In the reflectance image IMG2, an area in which three representative colors, vermilion C21, ocher C22, and gray C23 are used.

  In this example, the determination unit 15 determines that the number of representative colors is 3. The determination unit 15 tries whether or not the BIC decreases when, for example, brown is added as the fourth representative color in addition to the vermilion C21, the ocher color C22, and the gray C23. If representative colors are added and the BIC increases, the representative colors may be three. In this example, when brown is added as the fourth representative color, the BIC increases, so the representative colors are three colors of vermilion C21, ocher C22, and gray C23. If fine adjustment is required in editing the reflectance image, the representative color may be allowed to be increased by user input.

  FIG. 5 is an example of the shadow image IMG3 decomposed from the image IMG1 by the image editing apparatus 10 according to the embodiment of the present invention. The reflectance image IMG2 is a grayscale image, and is an image representing the shadow of a target such as a bed, two cushions placed on the bed, a carpet, and a sofa, and is different from the image IMG1 in that the color of the target is not represented. . The reflectance image IMG2 is an image in which the value representing saturation is 0 when the pixel value is represented in the HSV color space.

  FIG. 6 is an example of the first modified image IMG4 generated by the image editing apparatus 10 according to the embodiment of the present invention. In this example, the first generation unit 13 changes the reflectance image from the reflectance image IMG2 based on the fact that the vermilion color C21 among the plurality of representative colors C21, ocher C22, and gray C23 is changed to white C41. A first changed image IMG4 in which the color related to the vermilion C21 included in the IMG2 is changed is generated. The first generation unit 13 generates the first changed image IMG4 from the reflectance image IMG2 based on the fact that the plurality of representative colors are changed independently of each other, and is related to the ocher color C22 and gray C23 that have not been changed. The color to be used is common to the reflectance image IMG2 and the first modified image IMG4.

  In the present example, in the first changed image IMG4, an area where the changed representative color white C41 is used, and an area where the ocher colors C22 and gray C23 which are not changed are used. Is shown. Since the first changed image IMG4 is generated from the reflectance image IMG2 that does not include information relating to the shadow, it can be read that the color of the entire sofa has been changed from vermilion C21 to white C41.

  FIG. 7 is an example of the second modified image IMG5 generated by the image editing apparatus 10 according to the embodiment of the present invention. The first generation unit 13 generates a second changed image IMG5 in which the color of the shadow image IMG3 is changed from the shadow image IMG3 based on the change of the color of the shadow image IMG3. In this example, the first generation unit 13 changes from at least one of the hue, saturation, and brightness of the shadow image IMG3 from the shadow image IMG3 based on the change of at least one of the hue, saturation, and brightness of the shadow image IMG3. A second changed image IMG5 in which either one is changed is generated. More specifically, the first generation unit 13 changes the entire shadow image IMG3 from the shadow image IMG3 to light yellow based on a change in which the hue and saturation of the shadow image IMG3 are values representing light yellow. A second modified image IMG5 is generated.

  FIG. 8 is an example of the composite image IMG6 generated by the image editing apparatus 10 according to the embodiment of the present invention. In this example, the second generation unit 14 combines the first changed image IMG4 and the second changed image IMG5 to generate a combined image IMG6. The composite image IMG6 includes objects such as an ocher bed, two brown cushions placed on the bed, a gray carpet, and a white C41 sofa. Compared with the image IMG1 shown in FIG. It differs in that it is a warm color and the color of the sofa is white C41.

  When the composite image IMG6 is compared with the image IMG11 edited by the conventional image editing apparatus shown in FIG. 13, there is no occurrence of whiteout and the color of the shadow portion of the sofa is also changed to white C41. It differs in that it is a natural image. As described above, according to the image editing apparatus 10 according to the present embodiment, by brightening a dark color before the change, a brighter color is changed to be brighter and whiteout occurs in the image. Is prevented. On the other hand, darkening the bright color before the change prevents blackout from occurring in the image. Furthermore, not only the color of a part of the image can be changed, but also the brightness and color of the entire image can be changed to reproduce the change in illumination.

  FIG. 9 is a flowchart of a first example of image editing processing executed by the image editing apparatus 10 according to the embodiment of the present invention. First, the image editing apparatus 10 decomposes the original image into a reflectance image and a shadow image (S10). Then, the image editing apparatus 10 selects one having a large number of pixels as a representative color from a plurality of colors included in the reflectance image (S11). For example, the image editing apparatus 10 may select a representative color that has a large number of pixels from a plurality of colors included in the reflectance image or a similar color of a plurality of colors included in the reflectance image.

  The image editing apparatus 10 determines whether or not the BIC increases when one representative color is added (S12). When the BIC does not increase (S12: NO), the image editing apparatus 10 represents the color having the next largest number of pixels from the plurality of colors included in the reflectance image or the similar colors of the plurality of colors included in the reflectance image. It adds as a color (S13). On the other hand, if one representative color is added and the BIC increases (S12: YES), the image editing apparatus 10 determines the number of representative colors without adding a representative color, and determines one or more representative colors. (S14). If fine adjustment is required in editing the reflectance image, the representative color may be allowed to be increased by user input.

  Thereafter, the image editing apparatus 10 changes the plurality of representative colors independently from each other based on the input by the input unit 10e (S15). Then, the image editing apparatus 10 generates a first changed image from the reflectance image based on the change of the representative color (S16).

  Further, the image editing apparatus 10 changes the color of the shadow image based on the input by the input unit 10e (S17). Then, the image editing apparatus 10 generates a second changed image from the shadow image based on the color change (S18). It should be noted that a process for changing a plurality of representative colors (S15), a process for generating a first changed image (S16), a process for changing the color of a shadow image (S17), and a process for generating a second changed image (S18). And may be executed in parallel. Thereafter, the image editing apparatus 10 combines the first changed image and the second changed image to generate a combined image (S19). When the image editing is to be ended (S20: YES), the image editing apparatus 10 ends the image editing process. On the other hand, when the image editing is continued (S20: NO), the processes of reference numerals S15 to S19 are executed again.

  FIG. 10 is an example of an image editing screen DP displayed by the image editing apparatus 10 according to the embodiment of the present invention. The image editing apparatus 10 includes a display unit 10f. The display unit 10f includes a reflectance image IMG2, a shadow image IMG3, and an original image IMG1, a first modified image IMG4, a second modified image IMG5, and a composite image IMG6. You may display so that comparison is possible. When there is no change in any of the reflectance image IMG2 and the shadow image IMG3, the display unit 10f may display the first changed image IMG4, the shadow image IMG3, and the composite image IMG6, or the reflectance image IMG2, A two-change image IMG5 and a composite image IMG6 may be displayed.

  The image editing screen DP includes a reflectance image IMG2, a shadow image IMG3, and an original image IMG1 in the upper stage, and includes a first changed image IMG4, a second changed image IMG5, and a synthesized image IMG6 in the lower stage. Arrows pointing from the original image IMG1 to the reflectance image IMG2 and the shadow image IMG3 are displayed, indicating that the reflectance image IMG2 and the shadow image IMG3 are decomposed from the image IMG1. Further, an arrow pointing from the reflectance image IMG2 to the first changed image IMG4 is displayed, indicating that the first changed image IMG4 is generated from the reflectance image IMG2. Similarly, an arrow pointing from the shadow image IMG3 to the second changed image IMG5 is displayed, indicating that the second changed image IMG5 is generated from the shadow image IMG3. Furthermore, an arrow pointing to the composite image IMG6 from the first change image IMG4 and the second change image IMG5 is displayed, indicating that the composite image IMG6 is a composite from the first change image IMG4 and the second change image IMG5. ing.

  By displaying the images before and after the color change by the image editing device 10 so that they can be compared, it is possible to grasp at a glance what kind of change has been made, and to support the editing of the color of the image more smoothly. Can do. For example, as in the image editing screen DP of this example, the reflectance image IMG2 and the first modified image IMG4 are displayed so as to correspond in the upper and lower stages, and the shadow image IMG3 and the second modified image IMG5 are associated in the upper and lower stages. The original image IMG1 and the composite image IMG6 are displayed so as to correspond to each other in the upper stage and the lower stage, so that the images before and after the color change can be easily compared. You can see at a glance whether you went. In the case of this example, specifically, it can be easily confirmed that the color of the sofa is different between the reflectance image IMG2 and the first modified image IMG4. Further, it can be easily confirmed that the color and brightness of the entire image are different between the shadow image IMG3 and the second modified image IMG5. Also, it can be easily confirmed that the color of the sofa, the color of the entire image, and the brightness of the entire image are different between the original image IMG1 and the composite image IMG6. Note that the arrangement of the images shown in this example is an example, and the images before and after the change can be displayed side by side, or other arrangements can be made.

  The display unit 10f may update and display the first modified image IMG4 and the composite image IMG6 when the first modified image IMG4 is generated by at least the first generating unit 13. The image editing screen DP may include a first indicator I1 for adjusting the HSV value of the representative color selected from the reflectance image IMG2, and a second indicator I2 for adjusting the HSV value of the shadow image IMG3. Changes in hue, saturation and lightness may be accepted. The first indicator I1 is an example of an indicator that changes the representative color of the reflectance image IMG2, and the value to be adjusted is not limited to the HSV value, and may be an arbitrary value. Similarly, the second indicator I2 is an example of an indicator for changing the color of the shadow image IMG3, and the value to be adjusted is not limited to the HSV value, and may be an arbitrary value. The image editing screen DP may include a selection window S for designating a representative color to be changed among a plurality of representative colors selected from the reflectance image IMG2. When a specific representative color is designated and the first indicator I1 is adjusted, the first generation unit 13 updates the first changed image IMG4 by changing the designated representative color according to the value. In addition, the second generation unit 14 combines the new first changed image IMG4 and the second changed image IMG5, and updates the combined image IMG6. Similarly, when the 2nd indicator I2 is adjusted, the 1st production | generation part 13 updates 2nd change image IMG5 according to the value. Moreover, the 2nd production | generation part 14 may synthesize | combine 1st change image IMG4 and new 2nd change image IMG5, and may update synthetic | combination image IMG6.

  According to the image editing apparatus 10 according to the present embodiment, when the representative color of the reflectance image IMG2 is changed, the first changed image IMG4 and the composite image IMG6 are updated and displayed so as to be linked to the change, It is possible to grasp at a glance how the image changes by changing the representative color. When the color of the shadow image IMG3 is changed, the second changed image IMG5 and the composite image IMG6 are updated and displayed so as to be linked to the change, and how the image changes due to the change of the shadow image IMG3. Can be seen at a glance. Thereby, it can assist so that the edit of the color of an image may become smoother. When the color of the shadow image IMG3 is changed, the second changed image IMG5 and the composite image IMG6 are updated and displayed so as to be linked to the change, and how the image changes due to the change of the shadow image IMG3. Can be grasped at a glance, and the editing of the color of the image can be supported more smoothly.

  FIG. 11 is a flowchart of a second example of the image editing process executed by the image editing apparatus 10 according to the embodiment of the present invention. First, the image editing apparatus 10 displays the reflectance image, the shadow image, and the original image, the first modified image, the second modified image, and the composite image so as to be comparable (S30).

  Then, the image editing apparatus 10 changes the hue, saturation, and brightness of the plurality of representative colors independently from each other based on the input from the input unit 10e (S31). Based on the change, the image editing apparatus 10 generates a new first changed image, combines the new first changed image and the second changed image to generate a new combined image (S32), and creates a new one. The first changed image and the new composite image are updated and displayed (S33).

  Further, the image editing apparatus 10 changes the hue, saturation, and brightness of the shadow image independently from each other based on the input from the input unit 10e (S34). Based on the change, the image editing apparatus 10 generates a new second changed image, combines the first changed image and the new second changed image to generate a new combined image (S35), and creates a new one. The second changed image and the new composite image are updated and displayed (S36). A process for changing a plurality of representative colors (S31), a process for generating a first changed image (S32), a process for generating a composite image (S33), a process for changing the color of a shadow image (S34), a first The process for generating the two-change image (S35) and the process for generating the composite image (S36) may be executed in parallel. When the image editing is to be ended (S37: YES), the image editing apparatus 10 ends the image editing process. On the other hand, when the image editing is continued (S37: NO), the processes of S31 to S36 are executed again.

  The embodiments described above are for facilitating the understanding of the present invention, and are not intended to limit the present invention. Each element included in the embodiment and its arrangement, material, condition, shape, size, and the like are not limited to those illustrated, and can be changed as appropriate. In addition, the structures shown in different embodiments can be partially replaced or combined.

  DESCRIPTION OF SYMBOLS 10 ... Image editing apparatus, 10a ... CPU, 10b ... RAM, 10c ... ROM, 10d ... Communication part, 10e ... Input part, 10f ... Display part, 11 ... Decomposition part, 12 ... Selection part, 13 ... 1st production | generation part, 14 ... 2nd production | generation part, 15 ... Determination part, 16 ... Memory | storage part

Claims (8)

A decomposition unit that decomposes an image obtained by photographing an object into a reflectance image that represents the reflectance of the object and a shadow image that represents the shadow of the object;
A selection unit that selects a plurality of representative colors based on the number of pixels of a plurality of colors included in the reflectance image or the number of pixels of similar colors of the plurality of colors;
Based on the fact that any one of the plurality of representative colors is changed to a different color, a first changed image is generated by changing any one of the plurality of colors included in the reflectance image from the reflectance image. A generator,
A second generation unit that combines the first modified image and the shadow image to generate a combined image;
An image editing apparatus comprising: A determining unit that determines the number of the plurality of representative colors such that the plurality of colors included in the reflectance image are represented by the plurality of representative colors while suppressing an increase in the number of the plurality of representative colors. Further comprising
The image editing apparatus according to claim 1. The first generation unit generates the first changed image from the reflectance image based on changing the plurality of representative colors independently of each other.
The image editing apparatus according to claim 1. The first generation unit generates a second changed image in which the color of the shadow image is changed from the shadow image based on the change of the color of the shadow image,
The second generation unit generates the composite image by combining the first change image and the second change image;
The image editing apparatus according to any one of claims 1 to 3. A display unit that displays the reflectance image, the shadow image, and the image, and the first modified image, the second modified image, and the composite image in a comparable manner;
The image editing apparatus according to claim 4. The display unit updates and displays the first modified image and the composite image when the first modified image is generated by at least the first generating unit, and the second modified image is generated. If done, update and display the second modified image and the composite image;
The image editing apparatus according to claim 5. Decomposing an image obtained by photographing an object into a reflectance image representing the reflectance of the object and a shadow image representing a shadow of the object;
Selecting a plurality of representative colors based on the number of pixels of a plurality of colors included in the reflectance image or the number of pixels of a similar color of the plurality of colors;
Generating a first changed image in which one of the plurality of colors included in the reflectance image is changed from the reflectance image based on changing any of the plurality of representative colors to a different color; ,
Combining the first modified image and the shadow image to generate a combined image;
Image editing method including A computer equipped with an image editing device
A decomposition unit that decomposes an image obtained by capturing an object into a reflectance image that represents the reflectance of the object and a shadow image that represents the shadow of the object;
A selection unit that selects a plurality of representative colors based on the number of pixels of a plurality of colors included in the reflectance image or the number of pixels of a similar color of the plurality of colors;
Based on the fact that any one of the plurality of representative colors is changed to a different color, a first changed image is generated by changing any one of the plurality of colors included in the reflectance image from the reflectance image. A generating unit; and a second generating unit that combines the first changed image and the shadow image to generate a combined image;
Image editing program to function as.