JP7421607B2 - Image processing device and method - Google Patents

Description

The present invention relates to image processing that reproduces translucent objects by taking internally scattered light into account.

In recent years, in the field of computer graphics (CG), technology has been developed to draw translucent objects in which light is scattered not only on the object's surface but also inside the object. Here, human skin is particularly difficult to draw because it is important not only to reproduce internally scattered light but also to reproduce surface wrinkles, pores, and the like. Surface shapes such as wrinkles and pores are hereinafter referred to as "fine irregularities of an object surface (FIOS)." A translucent object is an object in which light is transmitted and scattered, such as cloudy glass or plastic, cloudy liquid, or biological cells.

As a method for reproducing internally scattered light, the method disclosed in Patent Document 1 is known. Patent Document 1 performs sampling at multiple points, integrates pixel values using a bidirectional scattering surface reflectance distribution function (BSSRDF), and performs blurring processing. A method is also known in which internally scattered light is reproduced by performing blurring processing in which the amount of blurring differs for each wavelength channel of an image value. Furthermore, as a method for reproducing FIOS, there is a process that uses a bump map that represents the surface shape. When reproducing human skin using CG, blurring processing is usually performed after reproducing FIOS.

Figure 1 explains the appearance of a translucent object with FIOS affected by internally scattered light. FIG. 1 shows the spread of internally scattered light when light is incident on a translucent object 101 from a cross-sectional direction. As shown in FIG. 1(a), when light enters the surface of a semi-transparent object 101 from a direction perpendicular to the surface, the output light exhibits a characteristic in which long-wavelength light R is spread more widely than short-wavelength light B.

On the other hand, FIG. 1(b) shows a case where light is obliquely incident on the semitransparent object 101. Light incident obliquely has the characteristic of spreading in the direction of incidence. Therefore, the right side of the diagonally cut translucent object 101 (hereinafter referred to as the cut part) is a part where no light would normally enter and a shadow would occur, but the cut part becomes brighter because internally scattered light is emitted. . At this time, more long-wavelength light that spreads more easily is emitted from the cut portion, and if there is no internally scattered light, the cut portion, which is a shadow portion, is observed to be red and bright due to the internally scattered light. The cut portion, which is a shadow portion when there is no internally scattered light, corresponds to FIOS such as wrinkles and pores in human skin.

The technique of Patent Document 1 uniformly performs blurring processing without considering the influence of FIOS, and therefore cannot reproduce the characteristics of FIOS becoming red and bright. Since FIOS greatly affects the appearance of translucent objects, it is necessary to faithfully reproduce the characteristic of FIOS becoming red and bright due to internally scattered light (hereinafter referred to as brightness increase).

Special Publication No. 2006-506742

The present invention aims to reproduce a translucent object by considering internally scattered light.

The present invention includes the following configuration as one means for achieving the above object.

An image processing apparatus according to the present invention includes an input means for inputting a first image obtained by capturing an image of a subject having an uneven surface;
A second image obtained when the subject in the first image is virtually projected with illumination light different from the illumination light with which the first image was captured, and the first image is corrected. and generating means for generating by.
The generating means corrects the first image so that in the second image, a part of the shadow area caused by the unevenness becomes redder than other shadow areas ,
The part is redder and brighter than the other shadow area.
It is characterized by

According to the present invention, it is possible to reproduce a translucent object by considering internally scattered light. For example, it becomes possible to reproduce the increase in brightness of fine irregularities on the surface of an object (FIOS) due to internally scattered light.

A diagram illustrating the appearance of a translucent object with FIOS affected by internally scattered light. FIG. 1 is a block diagram illustrating a configuration example of an information processing device that functions as an image processing device according to an embodiment. FIG. 1 is a diagram illustrating an example of a processing configuration of an image processing device. FIG. 3 is a diagram illustrating an overview of a normal mapping method in drawing processing. FIG. 4 is a diagram illustrating normal mapping processing in detail. 5 is a flowchart illustrating drawing processing executed by the image processing device. A diagram explaining height information λ-h. 5 is a flowchart illustrating a normal map generation process for each wavelength by a normal map generation unit. 5 is a flowchart illustrating calculation image generation processing by a calculation image generation unit. FIG. 3 is a diagram illustrating shading processing. FIG. 3 is a block diagram showing an example of a processing configuration of an image processing apparatus according to a second embodiment. 5 is a flowchart illustrating drawing processing executed by the image processing device. A diagram illustrating the concept of wraparound of internally scattered light. 10 is a flowchart illustrating a process for creating a normal map for each wavelength including a process for selecting a high frequency area according to the third embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Image processing apparatuses and image processing methods according to embodiments of the present invention will be described in detail below with reference to the drawings. Note that the examples do not limit the scope of the claimed invention, and not all combinations of configurations described in the examples are essential to the solution of the invention.

In recent years, an image processing method called relighting that changes or adds illumination light to a captured image has been proposed. The relighting process uses the shape information of the object to perform drawing processing under a new lighting environment. Particularly when relighting human skin, if the influence of internally scattered light is not taken into account, an unnatural appearance that differs from actual human skin will result.

In the following example, a method for reproducing a semi-transparent object will be described in consideration of the influence of internally scattered light on the appearance of fine irregularities (FIOS) on the surface of the object. In order to faithfully reproduce human skin, by adding an ideal FIOS shape (texture) to the image area where human skin is photographed and applying the reproduction method described in the example, it is possible to faithfully reproduce the internal scattered light of human skin. becomes possible to reproduce.

As Example 1, a method for reproducing a translucent object using a drawing technique that changes information indicating the fine irregularities (FIOS) on the surface of the object to be drawn according to the wavelength will be described.

[Device configuration]
The block diagram in FIG. 2 shows an example of the configuration of an information processing device that functions as an image processing device according to the embodiment. A microprocessor (CPU) 201 uses a main memory 202 as a work memory, executes programs stored in a storage unit 203, performs various arithmetic operations, and controls the configuration described below via a system bus 206. Main memory 202 is composed of, for example, random access memory (RAM) or read-only memory (ROM).

The storage unit 203 is a hard disk drive (HDD) or solid state drive (SSD), and stores an image processing program, data representing an object to be drawn, characteristic data of a light source, and the like. The input unit 204 is a keyboard, a mouse, or the like, and is a device for inputting user operations and user instructions. The display unit 205 displays images and user interfaces (UI) generated by the CPU 201 on a monitor 207 such as a liquid crystal panel.

The information processing device, which is a computer device, may include a network interface (not shown). In that case, the server device can be accessed via a wired or wireless network, various programs and characteristic data can be acquired from the server device, and arithmetic processing results can be output to the server device. Further, the information processing device may be a tablet or a smartphone that executes a program for implementing image processing, which will be described later. In that case, the monitor 207 and the input unit 204 are provided as an integrated touch panel.

●Image processing device An example of the processing configuration of the image processing device will be explained with reference to FIG. The processing configuration shown in FIG. 3 is realized by the CPU 201 executing an image processing program. Input units 301-305 each input the following information via system bus 206.

The vertex information input unit 301 inputs vertex information representing the shape of the object to be drawn from the storage unit 203 or the like. The light source information input unit 302 inputs light source information from the storage unit 203, the input unit 204, or the like. The diffuse color information input unit 303 inputs a diffuse color map representing the color of the object to be drawn from the storage unit 203 or the like. The FIOS information input unit 304 inputs, from the storage unit 203 or the like, a height map, which will be described in detail later, which represents the relative height of the surface of the object to be drawn, as first information FIOS1 of the FIOS. The height information λ-h input unit 305 receives height information λ-h, the details of which will be described later, from the storage unit 203 or the input unit 204.

The vertex processing unit 306 performs vertex processing based on the input vertex information, and outputs the vertex processing result to the calculated image generation unit 308. The normal map generation unit 307 creates a normal map for each wavelength as second information FIOS2 of FIOS based on the height map FIOS1 and height information λ-h, and calculates these normal maps FIOS2i.The image generation unit Output to 308.

The calculated image generation section 308 includes a drawing processing section 309 and an image composition section 310. The drawing processing unit 309 performs drawing including shading processing for each wavelength based on the vertex processing result of the vertex processing unit 306, the light source information, the diffuse color map, and the normal map FIOS2i for each wavelength input from the normal map generation unit 307. Perform processing. The image composition unit 310 generates a composite image by combining the shadow-processed images, which are objects for each wavelength created by the drawing processing unit 309. The composite image is output from the output unit 311 as a calculated image representing the object to be drawn.

[Overview of normal mapping method]
Since FIOS is minute, drawing FIOS using polygons requires computation of a huge amount of data. Therefore, a method is needed to reproduce FIOS with a small amount of calculation. The normal mapping method is a method of pseudo-reproducing FIOS by using image data (normal map) that stores normal directions instead of polygons.

An overview of the normal mapping method in drawing processing will be explained with reference to FIG. As shown in FIG. 4(a), the normal line always points upward on the plane. When a slope is included as shown in FIG. 4(b), a tilt occurs between the normal on the slope and the normal on the plane. As shown in FIG. 4(c), when the normal of a slope is assigned to a plane, a shadow is generated in the same way as the slope, so it is possible to give the appearance of a slope even though it is a plane. In other words, the normal map can represent the inclination of the surface of the object to be drawn. Furthermore, the accuracy of objects drawn by normal mapping processing depends on the density of the number of normals stored in the normal map.

The normal mapping process will be explained in detail with reference to FIG. The FIOS information has, for example, "0.0" as a value indicating the same height as the reference plane of the object to be drawn, and "1.0" as a value indicating the highest position from the reference plane. The height map FIOS1 is gray scale data indicating values of FIOS information from 0.0 to 1.0. For example, the bottom (deepest part) of the FIOS of the object to be drawn may be used as the reference plane. Alternatively, FIOS information may be created using the average surface of the object to be drawn, excluding FIOS, as a reference plane. In this case, the FIOS information has positive and negative values, so by normalizing the FIOS information with the minimum value (negative value) as "0.0", the deepest point is "0.0" and the highest point is "1.0". FIOS information with ” is obtained.

The normal map generation unit 307 acquires the change in height from the height map FIOS1, and generates a normal map FIOS2 in which vector information (unit vector of normal) based on the change in height is stored in each pixel. Although we have given an example of creating a normal map from a height map, it is also possible to create a normal map based on information that can obtain height information λ-h, or if the normal map is input directly. Good too.

Furthermore, the drawing processing unit 309 generates an illumination map storing vector information from the light source information, similar to the normal map FIOS2. Each pixel of the illumination map stores a vector oriented toward a light source and having a length corresponding to the amount of incident light. By taking the inner product of the vectors of the corresponding pixels of the normal map FIOS2 and the illumination map, the brightness value of the pixel to be drawn is calculated.

[Drawing processing]
The drawing process executed by the image processing device will be explained with reference to the flowchart in FIG. The input unit 301 inputs vertex information representing the shape of the object to be drawn (S601). The vertex processing unit 306 performs vertex processing based on the input vertex information (S602). Vertex processing includes CG processing such as a vertex shader, clipping, perspective transformation, and viewport transformation, but these techniques are well known and will not be described here.

Next, the input unit 302 inputs light source information (S603). The light source information indicates spectral radiance, which is brightness information for each wavelength of the light source, and the light source position in virtual space. The input unit 303 inputs a diffuse color map representing the color of the object to be drawn (S604). The input unit 304 inputs the height map FIOS1, which is FIOS information of the object to be drawn (S605). The input unit 305 inputs height information λ-h (S606). Note that the input order of each piece of information is an example, and is not limited to the above, and each piece of information can be input in any order.

The height information λ-h will be explained with reference to FIG. The height information λ-h indicates the relationship between the wavelength and the maximum height of the height map. FIG. 7(a) is an example of height information λ-h, which is data in a table format indicating the maximum value h of the height of the height map at wavelength λi. For example, assuming that wavelength w1 corresponds to the red (R) component, wavelength w2 to the green (G) component, and wavelength w3 to the blue (B) component, there is a relationship of w1 > w2 > w3, and in this case, each wavelength corresponds to h1<h2<h3 is set as the relationship of the maximum height. In other words, the height information λ-h is information in which the longer the wavelength is, the smaller the maximum value h is, and the shorter the wavelength is, the larger the maximum value h is. Furthermore, by setting h1=h2=h3 as the relationship between the maximum height values, it is possible to prohibit the process of increasing the brightness of the covered portion due to an opaque object or an opaque object on a semi-transparent object.

Changes in the direction of the normal vector depend on the inclination of the slope, as shown in Figure 4. As shown in FIG. 7(b), when the maximum value h is small and the slope is small, the direction of the normal vector exhibits a distribution close to the direction perpendicular to the plane (dashed line arrow in FIG. 7(b)). On the other hand, when the maximum value h is large and the slope is large, the direction of the normal vector shows a more dispersed distribution. Therefore, the longer the wavelength of light, the smaller the maximum value h is set to reduce the change in the direction of the normal vector, and the shorter the wavelength light, the larger the maximum value h is set to make the change in the normal vector direction larger. Can be done. In other words, by giving a difference in the maximum height value h to the wavelength λ, the brightness of the FIOS (which is a shadow part when there is no internally scattered light) is obtained in the pixel value calculation process described later.

Figure 7(c) shows the value of the height map FIOS1 (horizontal axis) and the value of the height map FIOS1i when the height map FIOS1i is generated using the height information λ-h shown in Figure 7(a). (Vertical axis) shows the relationship. Note that the definition of the relationship between the value of the height map and the height is not limited to a table format, and may be defined using a polynomial or the like.

Next, the normal map generation unit 307 uses the height map FIOS1 and the height information λ-h to generate a normal map FIOS2 for each wavelength (S607). The calculated image generation unit 308 uses the information input or generated in steps S602 to S607 to perform pixel value calculation processing (drawing processing) including shading processing, and draws a shading processed image that is an object for each wavelength λi. (S608). Then, the shading processed images, which are objects for each wavelength λi, are combined (S609), and the calculated image obtained by the combination is outputted via the output unit 311 (S610). The calculated image representing the object to be drawn is displayed on the monitor 207 by the display unit 205 or stored in the storage unit 203 or the like.

●Normal map generation unit The normal map generation process for each wavelength (S607) by the normal map generation unit 307 will be explained with reference to the flowchart of FIG. The normal map generation unit 307 initializes a counter i to 0 (S801), and obtains the i-th wavelength λi and the maximum height value hi from the height information λ-h (S802). For example, when the height information λ-h shown in FIG. 7(a) is input, the wavelength w1 and the maximum value h1 are initially obtained. Next, the normal map generation unit 307 creates a height map FIOS1i of the wavelength λi with the maximum height as the maximum value hi (S803), and creates a normal map FIOS2i of the wavelength λi based on the height map FIOS1i. (S804) Note that the corresponding wavelength λi is recorded as header information of the normal map FIOS2i.

Next, the normal map generation unit 307 increments the counter i (S805) and compares the count value i with the number of wavelengths N of the height information λ-h, thereby performing normal map generation processing for each wavelength. It is determined whether or not to end (S806). That is, if i≦N, the process returns to step S802 to create a normal map for the next wavelength λi, and if i>N, the normal map generation process for each wavelength ends.

●Calculated Image Generation Unit Calculated image generation processing (S608-S609) by the calculated image generation unit 308 will be explained with reference to the flowchart in FIG. In the calculated image generation unit 308, the drawing processing unit 309 generates an illumination map based on the light source information (S901), and initializes a counter i to 0 (S902). Next, the drawing processing unit 309 obtains the wavelength λi from the header information of the normal map FIOS2i (S903), and extracts the diffuse color map corresponding to the wavelength λi from the input diffuse color map (S904). Then, drawing processing including shading processing is performed using the vertex processing result, the illumination map, the diffuse color map corresponding to the wavelength λi, and the normal line map FIOS2i to generate a shading processed image that is an object of the wavelength λi (S905).

The shading processed image for each wavelength corresponds to, for example, a grayscale image of an R component, a G component, or a B component. However, the wavelength λi is not limited to the dominant wavelength of the R component, the G component, or the B component, and may be three or more wavelengths, as long as a full-color image can be obtained by combining each shading-processed image of the wavelength λi.

Shading processing will be explained with reference to FIG. FIG. 10(a) shows the diffused colors of wavelengths w1, w2, and w3. Note that, as described above, the wavelengths have a relationship of w1>w2>w3. FIG. 10(b) shows the normal map FIOS2i corresponding to wavelengths w1, w2, and w3. As mentioned above, the height corresponding to each wavelength has the relationship h1≦h2≦h3, so in the normal map FIOS2i, the change in the direction of the vector of wavelength w1 is smaller than the change in the direction of the vector of wavelength w3. is small. FIG. 10(c) shows the shading processed image created in step S905. By using the normal map FIOS2i, in which the direction of the vector changes less as the wavelength becomes longer, the brightness of the FIOS at wavelength w1 increases compared to the brightness of FIOS at wavelength w3, as shown in Figure 10(c). You can get an increase in brightness.

Next, the drawing processing unit 309 increments the counter i (S906) and compares the count value i with the number N of normal maps FIOS2 to determine whether or not to end the drawing process (S907). That is, if i≦N, the process returns to step S902 and the drawing process for the next wavelength λi is performed, and if i>N, the drawing process ends. When the drawing process is completed, the image composition unit 310 generates a color image by combining the shadow-processed images of the wavelength λi as a calculated image (S908), and the calculation image generation process ends.

In this way, by increasing the brightness of FIOS (which is a shadow area when there is no internally scattered light), it is possible to reproduce the red and bright characteristics of FIOS, and more faithfully reproduce translucent objects with internal scattering characteristics. It becomes possible to do so.

In the above, an example was explained in which the height map FIOS1 input as the first surface fine unevenness information is converted to the normal map FIOS2i for each wavelength, which is the second surface fine unevenness information. Surface fine unevenness information is not limited to a height map or a normal line map. For example, any information that includes FIOS information, such as distance information measured by a camera or distance measuring device, can be used. Furthermore, when converting from FIOS1 to FIOS2i, it is possible to use not only the method of changing the height of the height map, but also the method of applying different blurring processing to FIOS1 for each wavelength.

Embodiment 2 An image processing apparatus and an image processing method according to a second embodiment of the present invention will be described below. Note that in the second embodiment, components that are substantially the same as those in the first embodiment may be denoted by the same reference numerals, and detailed description thereof may be omitted.

When the processing of Example 1 is performed, since the shape of FIOS differs depending on the wavelength, the size and shape of the gloss may differ somewhat depending on the wavelength, which may appear as color shift after synthesis. In the second embodiment, the calculation of the glossy part is performed separately from one normal map, and the image of the glossy part and the shading processed image are combined to eliminate color shift in the glossy part.

The block diagram in FIG. 11 shows an example of the processing configuration of the image processing device. The configuration of the image processing device of Example 2 is such that a glossy image calculation section 321 and a glossy image composition section 322 are added to the configuration shown in FIG. The glossy image calculation unit 321 draws a glossy image using one normal map FIOS2i acquired from the normal map generation unit 307 and the illumination map acquired from the drawing processing unit 309, and converts the glossy image into a glossy image composition unit 322. Output to.

The glossy image composition unit 322 generates a calculated image by combining the composite image of the shading processed image inputted from the calculated image generation unit 308 and the glossy image inputted from the glossy image calculation unit 321. Note that if the composite image of the shading-processed image already has a glossy part, the glossy image composition unit 322 performs processing such as deleting the glossy part, and then composites the glossy image with the composite image of the shading-processed image.

The drawing process executed by the image processing apparatus of the second embodiment will be explained with reference to the flowchart in FIG. The processing in steps S601-S609 is the same as in the first embodiment, and detailed explanation will be omitted. The glossy image synthesis unit 322 obtains one of the normal maps FIOS2i created by the normal map generation unit 307 and the illumination map (S621), and draws a glossy image using them (S622).

Next, the glossy image composition unit 322 generates a calculated image by combining the composite image of the shading processed image inputted from the calculated image generation unit 308 and the glossy image inputted from the glossy image calculation unit 321 (S623), and The image is output via the output unit 311 (S624). The calculated image is displayed on the monitor 207 by the display unit 205 or stored in the storage unit 203 or the like. In this way, color shift in the glossy portion can be eliminated.

Embodiment 3 An image processing apparatus and an image processing/method according to a third embodiment of the present invention will be described below. Note that in the third embodiment, components that are substantially the same as those in the first and second embodiments are denoted by the same reference numerals, and detailed description thereof may be omitted.

As a third embodiment, a method will be described in which a translucent object reproduction method is selectively applied to an area where the spatial frequency of FIOS is high (hereinafter referred to as a high frequency area) where the influence of internally scattered light is noticeable. FIG. 13 shows the concept of wraparound of internally scattered light. Figure 13(a) shows the wrap-around of internally scattered light in a semi-transparent object with a high FIOS spatial frequency (hereinafter referred to as a high-frequency object), and Figure 13(b) shows a semi-transparent object with a low FIOS spatial frequency. (hereinafter referred to as a low-frequency object) shows the wraparound of internally scattered light.

As shown in FIG. 13(a), the distance that light travels inside a high-frequency object is short. On the other hand, as shown in FIG. 13(b), the distance that light travels inside a low-frequency object is long. Since the amount of absorbed light increases according to the distance traveled by the light, the amount of light emitted from a high frequency object is large, and the amount of light emitted from a low frequency object is small. Therefore, in the low-frequency object shown in FIG. 13(b), internally scattered light is not emitted to the extent that an increase in brightness is required. Therefore, it can be said that it is effective to selectively apply the translucent object reproduction method to high frequency areas that are more significantly affected by internally scattered light.

The normal map creation process for each wavelength (S607) including the high frequency area selection process will be explained with reference to the flowchart in FIG. After initializing the counter i (S801), the normal map creation unit 307 extracts a high frequency area based on the FIOS information (for example, the height map FIOS1) (S811). Next, the i-th wavelength λi and the maximum height value hi are obtained from the height information λ-h (S802), and a height map FIOS1i of the wavelength λi is created for the extracted high frequency area (S812). The processing in steps S805 and S806 is the same as in the first embodiment, and detailed explanation will be omitted.

When the determination result in step S806 is i>N, the normal map creation unit 307 generates information indicating a predetermined height (for example, the height of the reference plane) for the non-extracted area that has not been extracted as a high frequency area. The height map FIOS1i for each wavelength λi is set (S813). Then, a normal map FIOS2i for each wavelength λi is created based on the height map FIOS1i for each wavelength λi (S814), and the normal map generation process for each wavelength is completed.

In this way, a height map FIOS1i is created in which the maximum height differs for each wavelength in the high frequency area and the height information matches in the non-extracted area. Therefore, it is possible to selectively apply the translucent object reproduction method to areas where the spatial frequency of FIOS is high, where the influence of internally scattered light is noticeable.

[Modified example]
For example, if the light source does not include long wavelength light, or if the diffuse color map does not include colors in the long wavelength range, no increase in brightness, or even if it is observed, may be minimal. In order to reduce the load of reproduction processing of brightness increase in such a case, the normal map generation unit 307 determines whether to create a normal map FIOS2i for each wavelength from the diffuse color map and light source information. Good too. When not creating the normal map FIOS2i for each wavelength, the normal map generation unit 307 outputs the normal map FIOS2 corresponding to the height map FIOS1.

Furthermore, the above description has been made on the assumption that the object to be drawn is a translucent object. When a part of a translucent object is covered by an opaque object, the reproduction method of Examples 1-3 is applicable to the uncovered portion of the translucent object that is not covered by the opaque object. In this case, the covered portion may be outside the applicable range of the reproduction method, or the same reproduction method as for the non-covered portion may be applied as an area in which the normal map FIOS2i for each wavelength is not created.

Furthermore, when light enters the surface of the object to be drawn at a shallow angle (an angle of incidence close to 90 degrees), the influence of internally scattered light on appearance is smaller than that of reflected light. Therefore, similar to the high frequency area in Example 3, it is also effective to extract areas where the angle of incidence of illumination light that enters the surface of the object to be drawn is less than a predetermined angle (low incidence angle area). In this case, a height map FIOS1i with different maximum heights for each wavelength for the extraction area and matching height information for the non-extraction area is created, and the normal of each wavelength λi is created based on the height map FIOS1i of each wavelength λi. Map FIOS2i is created. Therefore, the translucent object reproduction method can be selectively applied to low incident angle areas where the influence of internally scattered light is noticeable.

[Other Examples]
The present invention provides a program that implements one or more functions of the above-described embodiments to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. This can also be achieved through processing. It can also be implemented by a circuit (eg, ASIC) that implements one or more functions.

304... Input section, 307... Normal map generation section, 309... Drawing processing section, 310... Image composition section

Claims (17)

an input means for inputting a first image obtained by imaging a subject having an uneven surface;
Correcting the first image to obtain a second image obtained when the subject in the first image is imaged by virtually projecting illumination light different from the illumination light with which the first image was captured. and generating means for generating by.
The generating means corrects the first image so that in the second image, a part of the shadow area caused by the unevenness becomes redder than other shadow areas ,
The part is redder and brighter than the other shadow area.
An image processing device characterized by: 2. The image processing apparatus according to claim 1, wherein the shadow region is a region of a concave portion caused by the concavo-convex portion of the second image, and is an area adjacent to a convex portion of the concave-convex portion. The first image has a plurality of color components, the plurality of color components are red, blue, and green, and each pixel in the first image has a pixel value of each of the plurality of color components. The image processing apparatus according to claim 1 or 2 . The generating means corrects the first image so that the ratio of red in the red, blue, and green ratios of the pixel values becomes larger for pixels in the human skin area in the part. The image processing device according to item 3 . The image processing device according to claim 3 or 4 , wherein the degree of change from the first image to the second image after correction by the generating means is different for each of the plurality of color components. . The generating means corrects the first image using a height map representing the height of the unevenness and height information created for each wavelength corresponding to the plurality of color components. The image processing apparatus according to any one of claims 3 to 5 . The image processing apparatus according to claim 6 , wherein the height information is information for limiting the height of the red color in the height map. The generation means includes a creation means for creating a normal map using a height map representing the height of the unevenness and height information created for each wavelength corresponding to the plurality of color components. ,
8. The image processing apparatus according to claim 6 , wherein the creation means creates the normal map when long wavelength light is present in the plurality of color components. 9. The creating means does not create the normal map for the subject area in the first image if the area is covered by an object that is not a translucent object. The image processing device described. 9. The generating means corrects the first image by combining the corrected image obtained by correcting the first image with the first image. The image processing device according to item 1. In the second image, for a region where the angle formed between the illumination light different from the illumination light when the first image was captured and the surface of the subject is shallow, the height information is determined based on the red color in the height map. 10. The image processing apparatus according to claim 6 , wherein the height restriction of the color component matches the height restriction of other color components. comprising calculation means for calculating a gloss image representing the gloss of the subject in the second image,
The image processing apparatus according to any one of claims 1 to 10 , wherein the generating unit composes the glossy image with the first image to correct the first image. The image processing apparatus according to any one of claims 1 to 10 , wherein the part is a high frequency region compared to the other shadow region. 14. The image processing apparatus according to claim 1, wherein the part is a part of the shadow area when there is a plurality of shadow areas. The image processing apparatus according to any one of claims 1 to 14 , wherein the input means acquires the first image from a server device via a network. An image processing method performed by an image processing device, the method comprising:
an input step in which the input means of the image processing device inputs a first image obtained by capturing an image of a subject having an uneven surface;
A second image obtained when the generating means of the image processing device virtually projects illumination light different from the illumination light used when capturing the first image onto the subject in the first image and captures the image. a generation step of generating an image by correcting the first image,
In the generation step, the first image is corrected so that in the second image, a part of the shadow area caused by the unevenness becomes redder than other shadow areas ,
The part is redder and brighter than the other shadow area.
An image processing method characterized by: A computer program for causing a computer to function as each means of the image processing apparatus according to any one of claims 1 to 15 .

Images