JP2020160489A - Image processing device, image processing method, and program - Google Patents

Abstract

To provide an image processing device capable of making corrections less noticeable after correcting an object image.SOLUTION: An image processing device (image capturing device 1) comprises: acquisition means (image capturing unit 106) for acquiring an object image; texture information acquisition means (analysis unit 109) configured to obtain object texture information from the object image acquired by the acquisition means; and processing means (second image processing unit 110) configured to process the object image based on the texture information obtained by texture information acquisition means.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image processing apparatus, an image processing method, and a program.

Conventionally, a technique for correcting a face image has been proposed. For example, in Patent Document 1, a facial image that reproduces a smooth skin condition by applying a smoothing treatment to the remaining skin region that masks characteristic regions such as the eyes, nose, and mouth of the face to make fine rough skin inconspicuous. Is described to generate.

JP-A-2009-702260

However, with the technique of Patent Document 1, it is difficult to correct only the singular region different from the surrounding skin, and when the correction processing is applied, it is foreseen that the difference from the uncorrected peripheral portion becomes conspicuous. Will be done.

The present invention has been made in view of such problems, and an object of the present invention is to make the corrected portion inconspicuous when the subject image is corrected.

In order to solve the above problems, the image processing apparatus of the present invention
An acquisition method for acquiring a subject image and
The texture information acquisition means for acquiring the texture information of the subject from the subject image acquired by the acquisition means, and
A processing means for processing the subject image based on the texture information acquired by the texture information acquisition means, and
It is characterized by having.

Further, the image processing apparatus of the present invention is
An acquisition method for acquiring a subject image and
A detection means for detecting a singular region different from the surroundings from the subject image,
A correction means for correcting the singular region detected by the detection means so as to make it inconspicuous with respect to the periphery.
A control means for controlling the singular region corrected so as to be inconspicuous by the correction means so as to perform processing for adding the surface state of the subject in the subject image before the correction by the correction means.
It is characterized by being equipped with.

Further, the image processing apparatus of the present invention is
An acquisition method for acquiring a subject image and
A detection means for detecting a singular region different from the periphery in the subject image acquired by the acquisition means, and
Using mask data of a plurality of sizes imitating a shape surrounding the singular region detected by the detection means, a processing means for performing predetermined processing on the image region including the singular region surrounded by the mask data, respectively.
A reliability information acquisition means for acquiring reliability information of each of the image regions subjected to the predetermined processing using each of the mask data of a plurality of sizes.
Based on the reliability information acquired by the reliability information acquisition means, a synthesis means for synthesizing the results of predetermined processing in each mask data processed by the processing means with different degrees of influence.
It is characterized by being equipped with.

According to the present invention, it is possible to make the corrected portion inconspicuous when the subject image is corrected.

It is a block diagram which shows the functional structure of the image pickup apparatus which concerns on this embodiment. It is a figure for demonstrating the outline of the processing in the analysis part of FIG. 1 and the 2nd image processing part. It is a figure which shows the flow of processing in the analysis part and the 2nd image processing part of FIG.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. The present invention is not limited to the illustrated examples.

[Configuration of Imaging Device 1]
FIG. 1 is a block diagram showing a functional configuration of the image pickup apparatus 1 according to the present embodiment. As shown in FIG. 1, the image pickup device 1 includes a control unit 101, a storage unit 102, a display unit 103, a communication unit 104, an operation unit 105, an image pickup unit 106, a first image processing unit 107, an image memory 108, and an analysis unit. It is configured to include 109, a second image processing unit 110, a lighting control unit 111, a light emitting unit 112, and the like. The control unit 101 and each unit are connected by a bus.

The control unit 101 is configured to include a CPU (Central Processing Unit), a RAM (Random Access Memory), and the like, and executes various programs stored in the storage unit 102 to perform predetermined calculations and control each unit.

The storage unit 102 is composed of a non-volatile semiconductor memory, a hard disk, or the like. The storage unit 102 stores a system program executed by the control unit 101, various processing programs, data necessary for executing these programs, and the like.

The display unit 103 is composed of an LCD (Liquid Crystal Display) or the like, and according to a display control signal from the control unit 101, for example, a live view image showing a state imaged by the imaging unit 106 or an imaging unit according to a shooting instruction. The image or the like acquired by 106 is displayed.

The communication unit 104 is an interface for wirelessly or wired data communication with an external device.

The operation unit 105 is provided with various function buttons such as a shutter key and a transparent touch panel stacked on the display unit 103, and receives and operates each button press input, touch panel touch, slide operation, etc. by the user. The information is output to the control unit 101.

The image pickup unit 106 is configured to include an image pickup lens 106a, an image pickup element composed of an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor) (not shown), an A / D conversion circuit, and the like. The optical image that has passed through the 106a is converted into a two-dimensional image signal by the image sensor, and the image data (RGB image data) of the subject is acquired.

The first image processing unit 107 is composed of a digital value luminance signal Y and color difference signals Cb and Cr after performing color process processing including pixel interpolation processing and γ correction processing on the RGB data acquired by the imaging unit 106. It is converted into image data (image data of each component of Y, U, and V).

The image memory 108 is a recording means that is composed of, for example, a flash memory or the like, and records image data that has been imaged by the image capturing unit 106 and processed by the first image processing unit 107.

The analysis unit 109 analyzes the image data (subject image) of the subject acquired by the imaging unit 106 and processed by the first image processing unit 107 when the correction mode according to the present invention is set in advance. The texture information (skin surface information) of the subject's skin is acquired and output to the second image processing unit 110. Further, the analysis unit 109 analyzes the image area around the singular region (for example, the region where foreign matter such as a breakout exists) different from the surrounding skin in the image data processed by the first image processing unit 107. The second image processing unit 110 generates the texture protection maps M11 and M12 and the shadow protection maps M21 and M22 (see FIG. 2) used when the singular region is corrected. Here, the texture protection maps M11 and M12 and the shading protection maps M21 and M22 protect the image areas to be excluded from the correction target among the image areas around the singular area when the correction process is applied to the singular area. It is mask data for (masking).
The analysis unit 109 functions as a texture information acquisition means, an edge component acquisition means, a detection means, an analysis means, and a correction means.

The second image processing unit 110 has the skin protection maps M11, M12, generated by the analysis unit 109 with respect to the singular region of the image data acquired by the imaging unit 106 and processed by the first image processing unit 107. And correction processing such as smoothing processing is performed using the protection maps M21 and M22 for shading, and the corrected image is generated by adding texture information to the processing result and synthesizing it with the original image data (original image). ..
The second image processing unit 110 functions as a processing means and a processing means.

The analysis unit 109 and the second image processing unit 110 are executed in cooperation with the control unit 101 and the program stored in the storage unit 102, but may be realized by dedicated hardware. ..

The lighting control unit 111 (driver) is connected to each of the LEDs (Light Emitting Diodes) of the light emitting unit 112, and controls the amount of light and the lighting / extinguishing of the light emitting unit 112 according to the instruction from the control unit 101.

The light emitting unit 112 is composed of an LED or the like, and irradiates the subject with light.

[Operation of imaging device 1]
Next, the operations of the analysis unit 109 and the second image processing unit 110 of the image pickup apparatus 1 will be described.
FIG. 2 is a diagram showing an outline of processing executed by the analysis unit 109 and the second image processing unit 110. G1 of FIG. 2 is a diagram schematically showing a Y component image (luminance component image) of a singular region in the original image and its surroundings, using the image data processed by the first image processing unit 107 as the original image. .. In G1 of FIG. 2, the diagonal line 51 represents a singular region, the dot 52 represents the texture of the skin, and the shaded portion 53 represents the shadow applied to the skin. Some of these shadows are caused by unevenness caused by facial organs and wrinkles, while others are caused by hair, eyelashes, eyebrows, or their shadows. In FIG. 2, the texture 52 exists in the entire G1. In addition to the singular region 51 (an image of a foreign substance such as a breakout), a shadow 53 exists around the singular region.

In the original image corresponding to the Y component image G1, if the correction process for making the singular region 51 inconspicuous is performed, the surface condition (texture) of the skin in the region to be corrected disappears, resulting in an unnatural image. It ends up. For example, G2 in FIG. 2 shows an example in which the texture of the singular region disappears. When the area to be corrected is expanded to the periphery, the shadows around the singular region disappear, resulting in a more unnatural image.

Therefore, in the present embodiment, first, the analysis unit 109 generates a Y component image G1 in the singular region and its peripheral portion from the original image. Next, as shown in FIG. 2, the Y component image G1 is analyzed, and based on the analysis results, the texture protection map M11 (large), the texture protection map M12 (small), the shadow protection map M21 (large), Generates a shadow protection map M22 (small). Then, the second image processing unit 110 uses the generated texture protection maps M11 and M12 and shadow protection maps M21 and M22 to perform correction processing to make the singular region inconspicuous in the original image. In FIG. 2, the image after the correction process is shown by G2. Further, the analysis unit 109 analyzes the Y component image G1 before the correction to acquire information (texture information) indicating the surface state (texture) of the subject's skin, and the second image processing unit 110 analyzes it. In part 109, the texture information acquired from the information before the correction is added to the image G2 after the correction process to generate the corrected image G3 in which the texture and the shadow are not erased.

Here, the method for detecting the singular region to be corrected from the original image is not particularly limited, and any method may be used. For example, a region designated by the user by the operation unit 105 may be detected as a singular region from the face area of the original image displayed on the display unit 103. Alternatively, the face region is detected from the original image, the image of the detected face region is cut out, and the V component image data in the contour region of the face region is a pixel different from the surroundings from the data masking the eyes, nose, and mouth. A region of values (suddenly occurring pixel values) may be extracted and a singular region may be detected based on the size of the region.

FIG. 3 is a block diagram showing a functional configuration of the analysis unit 109 and the second image processing unit 110.
As shown in FIG. 3, the analysis unit 109 includes a skin analysis unit 20, a protection map data setting unit 21, a low frequency data generation unit 22, an edge information generation unit 23, an area division unit 24, and a shadow protection map generation unit 25. It is configured to include a skin protection map generation unit 26. The second image processing unit 110 includes a protection map boundary reliability determination unit 27, a correction processing unit 28, a transmission composition processing unit 29, and a composition processing unit 30. Hereinafter, the processing of the analysis unit 109 and the second image processing unit 110 will be described with reference to FIG.

First, the analysis unit 109 generates a Y component image composed of a Y (luminance) component from an original image having each component of YUV, and further centers a singular region detected from the face region of the Y component image. A rectangular area having a predetermined size included in the above is cut out and input to the skin analysis unit 20 and the edge information generation unit 23.

The skin analysis unit 20 analyzes the Y component image in the input rectangular region to detect the image region of the pores, and the pore size data (size) and the contrast data (con) of the pore portion in the image region of the pores. To get. Then, for the acquired pore size data, the average data (size_ave) of the pore size data and the standard deviation data (size_sd) of the pore size data are calculated and output to the protection map data setting unit 21 and the low frequency data generation unit 22. To do. Further, with respect to the acquired contrast data of the pore portion, the average data (con_ave) of the contrast data of the pore portion and the standard deviation data (con_sd) of the contrast data of the pore portion are calculated and output to the texture protection map generation unit 26. ..

When correcting the singular region, the protection map data setting unit 21 has two types of elliptical protection exclusion regions 61 and 62 (see FIG. 2) having different sizes so that the other regions are not corrected. The protection maps M1 and M2 composed of the mask data having the above are generated and output to the area division unit 24 and the texture protection map generation unit 26. The singular region is a substantially circular region where breakouts such as acne and undulations conspicuous from the surroundings are present, and the protection exclusion regions 61 and 62 of the protection maps M1 and M2 have an elliptical shape surrounding the singular region.
For example, the protection map data setting unit 21 first reads out a basic protection map having a basic ellipse-sized protection exclusion area stored in the storage unit 102. Next, the radius of the pore size is calculated based on the sum of the average data (size_ave) of the pore size data calculated by the skin analysis unit 20 and the standard deviation data (size_sd) of the pore size data, and the calculated radius is used as the basis. A protection map M1 having a protection exclusion area added to the radius of the ellipse of the protection map of the above, and a protection map M2 having an ellipse-sized protection exclusion area obtained by subtracting the obtained radius from the radius of the ellipse of the basic protection map are generated.
Since the pore size generally has little individual difference, two types of protection maps M1 and M2 may be generated in advance and stored in the storage unit 102. Further, in the present embodiment, two types of protection maps M1 and M2 are generated as templates, but the present invention is not limited to this, and a plurality of types may be generated depending on the size. The larger the size, the more accurate the correction of the singular region becomes possible.

The low frequency data generation unit 22 uses a value obtained by adding the average data (size_ave) of the pore size data input from the skin analysis unit 20 and the standard deviation data (size_sd) of the pore size data as a parameter for the Y component image. The edge component in the Y component image is removed by smoothing using the average filter, and low frequency data (low frequency band data) showing a rough shadow state is generated. This makes it possible to remove the effects of pores from the shadows of the skin. The low frequency data generation unit 22 outputs the generated low frequency data to the edge information generation unit 23 and the area division unit 24.

The edge information generation unit 23 generates positive (positive value) difference data and negative (negative value) difference data between the low frequency data generated by the low frequency data generation unit 22 and the Y component image, respectively. Generate / acquire. As a result, the texture in the frequency band higher than the pores can be extracted. The edge information generation unit 23 outputs the difference data in the positive direction and the difference data in the negative direction to the synthesis processing unit 30 as edge information (texture information) corresponding to the texture.

The area division unit 24 is an annular region between the ellipse of the protection map M1 and the ellipse of the protection map M2 input from the protection map data setting unit 21 based on the low frequency data input from the low frequency data generation unit 22. In, the shadow area that affects the original shadow expression, such as the shadow disappearing when the correction is applied, is specified, and this is divided from other areas.

Specifically, the region division unit 24 is a pixel value in an annular region where the size difference between the ellipse of the protection map M1 and the ellipse of the protection map M2 when the large and small protection maps M1 and M2 are superimposed on the low frequency data. A histogram of (pixel value of low frequency data) is calculated, the mode value (peak) is obtained from this histogram, and the area of the pixel value up to the range where the minimum value exists around the determined mode value is used as the shadow area. Divide with other areas. Then, the information of the shadow area is output to the shadow protection map generation unit 25.

The shading protection map generation unit 25 adds the shadow area to the protection target area by the protection maps M1 and M2 based on the information of the shadow area input from the area division unit 24, and is for shading as shown in FIG. Generate protection maps M11 and M12. As a result, although it is outside the singular region, if it is processed using the protection maps M1 and M2, it will enter the protection exclusion region, and some or all (in many cases, part) information will be lacking, resulting in an unnatural shadow. It is possible to add the entire area to the protection target. That is, by causing the correction processing unit 28 to perform the correction processing using the shadow protection maps M11 and M12, it is possible to perform the correction processing excluding the shadow areas included in the protection exclusion areas 61 and 62. The shadow protection map generation unit 25 outputs the generated shadow protection maps M11 and M12 to the correction processing unit 28 and the protection map boundary reliability determination unit 27.

The skin protection map generation unit 26 adds the average data (con_ave) of the contrast data of the pore portion acquired from the skin analysis unit 20 and the standard deviation data (con_sd) of the contrast data of the pore portion, and has a higher intensity. A pixel having a positive edge (an edge having a higher brightness than the surroundings) and a pixel having a negative edge (an edge having a lower brightness than the surroundings) are obtained. Next, in each of the protection maps M1 and M2, the pixels of the positive direction edge and the pixels of the negative direction edge in the pixel area on the boundary with the protection exclusion area are added to the protection target area of the protection maps M1 and M2 to give a texture. For protection maps M21 and M22 are generated. By doing so, it is possible to exclude pixels having an edge strength equal to or higher than the pores existing at the boundary with the unprotected area of the protection map from the correction processing target, and eliminate the unnaturalness remaining in the corrected image. Can be done. The texture protection map generation unit 26 outputs the generated texture protection maps M21 and M22 to the correction processing unit 28 and the protection map boundary reliability determination unit 27.

The protection map boundary reliability determination unit 27 includes a large-sized protection map in which two large-sized protection maps M11 and M21 generated by the shadow protection map generation unit 25 and the texture protection map generation unit 26 are combined, and 2 The reliability of each of the corrected image areas is determined by using each of the small size protection maps M12 and M22 combined. Then, in the transmission synthesis processing described later, a map that increases the influence of the pixel with higher reliability and lowers the influence of the pixel with lower reliability is generated and output to the transmission synthesis processing unit 29. Before the correction process, the reliability of each image area after the correction process may be determined.

Here, the image area where the protection target areas of the large protection map and the protection target area of the small protection map overlap, and the image area where both protection exclusion areas overlap, can be corrected by using either the large or small protection map. Since the processing results are the same, the reliability is the same. Therefore, here, the reliability is mainly determined for each pixel of the image area sandwiched between the boundary between the protection exclusion area of the large protection map and the protection exclusion area of the small protection map. In the present embodiment, it is determined that the reliability is higher when the processing can be performed in a more elliptical region. For example, the boundary sharing rate, the area sharing rate, the boundary regression analysis match rate, etc. are calculated for each pixel, and the reliability of the large and small protection maps is determined based on the calculation results.

Here, the boundary sharing ratio matches the length of the boundary between the large-sized protection map and the small-sized protection map with the protection exclusion area in each, and the boundary between the large and small protection maps and the protection exclusion area (sharing). It is the ratio of the length. If the protection exclusion area (that is, the correction target area) in the large and small protection maps is elliptical, the boundary with the protection exclusion area of the large and small protection maps is not shared. Therefore, the higher the ratio of the shared portion, the lower the reliability.
The area sharing ratio is the ratio of the area of the protected edge (the area added to the protected area) to the area of the protected map in each of the large-sized protected map and the small-sized protected map. The one with a small proportion of protected edges (the one with a low sharing rate) has higher reliability.
Boundary regression analysis coincidence rate is the number of boundary values within the range of regression analysis results when regression analysis is performed on the boundary with the protection exclusion area in each of the large size protection map and the small size protection map. This is the value shown. Boundary regression analysis If the concordance rate is low, reduce the reliability.
The above-mentioned reliability calculation method is an example, and the concept and specific method of reliability determination are not limited to this.

The correction processing unit 28 has two large-sized protection maps M11 generated by the shadow protection map generation unit 25 and the texture protection map generation unit 26 in a rectangular region including the singular region of the image of each YUV component of the original image. A large-sized protection map including M21 is applied, and correction processing is performed to make the boundary between the image area of the protection exclusion area surrounded by the protection map and the other image areas and the peculiar state inconspicuous. Similarly, the two small-sized protection maps M12 and M22 generated by the shadow protection map generation unit 25 and the texture protection map generation unit 26 are combined with the rectangular area including the singular region of the image of each YUV component of the original image. A small-sized protection map is applied, and correction processing is performed to make the boundary and peculiar state between the image area of the protection exclusion area surrounded by the protection map and the other image areas inconspicuous.
The above correction process mainly uses a smoothing process (a process for smoothing the skin condition and a process for giving a feeling of transparency), but in addition to this, a color correction process (a process for correcting a more natural skin color) is also used. It may be used.
The correction processing unit 28 outputs the YUV component image corrected by the large-sized protection map and the YUV component image corrected by the small-sized protection map to the transmission synthesis processing unit 29.

Based on the map output from the protection map boundary reliability determination unit 27, the transmission synthesis processing unit 29 corrected the YUV component images corrected by the large size protection map and the correction processing by the small size protection map. Each YUV component image is transparently combined by setting the α value so that the influence of the pixel judged to be highly reliable is high for each pixel targeted for correction processing by the two protection maps. Output.

The composition processing unit 30 adds the edge information corresponding to the texture input from the edge information generation unit 23 to the correction processing target area of the YUV component image transmitted and synthesized by the transmission composition processing unit 29, and adds the texture state. The corrected image (corrected image) and the original image are α-blended at a predetermined blend ratio, and the corrected image is output.

By performing the above processing in the analysis unit 109 and the second image processing unit 110, as shown in the corrected image G3 of FIG. 2, while maintaining the state of shadows and texture due to the influence of facial irregularities and wrinkles, Image processing can be performed to make singular areas such as breakouts inconspicuous.

As described above, according to the image pickup apparatus 1, the analysis unit 109 acquires the subject image and acquires the surface state information of the subject from the acquired subject image. The second image processing unit 110 processes the subject image based on the surface state information acquired by the analysis unit 109.
Therefore, it is possible to obtain a subject image that retains the surface state of the subject based on the surface state information of the subject. Further, when the surface state of the subject is lost by correcting the subject image, the corrected portion can be made inconspicuous based on the surface state information.

Further, for example, the analysis unit 109 acquires an edge component of a predetermined frequency band in the image region of the subject image, and acquires the acquired edge component as surface state information.
Therefore, it is possible to obtain a subject image that retains the surface state of the subject composed of edges in a predetermined frequency band.

Further, for example, the second image processing unit 110 performs processing on the subject image to which the predetermined processing has been performed based on the acquired surface state information. Therefore, it is possible to add the surface state of the subject to the subject image that has been subjected to the predetermined processing.

Further, for example, the second image processing unit 110 performs processing based on the surface state information acquired by the analysis unit 109 when a process of making a singular region in the subject image inconspicuous, for example, a smoothing process is performed. Give. Therefore, even if the surface state of the subject is lost by performing the process of making the singular region inconspicuous in the subject image, it is possible to make the processed portion inconspicuous by performing the processing based on the surface state.

Further, for example, the second image processing unit 110 uses protection maps of a plurality of sizes imitating a shape surrounding the singular region, and performs predetermined processing in the image region including the singular region surrounded by the protection map, respectively. Obtains the reliability information of each image area that has been subjected to predetermined processing using each of the protection maps of various sizes, and based on the acquired reliability information, predetermined processing by the plurality of protection maps. The results of are synthesized with different degrees of influence. Therefore, the processing result processed by the more reliable protection map can be reflected in the subject image.

Further, for example, the analysis unit 109 analyzes the luminance information of the subject image and corrects the shapes of a plurality of types of protection maps based on the analysis result. Therefore, it is possible to process a peculiar region of the subject image by using a mask-shaped protection map according to the subject image.

Further, since the analysis unit 109 and the second image processing unit 110 of the imaging device 1 perform the above processing on the subject image including the human skin region, it is possible to obtain a subject image that retains the texture of the human skin.

Further, the second image processing unit 110 of the image pickup apparatus 1 corrects the singular region in the subject image so as to make it inconspicuous with respect to the periphery, and makes the corrected singular region the surface state of the subject in the subject image before the correction. Perform additional processing. Therefore, when the correction for making the singular region of the subject image inconspicuous is performed, the corrected portion can be made inconspicuous.

The description content in the above embodiment is a preferable example of the present invention, and is not limited thereto.

For example, in the above embodiment, the case where the image processing apparatus of the present invention is provided in the image pickup apparatus has been described as an example, but the image processing apparatus of the present invention may be a separate body from the image pickup apparatus. For example, it may be an image processing device that performs the processing described with reference to FIGS. 2 to 3 on the image data received from the image pickup device. Further, the original image is not limited to the recorded image, and may be a live view image that is sequentially captured by the imaging unit 106 and sequentially displayed on the display unit 103. By doing so, it is possible to provide the user with a subject image (face) that always reflects the corrected result.

Further, in the above embodiment, the case where the image data of YUV is used has been described as an example, but the type of the image data is not particularly limited.

Further, in the above embodiment, the case of protecting both the texture and the shadow has been described as an example, but only one of them may be protected.
When protecting only the texture, multiple skin protection maps with different sizes of the protection exclusion area are generated based on the Y component image of the original image, and the YUV component of the original image is used using each of the multiple skin protection maps. The image is corrected, the reliability of each of the corrected image areas is obtained using each of the multiple protection maps for texture, and the α value is calculated for each pixel based on the reliability for multiple textures. Transparent composition of a plurality of corrected images is performed using each of the protection maps. Further, the texture information acquired from the Y component image is added to the transparently synthesized image and combined with the YUV component image of the original image at a predetermined α value.
When protecting only the shading, multiple shading protection maps with different sizes of the protection exclusion area are generated based on the Y component image of the original image, and the YUV component of the original image is used using each of the multiple shading protection maps. The image is corrected, the reliability of each of the corrected image areas is obtained using each of the multiple shadow protection maps, and the α value is calculated for each pixel based on the reliability for multiple shadows. Transparent composition of a plurality of corrected images is performed using each of the protection maps. Then, the transmitted-combined image is combined with the YUV component image of the original image at a predetermined α value.

Further, for example, in the above description, an example in which a hard disk, a non-volatile memory of a semiconductor, or the like is used as a computer-readable medium of the program according to the present invention has been disclosed, but the present invention is not limited to this example. As another computer-readable medium, a portable recording medium such as a CD-ROM can be applied. A carrier wave is also applied as a medium for providing data of the program according to the present invention via a communication line.

In addition, the detailed configuration and detailed operation of each device constituting the image pickup device can be appropriately changed without departing from the spirit of the invention.

Although some embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and the equivalent scope thereof.
The inventions described in the claims originally attached to the application of this application are added below. The item number of the claim described in the appendix is the scope of the claims originally attached to the application of this application.
[Additional Notes]
<Claim 1>
An acquisition method for acquiring a subject image and
The texture information acquisition means for acquiring the texture information of the subject from the subject image acquired by the acquisition means, and
A processing means for processing the subject image based on the texture information acquired by the texture information acquisition means, and
An image processing device characterized by comprising.
<Claim 2>
An edge component acquisition means for acquiring an edge component of a predetermined frequency band in an image region of the subject image is provided.
The image processing apparatus according to claim 1, wherein the texture information acquisition means acquires the edge component acquired by the edge component acquisition means as the texture information.
<Claim 3>
Further provided with a processing means for performing a predetermined image processing on the subject image,
The image according to claim 1 or 2, wherein the processing means performs processing on a subject image that has been subjected to predetermined processing by the processing means based on the texture information acquired by the texture information acquisition means. Processing equipment.
<Claim 4>
Further, a detection means for detecting a singular region different from the periphery in the subject image is provided.
The image processing apparatus according to claim 3, wherein the predetermined process is a process for making the singular region inconspicuous in the subject image.
<Claim 5>
The processing means
Using mask data of a plurality of sizes that imitates the shape surrounding the singular region, the predetermined processing is performed on the image region including the singular region surrounded by the mask data, respectively.
A reliability information acquisition means for acquiring reliability information of each of the image regions subjected to the predetermined processing using each of the mask data of a plurality of sizes.
Based on the reliability information acquired by the reliability information acquisition means, a synthesis means for synthesizing the results of predetermined processing in each mask data processed by the processing means with different degrees of influence.
4. The image processing apparatus according to claim 4, further comprising.
<Claim 6>
An analysis means for analyzing the luminance information of the subject image and
Based on the analysis result by the analysis means, the correction means for correcting the shape of the plurality of types of mask data, and the correction means.
5. The image processing apparatus according to claim 5, further comprising.
<Claim 7>
The image processing apparatus according to claim 4, wherein the predetermined process is a smoothing process.
<Claim 8>
The image processing apparatus according to any one of claims 1 to 7, wherein the subject image is an image including a human skin region.
<Claim 9>
An acquisition method for acquiring a subject image and
A detection means for detecting a singular region different from the surroundings from the subject image,
A correction means for correcting the singular region detected by the detection means so as to make it inconspicuous with respect to the periphery.
A control means for controlling the singular region corrected so as to be inconspicuous by the correction means so as to perform processing for adding the surface state of the subject in the subject image before the correction by the correction means.
An image processing device characterized by being equipped with.
<Claim 10>
An acquisition method for acquiring a subject image and
A detection means for detecting a singular region different from the periphery in the subject image acquired by the acquisition means, and
Using mask data of a plurality of sizes imitating a shape surrounding the singular region detected by the detection means, a processing means for performing predetermined processing on the image region including the singular region surrounded by the mask data, respectively.
A reliability information acquisition means for acquiring reliability information of each of the image regions subjected to the predetermined processing using each of the mask data of a plurality of sizes.
Based on the reliability information acquired by the reliability information acquisition means, a synthesis means for synthesizing the results of predetermined processing in each mask data processed by the processing means with different degrees of influence.
An image processing device characterized by being equipped with.
<Claim 11>
This is an image processing method in an image processing device.
The process of acquiring the subject image and
The process of acquiring the texture information of the subject from the acquired subject image and
The process of processing the subject image based on the acquired texture information, and
An image processing method characterized by including.
<Claim 12>
This is an image processing method in an image processing device.
The process of acquiring the subject image and
A step of detecting a singular region different from the surroundings from the subject image, and
A step of correcting the detected singular region so as to be inconspicuous with respect to the periphery, and
An image processing method comprising a step of adding a process of adding a surface state of a subject in the subject image before the correction to the singular region corrected so as to be inconspicuous.
<Claim 13>
This is an image processing method in an image processing device.
The process of acquiring the subject image and
A step of detecting a singular region different from the periphery in the acquired subject image, and
Using mask data of a plurality of sizes that imitates the shape surrounding the detected singular region, a step of performing predetermined processing on the image region including the singular region surrounded by the mask data, respectively.
A step of acquiring reliability information of each of the image regions subjected to the predetermined processing by using each of the mask data of a plurality of types of sizes, and
Based on the acquired reliability information, a synthesis means for synthesizing the results of the predetermined processing in each of the mask data with different degrees of influence, and
An image processing method characterized by including.
<Claim 14>
Computer,
Acquisition method for acquiring subject images,
A texture information acquisition means for acquiring the texture information of a subject from the subject image acquired by the acquisition means,
A processing means for processing the subject image based on the texture information acquired by the texture information acquisition means,
A program to function as.
<Claim 15>
Computer,
Acquisition method for acquiring subject images,
A detection means for detecting a singular region different from the surroundings from the subject image,
A correction means for correcting a singular region detected by the detection means so as to make it inconspicuous with respect to the periphery.
A control means for controlling a singular region corrected so as to be inconspicuous by the correction means so as to perform processing for adding the surface state of the subject in the subject image before the correction by the correction means.
A program to function as.
<Claim 16>
Computer,
Acquisition method for acquiring subject images,
A detection means for detecting a singular region different from the periphery in the subject image acquired by the acquisition means,
A processing means for performing predetermined processing in an image region including the singular region surrounded by the mask data using mask data of a plurality of sizes imitating a shape surrounding the singular region detected by the detecting means.
A reliability information acquisition means for acquiring reliability information of each of the image regions subjected to the predetermined processing using each of the mask data of a plurality of sizes.
A synthesis means for synthesizing the results of predetermined processing in each mask data processed by the processing means with different degrees of influence based on the reliability information acquired by the reliability information acquisition means.
A program to function as.

1 Image pickup device 101 Control unit 102 Storage unit 103 Display unit 104 Communication unit 105 Operation unit 106 Image pickup unit 106a Image pickup lens 107 First image processing unit 108 Image memory 109 Analysis unit 20 Skin analysis unit 21 Protection map data setting unit 22 Low frequency Data generation unit 23 Edge information generation unit 24 Area division unit 25 Shading protection map generation unit 26 Texture protection map generation unit 27 Protection map boundary reliability judgment unit 110 Second image processing unit 28 Correction processing unit 29 Transmission synthesis processing unit 30 Synthesis processing unit 111 Lighting control unit 112 Light emitting unit

Claims (16)

An acquisition method for acquiring a subject image and
The texture information acquisition means for acquiring the texture information of the subject from the subject image acquired by the acquisition means, and
A processing means for processing the subject image based on the texture information acquired by the texture information acquisition means, and
An image processing device characterized by comprising. An edge component acquisition means for acquiring an edge component of a predetermined frequency band in an image region of the subject image is provided.
The image processing apparatus according to claim 1, wherein the texture information acquisition means acquires the edge component acquired by the edge component acquisition means as the texture information. Further provided with a processing means for performing a predetermined image processing on the subject image,
The image according to claim 1 or 2, wherein the processing means performs processing on a subject image that has been subjected to predetermined processing by the processing means based on the texture information acquired by the texture information acquisition means. Processing equipment. Further, a detection means for detecting a singular region different from the periphery in the subject image is provided.
The image processing apparatus according to claim 3, wherein the predetermined process is a process for making the singular region inconspicuous in the subject image. The processing means
Using mask data of a plurality of sizes that imitates the shape surrounding the singular region, the predetermined processing is performed on the image region including the singular region surrounded by the mask data, respectively.
A reliability information acquisition means for acquiring reliability information of each of the image regions subjected to the predetermined processing using each of the mask data of a plurality of sizes.
Based on the reliability information acquired by the reliability information acquisition means, a synthesis means for synthesizing the results of predetermined processing in each mask data processed by the processing means with different degrees of influence.
4. The image processing apparatus according to claim 4, further comprising. An analysis means for analyzing the luminance information of the subject image and
Based on the analysis result by the analysis means, the correction means for correcting the shape of the plurality of types of mask data, and the correction means.
5. The image processing apparatus according to claim 5, further comprising. The image processing apparatus according to claim 4, wherein the predetermined process is a smoothing process. The image processing apparatus according to any one of claims 1 to 7, wherein the subject image is an image including a human skin region. An acquisition method for acquiring a subject image and
A detection means for detecting a singular region different from the surroundings from the subject image,
A correction means for correcting the singular region detected by the detection means so as to make it inconspicuous with respect to the periphery.
A control means for controlling the singular region corrected so as to be inconspicuous by the correction means so as to perform processing for adding the surface state of the subject in the subject image before the correction by the correction means.
An image processing device characterized by being equipped with. An acquisition method for acquiring a subject image and
A detection means for detecting a singular region different from the periphery in the subject image acquired by the acquisition means, and
Using mask data of a plurality of sizes imitating a shape surrounding the singular region detected by the detection means, a processing means for performing predetermined processing on the image region including the singular region surrounded by the mask data, respectively.
A reliability information acquisition means for acquiring reliability information of each of the image regions subjected to the predetermined processing using each of the mask data of a plurality of sizes.
Based on the reliability information acquired by the reliability information acquisition means, a synthesis means for synthesizing the results of predetermined processing in each mask data processed by the processing means with different degrees of influence.
An image processing device characterized by being equipped with. This is an image processing method in an image processing device.
The process of acquiring the subject image and
The process of acquiring the texture information of the subject from the acquired subject image and
The process of processing the subject image based on the acquired texture information, and
An image processing method characterized by including. This is an image processing method in an image processing device.
The process of acquiring the subject image and
A step of detecting a singular region different from the surroundings from the subject image, and
A step of correcting the detected singular region so as to be inconspicuous with respect to the periphery, and
An image processing method comprising a step of adding a process of adding a surface state of a subject in the subject image before the correction to the singular region corrected so as to be inconspicuous. This is an image processing method in an image processing device.
The process of acquiring the subject image and
A step of detecting a singular region different from the periphery in the acquired subject image, and
Using mask data of a plurality of sizes that imitates the shape surrounding the detected singular region, a step of performing predetermined processing on the image region including the singular region surrounded by the mask data, respectively.
A step of acquiring reliability information of each of the image regions subjected to the predetermined processing by using each of the mask data of a plurality of types of sizes, and
Based on the acquired reliability information, a synthesis means for synthesizing the results of the predetermined processing in each of the mask data with different degrees of influence, and
An image processing method characterized by including. Computer,
Acquisition method for acquiring subject images,
A texture information acquisition means for acquiring the texture information of a subject from the subject image acquired by the acquisition means,
A processing means for processing the subject image based on the texture information acquired by the texture information acquisition means,
A program to function as. Computer,
Acquisition method for acquiring subject images,
A detection means for detecting a singular region different from the surroundings from the subject image,
A correction means for correcting a singular region detected by the detection means so as to make it inconspicuous with respect to the periphery.
A control means for controlling a singular region corrected so as to be inconspicuous by the correction means so as to perform processing for adding the surface state of the subject in the subject image before the correction by the correction means.
A program to function as. Computer,
Acquisition method for acquiring subject images,
A detection means for detecting a singular region different from the periphery in the subject image acquired by the acquisition means,
A processing means for performing predetermined processing in an image region including the singular region surrounded by the mask data using mask data of a plurality of sizes imitating a shape surrounding the singular region detected by the detecting means.
A reliability information acquisition means for acquiring reliability information of each of the image regions subjected to the predetermined processing using each of the mask data of a plurality of sizes.
A synthesis means for synthesizing the results of predetermined processing in each mask data processed by the processing means with different degrees of influence based on the reliability information acquired by the reliability information acquisition means.
A program to function as.