JP2024013339A - Image processing device, image processing method, image processing program and recording medium - Google Patents

Abstract

To provide an image processing device which can visualize a living body surface such as a skin surface of a human with high accuracy in comparison with a conventional technique.SOLUTION: An image processing device according to the present invention comprises: an image sensor which images a surface of a processing object to generate polarization image data and non-polarization image data; a first calculation unit which calculates and outputs a ratio of luminance or a difference of luminance for each pixel between the polarization image data and the non-polarization image data; a background extraction processing unit which extracts and outputs background image data related to a background image from image data being any of (1) image data including the ratio of luminance for each pixel from the first calculation unit, (2) image data including the difference of luminance for each pixel from the first calculation unit, (3) the polarization image data and (4) the non-polarization image data; and a second calculation unit which subtracts the background image data from the background extraction processing unit from the image data including the ratio of luminance or the difference of luminance for each pixel from the first calculation unit to generate image data including the difference for each pixel.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image processing device, an image processing method, an image processing program, and a recording medium storing the image processing program, for example, for visualizing the uneven state of a living body surface.

For example, from a medical perspective, the human skin is an organ that protects the body from external factors and maintains homeostasis, but evaluation of its properties is also important from a cosmetic perspective. Skin conditions can be expressed in various terms, such as the appearance of makeup, dullness, or translucence, but unlike general dermatology, which diagnoses pathological skin findings, medical treatment uses skin that is included in the normal category. It is difficult to quantitatively capture subtle changes in skin conditions from a cosmetic perspective, and a practical method has not yet been established.

When human skin is observed under magnification, a pattern of fine irregularities called "wrinkles" or "texture" can be seen on the surface.

Special Publication No. 2010-515489

Feichtinger, Hans, G. et al., "Gabor analysis and algorithms: Theory and applications," Springer Science & Business Media, 2012.

On the surface of human skin, there are many wrinkles and textures that are grid-like grooves with a depth of about 10 to 20 μm. Conventionally, visual confirmation methods have been used to easily obtain the distribution of wrinkles and texture, but due to accuracy and cost issues, it is possible to automatically obtain wrinkle and texture distribution information with high precision and at low cost. A method that can be used to obtain such data is needed. To easily obtain skin information, it is common to photograph the skin surface with an RGB camera, but since wrinkles and textured areas are very similar in color to their surroundings, only RGB images (for example, non-polarized images) are required. It is difficult to visualize accurately using

For example, a conventional imaging device has been disclosed in order to accurately visualize the skin surface (see, for example, Patent Document 1). The non-invasive imaging system and method includes an illumination source including an incident light source that directs light onto the skin and a detector for detecting the degree of deflection of the light reflected from the skin. The system and method for determining skin condition is also based on one aspect of the polarization of the reflected light. That is, it is disclosed that the conventional imaging device determines the skin condition based on the polarization mode of the reflected light.

However, even in this conventional imaging device, there is a problem in that it is not possible to visualize the uneven state of the surface of the object to be photographed, such as the state of human skin, with high precision.

A first object of the present invention is to solve the above-mentioned problems, and provide image processing that is capable of visualizing the unevenness of the surface of an object to be photographed, such as a biological surface such as a human skin surface, with higher accuracy than conventional techniques. An object of the present invention is to provide an apparatus, an image processing method, an image processing program, and a recording medium storing the image processing program.

A second object of the present invention is to provide an image that can analyze the uneven state of the surface of an object to be photographed, such as a biological surface such as a human skin surface, using predetermined feature quantities with higher accuracy than the conventional technology. The present invention provides a processing device, an image processing method, an image processing program, and a recording medium storing the image processing program.

The image processing device according to the first aspect of the present invention includes:
an image sensor that images the surface of the object to be processed and generates polarized image data and non-polarized image data;
a first calculation unit that calculates and outputs a brightness ratio or brightness difference for each pixel between the polarized image data and the non-polarized image data;
(1) image data including a brightness ratio for each pixel from the first calculation unit;
(2) image data including a difference in brightness for each pixel from the first calculation unit;
(3) the polarization image data;
(4) the non-polarized image data;
a background extraction processing unit that extracts and outputs background image data related to a background image from any of the image data;
Subtracting the background image data from the background extraction processing unit from the image data including the brightness ratio or difference in brightness for each pixel from the first calculation unit to generate image data including the difference for each pixel. and a second calculation unit.

An image processing device according to a second aspect of the present invention is the image processing device according to the first aspect, comprising:
By using a Gapol filter on the image data from the post-processing unit and calculating Gapol features by changing at least one of the angle and the period, the angular distribution of brightness values and the brightness are calculated. The apparatus further includes a feature amount analysis unit that generates a periodic distribution of values and a luminance value distribution in a two-dimensional plane of angle and period.

Therefore, according to the image processing device and the like according to the first aspect of the present invention, the uneven state of the surface of the object to be photographed, such as the surface of a living body such as the surface of human skin, can be visualized with higher precision than in the prior art. can.

Further, according to the image processing apparatus and the like according to the second aspect of the present invention, the uneven state of the surface of the object to be photographed, such as the surface of a living body such as the surface of human skin, can be detected with higher precision than in the prior art. It can be analyzed using predetermined feature quantities.

1 is a block diagram showing a configuration example of an image processing system according to a first embodiment; FIG. 2 is a flowchart showing image processing performed by the image processing unit 12 of FIG. 1. FIG. This is a captured image showing a well-textured skin surface of a human (human body) that is the subject of image processing. This is a captured image of the skin of a human being (human body) that is the subject of image processing, and shows the surface of the skin with disordered texture. 2 is a captured image showing a polarized light image captured by camera 1 in FIG. 1. FIG. 2 is a captured image showing a non-polarized image captured by camera 1 in FIG. 1. FIG. This is a captured image showing a UV image captured by another ultraviolet (UV) camera. 4A is a captured image showing the polarization image of FIG. 4A. 4A is a captured image showing an image of the R (red) channel of the polarization image of FIG. 4A. 4A is a captured image showing an image of the G (green) channel of the polarization image of FIG. 4A. 4A is a captured image showing an image of the B (blue) channel of the polarization image of FIG. 4A. 4B is a captured image showing the non-polarized image of FIG. 4B. 4B is a captured image showing an image of the R (red) channel of the non-polarized image of FIG. 4B. 4B is a captured image showing a G (green) channel image of the non-polarized image of FIG. 4B. 4B is a captured image showing an image of the B (blue) channel of the non-polarized image of FIG. 4B. This is a captured image for considering the visualization of wrinkles or texture, and is a captured image showing a B channel image of a polarized light image. This is a captured image for considering the visualization of wrinkles or texture, and is a captured image showing a B channel image of a non-polarized light image. FIG. 3 is a conceptual diagram of image processing showing the image processing of FIG. 2 using a captured image and an image-processed image. This is an image according to Visualization Example 1, and is a captured image showing a polarization image to be processed. This is an image according to Visualization Example 1, and is a captured image showing a non-polarized image to be processed. This is an image according to Visualization Example 1, which is a visualized image after the image processing of FIGS. 2 and 8. This is an image related to Visualization Example 2, and is a captured image showing a polarized image to be processed. This is an image related to Visualization Example 2, and is a captured image showing a non-polarized image to be processed. This is an image according to Visualization Example 2, which is a visualized image after the image processing of FIGS. 2 and 8. This is an image related to Visualization Example 3, and is a captured image showing a polarized image to be processed. This is an image related to Visualization Example 3, and is a captured image showing a non-polarized image to be processed. This is an image according to Visualization Example 3, which is a visualized image after the image processing of FIGS. 2 and 8. This is an image related to Visualization Example 4, and is a captured image showing a polarization image to be processed. This is an image related to Visualization Example 4, and is a captured image showing a non-polarized image to be processed. This is an image related to Visualization Example 4, which is a visualized image after the image processing of FIGS. 2 and 8. This is an image related to Visualization Example 5, and is a captured image showing a polarized light image to be processed. This is an image according to Visualization Example 5, and is a captured image showing a non-polarized image to be processed. This is an image according to Visualization Example 5, which is a visualized image after the image processing of FIGS. 2 and 8. This is an image related to Visualization Example 6, and is a captured image showing a polarization image to be processed. This is an image related to Visualization Example 6, and is a captured image showing a non-polarized image to be processed. This is an image according to Visualization Example 6, which is a visualized image after the image processing of FIGS. 2 and 8. This is an image related to Visualization Example 7, and is a captured image showing a polarized image to be processed. This is an image related to Visualization Example 7, and is a captured image showing a non-polarized image to be processed. This is an image according to Visualization Example 7, which is a visualized image after the image processing of FIGS. 2 and 8. FIG. 2 is a block diagram illustrating a configuration example of an image processing system according to a second embodiment. 17 is a flowchart showing image processing performed by the image processing unit 12A of FIG. 16. 17 is a diagram for explaining the concept of a one-dimensional Gapore function used in the feature quantity analysis section 27 of the image processing section 12A in FIG. 16. FIG. 17 is a diagram for explaining the concept of a two-dimensional Gapore function used in the feature quantity analysis section 27 of the image processing section 12A in FIG. 16. FIG. This is an image showing analysis example 1 (directivity) of the feature amount analysis unit 27, and is a visualization processed image (DWE (Dermo Wrinkle Emphasis) image) before linear filtering. This is an image showing analysis example 1 (directivity) of the feature amount analysis unit 27, and is an analysis processed image after linear filtering. It is a graph showing analysis example 2 (directivity) of the feature quantity analysis unit 27, and is a graph showing the angular distribution of average luminance after linear filtering. It is a graph showing analysis example 2 (directivity) of the feature amount analysis unit 27, and is an analysis processed image at maximum brightness Lmax after linear filtering. It is a graph showing analysis example 2 (directivity) of the feature amount analysis unit 27, and is an analysis processed image when the minimum brightness Lmin is obtained after linear filtering. This is an image showing analysis example 3 (periodicity) by the feature quantity analysis unit 27, and is a visualization processed image (DWE image) before linear filtering. This is an image showing analysis example 3 (periodicity) of the feature quantity analysis unit 27, and is an analysis processed image after linear filtering. It is a graph showing analysis example 4 (periodicity) of the feature amount analysis unit 27, and is a graph showing a periodic distribution of average luminance after linear filtering. It is a graph showing analysis example 4 (periodicity) of the feature amount analysis unit 27, and is an analysis processed image at the time of maximum brightness Lmax after linear filtering. This is a graph showing analysis example 5 (directivity and periodicity) of the feature quantity analysis unit 27, and is a graph image in which brightness values are expressed in a contour heat map format in a two-dimensional plane of periodicity with respect to the angle of directionality. This is an image showing analysis example 5 (directivity and periodicity) of the feature amount analysis unit 27, and is an RGB image before processing (shown as a black and white image for patent drawings). This is an image showing analysis example 5 (directivity and periodicity) of the feature quantity analysis unit 27, and is a visualization processed image (DWE image) after visualization processing. This is a graph showing analysis example 5 (directivity and periodicity) of the feature quantity analysis unit 27, and is a graph image in which brightness values are expressed in a contour heat map format in a two-dimensional plane of periodicity with respect to the angle of directionality. It is a graph showing analysis example 5 (directivity and periodicity) of the feature amount analysis unit 27, and is a graph showing the angular distribution of average luminance after linear filtering. It is a graph showing analysis example 5 (directivity and periodicity) of the feature amount analysis unit 27, and is a graph showing a periodic distribution of average luminance after linear filtering. This is an image showing analysis example 6 (directivity and periodicity) of the feature quantity analysis unit 27, and is an RGB image before processing. This is an image showing analysis example 6 (directivity and periodicity) of the feature quantity analysis unit 27, and is a visualization processed image (DWE image) after visualization processing. This is a graph showing analysis example 6 (directivity and periodicity) of the feature quantity analysis unit 27, and is a graph image in which brightness values are expressed in a contour heat map format in a two-dimensional plane of periodicity with respect to the angle of directionality. It is a graph showing analysis example 6 (directivity and periodicity) of the feature amount analysis unit 27, and is a graph showing the angular distribution of average luminance after linear filtering. It is a graph showing analysis example 6 (directivity and periodicity) of the feature amount analysis unit 27, and is a graph showing a periodic distribution of average luminance after linear filtering. This is an image showing analysis example 7 (directivity and periodicity) of the feature quantity analysis unit 27, and is an RGB image before processing. This is an image showing analysis example 7 (directivity and periodicity) of the feature amount analysis unit 27, and is a visualization processed image (DWE image) after visualization processing. This is a graph showing analysis example 7 (directivity and periodicity) of the feature quantity analysis unit 27, and is a graph image in which brightness values are expressed in a contour heat map format in a two-dimensional plane of periodicity with respect to the angle of directionality. It is a graph showing analysis example 7 (directivity and periodicity) of the feature quantity analysis unit 27, and is a graph showing the angular distribution of average luminance after linear filtering. It is a graph showing analysis example 7 (directivity and periodicity) of the feature amount analysis unit 27, and is a graph showing a periodic distribution of average luminance after linear filtering. This is an image showing analysis example 8 (directivity and periodicity) of the feature amount analysis unit 27, and is an RGB image before processing. This is an image showing analysis example 8 (directivity and periodicity) of the feature quantity analysis unit 27, and is a visualization processed image (DWE image) after visualization processing. This is a graph showing analysis example 8 (directivity and periodicity) of the feature amount analysis unit 27, and is a graph image in which brightness values are expressed in a contour heat map format in a two-dimensional plane of periodicity with respect to the angle of directionality. It is a graph showing analysis example 8 (directivity and periodicity) of the feature quantity analysis unit 27, and is a graph showing the angular distribution of average luminance after linear filtering. It is a graph showing analysis example 8 (directivity and periodicity) of the feature amount analysis unit 27, and is a graph showing a periodic distribution of average luminance after linear filtering. This is an image showing analysis example 9 (directivity and periodicity) of the feature quantity analysis unit 27, and is an RGB image before processing. This is an image showing analysis example 9 (directivity and periodicity) of the feature quantity analysis unit 27, and is a visualization processed image (DWE image) after visualization processing. This is a graph showing analysis example 9 (directivity and periodicity) of the feature quantity analysis unit 27, and is a graph image in which brightness values are expressed in a contour heat map format in a two-dimensional plane of periodicity with respect to the angle of directionality. It is a graph showing analysis example 9 (directivity and periodicity) of the feature amount analysis unit 27, and is a graph showing the angular distribution of average luminance after linear filtering. It is a graph showing analysis example 9 (directivity and periodicity) of the feature amount analysis unit 27, and is a graph showing a periodic distribution of average luminance after linear filtering. FIG. 3 is a block diagram illustrating a configuration example of an image processing system according to a third embodiment.

Embodiments and modifications of the present invention will be described below with reference to the drawings. Note that the same or similar components are given the same reference numerals.

(Inventor's knowledge)
For example, in order to easily obtain skin information such as human skin, it is common to photograph the skin surface with an RGB camera, but since wrinkles and textured areas are very similar in color to their surroundings, RGB images ( For example, it has been difficult to visualize accurately using only non-polarized light images.

Therefore, in the present invention, wrinkles and texture are reduced by processing a close-up image of the skin taken using a dermo camera (registered trademark) that can simultaneously obtain polarized and non-polarized images, rather than a general RGB camera image. I tried to visualize it. A non-polarized image is a general RGB image that does not take polarization into account, while a polarized image is a photograph of the light that has passed through a polarizing plate by shining polarized light on the subject, suppressing the direct reflection of light on the skin surface, This image makes it easy to obtain information about the inner surface of the skin.

Wrinkles and texture, which are lattice-like grooves, reflect light in a complex manner, and the skin color differs slightly, so there is a slight difference in brightness between wrinkles and texture and other areas in non-polarized images. . On the other hand, since a polarized image mainly captures light scattered below the surface, information about wrinkles and texture is less likely to be included in the image. Therefore, the difference between these two images is mainly the presence or absence of direct reflected light on the skin surface, which corresponds to brightness information from reflected light at wrinkles and texture. In addition, although both polarized and non-polarized images contain RGB information, wrinkles and texture are more visible in B channel images than in G and R channels due to optical characteristics and skin color caused by the shape of microscopic grooves in wrinkles and texture. There is a large difference in brightness between parts. Therefore, only the B channel information is used for visualization processing.

In the following embodiments of the present invention, with the aim of more accurately diagnosing the health condition of the skin, we will use predetermined image processing to accurately visualize the wrinkles and texture of the skin, and then analyze the wrinkle and texture patterns. We devised an image processing device that can perform analysis.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration example of an image processing system according to the first embodiment. The image signal processing system shown in FIG. 1 includes a camera 1, an image processing device 10, and a display 2. Here, the image processing device 10 includes an image interface 11 having an image memory 11m, an image processing section 12, and an image interface 13 having an image memory 13m. The image processing unit 12 controls the operations of the brightness ratio calculation unit 21, the background extraction processing unit 22, the image memory 23, the brightness difference calculation unit 24, the post-processing unit 25, and the processing units 21 to 25. The control unit 20 is configured to include a control unit 20 that

In FIG. 1, camera 1 is, for example, "Dermo Camera" (registered trademark), which is a DZ-D100 skin observation and photographing camera manufactured by Casio Computer Co., Ltd., and is capable of capturing, for example, polarized and non-polarized images of the human skin surface. Then, the image data is outputted to the brightness ratio calculation section 21 via the image interface 11. Here, "Dermo Camera" (registered trademark) can capture ultraviolet (UV) images in addition to polarized and non-polarized images, but in the embodiment of the present invention, it is essential to capture UV images. isn't it. That is, the camera 1 only needs to be capable of capturing a polarized image and a non-polarized image of the object to be photographed, and the polarized image and the non-polarized image may be captured using separate image sensors or the like.

Here, polarized light imaging is generally characterized by suppressing light reflection and being able to capture the color and structure inside the skin just beneath the skin's thin layer, whereas non-polarized light imaging is generally It is unique in that it can photograph lesions on the surface of the skin, and UV images are generally used to capture peripheral areas such as hidden "spots" and blurred "moles" that do not come out with polarized light images. It is characterized by being able to take pictures of The "dermo camera" can capture these three types of images at the same angle of view.

The image interface 11 stores input image data in a built-in image memory 11m, converts the stored image data into a predetermined image format, and outputs the converted image data to the brightness ratio calculation section 21 at the next stage. Further, the image interface 13 stores input image data in a built-in image memory 13m, converts the stored image data into a predetermined image format, and outputs the converted image data to the next stage display 2.

FIG. 2 is a flowchart showing image processing performed by the image processing section 12 of FIG.

In step S1 of FIG. 2, the brightness ratio calculation unit 21 inputs the polarized image data and non-polarized image data captured by the camera 1 via the image interface 11, and The luminance ratio (ratio of luminance values) of each pixel is calculated, and image data having a luminance ratio value for each pixel is generated and output. Next, in step S2, the background extraction processing unit 22 performs a background extraction process on the image data output from the brightness ratio calculation unit 21 using a smoothing filter such as a Gaussian filter, thereby determining the background color. Generate and output image data that is extracted and retained.

Furthermore, in step S3, the brightness difference calculation unit 24 calculates each corresponding pixel between the image data output from the background extraction processing unit 22 and the image data output from the brightness ratio calculation unit 21 via the image memory 23. By calculating the brightness difference (difference in brightness values), image data having a brightness difference in each pixel, in which the background color is deleted and wrinkles or texture information is emphasized, is generated and output. In step S4, the post-processing unit 25 visualizes wrinkles or texture information by performing post-processing including binarization processing and closing processing on the image data output from the brightness difference calculation unit 24. The generated image data is output and displayed on the display 2 via the image interface 13, and the image processing is completed.

Although the brightness ratio calculation unit 21 in FIG. 1 calculates the brightness ratio (ratio of brightness values) of each corresponding pixel of polarized image data and non-polarized image data, the present invention is not limited to this. , the brightness difference (difference in brightness value) between each corresponding pixel of the polarized image data and the non-polarized image data may be calculated. This is because the brightness ratio of each corresponding pixel of polarized image data and non-polarized image data may correspond to a relative difference between them.

Next, the above image processing will be explained in detail below.

FIG. 3A is a captured image of the skin of a human being (human body) that is the subject of image processing, showing a well-textured skin surface. Further, FIG. 3B is a captured image of the skin of a human being (human body) that is the subject of image processing, showing the surface of the skin with disordered texture.

Possible causes of loss of human skin texture include photoaging of the skin due to ultraviolet rays, dryness of the skin, deterioration of blood circulation, and slowing of metabolism. Therefore, it can be said that skin wrinkle and texture information is useful for evaluating the health condition of the skin. Conventional methods for measuring skin wrinkles and texture include the following.
(1) Measure wrinkles and texture of the skin based on a skin image taken with an RGB camera. In this case, there is a characteristic that the skin color is uneven, or that the color information of wrinkles and textured areas is similar to that of other areas.
(2) Measure the wrinkles and texture of the skin using three-dimensional data of the skin measured with a three-dimensional scanner. In this case, the measuring equipment is expensive, the measurement time is long, and the amount of data is enormous.

Therefore, with the measurement method according to the conventional technology, there is a problem that it is difficult to obtain skin wrinkle and texture information easily and inexpensively.

FIG. 4A is a captured image showing a polarized light image captured by the camera 1 of FIG. Further, FIG. 4B is a captured image showing a non-polarized light image captured by the camera 1 of FIG. Further, FIG. 4C is a captured image showing a UV image captured by another ultraviolet (UV) camera. From FIG. 4A, it can be seen that the color and structure inside the skin are visible, but wrinkles and texture are not visible. Further, from FIG. 4B, information on the skin surface is visible, and some wrinkles and texture are visible. From FIG. 4C, "spots" and "moles" can be seen, as well as "wrinkles" and "texture".

5A is a captured image showing the polarization image of FIG. 4A, FIG. 5B is a captured image showing the R channel image of the polarization image of FIG. 4A, and FIG. 5C is a captured image showing the G channel image of the polarization image of FIG. 4A. FIG. 5D is a captured image showing an image of the B channel of the polarization image of FIG. 4A. 5B to 5D are images in which the polarized light image in FIG. 4A is separated into images of each RGB channel and displayed. As a result, it was found that even in polarized images in which reflected light from the skin surface is suppressed, the B channel image on the short wavelength side contains some information about the skin surface. That is, in this embodiment, it is preferable to detect the unevenness of the human skin surface using the B channel image, and the image data input to the brightness ratio calculation unit 21 in FIG. 1 is also different from the B channel polarization image data. Preferably, polarized image data is used.

6A is a captured image showing the non-polarized image in FIG. 4B, FIG. 6B is a captured image showing the R channel image in the non-polarized image in FIG. 4B, and FIG. 6C is a captured image showing the G channel image in the non-polarized image in FIG. 4B. FIG. 6D is a captured image showing the B channel image of the non-polarized image of FIG. 4B. 6B to 6D are images in which the non-polarized image in FIG. 4B is separated into images of each RGB channel and displayed. The results show that non-polarized images contain more information about the skin surface than polarized images. That is, in this embodiment, it is preferable to detect the unevenness of the human skin surface using a non-polarized image, and the image processing unit 12 in FIG. 1 is configured to extract information from the non-polarized image data. It is preferable.

FIG. 7A is a captured image for considering the visualization of wrinkles or texture, and is a captured image showing an image of the B channel of a polarized light image. Further, FIG. 7B is a captured image for considering visualization of wrinkles or texture, and is a captured image showing a B channel image of a non-polarized image. Although the B channel image of the polarized image in FIG. 7A has little information on wrinkles and texture, it can be said that it contains sufficient color information inside the skin. On the other hand, the B channel image of the non-polarized image shown in FIG. 7B includes information on wrinkles and texture, and also includes sufficient color information inside the skin. Therefore, in this embodiment, wrinkles and texture are visualized from the B channel information of the polarized image and the non-polarized image. Note that it can be said that UV images are unsuitable for visualizing wrinkles and texture because of the information on spots other than wrinkles and texture.

FIG. 8 is a conceptual diagram of image processing showing the image processing of FIG. 2 using a captured image and an image-processed image.

In FIG. 8, first, a brightness ratio image of each pixel of the B channel of a polarized image and a non-polarized image is calculated. For areas other than wrinkles and texture, the polarized image and non-polarized image are very similar, so the brightness ratio is small, but the brightness ratio for wrinkles and texture is large, so wrinkles and texture are emphasized in the brightness ratio image. . Next, in order to remove uneven skin color and shadows caused by the way the incident light hits the brightness ratio image, a background extraction process using a smoothing filter such as a Gaussian filter is applied to the brightness ratio image. By taking the difference between the brightness ratio images before and after application, an image in which only wrinkles and textured areas are significantly emphasized is generated. Finally, by performing post-processing including binarization processing and closing processing, a visualized image of wrinkles and texture (DWE (Dermo Wrinkle Emphasis) image) can be generated.

Further, the images of reflected light and scattered light used in the image processing apparatus 10 described above will be supplementally explained below.

A non-polarized image is an image taken of general light, and is captured by the image sensor of the camera 1 receiving the reflected light reflected from the surface of the target object and the scattered light scattered by the surface layer of the target object. An image signal containing data is generated. Moreover, a polarized light image is an image obtained mainly by photographing light scattered on the surface layer of a target object. Linearly polarized light is irradiated onto the target object, and the image sensor of camera 1 receives the light that passes through a polarizing plate installed in a direction that cuts off light polarized in the same direction, and the image is captured and image data is generated. An image signal containing the image is generated.

The reflected light reflected from the surface of the target object does not pass through the polarizing plate because its polarization direction does not change, but the scattered light does not pass through the polarizing plate because its polarization direction changes. Thereby, the polarized light image can capture only the light scattered on the surface layer of the target object. In addition, short-wavelength light (B channel of an RGB image) has a lower rate of penetration into the target object than long-wavelength light (R channel of an RGB image), and tends to be more likely to be reflected on the surface of the target object. The B channel image contains a lot of information about minute irregularities on the surface of the target object. Therefore, in this embodiment, it is preferable to visualize the wrinkles and texture of the skin using the B channel image. Furthermore, when visualizing a target object other than the skin, it is considered necessary to change the wavelength information (RGB information) to be utilized depending on the scattering characteristics of the surface layer of the target object.

Furthermore, an example of visualization of image processing using the image processing apparatus 10 according to the first embodiment will be described below.

(Visualization example 1)
FIG. 9A is an image according to visualization example 1, which is a captured image showing a polarized image to be processed, and FIG. 9B is an image related to visualization example 1, which is a captured image showing a non-polarized image to be processed. . Further, FIG. 9C is an image according to Visualization Example 1, which is a visualized image after the image processing of FIGS. 2 and 8. As is clear from FIG. 9C, it is possible to obtain an image in which wrinkles and texture are emphasized and visualized.

(Visualization example 2)
FIG. 10A is an image according to visualization example 2, which is a captured image showing a polarized image to be processed, and FIG. 10B is an image related to visualization example 2, which is a captured image showing a non-polarized image to be processed. . Further, FIG. 10C is an image according to visualization example 2, which is a visualized image after the image processing of FIGS. 2 and 8. As is clear from FIG. 10C, it is possible to obtain an image in which wrinkles and texture are emphasized and visualized.

(Visualization example 3)
FIG. 11A is an image according to visualization example 3, which is a captured image showing a polarized image to be processed, and FIG. 11B is an image related to visualization example 3, which is a captured image showing a non-polarized image to be processed. . Further, FIG. 11C is an image according to visualization example 3, which is a visualized image after the image processing of FIGS. 2 and 8. As is clear from FIG. 11C, it is possible to obtain an image in which wrinkles and texture are emphasized and visualized.

(Visualization example 4)
FIG. 12A is an image related to Visualization Example 4, which is a captured image showing a polarized image to be processed, and FIG. 12B is an image related to Visualization Example 4, which is a captured image showing a non-polarized image to be processed. . Further, FIG. 12C is an image according to visualization example 4, which is a visualized image after the image processing of FIGS. 2 and 8. As is clear from FIG. 12C, it is possible to obtain an image in which wrinkles and texture are emphasized and visualized.

(Visualization example 5)
FIG. 13A is an image according to visualization example 5, which is a captured image showing a polarized image to be processed, and FIG. 13B is an image related to visualization example 5, which is a captured image showing a non-polarized image to be processed. . Further, FIG. 13C is an image according to visualization example 5, which is a visualized image after the image processing of FIGS. 2 and 8. As is clear from FIG. 13C, it is possible to obtain an image in which wrinkles and texture are emphasized and visualized.

(Visualization example 6)
FIG. 14A is an image related to visualization example 6, which is a captured image showing a polarized image to be processed, and FIG. 14B is an image related to visualization example 6, which is a captured image showing a non-polarized image to be processed. . Further, FIG. 14C is an image according to visualization example 6, which is a visualized image after the image processing of FIGS. 2 and 8. As is clear from FIG. 14C, it is possible to obtain an image in which wrinkles and texture are emphasized and visualized.

(Visualization example 7)
FIG. 15A is an image according to visualization example 7, which is a captured image showing a polarized image to be processed, and FIG. 15B is an image related to visualization example 7, which is a captured image showing a non-polarized image to be processed. . Further, FIG. 15C is an image according to visualization example 7, which is a visualized image after the image processing of FIGS. 2 and 8. As is clear from FIG. 15C, it is possible to obtain an image in which wrinkles and texture are emphasized and visualized.

(Summary of Embodiment 1)
As explained above, wrinkles and texture can be visualized by the method according to the present embodiment, and in particular, by utilizing the feature amount based on the brightness difference information of the B channel of the polarized image and the non-polarized image, the skin texture can be visualized. It can be seen that wrinkles and texture can be visualized. That is, according to the image processing device 10 according to the first embodiment, it is possible to visualize the uneven state of the surface of an object to be imaged, such as a biological surface such as a human skin surface, with higher accuracy than in the conventional technology.

(Modified example)
In the first embodiment described above, post-processing is performed in the post-processing section 25, but the present invention is not limited to this, and this may be omitted. Furthermore, the post-processing may include at least binarization processing.

In the first embodiment described above, the image processing section 12 can be configured by, for example, a digital computer having an image memory. The image processing in FIG. 2 may be configured as an image processing program and executed by a CPU, which is hardware. Alternatively, the image processing program may be recorded on a computer-readable recording medium (program product) such as an optical disk, and the storage medium may be inserted into an optical disk drive device, read out, and executed by the computer.

In the first embodiment described above, skin wrinkles and texture are preferably visualized by using features based on brightness difference information using B channel polarized images and non-polarized images. The invention is not limited to this, and features based on luminance difference information may be used using images of other wavelengths in addition to the R channel or G channel. Even in this case, the accuracy is higher than that of the conventional technology. You can visualize wrinkles and texture of the skin.

(Embodiment 2)
FIG. 16 is a block diagram showing a configuration example of an image processing system according to the second embodiment. The image processing system in FIG. 16 differs from the image processing system in FIG. 1 in the following points.
(1) Instead of the image processing section 12, an image processing section 12A including an image memory 26 and a feature quantity analysis section 27 is provided between the post-processing section 25 and the image interface 13.
(2) The image processing section 12A includes a control section 20A instead of the control section 20.
(3) In place of the image processing device 10, an image processing device 10A including an image processing section 12A is provided.
The differences will be explained below.

FIG. 17 is a flowchart showing image processing performed by the image processing section 12A of FIG. 16. The image processing in FIG. 17 differs from the image processing in FIG. 2 in the following points.
(1) Instead of the process in step S4, the process in step S4A is executed.
(2) After the process in step S4A, the process in step S5 is executed.
The differences will be explained below.

In step S4A in FIG. 17, the post-processing unit 25 performs post-processing including binarization processing and closing processing on the image data output from the brightness difference calculation unit 24 to remove wrinkles or texture. Image data that visualizes the information is generated and output to the feature analysis unit 27 via the image memory 26 and to the display 2 via the image memory 26 and the image interface 13. Next, in step S5, the feature amount analysis unit 27 performs feature amount analysis using a Gapore filter on the image data output from the post-processing unit 25 via the image memory 26, thereby obtaining (1 ) a graph image showing the angular distribution of the average brightness value, (2) a graph image showing the periodic distribution of the average brightness value, and (3) a graph image showing the average brightness value (distribution of the average brightness value) in a two-dimensional plane of angle and period. A gray scale (or color) image (for example, it may be a contour heat map diagram) is generated and output to the display 2 via the image interface 13. Here, at least one image designated by the user among the three images may be generated and output.

FIG. 18A is a diagram for explaining the concept of a one-dimensional Gapor function used in the feature amount analysis section 27 of the image processing section 12A in FIG. FIG. 3 is a diagram for explaining the concept of a two-dimensional Gapore function to be used.

Embodiment 2 is characterized in that the spatial distribution (interval (period) and directionality) of skin wrinkles and texture is detected using Gabor features. Specifically, a linear filter used for texture analysis in image processing, for example, the Gabor filter disclosed in Non-Patent Document 1, is used to calculate the Extract specific frequency components. As shown in FIG. 18A, the Gabor filter uses a Gabor function generated by multiplying a Gaussian function of a normal distribution and a trigonometric function of a cosine wave. In this embodiment, it is particularly used for biometric recognition such as face, iris, and fingerprint authentication. Here, the Gabor filter is said to be a model of early visual processing in the human brain.

Next, parameters including the period and angle of the Gabor filter will be explained below.

(1) Gabor filter parameters In the Gabor filter applied to the visualized image of skin wrinkles and texture, the period and angle parameters can be set. In this embodiment, the response value (average brightness) when a Gabor filter is applied to a visualized image by changing the period and angle of a black and white pattern (kernel in which a sin/cos function is localized with a Gaussian function) The direction and angle of the black and white pattern included in the visualized image are detected from the strength of the image.

(2) Period of Gabor filter When the interval between wrinkles and textures on the visualized image (interval between continuous white lines) and the interval between the black and white patterns of the Gabor filter are most similar, the response when applying the Gabor filter will be strong. In other words, when Gabor filters with various periods are applied to visualized images of wrinkles and texture, the period parameter of the Gabor filter that shows the maximum response corresponds to the interval between wrinkles and texture.

(3) Angle of Gabor filter When the direction of wrinkles and texture on the visualized image (direction of continuous white lines) and the direction of the black and white pattern of the Gabor filter are most similar, the response when applying the Gabor filter will be strong. The difference between the maximum response value (average brightness after applying the filter) and the minimum response value (average brightness after applying the filter) corresponds to the strength of the directionality of wrinkles and texture. If the difference between the maximum response value (average brightness after applying the filter) and the minimum response value (average brightness after applying the filter) is small, it means that the same wrinkles and texture are present in any direction. In other words, wrinkles and texture are not distorted. If the difference between the maximum response value (average brightness after applying the filter) and the minimum response value (average brightness after applying the filter) is large, there are strong wrinkles and texture in one direction, and wrinkles and texture in another direction. And the texture is weak. In other words, wrinkles and texture become distorted.

(Analysis example 1)
FIG. 19A is an image showing analysis example 1 (directivity) of the feature amount analysis unit 27, which is a visualization processed image (DWE image) before linear filtering, and FIG. 19B is an image showing analysis example 1 (directivity) of the feature amount analysis unit 27. This is an image showing the directionality) and is an analysis processed image after linear filtering. The figure shows a DWE image (period: 20 pixels/cycle) when Gabor filters in various directions are applied to the visualized image (DWE image) in FIG. 19A by fixing the period of the Gabor filter and changing the direction. Shown in 19B. This shows that the strength of the directionality of wrinkles and texture can be detected.

(Analysis example 2)
FIG. 20A is a graph showing analysis example 2 (directivity) of the feature quantity analysis unit 27, and is a graph showing the angular distribution of average luminance after linear filtering. Further, FIG. 20B is a graph showing analysis example 2 (directivity) of the feature quantity analysis unit 27, and is an analysis processed image at the maximum brightness Lmax after linear filtering. Further, FIG. 20C is a graph showing analysis example 2 (directivity) of the feature amount analysis unit 27, and is an analysis processed image at the minimum brightness Lmin after linear filtering. Here, the luminance difference ΔL=Lmax−Lmin.

As is clear from FIGS. 20A to 20C, the maximum brightness value (maximum response value) Lmax is shown at an angle of 56 degrees, and the minimum brightness value (minimum response value) Lmin is shown at an angle of 157 degrees. That is, it can be seen that a strong response is detected when the Gabor filter with an angle of 56 degrees is applied, and the strength of the directionality of wrinkles and texture can be detected.

(Analysis example 3)
FIG. 21A is an image showing analysis example 3 (periodicity) by the feature quantity analysis unit 27, and is a visualization processed image (DWE image) before linear filtering. Further, FIG. 21B is an image showing analysis example 3 (periodicity) by the feature amount analysis unit 27, and is an analysis processed image after linear filtering. DWE images (angle: 55 degrees) are obtained by applying Gabor filters with various periods while fixing the direction (angle) of the Gabor filter and changing the period to the visualization processed image (DWE image) in FIG. 21A. Shown in FIG. 21B. This shows that the strength of the appearance interval (appearance period) of wrinkles and texture can be detected.

(Analysis example 4)
FIG. 22A is a graph showing analysis example 4 (periodicity) by the feature amount analysis unit 27, and is a graph showing a periodic distribution of average luminance after linear filtering. Further, FIG. 22B is a graph showing analysis example 4 (periodicity) by the feature amount analysis unit 27, and is an analysis processed image at the maximum brightness Lmax after linear filtering.

As is clear from FIGS. 22A and 22B, the period is 86 pixels/cycle, the maximum brightness is 87.4, and a strong response is detected when the Gabor filter with the period 86 is applied. This shows that it is possible to detect the strength of the appearance interval (=appearance period) of wrinkles and texture.

(Analysis example 5)
FIG. 23 is a graph showing analysis example 5 (directivity and periodicity) of the feature quantity analysis unit 27, and is a graph image in which brightness values are expressed in a contour heat map format in a two-dimensional plane of periodicity with respect to directionality angle. be. FIG. 23 shows the luminance change of a DWE image to which a Gabor filter is applied, and the response values when both the direction and period of the Gabor filter are changed are displayed as a heat map (contour line display) to show the feature amount of the DWE image. This is a visualization of the Here, in order to show the patent drawing as a black and white image, the response values are changed by cross hatching or hatching intervals. The same applies to FIG. 24 and subsequent figures.

24A to 24E are images showing analysis example 5 (directivity and periodicity) of the feature quantity analysis unit 27, and FIG. 24A is an RGB image before processing (shown as a black and white image for patent drawings). , FIG. 24B is a visualized image (DWE image) after the visualization process, and FIG. 24C is a graph image representing brightness values in a contour heat map format in a two-dimensional plane of periodicity with respect to directional angle. Further, FIG. 24D is a graph showing the angular distribution of average brightness after linear filtering, and FIG. 24E is a graph showing the periodic distribution of average brightness after linear filtering. In FIGS. 24A and 24B, the rectangle within the image indicates the area to be analyzed.

As is clear from FIGS. 24C and 24D, it is possible to detect angles where the brightness difference in the angular distribution is large, and it is possible to confirm strong wrinkles and texture only in a certain direction. Furthermore, as is clear from FIGS. 24C and 24E, a period with a strong response can be detected, and it can be seen from FIGS. 24C and 24E that the intervals between wrinkles and textures are wide.

(Analysis example 6)
25A to 25E are images showing analysis example 6 (directivity and periodicity) of the feature quantity analysis unit 27, in which FIG. 25A is an RGB image before processing, and FIG. 25B is a visualized image after visualization processing. (DWE image), and FIG. 25C is a graph image in which brightness values are expressed in a contour heat map format in a two-dimensional plane of periodicity with respect to directional angle. Further, FIG. 25D is a graph showing the angular distribution of the average brightness after linear filtering, and FIG. 25E is a graph showing the periodic distribution of the average brightness after linear filtering. In FIGS. 25A and 25B, the rectangle within the image indicates the area to be analyzed.

As is clear from FIGS. 25C and 25D, it can be detected that the brightness difference in the angular distribution is small and the peak positions are scattered. This shows that wrinkles and texture directionality are weak. Furthermore, as is clear from FIGS. 25C and 25E, the periodic distribution has only one peak position (Lmax), and the intervals between wrinkles and textures are uniform.

(Analysis example 7)
26A to 26E are images showing analysis example 7 (directivity and periodicity) of the feature quantity analysis unit 27, in which FIG. 26A is an RGB image before processing, and FIG. 26B is a visualized image after visualization processing. (DWE image), and FIG. 26C is a graph image in which brightness values are expressed in a contour heat map format in a two-dimensional plane of periods with respect to directional angles. Further, FIG. 26D is a graph showing the angular distribution of the average brightness after linear filtering, and FIG. 26E is a graph showing the periodic distribution of the average brightness after linear filtering. In FIGS. 26A and 26B, the rectangle within the image indicates the area to be analyzed.

As is clear from FIGS. 26C and 26D, it can be seen that there is some brightness difference in the angular distribution, and it can be seen that there are some wrinkles and texture directionality. Furthermore, as is clear from FIGS. 26C and 26E, the periodic distribution has only one peak position (Lmax), and the intervals between wrinkles and textures are uniform.

(Analysis example 8)
27A to 27E are images showing analysis example 8 (directivity and periodicity) of the feature amount analysis unit 27 for the human left shoulder blade (scar after suturing), and FIG. 27A is an RGB image before processing. 27B is a visualized image (DWE image) after the visualization process, and FIG. 27C is a graph image representing the luminance value in a contour heat map format in a two-dimensional plane of the period with respect to the directional angle. Further, FIG. 27D is a graph showing the angular distribution of the average brightness after linear filtering, and FIG. 27E is a graph showing the periodic distribution of the average brightness after linear filtering. In FIGS. 27A and 27B, the rectangle within the image indicates the area to be analyzed.

As is clear from FIGS. 27C and 27D, it can be seen that the brightness difference in the angular distribution is small, which indicates that the directionality (angular change) of wrinkles or texture is weak (angular change is small). Furthermore, as is clear from FIGS. 27C and 27E, it can be seen that the peak position of the periodic distribution is small, which indicates that the intervals between wrinkles and textures are narrow (the period is small).

(Analysis example 9)
28A to 28E are images showing analysis example 9 (directivity and periodicity) of the feature quantity analysis unit 27 for the human left shoulder blade (scar after suturing), and FIG. 28A is an RGB image of Rizen. 28B is a visualized image (DWE image) after the visualization process, and FIG. 28C is a graph image representing the luminance value in a contour heat map format in a two-dimensional plane of the period with respect to the directional angle. Further, FIG. 28D is a graph showing analysis example 9 (directivity and periodicity) of the feature amount analysis unit 27, and is a graph showing the angular distribution of the average luminance after linear filtering, and FIG. 28E is a graph showing the angular distribution of the average luminance after linear filtering. 27 is a graph showing analysis example 9 (directivity and periodicity), and is a graph showing a periodic distribution of average luminance after linear filtering. In FIGS. 28A and 28B, the rectangle within the image indicates the area to be analyzed.

As is clear from FIGS. 28C and 28D, the brightness difference in the angular distribution is small and there are two peaks. This shows that although wrinkles and texture directionality are weak, brightness tends to be strong in two directions. Furthermore, as is clear from FIGS. 28C and 28E, it can be seen that the surrounding peak positions are slightly smaller, which indicates that the intervals between wrinkles and textures are slightly narrower (the period is slightly smaller).

(Summary of Embodiment 2)
As described above, according to the second embodiment, the Gabor feature amount using the Gabor filter was analyzed regarding the interval and directionality of wrinkles and texture. Here, the Gabor filter is a linear filter used for texture analysis in image processing, etc., and extracts specific frequency components for each direction. We calculated the response value (average brightness value) when applying a Gabor filter of arbitrary direction and period to the visualized image of wrinkles and texture, and quantitatively evaluated the distortion and spacing of the lattice pattern of wrinkles and texture. Ta. Regarding the pattern distortion, we focused on the strength of the response when applying a Gabor filter in an arbitrary direction, and the difference between the maximum and minimum responses corresponds to the amount of pattern distortion. Regarding the intervals between wrinkles and textures, attention is paid to the strength of the response when a Gabor filter with an arbitrary period is applied, and the period parameter (pixel/cycle) of the Gabor filter at the time of maximum response is considered to correspond to the interval.

Here, we will particularly explain the cases of (1) FIGS. 25A to 25E (analysis example 6), (2) FIGS. 27A to 27E (analysis example 8), and (3) FIGS. 28A to 28E (analysis example 9). do.

Table 1 shows the angle, period, and difference in response at maximum response. However, the angle depends on the direction of the camera when shooting. Here, analysis examples 8 and 9 are examples of scars after suturing, and the distance between wrinkles and textures and the way of distortion are different between the periphery of the sutured part according to analysis example 8 and the sutured part according to analysis example 9. , can be confirmed from the visualized image and Gabor features. These results show that quantitative evaluation is possible from parameters corresponding to the distortion and spacing of the lattice pattern of wrinkles and texture, and that feature analysis of wrinkles and texture using a Gabor filter is useful.

In the second embodiment, a method for visualizing wrinkles and texture using B channel information of polarized and non-polarized images of the skin taken with a dermo camera was proposed. Then, the intervals and directionality of wrinkles and texture were evaluated using the Gabor feature amount of the visualized image. Furthermore, the effectiveness of evaluation using the proposed method in diagnosing diseased areas of the skin was suggested.

As explained above, according to the image processing device 10A according to the second embodiment, the uneven state of the surface of the object to be photographed, such as the surface of a biological body such as the surface of human skin, can be detected with higher precision than that of the conventional technology. It can be analyzed using predetermined feature quantities.

In particular, what kind of medical judgment or diagnosis can be made according to Embodiments 1 and 2 will be described below. Generally, it is difficult to visualize the state of skin wrinkles and texture, so visualized wrinkle and texture information is useful for diagnosing the health state of the skin and making cosmetic decisions. The appearance of wrinkles and texture varies depending on the area of the skin, and healthy wrinkles and texture appear in each area, so it is possible to diagnose diseased areas by examining the appearance of abnormal wrinkles and texture. I think it is possible. In particular, visual information on wrinkles and texture is useful for diagnosing the healing process of scarred areas. Scarred areas may have noticeable scars or not after healing, and the state of wrinkles and texture during the healing process (when the skin is stretched Imaging diagnosis and quantitative evaluation (feature analysis using Gabor filters) are considered useful for follow-up observation.

In the above-described second embodiment, the post-processing unit 25 executes the binarization process and the closing process, but the present invention is not limited to this. The state can be expressed with high precision.

(Embodiment 3)
FIG. 29 is a block diagram showing a configuration example of an image processing system according to the third embodiment. The image processing system in FIG. 29 differs from the image processing system in FIG. 1 in the following points.
(1) In place of the image processing section 12, an image processing section 12B was provided.
(2) The image processing section 12B includes a control section 20B instead of the control section 20.
(3) In place of the image processing device 10, an image processing device 10B including an image processing section 12B and an external interface 14 is provided.
(4) An optical disk drive 30 connected to the external interface 14 and into which an optical disk 31 is inserted is further provided.
The differences will be explained below.

In the third embodiment, the image processing program shown in FIG. 2 or FIG. For example, the storage medium may be inserted into a drive device such as the optical disk drive 30, read out, loaded into the memory of the control unit 20B via the external interface 14, and executed by the control unit 20B (computer).

In the second embodiment described above, the image processing system of the first embodiment is provided with the external interface 14 and the optical disk drive 30, but the present invention is not limited to this. 14 and an optical disk drive 30.

(Other variations)
In the above embodiment, the background extraction processing unit 22 extracts background image data from the image data from the brightness ratio calculation unit 21 (or image data including the difference in brightness for each pixel (modified example)). However, the present invention is not limited to this, and background image data may be extracted from polarized image data or non-polarized image data input to the brightness ratio calculation section 21. That is, the background extraction processing unit 22
(1) Image data including the brightness ratio of each pixel from the brightness ratio calculation unit 21,
(2) image data including the difference in brightness for each pixel;
(3) Polarization image data input to the brightness ratio calculation unit 21;
(4) non-polarized image data input to the brightness ratio calculation unit 21;
Background image data related to a background image may be extracted from any of the image data and output.

In the above embodiment, post-processing is performed in the post-processing section 25, but the present invention is not limited to this, and this may be omitted. Furthermore, the post-processing may include at least binarization processing.

In the embodiments described above, the image processing sections 12, 12A, and 12B can be configured by, for example, a digital computer having an image memory. The image processing in FIGS. 2 and 17 may be configured as an image processing program and executed by a CPU, which is hardware.

As detailed above, according to the present invention, it is an object of the present invention to provide an image processing device and an image processing method that can visualize a living body surface, such as a human skin surface, with higher precision than in the prior art. Further, according to the present invention, it is possible to provide an image processing device and an image processing method that can analyze a biological surface, such as a human skin surface, using predetermined parameters with higher precision than in the prior art.

In the present invention, the following photographic objects can be used for image processing. Basically, anything that has a certain degree of fine irregularities on its surface and has a predetermined high degree of transparency, such as the skin or surface of a living body including a human body, an animal, or a living thing, can be used as an object to be photographed. Specifically, it is possible to visualize the unevenness of the surfaces of human skin, animal skin, leaves of living things, mushrooms, and the like.

1 Camera 2 Display 10, 10A, 10B Image processing device 11 Image interface 11m Image memory 12, 12A, 12B Image processing section 13 Image interface 13m Image memory 14 External interface 20, 20A Control section 21 Brightness ratio calculation section 22 Background extraction processing section 23 Image memory 24 Brightness difference calculation section 25 Post-processing section 26 Image memory 27 Feature amount analysis section 30 Optical disk drive 31 Optical disk

Claims (24)

an image sensor that images the surface of the object to be processed and generates polarized image data and non-polarized image data;
a first calculation unit that calculates and outputs a brightness ratio or brightness difference for each pixel between the polarized image data and the non-polarized image data;
(1) image data including a brightness ratio for each pixel from the first calculation unit;
(2) image data including a difference in brightness for each pixel from the first calculation unit;
(3) the polarization image data;
(4) the non-polarized image data;
a background extraction processing unit that extracts and outputs background image data related to a background image from any of the image data;
Subtracting the background image data from the background extraction processing unit from the image data including the brightness ratio or difference in brightness for each pixel from the first calculation unit to generate image data including the difference for each pixel. a second calculation unit that performs
An image processing device comprising: The first calculation unit is configured to calculate the difference between B (blue) channel image data of the polarized image data and B (blue) channel image data of the non-polarized image data among the polarized image data and the non-polarized image data. The image processing device according to claim 1, wherein the image processing device calculates a brightness ratio or a brightness difference for each pixel. The image processing device according to claim 1, wherein the background extraction processing unit extracts background image data related to a background image from the image data using a Gaussian filter. The image processing device according to claim 1, further comprising a post-processing unit that performs binarization processing on the image data from the second calculation unit and outputs the result. The image processing device according to claim 4, wherein the post-processing section performs binarization processing and closing processing on the image data from the second calculation section and outputs the resultant data. By using a Gapol filter on the image data from the post-processing unit and calculating Gapol features by changing at least one of the angle and the period, the angular distribution of brightness values and the brightness are calculated. The image processing device according to claim 4 or 5, further comprising a feature amount analysis unit that generates a periodic distribution of values and a luminance value distribution in a two-dimensional plane of angles and periods. an image sensor capturing an image of the surface of the object to be processed to generate polarized image data and non-polarized image data;
a first calculation unit calculating and outputting a brightness ratio or brightness difference for each pixel between the polarized image data and the non-polarized image data;
The background extraction processing section
(1) image data including a brightness ratio for each pixel from the first calculation unit;
(2) image data including a difference in brightness for each pixel from the first calculation unit;
(3) the polarization image data;
(4) the non-polarized image data;
extracting and outputting background image data related to a background image from any of the image data;
A second calculation unit subtracts the background image data from the background extraction processing unit from the image data including the brightness ratio or difference in brightness for each pixel from the first calculation unit to obtain an image for each pixel. generating image data including a difference;
image processing methods including; The step of calculating the brightness ratio or brightness difference includes calculating the B (blue) channel image data of the polarized image data and the B (blue) channel image data of the non-polarized image data among the polarized image data and non-polarized image data. calculating a brightness ratio or brightness difference for each pixel between channel image data;
The image processing method according to claim 7. The step of extracting the background image data includes extracting background image data related to the background image from the image data using a Gaussian filter.
The image processing method according to claim 7. 8. The image processing method according to claim 7, further comprising a step in which the post-processing section performs binarization processing on the image data from the second calculation section and outputs the result. The step of performing binarization processing and outputting includes performing binarization processing and closing processing on the image data from the second calculation unit and outputting the resultant image data.
The image processing method according to claim 10. The feature quantity analysis unit calculates the luminance value by using a Gapol filter on the image data from the post-processing unit and changing at least one of the angle and the period. 12. The image processing method according to claim 10, further comprising the step of generating an angular distribution of , a periodic distribution of brightness values, and a brightness value distribution of angles and periods in a two-dimensional plane. An image processing program executed by a computer,
an image sensor capturing an image of the surface of the object to be processed to generate polarized image data and non-polarized image data;
a first calculation unit calculating and outputting a brightness ratio or brightness difference for each pixel between the polarized image data and the non-polarized image data;
The background extraction processing section
(1) image data including a brightness ratio for each pixel from the first calculation unit;
(2) image data including a difference in brightness for each pixel from the first calculation unit;
(3) the polarization image data;
(4) the non-polarized image data;
extracting and outputting background image data related to a background image from any of the image data;
A second calculation unit subtracts the background image data from the background extraction processing unit from the image data including the brightness ratio or difference in brightness for each pixel from the first calculation unit to obtain an image for each pixel. generating image data including a difference;
image processing programs including; The step of calculating the brightness ratio or brightness difference includes calculating the B (blue) channel image data of the polarized image data and the B (blue) channel image data of the non-polarized image data among the polarized image data and non-polarized image data. calculating a brightness ratio or brightness difference for each pixel between channel image data;
The image processing program according to claim 13. The step of extracting the background image data includes extracting background image data related to the background image from the image data using a Gaussian filter.
The image processing program according to claim 13. 14. The image processing program according to claim 13, further comprising a step in which the post-processing section performs binarization processing on the image data from the second calculation section and outputs the result. The step of performing binarization processing and outputting includes performing binarization processing and closing processing on the image data from the second calculation unit and outputting the resultant image data.
The image processing program according to claim 16. The feature quantity analysis unit calculates the luminance value by using a Gapol filter on the image data from the post-processing unit and changing at least one of the angle and the period. 18. The image processing program according to claim 16, further comprising the step of generating an angular distribution of , a periodic distribution of brightness values, and a brightness value distribution of angles and periods in a two-dimensional plane. A recording medium that stores an image processing program readable by a computer,
The image processing program includes:
an image sensor capturing an image of the surface of the object to be processed to generate polarized image data and non-polarized image data;
a first calculation unit calculating and outputting a brightness ratio or brightness difference for each pixel between the polarized image data and the non-polarized image data;
The background extraction processing section
(1) image data including a brightness ratio for each pixel from the first calculation unit;
(2) image data including a difference in brightness for each pixel from the first calculation unit;
(3) the polarization image data;
(4) the non-polarized image data;
extracting and outputting background image data related to a background image from any of the image data;
A second calculation unit subtracts the background image data from the background extraction processing unit from the image data including the brightness ratio or difference in brightness for each pixel from the first calculation unit to obtain an image for each pixel. generating image data including a difference;
Recording media including. The step of calculating the brightness ratio or brightness difference includes calculating the B (blue) channel image data of the polarized image data and the B (blue) channel image data of the non-polarized image data among the polarized image data and non-polarized image data. calculating a brightness ratio or brightness difference for each pixel between channel image data;
The recording medium according to claim 19. The step of extracting the background image data includes extracting background image data related to the background image from the image data using a Gaussian filter.
The recording medium according to claim 19. The image processing program includes:
20. The recording medium according to claim 19, further comprising a step in which the post-processing section performs binarization processing on the image data from the second calculation section and outputs the result. The step of performing binarization processing and outputting includes performing binarization processing and closing processing on the image data from the second calculation unit and outputting the resultant image data.
The recording medium according to claim 22. The image processing program includes:
The feature quantity analysis unit calculates the luminance value by using a Gapol filter on the image data from the post-processing unit and changing at least one of the angle and the period. The recording medium according to claim 22 or 23, further comprising the step of generating an angular distribution of , a periodic distribution of brightness values, and a brightness value distribution of angles and periods in a two-dimensional plane.

Images