JP2005149390A - Image processing method and device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make data size of image information required for texture reproduction small, to shorten image processing time and to easily obtain a texture reproduced image. <P>SOLUTION: When an object image is acquired by photographing an object, texture information data is compressed by acquiring the texture information data expressing the texture of the object, separating the object image into each component in which feature values of the texture information data have approximately the same value based on the feature values and making a representative value of the feature values in each component correspond to each separated component. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image processing method and apparatus, and in particular, an image processing method that reduces the load when processing texture image data for faithfully reproducing the texture of an object and easily reproduces an image that emphasizes the texture. And device.

  Today, with advances in optical equipment and image processing technology, it has become possible to capture a subject with a camera or read a subject with a scanner to obtain a high-quality image. It is difficult to reproduce the texture such as fine irregularities on the surface, and it is desired to reproduce the texture of the subject more realistically.

An actual subject is usually a three-dimensional object, and a light source that illuminates the subject is necessary for photographing the subject. When the subject is illuminated, the brightness of the subject surface changes depending on the location. The brightness varies depending on the manner of reflection on the subject surface, and the manner of reflection varies depending on the material of the subject surface.
In general, the reflection of light on the surface of an object includes diffuse (internal) reflection that represents the color of an object and specular (surface) reflection that represents gloss.

Diffuse reflected light is light that is incident on the surface and then exits the surface after multiple scattering within the material, and is reflected with the same intensity in all directions. Diffuse reflections are often seen on rough surfaces such as plaster and chalk.
The specular reflection light is generated by direct reflection on the surface of the subject. Specular reflection causes highlights on parts of the surface when metal or well-polished pottery is illuminated. The specular reflection light is reflected most strongly in the direction of regular reflection where the incident surface and the reflection surface are equal. In general, when a subject is illuminated, both diffuse reflection light and specular reflection light are generated.
Therefore, in order to obtain an image that realistically reproduces the subject, it is necessary to faithfully reproduce the change in reflected light due to the difference in the material of the subject surface.

  The texture image data including information for reproducing the texture of the object is data holding information such as declination (variation) information, spectral information, distance information, etc., and combining them to calculate a predicted image It is possible to generate and display an image that faithfully achieves the reproduction of the texture.

For example, as a color reproduction method when designing paint colors for the exterior of automobiles, etc., the variable angle spectral reflectance of the outer color coating plate is measured, and the characteristic amount related to the variable angle and the characteristic regarding the wavelength from the measured variable spectral reflectance A color reproduction method for displaying an image of a paint color by extracting the amount and reconstructing the variable angle spectral reflectance using these is known (see, for example, Patent Document 1).
JP-A-9-33347

  However, in the conventional texture image reproduction method as described above, the amount of information of the texture image data necessary for the texture reproduction is very large, and the load in image processing including image transmission and recording is large. There is a problem that it takes a long time.

  The present invention has been made in view of the above-described conventional problems, and can reduce the data size of image information necessary for texture reproduction, shorten the image processing time, and easily obtain a texture reproduction image. It is an object to provide a method and apparatus.

  In order to solve the above-described problem, according to a first aspect of the present invention, when a subject is photographed to obtain a subject image, the material information data representing the texture of the subject is obtained, and the feature amount of the texture information data is obtained. Based on the subject image, the feature amount is separated into each component having substantially the same value, and a representative value of the feature amount in each component is associated with each separated component. The image processing method is characterized in that the texture information data is compressed.

  Further, it is preferable to use two-dimensional distance information and declination spectral information at each point on the surface of the subject as the feature amount of the texture information data.

  Further, it is preferable that the declination spectral information of the subject is decomposed into declination information and spectral information and used as the feature amount.

  Further, it is preferable that the normal vector direction of each point on the subject surface is calculated from the two-dimensional distance information of the subject and used as the feature amount.

  Similarly, in order to solve the above-described problem, the second aspect of the present invention provides a means for acquiring texture information data representing the texture of the subject when photographing the subject and obtaining the subject image, and the texture Based on the feature amount of the information data, the means for separating the subject image into each component having the substantially same value as the feature amount, and the feature amount in each component in each separated component An image processing apparatus characterized in that the texture information data is compressed.

  The means for obtaining the texture information data representing the texture of the subject includes distance information obtaining means for obtaining two-dimensional distance information at each point on the surface of the subject, and bias at each point on the surface of the subject. A declination spectroscopic information acquisition unit that acquires angular spectroscopic information is preferable.

  The image processing apparatus preferably further includes means for decomposing the declination spectral information of the subject into declination information and spectral information.

  The image processing apparatus preferably further includes means for calculating a normal vector direction of each point on the subject surface from the two-dimensional distance information of the subject.

  Similarly, in order to solve the above problem, a third aspect of the present invention provides a program for executing an image processing method.

  According to the present invention, a subject image is separated into constituent elements having the same (uniform) texture, and the texture information data representing the texture for each constituent element is represented by one representative point, thereby providing a texture image. By compressing the data amount, it is possible to save the storage capacity when storing and holding the data and reduce the traffic and time at the time of data transmission, and it is possible to easily obtain an image that faithfully reproduces the texture.

  Hereinafter, an image processing method and apparatus according to the present invention will be described in detail based on preferred embodiments shown in the accompanying drawings.

FIG. 1 is a block diagram showing a schematic configuration of an embodiment of an image processing apparatus that executes an image processing method according to the present invention.
The image processing apparatus according to the present embodiment shoots a subject, acquires texture information data necessary to reproduce the texture of the subject, and sets the subject in the image to a value with substantially the same feature amount of the texture information data. An image for reproducing the texture after compressing the texture information data (reducing the data amount) by associating each component with a representative value of the feature value of the component. It is characterized by facilitating processing.

As shown in FIG. 1, when the image processing apparatus according to the present embodiment captures a subject 10 and obtains a subject image, means 12 for obtaining texture information data representing the texture of the subject 10, and feature amounts of the texture information data The component separation means 14 for separating the subject image into the respective components having the same texture, the representative value handling means 16 for associating the representative values of the feature values of the texture information data with the respective components, and the respective components Image processing means 18 for performing image processing according to the texture.
The texture information acquisition unit 12 that acquires the texture information data representing the texture of the subject 10 includes a declination spectral information acquisition unit 20 and a distance information acquisition unit 22.

  The image processing apparatus according to the present embodiment also includes the declination information acquisition unit 24 and the spectroscopic unit for decomposing the declination spectral information acquired by the declination spectral information acquisition unit 20 into declination information and spectral information as optical attribute data. Information acquisition means 26 is provided. The image processing apparatus further includes normal vector direction calculation means 28 for calculating the direction of the normal vector at each point on the subject surface and acquiring normal direction data.

The declination spectral information acquisition means 20 changes the declination (variation) spectroscopic information of the subject 10, that is, the position (angle) of the light source that illuminates the subject 10 and the light receiving angle for receiving the reflected light from the subject 10. The spectral reflectance for each variable angle is acquired.
Specifically, this is not particularly limited, but for example, a multiband camera capable of capturing a plurality of spectral images with a plurality of channels can be used. That is, using a multiband camera, the angle of the light source with respect to the subject 10 may be changed, and the subject 10 may be photographed from a plurality of directions to obtain a plurality of spectral images.
Alternatively, for example, a spectral distribution for each variable angle may be measured using a variable angle spectrophotometer such as GCMS-3 manufactured by Murakami Color Research Laboratory. In this case, it is necessary to take a separate image and acquire the image data.

  The distance information acquisition unit 22 measures the shooting distance of each point on the surface of the subject 10. That is, for each point on the subject 10, the distance from the distance information acquisition means 22 to each point is measured, and so-called two-dimensional distance information for each point is obtained. It can also be said that this distance information is information representing the surface shape of the subject 10. The distance information acquisition unit 22 is not particularly limited, and may be a distance sensor using a light beam, for example, and uses a three-dimensional image measurement method for obtaining a distance to a measurement object based on the principle of the optical triangulation method. It may be a thing.

The declination information acquisition unit 24 estimates a diffuse reflection component as declination information of the subject 10 from the declination spectral information. The declination information acquisition unit 24 extracts, for each pixel of the subject image, the one having the lowest intensity of the declination spectral image acquired by the declination spectral information acquisition unit 20 and uses it as a diffuse reflection component. Is.
At this time, the distance information (surface shape information) of the subject 10 and the information on the direction of the light source indicate the shadow area in the subject 10, but the shadow region does not contain so much diffuse reflection component and is weak in intensity. Except for this, the diffuse reflection component of the subject 10 is estimated.

The spectral information acquisition unit 26 assumes a Phong model as a spectral reflection model of the subject 10 and estimates the spectral reflectance of the subject 10 when the light source direction (incident angle) and the observation direction (light reception angle) change. Therefore, the parameters α and n in the Phong model are calculated.
Here, the Phong model will be described.

FIG. 2 is a diagram for explaining the Phong model. Light emitted from the light source 30 is reflected on the surface 32 and observed (received).
At this time, the unit vector in the normal direction of the surface 32 is N, the incident angle is θ _i , the light receiving angle is θ _r , the unit vector in the light source direction is L, the unit vector in the observation direction is V, and the specular reflection direction of the light receiving point is If the unit vector to be represented is R _v , the light source spectrum is E (λ), and the diffuse reflectance of the subject (surface 52) is S _d (λ), then the observed reflectance Y (λ) is , Α and n are expressed by the following equation (1), with parameters depending on the material of the subject. In equation (1), (A, B) represents the inner product of vectors A and B.
Y (λ) = α (R _v · L) ⁿ E (λ) + (N · L) S _d (λ) E (λ)
... (1)

As shown in the above equation (1), the Phong's model shows the ratio of the decrease in the specular reflection component due to the deviation of the observation direction from the specular reflection direction, and the cos of the deviation angle (θ _r −θ _i ) It is approximated by the nth power of the inner product (equal to _Rv · L).
The undetermined parameters α and n in the model equation (1) can be uniquely determined by recording declination measurements at two or more angles. However, in practice, it is desirable to calculate an optimum value from a large amount of deviation angle measurement data.

The normal vector direction calculation means 28 calculates the direction of the outward normal vector at each point on the surface of the subject 10 from the distance information (surface shape information) received from the distance information acquisition means 22.
This outward normal vector direction is equivalent to a value obtained by differentiating the distance information at each point on the surface of the subject 10 in the X, Y, and Z directions (that is, the vertical, horizontal, and depth directions).

The component separating means 14 is a declination spectroscopic information that is optical attribute data representing the color, gloss, etc. of the subject 10 (specifically, diffuse reflectance data that is declination information and a parameter α of the Phong model that is spectroscopic information, n) and the distance information representing the surface shape of the subject 10 (specifically, the normal vector direction of each point on the subject 10 obtained by differentiating the distance information in the three-dimensional direction) In other words, it is separated into components having the same texture.
In this separation into components, components having the same texture are considered to have the same optical attribute data and normal direction data, so that the diffuse reflectance and the parameters α, n or the normal vector direction are substantially the same value. This is done by separating regions consisting of such points as components having the same texture.

The feature quantity representative value handling unit 16 applies, for example, the diffuse reflectance at each point of the component, the parameter α, n of the Phong model, or the normal vector direction to each component separated as a region having the same texture. Is associated with the representative value of each feature quantity, that is, the data of the representative point of each component.
At this time, the value of the feature amount itself may not be associated, but a code uniquely corresponding to the value may be associated. As a result, the data amount of the texture information data can be greatly compressed.

The image processing means 18 performs image processing corresponding to the texture for each separated component, and forms an image that faithfully reproduces the texture.
That is, the data of the representative point corresponding to each component, the spectrum data of the light source, the position data in the direction of the light source, and the observation position data are applied to the Phong model, and the spectrum in the observation direction for each point. The reflectance data is calculated, texture reproduction image data for each component is created, and a texture reproduction image is created by combining the data for each component.

  Hereinafter, as an operation of the present embodiment, an image processing method according to the present invention will be described. The image processing method of the present invention separates the subject into components that can be regarded as having the same (uniform) texture, and associates the data representing the texture with the representative points of each component, thereby reducing the data amount of the texture information data. This makes it easy to create a texture reproduction image by compression. FIG. 3 is a flowchart showing the processing flow of the image processing method of this embodiment. Hereinafter, it demonstrates along the flowchart of FIG.

  First, in step 100 of FIG. 3, an image of the subject 10 is taken. In the present embodiment, a triangular pyramid 40 made of metal as shown in FIG. The triangular pyramid 40 is entirely made of metal, and the left surface 42 and the right surface 44 are made of the same material and have the same color. However, the normal directions of the left surface 42 and the right surface 44 are different, and it is desirable that the difference between the left surface 42 and the right surface 44 can be expressed in the reproduced image.

  In step 110, texture information data of the subject 10 (triangular pyramid 40) is acquired together with the above photographing. Here, declination spectroscopic information and distance information are used as the texture information data. The method of acquiring the declination spectral information is not particularly limited. For example, the triangular pyramid 40 is photographed with a multiband camera while moving at least one of the light source position and the photographing position of the camera, and a plurality of spectral images are obtained. Thus, the declination spectroscopic information may be obtained from this, and as described above, it may be directly measured using a commercially available declination spectrophotometer. Further, the distance information acquisition unit 22 measures the distance from the distance information acquisition unit 22 to each point on the triangular pyramid 40 and acquires the distance information.

Next, in step 120, the declination information acquisition unit 24 acquires declination information from the declination spectral information. The declination information is a diffuse reflection component that represents the color of the subject 10, and is obtained, for example, by extracting the pixel having the lowest declination spectral image intensity for each pixel of the subject 10 (triangular pyramid 40).
In step 130, the spectral information acquisition unit 26 acquires spectral information from the declination spectral information. That is, the two parameters α and n in the above-described phon model are calculated from the declination spectral image of the subject 10 (triangular pyramid 40). In the present embodiment, the subject 10 is a triangular pyramid 40 made entirely of metal, and the parameters α and n of the Phong model have similar values on both the left surface 42 and the right surface 44, and diffuse reflection that indicates color information. Values close to the components were obtained. Thereby, it can be determined that the subject 10 (triangular pyramid 40) is actually made of a common material on both sides.

Next, in step 140, the normal vector direction calculation means 28 calculates the direction of the outward normal vector of each point on the subject 10 (triangular pyramid 40) from the distance information. The outward normal vector direction is obtained by differentiating the distance information at each point in the X, Y, and Z directions (vertical, horizontal, and depth directions).
When the outward normal vector direction is calculated for the triangular pyramid 40 that is the subject 10 in the present embodiment, the left surface 42 is approximately ((1/3) ^1/2 , (1/3) ^1/2 , (1 / 3) ^1/2 ), and the right surface 44 obtained values of about (-(1/3) ^1/2 , (1/3) ^1/2 , (1/3) ^1/2 ).

Next, in step 150, in the component separation means 14, the subject 10 has the same texture (uniform) by the optical attribute data, that is, declination information and spectral information, and the normal direction data, that is, the outward normal vector direction. Separate into each possible component.
The component separation unit 14 uses the diffuse reflection spectrum on the subject 10 obtained above, the Phong parameters (α, n), and the normal vector direction, and the feature quantities expressing these textures have substantially the same values. And the subject 10 is separated into the respective components. At this time, if the diffuse reflection spectrum is the same, it shows the same color, and if the Phong parameters (α, n) are the same, it has the same specular reflection characteristic (gloss), and the normal vector direction is the same. If so, it is considered to be on the same plane. Therefore, the subject 10 is separated into constituent elements by appropriately combining them.

In the case of the triangular pyramid 40 used as the subject 10 this time, the material and the color are the same, and the diffuse reflection spectrum and the Phong parameter (α, n) are substantially the same values on the left surface 42 and the right surface 44. As described above, the outward normal vector direction is about ((1/3) ^1/2 , (1/3) ^1/2 , (1/3) ^1/2 ) on the left surface 42, and on the right surface 44. , About (− (1/3) ^1/2 , (1/3) ^1/2 , (1/3) ^1/2 ).
Therefore, in the triangular pyramid 40, the left surface 42 and the right surface 44 have the same color and material, but are separated as different components depending on their shapes.

Next, in step 160, the feature quantity representative value handling means 16 associates the representative value of the feature quantity with each component separated above.
In the case of the triangular pyramid 40, since it is judged that the whole is made of the same material, the Phong parameter (α, n) is not held for each point but is represented by a set of parameters (α, n). This representative point is determined by averaging the parameters (α, n) of each point. Further, the diffuse reflection components are averaged, and are made common by representing them with one diffuse reflection spectrum. Thus, by holding only representative points, the amount of data to be held can be reduced.

As for the normal vector direction, the normal vector directions are ((1/3) ^1/2 , (1/3) ^1/2 , (1/3) ^1/2 ) at all points on the left surface 42. The normal vector directions are represented by (− (1/3) ^1/2 , (1/3) ^1/2 , (1/3) ^1/2 ) at all points on the right surface 44.
At this time, in general, when the normal vector direction is made common, first, it is determined how many normal direction changes should be made common. For example, data within several degrees (for example, 5 degrees) in each of the X, Y, and Z directions is shared. Next, the average value in the direction of the common normal vector is calculated to determine the representative value.

Next, in the case of the triangular pyramid 40, the normal vector data ((1/3) ^1/2 , (1/3) ^1/2 , (1/3) ^1/2 ) which is a representative value of the left surface 42 is numbered. 1 is assigned, and number 2 is assigned to the normal vector data (-(1/3) ^1/2 , (1/3) ^1/2 , (1/3) ^1/2 ), which is a representative value of the right surface 44. To do. Each point is given the number 1 or 2 given above instead of having the normal vector data.
That is, the normal vector data is 1 at each point on the left surface 42, and the normal vector data is 2 at each point on the right surface 44. Separately, the correspondence between the numbers 1 and 2 and the normal vector data represented by the numbers 1 and 2 is maintained. As a result, the amount of data can be reduced to about 比 べ as compared with the case where normal vector data is held for each point.

  In this way, the subject is separated into each component, and in each component, diffuse reflectance data and Phong model parameters (α, n) are held in common by one representative point, and the normal line The vector data is also represented by a single value, and the texture information data, which is data for reproducing the texture, can be greatly reduced.

Further, as described above, when image processing is performed using texture information data compressed and held to create an image in which the texture is reproduced, the processing is performed as follows.
First, in step 170, texture reproduction image data is created for each separated component. By applying the compressed texture information data, light source spectrum data, light source position data, and observation position data to the Phong model, spectral reflectance data to the observation position for each point can be calculated. .
That is, the spectral distribution value of the object in the observation direction is calculated by multiplying the normal vector calculated from the object position information, the position information of the light source, and the observation position information by the parameters of the Phong model and the diffuse reflectance data. Then, material reproduction image data is created.

Next, in step 180, a texture reproduction image that emphasizes the texture reproducibility of the subject is created by combining the texture reproduction image data created for each component.
As described above, in the present embodiment, using the feature amount as the texture information data such as declination information (diffuse reflectance data), spectral information (Phong's model parameter), and distance information (normal vector direction), The subject image is separated into each component having the same texture (uniform), and the texture information data representing the texture is represented by one representative point for each component, thereby compressing the amount of the texture image data, It is possible to save storage capacity when storing and holding data and reduce traffic and time during data transmission.

  Also, the image processing method of the present invention can be executed on an arbitrary computer by assembling a program for executing the above-described image processing method, storing the program in a predetermined recording medium, and reading or downloading the program through a network or the like. Therefore, it is possible to easily obtain an image that faithfully reproduces the texture.

  The image processing method and apparatus of the present invention have been described in detail above. However, the present invention is not limited to the above embodiments, and various improvements and modifications can be made without departing from the gist of the present invention. Of course you can go.

1 is a block diagram illustrating a schematic configuration of an embodiment of an image processing apparatus that executes an image processing method according to the present invention. It is a figure for demonstrating the model of Phong. It is a flowchart which shows the flow of a process of the image processing method of this embodiment. It is a perspective view showing the triangular pyramid used as a to-be-photographed object in this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Subject 12 Texture information data acquisition means 14 Component element separation means 16 Feature quantity representative value correspondence means 18 Image processing means 20 Declination spectral information acquisition means 22 Distance information acquisition means 24 Declination information acquisition means 26 Spectral information acquisition means 28 Normal line Vector direction calculation means 30 Light source 32 Surface 40 Triangular pyramid 42 Left side 44 Right side

Claims (9)

When capturing a subject image by capturing a subject, acquiring material information data representing the texture of the subject,
Based on the feature amount of the texture information data, the subject image is separated into constituent elements having the same feature value.
An image processing method, wherein the texture information data is compressed by associating each separated component with a representative value of the feature amount in each component.   The image processing method according to claim 1, wherein two-dimensional distance information and declination spectral information at each point on the surface of the subject are used as the feature amount of the texture information data.   The image processing method according to claim 2, wherein the declination spectral information of the subject is decomposed into declination information and spectral information and used as the feature amount.   The image processing method according to claim 2 or 3, wherein a normal vector direction of each point on the subject surface is calculated from the two-dimensional distance information of the subject and used as the feature amount. Means for acquiring texture information data representing the texture of the subject when photographing the subject and obtaining the subject image;
Means for separating the subject image into components having substantially the same value based on the feature amount of the texture information data;
Means for associating each separated component with a representative value of the feature value in each component;
And an image processing apparatus characterized in that the texture information data is compressed.   Means for obtaining texture information data representing the texture of the subject includes distance information obtaining means for obtaining two-dimensional distance information at each point on the surface of the subject and declination spectroscopy at each point on the surface of the subject. The image processing apparatus according to claim 5, which is a declination spectral information acquisition unit that acquires information.   The image processing apparatus according to claim 6, further comprising means for decomposing the declination spectral information of the subject into declination information and spectral information.   8. The image processing apparatus according to claim 6, further comprising means for calculating a normal vector direction of each point on the subject surface from two-dimensional distance information of the subject. The program for performing the image processing method in any one of Claims 1-4.

Images