JP2003302211A - Three-dimensional image processing unit and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide three-dimensional image data for realistically reproducing texture, glossiness and three-dimensional effect of an object according to the observation environment specified by an observer. <P>SOLUTION: A control part 12 controls a surface attribute 11 and a three- dimensional shape obtaining part 13 according to an input environment parameter, to thereby obtain the surface attribute data and three-dimensional shape model of an object. A three-dimensional data integrating part generates integrated data representing the three-dimensional shape model of the object body and the surface attribute of the respective parts using the measurement data. A data authenticity discriminating part 14 checks the authenticity in generating the integrated data to determine whether or not further acquisition of measurement data by the control part 12 is needed. When it is determined that further acquisition of measurement data is needed, the data authenticity discriminating part 14 updates an input environment parameter according to the authenticity, and the control part 12 re-measures the object body using the updated parameter. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional image input method, a three-dimensional image input device, and a three-dimensional image processing method for making it possible to present the shape, color, texture, etc. of a target real object more realistically. And a three-dimensional image processing device.

[0002]

2. Description of the Related Art Conventionally, a method using a contact type position sensor is known as an image processing apparatus for inputting three-dimensional information (shape, surface attribute) of an existing object. In this method, the probe is brought into contact with each point of the object, the three-dimensional position coordinates of the probe are detected by a position sensor, and the three-dimensional position information of each point of the object is acquired. However, in the method using the contact-type position sensor, it is necessary to bring the probe into contact with each point of the object. There was a limitation that it took time.

Also, conventionally, a non-contact type three-dimensional measuring device has been known. Since the non-contact type is capable of faster measurement than the contact type, it is used for data input to a CG system or a CAD system, body measurement, visual recognition of a robot, and the like.

A slit light projection method (also called a light section method) or a pattern projection method is known as a non-contact three-dimensional measurement method. These methods actively irradiate a measurement object with specific reference light (also referred to as detection light) and obtain a distance image (also referred to as three-dimensional image, three-dimensional data, or three-dimensional shape data) based on the principle of triangulation. It is a kind of measurement method.
In the slit light projection method, a measurement target is scanned by irradiating and deflecting slit light. In the pattern projection method, a plurality of two-dimensional pattern lights are sequentially irradiated. The obtained distance image is a set of pixels indicating the three-dimensional positions of a plurality of parts on the measurement target.

In such a three-dimensional measuring apparatus, a distance measuring optical system for obtaining a distance image of the object to be measured and a color optical system (also called a monitor optical system) for acquiring texture information of the surface of the object. And are provided. The distance measuring optical system includes a light projecting unit that irradiates the measurement target object with the reference light, a light receiving sensor that receives the reflected light of the reference light from the measurement target object, and the like. Three-dimensional shape data is calculated based on the output from the light receiving sensor.

Further, the color optical system includes an image sensor for picking up a color image (also referred to as a monitor image, two-dimensional image, two-dimensional data, or two-dimensional image data) of the object to be measured. The color image obtained by the color optical system is used to obtain the texture information of the surface of the target object, and the range of the distance image obtained by the distance measurement is confirmed in advance when the measurement by the distance measuring optical system is started. It is also used to specify the correction point when correcting the obtained range image.

[0007]

When the photographing environment and the reproduction environment (observation environment) are different, for example, the shape of the luminaire,
If the position and color are different, the specular reflection state of the target object will change, and the appearance will also change significantly. However, in the conventional three-dimensional image processing apparatus, the texture is simply pasted on the surface of the target object, and the reproduction of the texture and glossiness of the target object is not considered. When the image processing apparatus is used, it is difficult to accurately convey the above-described texture and glossiness of the target object.

The present invention has been made in view of the above problems, and it is possible to more realistically reproduce the texture, glossiness, and stereoscopic effect of a target object in accordance with the observation environment designated by the observer. The purpose is to provide three-dimensional image data.

[0009]

A three-dimensional image processing method according to the present invention for achieving the above object sets an input environment according to setting information, measures a target object, and expresses a three-dimensional shape and a surface attribute. An acquisition step of acquiring measurement data; a generation step of analyzing the measurement data acquired in the acquisition step to generate integrated data representing surface attributes of the three-dimensional shape model of the target object and each part thereof; In the generation of, a determination step of determining whether or not it is necessary to acquire further measurement data by the acquisition step for the target object, and if it is determined by the determination step that further measurement data is necessary, the setting And an execution step of updating the information and executing the processing of the acquisition step.

A three-dimensional image processing apparatus according to the present invention for achieving the above object has the following configuration. That is, the input object is set according to the setting information, the target object is measured, and an acquisition unit that acquires the measurement data representing the three-dimensional shape and the surface attribute, and the measurement data acquired by the acquisition unit is analyzed to analyze the target object. Generating means for generating integrated data representing the surface attributes of the three-dimensional shape model and each part thereof, and at the time of generating the integrated data, it is determined whether or not it is necessary to acquire further measurement data for the target object by the acquiring means. And a executing unit that updates the setting information and executes the process of the acquiring unit when the measuring unit determines that further measurement data needs to be acquired.

[0011]

Preferred embodiments of the present invention will be described below with reference to the accompanying drawings.

<First Embodiment> FIG. 1 is a block diagram showing the schematic arrangement of a three-dimensional image processing apparatus according to the first embodiment. The three-dimensional image processing apparatus according to the present embodiment, when reproducing the target object, can arbitrarily set the illumination environment used for observation, and can freely change the position and orientation of the target object in the observation space. It is possible to display an image of a target object under an observation environment and obtain a printout.

The three-dimensional image processing apparatus of this embodiment includes a three-dimensional shape acquisition unit 13 for acquiring the three-dimensional shape of a target object,
The surface attribute acquisition unit 11 that acquires the color, gloss, texture, etc. of the target object, the three-dimensional data integration unit 15 that integrates these three-dimensional data, and the reliability of the created three-dimensional integrated data are determined and re-measured. In order to determine the necessity and to perform three-dimensional shape acquisition or surface attribute acquisition in accordance with the data credibility determination unit 14 that updates the input environment parameter based on the determined credibility, and the input environment parameter updated by the data credibility determination unit 14. An image of the target object based on the control unit 12 that controls the three-dimensional shape acquisition unit 13 and the surface attribute acquisition unit 11, the operation unit 17 that sets the observation environment during reproduction, and the three-dimensional integrated data and the observation environment data during reproduction. And an arbitrary environment image generation unit 16 for reconstructing the image and an image output unit 18 for displaying and printing out.

The three-dimensional shape acquisition unit 13 uses a laser radar method, a slit light projection method, a pattern projection method, or the like.
A three-dimensional solid shape model is generated from the measurement data obtained from the dimension measuring device. Here, as the three-dimensional solid shape model, for example, a surface model with polygons or a model with a set of surface shape elements of different shapes is used to
Represents a three-dimensional shape.

The surface attribute acquisition unit 11 acquires image data obtained from an image input device such as a digital still camera, a video camera, a multispectral camera, etc., which photoelectrically converts an optical image into image data and inputs the image data, and a three-dimensional shape acquisition. Using the three-dimensional solid shape model obtained by the unit 13, the surface attribute parameters related to the color, gloss, and texture of the target object are estimated, and the surface attribute data is generated. With this surface attribute data, the position, spread, shape, intensity, etc. of specular reflection in the reproduced image can be determined according to changes in the observation environment such as the positional relationship of the target object, illumination light source, viewpoint during reproduction, and the color and shape of the illumination light source. It is possible to change, and it is possible to more realistically express the texture, glossiness, stereoscopic effect, etc. of the target object. Here, the three-dimensional measuring device for generating the three-dimensional solid shape model and the image input device for generating the surface attribute data may be separate or integrated.

The three-dimensional data integration unit 15 integrates the three-dimensional solid shape model obtained by the three-dimensional shape acquisition unit 13 and the surface attribute data obtained by the surface attribute acquisition unit 11 to obtain the three-dimensional shape of the target object. Create integrated data.

The data credibility discriminator 14 uses the created 3
The credibility of dimensional integrated data is determined, and the necessity of re-measurement is determined. If the reliability of the data is sufficient and re-measurement is not required, the three-dimensional integrated data at that time is confirmed. On the other hand, when the reliability of the data is insufficient and it is determined that the re-measurement is necessary, the input environment parameter used in the immediately preceding measurement is changed and passed to the control unit 12.

The control unit 12 again acquires the three-dimensional shape or the surface attribute according to the input environment parameter. The above processing is repeatedly performed until the data credibility determination unit 14 determines that the credibility of the three-dimensional integrated data is sufficient.

The observer can set the observation environment such as desired illumination conditions and the position and orientation of the target object through the operation unit 17. The operation unit 17 generates the inspector today data according to the setting by the observer and sends it to the arbitrary environment image generation unit 16. The arbitrary environment image generation unit 16 reconstructs an image of the target object under the observation environment desired by the observer according to the observation environment data set by the operation unit 17. The image output unit 18 includes a TV monitor, a printer, and the like, and displays and prints out the image reconstructed by the arbitrary environment image generation unit 16.

FIG. 2 is a schematic view of an apparatus for three-dimensional shape acquisition and surface attribute acquisition of this embodiment. The target object 21 is placed on a computer-controllable rotary stage 25. The rotation stage 25 is a computer 29.
It continuously rotates about the rotary shaft 26 at an angle instructed by. The position of the rotary shaft 26 in the measurement environment is known.

The illumination light source 27 is an illumination used when acquiring the surface attribute data, and one or plural illumination light sources are arranged, and according to a command from the computer 29, the illumination light spectrum, the light intensity, the illumination light shape, the number, and the like. It is possible to control the position in the measurement environment.

The three-dimensional measuring device / image input device 23 is arranged at a position distant from the target object 21, and the computer 2
By the command from 9, it is possible to move to an arbitrary position in the measurement space, perform measurement, and input data.

The computer 29 includes a rotary stage 25,
A command is sent to the illumination light source 27 to determine the measurement environment. Then, a command is sent to the three-dimensional measuring device / image input device 23 to perform three-dimensional shape measurement and image input.
Here, the three-dimensional measuring device and the image input device may be integrated or separate. Based on the measured data thus obtained, the three-dimensional shape and surface attribute data are acquired, and the three-dimensional integrated data is created.

Next, the third embodiment of the present invention having the above configuration
The operation of the three-dimensional image processing apparatus will be described in detail. First, the process from acquisition of a three-dimensional shape model and surface attribute data by actual measurement of a target object to generation of three-dimensional integrated data will be described.

The surface attribute acquisition unit 11 uses the measured data and the three-dimensional shape data to estimate the surface attribute parameters related to the color, gloss, and texture of the target object, and generate the surface attribute data. FIG. 3 is a block diagram showing the configuration of the surface attribute acquisition unit 11. In FIG. 3, the actual measurement data sorting section 31 sorts out actual measurement data, for example, by removing data from the input actual measurement data that lacks information due to influences of noise, shadow, occlusion, color mixture, or has a large error. Make a choice. The diffuse reflection area extraction unit 33
Only the diffuse reflection component data is extracted from the selected actual measurement data. The specular reflection component separation unit 35 separates the specular reflection component from the data including the specular reflection component. The flow of the estimation process will be described later in detail.

The control unit 12 acquires a three-dimensional shape and a surface attribute according to the input environment parameters. FIG. 4 is a block diagram showing the configuration of the control unit 12. In FIG.
The rotation stage control unit 47 sequentially controls the rotation angle of the rotation stage at each rotation interval determined by the input environment parameter. The illumination light source control unit 48
It controls the illumination light spectrum, light intensity, illumination light shape, number, and position in the measurement environment. The three-dimensional measuring device control unit 45 moves the three-dimensional measuring device to a measurement position obtained from the input environment parameter and continuously inputs data.
The image input device control unit 46 moves the image input device to a measurement position obtained from the input environment parameter and continuously inputs data. It should be noted that the input environment description data (43) is used in the second embodiment and will not be described here.

Next, the processing from the actual measurement of the target object to the creation of the three-dimensional integrated data in the three-dimensional image processing apparatus of the first embodiment will be described with reference to the flowchart of FIG.

In step S51, the three-dimensional measuring device / image input device 23 for each vertex on the surface of the target object.
The actual measurement data is input by, and in step S52,
Three-dimensional shape data is created using the actual measurement data acquired by the three-dimensional measuring device. In step S53, the data credibility determination unit 14 evaluates the credibility of the three-dimensional shape data and determines whether re-measurement is necessary (the shape data determination unit 141 (FIG. 14) described later according to the third embodiment).
Done by). If there is a portion where the reliability of the data is not sufficient, the process proceeds to step S62, the input environment parameter is changed for re-measurement, and then the process returns to step S51. The input environment parameters include the rotation interval of the rotary stage 25 to be actually measured, the position of the three-dimensional measuring device / image input device 23, the illumination light spectrum of the illumination light source 27,
It includes the intensity of light, the shape of illumination light, the number, the position, etc., and these are determined as one set of values. It should be noted that "determined by one value" means that the position of the three-dimensional measuring device / image input device, the illumination light spectrum of the illumination light source, the light intensity, the illumination light shape, the number, and the position for each data input. It means that all data is input in the same environment without changing the above (however, the stage is rotated based on the rotation interval of the rotation stage).

On the other hand, if it is determined in step S53 that the reliability of the three-dimensional shape data is sufficiently high and there is no need for re-measurement, surface attribute estimation processing is performed in step S54 and subsequent steps. Here, the characteristic of the dichroic reflection model that the reflected light on the surface of the object is divided into two components, a diffuse reflection component and a specular reflection component. This allows
By analyzing the changes in the brightness and color of the corresponding points of each vertex in the actual measurement data measured under a plurality of environments, the surface attribute at each vertex can be estimated.

First, in step S54, the actual measurement data selection section 31 lacks information due to the influence of noise, shadow, occlusion, color mixture, etc. in the actual measurement data input from the image input device in step S51. Select the actual measurement data, such as removing the data with large error. Then, in step S55,
Only the actual measurement data selected in step S54 is used to extract the data including only the diffuse reflection component. Then, in step S56, from the extracted data,
Of the surface attribute parameters, the parameter related to the object color is estimated.

Subsequently, in step S57, the specular reflection component is separated from the data including both the diffuse reflection component and the specular reflection component, and in step S58, the parameter relating to the reflection characteristic of the surface attribute parameters is estimated. Note that there are two types of actual measurement data, data that includes only diffuse reflection components (data that does not include specular reflection components) and data that includes both diffuse reflection components and specular reflection components, and is extracted in step S55. Is the former (data of only the diffuse reflection component). In step S57, the latter data (data including both the diffuse reflection component and the specular reflection component) is separated into the diffuse reflection component and the specular reflection component.

The surface attribute parameter at each vertex is estimated by performing the above processing for each vertex, and the entire surface of the target object is estimated (step S59). When the estimation of the entire surface of the target object is completed, the process proceeds to step S60, and the estimation result is used to create three-dimensional integrated data as described later with reference to FIG. Then, in step S61,
The reliability of the created three-dimensional integrated data is evaluated, and it is determined whether or not re-measurement is necessary (performed by the surface attribute data determination unit 142 (FIG. 14) described later according to the third embodiment). If it is determined that the credibility of the estimated values at all the vertices is sufficiently high and there is no need for re-measurement, this processing ends. On the other hand, if there is a vertex where the reliability of the estimated value is not sufficient, the process proceeds to step S62, and the input environment parameter is changed for re-measurement. Then, the process returns to step S51, and the process from the actual measurement is repeated based on the changed input environment parameter.

The above processing is repeated until it is determined that the reliability of the data is sufficient. Here, the constituent unit of the surface of the target object is described as a vertex, but it is also possible to use a minute surface instead of the vertex as described later.

FIG. 6 is a diagram showing an example of the three-dimensional integrated data created by the three-dimensional data integrating unit 15 in the first embodiment. The three-dimensional coordinates of one vertex and the structure for storing the surface attribute parameters such as the object color information and the reflection characteristic information are used as elements, and the elements are prepared for the number of vertices. Even in one measurement, the data of each vertex is not obtained at the same time, but is obtained point by point. Then, each time a new vertex is generated and added as the three-dimensional integrated information, the number of elements of the three-dimensional integrated data is expanded.

It is also possible to use a minute surface instead of the vertex as a constituent element. In this case, the structure that stores the three-dimensional position information and the surface attribute information of the microfacet becomes an element, and this element is prepared by the number of microfacets.
The minute surface which is an element is generated by integrating the neighboring vertices or minute surfaces having similar surface attribute information. Here, the joining of the minute surfaces is performed to the extent that the three-dimensional shape is not impaired. The three-dimensional position information of the minute surface is represented by, for example, the coordinates of the three vertices of the triangle and the normal vector. Further, the surface attribute information of the minute surface is composed of object color information and reflection characteristic information, and is expressed in a parameter format or an image format corresponding to the minute surface (for example, a texture image corresponding to the minute surface). The number of elements of the three-dimensional integrated data is expanded each time a new minute surface is generated and added as the three-dimensional integrated information.

The above-mentioned three-dimensional integrated data is placed in the memory of the device during the acquisition of the three-dimensional shape and the surface attribute, and when the acquisition of the three-dimensional shape and the surface attribute is completed, it is stored in a storage medium such as a hard disk. Saved.

Next, a process of generating an image of a target object in a desired observation environment using the three-dimensional integrated data generated as described above will be described.

The operating section 17 is operated by an observer to input
Create observation environment data such as desired lighting conditions and the position and orientation of the target object. FIG. 7 is a block diagram illustrating the configuration of the operation unit 17. In FIG. 7, an illumination light source setting unit 71
Is for setting the illumination light to be emitted to the target object at the time of reproduction. For example, the color, brightness, position, shape (point light source, line light source, surface light source, parallel light, etc.), the number of illumination light, etc. are set. The target object placement setting unit 73 performs setting relating to the position of the target object, and instructs, for example, three-dimensional movement of the target object up and down, left and right, front and back, and rotation about an arbitrary axis. Based on these set values, create observation environment data to be used for reproduction.

Next, in the three-dimensional image processing apparatus of this embodiment, the operation unit 17, the arbitrary environment image generation unit 16 and the image output unit for reproducing the image of the target object under the observation environment desired by the observer. The operation of 18 will be described.

FIG. 8 is a diagram showing a user interface for presenting an image in an arbitrary environment of the three-dimensional image processing apparatus according to this embodiment. In the present embodiment, the operation unit 17 is realized by a user interface on the screen as shown in FIG.

The observer sets the color, brightness, position, and shape (point light source, line light source, surface light source, parallel light, etc.) of the illumination light with which the target object is illuminated, using the illumination light source setting panel 83.
Here, a plurality of illumination light sources can be set. This allows any color in the observation space,
One or more illumination sources having brightness and shape can be placed at any location as shown at 82. The illumination light source setting panel 83 is provided by the illumination light source setting unit 71 of FIG. 7.

Further, the observer operates the operation buttons 84 to 88.
With the target object placement setting panel 89 provided with, three-dimensional movements up and down, left and right, front and back, and rotation about an arbitrary axis are set with respect to the target object. Specifically, the button 84 can be used to move the target object vertically and horizontally, and
With 5, it is possible to move to the front and back. Also, buttons 86, 8
Reference numerals 7 and 88 respectively indicate the rotation of the target object about the axes Y, Z, and X.

As a result, the target object 81 can be moved and rotated in an arbitrary direction in the observation space, and the target object can be observed from a desired direction.

FIG. 9 shows the operation unit 17 and the arbitrary environment image generation unit 1.
6 is a flowchart illustrating an image generation process of a target object according to No. 6. When the setting of the illumination light source or the setting of the target object arrangement is changed in the operation unit 17, the observation environment data is updated (steps S91 and S92). The arbitrary environment image generation unit 16 converts the observation environment data set by the illumination light source setting unit 71 (83) and the target object arrangement setting unit 73 (89) into the three-dimensional data generated by the three-dimensional data integration unit 15. Based on this, the image of the target object under the observation environment desired by the observer is reconstructed (step S9
3), and present this in the window 80 (step S9)
4). The presentation of the image of the target object is not limited to the display, and may be a printout by a printer. It should be noted that the integrated data is the modeling data of the object in CG, and the rendering processing corresponding to the observation environment is performed on this modeling data in the construction of the image.

As described above, the position, spread, and shape of specular reflection in the reproduced image are set according to the observation object such as the positional relationship among the target object, the illumination light source, and the viewpoint set by the observer and the color, brightness, and shape of the illumination light source. -Because the strength and the like can be changed, it is possible to more realistically express the texture, glossiness, and stereoscopic effect of the target object.

As described above, the three-dimensional shape and surface attributes of the target object are acquired, integrated as three-dimensional integrated data, and the image of the target object under the observation environment set by the observer is reconstructed and presented. As a result, it is possible to more realistically convey the texture, glossiness, and stereoscopic effect of the target object to the observer. In addition, it is possible to improve the accuracy of the three-dimensional shape measurement and the surface attribute estimation by automatically determining the credibility of the acquired data and repeating the three-dimensional shape acquisition or the surface attribute acquisition as needed, and thus the reality can be improved. High object drawing is possible.

<Second Embodiment> Next, a second embodiment will be described.

FIG. 10 is a block diagram showing the schematic arrangement of a three-dimensional image processing apparatus according to the second embodiment. 10, components having the same functions as those shown in FIG. 1 are designated by the same reference numerals. The three-dimensional image processing apparatus according to the second embodiment, like the first embodiment,
When reproducing the target object, it is possible to arbitrarily set the illumination environment used for observation, and it is possible to freely change the position and orientation of the target object within the observation space. It is possible to display or print out an image.

The three-dimensional image processing apparatus according to the second embodiment has a three-dimensional shape acquisition unit 13 for acquiring the three-dimensional shape of a target object and a surface attribute acquisition unit 11 for acquiring the color, gloss, texture, etc. of the target object. And integrating these 3D data 3
The dimensional data integration unit 105, the three-dimensional intermediate data holding unit 108 that holds the intermediate data of the three-dimensional integrated data, and the data credibility determination unit that determines the credibility of the created three-dimensional intermediate data and determines the necessity of re-measurement. 104, an input environment description data holding unit 106 that holds the input environment description data created by the data credibility determination unit 104, and a three-dimensional shape acquisition unit 13 for performing three-dimensional shape acquisition or surface attribute acquisition according to the input environment description data. And surface attribute acquisition unit 1
1, a control unit 102 for controlling 1, an operation unit 17 for setting an observation environment at the time of reproduction, and an arbitrary environment image generation unit 16 for reconstructing an image of the target object based on the three-dimensional integrated data and the observation environment data at the time of reproduction. , And an image output unit 18 for displaying and printing out. In the input environment description data used in the second embodiment, not the rotation interval but all the rotation positions of the rotary stage at each data input are described. Each time the data is input, it is described in the input environment description data. The stage is set to the rotated position.

Three-dimensional shape acquisition unit 13, surface attribute acquisition unit 1
1, the operation unit 17, the arbitrary environment image generation unit 16, and the image output unit 18 are the same as those in the first embodiment, and therefore description thereof will be omitted.

The three-dimensional data integration unit 105 integrates the three-dimensional solid shape model obtained by the three-dimensional shape acquisition unit 13 and the surface attribute data obtained by the surface attribute acquisition unit 11 into three-dimensional data of the target object. The three-dimensional intermediate data holding unit 1 updates the three-dimensional intermediate data that is the intermediate data of the integrated data.
Hold at 08.

The data credibility discriminating unit 104 discriminates the credibility of the created three-dimensional intermediate data and judges the necessity of re-measurement. When it is determined that the reliability of the data is sufficient and re-measurement is unnecessary, the three-dimensional data integration unit 105
Creates three-dimensional integrated data from the held three-dimensional intermediate data. On the other hand, when the reliability of the data is insufficient and it is determined that the re-measurement is necessary, the input environment description data for the re-measurement is created and held in the input environment description data holding unit 106.

The control unit 102 again acquires the three-dimensional shape or the surface attribute according to the input environment description data. The above processing is repeated until it is determined that the reliability of the data is sufficient. Here, the re-measurement does not necessarily have to be performed on the entire surface of the target object. Therefore, the second
In the embodiment of the present invention, by describing information such as the actual measurement position in the input environment description data, only the area where the credibility of the data is low is intensively measured, and only the corresponding portion of the three-dimensional intermediate data is measured each time. It is possible to update
To enable efficient measurement. here,
Correspondence between the element obtained by the current measurement and the element of the intermediate data obtained by the previous measurements is determined based on the three-dimensional position information.

Next, the configuration of the control unit 102 will be described with reference to FIG. In the second embodiment, the control unit 102
Instead of the input environment parameters in the first embodiment, the input environment description data 43 is used to perform three-dimensional shape acquisition and surface attribute acquisition. Rotation stage controller 47
Controls the rotation angle of the rotary stage based on the input environment description data. Further, the illumination light source control unit 48 controls the illumination light spectrum, the light intensity, the illumination light shape, the number, and the position in the measurement environment for the illumination light source based on the input environment description data. The three-dimensional measuring device control unit 45 controls the position of the three-dimensional measuring device based on the input environment description data and inputs data. Image input device control unit 46
Controls the position of the image input device based on the input environment description data and inputs data.

With the input environment parameter in the first embodiment, continuous measurement is performed only for a certain set value, but in this embodiment, by using the input environment description data, only a part of the target object is emphasized. Actually measured,
Since it is possible to perform finer control such as actual measurement while changing the illumination light source, it is possible to further improve the accuracy of data acquisition.

FIG. 11 is a diagram showing an example of three-dimensional intermediate data created by the three-dimensional data integration unit 105. The three-dimensional coordinates of each apex, the surface attribute parameters such as object color information and reflection characteristic information, and the structure for storing the reliability of the data are used as elements. Each time a new vertex is generated and added as the three-dimensional integrated information, the number of elements of the three-dimensional integrated data is expanded. In addition, when data with higher credibility can be acquired for some elements by re-measurement, only the data of the corresponding element is rewritten to update the three-dimensional intermediate data. Whether or not the reliability of the data obtained by the re-measurement is higher can be determined by using the reliability stored as an element.

Here, a minute surface may be used instead of the vertex as a constituent element. In this case, the structure for storing the three-dimensional position information and the surface attribute information of the minute surface is used as an element, and the elements are prepared by the number of minute surfaces. The minute surface which is an element is generated by integrating the neighboring vertices or minute surfaces having similar surface attribute information. Here, the joining of the minute surfaces is performed to the extent that the three-dimensional shape is not impaired. The three-dimensional position information of the minute surface is represented by, for example, the coordinates of the three vertices of the triangle and the normal vector. The surface attribute information of the minute surface is composed of object color information and reflection characteristic information, and is expressed in a parameter format or an image format corresponding to the minute surface.

FIG. 12 is a diagram showing an example of the input environment description data created by the data credibility determination unit 104. The position of the three-dimensional measuring device and the position of the image input device for performing three-dimensional measurement and image input for each data input environment,
It has a structure that stores the rotation angle of the rotary stage and the position, spectrum, intensity, shape, and number of illumination light sources, and holds this structure for the number of times of data input. Based on this description, the control unit 102 performs three-dimensional shape acquisition and surface attribute acquisition.

Next, the processing from the actual measurement to the creation of the three-dimensional integrated data in the three-dimensional image processing apparatus of the second embodiment will be described with reference to the flowchart of FIG. 13 and FIG.

In step S131, the three-dimensional measuring device / image input device 2 is used for each vertex on the surface of the target object.
The actual measurement data is input by 3, and in step S132, the three-dimensional shape data is created using the actual measurement data acquired by the three-dimensional measuring device. In step S133, the reliability of the three-dimensional shape data is evaluated, and it is determined whether or not re-measurement is necessary (performed by the shape data determination unit 141 (FIG. 14) described later according to the third embodiment). If there is a portion where the reliability of the data is not sufficient, the process proceeds to step S143, and input environment description data is created for re-measurement. In the input environment description data, as described with reference to FIG. 12, the rotation angle of the input environment data rotation stage 25 for the number of data inputs to be performed (N times in FIG. 12),
Position of three-dimensional measuring device / image input device 23, illumination light source 2
7 illumination light spectrum, light intensity, illumination light shape, number,
The position is described.

If the reliability of the data is sufficiently high and it is judged that there is no need for re-measurement, then the surface attribute estimation processing is performed. Here, the characteristic of the dichroic reflection model that the reflected light on the surface of the object is divided into two components, a diffuse reflection component and a specular reflection component. Thus, the surface attribute at each vertex can be estimated by analyzing the change in the brightness / color of the corresponding point of each vertex in the measured data measured under a plurality of environments.

First, in step S134, the actually measured data selection section 31 selects, from the actually measured data input in step S131, data that lacks information or contains large errors due to the effects of noise, shadow, occlusion, color mixture, and the like. The data is removed and the actual measurement data is selected (step S134). Subsequently, in step S135, the diffuse reflection area extraction unit 33 extracts the data including only the diffuse reflection component using the selected actual measurement data. Then, in step S136, of the surface attribute parameters, the parameter relating to the object color is estimated using the data extracted in step S135.

Subsequently, in step S137, the specular reflection component separation unit 35 separates the specular reflection component from the data including both the diffuse reflection component and the specular reflection component, and in step S138, among the surface attribute parameters. Estimate parameters related to reflection characteristics. At the same time, the reliability of the data is calculated, and the value of the three-dimensional intermediate data is updated for the vertex whose reliability is higher than the currently held three-dimensional intermediate data (step S139).

By performing the above processing for each vertex, the surface attribute parameter at each vertex is estimated, and the entire surface of the target object is estimated. When the estimation of the entire surface of the target object is completed (step S140), the reliability of the held three-dimensional intermediate data is evaluated and it is determined whether re-measurement is necessary (step S141) (third embodiment). By the surface attribute data discriminating unit 142 (FIG. 14) described later.

If the reliability of the estimated values at all the vertices is sufficiently high and it is judged that there is no need for re-measurement, three-dimensional integrated data is created from the three-dimensional intermediate data (step S142), and this processing ends. To do. On the other hand, if there is a vertex where the reliability of the estimated value is not sufficient, the process proceeds to step S143 to perform the actual measurement again, and the input environment description data is created. Then, the process returns to step S131, and the process from the actual measurement is performed again based on the created input environment description data. By repeating the above processing and updating a part of the three-dimensional intermediate data as needed, three-dimensional integrated data with a sufficiently high degree of credibility is created. Although the constituent unit of the surface of the target object has been described as a vertex here, a minute surface can be used instead of the vertex.

The three-dimensional integrated data created by the three-dimensional data integrating unit 105, the configuration of the operating unit 17, the operating unit 17 for reproducing the image of the target object under the observation environment desired by the observer, and the arbitrary The processing of the environment image generation unit 16 is the same as that of the first embodiment (FIGS. 6 to 9).

As described above, it is possible to efficiently measure only a part of the target object by using the input environment description data and to update only the corresponding part of the three-dimensional intermediate data at any time, thereby efficiently Actual measurement can be performed. In addition, since it is possible to perform finer control such as focusing on only the area where the reliability of the data is low or measuring while changing the illumination light source, it is possible to further improve the accuracy of data acquisition. <Third Embodiment> In this embodiment, the data credibility determination unit used in the first and second embodiments will be described.

FIG. 14 is a schematic configuration diagram showing the configuration of the data credibility determination unit (14, 104) according to the third embodiment. The credibility discriminating units 14 and 104 discriminate the credibility of the created three-dimensional shape data and surface attribute data, and update the input environment parameters or the input environment description data for re-measurement as necessary.

In FIG. 14, the shape data discrimination unit 141
Determines the presence or absence of occlusion and the complexity of the shape based on the three-dimensional shape data obtained by the three-dimensional shape acquisition unit 13. If the actual measurement is insufficient for the degree of occlusion or the complexity of the shape,
The rotation interval of the rotary stage in the input environment parameter is changed, and the re-measurement is instructed to the control units 12 and 102. At this time, the position of the three-dimensional measuring device / image input device 23 may be changed together. Alternatively, in the case of input environment description data, the position of the three-dimensional measuring device / image input device 23 and the rotation angle of the rotary stage 25 are changed to optimum values.

Based on the surface attribute data obtained by the surface attribute acquisition section 11, the surface attribute data determination section 142 determines the similarity between the illumination light source color and the target object color, the state of the specular reflection component peak value, and the diffuse reflection area. Whether or not remeasurement is necessary is judged based on the presence / absence, the ratio of specular reflection components, the uniformity of the color distribution of corresponding points, and the variation in attribute data between neighboring areas. If remeasurement is necessary, input environment parameters or input environment Generate descriptive data,
Update. For example:

(1) When the illumination light source color and the target object color are very similar: In this case, the estimation accuracy of the target object color is lowered. Therefore, when it is determined that the similarity between the illumination light source color and the target object color is high, the illumination light source spectrum in the input environment parameter, or the position of the three-dimensional measuring device / image input device 23 in the input environment description data,
The rotation angle of the rotary stage 25 and the illumination light source spectrum are changed to optimum values.

(2) When there are not many areas of diffuse reflection component only: In this case as well, the estimation accuracy of the target object color is reduced. Therefore, when it is determined that the area of only the diffuse reflection component is insufficient, the position and shape of the illumination light source in the input environment parameter, or the position and rotation of the three-dimensional measuring device / image input device 23 in the input environment description data. The rotation angle of the stage 25 and the position / shape / number of the illumination light sources 27 are changed to optimum values.

(3) When the degree of specular reflection is strong and the peak shape is collapsed, or conversely, when the degree is weak and the peak shape is not clearly shown: In these cases also, the illumination light source intensity in the input environment parameter, or The position of the three-dimensional measuring device / image input device, the rotation angle of the rotary stage, and the illumination light source intensity in the input environment description data are changed to optimum values to improve the accuracy of specular reflection characteristic estimation.

(4) When it is determined that the data including the specular reflection component is insufficient: In this case, the illumination light source position / shape in the input environment parameter, or the three-dimensional measuring device / image in the input environment description data The position of the input device, the rotation angle of the rotary stage, and the position / shape / number of illumination light sources are changed to optimum values.

(5) When there is a large variation in the color distribution of the corresponding points of a certain constituent element, or when there is a large change in the surface attribute data between constituent elements in the neighboring area: In these cases,
Considering that the constituent element is an edge area where the surface attribute changes, the position of the three-dimensional measuring device / image input device 23 in the input environment parameter or the three-dimensional measuring device in the input environment description data can be measured so as to be measured with sufficient resolution. / Change the position of the image input device 23 and the rotation angle of the rotary stage 25 to optimum values.

As described above, the credibility of the three-dimensional shape data and the surface attribute data is discriminated, and the accuracy of the three-dimensional shape measurement and the surface attribute estimation is improved by feeding back to the environment setting for re-measurement if necessary. Can be made.

<Fourth Embodiment> In the fourth embodiment,
In the three-dimensional image processing device shown in the second embodiment,
A configuration for reproducing and displaying a target object in a desired observation environment using the three-dimensional integrated data is provided in an external device connected by a network.

FIG. 15 is a block diagram showing the schematic arrangement of a three-dimensional image processing apparatus according to the fourth embodiment. In the three-dimensional image processing apparatus according to the fourth embodiment, the device that captures the data of the target object and the device that reproduces the image are located apart from each other, and the image is reproduced using data transmission via the network.

In FIG. 15, the device for fetching the data of the target object has a three-dimensional shape acquisition unit 13 for acquiring the three-dimensional shape of the target object and a surface attribute acquisition for acquiring the color, gloss, texture, etc. of the target object. A unit 11, a three-dimensional data integration unit 105 that integrates these three-dimensional data to create the three-dimensional integrated data shown in FIG. 11, and a network interface 151 that transmits the created three-dimensional integrated data through a network. A three-dimensional intermediate data holding unit 108 that holds intermediate data of the three-dimensional integrated data, a data credibility determination unit 104 that determines the credibility of the created three-dimensional intermediate data and determines the necessity of re-measurement, and a data credibility determination An input environment description data holding unit 1 that holds the input environment description data as shown in FIG. 12 created by the unit 104.
06, and a control unit 102 that controls the three-dimensional shape acquisition unit 13 and the surface attribute acquisition unit 11 to perform three-dimensional shape acquisition or surface attribute acquisition according to the input environment description data. The configuration of the device on the side that captures the data of these target objects is the same as that of the second embodiment (FIG. 10) except for the network interface 151.

The device on the side that reproduces the target object using the three-dimensional integrated data receives the three-dimensional integrated data transmitted through the operation unit 152 and the network for setting the observation environment at the time of reproduction. An arbitrary environment image generation unit 153 that reconstructs an image of the target object based on the observation environment at reproduction (set by the operation unit 152) and an image output unit 154 that displays and prints out.

In the above configuration, the three-dimensional data integration unit 1
If the data credibility discriminating unit 104 determines that the credibility of the three-dimensional intermediate data is sufficient and does not require re-measurement, the data 05 determines the three-dimensional integrated data of the target object from the held three-dimensional intermediate data. It is created and sent to the network interface 151. The network interface 151 transmits the received three-dimensional integrated data to the arbitrary environment image generation unit 152 via the network.

In the device for reproducing the target object,
The observer uses the operation unit 152 to set the observation environment such as desired illumination conditions and the position and orientation of the target object.

The arbitrary environment image generation unit 153 receives the three-dimensional integrated data transmitted through the network,
According to the observation environment data set by the operation unit 152,
The image of the target object under the observation environment desired by the observer is reconstructed. The image output unit 154 includes a TV monitor, a printer, and the like, and displays and prints out the image reconstructed by the arbitrary environment image generation unit 153.

The configuration of the operation unit 152, and the processes of the operation unit 152 and the arbitrary environment image generation unit 153 for reproducing the image of the target object under the observation environment desired by the observer are the first embodiment. (FIGS. 6 to 9).

As described above, by making it possible to transmit and receive the three-dimensional integrated data via the network, in the case where the device that takes in the data of the target object and the device that reproduces the image are located at distant places. However, it is possible to reproduce the image of the target object under the observation environment desired by the observer, and to convey the texture, glossiness, stereoscopic effect, etc. of the target object to the observer more realistically.

As is clear from the above description, according to each of the above embodiments, the three-dimensional shape and the surface attribute of the target object are acquired, integrated as three-dimensional integrated data, and under the observation environment set by the observer. The image of the target object is reconstructed and presented. Therefore, the shape, texture, and
Not only the color, but also the shape / position / direction / color of the illuminating light, the position / direction of the observer, the position / orientation of the target object, etc. It is possible to generate an image that conveys feelings in a more realistic manner.

Further, it is possible to improve the accuracy of the three-dimensional shape measurement and the surface attribute estimation by discriminating the credibility of the three-dimensional shape data and the surface attribute data and feeding back to the environment setting if necessary and re-measuring. it can. Therefore, it is possible to draw an object with higher reality.

Further, according to the second embodiment, only a part of the target object is intensively measured using the input environment description data, and only the corresponding part of the three-dimensional intermediate data is updated at any time. Thus, it is possible to efficiently perform the actual measurement for generating the three-dimensional integrated data. In addition, since it is possible to perform finer control such as focusing on only the area where the reliability of the data is low or measuring while changing the illumination light source, it is possible to further improve the accuracy of data acquisition.

Further, according to the fourth embodiment, the three-dimensional integrated data can be transmitted and received via the network, so that the place where the data of the target object is taken in and the place where the image is reproduced are at remote places. Even in such a case, it is possible to reproduce the image of the target object under the observation environment desired by the observer, and to convey the texture, glossiness, stereoscopic effect, etc. of the target object to the observer in a more realistic manner. Become.

Further, an object of the present invention is to supply a storage medium having a program code of software for realizing the functions of the above-described embodiments to a system or apparatus, and to supply a computer (or CPU) of the system or apparatus.
It is needless to say that it can be achieved by reading and executing the program code stored in the storage medium.

In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention.

As a storage medium for supplying the program code, for example, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD
-R, magnetic tape, non-volatile memory card, ROM, etc. can be used.

Further, not only the functions of the above-described embodiments are realized by executing the program code read by the computer, but also the OS (operating system) running on the computer based on the instructions of the program code. It is needless to say that this also includes a case where the above) performs a part or all of the actual processing and the processing realizes the functions of the above-described embodiments.

Further, after the program code read from the storage medium is written in the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, based on the instruction of the program code, It goes without saying that a case where the CPU or the like included in the function expansion board or the function expansion unit performs some or all of the actual processing and the processing realizes the functions of the above-described embodiments is also included.

[0095]

As described above, according to the present invention,
Three-dimensional image data capable of more realistically reproducing the texture, glossiness, stereoscopic effect, etc. of the target object is provided according to the observation environment designated by the observer.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a three-dimensional image processing apparatus according to a first embodiment.

FIG. 2 is a schematic diagram of an apparatus for three-dimensional shape acquisition and surface attribute acquisition according to the first embodiment.

FIG. 3 is a block diagram showing a configuration of a surface attribute acquisition unit.

FIG. 4 is a block diagram showing a configuration of a control unit.

FIG. 5 is a graph 3 from the actual measurement of the target object according to the first embodiment.
It is a flow chart explaining processing until creation of dimensional integration data.

FIG. 6 is a diagram showing an example of three-dimensional integrated data created by a three-dimensional data integrating unit in the first embodiment.

FIG. 7 is a block diagram illustrating a configuration of an operation unit.

FIG. 8 is a diagram showing a user interface for presenting an image under an arbitrary environment of the three-dimensional image processing apparatus according to the embodiment.

FIG. 9 is a flowchart illustrating an image generation process of a target object by an operation unit and an arbitrary environment image generation unit.

FIG. 10 is a block diagram showing a schematic configuration of a three-dimensional image processing device according to a second embodiment.

FIG. 11 is a diagram showing an example of three-dimensional intermediate data created by a three-dimensional data integration unit.

FIG. 12 is a diagram showing an example of input environment description data created by a data credibility determination unit.

FIG. 13 is a flowchart illustrating a process from actual measurement of a target object to creation of three-dimensional integrated data according to the second embodiment.

FIG. 14 is a schematic configuration diagram showing a configuration of a data credibility determination unit (14, 104) according to the third embodiment.

FIG. 15 is a block diagram showing a schematic configuration of a three-dimensional image processing device according to a fourth embodiment.

Claims (26)

[Claims] 1. An acquisition step of setting an input environment according to setting information, measuring a target object, and acquiring measurement data representing a three-dimensional shape and a surface attribute, and analyzing the measurement data acquired in the acquisition step. A generation step of generating integrated data representing a three-dimensional shape model of the target object and surface attributes of respective parts thereof, and whether or not further measurement data acquisition of the target object by the acquisition step is necessary in generating the integrated data. And a determination step of determining whether or not it is necessary to acquire further measurement data by the determination step, and an execution step of updating the setting information and executing the processing of the acquisition step. Three-dimensional image processing method. 2. The integrated data includes data indicating positions of vertices forming a three-dimensional shape model, and surface attribute reproduction parameters corresponding to the respective vertices. Three-dimensional image processing method. 3. The integrated data includes data indicating a position and a direction of a polygonal surface forming a three-dimensional shape model, and a surface attribute reproduction parameter corresponding to each polygonal surface. The three-dimensional image processing method according to claim 1. 4. A first determining step of determining a parameter relating to an object color of each part of a three-dimensional shape model by obtaining a diffuse reflection component from data representing a surface attribute of the measurement data, and the surface generating step. 3 is obtained by obtaining the specular reflection component from the data representing the attribute.
The three-dimensional image processing method according to claim 1, further comprising a second determining step of determining a parameter relating to a reflection characteristic of each part of the three-dimensional shape model. 5. The generating step further includes a removing step of previously removing data of a portion determined to have a larger error than the data representing the surface attribute, and then providing the data to the first and second determining steps. The three-dimensional image processing method according to claim 4, wherein. 6. The setting information includes a position of a measuring device used in the acquisition step, an attribute of an illumination system, a setting parameter that defines a rotation interval of a target object, and the acquisition step performs measurement based on the setting parameter. The three-dimensional image processing method according to claim 1, wherein the device is moved, an illumination system is set, and the measurement data is acquired while rotating the target object in accordance with the rotation interval. 7. The method according to claim 6, wherein the execution step changes at least one of a position of the measuring device, an attribute of an illumination system, and a rotation interval based on a result of the analysis by the generating step. The described three-dimensional image processing method. 8. The setting information describes a plurality of setting parameters that define the position of the measuring device, the illumination system, and the rotation angle of the target object included in the acquisition step, and the acquisition step includes the plurality of setting parameters. The three-dimensional image processing method according to claim 1, wherein measurement data is acquired by sequentially performing measurement according to each of the setting parameters. 9. The execution step describes a plurality of setting parameters such that a predetermined region of the target object is intensively measured in the acquisition step based on the analysis result of the generation step. The three-dimensional image processing method according to claim 8. 10. The generation step analyzes the measurement data acquired in the acquisition step to analyze the three-dimensional shape model of the target object, data indicating surface attributes of respective parts thereof, and data indicating reliability thereof. A holding step of generating data and holding the data, and a process of the acquisition step performed by the execution step, and as a result of analyzing the acquired measurement data, the reliability of the data is higher than that of the data held by the holding step. When new data is obtained, an updating step of updating the portion with the new data is provided, and when it is determined that further acquisition of data is unnecessary in the determination step, the data is held in the holding step. The three-dimensional image processing method according to claim 1, wherein the integrated data is generated based on final intermediate data. 11. A setting step of arbitrarily setting an observation environment including an attribute of an illumination system and a position and orientation of a target object, based on the observation environment set in the setting step and integrated data generated in the generation step. The three-dimensional image processing method according to claim 1, further comprising: an image generation step of generating image data of the target object, and an output step of outputting the image data generated in the image generation step. . 12. The three-dimensional image processing method according to claim 1, further comprising an output step of outputting the integrated data to an external device. 13. An acquisition unit for measuring an object by setting an input environment according to setting information and acquiring measurement data representing a three-dimensional shape and a surface attribute, and analyzing the measurement data acquired by the acquisition unit. Generating means for generating integrated data representing the three-dimensional shape model of the target object and surface attributes of each part thereof; and, in generating the integrated data, whether or not it is necessary to acquire further measurement data for the target object by the acquiring means. A determination unit that determines whether or not the determination unit determines that further measurement data needs to be acquired, and an execution unit that updates the setting information and executes the process of the acquisition unit. 3D image processing device. 14. The integrated data according to claim 13, wherein the integrated data includes data indicating positions of vertices forming a three-dimensional shape model, and surface attribute reproduction parameters corresponding to the respective vertices. Three-dimensional image processing device. 15. The integrated data includes data indicating a position and a direction of a polygonal surface forming a three-dimensional shape model, and a surface attribute reproducing parameter corresponding to each polygonal surface. The three-dimensional image processing device according to claim 13. 16. The first determining means determines the parameter relating to the object color of each part of the three-dimensional shape model by obtaining a diffuse reflection component from the data representing the surface attribute of the measurement data, and the surface. 3 is obtained by obtaining the specular reflection component from the data representing the attribute.
The three-dimensional image processing apparatus according to claim 13, further comprising a second determining unit that determines a parameter relating to a reflection characteristic of each part of the three-dimensional shape model. 17. The generating means further comprises a removing means for removing data of a portion determined to have a larger error than the data representing the surface attribute, and then providing the data to the first and second determining means. The three-dimensional image processing device according to claim 16, characterized in that. 18. The setting information includes a position of a measuring device used in the acquisition unit, an attribute of an illumination system, and a setting parameter that defines a rotation interval of a target object, and the acquisition unit measures based on the setting parameter. The three-dimensional image processing apparatus according to claim 13, wherein the measurement data is acquired while moving the apparatus, setting an illumination system, and rotating the target object in accordance with the rotation interval. 19. The method according to claim 18, wherein the executing unit changes at least one of the position of the measuring device, the attribute of the illumination system, and the rotation interval based on the result of the analysis by the generating unit. The three-dimensional image processing device described. 20. The setting information describes a plurality of setting parameters that define a position of a measuring device, an illumination system, and a rotation angle of a target object included in the acquisition unit, and the acquisition unit includes the plurality of setting parameters. The three-dimensional image processing apparatus according to claim 13, wherein measurement is sequentially performed according to each of the setting parameters to acquire measurement data. 21. The executing means describes a plurality of setting parameters based on a result of the analysis by the generating means so that a predetermined area of the target object is intensively measured by the acquiring means. 3. The method according to claim 2,
The three-dimensional image processing device according to item 0. 22. The generating means analyzes the measurement data acquired by the acquiring means, and includes a three-dimensional shape model of the target object, data representing surface attributes of respective parts thereof, and data representing their credibility. A holding unit that generates data and holds the data, and a process of the acquisition unit is executed by the execution unit, and as a result of analyzing the acquired measurement data, the reliability is higher than the data held by the holding unit. When new data is obtained, it is provided with updating means for updating that portion with the new data, and when the determination means determines that further acquisition of data is unnecessary, the data is held in the holding means. The three-dimensional image processing device according to claim 13, wherein the integrated data is generated based on final intermediate data. 23. Based on the setting means for arbitrarily setting an observation environment including the attributes of the illumination system and the position and orientation of the target object, the observation environment set by the setting means, and the integrated data generated by the generating means. 14. The three-dimensional image processing apparatus according to claim 13, further comprising: an image generating unit that generates image data of the target object; and an output unit that outputs the image data generated by the image generating unit. . 24. The three-dimensional image processing apparatus according to claim 13, further comprising an output unit that outputs the integrated data to an external device. 25. A control program for causing a computer to execute the three-dimensional image processing method according to any one of claims 1 to 12. 26. A storage medium storing a control program for causing a computer to execute the three-dimensional image processing method according to claim 1.