JP2018132452A - Image processor, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a process for obtaining the three-dimensional shape of an object with high accuracy without increasing the amount of exposure of the object to light from the conventional one.SOLUTION: Provided is an image processor comprising: first acquisition means for acquiring height data that indicates the height distribution of an object that was generated on the basis of image data obtained by imaging the object to which a plurality of mutually different patterns were projected; second acquisition means for acquiring reflectance data that indicates the reflectance distribution of the object that corresponds to the height distribution; and processing means for smoothing the height data by a smoothing filter in such a way that the cut-off frequency of a smoothing filter used for a first region in the reflectance distribution is lower than the cut-off frequency of a smoothing filter used for a second region whose reflectance is larger than the first region.SELECTED DRAWING: Figure 6

Description

  The present invention relates to an image processing technique related to measurement of a three-dimensional shape of an object.

  Conventionally, a phase shift method is known as a method for measuring the three-dimensional shape of an object. In the phase shift method, a bright and dark stripe pattern is projected onto the target object a plurality of times while shifting the phase, and the target object on which the stripe pattern is projected is imaged. For example, three types of images are acquired by projecting a pattern whose phase is shifted by 2π / 3 and 4π / 3 with respect to the initially projected fringe pattern and imaging the target object. At this time, the phase of each pixel is obtained by performing arithmetic processing using the pixel value (luminance) at the same pixel position in each captured image. Since the phase corresponds to the projection angle in the projection device, if the positional relationship between the projection device and the imaging device is known, the three-dimensional shape of the object can be obtained from the principle of triangulation.

  In the phase shift method, the contrast of the projected fringe pattern differs according to the light reflectance of the target object. When the reflectance of the target object is low, the captured stripe pattern is buried in noise. On the other hand, when the reflectance is high, a part of the captured stripe pattern is saturated. That is, in any case, the phase of each pixel cannot be obtained accurately, and the three-dimensional shape of the object cannot be obtained accurately. As a technique for accurately obtaining a three-dimensional shape of an object by a phase shift method, there is a technique disclosed in Patent Document 1. In the technique of Patent Document 1, a projection pattern is projected with a plurality of light amounts, and a pattern with a high contrast of a stripe pattern is selected for each pixel from a plurality of captured images.

JP 2005-214653 A

  However, since the technique of Patent Document 1 projects and images a fringe pattern on a target object with a plurality of light quantities, there is a problem in that the amount of light exposure to the target object increases and damage to the target object increases. .

  Therefore, an object of the present invention is to provide a process for obtaining a three-dimensional shape of a target object with high accuracy without increasing the amount of exposure of light to the target object as compared with the prior art.

  In order to solve the above problems, an image processing apparatus according to the present invention uses a height distribution of an object generated based on image data obtained by imaging an object on which a plurality of different patterns are projected. Based on the reflectance data, the first obtaining means for obtaining the representing height data, the second obtaining means for obtaining the reflectance data representing the reflectance distribution of the object corresponding to the height distribution, and the reflectance data. The high frequency is set so that the cutoff frequency of the smoothing filter used for the second region having a higher reflectance than the first region is higher than the cutoff frequency of the smoothing filter used for the first region in the rate distribution. And processing means for performing a smoothing process on the data with a smoothing filter.

  The present invention can obtain the three-dimensional shape of the target object with high accuracy without increasing the amount of exposure of light to the target object as compared with the prior art.

Block diagram showing device configuration of 3D shape measurement system Block diagram showing a functional configuration of a program executed by the image processing apparatus 105 A flowchart showing the flow of processing of the measurement application 209 The figure explaining the pattern image which the projection apparatus 103 projects The block diagram which shows the function structure of the shape calculation part 208 The flowchart which shows the flow of the process (S302) which calculates a three-dimensional shape The flowchart which shows the flow of the process (S605) which reduces noise. Diagram explaining the characteristics of noise The block diagram which shows the function structure of the shape calculation part 208

[Example 1]
<Device configuration of 3D shape system>
FIG. 1 is a diagram illustrating a device configuration of a three-dimensional shape measurement system. Reference numeral 101 denotes a measurement object to be measured, and reference numeral 102 denotes a sample stage for fixing the measurement object 101. A projection apparatus 103 projects a two-dimensional pattern image necessary for three-dimensional shape measurement onto the measurement object 101. As the projector 103, a single-plate monochrome DLP projector using 640 × 480 pixels and an LED light source is used, but the present invention is not limited to this. Any object can be used as long as it can project a two-dimensional pattern image on the object 101 to be measured. Reference numeral 104 denotes an imaging device that captures an image of the measured object 101 onto which the pattern image is projected by the projection device 103. As the imaging device 104, a digital single lens reflex camera (DSLR) having a CMOS area sensor with 8688 × 5792 pixels combined with a macro lens with a focal length of 100 mm is used. Further, it is assumed that the imaging device 104 has a photoelectric conversion characteristic that obtains a linear signal value with respect to the luminance on the measured object 101. Also, it is assumed that the image data to be recorded has color information of RGB 3 channels in each pixel, and each channel is quantized with 16 bits. Note that the imaging device 104 is not limited to the above example such as photoelectric conversion characteristics, the number of channels, and the number of quantization bits. An image processing apparatus 105 performs processing for calculating a three-dimensional shape of the object 101 to be measured from image data obtained by controlling the projection apparatus 103 and the imaging apparatus 104. The image processing apparatus 105 in this embodiment will be described as a PC (Personal Computer) including a main storage device such as a CPU and a RAM (Random Access Memory), and an auxiliary storage device such as an HD (Hard Disk) and a flash memory. However, the image processing apparatus 105 is not limited to a PC, and may be a microcomputer or the like.

  The projection device 103 and the imaging device 104 are connected to the image processing device 105 via an interface (not shown) such as a USB (Universal Serial Bus). As shown in FIG. 1, in this embodiment, the imaging device 104 is disposed above the object to be measured 101 and the projection device 103 is disposed obliquely above.

<Functional Configuration of Program Executed in Image Processing Device 105>
FIG. 2 is a block diagram illustrating a functional configuration of a program executed in the image processing apparatus 105. Reference numeral 201 denotes an input device that receives an operation from the user, and is a device such as a keyboard or a mouse. Reference numeral 202 denotes a display device that presents to the user the contents, measurement conditions, and measurement results input by the user, and is a device such as a liquid crystal monitor. The input device 201 and the display device 202 are not shown in FIG.

  The imaging driver 203 is a group of instructions for controlling the imaging device 104. An instruction to change the imaging conditions of the imaging device 104, an instruction to instruct the execution of imaging, and image data obtained by imaging are imaged from the imaging device 104. Instructions for transmission to the processing device 105 are included. The projection driver 204 is a group of commands for controlling the projection device 103, and includes a command for transmitting pattern image data from the image processing device 105 to the projection device 103, and a command for starting and ending projection. Yes. Similarly, the input driver 205 is a command group for controlling the input device 201, and the display driver 206 is a command group for controlling the display device 202.

  The acquisition control unit 207 is a command group that sends a command to the projection driver 204 and the imaging driver 203 and performs a series of processes for acquiring captured image data necessary to calculate the three-dimensional shape of the measured object 101. The shape calculation unit 208 is a command group that calculates the three-dimensional shape of the measured object 101 from the captured image data acquired by the acquisition control unit 207. The UI management unit 209 is a command group having a user interface (UI) function for performing processing such as management of information input by the user to the input device 201 and display of measurement results on the display device 202. The measurement application 210 is an instruction group for linking the instruction groups 207 to 209 to function as one measurement application.

<About the processing flow of the measurement application 210>
FIG. 3 is a flowchart for explaining the processing flow of the measurement application 210. First, in step S301, the measurement application 210 executes a series of processes for projecting and capturing a pattern image on the object to be measured 101 using the command group of the acquisition control unit 207. When the process of step S301 ends, the command group of the shape calculation unit 208 is executed in step S302, and the three-dimensional shape information of the measured object 101 is calculated from the image data obtained by imaging. The three-dimensional shape information obtained in the present embodiment is a height image (height data) in which the height information of each pixel of the area sensor of the imaging device 104 is recorded as a 32-bit floating point, but is not limited thereto. is not. The height represented by the height information is the height of the measured object 101 from the surface of the sample table. Finally, in step S303, the three-dimensional shape information obtained in step S302 is displayed on the display device 202, and the process ends. In this process, the height recorded in each pixel of the height data, which is three-dimensional shape information, is displayed in shades, and a two-dimensional distribution (height distribution) is displayed. Note that the processing in step S303 is not limited to the above processing, and it may be configured to record directly on an HD, a flash memory, or the like (not shown) without displaying on the display device 202.

<Regarding Pattern Image Projected by Projector 103>
FIG. 4 is a diagram for explaining a pattern image projected by the projection device 103. The pattern image shown in FIG. 4 is assumed to be an 8-bit grayscale image having pixel values of 0 (black) to 255 (white) of 640 × 480 pixels corresponding to the pixels of the projection device 103 on a one-to-one basis. To do. The number of pixels, the bit depth, the number of colors, etc. are not limited to this.

  4A to 4C are pattern images representing stripe patterns. In this embodiment, these pattern images are projected onto the measured object 101, the phase is calculated from the captured image, and the three-dimensional shape is calculated. This fringe pattern is a sine wave pattern, and the pixel value can be determined by the following equations (1) to (3).

Here, I ₁ (j, i) indicates the pixel value at position (j, i) in FIG. Similarly, I ₂ (j, i) represents the pixel value in FIG. 4B, and I ₃ (j, i) represents the pixel value in FIG. 4C. Note that j is a subscript indicating the position of the image in the horizontal direction, and i is the vertical position of the image. N _cycle is a parameter indicating the number of pixels in the horizontal direction per _cycle of the sine wave. FIG. 4D is a diagram illustrating changes in pixel values with respect to j of I ₁ , I ₂ , and I ₃ . As described above, the stripe patterns shown in FIGS. 4A to 4C are patterns in which the phase is shifted by 1/3 period. FIG. 4 (e) is a diagram showing a change in phase with respect to j in FIG. 4 (d). Note that the phase Φ is calculated from I ₁ , I ₂ , and I _{3 according} to the equation (4).

  However, (phi) (j, i) shall take the value of 0-2 (pi). In the phase shift method, a three-dimensional shape is estimated from a change in phase when these fringe patterns are projected onto the object 101 to be measured. In this embodiment, three striped patterns whose phases are shifted by 1/3 period are used. However, the present invention is not limited to this. For example, it may be six striped patterns whose phases are shifted by 1/6 period. Further, the waveform of the stripe pattern is not limited to a sine wave shape, and may be any waveform as long as the phase can be estimated, such as a triangular wave.

<About the process of calculating the three-dimensional shape (S302)>
FIG. 5 is a block diagram illustrating a functional configuration of the shape calculation unit 208 that executes Step S302. The reading unit 501 reads captured image data captured by the imaging device 104 and recorded on a recording medium such as an HD. The first generation unit 502 calculates the phase of each pixel from the three pieces of image data corresponding to the pattern images of FIGS. 4A to 4C according to the equation (4), and generates phase image data. At this time, the phase is calculated using only the G channel of the image data. In addition, you may calculate from the brightness | luminance information which added each channel of R, G, B with the predetermined weight. The phase image data is recorded on a recording medium such as a RAM (not shown) as image data in which 32-bit floating-point pixel values each having a value of 0 to 2π are recorded for each pixel channel. However, it is not limited to this.

  The second generation unit 503 estimates the reflectance distribution of the measured object 101 and generates reflectance data representing the reflectance distribution of the measured object 101. This reflectance data is obtained by calculating the amplitude value corresponding to each pixel of the phase image data as the reflectance. The reflectance (amplitude value) A of each pixel can be calculated by Expression (5).

  The reflectance data is recorded on a recording medium such as a RAM (not shown) as image data in which a 32-bit floating point reflectance (amplitude value) is recorded.

Similar to the diagram shown in FIG. 4E, the phase image represented by the phase image data repeats according to the period of the fringe pattern onto which the value of 0 to 2π is projected, and therefore the phase value is not between 0 and 2π. It will be continuous. The connecting unit 504 connects the phases by adding 2πN to the phase value of the Nth period so that this discontinuity does not occur. The number N indicating the period can be specified by performing a process of adding 1 to N if a discontinuous point appears when the phase image is scanned in the width direction. Alternatively, the period may be specified by projecting and capturing a pattern image such as a gray code for specifying the period. Hereinafter, a phase coupled phase value of the phase image data and [Phi _C.

Third generation unit 505 calculates the height H from the phase value [Phi _C representing the phase image data is phase connected, and generates height data representing the height H. The height of each pixel can be calculated by equation (6).

H (j, i) = α (Φ _C (j, i) −Φ _R (j, i)) (6)
Here, [Phi _R in the case of measuring the sample stage 102 as a reference, a phase value representing the phase image data after phase unwrapping. The reference phase image data is stored in advance in HD or the like. The [Phi _R can be acquired by performing measurement and processing up phase connection by connection unit 504 only to the sample stage 102 except for the object 101 to be measured. Α is a coefficient, which is a value determined in advance according to the length of the phase period and the angle between the optical axis of the projection apparatus 103 and the normal line of the sample stage 102.

  The noise reduction unit 506 performs noise reduction processing on the height data generated by the third generation unit 505 based on the reflectance data generated by the second generation unit 503. Details of the processing will be described later.

  FIG. 6 is a flowchart for explaining the flow of the process (S302) for calculating the three-dimensional shape. First, in step S601, the first generation unit 502 generates phase image data. Next, in step S602, the second generation unit 503 generates reflectance data. In step S603, the connection unit 504 performs phase connection, and in step S604, the third generation unit 505 generates height data. Finally, in step S605, the noise reduction unit 506 performs noise reduction processing on the height data.

<Noise Reduction Processing (S605)>
FIG. 8 is a diagram in which the standard deviation of the phase value of the sine wave signal represented by I ₁ , I ₂ , and I ₃ in Equation (4) is plotted with the reflectance (amplitude value) on the horizontal axis. Note that noise having a normal distribution characteristic with a standard deviation of 0.4 is added to the sine wave signal. From this figure, it can be seen that when the phase value is calculated by equation (4), the standard deviation of the phase value suddenly increases and the phase value varies when the amplitude value of the sine wave is low. In general, when the reflectance of the object to be measured 101 is low, the projected sine wave pattern is reflected by the object to be measured 101 and the amount of light entering the imaging device 104 is reduced. For this reason, the amplitude value of the imaged sine wave is reduced, and when noise from the imaging device 104 is further added, the phase value varies, and the height calculated by the equation (6) varies accordingly. In particular, when the reflectance differs greatly for each region of the object to be measured 101, the noise is small where the reflectance is high and bright, and the noise is large where the reflectance is low and dark. If the height data in which the noise increases in the dark part is reproduced by a 3D printer or the like, an unnatural reproduction with a sense of incongruity will occur. Therefore, in this embodiment, noise reduction is performed so that the noise becomes uniform at any reflectance. In the present embodiment, noise is reduced by performing filter processing (smoothing processing) using a smoothing filter on the height data.

  FIG. 7 is a flowchart illustrating the flow of the noise reduction process (S605) performed by the noise reduction unit 506. First, in step S701, a filter coefficient of a smoothing filter used for noise reduction processing is determined. For each pixel of the height data generated in step S604, a filter coefficient in a peripheral pixel of the target pixel is determined. Specifically, when the position of the peripheral pixel of the pixel of interest (j, i) is (x, y), the coefficient C is determined by Expression (7).

if (│j-x│ <N / 2) ∩ (│i-y│ <N / 2)
C (x, y) = 1
otherwise
C (x, y) = 1
... Formula (7)
However, N is a filter size, and is determined by Expression (8).

N = αA (j, i) ^−β (8)
The filter size N is determined to be inversely proportional to the β power of the reflectance A represented by the reflectance data. Α and β are preset constants and take values of α ≧ 0 and β ≧ 1.

  In step S702, based on the filter coefficient calculated in step S701, the height data representing the height H is subjected to smoothing processing according to equation (9).

H ′ (j, i) = Σx _{, y} C (x, y) H (j + x, i + y) / Σ _{x, y} C (x, y)
... Formula (9)
Here, H ′ is the height represented by the height data after the smoothing process.

  In step S703, it is confirmed whether all the pixels have been processed as the target pixel. If not, the variables j and i indicating the pixel positions are updated, and the process returns to step S701. If all pixels have been processed, the height data after the smoothing process is stored in a memory such as RAM or HD, and the process is terminated.

  In equation (8), the value of β can be determined by fitting the horizontal axis of the characteristic shown in FIG. 8 as the value of reflectance A. In general, assuming that the population follows a normal distribution, the estimation error of the sample mean is 1 / √M when the number of samples is M. Smoothing by the methods shown in Equations (7) and (9) is almost equivalent to estimating the average using N × N pixels as samples, and the estimation accuracy of the pixels after smoothing, that is, the noise amount is 1 / N. Therefore, as shown in Expression (8), if N is determined in proportion to the amount of noise in the height data according to the reflectance, height data with a uniform amount of noise can be obtained at any reflectance. it can.

  As described above, in this embodiment, noise reduction is adaptively performed according to the reflectance distribution of the measured object 101. This makes it possible to obtain height data (three-dimensional shape information) with a uniform amount of noise at any reflectivity without having to change the amount of light of the projection pattern and perform imaging multiple times. In addition, since noise is adaptively reduced according to the reflectance, that is, the amount of light reflected by the object 101 to be measured, the reflectance of the object 101 to be measured even when a projection apparatus having a relatively low light intensity is used as the projection apparatus 103. It is possible to obtain three-dimensional shape information with uniform noise regardless of the distribution. That is, it is possible to reduce damage caused by exposure of light to the measured object 101. In addition, since three-dimensional shape information with uniform noise can be obtained even if the exposure amount of the imaging device 104 is small, a cheaper device can be used as the imaging device, and at the same time, the shutter speed can be set short, so that the measurement time Can be shortened.

[Modification]
<Modification 1>
In the first embodiment, the image processing apparatus 105 acquires the image data by controlling the projection apparatus 103 and the imaging apparatus 104, and generates the reflectance data and the height data from the acquired image data. It is not limited to. Reflectance data and height data are generated in advance from the same processing as in the first embodiment using the projection device 103 and the imaging device 104, and stored in a memory such as a RAM or an HD. The image processing apparatus 105 may only obtain the stored reflectance data and height data from the memory and perform the noise reduction processing of the first embodiment. FIG. 9 shows a functional configuration of the shape calculation unit 208 in the image processing apparatus 105 in this case. With this configuration, three-dimensional shape information with uniform noise can be obtained without generating reflectance data and height data.

<Modification 2>
The smoothing filter in Example 1 is a known moving average filter, and the filter coefficient was determined by Equation (7). However, the smoothing filter is not limited to the moving average filter, and a known Gaussian filter may be used. In this case, the filter coefficient is determined using the following equation (10).

  Here, σ = N / 4. Since the coefficient is determined to be proportional to the Gaussian distribution function in this way, the smoothing weight can be set according to the distance between the pixel of interest (j, i) and the surrounding pixel (x, y) in the smoothing filter. More suitable smoothing processing can be performed.

<Modification 3>
In the first embodiment, N is determined by Expression (8) on the assumption that the filter size N is inversely proportional to the reflectance A, but is not limited to the above example. The filter size N may be determined using Equation (11) assuming that the characteristic of FIG. 8 is proportional to the logarithm of the horizontal axis (reflectance).

N = α−β · log (A (j, i)) (11)
At this time, the constants α and β have values of α ≧ 0 and β ≧ 0.

Equation (10) and Fourier transform, if the frequency at which the amplitude of the cut-off frequency omega _C becomes 1 / √2, of formula (12) below is obtained.

  A similar result can be obtained by calculating the cut-off frequency by Fourier transform of Equation (7). From equation (12), it can be seen that the filter size decreases as the cutoff frequency of the smoothing filter increases. The smoothing filter used in the first embodiment is designed so that the filter size becomes small in a region where the reflectance is large, as in Expression (8) or Expression (11). For this reason, the smoothing filter used in Example 1 should just be designed so that a cutoff frequency may become high in the area | region where a reflectance is large (the amount of light is large).

<Modification 4>
In the first embodiment, an example in which one imaging device 104 is used and three-dimensional shape measurement is performed by the phase shift method has been described. However, the present invention is not limited to this, and a three-dimensional shape measurement method that calculates a phase image is used. Any method may be used. For example, a so-called stereo phase shift method may be used in which a plurality of imaging devices are used, phase images calculated therefrom are made to correspond by the stereo matching method, and a three-dimensional shape is acquired by the principle of triangulation.

<Modification 5>
The second generation unit 503 generates the reflectance data by calculating the amplitude value corresponding to the phase, but is not limited thereto. For example, brightness information is calculated from image data obtained by projecting an entire white projection pattern from the projection device 103 and the projected measurement object 101 is imaged by the imaging device 104, and used as reflectance data. May be. The data generated by the second generation unit is not limited to the reflectance data representing the reflectance distribution. If the reflectance distribution of the light of the measured object 101 is known, the distribution of the amount of light reflected by the measured object 101 is determined. It may be light amount data representing (light amount distribution) or luminance data representing a luminance distribution.

<Modification 6>
In the first embodiment, the height data after the noise reduction process is displayed on the display device 202, or directly recorded on an HD or a flash memory. However, it is not limited to the above example. For example, even if a printer such as an ink jet printer or a 3D printer generates print data for printing an image or a three-dimensional shape representing the measured object 101 based on the height data after the noise reduction processing, and outputs the print data to the printer Good. In this case, the shape calculation unit 208 includes a print data generation unit and a print data output unit. Alternatively, the image processing apparatus 105 may include a printing unit that performs printing based on the print data. The print data is, for example, color material amount data representing the color material amount, binary data representing whether or not to record the color material, STL data representing a three-dimensional shape, and the like.

[Other Examples]
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

105 Image Processing Device 901 First Acquisition Unit 902 Second Acquisition Unit 506 Noise Reduction Unit

Claims (18)

First acquisition means for acquiring height data representing a height distribution of the object, generated based on image data obtained by imaging an object on which a plurality of different patterns are projected;
Second acquisition means for acquiring reflectance data representing the reflectance distribution of the object corresponding to the height distribution;
Based on the reflectance data, the cutoff frequency of the smoothing filter used for the second region having a higher reflectance than the first region than the cutoff frequency of the smoothing filter used for the first region in the reflectance distribution. Processing means for performing a smoothing process using a smoothing filter on the height data so that the height data is higher;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the height data is data generated by a phase shift method.   The image processing apparatus according to claim 1, wherein the smoothing filter is a Gaussian filter.   The image processing apparatus according to claim 1, wherein the smoothing filter is a moving average filter.   The image processing apparatus according to claim 1, wherein the reflectance data is data generated based on the image data.   The image data was obtained by imaging the object on which a pattern image representing a pattern in which luminance changes sinusoidally and a pattern image representing a pattern in which the phase of the sine wave is shifted from the pattern image are projected. The image processing apparatus according to claim 1, wherein the image processing apparatus is a plurality of image data.   Based on the reflectance data, the processing means performs smoothing used for a second region having a reflectance higher than that of the first region than a filter size of a smoothing filter used for the first region in the reflectance distribution. The image processing according to any one of claims 1 to 6, wherein a smoothing process using a smoothing filter is performed on the height data so that a filter size of the filter becomes smaller. apparatus.   The image processing apparatus according to claim 7, wherein a filter size of the smoothing filter is proportional to a logarithm of reflectance in a reflectance distribution represented by the reflectance data.   The image processing apparatus according to claim 7, wherein a filter size of the smoothing filter is inversely proportional to a reflectance in a reflectance distribution represented by the reflectance data. First control means for controlling the projection device to project a plurality of different patterns on the object;
Second control means for controlling the imaging apparatus so as to image the object on which a plurality of different patterns are projected by the projection apparatus controlled by the first control means;
Height data generating means for generating the height data based on a plurality of the image data obtained by imaging of the imaging means controlled by the second control means,
The image processing apparatus according to any one of claims 1 to 9, wherein the first acquisition unit acquires the height data generated by the height data generation unit. Projecting means for projecting a plurality of different patterns onto the object;
Imaging means for imaging the object on which a plurality of different patterns are projected by the projection means;
Height data generating means for generating the height data based on a plurality of the image data obtained by imaging by the imaging means,
The image processing apparatus according to any one of claims 1 to 9, wherein the first acquisition unit acquires the height data generated by the height data generation unit. Print data generation means for generating print data for a printer to print a three-dimensional shape representing the object based on the height data smoothed by the processing means;
Print data output means for outputting the print data to the printer;
The image processing apparatus according to claim 1, further comprising: Print data generation means for generating print data for a printer to print a three-dimensional shape representing the object, based on the height data smoothed by the processing means;
Printing means for printing a three-dimensional shape representing the object based on the print data;
The image processing apparatus according to claim 1, further comprising:   14. The image processing apparatus according to claim 1, further comprising height data output means for outputting the height data smoothed by the processing means to a display device. .   The image processing apparatus according to claim 1, further comprising display means for displaying the height data smoothed by the processing means. First acquisition means for acquiring height data representing a height distribution of the object, generated based on image data obtained by imaging an object on which a plurality of different patterns are projected;
Second acquisition means for acquiring light amount data representing a light amount distribution of light reflected from the object corresponding to the height distribution;
Based on the light amount data, the cutoff frequency of the smoothing filter used for the second region having a larger amount of light than the first region is greater than the cutoff frequency of the smoothing filter used for the first region in the light amount distribution. Processing means for performing a smoothing process by a smoothing filter on the height data so as to be higher;
An image processing apparatus comprising: A first acquisition step of acquiring height data representing a height distribution of the object generated based on image data obtained by imaging an object on which a plurality of different patterns are projected;
A second acquisition step of acquiring reflectance data representing the reflectance distribution of the object corresponding to the height distribution;
Based on the reflectance data, the cutoff frequency of the smoothing filter used for the second region having a higher reflectance than the first region than the cutoff frequency of the smoothing filter used for the first region in the reflectance distribution. A processing step of performing a smoothing process by a smoothing filter on the height data so that the height is higher;
An image processing method comprising:   A program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 16.

Images