JP5430643B2 - Method for creating a three-dimensional profile map for image composition - Google Patents

Description

  The present invention relates to a method for creating a three-dimensional profile map. More specifically, a camera device such as an optical microscope is used to obtain a large number of images corresponding to a focal length with respect to one three-dimensional sample. The present invention relates to a method for creating a three-dimensional profile map that displays sample height information so that a single two-dimensional composite image in focus (in focus) can be obtained by combining images.

  In general, in order to obtain a three-dimensional profile (height information) of a sample, sample images having different focal lengths are obtained, brightness contrast (contrast) is obtained for each region from each sample image, and each region is obtained. The image map is created by combining the focus information (that is, the height information) of the image having the largest contrast between the light and dark. Such a three-dimensional profile is used to obtain one vivid composite image from a large number of sample images having different focal lengths. At this time, due to image noise and blur (blur), a portion showing a large contrast between light and dark does not necessarily mean a boundary line inside the image, so various attempts have been made to complement this. For example, in one image formed by combining images having the maximum contrast, a window for determining consistency is set for each region, and the number of the input image indicating the most frequent contrast frequency in the window The mode selection method for selecting a representative value as the representative value, the median selection method for selecting the median value for the removal of black-white noise mainly generated instead of the mode number, etc. are used alone or in combination with other methods. ing. However, the biggest problem with the above method is that there is no boundary line for an image that is not in focus at all, so that correct information cannot be obtained. In fact, most existing 3D profilers process out-of-focus areas using a simple low-pass filter method, so if the out-of-focus areas are widely distributed, the atypical height map will swing up and down. Is formed. Nevertheless, since it is difficult to focus on the entire input image, an appropriate processing method for a non-focused area is necessary.

  On the other hand, various methods for determining the degree of focus matching of images have been developed. Since it is necessary to obtain height information for each point in the image, the degree of focus matching is obtained in units of pixels or specific regions. There are various ways to wrap areas, such as wrapping in areas that have similar characteristics, wrapping in the same shape window, and the resolution of the input image even if the windows are the same shape. Depending on the window size, the window size may be set differently. Methods for obtaining the focus matching degree include mask-based methods such as a Sobel filter and LoG, a Fourier transform method using a window, a wavelet method, and the like, and various methods are used within each large classification. It is done. Note that even if an image map composed only of the number (height) of the focused image is acquired, correct three-dimensional information is not always obtained. This is because it is difficult to distinguish between a case where the gradient increases sharply and a case where it gradually increases even if the discretized height information is corrected in the three-dimensional modeling process.

  An object of the present invention is to provide a method for creating a three-dimensional profile map that grasps accurate position information of a focused area in a sample image and correctly reflects a height change according to the position.

  Another object of the present invention is to provide a three-dimensional profile in which a non-focused area that does not have an in-focus image portion does not hinder information grasping of a region of interest and has a continuity in height from the surrounding focused sections. It is to provide a method for creating a map.

  Still another object of the present invention is to provide a method of creating a three-dimensional profile map that can accurately reflect the degree of focus matching of an image.

In order to achieve the above object, the present invention includes a step of imaging a sample from different heights to obtain a number of two-dimensional sample images having different focal sites, and a discrete wavelet transform for the number of sample images. In the step of performing (Discrete Wavelet Transform: DWT) and the detailed sub-band obtained by performing the discrete wavelet transform (DWT) on each sample image, the sub-band of each pixel (sub-band) ) Compare the coefficient values and create an initial height map for each pixel at the imaging height of the image showing the maximum coefficient value, and discrete wavelet transform (DWT) was performed on each input image Calculate the degree of focus (degree of focus matching) for the approximate sub-band region of the image Filtering the focus matching degree so that the height information of the focused pixel (boundary point) remains, and the height information of the non-focused pixel (nonboundary point) is removed; Calculating a height of a pixel (non-boundary point) removed without passing through the filter by interpolating from a height value of a pixel (boundary point) passing through the filter, and the height information is removed And a step of creating a height map by substituting the height calculated by the interpolation with respect to a pixel.

According to the method for creating a three-dimensional profile map according to the present invention, using a large number of input images input together with different height information, the height information of the portion of the sample is obtained from the entire region of the focal point, It is possible to provide a highly consistent height map without the problem of an inaccurate height profile associated with inputting a discontinuous height with respect to a non-focused portion. Using the three-dimensional profile map produced in accordance with the present invention, it is possible to accurately measure and observe the sample, and an image of the sample in which the focal portion varies depending on the height is combined with a single composite image. Can be synthesized as

1 is a configuration block diagram of an optical microscope capable of realizing a three-dimensional profile map creation method according to the present invention. 6 is a flowchart for explaining a method of creating a three-dimensional profile map according to an embodiment of the present invention. The figure for demonstrating the result of having performed discrete wavelet transform (DWT) with respect to the two-dimensional image of a sample. The figure which shows an example of the result of having performed discrete wavelet transform (DWT) with respect to the two-dimensional image of a sample. The figure which shows an example of the map (initial height map) which shows the initial height of the image produced in the pixel unit. The figure which shows an example of the height map primary-corrected so that the height of a pixel may correspond with the height of a surrounding pixel. The figure which shows an example of the height distribution of an actual sample in case height information is input discontinuously. The figure which shows an example of the height distribution calculated from the focus matching degree, when height information is discontinuously input. The figure for demonstrating the process of calculating a focus matching degree with respect to the approximate sub-band area | region of the image in which the discrete wavelet transform (DWT) was performed, and extracting a boundary point. The figure which shows a height map in case the height value of the pixel of a non-focal point is removed enough with respect to the approximate sub-band area | region of the image in which the discrete wavelet transform (DWT) was performed. The figure which shows a height map in case the height value of a non-focus point pixel is fully removed with respect to the approximate sub-band area | region of the image in which the discrete wavelet transform (DWT) was performed. In samples with continuously changing heights, the remaining points (1, 2, 3, 4) after filtering the degree of focus matching and the points to be erased (1 ', 2', 3 ', 4') The figure which shows the height distribution value obtained after application of the height interpolation method. In a sample whose height changes discontinuously in a stepped manner, a point (1, 2, 3, 4) remaining after filtering the focus matching degree and a point (1 ′, 2 ′, 3 ′) to be erased 4 ') and a diagram showing height distribution values obtained after application of the height interpolation method. The figure which shows the process in which the boundary point used for interpolation is extracted in order to calculate the height of the point erased because it was out of focus. The figure which shows an example of the three-dimensional profile (height) map completed according to this invention.

  Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a configuration block diagram of an optical microscope capable of realizing a method for creating a three-dimensional profile map according to the present invention. As shown in the figure, the optical microscope that can be employed in the present invention includes illumination for irradiating a sample with illumination light, an optical barrel for enlarging the image of the sample, and a camera for imaging the sample image. A microscope unit 10 for obtaining the obtained image, a microscope operation control unit 12 for adjusting the height of the microscope unit 10, that is, a focal length, and an enlarged image of the sample from the microscope unit 10 and receiving the digital image signal. An image input unit 22 to be converted, a height input unit 24 to which a height at which a sample image is obtained from the microscope operation control unit 12 (a height of the microscope) is input, and a digital image signal from the image input unit 22 The height information from which the sample image was obtained is input from the height input unit 24, the focus matching degree of the input image is evaluated, and a three-dimensional profile map (height map) of the sample is generated. It includes an image processing unit 26, a. The image processing unit 26 can generate an image with an extended depth, which is an in-focus image, using the three-dimensional profile map.

  FIG. 2 is a flowchart for explaining a method (algorithm) for creating a three-dimensional profile map according to an embodiment of the present invention. As shown in FIG. 1 and FIG. 2, in the method of creating a three-dimensional profile map according to the present invention, a sample is imaged from different heights to obtain a number of two-dimensional sample images having different focal sites (S10). A step of performing discrete wavelet transform (DWT) on the sample image (S20), and a detailed sub-band obtained by performing discrete wavelet transform (DWT) on each sample image In step S22, the detailed sub-band coefficient value of each pixel is compared to create an initial height map for each pixel at the imaging height of the image indicating the maximum coefficient value. A step of calculating a focus matching degree for the approximate sub-band region of the image subjected to the wavelet transform (DWT) (S28). A step of filtering the focus matching degree so that the height information of the in-focus pixel remains, and the height information of the non-focus pixel is removed (S30); Calculating the height of the pixel obtained by interpolating from the height value of the pixel that has passed through the filter (S32), and the height calculated by the interpolation for the pixel from which the height information has been removed. Substituting, and creating a height map (S34). Here, between the step (S22) and step (S28), the detailed sub-band wavelet coefficient of the pixel calculated as the same height is added in the verification window centered on each pixel, and the height is calculated. The sum of the corresponding wavelet coefficients is compared and the height giving the largest value is corrected with the actual height of the pixel (S24) and / or the median filter is applied to the (corrected) height map. It is preferable to further include a step (S26) of creating a height map from which impulse noise is removed. Here, the step S10 for obtaining the sample image is performed by the microscope unit 10, and the obtained sample image is converted into a digital image signal by the image input unit 22, and then the discrete wavelet transform (DWT) is performed by the image processing unit 26. ) Etc. may be performed. If necessary, the (color) digital image signal is converted into a black-and-white digital image signal by a normal method in the image processing unit 26 (S12), and the remaining processes such as discrete wavelet transform (DWT) are performed. It may be done. Such black-and-white image conversion can reduce the number of future image signal data and reduce the calculation burden on the image processing unit 26.

  Hereinafter, a method for creating a three-dimensional profile map according to the present invention will be described more specifically. A method for creating a three-dimensional profile map according to the present invention includes (a) a step of obtaining height information for the entire sample image (creation of an initial height map), and (b) a boundary line (boundary point) of the sample. 2) extracting information on the area corresponding to), and calculating the height of the area other than the boundary line by interpolation using the height information of the boundary line area obtained in step (a). Broadly divided into steps. The purpose of the step (a) is to obtain accurate height information regarding the in-focus area. The present invention obtains more accurate height information in the step (a), judges the consistency of the height information in the step (b), and corrects the height information of the pixel having low consistency with good consistency. To process the out-of-focus area.

  In the step (a), in order to obtain the height information of the image, first, the sample is imaged from different heights (distances), and a large number (for example, 2 to 8 pieces, preferably 2) having different focal sites. ~ 5) two-dimensional images are obtained (S10), and the position information and height information of the in-focus portion are combined from the two-dimensional image to form a height map (for example, the height of the position on the xy plane). A two-dimensional map that is data in a form in which values (input image numbers) are displayed. In order to make it easy to distinguish, a height may be displayed for each color. For this purpose, for each pixel of the obtained two-dimensional image, (i) evaluation of the focus matching degree, (ii) calculation of height information of each pixel from the focus matching degree (focus information), and (iii) 3) Perform three steps, such as creating a height map using the combined height information.

  First, (i) the method of evaluating the degree of focus matching for each part of the sample image is as follows. In general, the in-focus image has a well-defined boundary, and at the most in-focus position (the best in-focus), the light intensity and the contrast of light and dark are maximized. The focus matching degree can be evaluated by the method. In the present invention, a discrete wavelet transform (DWT) method, which is a normal signal or image analysis method, is employed as the boundary detection method. The discrete wavelet transform (DWT) method is a method of analyzing a signal using a localized wavelet. Signal analysis with various resolutions can be performed by multilayer structure analysis, which is useful for analysis of a local signal. In particular, by performing discrete wavelet transform (DWT) repeatedly (multiple discrete wavelet transform), the average value smoothing effect increases as it goes up, so that there is a merit that non-impulse noise such as the same type noise is removed. However, since the impulse noise still remains, the impulse map is removed by post-processing so that the height map is consistent.

  When the discrete wavelet transform (DWT) is performed on the two-dimensional image of the sample (S20), the image is divided into four sub-bands composed of LL, HL, LH, and HH, as shown at A in FIG. 3A. Here, LL is a reduced image of the original image with the resolution reduced to ¼, which is called an approximate sub-band, and the HL sub-band is a degree of change in image information in the vertical direction (for example, When the discrete wavelet transform (DWT) is performed once on the original image, the HL sub-band coefficient indicates the degree of change in the brightness of the image in the vertical direction, and the discrete wavelet transform (DWT) is performed twice or more. In this case, the HL sub-band coefficient represents the information change of the reduced image.), LH represents the degree of change of the image information in the horizontal direction, and HH represents the degree of change of the image information in the diagonal direction. For example, a large LH means that there is a large change in image information in the horizontal direction, and therefore there is a vertical boundary. The LH, HL, and HH are referred to as detailed sub-bands (see, for example, US Pat. No. 6,151,415). Such discrete wavelet transform (DWT) is performed many times, such as twice (B in FIG. 3A and FIG. 3B) and three times (C in FIG. 3A) for one original image, depending on the resolution of the original image. Can be performed over a wide range. When performing discrete wavelet transform (DWT) on an image that has already been subjected to discrete wavelet transform (DWT), by performing discrete wavelet transform (DWT) on the currently obtained LL sub-band, An image having a multi-layer structure such as C is obtained. As described above, the detailed sub-band wavelet coefficients obtained by performing the discrete wavelet transform (DWT) (for example, the wavelet coefficients for 16 pixels are shown in FIG. 3B) are the contrasts of the contrast of the image. In order to reflect the characteristics, among the detailed sub-band wavelet coefficients, the boundary detection becomes possible by combining the values having the maximum absolute value. Since the approximate sub-band value is a low-resolution image from which noise has been removed, image processing such as boundary detection can be performed using this image. That is, the present invention applies a discrete wavelet transform (DWT) method that allows local data access at various resolutions instead of the Fourier analysis conventionally used for image boundary detection and signal analysis. Evaluate (calculate) consistency. An example of a method for evaluating the degree of focus matching using such discrete wavelet transform (DWT) is also disclosed in Korean Patent Application No. 10-2010-0054861 (filing date: June 10, 2010) by the present applicant. The contents of said patent application are incorporated herein by reference. For example, in order to obtain an image having a resolution of about 100 × 100 from an image having an input resolution of 1600 × 1200, details obtained by performing discrete wavelet transform (DWT) and performing discrete wavelet transform (DWT) about three times. The maximum value of the pixel coefficient of the sub-band (LH, HL, HH) (that is, the maximum value among the LH, HL, and HH sub-band coefficient values) is determined as the focus matching degree of each pixel in the image. Similarly, the maximum value of the pixel coefficient is obtained for each input image. In the present invention, the discrete wavelet transform (DWT) is performed one or more times, and the number of transforms is 1 to 10 times, preferably depending on the resolution of the original image, the size of the pixel to be synthesized, etc. 2 to 5 times.

  Next, for each pixel, the image height (imaging or focal length) at which the coefficient value of the pixel is maximum is determined to be the focal point position (height) with respect to the pixel, and Specify the height. This is a process of calculating the height information of each pixel from the (ii) focus matching degree (focus information). That is, the height information of each pixel is obtained from the focus matching degree (focus information) for each pixel of the image. Next, in the step of (iii) creating a height map using the combined height information, an image height map is created in pixel units, and an example of the initial height map thus created is It is shown in FIG. FIG. 4 is an initial height map of a sample in which four metal plates and a circular metal structure are configured to have almost the same height, and the central part of the circular metal structure (the middle black color in FIG. 4). Part) is partially recessed (recessed).

  In the initial height map obtained in this way, as described above, the impulse-type noise is not removed, and the contrast value in units of pixels is sensitive to external environmental elements. It is preferable to confirm the consistency of the height obtained. In order to retain the boundary value information of wavelet coefficients while reducing the sensitivity of each pixel, it is effective to set a window around the pixel and compare the wavelet coefficients within this window (A wavelet− based image fusion tutoral, G. Pagesares and JM de la Cruz, Pattern Recognition 37 (2004) 1855-1872). Such a window-based composition method is used to determine the focus matching degree when creating a height map, or simply used to apply a filter when checking the consistency. However, when such a window-based method is used, a window operation is required for all pixels of all input images, resulting in a slow speed. For this reason, in the present invention, in a window (for example, a verification window having a 3 × 3 pixel size centered on the pixel) so that the calculation can be easily performed while maintaining the meaning (degree of focus matching) of the wavelet coefficient. Group wavelet coefficients according to input height. In general, not many samples have a specific height of only one or two pixels, so if there is a boundary in the window, it will almost always be in the form of a boundary line that is not a boundary point, and will have the same height. The sum of the wavelet coefficients of the boundary line with is greater than the sum of the wavelet coefficients with different heights. Even if any pixel in the window has a large wavelet coefficient in the form of impulse noise that differs greatly in height from the surrounding area, if there are boundaries in the window, the number of boundary points is generally greater than impulse noise. Since the wavelet coefficients at the boundary points have a large value, the sum of the wavelet coefficients at the boundary points at the same height is usually larger than the wavelet coefficients of the impulse noise at the same height. Therefore, in the present invention, the pixels calculated as the same height in the window are grouped, the wavelet coefficients of the grouped pixels are added, the sum of the wavelet coefficients according to the height is compared, and the most The height giving a large value is corrected by the actual height of the pixel (S24). Hereinafter, this method is referred to as a grouped coefficient summation verification (GCSV) method. Within the verification window, wavelet coefficients having the same height are grouped based on the input image number (height) of the pixel obtained in step (ii). That is, the same height wavelet coefficients are less likely to be in the same group. For example, when imaging a sample from three different heights (1, 2 and 3) and obtaining three two-dimensional images with different focal sites, the wavelet coefficients corresponding to height 1 are less likely to be the same group, Wavelet coefficients corresponding to height 2 are less likely to be the same group, and wavelet coefficients corresponding to height 3 are less likely to be the same group. Since the impulse noise is not completely removed only by the above-described method, the first corrected height map from which the impulse noise is removed is created by performing median filtering (median filtering) at least once. It is preferable (S26). In order to perform the median filtering, a window of an appropriate area (for example, 5 × 5 pixels) is set around the pixel to be filtered, and all the height information existing in the window is set in ascending order or Sort in descending order and correct (replace) the height information of the first selected pixel with its median. FIG. 5 shows an example of an image map that is first corrected so that the pixel height matches the height of the surrounding pixels, reflecting the accurate height with respect to the boundary region. FIG. 4 is a height map simply plotted using the degree of focus matching after the discrete wavelet transform (DWT), and FIG. 5 shows the grouped coefficient sum verification method for the height map of FIG. It is a height map obtained by performing (GCSV) and median filtering. FIG. 5 shows that the variation in height between adjacent pixels is reduced compared to FIG.

  As shown in FIG. 5, the primary corrected height map created in this way has a significant variation in height in the non-focused region, and height information is discontinuously input. There is a problem that the change in height cannot be displayed correctly. When height information is input discontinuously (discretized) and the actual sample has a continuous height distribution as shown in FIG. 6, the horizontal axis indicates the position of the sample, and the vertical axis indicates the sample. As shown in FIG. 6, a portion higher than the dotted line parallel to the horizontal axis in FIG. 6 indicates a height distribution having an H value, and a portion lower than the dotted line has an L value. In FIG. 6 and FIG. 7, the graph shows the profile of the actual sample, the number in the box is the height information of the sample input in digital form, for example, 0 is L, 1 is H Since the median may be set to H or L, the value 0.5 is used. Here, once the median is determined, it cannot be changed. For this reason, in this method, it cannot be distinguished whether the sample has a stepped gradient or a continuous gradient. However, if the positions of pixels with discontinuous gradients are compared, it is possible to distinguish between a continuous height change and a discontinuous height change. In FIG. 7, the position of the discontinuous gradient is one point where the staircase starts, or when the staircase gradient is slightly gentle, there are two points: the point where the staircase starts to rise and the point where the staircase ends. . Even in the case of a sample whose height continuously changes, there are two points: a point where the height starts to rise and a point where the height does not rise any further. Comparing the two cases, the gradient is steep when the point at which the discontinuous gradient appears is close, and the gradient is gentle otherwise. For this reason, the discontinuity of the gradient is an important basis for judging the gradient.

  The discontinuity of the height gradient, which is an important basis for determining the height distribution, can be grasped by the degree of focus matching. For this purpose, discrete wavelet transform (DWT) is performed on each input image, preferably discrete wavelet transform (DWT) is performed up to the highest level, and if necessary, non-impulse type noise is performed. The degree of focus matching is calculated for the approximate sub-band region of the image from which is removed (S28). FIG. 8 is a diagram showing three lights whose focal points are tied to the height L and a sample whose height changes continuously when measuring two types of heights L and H. FIG. Since the lights 1 and 2 that have reached the position L are accurately focused, the degree of focus matching is highly evaluated. On the other hand, the light that is not in focus (non-focused) 3 has a low degree of focus matching. Is done. In the approximate sub-band region, the focus matching degree of a specific pixel is expressed by the brightness (I (x, y)) of the pixel (coordinate: (x, y)) and the surrounding four points as shown in the following equation 1. It can be calculated as the sum of absolute values of differences from the brightness of (coordinates: ((x, y + 1), (x, y-1), (x + 1, y), (x-1, y))).

  At this time, if the degree of focus matching is filtered, for example, a threshold is set and a high-pass filter is applied (S30), the in-focus point (points 1 and 2) remains, and the non-in-focus point The point (3) disappears. For example, if the difference in intensity between two adjacent pixels is 40 or more, the boundary between the pixels appears clearly. However, the pixel indicating such a difference in brightness is at least 2 in the four directions, up, down, left, and right. The threshold condition is when there are two pixels, that is, when the pixel to be filtered has a meaning as a non-isolated line. As a result of the experiment, in this case, when the threshold is 80 to 120, only the shape of the outline remains. For this reason, the threshold value in this case can be set to 80, preferably 100. Therefore, in the present invention, the degree of focus matching is less than 3 times, preferably less than 2.5 times, more preferably less than the brightness difference where the boundary between adjacent pixels appears (that is, becomes clear). If it is less than twice, it is regarded as an out-of-focus pixel and the height information is removed. Here, “the difference in brightness at which the boundary between adjacent pixels appears” can experimentally determine the difference in brightness at which the outline of the image appears. As described above, when the focus matching degree is filtered, the remaining points are boundary points where the boundary is clear, and this includes a point where the gradient is discontinuous (position 2 in FIG. 8). . At this time, if the threshold is set too low (for example, 50), a non-focused portion remains (see FIG. 9), and if the threshold is set appropriately (for example, 100), the boundary point remains. Then, the points in the out-of-focus area can be deleted (see FIG. 10). 9 and 10, white portions in a circular sample image are recognized as boundary lines and remain without being erased, and black portions are portions recognized and erased as portions other than the boundary line. is there. As shown in FIG. 9, if there are many unerased points (outlined portions), the image is brightened as a whole, but the meaning as a boundary is reduced. In FIG. 10, points other than the boundary are sufficiently removed. Indicates the state. In erasing the pixel height information, it is preferable that only the pixel height information corresponding to the boundary remains and all the boundary lines of the image look vivid, even if some boundary information is lost. Preferably, the entire boundary information remains and any pixels other than the boundary line are erased. A non-limiting example of the number of pixels that are out of focus and whose height information is erased depends on the observed sample image, height change, etc. 20 to 50%, more specifically 30 to 40%. Here, although none of the boundary points that have passed through the threshold are points where the gradient is not continuous, as shown in FIG. 11 and FIG. The point at is the point where the gradient is not continuous. Specifically, in FIGS. 11 and 12, 1, 2, 3, and 4 are boundary points, and 1 ′, 2 ′, 3 ′, and 4 ′ are points eliminated by the high-pass filter.

In this way, the height of the point that has been erased without passing through the filter is set to the height value of the point that has passed through the filter (for example, the height value that has been subjected to the primary correction obtained in step S26). It is calculated by inserting (S32), and the height map is finally completed using this. Although the height discontinuity plays an important role in analyzing the height distribution, it does not need to be extracted separately when actually creating the height map. The boundary point uses the height value as it is, and the deleted point (non-boundary point) changes the gradient by interpolating the heights of the two closest boundary points in the vicinity to obtain the height. This is because sufficient information can be obtained. For example, in FIG. 11 and FIG. 12, a point that is not a boundary point but is at the same height as a point that remains as a boundary point, such as 1 ', is converted into a peripheral boundary point through interpolation of the peripheral boundary points. Have the same height. On the other hand, a point between the boundary points having different heights, such as 2 ', has an intermediate height by interpolating from the heights of the peripheral boundary points. At this time, the height of the point to be interpolated is to obtain a weighted average of the heights of the boundary points, and the weighting value is more greatly influenced by the near boundary value, so that the function of the reciprocal distance (1 / r) or It is given as a Gaussian function of distance. For example, when the Gaussian function and the weight value function are expanded by 1 / r (where r represents the distance between the point to be interpolated and the boundary point), 1 / r which is the lowest order is used as the weight value. or apply, or, in the case of a Gaussian function, using ^exp a (-r 2/1000) as a weighting value, it was possible to obtain good results.

On the other hand, unlike the one-dimensional samples shown in FIG. 6 and FIG. 7 and the like, in a two-dimensional actual sample, the boundary value may exist not only in the form of a closed curve but also in the form of an open curve, a collection of points, etc. Good. For this reason, in the present invention, it is preferable to perform interpolation from 3 to 7, for example, 5 boundary points located at a position closest to the deleted point whose height is to be calculated. The detection of boundary points used for interpolation uses a combination of concentric circle search centered on the point to be interpolated (erased point) and azimuth angle search. As shown in FIG. 13, a method in which concentric circle search and azimuth angle search are combined draws a concentric circle or a concentric rectangle with the point to be interpolated as the center, and searches from the center toward the outside. If found, search for more boundary points within the range of ± 30 °, preferably ± 15 ° (ie, the azimuth at which the boundary point was found) from the found boundary points. It is a method that does not. In FIG. 13, the hollow rectangle indicates the position of the point to be interpolated, the solid rectangle indicates the boundary point, the dotted rectangle indicates the concentric circle to be searched, and the triangular portion drawn by the light dotted line indicates the boundary point. The area of the azimuth angle excluded from the search target after leaving is shown. The above method is a method of searching for a single closed curve including a point to be interpolated. When interpolating the 2 ′ point in FIG. 11 which is one-dimensional, if the boundary point 2 is searched when searching to the left side. For example, it means that no further search is performed on the left side. Thus, if the boundary points are searched, the 1 / r or exp (-r ^2/1000) is set to the weighted value by performing interpolation to calculate the height of the points is erased, the A height map is finally completed by substituting the height calculated by the interpolation for the pixels from which the height information has been removed (S34). FIG. 14 shows an example of a three-dimensional profile (height) map finally completed according to the present invention.

  Thus, when the final height map of the sample image is completed, the highest wavelet sub-band value is reconstructed in the sub-band of the discrete wavelet transform (DWT) of the height, and each of R, G, and B is reconstructed. After the soft sub-band of the approximate sub-band of the highest level wavelet is used as a weighted average, the multiple inverse discrete wavelet transform (Inverse DWT) is performed to obtain a final composite image at the resolution level of the input image. It is done. The final composite image that is generated is a two-dimensional image that is in focus (focused) and extended in depth over the entire area of the image.

Claims (9)

Imaging a sample from different heights to obtain a number of two-dimensional sample images with different focal points;
Performing a discrete wavelet transform on the sample image;
In the detailed sub-band obtained by performing the discrete wavelet transform on each sample image, the detailed sub-band coefficient value of each pixel is compared, and the initial height for each pixel is obtained at the imaging height of the image showing the maximum coefficient value. Creating a height map;
In each input image, calculating a focus matching degree that is an evaluation value for detecting a pixel that is a focal point and a boundary point with respect to the approximate subband region of the image subjected to the discrete wavelet transform;
Filtering the focus matching degree to leave pixel height information that is in- focus and boundary points, and removing pixel height information that is not .
Calculating a height of a pixel (non-boundary point) removed without passing through the filter from a height value of a pixel (boundary point) that has passed through the filter;
Substituting the height calculated by the interpolation for the pixels from which the height information has been removed, and creating a height map;
Of creating a three-dimensional profile map including After the step of creating the initial height map, the detailed sub-band wavelet coefficients of the pixels calculated as the same height are added within the verification window centered on each pixel, and the sum of the wavelet coefficients according to the height is added. The method of creating a three-dimensional profile map according to claim 1, further comprising the step of comparing the heights that give the largest value with the actual height of the pixel.   The method according to claim 2, wherein the verification window has a pixel size of 3 × 3 with each pixel as a center.   The method of claim 2, wherein the method of selecting pixels to be grouped, i.e. grouped by the detailed sub-band wavelet coefficients, is to select pixels corresponding to the same height in the verification window as the same group. 3D profile map creation method. The focus matching degree of the approximation Sabubando region, as shown in the following equation 1, the pixel (coordinates: (x, y)) brightness of the (I (x, y)), peripheral four points (coordinates :( 3. The calculation according to claim 1, which is calculated as a sum of absolute values of differences from brightness of (x, y + 1), (x, y−1), (x + 1, y), (x−1, y)). How to create a dimension profile map.
  6. The three-dimensional profile according to claim 5, wherein if the focus matching degree is less than three times the brightness difference at which a boundary between adjacent pixels appears, it is regarded as a non-focus pixel and height information is removed. How to create a map.   The height of the point to be interpolated is obtained from the weighted average of the heights of the boundary points, and the weighted value function of the weighted average is a reciprocal distance (1 / r, where r is the point to be interpolated and the boundary The method of creating a three-dimensional profile map according to claim 1, which represents a distance between points) or a Gaussian function of distance. Weighting value a function of the weighted average is 1 / r or ^{exp (-r 2/1000),} 3 -dimensional profile map creation method according to claim 7.   Selection of a boundary point for interpolating the height of the non-boundary point is performed by drawing a concentric circle or a concentric rectangle centering on the point to be interpolated (boundary point) and searching from the center outward. If found, it is performed in such a manner that no more boundary points are searched within a range of ± 30 ° from the found boundary point (azimuth angle where the boundary point was found). A method for creating the described three-dimensional profile map.