KR102011910B1 - 3 dimensional shape detection apparatus and method - Google Patents

Abstract

According to the present invention, a position of a circuit component, which is a region of interest in a low-resolution pattern, is identified, the region of interest can be moved by an optical and a mechanical method for precise measurement, and an optimal high-resolution pattern for each region of interest can be selected and measured, thereby increasing a processing speed and making a precise depth measurement.

Description

3D shape measuring device and method {3 DIMENSIONAL SHAPE DETECTION APPARATUS AND METHOD}

Circuit boards having electronic integrated circuits and discrete electronic components are well known. The circuit board is provided with predetermined conductive paths and pads to receive the leads of electronic components such as integrated circuit chips, resistors or capacitors. During the assembly process of the circuit board, solder paste deposits are placed in a proper position on the circuit board. Solder paste precipitates are usually applied by placing a stencil screen on the substrate, applying solder paste through stencil openings, and removing the stencil from the substrate.

 The electronic components of the circuit board are then placed over the substrate with a machine that picks and places the leads of the electronic components overlying each solder paste precipitate. After all the components have been placed on the substrate, the circuit board passes through an oven to melt the solder paste precipitate to form an electrical and / or mechanical connection between the component and the substrate.

As miniaturization has become more emphasized in the electronics industry, the size of solder paste precipitates and electronic components and the accuracy with which they must be placed on substrates have become smaller and more stringent. The height of the solder paste precipitate can be as small as 50 microns and the depth of the solder paste brick should usually be measured within 1% of the design depth and size.

Discrete electronic components such as resistors and capacitors can be as small as 200x400 microns, and leads on micro ball grid array components can have center-center spacing less than 300 microns. The center-center spacing between the solder bricks can sometimes have a small size of 200 microns.

If the amount of solder paste is too small, there may be no electrical connection between the leads of the electronic components and the pads of the circuit board. Too much paste may cause bridging and short-circuiting between the leads of the part. After placement, it is also important to inspect the part to ensure that it is properly placed. Misplaced parts, missing parts, or low solder joints are typical defects that occur during the placement of parts and the reflow process of solder paste. After the reflow process, an automated optical inspection system can be used to check the proper placement of the parts and the quality of the reflowed solder joints to ensure that all parts are properly soldered and connected to the circuit board. Current optical inspection systems use 2D video images of circuit boards to detect defects. However, optical inspection systems that detect 3D depth images of circuit boards may enable detection of lifted leads, package coplanarity and misalignment such as tombstones and billboards of components. Or improve.

Techniques for detecting 3D depth images are well known in structured light. The structured light method projects a specially designed optical pattern onto an object for generating depth information, and then acquires image information on which the optical pattern is projected through a camera. Projected optical pattern decoding is performed on the acquired image to find a corresponding point between the image and the projected structured light pattern. After finding the corresponding point, depth information is calculated by triangulation.

 FIG. 1C illustrates an image of an image in which the structured light pattern having the form as shown in FIG. 1A is irradiated to the target object 100 as shown in FIG. 1B. As shown in FIG. 1C, distortion occurs in a portion of the target object 100 having a change in depth, thereby separating the same linear patterns (pattern D and pattern D ', pattern E and pattern E'), and separating the pattern D. It can be seen that parallax has occurred between and the pattern D 'and the pattern E' and the pattern E '. The number of pixels corresponding to the size of parallax may also be known through the captured image. For example, it can be seen that the pattern is separated from each other by 20 pixels between the pattern D and the pattern D 'and between the pattern E and the pattern E'.

To describe this in detail, referring to FIG. 2, b means a baseline which is a distance between the infrared light 210 and the lens 220 in the black and white sensor.

The reference plane 230 refers to any plane defined as a reference for measuring the depth of a specific object. ZO corresponds to a distance between the sensor and the reference plane 230, and may be predetermined.

f is the focal length and corresponds to the distance between the lens 220 in the monochrome sensor and the image plane 250 of the imaging device.

In the 3D depth measurement, the reference image, which is a specific reference, projects the structured light from the infrared light 210 on an arbitrary reference plane 230, and detects the structured light reflected from the reference plane by using a black and white sensor to image the image. It may mean.

In order to obtain the reference image, the structured light is projected from the infrared light 210 to the reference plane 230 spaced a predetermined distance ZO. The projected structured light is reflected by the reference plane 230 and sensed by the monochrome sensor.

The image of the pattern of the structured light projected and detected on the reference plane 230 becomes the reference image. In this case, the reference image may be generated in advance and stored in the memory.

Then, in order to measure the three-dimensional depth of the object 240, structured light having the same pattern as that used when acquiring the reference image may be projected onto the object 240. In FIG. 2, the object 240 is illustrated as one surface, but for convenience of description, structure light will be projected onto each surface of the object 240.

When the structured light reflected by the object 240 is obtained as an image by the black and white sensor, the positional movement of the corresponding patterns in the obtained image may be measured based on the reference image. The position shift of the pattern corresponds to d in FIG. 2, which means a disparity corresponding to k points of the object 240 based on the reference image.

According to one example, using the similarity of the triangle in Figure 2, it can be seen that the relationship of D / b = (ZO-ZK) / ZO, d / f = D / ZK.

Summarizing the above two equations can be summarized as the following equation.

Zk = Z0 / (1 + (Z0 x d) / (f x b))

In the above equation, f is a focal length, b is a baseline, and ZO is a distance to the reference plane 230, which may be predetermined. Therefore, if the parallax d for any point k of the object 240 is measured in the obtained image, it is possible to calculate the three-dimensional depth ZK, which is the distance to the object 240.

In order to apply the structured light three-dimensional depth measurement technology to a small object such as an optical inspection system, the pattern resolution is increased by densely spaced patterns, and the resolution is different for each region according to the arrangement state of circuit components. We need a way to investigate the pattern of.

In addition, processing a whole with a high resolution pattern takes a lot of processing time, so in order to shorten the processing time, the position of a circuit component as a region of interest is identified with a low resolution pattern, a region of interest is moved for precise measurement, and an optimal high resolution pattern for each region of interest is selected. A method that can be selected and measured is needed.

The present invention to solve the above problems, a method and apparatus for identifying the position of the circuit component of the region of interest in a low-resolution pattern, moving the region of interest for precise measurement and selecting the optimal high-resolution pattern for each region of interest Its purpose is to provide.

According to an aspect of the present invention to achieve the above or another object, at least one pattern projection unit for projecting a variety of optical patterns on the target object, at least one first optical filter unit for projecting a first optical pattern on the target object, A second optical filter unit for generating a second optical pattern and controlling a projection area to be projected onto a region of interest of the target object including the target object, a first camera to obtain a 2D image of the target object, and the target object A second camera for acquiring an optical pattern image projected onto the camera, a control unit for controlling the pattern projector and the camera, and performing image processing and depth calculation, wherein the controller detects an object candidate group using the 2D image, A depth map is calculated using the optical pattern image, and the zero interest of interest is formed using the object candidate group and the depth map. The identification and selection of a specific said area of interest and the second optical patterns, and wherein the or part the pattern projection to move the projection area of the second optical patterns, provides a three-dimensional shape measuring system for controlling the optical filter part.

The target object may be a circuit board, the target object may be a circuit component, the first camera may be a color camera, and the second camera may include an infrared camera.

The optical filter unit may project the first optical pattern periodically by using any one of a micro mirror and a movable prism for the light incident from the laser light source in the infrared region.

The second optical pattern selection may be performed by controlling the spot size of the laser light source in the infrared region incident on the optical filter unit to control the pattern size of the random speckle pattern generated by the diffuser plate, thereby controlling the size, position, and shape of the target object. The resolution may be adjusted according to any one or more of the above.

The optical filter unit may project a second optical pattern in the form of an aperiodic random speckle pattern using a laser light source and a diffusion plate in an infrared region.

The second optical pattern may have a higher resolution than the first optical pattern.

The movement of the projection area may move one or more pattern projectors up and down to move the projection area to a region of interest including the periphery of the target object.

The projection area movement may move the projection area to the target object position using an electrically controllable diffractive lens included in the optical filter unit.

The projection region movement may electrically control the liquid crystal in the diffractive lens to project only the random region to project the random speckle pattern to move the projection region to the target object position.

Further, according to another aspect of the invention, obtaining a 2D image of the target object, detecting a group of object candidates, projecting a first optical pattern on the target object, utilizing the obtained first pattern image using the target object Calculating a first depth map of the image, identifying a region of interest including the target object by using the depth map and the object candidate group, selecting a second optical pattern for each region of interest, and displaying a pattern projection region Moving to a region of interest, projecting a second optical pattern onto the target object, and calculating a second depth map of the target object by using the acquired second pattern image. do.

The target object may be a circuit board, the target object may be a circuit component, the 2D image may be a color image, and the first and second pattern images may be infrared images.

In the projecting of the first optical pattern, the light incident from the laser light source in the infrared region may be projected on the periodic first optical pattern using a micro mirror of the optical filter unit or a movable prism.

The selecting of the second optical pattern may include controlling a spot size of a laser light source in an infrared region to control a pattern size of a random speckle pattern generated by a diffuser plate, thereby controlling at least one of a size, a position, and a shape of a target object. You can adjust the resolution accordingly.

In the projecting of the second optical pattern, light incident from a laser light source in an infrared region may project a second optical pattern in the form of an aperiodic random speckle pattern using a diffuser plate of the optical filter unit.

The second optical pattern may have a higher resolution than the first optical pattern.

The detecting of the object candidate group may detect the object candidate group by edge detection in the 2D image of the target object.

The calculating of the first and second depth maps may calculate a depth map by triangulation by measuring a relative moving distance between the projected optical pattern and the pattern detected on the pattern image.

The moving of the projection area may move one or more pattern projection parts up and down to move the projection area to the region of interest including the periphery of the target object.

In the moving of the projection area, the projection area may be moved to a region of interest including the target object position by using an electrically controllable diffractive lens included in the optical filter unit in the pattern projector.

The moving of the projection area may electrically control the liquid crystal in the diffractive lens to project a random speckle pattern only in a specific area, thereby moving the projection area to a region of interest including the target object position.

The method may further include determining whether the target object is defective by comparing the second depth map with a reference depth map.

The calculating of the second depth map may include projecting a second optical pattern onto the same projection area by one or more pattern projectors which are moved up and down, and synchronizing with the second pattern image obtained a plurality of times to obtain a second depth map. Can improve the accuracy.

In the calculating of the second depth map, the depth resolution may be improved by first processing the second depth map by the pattern projector which is relatively close to the target object.

In the present invention, the location of the region of interest uses a low resolution pattern, and the precise depth measurement of the region of interest may utilize a high resolution pattern to shorten the processing time and improve the depth measurement accuracy.

1A is a structured light projection pattern of the prior art.
1B is a target object for depth measurement of the prior art.
1C is an image obtained for depth measurement of the prior art.
2 is a conceptual diagram illustrating a depth measuring method using structured light according to the related art.
3 is a block diagram of a three-dimensional shape measurement system according to an embodiment of the present invention,
4 is an image projecting the first broad pattern according to an embodiment of the present invention.
5 is an image projecting a second optical pattern according to an embodiment of the present invention.
FIG. 6 is an example of a random speckle pattern as an example of a second optical pattern according to an exemplary embodiment.
7 is a block diagram of an optical filter for generating and controlling a second optical pattern according to an embodiment of the present invention.
8 is a flowchart illustrating a 3D shape measuring method according to an embodiment of the present invention.
9 is a movable prism for generating a first optical pattern according to an embodiment of the present invention.
10 is a micromirror for generating a first optical pattern according to an embodiment of the present invention.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings, and the same or similar components are denoted by the same reference numerals regardless of the reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used in consideration of ease of specification, and do not have distinct meanings or roles from each other. In addition, in describing the embodiments disclosed herein, when it is determined that the detailed description of the related known technology may obscure the gist of the embodiments disclosed herein, the detailed description thereof will be omitted. In addition, the accompanying drawings are intended to facilitate understanding of the embodiments disclosed herein, but are not limited to the technical spirit disclosed herein by the accompanying drawings, all changes included in the spirit and scope of the present invention. It should be understood to include equivalents and substitutes.

Terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

Singular expressions include plural expressions unless the context clearly indicates otherwise.

In this application, the terms "comprises" or "having" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Referring to FIG. 3, FIG. 3 is a block diagram illustrating a three-dimensional measurement system related to the present invention.

The system 300 may include an infrared camera 301, an RGB camera 302, a pattern projector 303, an optical filter 304, a circuit board 305, and the like. The components shown in FIG. 3 are not essential to implementing a three dimensional measurement system, such that the three dimensional measurement system described herein may have more or fewer components than those listed above.

The pattern projector 303 may project light onto the circuit board 305 by generating various patterns according to the object through the optical filter unit 304 to the light emitted from the infrared laser light source, and includes one or more, Control the area where the pattern is irradiated by moving up and down.

The IR (infrared) camera 301 captures an image of the first optical pattern 401 projected onto the circuit board 305 which is a measurement object, and measures the amount of change of the pattern in the controller 306 to generate depth information.

The RGB (color) camera 302 measures an image of the circuit board 305 which is a measurement object, determines the edge of the circuit component 402 on the image by the controller 306, and determines the position of the circuit component 402. do.

Referring to FIG. 4, FIG. 4 is a diagram for describing an image in which a first optical pattern is irradiated to a circuit board, which is a measurement object of a three-dimensional measurement system according to the present invention. A plurality of circuit components 402 are disposed on the circuit board 305 and the first optical pattern 401 for three-dimensional depth measurement is irradiated.

The first optical pattern 401 corresponds to an infrared region, and although the pattern can be projected on the entire circuit board 305, there is a limit in increasing the spatial resolution in a pattern having periodicity such as a lattice form. Therefore, accurate depth information of the correct edge portion of the circuit component 402 cannot be obtained.

Referring to FIG. 5, FIG. 5 is a diagram for describing an image in which an optimal second optical pattern is irradiated to a circuit board, which is a measurement object of a three-dimensional measurement system according to the present invention.

A plurality of circuit components 402 are disposed on the circuit board 305, and the second optical pattern 501 for three-dimensional depth measurement is irradiated.

The second optical pattern 501 corresponds to an infrared region, and the optical filter unit 304 controls the traveling direction of the light to project an optimal pattern for each position of the circuit component 402, and has a randomness. You can increase the spatial resolution with the Random Speckle Pattern. Thus, accurate depth information of the correct edge portion of the circuit component 402 can be obtained.

6 is an example of a random speckle pattern projected onto an object in a second optical pattern.

7 is a detailed configuration diagram of the optical filter unit 304. Referring to FIG. 7, the optical filter unit 304 may include a diffuser 701 and an electrically controlled diffractive lens 702.

The diffuser plate 701 diffuses the laser light from the infrared laser light source to generate a random speckle pattern. In addition, the spot size of the laser light source may be adjusted to adjust the pattern size of the random speckle pattern. Therefore, the optimal pattern can be selected according to the position or shape of the circuit component.

The diffractive lens 702 may change the direction of the random speckle pattern and electrically control the liquid crystal element included therein to control the projection area such that the pattern is projected only on the required projection area.

8 is a flowchart illustrating a three-dimensional shape measuring method of a three-dimensional shape measuring system according to an exemplary embodiment of the present invention. An RGB camera 302 may be used to acquire a 2D image of a circuit board as an object (801). In an embodiment, the processes illustrated in FIG. 8 may be performed by the controller controlling another object.

An object candidate group may be derived through image processing for edge detection using the acquired 2D image (802).

One or more pattern projectors project the first optical pattern 401 onto the object (803). Although the first optical pattern 401 can be projected on the entire circuit board 305 in the infrared region, there is a limit in increasing the spatial resolution in a pattern having periodicity such as a lattice shape. The projection of the first pattern uses the optical element of any one of a micro mirror (see FIG. 10) and a movable prism (see Prism, see FIG. 9) installed in the optical filter unit to propagate light incident from the infrared laser light source. The direction may be controlled to scan the first optical pattern 401 on the object.

Acquire an image in which the first optical pattern is projected onto the circuit board by using the IR camera 301, measure a distance traveled by the pattern on the acquired image relative to the projected pattern, and include depth information for each region by triangulation. A depth map may be calculated (804).

The location of the circuit component may be determined using the object candidate group and the depth map in operation 805. Some of the object candidate groups obtained by edge detection may not have a height with a mark or the like formed on a substrate. Therefore, whether the circuit component 402 is determined based on the depth information of the corresponding region of the object candidate group and the position information on the 2D image may be determined.

An optimal second optical pattern is selected for each circuit component 402 (806). The necessary spatial resolution is selected according to any one or more of the position, size and shape of the circuit components, and the spot size of the laser light source is controlled to adjust the size of the random speckle pattern to secure the required spatial resolution. In an embodiment, the resolution of the second optical pattern may be higher than the resolution of the first optical pattern.

One or more pattern projectors 303 are moved up and down to move the pattern projection area around the circuit component 402 (807). One or more pattern projectors 303 installed at a specific angle are moved up and down to move the area where the pattern is projected. In this case, the projection areas of the plurality of pattern projectors 303 may be moved to overlap the periphery of the circuit component 402.

The optical filter portion 304 is controlled to match the projection area of the pattern with the position of the circuit component 402 (808). The liquid crystal portion of the diffraction lens 702 in the optical filter portion 304 can be electrically controlled to control the pattern to be irradiated only at the position of the circuit component 402. The optimal second optical pattern selected for each circuit component 402 is projected onto the designated projection area according to the position of the circuit component 402 (809).

A precision depth map is calculated from the image obtained by the IR camera 301 (810). By comparing the projected random speckle pattern (Random Speckle Pattern) and the acquired image to determine the relative pattern shift amount and to calculate the depth information. When the projection areas of the plurality of pattern projectors 303 are moved to overlap the periphery of the circuit component 402, the depth information of the projection area is synchronized by synchronizing the projection time of the pattern and the image acquisition time for each pattern projector 303. It is possible to improve the accuracy of the shape measurement by calculating a plurality of times. In this case, the pattern projector 303 having a close distance to the circuit component 402 has a higher depth resolution than the other pattern adverb, so that the depth information can be generated by reflecting it first.

The precision depth map is compared with information on a normal circuit board 305 having reference depth information to check whether the circuit board 305 is defective (811).

 The present invention described above can be embodied as computer readable codes on a medium in which a program is recorded. The computer-readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include hard disk drives (HDDs), solid state disks (SSDs), silicon disk drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and the like. This also includes implementations in the form of carrier waves (eg, transmission over the Internet). The computer may also include a controller.

Accordingly, the above detailed description should not be construed as limiting in all aspects and should be considered as illustrative. The scope of the invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the invention are included in the scope of the invention.

100: target object 210: infrared light
220: lens 220 in monochrome sensor 230: reference plane
240 object 250 Image plane of the imaging device
300: 3D shape measurement system 301: color camera
302: infrared camera 303: pattern projection unit
304: optical filter portion 305: circuit board
306: control unit 401: first optical pattern
402: circuit component 501: second optical pattern
701: diffuser plate 702: diffractive lens

Claims (23)

One or more pattern projectors for projecting various optical patterns onto a target object;
At least one first optical filter to project a first optical pattern onto the target object;
A second optical filter unit generating a second optical pattern and controlling a projection area to be projected onto a region of interest of the target object including a target object;
A first camera obtaining a 2D image of the target object;
A second camera obtaining an optical pattern image projected on the target object;
And a controller for controlling the pattern projector and the camera, and performing image processing and depth calculation.
The object candidate group is detected using the 2D image, a depth map is calculated using the optical pattern image, the region of interest is identified using the object candidate group and the depth map, and the second optical for each region of interest is used. Select the pattern and control the optical projection part or the pattern projection part to move the projection area of the second optical pattern,
The first optical pattern is an infrared band pattern having a lattice periodicity,
The optical filter unit
The light incident from the laser light source in the infrared region is projected on the periodic first optical pattern using either a micro mirror or a movable prism,
3D shape measurement system having a different resolution between the first optical pattern and the second optical pattern. The method of claim 1
The target object is a circuit board, the target object is a circuit component, the first camera is a color camera, and the second camera is an infrared camera. delete The method of claim 1
The second optical pattern selection
By controlling the spot size of the laser light source in the infrared region incident on the optical filter part, the pattern size of the random speckle pattern generated by the diffusion plate is controlled to adjust the resolution according to any one or more of the size, position and shape of the target object. Adjustable three-dimensional shape measurement system. The method of claim 1
The optical filter unit
A three-dimensional shape measurement system for projecting a second optical pattern in the form of an aperiodic random speckle pattern using a laser light source and a diffuser plate in the infrared region. The method of claim 1
And the second optical pattern has a higher resolution than the first optical pattern. The method of claim 1
The projection area shift is
3D shape measurement system for moving the at least one pattern projection unit up and down to move the projection area to the region of interest including the periphery of the target object. The method of claim 1
The projection area shift
3D shape measurement system for moving the projection area to the target object position using an electrically controllable diffraction lens included in the optical filter unit. The method of claim 8
The projection area shift is
Electrically controlling the liquid crystal in the diffractive lens to project a random speckle pattern only in a specific area to move the projection area to the target object position. Obtaining a 2D image of the target object and detecting an object candidate group;
Projecting a first optical pattern onto the target object and calculating a first depth map of the target object by using the obtained first pattern image;
Identifying an ROI including a target object by using the depth map and the object candidate group;
Selecting a second optical pattern for each region of interest;
Moving a pattern projection area to the region of interest;
Projecting a second optical pattern onto the target object and calculating a second depth map of the target object by using the acquired second pattern image;
Including,
The first optical pattern is an infrared band pattern having a lattice periodicity,
Projecting the first optical pattern
The light incident from the laser light source in the infrared region is projected to the first optical pattern by using a micro mirror of the optical filter unit or a movable prism,
The method of claim 3, wherein the first optical pattern and the second optical pattern have different resolutions. The method of claim 10
The target object is a circuit board, the target object is a circuit component, the 2D image is a color image, and the first and second pattern images are infrared images. delete The method of claim 10
The step of selecting the second optical pattern
Three-dimensional shape measurement by controlling the spot size of the laser light source in the infrared region to control the pattern size of the random speckle pattern generated by the diffuser to adjust the resolution according to at least one of the size, position, and shape of the target object Way. The method of claim 10
Projecting the second optical pattern
3. A method of measuring a three-dimensional shape in which light incident from a laser light source in an infrared region is projected using a diffusion plate to project a second optical pattern in the form of an aperiodic random speckle pattern. The method of claim 10
The second optical pattern has a higher resolution than the first optical pattern, characterized in that the three-dimensional shape measurement method. The method of claim 10
Detecting the object candidate group
3D shape measurement method for detecting the object candidate group by the edge detection in the 2D image of the target object. The method of claim 10
Computing the first and second depth maps
A three-dimensional shape measuring method for calculating a depth map by triangulation by measuring a relative moving distance between a projected optical pattern and a detected pattern on a pattern image. The method of claim 10
Moving the projection area
3. The method of claim 3, wherein the projection area is moved upward and downward to move the projection area to the region of interest including the periphery of the target object. The method of claim 10
Moving the projection area
3. The method of claim 3, wherein the projection area is moved to a region of interest including the target object position by using an electrically controllable diffraction lens included in an optical filter unit in a pattern projector. The method of claim 19
Moving the projection area
And electrically controlling the liquid crystal in the diffractive lens to project a random speckle pattern only to move the projection area to the region of interest including the target object position. The method of claim 10
And comparing the second depth map with a reference depth map to determine whether the target object is defective. The method of claim 10
Computing the second depth map
A three-dimensional shape measuring method of projecting a second optical pattern onto the same projection area in one or more pattern projectors moved up and down, and synchronizing with the second pattern image obtained a plurality of times to improve the accuracy of a second depth map. . The method of claim 22,
Computing the second depth map
3. The method of claim 3, wherein the depth resolution is improved by first processing the second depth map by the pattern projector which is relatively close to the target object.