KR20110124409A - Omni-directional distance measurement device and method based on structured light using difference image integral calculus algorithm - Google Patents

Abstract

PURPOSE: An omnibeary-distance facility and method using a forward direction mirror and difference image integral algorithm are provided to secure resistance to ambient illumination noises by reinforcing structured light pixels. CONSTITUTION: An omnibeary-distance facility comprises an irradiation part(110), a measuring part(130) and an analysis part(150). An irradiation part emits laser beam in a forward direction. The measuring part comprises a convex surface type mirror(131) and a camera(133). A background image is gotten repetitively from the convex surface type mirror by using camera. The analysis part calculates the object distance between the object and the camera.

Description

Omni-directional distance measurement device and method based on structured light using difference image integral calculus algorithm}

The present invention relates to a laser structured light image-based omnidirectional distance measurement, and more particularly, to a distance measuring apparatus and method considering a differential image integration algorithm.

In order to autonomously drive a moving object such as a mobile robot or an unmanned transport vehicle in a factory, a distance measuring sensor capable of measuring a distance to objects around the moving object is required. The distance information to the surrounding objects can be used to estimate the current position of the moving object itself through matching with a global database of known object maps, as well as whether there are obstacles, and also to incrementally connect local maps around the moving object. Can be used to create a global object map.

In order to determine not only whether there are obstacles in the direction to move, but which direction can be moved, or to estimate the current position of the moving object by local mapping, the distance information in the direction as wide as possible from the current position of the moving object is obtained. You should be able to get it. In general, since distance measuring sensors measure only one direction distance, in order to obtain distance information in a wide direction, a plurality of such distance measuring sensors are arranged in a circumferential shape, or a method of mechanically scanning one distance measuring sensor is used. . However, these methods are expensive and have a problem in that it takes a long time to obtain wide-angle information.

Also, to measure the distance, use two cameras to find the distance by applying trigonometry to the same point on the two image planes (for example, a stereo camera), and emit waves such as ultrasonic waves or electromagnetic waves to the object. There is a method of measuring distance by measuring a time-of-flight (TOF) until reflection and returning, and a method using a structured light image-based method. The structured light image-based method is a method of calculating a distance by irradiating a structured light separated from an ambient light, such as a visible light laser or an infrared laser, to an object, acquiring reflected light as a camera image, and analyzing distortion according to an object distance. .

The stereo imaging method using two cameras has a problem of slowing distance information acquisition and being very weak to ambient lighting noise because it requires a large amount of computation time to find the same point in the image plane. The method requires a very precise electronic circuit is expensive, and there is a problem in that only the distance in one direction is basically measured.

Accordingly, by irradiating structured light in all directions of 360 degrees, and obtaining structured light reflected by surrounding objects as a single camera image using an omnidirectional mirror, it is possible to measure a 360-degree omnidirectional distance quickly and efficiently. There is a need for an apparatus and method using an optical image based method.

On the other hand, since the structured light image is affected by ambient light such as the brightness of sunlight or fluorescent lamps, the resulting object distance information is sensitive to lighting noise. There is a method of using an optical filter as a conventional technique for separately extracting the structured light from the structured light image. For example, when a red laser having a wavelength of 650 nm is used as a structured light source, an optical filter that passes only a wavelength of 650 nm is used to filter out images other than the structured light, and only structured light is extracted from the image plane by passing only the structured light. can do. The same applies to the case of using an infrared laser having a wavelength of 780 nm which is generally used as a structured light source. However, this method has a cost burden of using a separate filter, and there is a problem that these components appear as image noise when they have light components in a wavelength band such as structured light in sunlight or fluorescent lamps.

Accordingly, there is a need for an apparatus and method for extracting only a portion of structured light from an image plane by using a method of extracting only an object image from an image including an object except a background image.

The technical problem to be achieved by the present invention is to irradiate structured light in 360 degrees omnidirectional, and to obtain structured light reflected by surrounding objects as a single camera image using an omnidirectional mirror very quickly and efficiently in 360 degrees omnidirectional. It is to develop an omnidirectional structured light image processing hardware that measures distance but is robust against ambient lighting noise and can be processed very quickly and efficiently.

An omnidirectional distance measuring apparatus according to the present invention comprises: an irradiation unit for irradiating laser structured light in all directions; And a convex mirror and a camera, wherein the camera acquires a background image from the convex mirror, the laser structure light irradiated from the irradiator is reflected on an object, and the reflected laser structure light is reflected into the convex mirror. A measuring unit repeatedly obtaining an incident distortion image; And an analyzer configured to acquire a trajectory of the laser structured light reflected by using the difference image between the background image and the distorted image, and to calculate an object distance between the camera and the object from the trajectory of the reflected laser structured light.

In the omnidirectional distance measuring method according to the present invention, irradiating a laser structured light in all directions, obtaining a background image which is an image before irradiating the laser structured light, the irradiated laser structured light is reflected on an object Repeatedly acquiring a distorted image, which is a later image, through the camera, acquiring a trajectory of the reflected laser structured light by integrating a difference between the repeatedly obtained background image and the distorted image; Calculating an object distance between the camera and the object from a trajectory.

The present invention has the effect of measuring the object distance in all directions of 360 degrees at a time by using the irradiation unit for irradiating the laser structure light in all directions of 360 degrees and the measuring unit for obtaining an image through a convex mirror and one camera.

The present invention is economical because it is possible to extract structured light pixels from the structured light image without a separate optical filter. In addition, by repeatedly integrating the structured light difference image sequence over time, the structured light pixels are strengthened compared to the surrounding pixels. In the optical image processing, there is an effect that it can secure the robustness to the ambient light noise.

The present invention has the effect of reducing the computation time by using an FPGA and a parallel memory structure without implementing the difference image integration algorithm with a general processor.

1 is a block diagram showing a structured light image-based omnidirectional distance measuring apparatus.
2 shows an example of a structured light image-based omnidirectional distance measuring apparatus according to the present invention.
3 is an example of irradiating laser structured light linearly according to the present invention.
4 is another example of irradiating laser structured light in all directions according to the present invention.
5 shows a vertical cross section of a omnidirectional imaging device structure including an omnidirectional mirror and a camera.
FIG. 6 is a diagram illustrating extraction of structured light using a difference imaging method in a state in which structured light is turned off and structured light is turned on according to the present invention.
7 illustrates a block diagram of the structured light integral integration image processing hardware of the analyzer 150.

1 is a block diagram showing a structured light image-based omnidirectional distance measuring apparatus.

According to FIG. 1, the omnidirectional distance measuring device includes an irradiation unit 110 for irradiating a laser structured light in all directions, a measuring unit 130 including a convex mirror 131 and a camera 133, the camera and the object. It includes an analysis unit 150 for calculating the distance between objects.

Here, the irradiation unit 110 includes a laser source 111, the laser source 111 may be a point laser source or a circular laser source, a red or green visible light laser source or an infrared laser You can use something that separates the light from the source. Also, the irradiator 110 may include a dispersion lens 113. Through the irradiation unit 110 to irradiate the laser structured light in a 360-degree direction.

In addition, the measuring unit 130 obtains a background image which is an image before irradiating the laser structured light from the convex mirror 131 through the camera 133, and the laser structured light irradiated by the irradiator 110 is reflected on the object. Afterwards, a distorted image incident on the convex mirror is obtained. If the process of repeatedly acquiring an image with the laser structure light turned off and obtaining an image with the laser structure light turned off, the background image and the distorted image may be repeatedly obtained.

In addition, the analysis unit 150 obtains the difference between the repeatedly obtained background image and the distorted image using pixel values, and obtains the trajectory of the reflected laser structured light using a difference image integration algorithm. The object distance between the camera and the object is calculated from the trajectory of the reflected laser structured light.

2 shows an example of a structured light image-based omnidirectional distance measuring apparatus according to the present invention.

According to FIG. 2, the omnidirectional distance measuring apparatus may include a laser source 201, an omnidirectional mirror 203, and a camera 205. There is a laser source 201 for irradiating structure light 207 to the bottom of the camera, the camera 205 may be installed in a vertical direction. The camera 205 may acquire an image of all directions in 360 degrees on a single screen through an omni-directional mirror 203 installed at an upper portion thereof. As the omnidirectional mirror 203, a convex mirror can be used, and the convex mirror includes a conical or hyperbolic mirror or the like.

3 is an example of irradiating laser structured light linearly according to the present invention.

According to FIG. 3, the point laser source can be converted linearly through the cylinder lens.

In general, the laser source generates a point-shaped structure light, thereby converting the point-type laser source to a linear. The converted linear laser may be dispersed at 70 degrees to 130 degrees depending on the characteristics of the cylinder lens. A plurality of distributed linear lasers may be arranged to cover the 360 degree omnidirectional direction. Since one linear laser covers an area of about 90 degrees or 120 degrees, it is possible to irradiate structured light in 360 degrees by using four linear lasers at about 90 degrees and three linear lasers at about 120 degrees.

4 is another example of irradiating laser structured light in all directions according to the present invention.

According to FIG. 4, the structured light may be irradiated in 360 degrees by using a conical mirror and a circular laser. When a circular laser source is reflected on a convex mirror to reflect the structured light in 360 degree directions.

5 shows a vertical cross section of a omnidirectional imaging device structure including an omnidirectional mirror and a camera. In FIG. 5, a conical mirror is shown as an omnidirectional mirror, but any parabolic, hyperbolic, or the like can be used.

에서 반사된 레이저 구조광은 볼록면형 거울면에서 다시 반사되어 카메라의 핀홀을 지나 영상 센서면에 상을 맺게 된다. 5 is a view based on an axis on which a laser is installed and irradiated, wherein the camera and the convex mirror are symmetrically in the circumferential direction. The camera includes a pinhole and the image sensor surface, which is irradiated from a laser source installed below

The reflected laser structure light is reflected back from the convex mirror surface and passes through the pinhole of the camera to form an image on the image sensor surface.

를 갖는 원뿔면(conic)형 거울을 사용하는 경우, 물체거리

은

와 같이 볼록면형 거울의 높이

, 카메라 핀홀의 높이

, 영상면에서 구조광 측정각

와 원뿔각

의 함수로 표현된다.The irradiated structured light has a property of being distorted according to the distance from the object in the image plane. In other words, the farther the distance is from the center of the image, the closer the distance is from the center of the image. The object distance can be obtained by analyzing the degree of the structure light distortion. Cone angle

When using a conic mirror with

silver

Height of convex mirror

, Height of camera pinhole

Structured light measurement angle

And cone angle

It is expressed as a function of.

는 는 원뿔형 거울을 사용하는 경우, 다음과 같이 얻어진다.·Specifically distance expression

When using a conical mirror, is obtained as follows:

이고,

와

는

을 계산하는 과정에서 나타난 매개변수로서, 원뿔형 거울면상의 반사점이고, 다음과 같이 구할 수 있다.here

ego,

Wow

Is

This parameter appears in the process of calculating, which is the reflection point on the conical mirror surface, and can be obtained as follows.

As a method for extracting laser structured light, a difference image sequence obtained by using a difference image processing algorithm and a difference image processing algorithm for extracting structured light from an image plane by software without using a separate optical filter is By accumulating, it is possible to use an integrated image processing algorithm that further highlights the structured light and makes it robust to ambient lighting noise.

FIG. 6 is a diagram illustrating extraction of structured light using a difference imaging method in a state in which structured light is turned off and structured light is turned on according to the present invention.

According to FIG. 6, an image is first obtained with the structured light turned off, and then an image is obtained with the structured light turned on. The difference between the two images can be calculated to extract only the laser structured light components. When calculating the difference between two images, the difference between corresponding pixel values of two images may be obtained. At this time, only the pixels of the portion irradiated with the laser structure light have different values, and the other background pixels have the same value. Therefore, if the difference between the corresponding pixels in the two images is obtained, the background pixel portion disappears and only the pixels of the portion corresponding to the structured light remain, so that only the image of the structured light component is obtained.

However, when the laser structured light is not bright, the values of the structured light pixels in the structured light image obtained by the difference imaging method may not be larger than those of the surrounding pixels. I can receive it.

The structured light difference image integration algorithm accumulates the structured light difference image sequence for each pixel over time, thereby obtaining a structured light image that is robust to ambient light noise by highlighting the structured light pixels relative to the surrounding pixels. The difference image integration algorithm is suitable for hardware implementation because it consists of very simple pixel-by-pixel operations. When the image acquisition and differential image integration algorithm synchronized with the structured light modulation is implemented by field-programmable gate array (FPGA) hardware, structured light image processing hardware that can be processed very quickly and efficiently can be implemented.

7 illustrates a block diagram of the structured light difference image integration hardware of the analyzer 150.

According to FIG. 7, an image signal is received through a camera interface module 730. The input signal is analyzed by the FPGA 720 and a distance value is calculated by a processor 740 such as a microprocessor.

M1 710, M2 711, M3 712, and M4 713 mean an image memory. In a filed-progrmmable gate array (FPGA) 720, a differential integral image is obtained using the image memory. When the structure light is repeatedly turned on and off, the image with the structure light turned on in the input images may be represented as an even number image, that is, I (0), I (2), etc. The image in the off state can be represented by an odd number image, i.e., I (1), I (3).

First, the image, I (0), is obtained and stored in the memory M1 while the structured light is turned on.

Next, take the image, I (1), with the structure light off and subtract this value from the image on M1 (M1-I (1) = I (0) -I (1)). To the M2 memory again.

Then we take the structure light again, I (2) and store it in M1,

OFF image of structured light, take I (3) and subtract it from the image in M1 memory (M1-I (3) = I (2) -I (3)), then take this value from M2 memory , I.e., I (0) -I (1) and store in M3. In M3, an image of I (2) -I (3) + I (0) -I (1) remains. In other words, an image integrating two difference images remains in M3.

Take the structure light again, I (4) and store it in M1,

OFF image of structured light, take I (5) and subtract it from the image in M1 memory (M1-I (5) = I (4) -I (5)), this value in M3 memory I.e., I (2) -I (3) + I (0) -I (1) and store in M2. In M2, an image of I (4) -I (5) + I (2) -I (3) + I (0) -I (1) remains. In other words, an image integrating three difference images remains.

Then we take the structured light on, I (6) and store it in M1,

OFF image of structured light, take I (7) and subtract it from the image in M1 memory (M1-I (7) = I (6) -I (7)), this value in M2 memory In other words, I (4) -I (5) + I (2) -I (3) + I (0) -I (1) and store in M3. In M3, images of I (6) -I (7) + I (4) -I (5) + I (2) -I (3) + I (0) -I (1) remain. In other words, an image integrating four difference images remains.

Then we take the structured light on, I (8) and store it in M1,

OFF image of structured light, take I (9) and subtract it from the image in M1 memory (M1-I (9) = I (8) -I (9)), this value in M3 memory That is, I (6) -I (7) + I (4) -I (5) + I (2) -I (3) + I (0) -I (1) and store in M4. M4 contains I (8) -I (9) + I (6) -I (7) + I (4) -I (5) + I (2) -I (3) + I (0) -I (1 ) The image will remain. In other words, an image integrating five difference images is left.

Through this process, if the five difference images are integrated, a value sufficiently different from noise can be obtained. Differential image integration can be implemented using M4 (1), M4 (2) and two M4 memories.

When the image data, which is integrated five times in M4 (1), is stored, the distance value is calculated using the processor 740. During the distance calculation process in the processor 740, the FPGA 720 acquires a difference integral image using M4 (2). When the image integrally stored in M4 (2) is stored and the distance value is calculated by the processor 740, the image data integrally stored again is stored using M4 (1). By repeating this process, the differential integral image acquisition and the distance calculation can be processed in parallel.

The operation of acquiring the difference image and the operation of integrating the difference image are very simple operations performed on a pixel-by-pixel basis, and also parallel to the difference image acquisition process and the difference image integration process in an FPGA (field-programmable gate array) 720 parallel memory structure. Because of this, it is possible to perform image acquisition and difference integration operation very quickly and efficiently as a whole.

If you want to apply the above image processing algorithm with a general microprocessor or computer, the whole process will be very slow because you cannot process the image acquisition process and the differential integration process in parallel and also have serial memory. Alternatively, the hardware of the parallel memory structure is more effective.

In the above example, the methods are described based on a flowchart as a series of steps or blocks, but the present invention is not limited to the order of steps, and any steps may occur in a different order or simultaneously from other steps as described above. . In addition, those skilled in the art will appreciate that the steps shown in the flowcharts are not exclusive and that other steps may be included or one or more steps in the flowcharts may be deleted without affecting the scope of the present invention.

The above-described embodiments include examples of various aspects. While it is not possible to describe every possible combination for expressing various aspects, one of ordinary skill in the art will recognize that other combinations are possible. Accordingly, it is intended that the invention include all alternatives, modifications and variations that fall within the scope of the following claims.

Claims (11)

An irradiation unit for irradiating the laser structured light in all directions;
And a convex mirror and a camera, and repeatedly acquire a background image from the convex mirror using the camera, and repeatedly acquire a distorted image incident on the convex mirror after the laser structure light is reflected by an object. Measuring unit; And
Omnidirectional distance measurement from the repeatedly obtained background image and the distorted image to obtain a trajectory of the laser structured light reflected by the difference image integration algorithm, and to calculate the object distance between the camera and the object Device. The method of claim 1,
The irradiation unit includes a plurality of dispersion lenses for dispersing the point laser into a plurality of linear lasers,
The plurality of linear lasers distributed in each of the plurality of dispersion lenses are irradiated at an angle between 70 and 120 degrees,
Some or all of the plurality of linear lasers dispersed in each of the plurality of dispersion lenses are arranged to have different angles,
And forming the laser structured light irradiated in all directions with the linear lasers irradiated from the plurality of dispersion lenses. The method of claim 1,
The irradiator comprises a circular laser source,
And a circular laser emitted from the circular laser source to the convex mirror to form the laser structured light irradiated in all directions. The method of claim 1,
The convex mirror is a conical mirror,
The analysis unit

Omnidirectional distance measuring device for finding object distance through
Where the object distance between the camera and the object

Height of convex mirror

, Height of camera pinhole

Structured light measurement angle

And cone angle

Is a function of.

ego,

Wow

Can be obtained as

The method of claim 1,
And the analyzing unit accumulatively calculates a difference between pixel values of the background image and the pixel values of the distorted image, which are repeatedly obtained, to obtain a trajectory of the reflected laser structured light.
Irradiating the laser structured light in all directions,
Acquiring a background image, which is an image before irradiating the laser structure light, by using a camera;
Acquiring a distorted image which is an image after the laser structure light is reflected by an object by using the camera;
Acquiring a plurality of background images and a plurality of distortion images by repeatedly obtaining the background image and the distortion image;
Acquiring a trajectory of the reflected laser structured light by integrating the difference between the plurality of background images and the plurality of distorted images; And
And calculating an object distance between the camera and the object from the trajectory of the reflected laser structured light.
The method according to claim 6,
Scatter the point laser into a plurality of linear lasers,
A part or all of the plurality of linear lasers are arranged to have different angles to form the laser structured light irradiated in all directions.
The method according to claim 6,
And a circular laser is irradiated to the convex mirror to form the laser structured light irradiated in all directions.
The method according to claim 6,
And cumulatively calculating a difference between each of the plurality of background images and the plurality of distorted images to obtain a trajectory of the reflected laser structured light.
The method of claim 9,
And cumulatively calculating a difference between pixel values of the plurality of background images and pixel values of each of the plurality of distorted images to obtain a trajectory of the reflected laser structured light.
A camera interface module for repeatedly acquiring a background image which is an image before irradiating laser structure light and a distortion image which is an image after the laser structure light is reflected by an object;
A plurality of memories configured to store pixel values of the background image and pixel values of the distorted image repeatedly acquired by using the camera interface module;
A filed-programmable gate array (FPGA) for integrating a difference between a pixel value of the background image and a pixel value of the distorted image using the plurality of memories;
Acquiring the trajectory of the laser structured light reflected from the difference between the pixel value of the background image and the pixel value of the distorted image integrated in the FPGA, and using the trajectory of the reflected laser structured light, the camera interface module and the object. Differential image integration algorithm processing hardware comprising a processor for calculating the object distance between.

Images