KR101207462B1 - System for sensor revision used image information and distant-angle information - Google Patents

Abstract

PURPOSE: A sensor compensation system using video information and distance-angular information is provide to restructure the coordinates of the real world by using distance information obtained through output of a distance-angle sensor, and to automatically calculate a physical arrangement error in the distance-angle sensor and an image sensor. CONSTITUTION: An image sensor unit clearly recognizes an object from an input image. A distance-angle sensor unit extracts a rotation angle of the object by a laser. An information processing unit extracts the real location of the object. The information processing unit extracts the rotation angle of the object according to the extracted real location. The information processing unit outputs an error compensation signal. The error compensation signal directs the difference between the extracted rotation angles within a predetermined range.

Description

Sensor correction system using image information and distance-angle information {System for sensor revision used image information and distant-angle information}

The present invention relates to a method of correcting an image sensor or a distance-angle sensor to be located on the same line by using image information obtained from an image sensor and distance-angle information obtained from a distance-angle sensor.

An image-based collision prevention and avoidance system mounted on a moving object is generally composed of an image acquisition unit, an image detection unit, and an alarm device that alerts whether a detected object is collided with the acquired image according to a detection result.

The traveling moving body generates vibrations (shakes) by various external factors such as discontinuous road conditions, mechanical vibrations, acceleration and deceleration of the driving device, and the like. As a result, images obtained from a camera mounted inside the vehicle also generate similar shakes (displacements) due to these vibrations, which prevents stable tracking of objects detected by these elements.

In general, a vehicle collects necessary data by using a data acquisition system (DAQ). That is, the data acquisition / display system performs a performance measurement of a distance-angle detection sensor such as a radar, a performance measurement of a distance-angle detection sensor and an image sensor composite device, and a convergence processing, such as a vehicle or a robot. The image sensor and the distance-angle detection sensor are mounted on the moving body at the same time. The data acquisition / display system is a system for acquiring distance-angle information obtained from an installed distance-angle sensor and image information (position of an object) obtained from an image sensor, and displaying the acquired information on one screen. .

The above-described data acquisition system precisely acquires distance-angle and image information, and needs detailed information on synchronization or correction of physical location information of two sensors in order to display the acquired information on the screen.

1 illustrates a mounting position of a distance-angle sensor and an image sensor mounted on a conventional vehicle. As shown in FIG. 1, the position at which the distance-angle sensor is mounted is different from the position at which the image sensor is mounted. In general, the distance-angle sensor is mounted near the bumper, which is the front end of the vehicle, and the image sensor is installed near the room mirror. Therefore, since the physical positions of the two sensors are different from each other, when the information obtained from the two sensors is displayed on one screen, the error due to the mounting position should be corrected. In particular, the error in question is particularly problematic in the X-axis direction as shown in FIG.

In other words, image information obtained by simply perspective projection and distance-angle information of an object detected by a distance-angle sensor such as a radar cannot be simply 1: 1 matched.

Therefore, the target that can be detected by the image sensor and the distance-angle detection sensor is placed at a specific position within the detection area, and the information error of the image sensor and the distance-angle detection sensor detected therefrom is measured directly, and then the physical installation error is adjusted. The error should be corrected.

The problem to be solved by the present invention proposes a method for correcting the image sensor and the distance-angle sensor mounted on the object to be located on the same line.

Another object of the present invention is to propose a method of correcting the image sensor and the distance-angle sensor on the same line by using the image information obtained from the image sensor and the distance-angle information obtained from the distance-angle sensor.

Another problem to be solved by the present invention is to automatically calibrate the image sensor and the distance-angle sensor by using the image information obtained from the image sensor and the distance-angle information obtained from the distance-angle sensor. Suggest.

To this end, the sensor correction system of the present invention recognizes an object from an input image, rotates the image sensor unit to extract the center of gravity of the recognized object, and the distance to the object and the front side using a laser in the horizontal direction. A distance-angle sensor unit for extracting the rotation angle θ1 of the object, an actual position of the object by using the center of gravity of the object provided from the image sensor unit and the distance of the object provided from the distance-angle sensor unit An error of extracting a rotation angle θ2 of the object rotated in the left and right direction with respect to the front of the image sensor unit by using the extracted actual position, and indicating that the difference between the extracted θ1 and θ2 is included within a set range And an information processor for outputting a correction signal.

To this end, the sensor correction system of the present invention recognizes an object from an input image, rotates the image sensor unit to extract the center of gravity of the recognized object, and the distance to the object and the front side using a laser in the horizontal direction. A distance-angle sensor unit for extracting the rotation angle θ1 of the object, an actual position of the object by using the center of gravity of the object provided from the image sensor unit and the distance of the object provided from the distance-angle sensor unit An extracting perspective projection unit, an angle information extractor extracting a rotation angle θ2 of the object rotated relative to the front of the image sensor unit by using the extracted actual position, and a range in which the difference between the extracted θ1 and θ2 is set A correction signal output unit for outputting an error correction signal indicating to be included within It should.

The present invention overcomes the problem that the user had to manually correct the position of the image sensor and the distance-angle sensor in the image and distance / angle simultaneous storage system, and obtains object information in the image obtained by applying image recognition technology. Using the distance information acquired through the output of the distance-angle sensor, the physical alignment error of the image sensor and the distance-angle sensor can be automatically calculated by reconstructing the coordinates of the real-world before the projection on the screen. Can be.

By using the method of calculating the physical alignment error as described above, the physical alignment error is corrected by using a distance-angle sensor or a transfer device of an image sensor installed in various moving or fixed devices, and precise image information and distance-angle information are obtained. It is possible to acquire at the same time.

1 illustrates a mounting position of a distance-angle sensor and an image sensor mounted on a conventional vehicle.
2 illustrates an example of displaying an image of a real world acquired through an image sensor according to an embodiment of the present invention on a screen by perspective perspective.
3 is a diagram illustrating a difference between an image projected by an image projection method and an image of a real world according to an exemplary embodiment.
4 is a block diagram showing the configuration of a sensor correction system according to an embodiment of the present invention.
5 illustrates an example of extracting an angle from an angle information extractor according to an exemplary embodiment of the present invention.

The foregoing and further aspects of the present invention will become more apparent through the preferred embodiments described with reference to the accompanying drawings. Hereinafter will be described in detail to enable those skilled in the art to easily understand and reproduce through this embodiment of the present invention.

2 illustrates an example of displaying an image of a real world acquired through an image sensor according to an embodiment of the present invention on a screen by perspective perspective. The screen displayed by the perspective view shows the vanishing point and the vanishing line. The image projected by the perspective view and displayed on the screen is different from the image of the real world.

3 is a diagram illustrating a difference between an image projected by an image projection method and an image of a real world according to an exemplary embodiment. Hereinafter, a process of determining a difference between an image projected by an image projection method and an image of the real world will be described with reference to FIG. 3.

In general, the three-dimensional coordinates of the real world are projected on a two-dimensional screen by the following formula. In other words, a particular object in the real world is reduced in position and size in inverse proportion to distance.

FIG. 3A illustrates an example in which an image acquired by an image sensor is projected onto a plane at a specific distance, and FIG. 3B illustrates a movement of a plane at a specific distance in a direction in which the image is located. An example is shown.

According to FIG. 3, the image acquired by the image sensor is projected onto a plane at distance f from the lens. f is the focal length of the camera and is usually provided by the image sensor. According to FIG. 3, an image of the real world is displayed on a plane at a specific distance as shown in Equation 1 below.

[Equation 1]

 f / Z = r / R

x / X = y / Y = z / Z

x = Xf / Z, y = Yf / Z, z = f

x, x, z: position coordinates of the object projected on the plane

X, Y, Z: Position coordinates of the object in the real world

f: focal length of camera

r: Distance from the camera lens to the object projected on the plane

R: Distance from the camera lens to the object in the real world

According to Equation 1, when x, y, and Z values other than the focal length of the camera are known, X, Y values of objects located in the real world are known. Hereinafter, a process of calculating X and Y values of an object located in the real world will be described.

4 is a block diagram showing the configuration of a sensor correction system according to an embodiment of the present invention. Hereinafter, a configuration of a sensor correction system according to an embodiment of the present invention will be described with reference to FIG. 4.

According to FIG. 4, the sensor correction system includes an image sensor unit, a distance-angle sensor unit, an information processor, and a sensor correction unit. The image sensor unit includes a camera, an object recognition unit, and an object center of gravity extractor, and the distance-angle sensor unit includes a radar signal transceiver, a radar signal processor, and an object information extractor. The information processing unit also includes a perspective projection processing unit, an angle information extraction unit, and a correction signal output unit. First, an operation performed by the image sensor unit will be described.

The camera 402 acquires an image input through the lens. The obtained image is transmitted to the object recognizer 404. The object recognizer 404 is a device capable of recognizing different objects according to the recognition learning parameter of the classification algorithm. The object recognition method uses a sliding window method that determines whether the object is a trained object by matching an image obtained by cutting the learned DB information by the window size from the image.

The object information finally recognized through the above process extracts the center coordinates of the object through the object center of gravity detection unit 406. Object center of gravity detection is the center of gravity of the final detected image (ie x, x) through a general center of gravity extraction method for finding center points based on the coordinates of each point of the outline of the object detected by the sliding windowing method. y coordinate) is extracted. As described above, the x and y coordinates of the image are obtained by using the center of gravity extraction method.

The radar signal transceiver 412 transmits the radar to the object and receives the reflected radar. The radar signal processor 414 receives the received radar and processes a signal according to a set program. The object information extracting unit 416 extracts an object from the signal processed by the radar signal processing unit, and extracts the distance and angle (direction) of the extracted object.

The distance from the image sensor, that is, the camera, to the object may be measured using the distance information extracted from the distance-angle sensor unit 410. As shown in FIG. 1, the horizontal mounting positions of the distance-angle sensor and the image sensor are not the same, but can be measured simply by using a general measuring tool.

The perspective projection processor 422 extracts X and Y coordinates of the real world of the object detected by the image sensor. X and Y coordinates of an object located in the real world may be extracted using distance information Z detected through the distance-angle sensor unit and coordinates (x, y) obtained through the image sensor unit.

As described above, the present invention obtains X and Y values in which an object moved on a two-dimensional plane is actually located using x and y information of an object extracted from an image sensor and a distance Z extracted from a distance-angle sensor.

The angle information extractor 424 extracts angle information of the center point of the object detected by the image sensor through X and Y of the real world extracted through the perspective projection process.

5 illustrates an example of extracting an angle from an angle information extractor according to an exemplary embodiment of the present invention. Hereinafter, a process of extracting an angle from the angle information extractor according to an embodiment of the present invention will be described with reference to FIG. 5.

The angle information extractor 424 calculates the rotation angle θ1 from the front of the image sensor by using the X and Y values which are actual positions of the received object.

The correction signal output unit 426 outputs the correction information by comparing the angle information extracted from the coordinate information of the real world through the perspective projection process with the angle information which is one of the output values of the distance-angle sensor unit 410. The output correction information is compared with the detection angle values θ1 and θ2 of the same object of the image sensor and the distance-angle sensor as shown in FIG. 5, and when the values are not the same or within an error range, the image sensor or the distance-angle A correction signal is output to move the position of the sensor.

As shown in FIG. 5, the angle value θ1 of the object detected through the image sensor unit 400 is larger than θ2, which is a detection value of the distance-speed sensor unit 410, and the movable sensor is an image sensor. In this case, the image sensor moves to the right. The correction signal output unit repeats the above-described process, and finally, when the output value is within zero (0) or the set threshold value (Thershold Value), the correction process through the correction signal output unit 426 is completed. If the movable sensor is a distance-angle sensor, the calibration procedure of the above-described process is reversed.

 When the angle value θ1 of the object detected by the image sensor 400 is smaller than θ2, which is the detection value of the distance-speed sensor 410, and the movable sensor is the image sensor, the image sensor moves to the left. Move. The correction signal output unit repeats the above-described process, and finally, when the output value is zero or less than a predetermined threshold value, the correction process through the correction signal output unit is completed.

The sensor corrector 430 moves the image sensor or the distance-angle sensor left or right according to the correction signal received from the correction signal output unit 426.

As described above, the present invention proposes a method of matching the X-axis (vehicle width direction) coordinates of the mounting position of the image sensor with the X-axis (vehicle width direction) coordinates of the mounting position of the distance-angle sensor.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the scope of the present invention .

400: image sensor 402: camera
404: object recognition unit 406: object center of gravity extraction unit
410: distance-angle sensor unit 412: radar signal transceiver
414: radar signal processing unit 416: object information extraction unit
420: information processing unit 422: perspective projection processing unit
424: angle information extraction unit 426: correction signal output unit
430: angle correction unit

Claims (8)

An image sensor unit recognizing an object from an input image and extracting a center of gravity of the recognized object;
A distance-angle sensor unit for extracting a rotation angle [theta] 1 of the object rotated in the horizontal direction based on the distance to the object and the front using a laser;
The actual position of the object is extracted using the center of gravity of the object provided from the image sensor unit and the distance of the object provided from the distance-angle sensor unit, and based on the front of the image sensor unit using the extracted actual position. And an information processor for extracting a rotation angle θ2 of the object rotated in the left and right directions, and outputting an error correction signal instructing the difference between the extracted θ1 and θ2 to fall within a set range. system.
The sensor correction system of claim 1, further comprising an angle correction unit configured to move at least one of the image sensor and the distance-angle sensor according to the error correction signal.
The method of claim 1, wherein the image sensor unit,
A camera for acquiring an image;
An object recognizing unit recognizing an object included in the image acquired from the camera;
And a center of gravity extraction unit for extracting the recognized center of gravity of the object.
The method of claim 3, wherein the distance-angle sensor unit,
A radar transceiver for transmitting a radar and receiving a reflected radar among the transmitted radars;
And an object extractor configured to extract the position and angle of the object using the received radar.
The method of claim 4, wherein the information processing unit,
A perspective projection performing unit extracting an actual position of the object by using the center of gravity of the object provided from the image sensor unit and the distance of the object provided from the distance-angle sensor unit;
An angle information extraction unit for extracting a rotation angle θ2 of the object rotated with respect to the front of the image sensor unit by using the extracted actual position;
And a correction signal output unit for outputting an error correction signal instructing the extracted difference between θ1 and θ2 to fall within a set range.
The method of claim 5, wherein the perspective projection performing unit,
The focal length of the previously stored camera, the x and y coordinate values projected on a plane in which the object input through the lens is located at a distance spaced from the lens by the focal length, and of the object provided from the distance-angle sensor unit. And a distance is obtained to obtain the actual position of the object.
An image sensor unit recognizing an object from an input image and extracting a center of gravity of the recognized object;
A distance-angle sensor unit for extracting a rotation angle [theta] 1 of the object rotated in the horizontal direction based on the distance to the object and the front using a laser;
A perspective projection performing unit extracting an actual position of the object by using the center of gravity of the object provided from the image sensor unit and the distance of the object provided from the distance-angle sensor unit;
An angle information extraction unit for extracting a rotation angle θ2 of the object rotated with respect to the front of the image sensor unit by using the extracted actual position;
And a correction signal output unit for outputting an error correction signal instructing the extracted difference between θ1 and θ2 to fall within a set range.
The method of claim 7, wherein the perspective projection performing unit,
The focal length of the previously stored camera, the x and y coordinate values projected on a plane in which the object input through the lens is located at a distance spaced from the lens by the focal length, and of the object provided from the distance-angle sensor unit. And a distance is obtained to obtain the actual position of the object.

Images