KR101531313B1 - Apparatus and method for object detection of vehicle floor - Google Patents

Abstract

Disclosed is an apparatus to detect a lower part of a vehicle comprising: a depth sensor to generate a depth image with respect to the lower part of a vehicle; a rotation module to rotate the depth sensor in a horizontal direction; and a control module to generate a first depth image corresponding to a vehicle lower part photographing image in absence of an object, and to detect an object by subtracting the first depth image from a second depth image corresponding to the vehicle lower part photographing image at an arbitrary view.

Description

[0001] APPARATUS AND METHOD FOR OBJECT DETECTION OF VEHICLE FLOOR [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for detecting an object under a vehicle, and more particularly to a method and apparatus for detecting an object by analyzing an image of a lower part of the vehicle, .

In recent years, accidents involving injuries caused by children being laid under the vehicle are increasing. These accidents usually occur when the driver starts driving without looking around the vehicle, even if the child is in a blind spot. That is, the main cause of such accidents is driver carelessness. In order to prevent such accidents, recent vehicles are equipped with front and rear side sensors for detecting a dead zone to warn the driver of an obstacle. However, such devices often fail to detect objects under the vehicle. Therefore, a method of detecting a dangerous object under the vehicle and informing the driver of the state of the under vehicle is requested.

It is an object of the present invention to provide a method and an apparatus for detecting an object under a vehicle. More particularly, it is an object of the present invention to provide a method and an apparatus for detecting an object by analyzing an image of a lower portion of a vehicle and discriminating whether or not the detected object is dangerous to provide an alarm to the driver.

According to one embodiment of the present invention, a vehicle lower detecting apparatus is provided. The apparatus comprising: a depth sensor for generating a depth image of a lower portion of the vehicle; A rotation module for horizontally rotating the depth sensor; A first depth image corresponding to a vehicle under-shooting image in a state in which there is no object is generated and a second depth image corresponding to a vehicle under-shooting image at an arbitrary time is subtracted from the first depth image, And a control module for detecting the control signal.

In accordance with another embodiment of the present disclosure, a method is provided for a vehicle under-detection device to detect an object under a vehicle. The method includes rotating a depth sensor in a horizontal direction to generate a first depth image corresponding to an under-vehicle image in an object-free state; Rotating the depth sensor in a horizontal direction to generate a second depth image corresponding to an under-vehicle image at an arbitrary time; And subtracting the second depth image and the first depth image to detect an object.

Embodiments of the present disclosure can help the user to know the situation of the under vehicle by accurately analyzing the condition of the under vehicle. In particular, the risk of an accident can be reduced by automatically detecting the existence of obstacles and dangerousness in the lower part of the vehicle and informing the driver.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram schematically illustrating the operation of a vehicle lower detecting apparatus according to an embodiment of the present invention; FIG.
2 is an image showing a difference method according to an embodiment of the present invention.
3 is a schematic view illustrating a filtering operation of the vehicle lower detecting apparatus according to the embodiment of the present invention.
4 is a diagram illustrating a filtering area according to an embodiment of the present invention.
5 is a view illustrating a direction angle measuring method according to an embodiment of the present invention.
6 is a block diagram of a vehicle lower detecting apparatus according to an embodiment of the present invention.
7 is a flowchart illustrating a method of detecting a vehicle lower portion according to an embodiment of the present invention.

It is noted that the technical terms used herein are used only to describe specific embodiments and are not intended to limit the invention. It is also to be understood that the technical terms used herein are to be interpreted in a sense generally understood by a person skilled in the art to which the present invention belongs, Should not be construed to mean, or be interpreted in an excessively reduced sense. Further, when a technical term used herein is an erroneous technical term that does not accurately express the spirit of the present invention, it should be understood that technical terms that can be understood by a person skilled in the art are replaced. In addition, the general terms used in the present invention should be interpreted according to a predefined or prior context, and should not be construed as being excessively reduced.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to like or similar elements throughout the several views, and redundant description thereof will be omitted. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. It is to be noted that the accompanying drawings are only for the purpose of facilitating understanding of the present invention, and should not be construed as limiting the scope of the present invention with reference to the accompanying drawings. The spirit of the present invention should be construed as extending to all modifications, equivalents, and alternatives in addition to the appended drawings.

1 is a conceptual diagram showing a vehicle lower detecting apparatus according to an embodiment of the present invention.

The detection of dangerous objects in the lower part of the vehicle means recognition of all objects in the area under the vehicle, and most of the conventional object detection systems were vision-based systems using RGB cameras. However, in frequently changing environments, object detection using RGB cameras often fails because of illumination changes. Recently, research on object detection using an infrared-based depth sensor which is not subjected to illumination interference has been increasing. The depth sensor extracts depth information from a single camera image to obtain distance information of a specific object, and combines the extracted information into a gray image. However, a depth sensor combined with a single camera is often insufficient to observe the entire area of the vehicle's bottom due to the limit of field of view (FOV).

Methods for combining multiple cameras or using special types of lenses (eg fisheye lenses) and mirrors (eg, aspherical mirrors) have been proposed for wide FOV observations. Although these methods have the advantage of observing a large area at a fixed point, image compensation is required to deal with additional image distortion, and irregular resolution lowers the reliability of observation. A method using a PTZ (Pan-Tilt-Zoom) camera can be applied for wide FOV observation. The PTZ camera is advantageous not only in unidirectional observation using a single camera using rotating parts but also in uniform resolution. However, since the PTZ-based observation system is mainly aimed at observing the fixed area, the background image-based subtraction method used by the system must be changed to match the characteristics of the moving vehicle.

Therefore, in this specification, we propose a technique to deal with a continuously changing background in order to apply the background subtraction method to a moving vehicle subarea. Also, in the present specification, a sub-vehicle detection device using a depth sensor, a rotation module and the like, which is resistant to a variable environment, is proposed. The depth sensor is used to acquire a depth map for object detection in various environments. The vehicle lower detection apparatus of the present invention uses a rotation module (e.g., a stepper motor) to acquire background images in all directions under the vehicle, and uses a background subtraction method for the real- Lt; / RTI > You can also use an external obstacle filtering algorithm to handle a continuously changing background.

The vehicle lower detecting apparatus according to an embodiment of the present invention may include a depth sensor, a rotation module, and a control module.

First, the operation of the vehicle lower detecting apparatus will be described with reference to FIG. 1. First, the vehicle lower detecting apparatus detects a full direction (without an object) using a depth map obtained from a depth sensor, And generates a background image.

At this time, the depth sensor may only perform a horizontal rotation operation due to the characteristics of the lower portion of the vehicle. That is, since the lower portion of the vehicle is not relatively large in vertical height, the rotation module can operate to obtain a depth image in the 360-degree direction only by turning clockwise. At this time, eight depth maps (submaps) may be acquired at a 45 degree photographing angle, and eight images may be combined into one background image.

Next, the vehicle lower detection apparatus compares a current depth image obtained in real time with a previously created background image, and detects an object using a difference method.

The vehicle lower detection apparatus can perform object extraction based on a background subtraction method. The background difference method is effective in a steady shooting condition and can obtain a clear background image. Since the depth sensor is a structured-light method that calculates and extracts depth information from the structural change of the infrared ray scanned on the object, the surface of the object is not uniform or the surface of the object reflects infrared rays Irregular holes may appear in the holes. In order to solve this problem, a morphological operation may be applied to the vehicle lower detecting apparatus. Erode operation and / or Dilate operation may also be applied to remove the isolated noise and merge adjacent pixels. Further, a labeling operation may be applied to remove all pixels except for the object to be detected.

2 is an image showing a difference method according to an embodiment of the present invention. 2 (a) is an original background image without an object to be detected, and Fig. 2 (b) is an image including an object. If the background is subtracted from the image including the object, the result as shown in FIG. 2 (c) is obtained. If a morphological operation and / or a labeling operation is applied to this, a clean image can be extracted as shown in FIG. 2 (d).

In conventional image difference-based object detection, the background image must always be the same as the previously generated image. But the background around the moving vehicle keeps changing. When a vehicle-related accident occurs mainly in an urban area, and the vehicle is usually parked on a flat ground, the depth image is independent of the texture, so the changing part in the background is another vehicle, obstacle at a predetermined height, pedestrian, and the like. In the embodiment of the present invention, in order to eliminate these unintended elements, the detection success rate is raised by removing (filtering) factors having a depth value larger than a specific value.

In the embodiment proposed here, an algorithm may be applied to define a filtering area and to remove objects outside the filtering area, in order to solve difficulties in application of the difference method due to the ever-changing background around the moving vehicle. Referring to FIG. 3, once an object (obstacle) is detected, the vehicle lower detecting apparatus calculates a center point of the object and determines the direction. Subsequently, the vehicle lower detecting device compares the distance to the detected object with the reference distance to determine whether the object is dangerous. (Only objects within the reference distance are judged to be dangerous.) The reference distance for each detection direction can be predefined together with the filtering area. The filtering region may be an elliptical region described by an elliptic equation. 4 (b), the filtering region is defined as an ellipse having a major axis of 5,000 mm and a minor axis of 2,800 mm. If the filtering region is defined as a rectangle, it is difficult to treat the distances in each direction as a mathematical relation. If the filtering region is defined as a circle, the region width becomes too large. Therefore, in this embodiment, the filtering region is defined as an ellipse.

The vehicle lower detection device first calculates the center point of the detected object to determine the direction of the detected object. The center point may be calculated as an average of the number of pixels occupied by the detected object in the image, and the following equation may be used in this process.

Where x 'and y' are the x and y coordinates of the center point, A is the total number of pixels of the detected object, and B [i, j] is the pixel coordinates of the detected object.

In the next step, the vehicle lower detecting device determines the direction of the detected object by correlating with the position of the depth sensor. For example, as shown in FIG. 5 (b), the horizontal distance between the center point of the object and the center point of the image is calculated in the captured image, and the calculated distance value and the known distance Calculate the direction angle (θ) of the detected object by applying the proportional formula using the horizontal viewing angle of the depth sensor. In FIG. 5 (b), the horizontal distance between the center point of the object and the center line of the image is 75 pixels, the width of the image is 320 pixels, and the viewing angle is 57 degrees. 5A, the sensor direction when the image is captured is A on the reference line, and if the detected object OA is on the left side of the image center line x, the final direction angle? 28-x).

When the position (direction) of the detected object is calculated, the distance to the detected object can be calculated. First, the distance between the directional line from the depth sensor to the object to be detected and the intersection S of the boundary line of the filtering area can be calculated using the elliptic equation. The mathematical formulas used in this process are as follows.

Where y is the distance from the depth sensor to the object to be detected and the intersection S of the boundary of the filtering region, m is the long axis distance of S, n is the short axis distance of S, B is the short axis of the ellipse constituting the filtering region, and? Is the direction angle of the detected object.

If the distance (depth value) to the center point of the detected object is larger than the y obtained as described above, the detected object is determined to be outside the filtering area and can be filtered out.

The above-described apparatus and method for detecting a vehicle lower object can perform effective object detection in both day / night and moving / stop situations.

FIG. 6 is a block diagram illustrating a vehicle lower detection apparatus according to an embodiment of the present invention.

The vehicle lower detection apparatus 100 may include a depth sensor 101, a rotation module 102, and a control module 103. Further, the vehicle lower detecting apparatus 100 may further include various elements of hardware such as a memory, a display module, and a user interface.

The vehicle lower detecting apparatus 100 performs the vehicle lower detecting method described with reference to FIGS. That is, the vehicle lower detection apparatus 100 acquires a background image in all directions under the vehicle using a rotation module (e.g., a stepper motor) and generates a background subtraction method for a real-time input image Use to detect objects. You can also use an external obstacle filtering algorithm to handle a continuously changing background.

The depth sensor 101 generates a depth image of the lower portion of the vehicle, and the rotation module 102 rotates the depth sensor in a horizontal direction.

The control module 103 controls the first depth image (e.g., depth map) and the second depth image (depth map) corresponding to the under vehicle image at an arbitrary time, (Eg depth map, depth map). The first depth image and the second depth image may be formed by combining depth images captured in a plurality of directions defined according to the viewing angle of the depth sensor. For example, if the viewing angle of the depth sensor is 57 degrees, eight depth maps (submaps) may be obtained at intervals of 45 degrees, and eight depth maps may be combined into a single background image.

The control module 103 subtracts the first depth image and the second depth image to detect an object. At this time, the control module 103 may apply a morphological operation for the process. Erode operation and / or Dilate operation may also be applied to remove isolated noise and merge adjacent pixels. Further, a labeling operation may be applied to remove all pixels except the object to be detected.

The control module 103 may perform an algorithm to define a filtering area and to remove objects outside the filtering area to resolve difficulties in application of the difference method due to the ever-changing background around the moving vehicle.

For example, the control module 103 may calculate the center point and the direction angle of the detected object, determine whether the detected object is located within the reference distance based on the calculated center point and the direction angle, It can be determined that the detected object is not a dangerous object.

The center point may be calculated based on the number of pixels occupied by the detected object in the second image. Wherein the angle of incidence is obtained by converting the horizontal distance between the center point and the centerline of the second image into an angle in proportion to the horizontal viewing angle of the second image and by adding or subtracting the converted angle from the angle of view of the second image Can be calculated.

Meanwhile, the reference distance may be defined as an elliptical shape. In this case, the control module 103 may calculate the reference distance according to the following equation.

Here, y is a reference distance,? Is a directional angle of the detected object, a is a major axis length of the ellipse, and b is a minor axis length of the ellipse.

If the distance to the detected object is larger than the reference distance y obtained as described above, the control module 103 can determine that the detected object is outside the filtering area and can filter out the detected object, If it is determined that the object is within the reference distance, an alarm can be sent to the driver.

7 is a flowchart illustrating a method of detecting a vehicle lower portion according to an embodiment of the present invention.

The above method can be performed by the vehicle lower detecting apparatus described in Figs. Meanwhile, the vehicle lower detecting apparatus obtains a background image in all directions under the vehicle using a rotation module (e.g., a stepper motor), and uses a background subtraction method for a real- Lt; / RTI > You can also use an external obstacle filtering algorithm to handle a continuously changing background.

The vehicle lower detecting apparatus may generate a first depth image corresponding to the vehicle lower imaging image in the absence of an object (S110). The first depth image may be generated by horizontally rotating the depth sensor.

Next, the vehicle lower detection apparatus may generate a second depth image corresponding to the vehicle lower image at an arbitrary time (S120). The second depth image may be generated by rotating the depth sensor horizontally.

The first depth image and the second depth image may be formed by combining depth images captured in a plurality of directions defined according to the viewing angle of the depth sensor. For example, if the viewing angle of the depth sensor is 57 degrees, eight depth maps (submaps) may be obtained at intervals of 45 degrees, and eight depth maps may be combined into a single background image.

The vehicle lower detecting apparatus may detect an object by subtracting the second depth image and the first depth image (S130). At this time, the vehicle lower detecting apparatus may apply a morphological operation for the process. Erode operation and / or Dilate operation may also be applied to remove isolated noise and merge adjacent pixels. Further, a labeling operation may be applied to remove all pixels except the object to be detected.

The under vehicle detection device may perform an algorithm to define a filtering area and to remove an object outside the filtering area to solve a difficulty in application of the difference method due to the continuously changing background around the moving vehicle.

For example, when the detected object is located outside the reference distance, the vehicle lower detecting apparatus may determine that the detected object is not a dangerous object (S140).

The center point may be calculated based on the number of pixels occupied by the detected object in the second image. Wherein the angle of incidence is obtained by converting the horizontal distance between the center point and the centerline of the second image into an angle in proportion to the horizontal viewing angle of the second image and by adding or subtracting the converted angle from the angle of view of the second image Can be calculated.

Meanwhile, the reference distance may be defined as an elliptical shape. In this case, the control module 103 may calculate the reference distance according to the following equation.

Here, y is a reference distance,? Is a directional angle of the detected object, a is a major axis length of the ellipse, and b is a minor axis length of the ellipse.

If the distance to the center of the detection object is larger than the reference distance y obtained as described above, the detection device may determine that the detection object is located outside the filtering area, If it is determined that the detected object is within the reference distance, an alarm can be sent to the driver.

On the other hand, the method of the present specification may be embodied as a computer-readable recording medium including instructions for performing the respective steps described in FIG. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that affects the machine readable propagation type signal, or a combination of one or more of the foregoing.

Implementations of the functional operations and the subject matter described herein may be implemented in digital electronic circuitry, or may be implemented in computer software, firmware, or hardware, including the structures disclosed herein, and structural equivalents thereof, It can be implemented. Implementations of the subject matter described herein may be implemented as one or more computer program products, i. E. One or more modules relating to computer program instructions encoded on a type of program storage medium for execution by, or control of, the operation of the processing system Can be implemented.

A computer program (also known as a program, software, software application, script or code) may be written in any form of programming language, including compiled or interpreted language, a priori or procedural language, Components, subroutines, or other units suitable for use in a computer environment. A computer program does not necessarily correspond to a file in the file system. The program may be stored in a single file provided to the requested program, or in multiple interactive files (e.g., a file storing one or more modules, subprograms, or portions of code) (E.g., one or more scripts stored in a markup language document). A computer program may be deployed to run on multiple computers or on one computer, located on a single site or distributed across multiple sites and interconnected by a communications network.

On the other hand, computer readable media suitable for storing computer program instructions and data include semiconductor memory devices such as, for example, EPROM, EEPROM and flash memory devices, such as magnetic disks such as internal hard disks or external disks, Non-volatile memory, media and memory devices, including ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuits.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

100: vehicle lower detection device
101: Depth sensor
102: rotation module
103: Control module

Claims (13)

A depth sensor for generating a depth image of a lower portion of the vehicle;
A rotation module for horizontally rotating the depth sensor;
Acquiring a first depth image corresponding to the under-vehicle image captured in the absence of an object,
And a control module for subtracting the first depth image and the second depth image corresponding to the under-vehicle shot image at an arbitrary time to detect an object,
The control module
Calculating a center point and a direction angle of the detected object, determining whether the detected object is located within a reference distance based on the calculated center point and a direction angle,
And determines that the detected object is not a dangerous object when the detected object is located outside the reference distance. The method according to claim 1,
Wherein the first depth image and the second depth image are composed of a combination of depth images photographed in a plurality of directions defined corresponding to a viewing angle of the depth sensor. delete The method according to claim 1,
The center point
The detected object is calculated based on the number of pixels occupied in the second image,
The direction angle
Converting the horizontal distance between the center point and the center line of the second image into an angle according to a proportional relationship with a horizontal viewing angle of the second image,
And the calculated angle is calculated by adding or subtracting the converted angle from the photographing angle of the second image. The method according to claim 1,
Wherein the reference distance is defined as an elliptical shape. 6. The method of claim 5,
The control module
The reference distance is calculated according to the following equation,

Wherein y is a reference distance,? Is a direction angle of the detected object, a is a major axis length of the ellipse, and b is a minor axis length of the ellipse. CLAIMS 1. A method for detecting an object below a vehicle,
Rotating a depth sensor in a horizontal direction to generate a first depth image corresponding to an under-vehicle shot image in the absence of an object;
Rotating the depth sensor in a horizontal direction to generate a second depth image corresponding to an under-vehicle image at an arbitrary time;
And subtracting the second depth image and the first depth image to detect an object,
The method of claim 1,
Calculating a center point and a direction angle of the detected object;
Determining whether the detected object is located within a reference distance based on the calculated center point and direction angle; And
And determining that the detected object is not a dangerous object when the detected object is located outside the reference distance. 8. The method of claim 7,
Wherein the first depth image and the second depth image are formed by combining depth images photographed in a plurality of directions defined corresponding to a viewing angle of the depth sensor. delete 8. The method of claim 7,
The center point
The detected object is calculated based on the number of pixels occupied in the second image,
The direction angle
Converting the horizontal distance between the center point and the center line of the second image into an angle according to a proportional relationship with a horizontal viewing angle of the second image,
And the calculated angle is calculated by adding or subtracting from the angle of view of the second image. 8. The method of claim 7,
Wherein the reference distance is defined as an elliptical shape. 12. The method of claim 11,
The reference distance is calculated according to the following equation,

Wherein y is a reference distance,? Is a direction angle of the detected object, a is a major axis length of the ellipse, and b is a minor axis length of the ellipse. There is provided a computer readable medium having instructions for an under vehicle detection device to detect an object under the vehicle,
Rotating a depth sensor in a horizontal direction to generate a first depth image corresponding to an under-vehicle shot image in the absence of an object;
Rotating the depth sensor in a horizontal direction to generate a second depth image corresponding to an under-vehicle image at an arbitrary time;
And subtracting the second depth image and the first depth image to detect an object,
The method of claim 1,
Calculating a center point and a direction angle of the detected object;
Determining whether the detected object is located within a reference distance based on the calculated center point and direction angle; And
And determining that the detected object is not a dangerous object when the detected object is located outside the reference distance.

Images