KR20170124310A - System and method for recognizing obstacle based on image - Google Patents

Abstract

According to an embodiment of the present invention, a system for recognizing an obstacle can comprise: an image acquisition unit acquiring an image outside a vehicle from a camera mounted in the vehicle; a vehicle speed acquisition unit acquiring the speed of the vehicle; and a control unit generating a plurality of difference frames composed of a difference value between a current frame of the image and each of a plurality of previous frames, generating an overlapped frame by enabling the difference frames to be overlapped, and recognizing an obstacle from the overlapped frame when the vehicle speed is lower than or equal to a predetermined speed.

Description

Technical Field [0001] The present invention relates to an image-based obstacle recognition system,

The present invention relates to an image-based obstacle recognition system and method.

There are various types of obstacles around the vehicle, and various ways of recognizing the obstacles have been proposed to be protected from the risk of such potential collision.

Although there is a method of using an image, an ultrasonic wave, a laser or the like to recognize an obstacle, a method of using an image is preferable as a method for a driver to directly identify an obstacle.

Among the methods of recognizing obstacles using the image, there is a method of applying the behavior information of the vehicle to the continuous change of the image by using the wide angle fisheye camera and estimating the presence of the obstacle in the area where the change and the other changes occur.

This method removes some of the changes in the image in consideration of the vehicle behavior, that is, the traveling speed of the vehicle, and recognizes the remaining portions as obstacles. In order to recognize the obstacles, the optical flow, the inverse perspective mapping ) Must be performed.

However, the conventional method requires a lot of computation steps to insure that the real time property is not guaranteed. When the speed of the vehicle, which is a judgment criterion, is slow, there is a problem that the change in the image is insufficient and the performance of recognizing the obstacle is poor.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to improve the obstacle recognition performance and reduce unnecessary operations in an image-based obstacle recognition method.

It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are not intended to limit the invention to the precise form disclosed. It can be understood.

An obstacle recognition system related to an example of the present invention for realizing the above-mentioned object includes an image acquisition unit for acquiring an image of the outside of the vehicle from a camera mounted on the vehicle, a vehicle speed acquiring unit for acquiring the speed of the vehicle, Generating a plurality of differential frames constituted by differential values between a current frame of the image and each of a plurality of previous frames and generating an overlapping frame by overlapping the plurality of differential frames, And a control unit for recognizing the image.

The control unit may correct the spatial position of the plurality of previous frames using the speed of the vehicle, and may generate the difference frame using the corrected plurality of previous frames.

When the speed of the vehicle is zero or the speed of the vehicle is higher than the predetermined speed, the control unit compares the current frame with at least one of the plurality of previous frames without generating the overlapping frame, The obstacle can be recognized.

In addition, when the camera is a fisheye lens camera, the controller can correct the distortion of the image using the distortion rate of the fisheye lens camera.

In addition, when the speed of the vehicle is 0, the control unit may not correct the distortion of the image.

Embodiments of the present invention can improve the obstacle recognition performance and reduce unnecessary operations in an image-based obstacle recognition method.

Also, the obstacle can be recognized in real time by using the low performance controller by applying the embodiment of the present invention.

It should be understood, however, that the effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those skilled in the art to which the present invention belongs It will be possible.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate preferred embodiments of the invention and, together with the description, serve to further the understanding of the technical idea of the invention, It should not be construed as limited.
1 is a block diagram of an obstacle recognition system according to an embodiment of the present invention.
FIG. 2A is an image taken by a fisheye lens camera according to an embodiment of the present invention, and FIG. 2B is an image obtained by correcting distortion of an image taken by a fisheye lens camera according to an embodiment of the present invention.
FIG. 3A shows a differential frame when the vehicle speed is 5 km / h according to an embodiment of the present invention, FIG. 3B shows a differential frame when the vehicle speed is 1 km / h according to an embodiment of the present invention .
FIG. 4A is a current frame of an image according to an embodiment of the present invention, and FIGS. 4B to 4D are previous frames of an image arranged in a reverse order of time according to an embodiment of the present invention.
FIG. 5A is a current frame of an image according to an embodiment of the present invention, and FIGS. 4B to 5D are previous frames of an image in which spatial positions are corrected in a reverse order of time according to an embodiment of the present invention.
6A to 6C are differential frames derived from the current frame and the previous frame of the image, respectively, according to an embodiment of the present invention.
7 is an overlapping frame according to an embodiment of the present invention.
8 is a flowchart illustrating an obstacle recognition method according to an embodiment of the present invention.
9 is a block diagram illustrating a computing system that implements an obstacle recognition method in accordance with an embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. It should be noted that, in adding reference numerals to the constituent elements of the drawings, the same constituent elements are denoted by the same reference numerals whenever possible, even if they are shown in different drawings. In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the difference that the embodiments of the present invention are not conclusive.

In describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. Also, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be interpreted in an ideal or overly formal sense unless explicitly defined in the present application Do not.

1 is a block diagram of an obstacle recognition system according to an embodiment of the present invention.

FIG. 2A is an image taken by a fisheye lens camera according to an embodiment of the present invention, and FIG. 2B is an image obtained by correcting distortion of an image taken by a fisheye lens camera according to an embodiment of the present invention.

FIG. 3A shows a differential frame when the vehicle speed is 5 km / h according to an embodiment of the present invention, FIG. 3B shows a differential frame when the vehicle speed is 1 km / h according to an embodiment of the present invention .

FIG. 4A is a current frame of an image according to an embodiment of the present invention, and FIGS. 4B to 4D are previous frames of an image arranged in a reverse order of time according to an embodiment of the present invention.

FIG. 5A is a current frame of an image according to an embodiment of the present invention, and FIGS. 4B to 5D are previous frames of an image in which spatial positions are corrected in a reverse order of time according to an embodiment of the present invention.

6A to 6C are differential frames derived from the current frame and the previous frame of the image, respectively, according to an embodiment of the present invention.

7 is an overlapping frame according to an embodiment of the present invention.

Referring to FIG. 1, the obstacle recognition system according to the present invention may include an image acquisition unit 100, a vehicle speed acquisition unit 200, and a control unit 300.

However, the components shown in Fig. 1 are not essential, and an obstacle recognition system having components having more or fewer components may be implemented.

The image acquiring unit 100 acquires an image outside the vehicle.

The image acquiring unit 100 may be a camera for photographing the outside of the vehicle or an image receiving unit for acquiring an image from a camera for photographing the outside of the vehicle.

Also, the camera may be a fisheye lens camera for photographing the omnidirectional image, and the image obtained from the fisheye lens camera may be a distorted image since it is a wide-angle image obtained with one lens.

The vehicle speed acquiring section 200 is a configuration for acquiring the speed of the vehicle, and can acquire not only the speed of the vehicle but also information on the direction in which the vehicle is moving. For example, the vehicle speed obtaining section 200 may be OBD (On-Board Diagnostics).

The control unit 300 recognizes an obstacle from the image acquired by the image acquiring unit 100 and can perform an operation according to the speed of the vehicle to recognize an obstacle from a plurality of frames of the image.

Hereinafter, a method by which the controller 300 recognizes an obstacle will be described in detail with reference to FIGS. 2A to 7. FIG.

If the acquired image is an image acquired by a fisheye lens camera, the controller 300 can correct the image distortion using the fisheye ratio of the fisheye lens camera. In order to apply the vehicle behavior information to the continuous input image change, it is necessary to match the actual distance information with the image information such as IPM (Inverse Perspective Mapping).

That is, as shown in FIG. 2A, the image obtained by the fisheye lens camera is distorted as compared with the actual image. The control unit 300 can derive the corrected image as shown in FIG. 2B by performing an operation method such as IPM on the distorted image.

However, if the acquired image is a non-distorted image, there is no need to perform an operation to perform correction separately.

In addition, even when the vehicle speed is zero, the distortion of the image may not be corrected. This is because it is not necessary to perform an operation of applying the vehicle behavior information to the image when the vehicle is stopped and only the obstacle moving through the comparison of the plurality of frames of the image is recognized.

The control unit 300 may determine whether the vehicle speed is slower than the predetermined speed Vth, and may perform an operation of enhancing the characteristics of the obstacle when the speed of the vehicle is equal to or lower than a predetermined speed.

As shown in FIG. 3A, in the differential frame derived from the image frame obtained from the vehicle moving at the speed of 5 km / h, the characteristics of the obstacle are clearly revealed. However, as shown in FIG. 3B, The feature of the obstacle is not clearly revealed in the differential frame derived by using the difference frame.

That is, when the speed of the vehicle is high, the size or position change of the obstacle between adjacent frames of the image becomes conspicuous, but when the vehicle speed is slow, the motion change of the obstacle represented in the adjacent frame is insignificant.

First, the controller 300 may extract a current frame and a plurality of previous frames from an image to perform an operation for enhancing a feature of the obstacle.

FIG. 4A shows the current frame in the image, and FIGS. 4B to 4D are frames previously taken temporally with respect to the current frame.

Since the vehicle moves slowly but the speed is not zero, the position of the fixed obstacle changes by the moving distance of the vehicle in the image.

Referring to FIGS. 4A to 4D, the position of the obstacle rises gradually with respect to the red line from FIG. 4A to FIG. 4D. 4A to 4D, the vehicle approaches the obstacle since FIG. 4D is the frame that is photographed first in time.

The control unit 300 extracts a current frame and a plurality of previous frames and then generates a difference frame between the current frame and each of the plurality of previous frames using a speed of the vehicle according to an embodiment of the present invention, Can be corrected.

That is, a plurality of previous frames are corrected by applying the behavior information of the vehicle, and the position of the obstacle present in each of the plurality of previous frames is corrected to be the same as the position of the obstacle present in the current frame.

FIGS. 5B to 5D illustrate a frame in which the spatial positions of a plurality of previous frames are corrected according to an embodiment of the present invention, and the positions of the obstacles indicated in the plurality of frames are the same with respect to the red line.

The control unit 300 can generate a plurality of differential frames each composed of a difference value between each of a plurality of previous frames whose spatial positions are corrected and a current frame.

That is, a frame in which only the difference between the current frame and the previous frame is shown is generated.

6A is a differential frame derived from FIGS. 5A and 5B, FIG. 6B is a differential frame derived from FIGS. 5A and 5C, and FIG. 6C is a differential frame derived from FIGS. 5A and 5D.

Referring to FIGS. 6A to 6C, it can be seen that the obstacle appears more clearly in FIG. 6C, which is a difference frame derived from FIG. 5A and FIG. 5D which are temporally further apart.

If the above spatial position is not corrected, the difference frame is derived from Figs. 4A and 4B, derived from Figs. 4A and 4C, and can be derived from Figs. 4A and 4D.

The control unit 300 can overlap the derived FIGs. 6A to 6C to generate the overlapping frame as shown in FIG.

Although the obstacle can be recognized in FIG. 6C, which is a difference frame derived from FIG. 5A and FIG. 5D which are further distant from each other in time, there may be a moving obstacle instead of a fixed obstacle in the vicinity of the vehicle. If they do, they will not be able to recognize fast-moving obstacles.

Therefore, when a plurality of differential frames are overlapped, an overlapping frame having a high temporal resolution is derived, and a fast moving obstacle can also appear in the overlapping frame.

The control unit 300 can recognize the obstacle from this overlapping frame.

On the other hand, when the speed of the vehicle exceeds the predetermined speed, the size or positional change of the obstacle between adjacent frames of the image is conspicuous as shown in FIG. 3A.

Therefore, the controller 300 can recognize an obstacle from a differential frame between adjacent frames of the image without performing the obstacle feature enhancing operation described with reference to FIG. 4A to FIG. Also, even when the vehicle speed is zero, the controller 300 can recognize an obstacle from a differential frame between neighboring frames of the image without performing the obstacle feature enhancing operation.

Hereinafter, a method for recognizing an obstacle will be described in detail based on the above-described configurations with reference to FIG.

8 is a flowchart illustrating an obstacle recognition method according to an embodiment of the present invention.

Referring to FIG. 8, an obstacle recognizing method according to an embodiment of the present invention includes an image acquiring step S100, a step S200 of determining whether the speed of the vehicle is greater than 0, A step S510 of generating an overlapping frame, a step S520 of generating a difference frame S520, and a step S520 of recognizing an obstacle, (S600).

Hereinafter, steps S100 to S600 will be described in detail with reference to FIGS. 1 to 7. FIG.

In step S100, the image obtaining unit 100 can obtain an image outside the vehicle. The image obtaining unit 100 can photograph the outside of the vehicle as a camera, and can acquire an image from a camera that photographs the outside of the vehicle.

Also, the obtained image may be an omnidirectional image captured by a fisheye lens camera, and the image captured by a fisheye lens camera may be a distorted image because it is a wide-angle image.

In step S200, the control unit 300 may determine whether the vehicle speed is greater than zero.

As described above, when the vehicle speed is 0, that is, when the vehicle is stopped, the control unit 300 can perform a step of generating a difference frame from an adjacent frame of an image to be described later, without performing an obstacle feature enhancing calculation .

If the vehicle speed is greater than zero, the control unit 300 can correct the image distortion in step S300.

In step S300, when the image obtained in step S100 is a distorted image obtained by a fisheye lens camera or the like, in order to apply the behavior information of the vehicle from the continuously input image change, And matching the actual distance information with the image information. That is, the image shown in FIG. 2A is corrected as shown in FIG. 2B.

At this time, the control unit 300 can correct the distortion of the image using the distortion rate of the camera that imaged the image.

In step S400, the control unit 300 may determine whether the vehicle speed is slower than the predetermined speed Vth.

If the vehicle speed is slower than the predetermined speed, the controller 300 may generate an overlapping frame by performing an obstacle feature enhancing operation as described with reference to FIGS. 4A to 7 (S510).

When the speed of the vehicle is faster than the preset speed, the characteristic of the obstacle is clearly revealed in the difference frame as shown in FIG. 3A, so that the control unit 300 does not perform any additional obstacle feature enhancing calculation, (S520). ≪ / RTI >

In step S600, the control unit 300 can recognize the obstacle from the overlapping frame or the difference frame.

9 is a block diagram illustrating a computing system that implements an obstacle recognition method in accordance with an embodiment of the present invention.

8, a computing system 1000 includes at least one processor 1100, a memory 1300, a user interface input device 1400, a user interface output device 1500, (1600), and a network interface (1700).

The processor 1100 may be a central processing unit (CPU) or a memory device 1300 and / or a semiconductor device that performs processing for instructions stored in the storage 1600. Memory 1300 and storage 1600 may include various types of volatile or non-volatile storage media. For example, the memory 1300 may include a ROM (Read Only Memory) and a RAM (Random Access Memory).

Thus, the steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by processor 1100, or in a combination of the two. The software module may reside in a storage medium (i.e., memory 1300 and / or storage 1600) such as a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, You may. An exemplary storage medium is coupled to the processor 1100, which can read information from, and write information to, the storage medium. Alternatively, the storage medium may be integral to the processor 1100. [ The processor and the storage medium may reside within an application specific integrated circuit (ASIC). The ASIC may reside within the user terminal. Alternatively, the processor and the storage medium may reside as discrete components in a user terminal.

The above-described obstacle recognition system and method are not limited to the configuration and method of the embodiments described above, but the embodiments may be modified such that all or some of the embodiments are selectively combined .

100:
200: vehicle speed obtaining section
300:
1000: Computing System
1100: Processor
1200: System bus
1300: Memory
1310: ROM
1320: RAM
1400: User interface

Claims (5)

An image acquiring unit acquiring an image of the outside of the vehicle from a camera mounted on the vehicle;
A vehicle speed obtaining section for obtaining a speed of the vehicle;
When the speed of the vehicle is not more than a predetermined speed,
And a controller for generating a plurality of differential frames each composed of a difference value between a current frame of the image and each of a plurality of previous frames, generating an overlapping frame by overlapping the plurality of difference frames, and recognizing an obstacle from the overlapping frame Obstacle recognition system. The method according to claim 1,
Wherein the controller corrects the spatial position of the plurality of previous frames using the velocity of the vehicle and generates the difference frame using the corrected plurality of previous frames. The method according to claim 1,
When the speed of the vehicle is zero or when the speed of the vehicle is higher than the predetermined speed,
Wherein the control unit recognizes the obstacle by comparing at least one of the current frame and the plurality of previous frames without generating the overlapping frame. The method according to claim 1,
When the camera is a fisheye lens camera,
Wherein the control unit corrects the distortion of the image using the distortion rate of the fisheye lens camera. 5. The method of claim 4,
If the speed of the vehicle is zero,
Wherein the control unit does not correct the distortion of the image.

Images