KR20170016143A - Lane recognition system using light fied camera and method therefor - Google Patents

Abstract

The present invention relates to a lane recognition system using a light-field camera and a method thereof, which improve the accuracy of lane recognition and a variety of changes in the environment, prevent a big accident by recognizing the distance between the vehicle and an external object, and greatly enhance user convenience. The system of the present invention comprises: an image shooting unit (100) which converts an image, which is shot by a light-field camera (111), into an all-in-focus image which has an estimated depth value, which is obtained by estimating the distance between an external object and the vehicle; an image converting unit (200) which removes noise by using the estimated depth value of the all-in-focus image and converts the image into an around view image; a template matching unit (300) which generates a template in the around view image of the image converting unit (200), sets a lane sign candidate area, and connects lane signs by matching template; an image processing unit (400) which extends and erodes the lane sign candidate area of the template matching unit (300) and connects the lanes which are straight lines and curves; a lane recognizing unit (500) which sets an area of interest in the connected lane sign candidate area of the image processing unit (400) and selects and detects a used lane; and a determination control unit (600) which determines the presence or absence of an error of the used lane, which is detected by the lane recognizing unit (500), and if the vehicle has escaped from the lane.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a lane recognition system and method using a photo-

The present invention relates to a lane recognition system and method using a photosystem camera. More particularly, the present invention not only improves the accuracy of road lane recognition, but also recognizes the distance between the vehicle and an external object, And more particularly, to a system and method for recognizing a lane using a photosystem camera that greatly improves user's convenience.

Conventional vehicle camera systems have been used including a depth camera (Time of Flight) system using a laser, an infrared sensor and the like and a binocular stereoscopic camera in a single camera.

In the binocular stereoscopic camera system, a horizontal movement method of rotating the optical axis of the left and right cameras and a horizontal movement method of moving the lens or sensor horizontally according to the distance of the object of interest by separating the lens and sensor of the left and right cameras, have.

In general, a system for introducing a separate sensor from the binocular stereoscopic camera is bulkier and bulkier than a conventional monocular camera, and since two cameras can not be accessed below 6 mm, it is difficult to capture close-up images, There is a problem such as inconsistency of the brightness, color, and image quality of the left and right cameras, inconsistency of the vertical and horizontal directions due to the inability of interlocking of the zoom and focus, inconsistency of magnification, and rotation of the screen.

Also, there is a problem that the depth camera method has a low resolution.

In order to solve the above problems, a lane recognition is improved by forming an image acquired from a camera of a vehicle as an illuminance invariant image, as in the registered patent No. 10-1498975, and the lane recognition rate is improved, Is described.

However, since the perspective is applied when recognizing the lane, it is difficult to see the road at a glance, and when the vehicle leaves the lane, it is difficult for the user to visually confirm the degree of departure.

In addition, since the lane candidate is detected by the edge filter or the lane emphasis filter and the lane candidate is detected at the boundary of the image using the brightness difference, there is a problem that it can be applied only to a road having a strong outline.

Accordingly, there is a need for a method for enhancing the recognition rate of a general road lane as well as a road having a lane marking and a strong outline, and improving the safety of a user by connecting a lane in real time to a real vehicle speed.

The present invention has been proposed in order to solve the above-described problems. The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide a road- The present invention provides a lane recognition system and method using a photosystem camera that can improve user's convenience and safety through all-in-focus and template matching.

Another object of the present invention is to provide a lane recognition system and a lane recognition system using a photosystem camera, which can improve lane detection accuracy by removing noise corresponding to a region other than a road used by a user.

It is another object of the present invention to provide a lane recognition system and method using a photosystem camera capable of reducing a calculation amount of a lane detecting process and improving a processing speed of connecting lanes in real time using a morphological image processing at a fast traveling speed of an actual vehicle .

It is another object of the present invention to provide a lane recognition system and method using a photosystem camera capable of recognizing a distance between a user's vehicle and an external object to prevent an accident.

The problems to be solved by the present invention are not limited to the above-mentioned problems, and other matters not mentioned can be clearly understood by those skilled in the art from the following description.

In order to accomplish the above object, the present invention provides a lane recognition system and method using a photosystem camera, the system comprising: an all-in-focus (AI) camera having a depth estimation value obtained by estimating a distance between an external object and a vehicle, in-focus image, an image converting unit 200 for removing noise using the depth estimation value of the all-in-focus image and converting the noise into an ambient view image, A template matching unit 300 for generating a template from the surrounding view image of the image transform unit 200, setting a lane marker candidate region and connecting lane markers by template matching, A lane marker candidate region of the lane marker candidate region of the lane marker candidate region and connecting the straight line and the curved lane to each other; Will be in the lane recognition unit 500 for detecting, characterized in that the configuration including the determination controller 600 for determining the lane departure whether the lane recognition unit used error and whether the vehicle in the lane is detected in 500. The

The image capturing unit 100 includes a capturing input unit 110 for capturing an image and inputting a captured image, a depth estimating unit 110 for estimating a distance between the external object and the vehicle stored in the image capturing input unit 110, An all-in-focus unit 130 that includes information of the depth estimator 120 and converts the image into a plurality of focus images to improve sharpness. do.

In addition, the image transforming unit 200 may include a noise removing unit 210 for removing an external object corresponding to noise using depth estimation information included in the all-in-focus unit 130, And an aver view 220 for transforming a lane of the image from which the noise is removed by the noise removing unit 210 into an averaged view with a predetermined width without a perspective view.

The template matching unit 300 includes a template generating unit 310 for generating a template using the surround view and a lane marker candidate region set by the template generating unit 310 and matching And a lane mark connecting part 320 connecting the curved lane.

The lane recognition unit 500 further includes a region of interest setting unit 510 for setting a region of interest of the lane marking candidate region, a lane selection unit 510 for selecting a lane to be used in the region of interest setting unit 510, And a lane selection unit 520 for selecting a lane.

The lane departure determination unit 600 further includes a lane error determination unit 610 that determines an error of the lane selected by the lane selection unit 520 and a lane error determination unit 610 that receives a determination signal indicating that there is no error in the lane error determination unit 610 And a lane departure determination unit 620 for determining whether or not the vehicle is departing from the lane and improving the safety of the user.

An image capturing step of receiving the image photographed from the photosystem camera 111 and converting the distance between the external object and the vehicle into an all-in-focus image having an estimated depth value; An image converting step of removing noise of the all-in-focus image and converting the noise into an all around view image; A template matching step of generating a template using the lane marker and the relative position information of the surround view image and matching the candidate template with the generated template and connecting the candidate area lane marker; An image processing step of extending and eroding the candidate area lane marker to connect lanes to a straight line and a curve; Selecting a lane to be used by setting a region of interest for detecting a lane in the candidate region; A lane error determination step of determining an error of the selected lane; A lane departure determination step of determining whether or not the selected selected lane has departed; And a control unit.

As described above, according to the present invention, a lane recognition system and method using a photosystem camera can recognize an external object as well as a lane, estimate a distance between the vehicle and an external object, greatly reduce an accident risk rate, It is possible to improve the convenience and safety of the user significantly.

Further, since the processing amount of the lane detecting process is reduced to improve the processing speed, the lane departure and warning situations can be easily coped by connecting the lane in real time.

In addition, the system cost can be reduced by using an inexpensive optical system camera.

1 is a lane recognition system using a photosystem camera according to an embodiment of the present invention.
2 is a photographing part of a lane recognition system using a photosystem camera according to an embodiment of the present invention.
3 is a block diagram of an image conversion unit of a lane recognition system using a photosystem camera according to an embodiment of the present invention.
4 is a block diagram of a template matching unit of a lane recognition system using a photosystem camera according to an exemplary embodiment of the present invention.
5 is a lane recognition part of a lane recognition system using a photosystem camera according to an embodiment of the present invention.
6 is a block diagram of a determination control unit of a lane recognition system using a photosystem camera according to an exemplary embodiment of the present invention.
FIG. 7 is a perspective view of a spectacle camera of a lane recognition system using a spectacle camera according to an embodiment of the present invention. FIG.
8 is a view illustrating a light field restoration process of a lane recognition system using a photosystem camera according to an embodiment of the present invention.
9 is a positional diagram of a lumen recognition system using a photosystem camera according to an embodiment of the present invention.
10 is a view of a lane recognition system using a photosystem camera according to an embodiment of the present invention.
11 is a lane detected using a lane recognition system using a photosystem camera according to an embodiment of the present invention.
12 is a flowchart of a lane recognition system using a photosystem camera according to an embodiment of the present invention.

Hereinafter, a system and method for recognizing a lane using a photosystem camera according to the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a lane recognition system using a photosystem camera according to an embodiment of the present invention. FIG. 2 is a photographing unit of a lane recognition system using a photosystem camera according to an embodiment of the present invention. And is an image converter of a lane recognition system using a photosystem camera according to an embodiment.

FIG. 5 is a lane recognition unit of a lane recognition system using a photosystem camera according to an embodiment of the present invention. FIG. 5 is a block diagram of a lane recognition unit of a lane recognition system using a photosystem camera according to an embodiment of the present invention. 6 is a block diagram of a lane recognition system using a photosystem camera according to an exemplary embodiment of the present invention.

8 is a view illustrating a light field reconstruction process of a lane recognition system using a photosystem camera according to an exemplary embodiment of the present invention. Referring to FIG. 7, FIG. 9 is a positional view of a pneumatic camera of a lane recognition system using a photosystem camera according to an embodiment of the present invention.

10 is a view of a lane recognition system using a photosystem camera according to an embodiment of the present invention. FIG. 11 is a lane detected using a lane recognition system using a photosystem camera according to an embodiment of the present invention And FIG. 12 is a flowchart of a lane recognition system using a photosystem camera according to an embodiment of the present invention.

In the drawings, the same reference numerals are given to the same elements even when they are shown in different drawings. In the drawings, the same reference numerals as used in the accompanying drawings are used to designate the same or similar elements. And detailed description of the configuration will be omitted. Also, directional terms such as "top", "bottom", "front", "back", "front", "forward", "rear", etc. are used in connection with the orientation of the disclosed drawing (s). Since the elements of the embodiments of the present invention can be positioned in various orientations, the directional terminology is used for illustrative purposes, not limitation.

As shown in FIG. 1 or FIG. 12, a system and method for recognizing a lane using a photosystem camera according to a preferred embodiment of the present invention is a system for estimating a distance between an external object and a vehicle In-focus image having an estimated depth value by using the depth estimation value of the all-in-focus image and a depth estimation value of the all-in-focus image, A template matching unit for generating a template from the surrounding view image of the image transform unit 200, setting a lane marker candidate region, and connecting lane markers by template matching, An image processing unit 400 for extending and eroding the lane marking candidate area of the template matching unit 300 and connecting a straight line and a curved lane to each other and a lane marker candidate area connected to the image processing unit 400,A lane recognition unit 500 for setting a shim area and selecting and using a lane to be used, and a determination control unit 600 for determining whether the used lane detected by the lane recognition unit 500 is erroneous or not, .

2, the image photographing unit 100 includes a photographing input unit 110 that photographs an image using a photosystem camera 111 and inputs a photographed image, A depth estimator 120 for estimating a distance between an external object and a vehicle of the image and an all-in-focus controller 120 that includes information of the depth estimator 120 and converts the image into a plurality of images to improve the sharpness. and a focus unit 130. [0027]

As shown in FIG. 7, the optical system camera of the image capturing unit 100 includes a diaphragm (not shown) for adjusting the amount of light passing through the main lens 112, A mask 114 for calculating a light field by modulating light passing through the diaphragm 113 and a light modulator 114 which is in close contact with the rear surface of the mask 114 and which is modulated by passing through the mask 114, A sensor 115 for storing a field and a microcontroller unit 116 for encoding an image coming from the sensor 115. [

The optical system camera 111 has a function of a three-dimensional camera by adding a mask, which is a light modulation filter, to a monocular camera, so that the lane recognition rate can be improved and the close-up photographing can be performed even on old roads or general roads where lane marking is not strong.

In addition, since the calculation amount of the matching operation is relatively smaller than that of the conventional camera, the size of the system is reduced, so that it can be quickly calculated and it is easy to store the image in real time on the real road where the image changes rapidly. If it is difficult to use, the focus acquisition and image processing of the object can be facilitated.

The mask 114 for modulating the light is a modulation function that is located between the sensor 115 and the main lens 112 and modulates the light field as a plane shield having the same size as the sensor.

In addition, the mask 114 may be a pinholes, a sum of sinusoids (SoS), a Modified Unirottle Dudundat Array (MURA) pattern, or the like.

는 아래 수학식 1 내지 수학식 4으로 나타낼 수 있다.At this time, the image modulated and stored in the sensor 115

Can be expressed by the following equations (1) to (4).

First, the value of the image sensor is calculated by summing up all the input rays as shown in Equation 1 below.

The one-dimensional Fourier transform of the image sensor is related to the two-dimensional Fourier transform of the input light field as shown in the following equation (2).

As described above, the horizontal slice of the light field two-dimensional Fourier transform becomes the same as the one-dimensional Fourier transform of the sensor by the Fourier projection-slice theorem, as shown in the following equation (3).

The image in which the modulated light field is stored in the sensor is expressed by Equation (4) below.

는 마스크이고, 상기

는 라이트 필드이며, 변수

는 각각 2차원 상에서 상대적인 두 가지의 매개변수의 각 축이다.remind

Is a mask,

Is a light field, and the variable

Are the respective axes of the two parameters relative to each other on the two dimensional plane.

는 메인렌즈의 직경과 메인렌즈와 센서와의 고정된 거리이다.Also,

Is the diameter of the main lens and the fixed distance between the main lens and the sensor.

Accordingly, the apparatus includes a photographing input unit 110 for storing the modulated image and inputting information of the stored image to the image processor to reverse-engineer the image to calculate light angle or intensity information.

The stored image is estimated by calculating the distance between the external object and the vehicle in the depth estimating unit 120 and receives the input image of the depth estimation value in the all-in-focus unit 130.

An all-in-focus process is performed by combining a refocused image of a light field finally reconstructed by the all-in-focus unit 130 and a method of combining a non-focus and a communication signal. focus image (S100).

Here, as shown in FIG. 8, the light field is modulated and restored in the frequency domain by applying the pattern of the mask 114.

는 대역제한적이라 가정하며, 상기

의 푸리에 변환을

로 표기한다.The input light field

It is assumed that the bandwidth is limited,

Fourier transform of

.

는 u,s의 절대적인 두 가지 매개변수인 각 축의 푸리에 변환이다.

Is the Fourier transform of each axis, which is an absolute two parameters of u, s.

-axis를 따라 오버샘플링된 조각을 추출한다.According to the Fourier-slice Theorem, first the light field extraction (131) is a high resolution camera sensor

Extract the oversampled fragments along the -axis.

를 얻을 수 있다. The light field modulation 132 is then applied to the light field modulated by the mask located in front of the sensor

Can be obtained.

와 기울기

를 갖는 마스크 스펙트럼의 컨볼루션과 동일하게 될 수 있으며, 아래 수학식 5와 같이 나타낼 수 있다.This is because

And tilt

And can be expressed by Equation (5) below. &Quot; (5) "

가

의 스펙트럼 복제품으로 구성되며, 라이트필드복원(134)은 상기 센서 조각을 재구성하여

를 예측한

를 얻을 수 있게 된다.Thereafter, the light field copy 133

end

And the light field reconstruction 134 reconstructs the sensor piece

Predicted

.

의 역푸리에 변환을 통해 최종적으로 라이트 필드를 복원할 수 있게 된다.Therefore,

It is possible to finally restore the light field through the inverse Fourier transform of the inverse Fourier transform.

The all-in-focus image can improve the sharpness of an image through a plurality of focal points without focusing on a conventional camera.

In other words, it is possible to reduce the blurring caused by the non-focused image edges before the conversion through the all-in-focus image and to enable a plurality of long lane transplantation.

Using the depth estimation information included in the all-in-focus unit 130, the image conversion unit 200 removes an external object corresponding to the noise as shown in FIG. 3 And the noise removing unit 210 so that the accuracy of scanning the lane transplantation can be significantly improved in an all-in-focus image.

However, the all-in-focus image in which the noise is removed by the noise removing unit 210 has a perspective, which narrows the lane and narrows the field of view.

In order to solve the above problem, the image conversion unit 200 includes an ambient view unit 220 for converting lanes of various roads of an all-in-focus image to a constant width without a perspective, thereby improving the recognition rate of the lane (S120).

As shown in FIG. 10, the surround view photographs the vicinity of 360 degrees of the vehicle with a plurality of cameras and shows the surroundings of the vehicle on the screen as viewed from the sky. In addition, The surroundings of the vehicle can be easily seen on the monitor.

In addition, the blind spot can be easily seen from a general camera, and the risk of a car accident can be reduced.

In this case, as shown in FIG. 9, it is preferable that a plurality of cameras are installed so that a blind spot of the converted image of the surround view unit 220 can be clearly seen.

Therefore, the surround view unit 220 can be greatly reduced in risk of a vehicle accident because there is no blind spot, and it is possible to easily apply a model-based method, which is a lane recognition algorithm that is resistant to noise and data loss.

In addition, the surround view unit 220 can reduce the calculation amount of the detection process by limiting the road area to the area below the horizontal vanishing point.

The template matching unit 300 may further include a template generating unit 310 and a lane marker connecting unit 320 as shown in FIG. 4 to match the template through the image of the surround view unit 220 Lt; / RTI >

The template generating unit 310 generates a template using the lane marker, the lane width, and the relative position information of the surround view unit 220.

The template should be extracted from the lane to be processed with a lane given in advance in the lane recognition.

는 아래 수학식 6로 나타낼 수 있다.The template

Can be expressed by Equation (6) below.

는 영상의 조건이나 환경에 맞추어 선정되는 것이 바람직하다.remind

Is preferably selected according to the conditions and the environment of the image.

After the template is created, a lane mark candidate region of the image of the surround view unit 220 is set in the lane marker connecting unit 320.

The lane markers of the set lane marker candidate areas are connected by curve model template matching such as curve fitting (S130).

In addition, since the curve model template matching has a large amount of computation, the processing speed can be greatly improved by using high morphological image processing.

The lane marker candidate region of the template matching unit 300 is extended and eroded in the image processing unit 400 to connect a straight line and a curved lane (S140).

5, the lane recognition unit 500 for recognizing the lane of the extended and eroded image through the image processing unit 400 includes a region of interest setting unit 510 and a lane select unit 520).

11, the ROI setting unit 510 sets the ROI in the lane mark candidate region, and then the lane selector 520 selects the lane in the ROI and the location of the actual user's vehicle Information matching the lanes to be used through the template matching unit 300, and selects the used lane on the road in the form of a line and a curve finally connected (S150).

In other words, the lane recognition unit 500 limits the range of the vehicle lane, thereby improving the peripheral noise removal and the processing speed.

A lane error determination unit 610 that determines whether the used lane detected by the lane recognition unit 500 is an error in the determination control unit 600 may be used to perform overall control of the present invention.

If the detected use lane is determined to be a lane not used by the user's vehicle, an error signal is sent to the lane recognition unit 500 to detect the used lane again (S160).

On the other hand, when the detected usage lane is determined to be a lane used by the user's vehicle, the vehicle should determine whether the user leaves the used lane without sending an error signal to the lane recognition unit 500.

The determination control unit 600 may further include a lane departure determination unit 620, as shown in FIG. 6, to determine whether or not the usage lane is departed.

When the lane departure determination unit 620 receives a signal indicating that there is no error in the used lane, the lane departure determination unit 620 determines whether the vehicle leaves the used lane.

The lane departure determination unit 620 may further include a hand control device for causing the vehicle to be positioned at the center of the lane of use according to the relative position of the alarm sound and the vehicle lane when the vehicle leaves the lane for use (S170).

Accordingly, the present invention has the effect of saving the unit price by using the inexpensive optical system camera 111, and can recognize the external object as well as the lane, and estimate the distance between the vehicle and the external object, thereby greatly reducing the accident risk.

In addition, by improving the recognition rate and accuracy of the lane, user convenience and safety are significantly improved. In order to improve the processing speed, the calculation amount of the lane detecting process is reduced and the lane departure is performed in real time. There is an effect that it is easy to cope with.

The embodiments of the present invention described above and shown in the drawings should not be construed as limiting the technical idea of the present invention. The scope of protection of the present invention is limited only by the matters described in the claims, and those skilled in the art will be able to modify the technical idea of the present invention in various forms. Accordingly, such improvements and modifications will fall within the scope of the present invention if they are apparent to those skilled in the art.

100: image capturing unit 110:
111: optical system camera 112: main lens
113: Aperture 114: Mask
115: sensor 116: microcontroller unit
120: Depth estimation unit 130: All-in-focus unit
131: Light field extraction 132: Light field modulation
133: Replication of Light Fields 134: Restoration of Light Fields
200: image converting unit 210: noise removing unit
220: a surround view unit 300: a template matching unit
310: template generating unit 320: lane marker connecting unit
400: image processing unit 500: lane recognition unit
510: ROI setting unit 520: Lane selection unit
600: Judgment control unit 610:
620: lane departure determination section

Claims (7)

An image capturing unit 100 for converting an image photographed by the optical system camera 111 into an all-in-focus image having a depth estimated value of the distance between the external object and the vehicle,
An image conversion unit 200 for removing noise using an estimated depth value of the all-in-focus image and converting the noise into an ambient view image,
A template matching unit 300 for generating a template from the surrounding view image of the image transform unit 200, setting a lane marker candidate region and connecting the lane markers by template matching,
An image processing unit 400 for expanding and eroding the lane marking candidate area of the template matching unit 300 and connecting a straight line and a curved lane,
A lane recognition unit 500 for setting a region of interest among the lane marker candidate areas connected to the image processing unit 400 and selecting and using a lane to be used,
And a judgment control unit (600) for judging whether the used lane detected by the lane recognizing unit (500) is erroneous or not and whether or not the lane departure of the vehicle is deviated.
The method according to claim 1,
The image capturing unit 100 includes a capturing input unit 110 for capturing an image and inputting the captured image,
A depth estimator 120 for estimating a distance between the external object and the vehicle stored in the image sensing input unit 110,
And an all-in-focus unit 130 that includes information of the depth estimator 120 and converts the image into a plurality of focus images to improve sharpness. Lane recognition system.
The method according to claim 1,
The image transforming unit 200 may include a noise removing unit 210 for removing an external object corresponding to noise using depth estimation information included in the all-in-focus unit 130,
(220) for converting a lane of an image from which the noise is removed by the noise removing unit (210) into an averaged view with a constant width without a perspective.
The method according to claim 1,
The template matching unit 300 includes a template generating unit 310 for generating a template using the surround view,
And a lane mark connecting unit (320) for connecting a straight line and a curved lane through matching the template of the template generating unit (310) and the lane mark candidate region set by the template generating unit (310).
The method according to claim 1,
The lane recognition unit 500 includes a region of interest setting unit 510 for setting a region of interest among the lane marking candidate regions,
And a lane selection unit (520) for selecting a lane to be used by the RO area setting unit (510) and detecting a selected lane.
The method according to claim 1,
The determination control unit 600 includes a lane error determination unit 610 for determining an error of the lane selected by the lane selection unit 520,
And a lane departure determination unit (620) for receiving a signal indicating that there is no error in the lane error determination unit (610) to determine whether or not the vehicle has departed from the lane so as to improve the safety of the user. Used lane recognition system.
An image capturing step of receiving an image photographed from the photosystem camera 111 and converting the distance between the external object and the vehicle into an all-in-focus image having an estimated depth value;
An image converting step of removing noise of the all-in-focus image and converting the noise into an all around view image;
A template matching step of generating a template using the lane marker and the relative position information of the surround view image and matching the candidate template with the generated template and connecting the candidate area lane marker;
An image processing step of extending and eroding the candidate area lane marker to connect lanes to a straight line and a curve;
Selecting a lane to be used by setting a region of interest for detecting a lane in the candidate region;
A lane error determination step of determining an error of the selected lane;
A lane departure determination step of determining whether or not the selected selected lane has departed;
Wherein the lane resting method comprises the steps of:

Images