CN103192830B - A kind of self-adaptive visual lane departure warning device - Google Patents

Abstract

A kind of self-adaptive visual lane departure warning device, relates to Vehicle security system.It is provided with vehicle power, decompression voltage regulator, attention device, FPGA core core, digital camera and steering angle induction apparatus；The input of decompression voltage regulator connects the outfan of vehicle power, the outfan of decompression voltage regulator connects the power module input of FPGA core core and the input of digital camera respectively, the output interface of FPGA core core connects the input of attention device, the input interface of the output termination fpga core plate of digital camera, the input interface of the output termination fpga core plate of digital camera, the input interface of the output termination fpga core plate of steering angle induction apparatus.Improve vision lane departure warning device Lane detection ability under complex illumination and complex road surface situation.Improve the vision lane departure warning device deviation pre-alerting ability to bend.Improve image procossing and the early warning response speed of existing vision lane departure warning device.

Description

A kind of self-adaptive visual lane departure warning device

Technical field

The present invention relates to Vehicle security system, especially relate to a kind of self-adaptive visual lane departure warning device.

Technical background

Along with the fast development of information and automatic control technology, car steering ancillary technique is accepted by auto vendor and user gradually.Lane departure warning device is possible to prevent or reduces the vehicle accident that driver causes due to driving fatigue, and has been increasingly becoming the standard configuration of high-end automobile brand.The lane line tracking and early warning device of view-based access control model is the mainstream technology route of lane departure warning product.

Vision lane line is followed the tracks of and traffic lane deviation pre-alarming device generally comprises 3 main modular: (1) video acquisition module；(2) image analysis module；(3) user's alert module.Wherein image analysis module is responsible for the task of identifying lane line, is the core technology of vision lane departure warning device.The Lane detection method adopted at present mainly has four kinds: (1) adopts the edge detection operator such as Sobel operator, Canny operator that original image is processed, higher processing costs is provided at lane line boundary, thus judging the position on border, and obtain the position (referring to Chinese patent CN102303609A and US Patent No. 4970653) of lane line from boundary information；(2) directly or adopt Hough transformation (HoughTransform) after application edge detection operator, calculate the straight line existed in image, and thus obtain the position (referring to Chinese patent CN102303609A and CN201712600U, US Patent No. 5790403) of lane line；(3) method adopting template matching, finds border along different angles, and thus obtains the position (referring to Chinese patent CN101804813A and US Patent No. 5398292) of lane line.(4) first gray level image is carried out binary conversion treatment, by selecting appropriate binary-state threshold, lane line and pavement image is made to be separated into bianry image and then the position of searching vehicle diatom (referring to Chinese patent CN101016052A and CN101016052A).

Rim detection is the Basic Problems of image recognition, has very long research history (referring to document: Zhang Yujin, " Image Engineering (middle volume) graphical analysis " second edition).Stable and sharp picture identification under complex environment is the ultimate aim of image recognition technology.Facts have proved, simple image binaryzation and edge detection method are difficult to adapt to various complex environments, and the image partition method combination based on model identifies that image binaryzation and edge detection process are retrained by clarification of objective, a lot of application achieve good effect (RobertHanek, Model-BasedImageSegmentationUsinglocalSelf-AdaptingSepar ationCriteria, LectureNotesinComputerScienceVolume2191,2001:1-8).Although the image partition method image segmentation based on model is better, but amount of calculation is very big, is unsuitable for the graphical analysis of real-time video, particularly on the vehicular platform calculating resource critical constraints.

Zheng Xinqian et al. (Zheng Xinqian etc., based on vision guided navigation volume the car design and the realization of FPGA, Xiamen University's journal natural science edition, 2012,2) proposes adapting to image binaryzation and the track identification method of a kind of novelty；Image binaryzation is combined with track identification width, feeds back binary-state threshold by track identification width.Based on the monorail vision guided navigation model dolly of this self-adaptive identification method, under multiple illumination and surface conditions, achieve good navigation effect.

Image is carried out square wave convolution, forms domatic leading line border, to form the key that associating between the leading line width of binaryzation and binary-state threshold is Zheng Xinqian et al. algorithm proposed.This algorithm can only realize in width leading line more than 30 pixels, and lane departure warning device needs to identify two very thin lane lines (generally only having 3～5 pixel width).So, the self adaptation track recognizer that Zheng Xinqian et al. proposes cannot directly be applied on lane departure warning device.

Job insecurity when except complex illumination and complex road surface, existing vision lane departure warning device is also undesirable to the early warning effect of bend.The lane departure warning device announced much adopts Hough straight line to convert, therefore the deviation of the straight line portion in straight turning road or bend can only be carried out early warning (referring to Chinese patent CN102303609A and CN201712600U, US Patent No. 5790403), the lane line near field can only be identified (referring to Chinese patent CN102951159A and CN101704367A) by some lane departure warning device, therefore the bend in None-identified far field is likely to driving the impact produced.Chinese patent CN102295004A discloses a kind of lane departure warning method, by measuring the speed of service and the corresponding maximum lateral acceleration arranged in advance and min. turning radius, assume that automobile travels by arc track, prediction is likely to the shortest time crossing lane line, and judges whether to give a warning with this.Whole method is excessively complicated, and operand is relatively big, not easily realizes.Simultaneously as maximum lateral acceleration is affected by surface conditions, so cannot accurately measure in advance so that whole method is difficulty with.

The image procossing of existing lane departure warning device substantially adopts single-chip microcomputer (referring to Chinese patent CN202686122U), digital signal processor (Chinese patent CN201712600U), computer (Chinese patent CN101016052A), or computer is plus special graphical analysis circuit (US Patent No. 5430810).The common ground of these image analysis equipment is to adopt serial process mode, and when processing multitude of video information, arithmetic speed is restricted.

Summary of the invention

It is an object of the invention to provide a kind of self-adaptive visual lane departure warning device.

The present invention is provided with vehicle power, decompression voltage regulator, attention device, field programmable gate array (FieldProgrammableGateArray, FPGA) core board (hereinafter referred to as FPGA core core), digital camera and steering angle induction apparatus；Described FPGA core core is provided with power module, crystal oscillator, fpga chip and input/output interface；

The input of described decompression voltage regulator connects the outfan of vehicle power, the outfan of decompression voltage regulator connects the power module input of FPGA core core and the input of digital camera respectively, the output interface of FPGA core core connects the input of attention device, the input interface of the output termination fpga core plate of described digital camera, the input interface of the output termination fpga core plate of digital camera, the input interface of the output termination fpga core plate of steering angle induction apparatus.

Compared with carrying out image procossing with existing main employing single-chip microcomputer, digital processing unit, computer or computer with the serial computing scheme that special circuit combines, the present invention adopts field programmable gate array chip, carry out image processing line by line parallel, significantly improve image processing speed and early warning response speed.The present invention, by image processing algorithms such as gray scale expansion and square wave convolution, making lane line broaden and forms domatic border, associating thus being formed between the binaryzation width and binary-state threshold of lane line；The measured value of lane width is fed back to image binaryzation threshold value, and makes corresponding adjustment, so that image binaryzation threshold value adapts to various complex illumination and surface conditions automatically.

The present invention is by assuming that circular arc traffic route calculates the reasonable steering angle matched with lane line, and compares with the reading of vehicle-mounted steering angle induction apparatus and judge.When the difference of the two is beyond time a range of, send deviation early warning signal by attention device module.Relative to the device that the straight line portion of rectilinear stretch or bend can only carry out early warning, bend deviation in far field can be sent early warning earlier by the present invention, strengthens the effect of safety precaution.

The present invention has advantage highlighted below:

1) vision lane departure warning device Lane detection ability under complex illumination and complex road surface situation is improved.

2) the vision lane departure warning device deviation pre-alerting ability to bend is improved.

3) image procossing and the early warning response speed of existing vision lane departure warning device are improved.

Accompanying drawing explanation

Fig. 1 is the structural representation of the embodiment of the present invention.

Fig. 2 is the decompression voltage regulator schematic diagram of the embodiment of the present invention.

Fig. 3 is the schematic diagram of the self adaptation lane mark identification algorithm of the embodiment of the present invention.

The gray scale that Fig. 4 is the embodiment of the present invention expands and the schematic diagram of effect after square wave convolutional filtering.

Fig. 5 is the schematic diagram of the planning travelling line of the embodiment of the present invention.

Detailed description of the invention

Following example will the present invention is further illustrated in conjunction with accompanying drawing.

Referring to Fig. 1～5, the embodiment of the present invention is provided with vehicle power 1, decompression voltage regulator 2, attention device 3, FPGA core core 4, digital camera 5 and steering angle induction apparatus 6；Described FPGA core core 4 is provided with power module, crystal oscillator, fpga chip and input/output interface；The input of described decompression voltage regulator 2 connects the outfan of vehicle power 1, the outfan of decompression voltage regulator 2 connects the power module input of FPGA core core 4 and the input of digital camera 5 respectively, the output interface of FPGA core core 4 connects the input of attention device 3, the input interface of the output termination fpga core plate 4 of described digital camera 5, the input interface of the output termination fpga core plate 4 of digital camera 5, the input interface of the output termination fpga core plate 4 of steering angle induction apparatus 6.

Described field programmable gate array (FPGA) core board is as key process unit, by the analysis to lane line image, determines relative position and the orientation of automobile and lane line, and when automobile is by run-off-road line, sends early warning signal；

Vehicle power 1 powers to FPGA core core 4 and digital camera 5 by decompression voltage regulator 2.Digital camera 5 receives image and by data information transfer to FPGA core core 4.Image is acquired and analyzes by FPGA core core 4, judges travel condition of vehicle in conjunction with the signal read from steering angle induction apparatus 6, and sends early warning signal when vehicle is by run-off-road to attention device 3.

The Core Feature of the present invention and key technique are the identification calculating with reasonable steering angle of lane line.In conjunction with Fig. 3, it is achieved specifically comprising the following steps that of the core link of the present invention

1) after digital camera 5 powers on, with fixing frame per second, gather road image, and transmit to field programmable gate array core board 4 line by line.

2) field programmable gate array core board 4 receiving step 1) described in a line image information, and the step shown in Fig. 3 be analyzed process.

3) be first according to equation (1) to step 2) described in a line image carry out gray scale dilation operation, increase the width of lane line.

p_i=max{p_k},i-7≤k≤i+8(1)

Wherein p_iAnd p_kFor the gray value of i-th and k pixel, max is the function of maximizing.

4) according to equation (2) to step 3) described in a line image carry out square wave convolution, eliminating while noise, forming figure as shown in Figure 4.

$p_i={\Σ}_{k=i-7}^{i+8}{p}_k\⊕1---\left(2\right)$

Wherein p_iAnd p_kFor the gray value of i-th and k pixel, for logic XOR.W in Fig. 4₀, W₁And W₂Respectively desirable binary-state threshold, too high binary-state threshold and calculate the lane width of acquisition under too low binary-state threshold.By square wave process of convolution so that calculate the lane width obtained and directly produce with binary-state threshold to associate, such that it is able to binary-state threshold is carried out self-adaptative adjustment.We claim the lane width W calculating acquisition under desirable binary-state threshold₀For standard lane width.Standard lane width W₀Obtain when surface conditions after auxiliary driving device starts learns, and use in binary-state threshold self-adaptative adjustment.Refer to step 12).

5) binary-state threshold that hands down of lastrow image is adopted, to step 4) obtained gray level image carries out binary conversion treatment.

6) in step 5) in a line image of obtained binaryzation, with the lane center position M that lastrow hands down₀Centered by, difference search track line boundary to the left and right sides.The method of search is that image does the jump function convolution to the left or to the right as shown in equation (3) and (4).

$E_L\left(i\right)={\mathrm{\Σ}}_{k=i-7}^i{p}_k\⊕0+{\mathrm{\Σ}}_{k=i+1}^{i+8}{p}_k\⊕1---\left(3\right)$

$E_R\left(i\right)={\mathrm{\Σ}}_{k=i-8}^{i-1}{p}_k\⊕1+{\mathrm{\Σ}}_{k=i}^{i+7}{p}_k\⊕0---\left(4\right)$

Wherein E_L(i) and E_RI left side dividing value that () is ith pixel and the right dividing value, p_kFor the image value of kth pixel, for logic XOR.Search for E respectively in the lateral direction_L(i) and E_R(i) value pixel more than 12, and it is demarcated as left-lane line boundary LB and right lane line boundary RB.

7) after searching the border of the left side or the right lane line, lane line is carried out template matching identification, to confirm the identification certainty of lane line.Concrete matching process is as follows:

$V_L={\mathrm{\Σ}}_{k=LB-2\×WD+1}^{LB-WD}{p}_k\⊕0+{\mathrm{\Σ}}_{k=LB-WD+1}^{LB}{p}_k\⊕1+{\mathrm{\Σ}}_{k=LB+1}^{LB+WD}{p}_k\⊕0---\left(5\right)$

$V_R={\mathrm{\Σ}}_{k=RB-WD}^{RB-1}{p}_k\⊕0+{\mathrm{\Σ}}_{RB}^{RB+WD-1}{p}_k\⊕1+{\mathrm{\Σ}}_{k=RB+WD}^{RB+2\×WD-1}{p}_k\⊕0---\left(6\right)$

Wherein V_LAnd V_RFor left-lane line and the template matching value having lane line, LB and RB is step 6) in the left-lane line boundary that searches and right lane line boundary, WD is the width of lane line, learns the stage on road surface and obtains, refers to step 13).The V obtained_LAnd V_RIf greater than the threshold value set in advance, it is determined that the left-lane line boundary LB and right lane line boundary RB of acquisition can use, otherwise cancel the value of LB or RB.

8) according to step 6) and 7) search and the left-lane line boundary LB confirmed and right lane line boundary RB calculate the width in track and center M.Due to the interruption of lane line, or due to light, surface conditions temporarily cannot find left margin or right margin time, with equation below calculate lane width:

Wherein RB and LB is step 6) in the left-lane line boundary that searches and right lane line boundary, M₀It is the midpoint, track found of lastrow, W₀It it is standard trajectory width.

9) according to calculating obtained width, with normal width W₀Compare, if more than normal width, and beyond certain limit, then when next line binaryzation, reduce threshold value 1-2 gray value；Otherwise then improve threshold value 1-2 gray value.Standard lane width W₀Obtain in the study stage, refer to step 12).

10), when processing last column, the radius of curvature R 7 of reasonable traffic route can be calculated according to the lane center position M obtained.According to Fig. 5, it can be deduced that the formula of radius of curvature R 7 calculated below is.

$R=\frac{a^2+b^2}{2b}---\left(8\right)$

Wherein a8 is the distance of lane center deviation vehicle centerline, and b9 is the visual field end distance to front vehicle wheel.B9 can actual measurement obtain in advance.A8 can be calculated by lane center position M and obtain, square journey (9)

$a=(M-\frac{D}{2})\×\mathrm{\λ}---\left(9\right)$

Wherein D is the width of view field image, and M is lane center position, and λ is the proportionality constant of transmitted distortion, can measure acquisition in advance.What Fig. 5 showed is eliminate the visual field after perspective distortion.

11) according to step 10) radius of curvature R 7 of calculated planning camber line, and the automobile front and back wheel interval S recorded in advance, it is possible to approximate calculation goes out the formula of the reasonable automobile steering angle α for planning camber line:

$\mathrm{\α}=\frac{S}{R}---\left(10\right)$

Read automobile actual steering angle by steering angle induction apparatus, and compare with calculating the reasonable automobile steering angle α obtained.When the difference of the two is beyond certain limit, send early warning signal by attention device module.

According to equation (8), when (9) and (10) calculate reasonable automobile steering angle α value, unique variable changed is parameter a8.Because the span of a8 is less than the width of image, so corresponding rationally automobile steering angle α value can be gone out calculated in advance, and set up look-up table.When plant running, only need to search correlation values, it is not necessary to calculate in real time, it is possible to reduce operation time and save logical resource.

12), after starting lane departure warning device, start to learn standard trajectory width parameter W₀.The first row in multiple image or first few lines image are carried out step 3) 7) operation, it is thus achieved that average track width value is as standard trajectory width parameter W₀。

$W_0=\frac{1}{N}{\mathrm{\Σ}}_{i=1}^N({\mathrm{RB}}_i-{\mathrm{LB}}_i)---\left(10\right)$

Wherein N is the number of image frames gathered.LB_iAnd RB_iIt is left-lane line boundary and the right lane line boundary of certain row image in the i-th frame.

13) in step 12) basis on, learn standard trajectory line width parameter WD.For the i-th two field picture gathered, search for optimal trajectory line width parameter WD_i, and N two field picture is averaging draws standard trajectory line width parameter WD.

WD_i=argmax{V_L(WD_i)+V_R(WD_i))

$WD=\frac{1}{N}{\mathrm{\Σ}}_{i=1}^N{\mathrm{WD}}_i---\left(11\right)$

Wherein V_LAnd V_RBeing the template matching value of definition in equation (5) and (6), argmax is so that V_L(WD_i)+V_R(WD_i) parameter WD when taking maximum_iValue, N be calculate frame number.

Disclosure one self-adaptive visual lane departure warning device.When there is clear carriageway line mark, coordinating vehicle-mounted steering angle induction installation, it is achieved the shooting to railway line, identifying, and lane departure warning.Adopt adaptive binaryzation Lane detection method so that this device can under more complicated light and surface conditions normal operation.Assume circular arc traffic route, directly measure the reasonable steering angle matched with lane line, improve the pre-alerting ability that bend is deviateed by device.Adopt FPGA (FieldProgrammableGateArray, field programmable gate array) chip is as primary processor, image can be carried out parallel processing so that this device has image procossing and early warning response speed faster than the common traffic lane deviation pre-alarming device based on the serial processor such as computer or microprocessor.

The present invention, according to the relative position between automobile and lane line and orientation, calculates rational automobile steering angle；Deviation between the actual steering and reasonable steering angle of automobile exceeds after to a certain degree, sends lane departur warning signal to driver.

In Lane detection process, by image processing algorithms such as gray scale expansion and square wave convolution, making lane line broaden and form domatic border, associating thus being formed between the binaryzation width and binary-state threshold of lane line；The measured value of lane width is fed back to binary-state threshold, and makes corresponding adjustment, so that binary-state threshold adapts to illumination and the surface conditions of various complexity automatically.

FPGA core core, as key process unit, is responsible for carrying out image acquisition, graphical analysis, reception induction apparatus signal and sending early warning signal.

Time required for the image processing process such as the process of image early stage, image binaryzation, Lane detection, and threshold feedback is less than the time of digital camera collection and transmission a line image.Image processing process carries out line by line in real time.

When negotiation of bends, the deviation of steering angle can be accurately identified, and implement early warning in time.Specific implementation method is according to the lane center position at visual field end, calculates the steering angle of reasonable traffic route, and compares with the actual steering angle of automobile.When steering angle deviation is beyond certain limit, reported to the police by attention device.

According to lane center when the reasonable steering angle of the position calculation of visual field end, the method adopting look-up table, directly give the steering angle of corresponding lane line remote location, thus saving calculating time and logical resource.

The present invention learns the dynamic track parameter needed for self-adapting estimation lane line automatically.The method of automatic learning dynamics track parameter is: search identifies the first row in multiple image or the lane line in first few lines image, calculates lane width and optimized vehicle road line width that search can be mated；Using the lane width of gained and lane line width, the meansigma methods in multiple image is as the standard lane width in self-adapting estimation and standard lane line width.

Identify that the function of lane line divides following 6 steps to complete:

(1) image is carried out gray scale expansion or other image processing methods, widen lane line.

(2) image is carried out square wave convolution or other image processing methods, make lane line form domatic border.

(3) image is carried out binaryzation.

(4) image is carried out jump function convolution to the left and to the right searching vehicle road line boundary.

(5) lane line found is carried out template matching, to confirm the reliability identified.

(6) lane width obtained is compared with the standard lane width of acquisition in learning process, and according to comparative result, binary-state threshold is adjusted.

Claims (3)

1. the self-adaptive visual lane departure warning device based on FPGA, it is characterised in that be provided with vehicle power, decompression voltage regulator, attention device, FPGA core core, digital camera and steering angle induction apparatus；Described FPGA core core is provided with power module, crystal oscillator, fpga chip and input/output interface；The input of described decompression voltage regulator connects the outfan of vehicle power, the outfan of decompression voltage regulator connects the power module input of FPGA core core and the input of digital camera respectively, the output interface of FPGA core core connects the input of attention device, the input interface of the output termination fpga core plate of described digital camera, the input interface of the output termination fpga core plate of steering angle induction apparatus；

Described fpga chip, as key process unit, is used for carrying out image acquisition, graphical analysis, accepting induced signal and send early warning signal；

Described key process unit, in Lane detection process, is expanded and square wave convolved image Processing Algorithm by gray scale, making lane line broaden and form domatic border, associating thus being formed between the binaryzation width and binary-state threshold of lane line；The measured value of lane width is fed back to binary-state threshold, and makes corresponding adjustment, so that binary-state threshold adapts to illumination and the surface conditions of various complexity automatically；

The function of described key process unit identification lane line divides following 6 steps to complete: image is carried out gray scale expansion by (1), widens lane line；(2) image is carried out square wave convolution, make lane line form domatic border；(3) image is carried out binaryzation；(4) image is carried out jump function convolution to the left and to the right searching vehicle road line boundary；(5) lane line found is carried out template matching, to confirm the reliability identified；(6) lane width obtained is compared with the standard lane width of acquisition in learning process, and according to comparative result, binary-state threshold is adjusted.

2. a kind of self-adaptive visual lane departure warning device based on FPGA as claimed in claim 1, it is characterised in that described key process unit learns the dynamic track parameter needed for self-adapting estimation lane line automatically.

3. a kind of self-adaptive visual lane departure warning device based on FPGA as claimed in claim 1, it is characterized in that described key process unit according to lane center when the reasonable steering angle of the position calculation of visual field end, the method adopting look-up table, directly gives the steering angle of corresponding lane line remote location.