CN116363614A - Device and method for road detection and polarization imaging - Google Patents
Device and method for road detection and polarization imaging Download PDFInfo
- Publication number
- CN116363614A CN116363614A CN202310304326.2A CN202310304326A CN116363614A CN 116363614 A CN116363614 A CN 116363614A CN 202310304326 A CN202310304326 A CN 202310304326A CN 116363614 A CN116363614 A CN 116363614A
- Authority
- CN
- China
- Prior art keywords
- road surface
- polarization
- image
- lane line
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
- B60W40/06—Road conditions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Automation & Control Theory (AREA)
- Mathematical Physics (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
A device and a method for road detection and polarization imaging belong to the technical field of polarization imaging and detection. The invention provides a vehicle-mounted high-definition polarization camera imaging system, which is used for acquiring images of a road surface in front of a vehicle running through adding polarization elements and combining corresponding optical imaging elements, detecting polarized images of the road through mutual coordination among a monochromatic brightness information processing unit, a polarization ratio information processing unit, a white line detection unit, a road surface edge detection unit, a road surface shape estimation unit, a lane line searching area determining unit, a lane line candidate point detection unit, a lane line detection unit and other modules, and obtaining brightness information so that a white line has a higher brightness level. The white road route on the road is accurately identified, and the white road route is used as a reference to guide the vehicle to travel, so that the vehicle driving method has higher identification precision on the vehicle driving road in rainy and snowy weather, and better plays a role in assisting the safe driving of the vehicle.
Description
Technical Field
The invention belongs to the technical field of polarized imaging and detection, and particularly relates to a device and a method for road detection and polarized imaging.
Background
With the acceleration of global road construction and the rapid increase of the level of motorization, the number of traffic accidents is also continuously rising. The frequent occurrence of traffic accidents not only can cause economic losses in various aspects such as social security, treatment cost, personnel disputes and the like, but also can threaten the life and property safety of human beings.
The reasons for the occurrence of road traffic accidents are mainly as follows: wrong operation of a driver, failure of vehicle performance and complex traffic environment, wherein the complex traffic environment is a primary cause of multiple traffic accidents, and particularly, the traffic accidents are more likely to occur under severe weather conditions (such as haze days, rain and snow days, cloudy days and the like) and complex driving environments with changeable road topography. Therefore, how to effectively improve the vehicle auxiliary safe driving technology and reduce the occurrence of traffic accidents is a focus of attention of researchers.
At present, various methods for assisting the safe driving of the vehicle are various, but no report for assisting the safe driving of the vehicle by accurately detecting a white line in a driving road image is available.
There is a need in the art for a new solution to this problem.
Disclosure of Invention
The technical problems to be solved by the invention are as follows: the device and the method for detecting the road and imaging the polarization are used for solving the technical problem that the white line in the driving road image is not accurately detected so as to assist the safe driving of the vehicle in the prior art.
The device for road detection and polarization imaging comprises a vehicle-mounted high-definition polarization camera imaging system, wherein the high-definition polarization camera imaging system comprises a polarization imaging unit, a first polarization image memory, a second polarization image memory and a display unit, and the polarization imaging unit is used for acquiring a first polarization image and a second polarization image of a vehicle driving road surface; the first polarized image memory is used for storing the acquired first polarized image; the second polarized image memory is used for storing the acquired second polarized image; the display unit displays each image;
the device for detecting the road and imaging the polarization further comprises a runway pavement detection and identification system, wherein the runway pavement detection and identification system comprises a white line detection unit, a pavement edge detection unit, a pavement shape estimation unit, a lane line searching area determination unit, a lane line candidate point detection unit, a lane line detection unit and a shape information storage unit;
the imaging system of the high-definition polarization camera further comprises a monochromatic brightness information processing unit and a polarization ratio information processing unit;
the single-color brightness information processing unit is used for generating a single-color brightness image and calculating and obtaining brightness information image data of brightness levels represented by pixels in the generated single-color brightness image based on a first polarization component and a second polarization component of the polarization image;
the polarization ratio information processing unit is used for obtaining polarization ratio information, and calculating and obtaining corresponding polarization ratio information data of each pixel based on the first polarization image and the second polarization image;
the white line detection unit is used for detecting and obtaining white line pixel information on a road surface from the brightness information image data obtained by the monochromatic brightness information processing unit;
the road surface edge detection unit scans a running road surface by adopting light beams to obtain a predicted image of the road surface edge, divides the image on the same scanning line, preliminarily divides the image information of the road surface edge and the image information of a roadside building, and finally respectively judges the image information of the road surface edge and the image information of the roadside building through a set threshold condition and sends corresponding image information data to the vehicle control unit for vehicle control;
the road surface shape estimating unit is used for estimating the shape of a road surface, detecting plane linearity formed on a running road surface by using polarization ratio information of each pixel, and dividing the road surface from roadside buildings which are positioned on two sides of the road surface and form a certain angle with the road surface;
the lane line search area determining unit is used for determining a lane line search area under the estimated road surface shape;
the lane line candidate point detection unit is used for judging lane line candidate points in the lane line searching area;
the lane line detection unit is used for determining a lane line in the lane line searching area, and the lane line detection unit is used for reducing the polarization ratio threshold value of the detected lane line in the lane line searching area when the lane line does not exist in the lane line searching area;
the shape information storage unit is used for storing the corresponding lane line information or the road surface shape information in the images obtained by the white line detection unit, the road surface edge detection unit and the road surface shape estimation unit, and synchronously displaying the corresponding information on the display unit in an image form so as to be convenient for a driver to check, and simultaneously sending the corresponding information to the vehicle control unit for vehicle control.
A method for road detection and polarization imaging, using the device for road detection and polarization imaging, comprising the following steps in sequence:
receiving first polarized light and second polarized light in reflected light of an object in an imaging area through a polarized imaging unit of a high-definition polarized camera imaging system, and obtaining first polarized images and second polarized images of road surfaces in front of a vehicle and storing the first polarized images and the second polarized images into corresponding memories respectively, wherein the first polarized light and the second polarized light have different polarization directions;
dividing the first polarized image and the second polarized image into a plurality of processing areas in a one-to-one correspondence manner by a monochromatic brightness information processing unit, respectively obtaining brightness level representation values of the first polarized image and the second polarized image in each processing area, and taking the sum of the brightness level representation values of the first polarized image and the second polarized image in the same processing area as a combined brightness level value for representing the processing area;
step three, a polarization ratio information processing unit obtains polarization ratio information of a first polarization image and a second polarization image, and obtains the ratio of the difference of brightness level characterization values of the first polarization image and the second polarization image in the same processing area to the combined brightness level value of the processing area;
step four, the white line detection unit extracts the brightness level value of the pixel satisfying the characteristic white line in the brightness level characteristic value, compares the brightness level value with a set threshold value, judges that the pixel corresponding to the value satisfying the set threshold value condition is white line pixel information on the road surface, and obtains the polarization ratio of the white line pixel information;
step five, the road surface edge detection unit scans the driving road surface by adopting light beams to obtain a predicted road surface edge image; dividing the image on the same scanning line according to different polarization rates of the road surface and the roadside building, and primarily dividing and obtaining image information of the road surface edge on the scanning line, image information of the roadside building and corresponding polarization ratio information of the road surface and the roadside building;
taking white line pixels obtained by a white line detection unit on the same scanning line as reference pixels;
comparing the difference value of the polarization ratio of the reference pixel information on the same scanning line and the polarization ratio information of the road surface edge image with a set threshold value, judging that the image data meets the threshold value condition as image data of the road surface edge, and transmitting the obtained data to a vehicle control unit for vehicle control;
comparing the difference value between the polarization ratio of the reference pixel information on the same scanning line and the polarization ratio information of the roadside building image with a set threshold value, judging that the threshold value condition is met as roadside building image data, and sending the obtained data to a display unit for display;
synchronously displaying the white line pixel information detected by the white line detection unit and the pavement edge image data detected by the pavement edge detection unit on a display unit in an image form so as to be convenient for a driver to check;
step seven, a road surface shape estimating unit detects plane linearity formed on a running road surface by using the polarization ratio information obtained in the step three, and divides the road surface and roadside buildings which are positioned on two sides of the road surface and form a certain angle with the road surface by using a set threshold value to obtain an estimated road surface shape, wherein the road surface shape comprises road surface gradient and width;
step eight, a lane line searching area determining unit obtains a lane line searching area according to the road surface gradient and the width in the estimated road surface shape;
step nine, a lane line candidate point detection unit detects a lane line on a road surface through polarization ratio information in an obtained lane line search area, and pixels meeting a set threshold condition are determined to be lane line candidate points;
step ten, the lane line detection unit judges whether the lane line candidate points are points on the lane line or not, calculates the width of a line where the lane line candidate points are located according to the points determined as the lane lines, judges the line as the lane line if the calculated line width is within a preset threshold range, and obtains the polarization ratio information of the lane line candidate points located at the edge of the lane line;
the lane line detection unit judges that no lane line is detected if no lane line candidate point exists in the lane line searching area or the width of a line where the lane line candidate point is located does not meet a set threshold value, and reduces the polarization ratio threshold value for detecting the lane line in the lane line searching area for next detection;
the shape information storage unit stores the corresponding lane line information or road surface shape information in the images obtained by the white line detection unit, the road surface edge detection unit and the road surface shape estimation unit, and synchronously displays the corresponding information on the display unit in an image form so as to be convenient for a driver to check, and simultaneously sends the corresponding information to the vehicle control unit for vehicle control.
In the seventh step, the specific steps of dividing the road surface and the roadside building which is positioned at two sides of the road surface and forms a certain angle with the road surface are as follows: binarizing the polarization ratio information obtained in the third step, performing a marking process on the binarized polarization ratio information according to a threshold value of a predetermined parameter, obtaining continuous polarization ratio information having road surface characteristics, thereby obtaining an estimated road surface shape, wherein the road surface shape includes a road surface gradient and a width.
The road surface edge detection unit determines the white line pixel detected in the polarization ratio information previously generated before the detection as the reference pixel under the condition that the white line pixel is not detected or under the condition that the road surface edge detection unit detects that the white line pixel ends in the middle of the vehicle running direction.
The road surface edge detection unit takes pixels of an extension line of the white line pixels which are finished in the middle as reference pixels under the condition that the white line pixels detected in the polarization ratio information generated in advance are finished in the middle of the vehicle running direction;
the road surface edge detection unit takes a pixel located at the center of the road surface as a reference pixel under the condition that no white line pixel is detected in the polarization ratio information generated previously.
The specific method for judging whether the lane line candidate point is a point on the lane line by the lane line detection unit in the step ten is as follows: the lane line detection unit divides the luminance image into an upper region and a lower region in the vehicle traveling direction, sets different luminance thresholds for the upper region and the lower region, and determines that the lane line candidate point is a point on the lane line by comparing the luminance level of each pixel in the upper region and the lower region with the corresponding luminance threshold one by one, the comparison result satisfying the set threshold range.
Through the design scheme, the invention has the following beneficial effects:
the invention provides a vehicle-mounted high-definition polarization camera imaging system, which is used for acquiring images of a road surface in front of a vehicle running through adding polarization elements and combining corresponding optical imaging elements, detecting polarized images of the road through mutual coordination among a monochromatic brightness information processing unit, a polarization ratio information processing unit, a white line detection unit, a road surface edge detection unit, a road surface shape estimation unit, a lane line searching area determining unit, a lane line candidate point detection unit, a lane line detection unit and other modules, and obtaining brightness information so that a white line has a higher brightness level. The white road route on the road is accurately identified, and the white road route is used as a reference to guide the vehicle to travel, so that the vehicle driving method has higher identification precision on the vehicle driving road in rainy and snowy weather, and better plays a role in assisting the safe driving of the vehicle.
Drawings
The invention is further described with reference to the drawings and detailed description which follow:
FIG. 1 is a block diagram of a road detection flow of the apparatus of the present invention;
fig. 2 is a block diagram of a lane line detection flow of the device of the present invention.
In the figure, a 1-1 imaging unit, a 3-1 first polarized image memory, a 3-2 second polarized image memory, a 3-3 monochromatic brightness information processing unit, a 3-4 polarization ratio information processing unit, a 3-5 white line detection unit, a 3-6 shape information storage unit, a 3-7 road surface edge detection unit, a 1-2 display unit, a 1-3 vehicle control unit, a 4-1 road surface shape estimation unit, a 4-2 lane line candidate point detection unit, a 4-3 lane line search area determination unit and a 4-4 lane line detection unit are arranged.
Detailed Description
For a clearer understanding of the objects, structures, principles and functions of the present invention, an apparatus and method for road detection and polarization imaging of the present invention will be described in further detail with reference to the accompanying drawings.
When light enters an interface between two substances having different refractive indexes at a certain angle, a horizontal polarization component (hereinafter, referred to as a P component) parallel to an incident surface, a vertical polarization component (hereinafter, referred to as an S component) perpendicular to the incident surface, and the reflectance of the P component is different from that of the S component. The P component decreases to zero at the brewster angle and then increases. Meanwhile, the S component only increases. Since the P component and the S component have different reflection characteristics, the polarization ratio expressed by the following formula (2) also varies according to the incident angle and the variation in the reflectance.
Typically, pavement is made of asphalt. Meanwhile, roadside buildings adjacent to the road surface and at an angle are made of materials other than asphalt, such as concrete, plants, or soil. In addition, the white lines and other road lines formed on the road surface are also made of materials other than asphalt.
Since different materials have different refractive indices, the polarization ratio of the pavement is different from that of a straight line or roadside building. Unlike the brightness difference, the difference in polarization ratio is not greatly affected by the intensity of incident light. Accordingly, the boundary between the road surface and the roadside building including the road shoulder, vegetation, soil, or the like can be detected by using the polarization ratio information.
Roadside buildings are located near the road surface and are at an angle to the road surface. When the normal direction of the object surface is different, the incident angle from the light source to the object and the reflection angle of light from the object to the camera are also different. Thus, the polarization ratio between the road surface and the adjacent roadside building may be different. The polarization ratio is obtained by the difference between the sum of the P-component and the S-component and the P-component and the S-component. Therefore, even in a dark environment where the luminance difference is small, the road surface edge can be detected using the polarization ratio information.
This also shows that the road shoulder, which is the edge of the roadside building adjacent to the road surface, can be detected by using the polarization ratio information. This method improves the accuracy of detecting the edges of roadside buildings in particular, because there is a difference in polarization ratio due to an angle difference in addition to a difference in polarization ratio due to a difference in material between the road surface and the roadside buildings.
Therefore, the boundary between the road surface and the lane line, and the boundary between the road surface and the roadside building can be detected from the difference in material and angle using the polarization ratio information.
FIG. 1 is a block flow diagram of road detection by an in-vehicle imaging system implemented in accordance with the present invention. The high-definition polarization camera imaging system is mounted on a vehicle. The imaging unit 1-1 photographs a scene image of a traveling direction in front of a vehicle on a road on which the vehicle travels, and obtains a vertical polarization component, a horizontal polarization component as original polarization image data. The obtained horizontally polarized image data is stored in the first polarized image memory 3-1, and the obtained vertically polarized image data is stored in the second polarized image memory 3-2. The horizontally polarized image data and the vertically polarized image data are sent to the monochrome luminance information processing unit 3-3 to perform monochrome luminance information processing, and luminance information representing the pixel luminance level of the generated monochrome luminance image is calculated and obtained. The horizontally polarized image data and the vertically polarized image data are also sent to the polarization ratio information processing unit 3-4, used as polarization ratio information generation, and calculate and obtain polarization ratio information of the polarized image pixels based on the P component and the S component.
The polarization ratio information processing unit 3-4 calculates and obtains polarization ratio information data indicating a polarization ratio using the following formula (1). The polarization ratio represents a ratio between polarization components, and can be calculated using the following formula (2).
Polarization ratio=p component/S component (1)
Polarization ratio= (P component-S component)/(P component+s component) (2)
The wet road condition behaves like a mirror, and thus the reflected light of the wet road has polarization properties. When the reflectance of the vertical polarization component Is defined as Rs and the reflectance of the horizontal polarization component Is defined as Rp, the intensities Is and Ip of the reflected light beams having the incident light intensity I satisfy the following relationship:
Is＝Rs*I (3)
Ip＝Rp*I (4)
where Is the intensity of reflected light of the vertically polarized component; ip is the reflected light intensity of the horizontally polarized component;
when the incident angle of the reflected light is equal to the brewster angle, the horizontal polarization component of the reflected light on the mirror is zero. The vertical polarization component of the reflected light is characterized in that the intensity of the reflected light gradually increases with an increase in the incident angle.
Due to the rough road surface when the road is dry, diffuse reflection dominates. Therefore, the reflected light does not have polarization characteristics, and thus the reflected light intensity of each polarization component is almost equal (i.e., rs=rp). The moisture information on the road surface can thus be obtained from the luminance information of the horizontally polarized image and the vertically polarized image based on the polarization characteristics. Specifically, the ratio of the reflected light intensities Is of the vertical polarized component to the reflected light intensity Ip of the horizontal polarized component, that Is, the image luminance ratio H Is as follows:
H＝Is/Ip＝Rs/Rp (5)
the average value of the brightness of the image brightness ratio H is obtained, and the wet state of the road surface is determined according to the magnitude of the average value. For example, when the road surface Is dry, the reflected light intensity Is of the vertical polarization component and the reflected light intensity Ip of the horizontal polarization component are substantially the same, and thus the luminance ratio H Is about l. In contrast, when the road surface Is completely wet, the reflected light intensity Ip of the horizontal polarization component Is much larger than the reflected light intensity Is of the vertical polarization component, and thus, the luminance ratio Is large. Further, when the road surface is slightly wet, the luminance ratio H is intermediate between the above cases. Therefore, the wet state of the road surface can be determined from the value of the luminance ratio H.
The white line detecting unit 3-5 functions as a line detecting unit and detects a white line on a road surface based on the luminance information calculated by the monochrome luminance information processing unit 3-3.
The road surface edge detection unit 3-7 detects and distinguishes the road surface edge from the roadside building based on the white line information obtained by the white line detection unit 3-5 and the polarization ratio information obtained by the polarization ratio information processing unit 3-4.
The road surface edge detection unit 3-7 scans a running road surface by adopting light beams, and segments images on the same scanning line according to different polarization rates of the road surface and a roadside building to obtain image information of the road surface edge on the scanning line, image information of the roadside building and corresponding polarization ratio information of the road surface edge and the roadside building;
comparing the difference value between the polarization ratio of the white line pixel information on the same scanning line and the polarization ratio information of the road surface edge image with a set threshold value, judging that the image data meets the threshold value condition as image data of the road surface edge, and transmitting the obtained data to a vehicle control unit for vehicle control;
and comparing the difference value between the polarization ratio of the white line pixel information on the same scanning line and the polarization ratio information of the roadside building image with a set threshold value, judging that the threshold value condition is met as the image data of the roadside building, and sending the obtained data to the display unit 1-2 for display.
The white line detected by the white line detecting unit 3-5 and the road surface edge detected by the road surface edge detecting unit 3-7 are displayed on the display unit 1-2, and the display unit 1-2 is provided with a CRT or liquid crystal display so as to be easily viewable by the driver. The data obtained by the road surface edge detection unit 3-7 may be transmitted to the vehicle control unit 1-3 for vehicle control. The shape information storage unit 3-6 receives and stores the information transmitted from the road surface edge detection unit 3-7.
The first polarized image memory 3-1, the second polarized image memory 3-2, the monochrome luminance information processing unit 3-3, the polarization ratio information processing unit 3-4, the white line detection unit 3-5, and the road surface edge detection unit 3-7 constitute an image processing unit.
According to a method described later, the edge of the white line is detected based on the obtained luminance information. The polarization ratio of the detected pixels within the white line, i.e., reference pixels, is set as the reference polarization ratio for scanning, and the polarized image is scanned based on the reference polarization ratio, each pixel of the polarized image having the polarization ratio. The road surface edge detection unit 3-7 scans using the light beam. Each line of pixels processed and generating polarization ratio information by the polarization ratio information processing unit 3-4 is also referred to as a scan line. The scan lines represent horizontal rows of pixels on the display to be scanned by the electron beam, the rows of pixels going from the left end to the right end.
The pixels on each scan line are sequentially processed in the left-right direction. The polarization ratio of the pixels on the same scan line as the reference pixels is compared with the corresponding reference polarization ratio. If the difference between the polarization ratio of the pixel and the reference polarization ratio is less than a predetermined threshold, a next pixel on the same scan line is processed. Meanwhile, if the difference is equal to or greater than the threshold value, the pixel is detected as a road surface edge point.
In this example, the polarization ratio of the pixels in the white line, i.e., the reference pixels, is used as the reference polarization ratio for scanning to reduce the influence of shadows generated by objects such as preceding vehicles. Roadside buildings are also protected from misunderstanding of road edges. Alternatively, the polarization ratio of the pixels located at the center of the respective scan lines, that is, the center of the polarized image, may be used as a reference polarization ratio to detect the road surface edge. When white line edges and road surface edges are detected, the scan lines are scanned from the bottom of the image where it is more reliable to the top of the image or from the x-axis or vertical direction of the screen.
After detecting the point indicating the white line edge and the point indicating the road surface edge in one screen or image, the approximate curves of the white line edge point and the road surface edge point are obtained through shape approximation. The approximate curve is obtained by the road surface edge detection unit 3-7, which also serves as an approximate curve acquisition unit. For example, a least squares method, a hough transform, or a model equation may be used for shape approximation. When an approximation curve is obtained by shape approximation, a high weight is given to reliable white line edge points and road surface edge points detected in the road image or in the lower part of the screen. In this way, even if the wrong white line edge point and the wrong road surface edge point are detected in the upper region of the road image, the lane line can be appropriately identified as long as the lane line edge point is correctly detected in the lower region of the road image.
In detecting white lines and road edges in real time, if similar white lines and similar edges are found in one or more images or polarized images acquired previously, it is determined that the detected white lines and road edges are reliable. According to the position of the white line in the previous frame, searching the white line edge and the road surface edge in the next frame and drawing lines. If the positions of the white line edge and the road surface edge are not detected in the five-frame image, the search is restarted from the center of the scanning line below the image.
As shown in fig. 2, a flow chart of lane line detection performed by the vehicle-mounted imaging system according to the present invention includes the following specific flow:
the imaging unit 1-1 captures an image of a scene in the traveling direction in front of the vehicle, and obtains a vertical polarization component, a horizontal polarization component as original polarization image data. The obtained horizontally polarized image data is stored in the first polarized image memory 3-1, and the obtained vertically polarized image data is stored in the second polarized image memory 3-2. The horizontally polarized image data and the vertically polarized image data are sent to the monochrome luminance information processing unit 3-3 to perform monochrome luminance information processing, and luminance information representing the pixel luminance level of the generated monochrome luminance image is calculated and obtained. The horizontally polarized image data and the vertically polarized image data are also sent to the polarization ratio information processing unit 3-4, used as polarization ratio information generation, and calculate and obtain polarization ratio information of the polarized image pixels based on the P component and the S component.
The polarization ratio information processing unit 3-4 calculates polarization ratio information using the formula (1), thereby obtaining polarization ratio information data. The polarization ratio represents a ratio between polarization components, and can be calculated using formula (2), and luminance information image data is generated and output using formula (3).
The road surface shape estimating unit 4-1 detects a planar line shape formed on a traveling road surface using the obtained polarization ratio information of the polarization image, and divides the road surface and roadside buildings positioned on both sides of the road surface and forming a certain angle with the road surface using a set threshold value to obtain an estimated road surface shape including a road surface inclination and a width;
the lane line candidate detection unit 4-2 detects candidate points of a possible lane line edge, also referred to as lane line candidate points, based on the polarization ratio information. The lane lines may represent any type of line (e.g., solid, dashed, or double lines) of any color (e.g., white or yellow) separating roadway or tram lanes. The lane line detection unit 4-4 detects lane lines on a road surface based on the polarization ratio information, and the ordinary asphalt road surface is black, and white lines are formed on the black road surface. The polarization ratio of the white line is close to zero, so the polarization ratio of the white line is sufficiently smaller than that of the other parts of the road, and the white line can be detected by determining the road where the polarization ratio is smaller than or equal to a predetermined value.
The lane line search area determining unit 4-3 obtains a lane line search area from the road surface inclination and width in the estimated road surface shape;
a lane line width is calculated based on the detected lane line candidate points, and it is determined whether the calculated white line width is within a predetermined range. If the calculated white line width is within the predetermined range, the lane line candidate point is determined as a white line edge on the road surface. Since the polarization ratio contrast between the lane line at the upper part of the image and the other part of the road surface is different from the polarization ratio contrast between the lane line at the lower part of the image and the other part of the road surface. Therefore, one frame of image is divided into an upper region and a lower region, and in the step of setting the polarization ratio threshold, different polarization ratio thresholds are set for the upper region and the lower region;
if no lane line edge is detected in the five-frame image, the search is started again from the center of the scanning line at the lower part of the image. If the lane line edge is successfully detected, storing the corresponding lane line information and road surface shape information in the obtained image into a shape information storage unit 3-6, and synchronously displaying the corresponding information in the form of an image on a display unit 1-2 so as to be convenient for a driver to check, and simultaneously sending the corresponding information to a vehicle control unit 1-3 for vehicle control.
It will be understood that the invention has been described in terms of several embodiments, and that various changes and equivalents may be made to these features and embodiments by those skilled in the art without departing from the spirit and scope of the invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from the essential scope thereof.
Claims (6)
1. The device for road detection and polarization imaging comprises a vehicle-mounted high-definition polarization camera imaging system, wherein the high-definition polarization camera imaging system comprises a polarization imaging unit, a first polarization image memory, a second polarization image memory and a display unit, and the polarization imaging unit is used for acquiring a first polarization image and a second polarization image of a vehicle driving road surface; the first polarized image memory is used for storing the acquired first polarized image; the second polarized image memory is used for storing the acquired second polarized image; the display unit displays each image; the method is characterized in that: the system comprises a white line detection unit, a pavement edge detection unit, a pavement shape estimation unit, a lane line searching area determination unit, a lane line candidate point detection unit, a lane line detection unit and a shape information storage unit;
the imaging system of the high-definition polarization camera further comprises a monochromatic brightness information processing unit and a polarization ratio information processing unit;
the single-color brightness information processing unit is used for generating a single-color brightness image and calculating and obtaining brightness information image data of brightness levels represented by pixels in the generated single-color brightness image based on a first polarization component and a second polarization component of the polarization image;
the polarization ratio information processing unit is used for obtaining polarization ratio information, and calculating and obtaining corresponding polarization ratio information data of each pixel based on the first polarization image and the second polarization image;
the white line detection unit is used for detecting and obtaining white line pixel information on a road surface from the brightness information image data obtained by the monochromatic brightness information processing unit;
the road surface edge detection unit scans a running road surface by adopting light beams to obtain a predicted image of the road surface edge, divides the image on the same scanning line, preliminarily divides the image information of the road surface edge and the image information of a roadside building, and finally respectively judges the image information of the road surface edge and the image information of the roadside building through a set threshold condition and sends corresponding image information data to the vehicle control unit for vehicle control;
the road surface shape estimating unit is used for estimating the shape of a road surface, detecting plane linearity formed on a running road surface by using polarization ratio information of each pixel, and dividing the road surface from roadside buildings which are positioned on two sides of the road surface and form a certain angle with the road surface;
the lane line search area determining unit is used for determining a lane line search area under the estimated road surface shape;
the lane line candidate point detection unit is used for judging lane line candidate points in the lane line searching area;
the lane line detection unit is used for determining a lane line in the lane line searching area, and the lane line detection unit is used for reducing the polarization ratio threshold value of the detected lane line in the lane line searching area when the lane line does not exist in the lane line searching area;
the shape information storage unit is used for storing the corresponding lane line information or the road surface shape information in the images obtained by the white line detection unit, the road surface edge detection unit and the road surface shape estimation unit, and synchronously displaying the corresponding information on the display unit in an image form so as to be convenient for a driver to check, and simultaneously sending the corresponding information to the vehicle control unit for vehicle control.
2. A method for road detection and polarization imaging using an apparatus for road detection and polarization imaging according to claim 1, characterized in that: comprising the following steps, and the following steps are carried out in sequence:
receiving first polarized light and second polarized light in reflected light of an object in an imaging area through a polarized imaging unit of a high-definition polarized camera imaging system, and obtaining first polarized images and second polarized images of road surfaces in front of a vehicle and storing the first polarized images and the second polarized images into corresponding memories respectively, wherein the first polarized light and the second polarized light have different polarization directions;
dividing the first polarized image and the second polarized image into a plurality of processing areas in a one-to-one correspondence manner by a monochromatic brightness information processing unit, respectively obtaining brightness level representation values of the first polarized image and the second polarized image in each processing area, and taking the sum of the brightness level representation values of the first polarized image and the second polarized image in the same processing area as a combined brightness level value for representing the processing area;
step three, a polarization ratio information processing unit obtains polarization ratio information of a first polarization image and a second polarization image, and obtains the ratio of the difference of brightness level characterization values of the first polarization image and the second polarization image in the same processing area to the combined brightness level value of the processing area;
step four, the white line detection unit extracts the brightness level value of the pixel satisfying the characteristic white line in the brightness level characteristic value, compares the brightness level value with a set threshold value, judges that the pixel corresponding to the value satisfying the set threshold value condition is white line pixel information on the road surface, and obtains the polarization ratio of the white line pixel information;
step five, the road surface edge detection unit scans the driving road surface by adopting light beams to obtain a predicted road surface edge image; dividing the image on the same scanning line according to different polarization rates of the road surface and the roadside building, and primarily dividing and obtaining image information of the road surface edge on the scanning line, image information of the roadside building and corresponding polarization ratio information of the road surface and the roadside building;
taking white line pixels obtained by a white line detection unit on the same scanning line as reference pixels;
comparing the difference value of the polarization ratio of the reference pixel information on the same scanning line and the polarization ratio information of the road surface edge image with a set threshold value, judging that the image data meets the threshold value condition as image data of the road surface edge, and transmitting the obtained data to a vehicle control unit for vehicle control;
comparing the difference value between the polarization ratio of the reference pixel information on the same scanning line and the polarization ratio information of the roadside building image with a set threshold value, judging that the threshold value condition is met as roadside building image data, and sending the obtained data to a display unit for display;
synchronously displaying the white line pixel information detected by the white line detection unit and the pavement edge image data detected by the pavement edge detection unit on a display unit in an image form so as to be convenient for a driver to check;
step seven, a road surface shape estimating unit detects plane linearity formed on a running road surface by using the polarization ratio information obtained in the step three, and divides the road surface and roadside buildings which are positioned on two sides of the road surface and form a certain angle with the road surface by using a set threshold value to obtain an estimated road surface shape, wherein the road surface shape comprises road surface gradient and width;
step eight, a lane line searching area determining unit obtains a lane line searching area according to the road surface gradient and the width in the estimated road surface shape;
step nine, a lane line candidate point detection unit detects a lane line on a road surface through polarization ratio information in an obtained lane line search area, and pixels meeting a set threshold condition are determined to be lane line candidate points;
step ten, the lane line detection unit judges whether the lane line candidate points are points on the lane line or not, calculates the width of a line where the lane line candidate points are located according to the points determined as the lane lines, judges the line as the lane line if the calculated line width is within a preset threshold range, and obtains the polarization ratio information of the lane line candidate points located at the edge of the lane line;
the lane line detection unit judges that no lane line is detected if no lane line candidate point exists in the lane line searching area or the width of a line where the lane line candidate point is located does not meet a set threshold value, and reduces the polarization ratio threshold value for detecting the lane line in the lane line searching area for next detection;
the shape information storage unit stores the corresponding lane line information or road surface shape information in the images obtained by the white line detection unit, the road surface edge detection unit and the road surface shape estimation unit, and synchronously displays the corresponding information on the display unit in an image form so as to be convenient for a driver to check, and simultaneously sends the corresponding information to the vehicle control unit for vehicle control.
3. A method for road detection and polarization imaging according to claim 2, characterized by: in the seventh step, the specific steps of dividing the road surface and the roadside building which is positioned at two sides of the road surface and forms a certain angle with the road surface are as follows: binarizing the polarization ratio information obtained in the third step, performing a marking process on the binarized polarization ratio information according to a threshold value of a predetermined parameter, obtaining continuous polarization ratio information having road surface characteristics, thereby obtaining an estimated road surface shape, wherein the road surface shape includes a road surface gradient and a width.
4. A method for road detection and polarization imaging according to claim 2, characterized by: the road surface edge detection unit determines the white line pixel detected in the polarization ratio information previously generated before the detection as the reference pixel under the condition that the white line pixel is not detected or under the condition that the road surface edge detection unit detects that the white line pixel ends in the middle of the vehicle running direction.
5. The method for road detection and polarization imaging of claim 4, wherein: the road surface edge detection unit takes pixels of an extension line of the white line pixels which are finished in the middle as reference pixels under the condition that the white line pixels detected in the polarization ratio information generated in advance are finished in the middle of the vehicle running direction;
the road surface edge detection unit takes a pixel located at the center of the road surface as a reference pixel under the condition that no white line pixel is detected in the polarization ratio information generated previously.
6. A method for road detection and polarization imaging according to claim 2, characterized by: the specific method for judging whether the lane line candidate point is a point on the lane line by the lane line detection unit in the step ten is as follows: the lane line detection unit divides the luminance image into an upper region and a lower region in the vehicle traveling direction, sets different luminance thresholds for the upper region and the lower region, and determines that the lane line candidate point is a point on the lane line by comparing the luminance level of each pixel in the upper region and the lower region with the corresponding luminance threshold one by one, the comparison result satisfying the set threshold range.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310304326.2A CN116363614A (en) | 2023-03-27 | 2023-03-27 | Device and method for road detection and polarization imaging |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310304326.2A CN116363614A (en) | 2023-03-27 | 2023-03-27 | Device and method for road detection and polarization imaging |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116363614A true CN116363614A (en) | 2023-06-30 |
Family
ID=86929902
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310304326.2A Pending CN116363614A (en) | 2023-03-27 | 2023-03-27 | Device and method for road detection and polarization imaging |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116363614A (en) |
-
2023
- 2023-03-27 CN CN202310304326.2A patent/CN116363614A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108960183B (en) | Curve target identification system and method based on multi-sensor fusion | |
| USRE48106E1 (en) | Detection of obstacles at night by analysis of shadows | |
| CN102668540B (en) | Imaging device, vehicle-mounted imaging system, road surface appearance detecting method and object detection apparatus | |
| US8831286B2 (en) | Object identification device | |
| RU2636120C2 (en) | Three-dimensional object detecting device | |
| US9569675B2 (en) | Three-dimensional object detection device, and three-dimensional object detection method | |
| RU2573110C1 (en) | On-board image recognition device | |
| CN100403332C (en) | Vehicle lane Robust identifying method for lane deviation warning | |
| RU2619059C2 (en) | Device for detecting three-dimensional objects and method for detecting three-dimensional objects | |
| CN107590470B (en) | Lane line detection method and device | |
| US20120147187A1 (en) | Vehicle detection device and vehicle detection method | |
| CN104899554A (en) | Vehicle ranging method based on monocular vision | |
| CN104520894A (en) | Roadside object detection device | |
| CN109635737B (en) | Auxiliary vehicle navigation positioning method based on road marking line visual identification | |
| CN102175613A (en) | Image-brightness-characteristic-based pan/tilt/zoom (PTZ) video visibility detection method | |
| KR20140109990A (en) | Determining a vertical profile of a vehicle environment by means of a 3d camera | |
| RU2635280C2 (en) | Device for detecting three-dimensional objects | |
| CN111694011A (en) | Road edge detection method based on data fusion of camera and three-dimensional laser radar | |
| Miclea et al. | Visibility detection in foggy environment | |
| Sasaki et al. | Development of ranging method for inter-vehicle distance using visible light communication and image processing | |
| CN103927523B (en) | Fog level detection method based on longitudinal gray features | |
| CN116363614A (en) | Device and method for road detection and polarization imaging | |
| CN115116025B (en) | Driving risk assessment system based on underground road vision distance | |
| CN112966782A (en) | Multi-view-angle feature-fused road surface water detection and identification method | |
| CN113963324A (en) | Road boundary identification method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |