CN111179152A - Road sign identification method and device, medium and terminal - Google Patents

Road sign identification method and device, medium and terminal Download PDF

Info

Publication number
CN111179152A
CN111179152A CN201811341411.1A CN201811341411A CN111179152A CN 111179152 A CN111179152 A CN 111179152A CN 201811341411 A CN201811341411 A CN 201811341411A CN 111179152 A CN111179152 A CN 111179152A
Authority
CN
China
Prior art keywords
dimensional
road
point cloud
cloud data
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811341411.1A
Other languages
Chinese (zh)
Other versions
CN111179152B (en
Inventor
吕天雄
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN201811341411.1A priority Critical patent/CN111179152B/en
Publication of CN111179152A publication Critical patent/CN111179152A/en
Application granted granted Critical
Publication of CN111179152B publication Critical patent/CN111179152B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G06T3/06—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/05—Geographic models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30256—Lane; Road marking

Abstract

The embodiment of the invention discloses a road sign identification method, a device, a medium and a terminal, wherein the identification method comprises the following steps: acquiring a three-dimensional point cloud data set, wherein the three-dimensional point cloud data set comprises a plurality of three-dimensional point cloud data points obtained by detecting a road; mapping the three-dimensional point cloud data set into a two-dimensional plane image; recognizing the road mark in the two-dimensional plane image by adopting a machine learning model; and inversely mapping the road identification in the two-dimensional plane image to the three-dimensional coordinate position. The technical scheme in the embodiment of the invention has higher accuracy.

Description

Road sign identification method and device, medium and terminal
Technical Field
The invention relates to the technical field of electronic maps, in particular to a road sign identification recognition method, a road sign identification recognition device, a medium and a terminal.
Background
An electronic map, i.e., a digital map, is a map that is stored and referred to digitally using computer technology, and is a map that is presented in a paperless manner using collected map data. Electronic maps are used in a wide range of applications, such as navigation or cruise processes.
The electronic map may include road signs of the road surface, and recognizing the road signs of the road surface is a common way to generate the road signs in the electronic map.
The accuracy of the existing road mark identification method needs to be improved.
Disclosure of Invention
The embodiment of the invention solves the technical problem of improving the accuracy of road identification.
In order to solve the above technical problem, an embodiment of the present invention provides a road sign identification method, which may include: acquiring a three-dimensional point cloud data set, wherein the three-dimensional point cloud data set comprises a plurality of three-dimensional point cloud data points obtained by detecting a road; mapping the three-dimensional point cloud data set into a two-dimensional plane image; recognizing the road mark in the two-dimensional plane image by adopting a machine learning model; and inversely mapping the road identification in the two-dimensional plane image to the three-dimensional coordinate position.
Optionally, the acquiring the three-dimensional point cloud data set includes: generating a three-dimensional frame at preset intervals along acquisition track information, wherein the acquisition track information is track information of equipment for acquiring the three-dimensional point cloud data points; acquiring three-dimensional point cloud data points in the three-dimensional frame; and obtaining the three-dimensional point cloud data set according to the three-dimensional point cloud data points in the three-dimensional frame.
Optionally, the cross section of the three-dimensional frame in the vertical height direction is square, and the preset interval is half of the side length of the square.
Optionally, the inverse mapping of the road identifier in the two-dimensional plane image to the three-dimensional coordinate further includes: and if the three-dimensional coordinate position of the road mark with the same type obtained by inverse mapping has an overlapped part on a plane vertical to the height direction, taking a circumscribed rectangle with the overlapped part as the three-dimensional coordinate position of the road mark.
Optionally, performing numerical value expansion by taking the road surface height value as a central value to determine the height range of the three-dimensional frame; the surface range of the three-dimensional frame is set according to the width of a road surface, and the surface range is a range which is perpendicular to the height in the three-dimensional frame.
Optionally, obtaining the three-dimensional point cloud data set according to the three-dimensional point cloud data points in the three-dimensional frame includes: performing plane fitting on the three-dimensional point cloud data points in the three-dimensional frame to obtain the height of a fitting plane; and acquiring three-dimensional point cloud data points with the height values and the height of the fitting plane within a preset range as the three-dimensional point cloud data set.
Optionally, mapping the three-dimensional point cloud data set into a two-dimensional plane image includes: and if the plurality of three-dimensional point cloud data points are mapped to the same coordinate in the two-dimensional plane image, taking the reflectivity average value of the plurality of three-dimensional point cloud data points as the reflectivity numerical value of the coordinate point.
Optionally, the mapping the three-dimensional point cloud data set into a two-dimensional plane image includes: orthogonally projecting the three-dimensional point cloud data set to obtain projection data; and carrying out affine transformation on the projection data to obtain the two-dimensional plane image.
Optionally, the recognizing, by using a machine learning model, the road identifier in the two-dimensional plane image includes: calculating the reflectivity range of three-dimensional point cloud data points related to the two-dimensional plane image, and normalizing the reflectivity range to a gray level image; and recognizing the gray level image by adopting a machine learning model to obtain the road mark in the two-dimensional plane image.
Optionally, recognizing the grayscale image by using a machine learning model includes: carrying out maximum value filtering on the gray level image to obtain a filtered gray level image; and recognizing the filtered gray level image by adopting a machine learning model to obtain the road identification in the two-dimensional plane image.
Optionally, the recognizing, by using a machine learning model, the road identifier in the two-dimensional plane image includes: and identifying the two-dimensional plane image by using a deep learning classification model.
Optionally, the recognizing, by using a machine learning model, the road identifier in the two-dimensional plane image includes: and determining the position and the identification type of the road identification in the two-dimensional plane image by adopting a machine learning model.
Optionally, the identifying, by using a machine learning model, the position of the road sign in the two-dimensional plane image includes: determining a road identifier identification position obtained by identifying the two-dimensional plane image by adopting a machine learning model; and scanning the maximum value distribution of pixel points in the edge range of the road identification position to determine the boundary of the road identification, wherein the edge range comprises the range for expanding the edge of the road identification position.
Optionally, the edge of the road sign recognition position includes: the outermost edge in the direction of the road.
Optionally, the inverse mapping the road identifier in the two-dimensional plane image to the three-dimensional coordinate position includes: inversely mapping the position of the road mark in the two-dimensional plane image to the two-dimensional coordinates of the point cloud through inverse affine transformation; and determining a height value according to three-dimensional cloud point data points adjacent to the data of the two-dimensional coordinate in the originally stored three-dimensional cloud point data points, wherein the two-dimensional coordinate is combined with the height value to generate a three-dimensional coordinate of the road identifier.
Optionally, the identification type includes at least one of the following: a left turn identification, a right turn identification, a straight line identification, and a text type identification.
Optionally, the method for identifying a road sign further includes: and associating the identification type of the road identification to the three-dimensional coordinate position.
Optionally, the position of the road sign in the two-dimensional plane image is a rectangle.
Optionally, mapping the three-dimensional point cloud data set into a two-dimensional plane image includes: mapping the three-dimensional point cloud data set to a PNG image, and compressing height information contained in position information in the three-dimensional point cloud data set to an alpha channel; inverse mapping the road sign within the two-dimensional planar image to a three-dimensional coordinate position comprises: and restoring the information in the alpha channel into a height coordinate of a three-dimensional coordinate position.
An embodiment of the present invention further provides a road sign recognition apparatus, including: the system comprises a point cloud data set acquisition unit, a road detection unit and a data acquisition unit, wherein the point cloud data set acquisition unit is suitable for acquiring a three-dimensional point cloud data set, and the three-dimensional point cloud data set comprises a plurality of three-dimensional point cloud data points obtained by detecting a road; the mapping unit is suitable for mapping the three-dimensional point cloud data set into a two-dimensional plane image; the recognition unit is suitable for recognizing the road mark in the two-dimensional plane image by adopting a machine learning model; and the inverse mapping unit is suitable for inversely mapping the road identification in the two-dimensional plane image to the three-dimensional coordinate position.
Optionally, the point cloud data set obtaining unit includes: the three-dimensional frame generating subunit is suitable for generating a three-dimensional frame at preset intervals along acquisition track information, wherein the acquisition track information is track information of equipment for acquiring the three-dimensional point cloud data points; the acquisition subunit is suitable for acquiring three-dimensional point cloud data points in the three-dimensional frame; and the data set determining subunit is suitable for obtaining the three-dimensional point cloud data set according to the three-dimensional point cloud data points in the three-dimensional frame.
Optionally, the cross section of the three-dimensional frame in the vertical height direction is square, and the preset interval is half of the side length of the square.
Optionally, the inverse mapping unit is further adapted to, after inversely mapping the road identifier in the two-dimensional plane image to the three-dimensional coordinate, if the three-dimensional coordinate position of the road identifier of the same type obtained by inverse mapping is located on a plane perpendicular to the height direction and has an overlapping portion, take a circumscribed rectangle having the overlapping portion as the three-dimensional coordinate position of the road identifier.
Optionally, the three-dimensional frame generation subunit is adapted to perform numerical expansion with the road surface height value as a central value to determine the height range of the three-dimensional frame; the surface range of the three-dimensional frame is set according to the width of a road surface, and the surface range is a range which is perpendicular to the height in the three-dimensional frame.
Optionally, the data set determining subunit includes: the plane fitting module is suitable for performing plane fitting on the three-dimensional point cloud data points in the three-dimensional frame to obtain the height of a fitting plane; and the data set module is suitable for acquiring three-dimensional point cloud data points with the height values and the height of the fitting plane within a preset range as the three-dimensional point cloud data set.
Optionally, the mapping unit is further adapted to: and when the plurality of three-dimensional point cloud data points are mapped to the same coordinate in the two-dimensional plane image, taking the reflectivity average value of the plurality of three-dimensional point cloud data points as the reflectivity numerical value of the coordinate point.
Optionally, the mapping unit includes: the projection subunit is suitable for orthogonally projecting the three-dimensional point cloud data set to obtain projection data; and the affine transformation subunit is suitable for carrying out affine transformation on the projection data to obtain the two-dimensional plane image.
Optionally, the identification unit includes: the normalization subunit is suitable for calculating the reflectivity range of the three-dimensional point cloud data points related to the two-dimensional plane image and performing normalization processing on the three-dimensional point cloud data points to obtain a gray level image; and the gray level image identification subunit is suitable for identifying the gray level image by adopting a machine learning model so as to obtain the road identification in the two-dimensional plane image.
Optionally, the grayscale image identifying subunit includes: the maximum filtering module is suitable for carrying out maximum filtering on the gray level image to obtain a filtered gray level image; and the recognition module is suitable for recognizing the filtered gray level image by adopting a machine learning model so as to obtain the road identification in the two-dimensional plane image.
Optionally, the recognition unit is adapted to recognize the two-dimensional plane image by using a deep learning classification model.
Optionally, the identification unit is adapted to determine the location and the type of the road sign within the two-dimensional plane image using a machine learning model.
Optionally, the identification unit includes: the road sign recognition position subunit is suitable for determining a road sign recognition position obtained by recognizing the two-dimensional plane image by adopting a machine learning model; and the scanning subunit is suitable for scanning the maximum value distribution of the pixel points in the edge range of the road identifier recognition position, and is suitable for determining the boundary of the road identifier, wherein the edge range comprises a range for expanding the edge of the road identifier recognition position.
Optionally, the edge of the road sign recognition position includes: the outermost edge in the direction of the road.
Optionally, the inverse mapping unit includes: an inverse affine transformation subunit adapted to inversely map, by inverse affine transformation, the position of the road sign within the two-dimensional planar image to the two-dimensional coordinates of the point cloud; and the three-dimensional coordinate generating subunit is suitable for determining a height value according to a three-dimensional cloud point data point adjacent to the data of the two-dimensional coordinate in the originally stored three-dimensional cloud point data points, and the two-dimensional coordinate is combined with the height value to generate a three-dimensional coordinate of the road identifier.
Optionally, the identification type includes at least one of the following: a left turn identification, a right turn identification, a straight line identification, and a text type identification.
Optionally, the road sign recognition apparatus further includes: and the association unit is suitable for associating the identification type of the road identification to the three-dimensional coordinate position.
Optionally, the position of the road sign in the two-dimensional plane image is a rectangle.
Optionally, the mapping unit is adapted to map a three-dimensional point cloud data set to a PNG image, and compress height information included in position information in the three-dimensional point cloud data set to an alpha channel; the inverse mapping unit is suitable for restoring the information in the alpha channel into a height coordinate of a three-dimensional coordinate position.
The embodiment of the invention also provides a computer readable storage medium, which stores computer instructions, and the computer instructions execute the steps of the road sign identification method when running.
The embodiment of the invention also provides a terminal, which comprises a memory and a processor, wherein the memory is stored with a computer instruction capable of running on the processor, and the step of the road identifier identification method is executed when the computer instruction runs.
Compared with the prior art, the technical scheme of the embodiment of the invention has the following beneficial effects:
in the embodiment of the invention, a three-dimensional point cloud data set is obtained, the three-dimensional point cloud data set is mapped into a two-dimensional plane image, and after a road identifier in the two-dimensional plane image is identified, the road identifier in the two-dimensional plane image is inversely mapped to a three-dimensional coordinate position. The machine learning model can be used for recognizing the road mark in the two-dimensional plane image, the machine learning model is used for recognizing the two-dimensional plane image, the adaptability of the recognition method is stronger by utilizing the diversity of training samples, the method is suitable for recognizing the two-dimensional plane image under different conditions, and the recognition result is more accurate, so that the accuracy of the road mark recognition method in the embodiment of the invention is higher.
Furthermore, the three-dimensional frames are set to be square, the preset interval is half of the side length of the three-dimensional frames, so that the three-dimensional frames are overlapped, the probability that the three-dimensional frames contain complete road identification marks can be improved, and the identification accuracy can be further improved.
Further, plane fitting is carried out on the three-dimensional point cloud data points in the three-dimensional frame to obtain the height of a fitting plane, the three-dimensional point cloud data points with the height values and the height of the fitting plane within a preset range are obtained to serve as the three-dimensional point cloud data set, the three-dimensional point cloud data can be cleaned, the three-dimensional point cloud data points near the road pavement height are screened out, data in the three-dimensional point cloud data set are more accurate, a follow-up identification process is carried out based on the more accurate three-dimensional point cloud data set, and the accuracy of the road identification method can be improved.
Furthermore, the gray level image obtained after normalization processing is carried out by calculating the reflectivity range of the three-dimensional point cloud data points related to the two-dimensional plane image, so that the method is more suitable for machine learning model identification, and further can improve the identification accuracy.
Furthermore, the gray level image is subjected to maximum value filtering, so that the filtered gray level image is subjected to mesh black holes caused by a scanning mode of three-dimensional point cloud data, the filtered gray level image is closer to the identification mark of a road surface, and the filtered gray level image is easier to be identified by a machine learning model, so that the filtered gray level image is identified, and the identification accuracy can be improved.
Drawings
Fig. 1 is a flowchart of a road sign recognition method according to an embodiment of the present invention;
FIG. 2 is a flow chart of one embodiment of the present invention for obtaining a three-dimensional point cloud data set;
FIG. 3 is a flowchart of a method for obtaining a three-dimensional point cloud data set from three-dimensional point cloud data points within a three-dimensional frame according to an embodiment of the present invention;
FIG. 4 is a schematic representation of an image form before affine transformation in an embodiment of the present invention;
FIG. 5 is a schematic representation of an affine transformed image form in an embodiment of the present invention;
FIG. 6 is a flow chart of one embodiment of identifying road signs in the two-dimensional plane image;
FIG. 7 is a flow chart of identifying the grayscale image according to one embodiment of the present invention;
FIG. 8 is a flow chart of one embodiment of identifying the location of a road sign within the two-dimensional planar image;
FIG. 9 is a flowchart illustrating a method for inverse mapping a road sign in a two-dimensional plane image to a three-dimensional coordinate position according to an embodiment of the present invention;
FIG. 10 is a diagram illustrating the result of fusing signatures in an embodiment of the present invention;
FIG. 11 is a schematic structural diagram of a road sign recognition apparatus according to an embodiment of the present invention;
FIG. 12 is a schematic structural diagram of a point cloud data collection obtaining unit according to an embodiment of the present invention;
FIG. 13 is a schematic structural diagram of a data set determination subunit according to an embodiment of the present invention;
FIG. 14 is a diagram illustrating a mapping unit according to an embodiment of the present invention;
FIG. 15 is a schematic structural diagram of an identification unit according to an embodiment of the present invention;
FIG. 16 is a schematic structural diagram of a gray scale image identification subunit according to an embodiment of the present invention;
FIG. 17 is a schematic diagram of another identification cell in an embodiment of the invention;
fig. 18 is a schematic structural diagram of an inverse mapping unit according to an embodiment of the present invention.
Detailed Description
As described above, the accuracy of the existing road sign recognition method needs to be improved.
In the road identification recognition method, an image of a road surface can be collected, and the road identification mark in the image of the road surface can be determined by recognizing the image of the road surface. The road surface image is usually recorded and played back manually, and the efficiency and the identification accuracy are low.
In another road mark identification method, three-dimensional point cloud data obtained by collecting road surfaces can be identified to obtain a road identification mark in the three-dimensional point cloud data. Compared with directly acquired image data, the method has the advantages that manual intervention is not needed in data acquisition, and efficiency is high. However, the accuracy of identifying the road identifier in the three-dimensional point cloud data still needs to be improved.
For example, when the ground mark is worn, blocked, stuck, etc., it is difficult to determine a complete ground mark area with a good accurate threshold, and this identification method usually requires a lane line to assist in generating a detection area. The requirement of high-precision map production is difficult to meet for the recognition precision and the automation degree of the road recognition mark.
In the embodiment of the invention, a three-dimensional point cloud data set is obtained, the three-dimensional point cloud data set is mapped into a two-dimensional plane image, and after a road identifier in the two-dimensional plane image is identified, the road identifier in the two-dimensional plane image is inversely mapped to a three-dimensional coordinate position.
By adopting the machine learning model to recognize the road mark in the two-dimensional plane image, the adaptability of the recognition method can be stronger by utilizing the diversity of the training samples, for example, the situations of abrasion, shielding, adhesion and the like of the ground mark can be covered in the training samples, so that the machine learning model can be suitable for recognizing the two-dimensional plane image under different situations, and the recognition result can be more accurate, therefore, the accuracy of the road mark recognition method in the embodiment of the invention is higher.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.
Fig. 1 is a flowchart of a road sign recognition method according to an embodiment of the present invention, which may specifically include steps S11 to S14.
Step S11, a three-dimensional point cloud data set is obtained, wherein the three-dimensional point cloud data set comprises a plurality of three-dimensional point cloud data points obtained by detecting a road.
The point cloud data may be a laser point cloud, and specifically, a laser radar (LiDAR) is used to obtain corresponding surface characteristics, such as reflectivity, for the spatial coordinates of each sampling point on the surface of the object in the same spatial reference system, and each sampling point may correspond to one three-dimensional point cloud data point.
In specific implementation, the three-dimensional point cloud data set may be point cloud data in a certain region range, or may also be point cloud data obtained by screening point cloud data in a certain region range.
And step S12, mapping the three-dimensional point cloud data set into a two-dimensional plane image.
The position coordinates of each data point in the three-dimensional point cloud data set are three-dimensional, each three-dimensional point cloud data point can be mapped to a coordinate point of the two-dimensional plane image, and the surface characteristic data, such as reflectivity data, of the three-dimensional point cloud data can be mapped to characteristic data, such as a gray value, of the plane image.
And step S13, recognizing the road mark in the two-dimensional plane image by adopting a machine learning model.
As described above, the three-dimensional point cloud data may include a plurality of data points, that is, three-dimensional point cloud data points, sampled by using a laser radar (LiDAR) in the same spatial reference system, and after the three-dimensional point cloud data set is converted into a two-dimensional plane image, the position information in the coordinate system may be restored from the coordinate points of the two-dimensional plane image, which is not possessed by the road surface image directly acquired, so that in an implementation manner of directly acquiring the road surface image, a video and manual intervention are generally required to be acquired.
In addition, the machine learning model is adopted to identify the road mark in the two-dimensional plane image, and the machine learning model is adopted to identify the two-dimensional plane image, so that the adaptability of the identification method is stronger by utilizing the diversity of training samples, the method is suitable for identifying the two-dimensional plane image under different conditions, and the identification result is more accurate.
The machine learning model can be a model adopted in a machine learning algorithm, an input image can be identified by adopting the machine learning model, and the input of the machine learning model can be an original two-dimensional plane image or an image obtained after the two-dimensional plane image is processed. The machine learning model recognizes the aforementioned input, and can determine the type of road sign in the two-dimensional plane image. The machine learning model may be any of various machine learning models that can be implemented by those skilled in the art, such as a deep learning model, or other models suitable for image recognition, and the like, and is not limited herein.
And step S14, inversely mapping the road mark in the two-dimensional plane image to the three-dimensional coordinate position.
As mentioned above, the two-dimensional plane may restore the position information, and the restored position information may be two-dimensional, and in a specific implementation, the three-dimensional position may be restored by determining a height value in the data near the two-dimensional coordinates in the originally stored three-dimensional point cloud data point and combining the height value.
In a specific implementation, referring to fig. 2, acquiring the three-dimensional point cloud data set may include steps S21 to S23.
And step S21, generating a three-dimensional frame at preset intervals along acquisition track information, wherein the acquisition track information is track information of equipment for acquiring the three-dimensional point cloud data points.
In a specific implementation, the device for acquiring the three-dimensional point cloud data can be a laser acquisition device and can be loaded on an acquisition vehicle. The trajectory information may be obtained from a Positioning point of the laser pickup device or a vehicle on which the laser pickup device is mounted by a Global Positioning System (GPS), and may be referred to as a pickup trajectory. Trajectory information may also be generated based on the positioning of other systems.
In particular implementations, the height range of the three-dimensional frame may be a range near the road surface, such as determining the road surface height as Z1The height range of the three-dimensional frame may be (Z)1+α，Z1α) wherein α may have a value of 0.5m, wherein the height Z of the road pavement is1The vehicle height may be subtracted from the height value in the GPS coordinates, and may be preset, and may be the height from the road surface of the location where the GPS device is mounted.
The section of the three-dimensional frame in the height direction can be square, the length of the side of the square can be not separated by half, and three-dimensional point cloud data can be acquired by the three-dimensional frame once. For example, the side length of the three-dimensional frame may be 20 meters, and the point cloud data in the three-dimensional frame may be acquired every 10 meters along the acquisition track.
And step S22, acquiring the three-dimensional point cloud data points in the three-dimensional frame.
As described above, a three-dimensional point cloud data point is a data point detected from a sampling point, which may include three-dimensional geographic location information and surface characteristics of the sampling point, such as reflectivity. The three-dimensional box may be a cubic box, a cross section of which perpendicular to the height direction may be a square, the three-dimensional box may define a spatial range, and the three-dimensional point cloud data points within the three-dimensional box may be data points whose positions fall within the three-dimensional box.
And step S23, obtaining the three-dimensional point cloud data set according to the three-dimensional point cloud data points in the three-dimensional frame.
In specific implementation, the three-dimensional point cloud data points in the three-dimensional frame can be further screened to obtain a three-dimensional point cloud data set.
For example, referring to fig. 3, deriving the three-dimensional point cloud data set from the three-dimensional point cloud data points within the three-dimensional frame may include:
step S31, performing plane fitting on the three-dimensional point cloud data points in the three-dimensional frame to obtain the height of a fitting plane;
and step S32, acquiring three-dimensional point cloud data points with the height values and the height of the fitting plane within a preset range as the three-dimensional point cloud data set.
In specific implementation, any method that can realize plane fitting by those skilled in the art can be used in step S31, and is not limited herein. If the height of the obtained fitting plane is Z2Then (Z) can be determined2+β，Z2β) is a preset range, (Z) is obtained by plane fitting2+β，Z2β) ratio (Z)1+α，Z1the range of-a) is more accurate.
Through the mode of firstly obtaining a rough height range and then further refining based on a plane fitting method, impurities in the data can be removed, for example, three-dimensional point cloud data points of objects with a cone barrel, guardrails and front and rear vehicles which are close to the ground at equal intervals can be removed, the influence of the impurity data on subsequent data is reduced, and the identification is more accurate.
After the plane fitting is performed, a plurality of three-dimensional point cloud data points can be obtained, and the position information of the three-dimensional point cloud data points can be changed from a (x, y, z) form to a (x, y, 0) form, wherein z is height information in the position information, and x and y are information of two dimensions of a two-dimensional plane perpendicular to the height direction respectively.
In specific implementation, when the three-dimensional point cloud data set is mapped to the two-dimensional plane image, if a plurality of three-dimensional point cloud data points are mapped to the same coordinate in the two-dimensional plane image, the reflectance mean value of the plurality of three-dimensional point cloud data points is taken as the reflectance value of the coordinate point.
For example, the position information of a plurality of three-dimensional data points is (x)3，y30), then these point cloud data points are mapped to the same coordinate in the two-dimensional planar image, where x3、y3Are specific values of two dimensions of the aforementioned two-dimensional plane. In such a scenario, the reflectance of the three-dimensional point cloud data points may be averaged, and the average value is referred to as the reflectance value of the two-dimensional plane image.
In a specific implementation, mapping the three-dimensional point cloud data set into a two-dimensional plane image may include: and performing orthogonal projection on the three-dimensional point cloud data set to obtain projection data, and performing affine transformation on the projection data to obtain the two-dimensional plane image.
The method of orthogonal projection may be to change the position information of the three-dimensional point cloud data point from (x, y, z) to (x, y, 0). The projection data may include a plurality of transformed data in a plurality of (x, y, 0) forms.
Referring to fig. 4 and 5, fig. 4 illustrates an image form before affine transformation, and fig. 5 illustrates an image form after affine transformation. In specific implementation, the affine transformation relation required for affine transformation can be established according to the position information of the data point cloud set.
For example, the affine transformation relationship may be established with the size and shape of the target image after the affine transformation according to the plane of the three-dimensional frame perpendicular to the height direction. The area range illustrated in fig. 4 may be a plane perpendicular to the height direction of the three-dimensional frame, and the area range illustrated in fig. 5 may be a size and a shape of the target image after the affine transformation. The affine transformation relationship may be presented in the form of an affine transformation matrix.
Affine transformation is carried out, the visual angles of projection data obtained after projection of different three-dimensional point cloud data sets can be unified, and data with the same visual angle can be identified during subsequent road identification, so that identification errors caused by non-unified standards can be avoided, and the identification accuracy rate is improved.
The coordinates of the three-dimensional point cloud data points in the plane perpendicular to the height direction can form a mapping relation with the coordinates of the data points in the two-dimensional image after affine transformation. Therefore, after the two-dimensional image is identified, the identification result can be inversely mapped to the three-dimensional coordinate position.
It will be understood by those skilled in the art that the graphs in fig. 4 and 5 are only illustrative and not limiting of the shape, reflectivity or gray scale value of the actual graph.
Referring to fig. 6, in a specific implementation, identifying road signs within the two-dimensional planar image using a machine learning model may include steps S61 and S62.
And step S61, calculating the reflectivity range of the three-dimensional point cloud data points related to the two-dimensional plane image, and normalizing the reflectivity range to obtain a gray level image.
In a specific implementation, the three-dimensional point cloud data points associated with the two-dimensional plane image may include each point cloud data point within a position range corresponding to the two-dimensional plane image.
Specifically, the point cloud data points may be data points after the plane fitting and data screening. Therefore, during normalization processing, impurities in the data can be prevented from influencing the result of the normalization processing, and the accuracy of subsequent identification can be higher.
In other embodiments, the data points may be data points obtained by obtaining the reflectance mean value through the aforementioned process when a plurality of three-dimensional point cloud data points are mapped to the same coordinate in the two-dimensional plane image. By calculating the reflectivity mean value, the data of the data point can be more accurate, and the identification result can be more accurate.
In a specific implementation, performing normalization processing may include: the maximum and minimum norm of the reflectivity of the two-dimensional image after affine transformation is obtained and mapped to a gray value space, for example, a numerical value space of 0 to 255.
The gray level image obtained after normalization processing is carried out by calculating the reflectivity range of the three-dimensional point cloud data points related to the two-dimensional plane image, so that the method is more suitable for machine learning model identification, and further can improve the identification accuracy.
And step S62, recognizing the gray level image by adopting a machine learning model to obtain the road mark in the two-dimensional plane image.
In a specific implementation, in the process of recognizing the grayscale image by using the machine learning model, the grayscale image may be processed first, and the processed grayscale image is recognized by using the machine learning model to obtain the road recognition identifier in the two-dimensional plane image.
For example, referring to fig. 7, identifying the grayscale image using a machine learning model may include:
step S71, carrying out maximum value filtering on the gray level image to obtain a filtered gray level image;
and step S72, recognizing the filtered gray-scale image by adopting a machine learning model to obtain a road mark in the two-dimensional plane image.
In specific implementation, when three-dimensional point cloud data is obtained, the laser acquisition equipment can acquire the numerical value of the reflectivity through the laser scanning line. Typically the scan lines may be interlaced, in which case if a two-dimensional image is acquired directly, it may include a mesh of black holes. These network black holes can be filled by maximum filtering of the grayscale image. The window size for maximum filtering of the grayscale image may be set to fill the mesh black hole.
By carrying out maximum value filtering on the gray level image, the mesh-shaped black holes caused by the scanning mode of the three-dimensional point cloud data in the filtered gray level image can be made, so that the filtered gray level image is closer to the identification mark of the road surface, and the filtered gray level image is easier to be identified by a machine learning model, so that the filtered gray level image is identified, and the identification accuracy can be improved.
In a specific implementation, identifying the road sign in the two-dimensional plane image by using the machine learning model may include: and identifying the two-dimensional plane image by using a deep learning classification model. Specifically, a Convolutional Neural Network (CNN) detection model, such as fast R-CNN, R-FCN, Mask R-CNN, etc., can be employed. Alternatively, other deep learning models suitable for image processing, which can be implemented by those skilled in the art, may be used for the recognition, and is not limited herein.
The two-dimensional plane image can be directly identified, or a gray scale image obtained by processing according to the method can be identified.
When training the deep learning model, can adopt diversified sample to train, for example can train as the sample with wearing and tearing, the sign that shelters from, gluing, so, when utilizing the deep learning model to discern, can promote the rate of accuracy of discernment.
In particular implementations, identifying the road marking within the two-dimensional planar image using a machine learning model may include identifying a location and a category of the road marking. The following further describes the present invention.
Referring to fig. 8, identifying the location of the road sign within the two-dimensional plane image may include step S81 and step S82.
And step S81, determining a road sign identification position obtained by identifying the two-dimensional plane image by adopting a machine learning model.
The machine learning model used for recognizing the image may be various, and for example, the deep learning classification model may be used for recognition. After recognition, a rough location of the road sign may be obtained, which may be referred to as a road sign recognition location.
Step S82, scanning the maximum value distribution of the pixel points in the edge range of the road sign identification position, and determining the boundary of the road sign, wherein the edge range comprises the range for expanding the edge of the road sign identification position.
The pixel points within the edge range may be pixel points whose distance from the edge of the road identifier recognition position is within a preset threshold range, with the edge as a center. The boundary of the road identification can be determined more accurately by scanning the pixel points in the edge range line by line, and the accuracy of the identification method is improved.
In a specific implementation, the position of the road sign in the two-dimensional plane image may be a position of a rectangle corresponding to the road sign. Specifically, the rectangle may be a circumscribed rectangle of the road sign. For example, referring to fig. 5, the position of the straight arrow may be the position indicated by the dashed box 51. Accordingly, the rectangle can be restored to a planar position in the three-dimensional coordinates perpendicular to the height direction.
In the scanning of the maximum value distribution, data within a preset range from the outer edge in the road direction, for example, data within a preset range from two sides in the AB direction of the dotted rectangular frame 51 in fig. 5, may be scanned.
In specific implementation, the result of the road sign identification can be used for generating an electronic map or for real-time cruising, and the boundary of the road sign in the road direction is more important, so that the boundary of the road sign in the road direction is more accurately determined, and a solid foundation can be laid for subsequent application.
Referring to fig. 9, inverse mapping the road sign within the two-dimensional plane image to the three-dimensional coordinate position may include:
step S91, inversely mapping the position of the road mark in the two-dimensional plane image to the two-dimensional coordinates of the point cloud through inverse affine transformation;
step S92, determining a height value according to the three-dimensional cloud point data points adjacent to the data of the two-dimensional coordinate in the originally stored three-dimensional cloud point data points, and generating the three-dimensional coordinate of the road sign by combining the two-dimensional coordinate with the height value.
The inverse affine transformation inverse mapping may be performed based on an affine transformation relationship applied in the affine transformation.
In specific implementation, the height value of the three-dimensional cloud point data point closest to the two-dimensional coordinate obtained by affine transformation and the corresponding two-dimensional coordinate in the originally stored three-dimensional cloud point data points can be used as the height value of the transformed three-dimensional coordinate.
In specific implementation, when the three-dimensional point cloud data set is mapped to a two-dimensional plane image, a 4-channel Portable Network Graphics (PNG) image may also be used to compress the ground height information to an alpha channel, so as to map the three-dimensional point cloud data to the two-dimensional image. Therefore, the height information, namely the z value, can be directly restored during inverse mapping, so that the z value can be prevented from being searched in the maintained three-dimensional point cloud data, and the identification efficiency is improved. Wherein the alpha channel is an 8-bit grayscale image channel.
As mentioned above, identifying the road sign in the two-dimensional plane image can obtain the position of the road sign and the sign type. In specific implementation, the identification type can be embodied in a three-dimensional coordinate position obtained by inversely mapping the road identification in the two-dimensional plane image. Or, when the position cannot embody the identification type, the identification type of the road identification may be associated to the three-dimensional coordinate position.
In particular implementations, the identification type can include at least one of: a left turn identification, a right turn identification, a straight line identification, and a text type identification. The identification type can be other types according to the requirements of the actual application scene. Correspondingly, the machine learning model is adopted to identify the road mark in the two-dimensional plane image, so that the mark type and the position of the rectangle corresponding to the road mark can be obtained.
In a specific implementation, when the data of the recognition result is stored, the data can be stored in the manner shown in table 1.
Wherein Obj _ id may be an index of the road identification mark, Left may be a center point of a Left boundary of the aforementioned rectangle, Top may be a center point of an upper boundary of the aforementioned rectangle, Right may be a center point of a Right boundary of the aforementioned rectangle, Bottom may be a center point of a lower boundary of the aforementioned rectangle, tpye may be a data type, Straight _ arrow and Left _ arrow may represent a Straight mark and a Left turn mark, respectively, and text may represent a text type mark.
Obj_id Left Top Right Bottom type
0000 X,y,z X,y,z X,y,z X,y,z Straight_arrow
0001 X,y,z X,y,z X,y,z X,y,z left_arrow
0002 X,y,z X,y,z X,y,z X,y,z text
TABLE 1
It will be understood by those skilled in the art that the storage means may be in other forms, and is not limited herein.
With continued reference to fig. 1, in a specific implementation, after inversely mapping the road identifier in the two-dimensional plane image to the three-dimensional coordinate, the method may further include: in step S15, if the three-dimensional coordinate positions of the road signs with the same type obtained by inverse mapping have overlapping parts on the plane perpendicular to the height direction, the circumscribed rectangle with the overlapping parts is taken as the three-dimensional coordinate position of the road sign.
For example, referring to fig. 10 in combination, if the rectangle DELK is the position of one road sign on the plane perpendicular to the height direction, and the rectangle CFHI is the position of the other road sign, and the road signs of the two are of the same type, the rectangle CFHI may be taken as the position of the road sign on the plane. The coordinates in the height direction may be determined by locating CFHI in the circumscribed rectangular frame by taking the points of CFHI on the circumscribed rectangular frame, for example, the midpoints of CF, FH, HI, CI on the four sides, respectively, and then determining the height coordinates of the midpoints in combination with the points adjacent to the midpoints in the originally stored three-dimensional point cloud data points.
By means of the method, the result of the identification mark is fused, the condition that the position information of the identification mark is inaccurate due to the fact that the three-dimensional frame does not contain the complete road identification mark can be avoided, and identification accuracy is improved.
Those skilled in the art can understand that, the specific implementation manners of the steps in the foregoing embodiments may be selected and combined according to the needs of an actual scenario, and are not limited herein.
In the embodiment of the invention, a three-dimensional point cloud data set is obtained, the three-dimensional point cloud data set is mapped into a two-dimensional plane image, and after a road identifier in the two-dimensional plane image is identified, the road identifier in the two-dimensional plane image is inversely mapped to a three-dimensional coordinate position. The machine learning model can be used for recognizing the road mark in the two-dimensional plane image, the machine learning model is used for recognizing the two-dimensional plane image, the adaptability of the recognition method is stronger by utilizing the diversity of training samples, the method is suitable for recognizing the two-dimensional plane image under different conditions, and the recognition result is more accurate, so that the accuracy of the road mark recognition method in the embodiment of the invention is higher.
The embodiment of the present invention further provides a road sign recognition apparatus, which has a schematic structural diagram as shown in fig. 11, and specifically includes:
a point cloud data set acquisition unit 111 adapted to acquire a three-dimensional point cloud data set including a plurality of three-dimensional point cloud data points obtained by detecting a road;
a mapping unit 112 adapted to map the three-dimensional point cloud data set into a two-dimensional plane image;
a recognition unit 113 adapted to recognize a road sign within the two-dimensional plane image using a machine learning model;
an inverse mapping unit 114 adapted to inverse map the road marking within the two-dimensional plane image to a three-dimensional coordinate position.
In a specific implementation, referring to fig. 12, the point cloud data set obtaining unit 111 may include:
a three-dimensional frame generating subunit 121 adapted to generate a three-dimensional frame at preset intervals along acquisition trajectory information, which is trajectory information of a device that acquires the three-dimensional point cloud data points;
an acquiring subunit 122 adapted to acquire three-dimensional point cloud data points within the three-dimensional frame;
and the data set determining subunit 123 is adapted to obtain the three-dimensional point cloud data set according to the three-dimensional point cloud data points in the three-dimensional frame.
In a specific implementation, the cross section of the three-dimensional frame in the vertical height direction is square, and the preset interval is half of the side length of the square.
With continued reference to fig. 11, in a specific implementation, the inverse mapping unit 114 is further adapted to, after inversely mapping the road identifier in the two-dimensional plane image to the three-dimensional coordinate, if the three-dimensional coordinate position of the road identifier with the same type obtained by inverse mapping has an overlapped portion on a plane perpendicular to the height direction, take a circumscribed rectangle with the overlapped portion as the three-dimensional coordinate position of the road identifier.
In a specific implementation, the height range of the three-dimensional frame can be obtained by performing numerical expansion by taking the road surface height value as a central value; the surface range of the three-dimensional frame is set according to the width of a road surface, and the surface range is a range which is perpendicular to the height in the three-dimensional frame.
With combined reference to fig. 12 and 13, in a specific implementation, the data set determining subunit 123 may include:
the plane fitting module 131 is adapted to perform plane fitting on the three-dimensional point cloud data points in the three-dimensional frame to obtain the height of a fitting plane;
and a data collection module 132 adapted to obtain three-dimensional point cloud data points with a height value and a height of the fitting plane within a preset range as the three-dimensional point cloud data collection.
With continued reference to fig. 11, in a specific implementation, the mapping unit 114 is further adapted to: and when the plurality of three-dimensional point cloud data points are mapped to the same coordinate in the two-dimensional plane image, taking the reflectivity average value of the plurality of three-dimensional point cloud data points as the reflectivity numerical value of the coordinate point.
Referring to fig. 14 and fig. 11 in combination, the mapping unit 114 may include:
a projection subunit 141, adapted to orthogonally project the three-dimensional point cloud data set to obtain projection data;
and an affine transformation subunit 142, adapted to perform affine transformation on the projection data to obtain the two-dimensional plane image.
With combined reference to fig. 15 and fig. 11, in a specific implementation, the identification unit 113 may include:
a normalization subunit 151 adapted to calculate a reflectivity range of a three-dimensional point cloud data point related to the two-dimensional plane image, and normalize the reflectivity range to a grayscale image;
a grayscale image identifying subunit 152 adapted to identify the grayscale image by using a machine learning model to obtain a road identifier in the two-dimensional plane image.
Referring to fig. 15 and 16 in combination, in a specific implementation, the grayscale image identifying subunit 152 may include:
a maximum filtering module 161, adapted to perform maximum filtering on the grayscale image to obtain a filtered grayscale image;
an identifying module 162 adapted to identify the filtered grayscale image using a machine learning model to obtain a road identifier in the two-dimensional plane image.
With continued reference to fig. 11, the recognition unit 113 is adapted to recognize the two-dimensional planar image using a deep learning classification model.
In a specific implementation, the recognition unit 113 is adapted to determine the location of the road sign and the sign type within the two-dimensional plane image using a machine learning model.
With combined reference to fig. 11 and fig. 17, in a specific implementation, the identification unit 113 may include:
a road sign recognition position subunit 171 that determines a road sign recognition position obtained by recognizing the two-dimensional planar image by using a machine learning model;
and the scanning subunit 172, configured to scan a maximum distribution of pixel points within an edge range of the road identifier identification position, and determine a boundary of the road identifier, where the edge range includes a range in which an edge of the road identifier identification position is extended.
In particular implementations, the edge of the road sign identification location may include: the outermost edge in the direction of the road.
With combined reference to fig. 11 and fig. 18, in a specific implementation, the inverse mapping unit 114 may include:
an inverse affine transformation sub-unit 181 adapted to inversely map the position of the road sign within the two-dimensional plane image to the two-dimensional coordinates of the point cloud by inverse affine transformation;
the three-dimensional coordinate generating subunit 182 is adapted to determine a height value according to a three-dimensional cloud point data point adjacent to the two-dimensional coordinate data point in the originally stored three-dimensional cloud point data points, and the two-dimensional coordinate generates a three-dimensional coordinate of the road identifier by combining the height value.
In a specific implementation, the identification type may include at least one of: a left turn identification, a right turn identification, a straight line identification, and a text type identification.
In a specific implementation, the road sign recognition device may further comprise an association unit adapted to associate the sign type of the road sign to the three-dimensional coordinate position.
In a specific implementation, the position of the road sign within the two-dimensional plane image may be a rectangle.
With continued reference to fig. 11, in an implementation, the mapping unit 112 is adapted to map the three-dimensional point cloud data set to the PNG image, and compress the height information included in the position information in the three-dimensional point cloud data set to an alpha channel; the inverse mapping unit 114 is adapted to restore the information in the alpha channel to the height coordinate of the three-dimensional coordinate position.
The explanation, the principle, the specific implementation and the beneficial effects of the noun related to the road sign recognition device in the embodiment of the present invention may refer to the road sign recognition method in the embodiment of the present invention, and are not described herein again.
The embodiment of the invention also provides a computer readable storage medium, on which computer instructions are stored, and when the computer instructions are executed, the steps of the road sign identification method can be executed.
The computer readable storage medium may be an optical disc, a mechanical hard disk, a solid state hard disk, etc.
The embodiment of the invention also provides a terminal, which comprises a memory and a processor, wherein the memory is stored with computer instructions capable of running on the processor, and the steps of the road sign identification method can be executed when the computer instructions run.
The terminal may be a portable computer with data processing capability, such as a vehicle-mounted smart device, a smart phone, a tablet computer, or the like, a single computer, a server, or a server cluster.
Although the present invention is disclosed above, the present invention is not limited thereto. Various changes and modifications may be effected therein by one skilled in the art without departing from the spirit and scope of the invention as defined in the appended claims.

Claims (40)

1. A road sign recognition method is characterized by comprising the following steps:
acquiring a three-dimensional point cloud data set, wherein the three-dimensional point cloud data set comprises a plurality of three-dimensional point cloud data points obtained by detecting a road;
mapping the three-dimensional point cloud data set into a two-dimensional plane image;
recognizing the road mark in the two-dimensional plane image by adopting a machine learning model;
and inversely mapping the road identification in the two-dimensional plane image to the three-dimensional coordinate position.
2. The road sign recognition method of claim 1, wherein the obtaining a three-dimensional point cloud data set comprises:
generating a three-dimensional frame at preset intervals along acquisition track information, wherein the acquisition track information is track information of equipment for acquiring the three-dimensional point cloud data points;
acquiring three-dimensional point cloud data points in the three-dimensional frame;
and obtaining the three-dimensional point cloud data set according to the three-dimensional point cloud data points in the three-dimensional frame.
3. The method for identifying road signs as claimed in claim 2, wherein the cross section of the three-dimensional frame in the vertical height direction is a square, and the predetermined interval is half of the side length of the square.
4. The method for recognizing road signs according to claim 2, wherein the inverse mapping of the road signs in the two-dimensional plane image to three-dimensional coordinates further comprises: and if the three-dimensional coordinate position of the road mark with the same type obtained by inverse mapping has an overlapped part on a plane vertical to the height direction, taking a circumscribed rectangle with the overlapped part as the three-dimensional coordinate position of the road mark.
5. The road sign recognition method according to claim 2, wherein the height range of the three-dimensional frame is determined by performing numerical expansion with the road surface height value as a center value; the surface range of the three-dimensional frame is set according to the width of a road surface, and the surface range is a range which is perpendicular to the height in the three-dimensional frame.
6. The method of claim 2, wherein obtaining the three-dimensional point cloud data set from the three-dimensional point cloud data points within the three-dimensional frame comprises:
performing plane fitting on the three-dimensional point cloud data points in the three-dimensional frame to obtain the height of a fitting plane;
and acquiring three-dimensional point cloud data points with the height values and the height of the fitting plane within a preset range as the three-dimensional point cloud data set.
7. The method of claim 1, wherein mapping the three-dimensional point cloud data set to a two-dimensional planar image comprises: and if the plurality of three-dimensional point cloud data points are mapped to the same coordinate in the two-dimensional plane image, taking the reflectivity average value of the plurality of three-dimensional point cloud data points as the reflectivity numerical value of the coordinate point.
8. The method of claim 1, wherein the mapping the three-dimensional point cloud data set to a two-dimensional plane image comprises:
orthogonally projecting the three-dimensional point cloud data set to obtain projection data;
and carrying out affine transformation on the projection data to obtain the two-dimensional plane image.
9. The method according to claim 1, wherein the recognizing the road sign in the two-dimensional plane image by using the machine learning model comprises:
calculating the reflectivity range of three-dimensional point cloud data points related to the two-dimensional plane image, and normalizing the reflectivity range to a gray level image;
and recognizing the gray level image by adopting a machine learning model to obtain the road mark in the two-dimensional plane image.
10. The method of claim 9, wherein recognizing the grayscale image using a machine learning model comprises:
carrying out maximum value filtering on the gray level image to obtain a filtered gray level image;
and recognizing the filtered gray level image by adopting a machine learning model to obtain the road identification in the two-dimensional plane image.
11. The method according to claim 1, wherein the recognizing the road sign in the two-dimensional plane image by using the machine learning model comprises: and identifying the two-dimensional plane image by using a deep learning classification model.
12. The method according to claim 1, wherein the recognizing the road sign in the two-dimensional plane image by using the machine learning model comprises: and determining the position and the identification type of the road identification in the two-dimensional plane image by adopting a machine learning model.
13. The method of claim 12, wherein the identifying the location of the road sign within the two-dimensional planar image using the machine learning model comprises:
determining a road identifier identification position obtained by identifying the two-dimensional plane image by adopting a machine learning model;
and scanning the maximum value distribution of pixel points in the edge range of the road identification position to determine the boundary of the road identification, wherein the edge range comprises the range for expanding the edge of the road identification position.
14. The method of claim 13, wherein the edge of the road sign recognition location comprises: the outermost edge in the direction of the road.
15. The method of claim 12, wherein the inverse mapping of the road sign in the two-dimensional plane image to a three-dimensional coordinate location comprises:
inversely mapping the position of the road mark in the two-dimensional plane image to the two-dimensional coordinates of the point cloud through inverse affine transformation;
and determining a height value according to three-dimensional cloud point data points adjacent to the data of the two-dimensional coordinate in the originally stored three-dimensional cloud point data points, wherein the two-dimensional coordinate is combined with the height value to generate a three-dimensional coordinate of the road identifier.
16. The method of claim 12, wherein the type of identification comprises at least one of: a left turn identification, a right turn identification, a straight line identification, and a text type identification.
17. The road sign recognition method according to claim 12, further comprising: and associating the identification type of the road identification to the three-dimensional coordinate position.
18. The method according to claim 12, wherein the position of the road sign in the two-dimensional plane image is a rectangle.
19. The method of claim 1, wherein mapping the three-dimensional point cloud data set to a two-dimensional planar image comprises: mapping the three-dimensional point cloud data set to a PNG image, and compressing height information contained in position information in the three-dimensional point cloud data set to an alpha channel;
inverse mapping the road sign within the two-dimensional planar image to a three-dimensional coordinate position comprises: and restoring the information in the alpha channel into a height coordinate of a three-dimensional coordinate position.
20. A road sign recognition device, comprising:
the system comprises a point cloud data set acquisition unit, a road detection unit and a data acquisition unit, wherein the point cloud data set acquisition unit is suitable for acquiring a three-dimensional point cloud data set, and the three-dimensional point cloud data set comprises a plurality of three-dimensional point cloud data points obtained by detecting a road;
the mapping unit is suitable for mapping the three-dimensional point cloud data set into a two-dimensional plane image;
the recognition unit is suitable for recognizing the road mark in the two-dimensional plane image by adopting a machine learning model;
and the inverse mapping unit is suitable for inversely mapping the road identification in the two-dimensional plane image to the three-dimensional coordinate position.
21. The road sign recognition device of claim 20, wherein the point cloud data set acquisition unit includes:
the three-dimensional frame generating subunit is suitable for generating a three-dimensional frame at preset intervals along acquisition track information, wherein the acquisition track information is track information of equipment for acquiring the three-dimensional point cloud data points;
the acquisition subunit is suitable for acquiring three-dimensional point cloud data points in the three-dimensional frame;
and the data set determining subunit is suitable for obtaining the three-dimensional point cloud data set according to the three-dimensional point cloud data points in the three-dimensional frame.
22. The road sign recognition device of claim 21, wherein the cross section of the three-dimensional frame in the vertical height direction is a square, and the predetermined interval is half of the side length of the square.
23. The road sign recognition device of claim 21, wherein the inverse mapping unit is further adapted to, after inverse mapping the road sign in the two-dimensional plane image to the three-dimensional coordinates, if the three-dimensional coordinate positions of the road signs of the same type obtained by inverse mapping have an overlapping portion on a plane perpendicular to the height direction, take a circumscribed rectangle having the overlapping portion as the three-dimensional coordinate position of the road sign.
24. The road sign recognition device of claim 21, wherein the three-dimensional frame generation subunit is adapted to determine the height range of the three-dimensional frame by performing numerical expansion with the road surface height value as a center value; the surface range of the three-dimensional frame is set according to the width of a road surface, and the surface range is a range which is perpendicular to the height in the three-dimensional frame.
25. The road sign recognition device of claim 21, wherein the data set determination subunit comprises:
the plane fitting module is suitable for performing plane fitting on the three-dimensional point cloud data points in the three-dimensional frame to obtain the height of a fitting plane;
and the data set module is suitable for acquiring three-dimensional point cloud data points with the height values and the height of the fitting plane within a preset range as the three-dimensional point cloud data set.
26. The road sign recognition device of claim 20, wherein the mapping unit is further adapted to: and when the plurality of three-dimensional point cloud data points are mapped to the same coordinate in the two-dimensional plane image, taking the reflectivity average value of the plurality of three-dimensional point cloud data points as the reflectivity numerical value of the coordinate point.
27. The road sign recognition device of claim 20, wherein the mapping unit comprises:
the projection subunit is suitable for orthogonally projecting the three-dimensional point cloud data set to obtain projection data;
and the affine transformation subunit is suitable for carrying out affine transformation on the projection data to obtain the two-dimensional plane image.
28. The road sign recognition device of claim 20, wherein the recognition unit comprises:
the normalization subunit is suitable for calculating the reflectivity range of the three-dimensional point cloud data points related to the two-dimensional plane image and performing normalization processing on the three-dimensional point cloud data points to obtain a gray level image;
and the gray level image identification subunit is suitable for identifying the gray level image by adopting a machine learning model so as to obtain the road identification in the two-dimensional plane image.
29. The road sign recognition device of claim 28, wherein the grayscale image recognition subunit comprises:
the maximum filtering module is suitable for carrying out maximum filtering on the gray level image to obtain a filtered gray level image;
and the recognition module is suitable for recognizing the filtered gray level image by adopting a machine learning model so as to obtain the road identification in the two-dimensional plane image.
30. The road sign recognition device of claim 20, wherein the recognition unit is adapted to recognize the two-dimensional plane image using a deep learning classification model.
31. The road sign recognition device of claim 20, wherein the recognition unit is adapted to determine the location of the road sign within the two-dimensional planar image and the sign type using a machine learning model.
32. The road sign recognition device of claim 31, wherein the recognition unit comprises:
the road sign recognition position subunit is suitable for determining a road sign recognition position obtained by recognizing the two-dimensional plane image by adopting a machine learning model;
and the scanning subunit is suitable for scanning the maximum value distribution of the pixel points in the edge range of the road identifier recognition position, and is suitable for determining the boundary of the road identifier, wherein the edge range comprises a range for expanding the edge of the road identifier recognition position.
33. The road sign recognition device of claim 32, wherein the edge of the road sign recognition location comprises: the outermost edge in the direction of the road.
34. The road sign recognition device of claim 31, wherein the inverse mapping unit comprises:
an inverse affine transformation subunit adapted to inversely map, by inverse affine transformation, the position of the road sign within the two-dimensional planar image to the two-dimensional coordinates of the point cloud;
and the three-dimensional coordinate generating subunit is suitable for determining a height value according to a three-dimensional cloud point data point adjacent to the data of the two-dimensional coordinate in the originally stored three-dimensional cloud point data points, and the two-dimensional coordinate is combined with the height value to generate a three-dimensional coordinate of the road identifier.
35. The road sign recognition device of claim 31, wherein the sign type comprises at least one of: a left turn identification, a right turn identification, a straight line identification, and a text type identification.
36. The road sign recognition device of claim 31, further comprising: and the association unit is suitable for associating the identification type of the road identification to the three-dimensional coordinate position.
37. The road sign recognition device of claim 31, wherein the position of the road sign within the two-dimensional planar image is a rectangle.
38. The road sign recognition device of claim 20, wherein the mapping unit is adapted to map a three-dimensional point cloud data set to a PNG image, and compress height information contained in position information in the three-dimensional point cloud data set to an alpha channel;
the inverse mapping unit is suitable for restoring the information in the alpha channel into a height coordinate of a three-dimensional coordinate position.
39. A computer-readable storage medium having stored thereon computer instructions, wherein the computer instructions are operable to perform the steps of the road sign recognition method of any one of claims 1 to 19.
40. A terminal comprising a memory and a processor, said memory having stored thereon computer instructions executable on said processor, wherein said computer instructions when executed perform the steps of the road sign recognition method of any one of claims 1 to 19.
CN201811341411.1A 2018-11-12 2018-11-12 Road identification recognition method and device, medium and terminal Active CN111179152B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811341411.1A CN111179152B (en) 2018-11-12 2018-11-12 Road identification recognition method and device, medium and terminal

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811341411.1A CN111179152B (en) 2018-11-12 2018-11-12 Road identification recognition method and device, medium and terminal

Publications (2)

Publication Number Publication Date
CN111179152A true CN111179152A (en) 2020-05-19
CN111179152B CN111179152B (en) 2023-04-28

Family

ID=70655263

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811341411.1A Active CN111179152B (en) 2018-11-12 2018-11-12 Road identification recognition method and device, medium and terminal

Country Status (1)

Country Link
CN (1) CN111179152B (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111695486A (en) * 2020-06-08 2020-09-22 武汉中海庭数据技术有限公司 High-precision direction signboard target extraction method based on point cloud
CN112102409A (en) * 2020-09-21 2020-12-18 杭州海康威视数字技术股份有限公司 Target detection method, device, equipment and storage medium
CN112132853A (en) * 2020-11-30 2020-12-25 湖北亿咖通科技有限公司 Method and device for constructing ground guide arrow, electronic equipment and storage medium
CN112446884A (en) * 2020-11-27 2021-03-05 广东电网有限责任公司肇庆供电局 Method and device for positioning power transmission line in laser point cloud and terminal equipment
CN112507891A (en) * 2020-12-12 2021-03-16 武汉中海庭数据技术有限公司 Method and device for automatically identifying high-speed intersection and constructing intersection vector
CN112683169A (en) * 2020-12-17 2021-04-20 深圳依时货拉拉科技有限公司 Object size measuring method, device, equipment and storage medium
CN112907746A (en) * 2021-03-25 2021-06-04 上海商汤临港智能科技有限公司 Method and device for generating electronic map, electronic equipment and storage medium
CN113205447A (en) * 2021-05-11 2021-08-03 北京车和家信息技术有限公司 Road picture marking method and device for lane line identification
WO2021238283A1 (en) * 2020-05-25 2021-12-02 华为技术有限公司 Positioning method and positioning apparatus
CN113808142A (en) * 2021-08-19 2021-12-17 高德软件有限公司 Ground identifier identification method and device and electronic equipment
CN114485671A (en) * 2022-01-24 2022-05-13 轮趣科技(东莞)有限公司 Automatic turning method and device for mobile equipment
CN114973910A (en) * 2022-07-27 2022-08-30 禾多科技（北京）有限公司 Map generation method and device, electronic equipment and computer readable medium
CN113808142B (en) * 2021-08-19 2024-04-26 高德软件有限公司 Ground identification recognition method and device and electronic equipment

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140098199A1 (en) * 2010-03-10 2014-04-10 Shapequest, Inc. Systems and methods for 2D image and spatial data capture for 3D stereo imaging
CN104766058A (en) * 2015-03-31 2015-07-08 百度在线网络技术（北京）有限公司 Method and device for obtaining lane line
CN107194962A (en) * 2017-04-01 2017-09-22 深圳市速腾聚创科技有限公司 Point cloud and plane picture fusion method and device
CN108107444A (en) * 2017-12-28 2018-06-01 国网黑龙江省电力有限公司检修公司 Substation's method for recognizing impurities based on laser data
CN108256446A (en) * 2017-12-29 2018-07-06 百度在线网络技术（北京）有限公司 For determining the method, apparatus of the lane line in road and equipment
CN108470159A (en) * 2018-03-09 2018-08-31 腾讯科技（深圳）有限公司 Lane line data processing method, device, computer equipment and storage medium
CN108573522A (en) * 2017-03-14 2018-09-25 腾讯科技（深圳）有限公司 A kind of methods of exhibiting and terminal of flag data

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140098199A1 (en) * 2010-03-10 2014-04-10 Shapequest, Inc. Systems and methods for 2D image and spatial data capture for 3D stereo imaging
CN104766058A (en) * 2015-03-31 2015-07-08 百度在线网络技术（北京）有限公司 Method and device for obtaining lane line
CN108573522A (en) * 2017-03-14 2018-09-25 腾讯科技（深圳）有限公司 A kind of methods of exhibiting and terminal of flag data
CN107194962A (en) * 2017-04-01 2017-09-22 深圳市速腾聚创科技有限公司 Point cloud and plane picture fusion method and device
CN108107444A (en) * 2017-12-28 2018-06-01 国网黑龙江省电力有限公司检修公司 Substation's method for recognizing impurities based on laser data
CN108256446A (en) * 2017-12-29 2018-07-06 百度在线网络技术（北京）有限公司 For determining the method, apparatus of the lane line in road and equipment
CN108470159A (en) * 2018-03-09 2018-08-31 腾讯科技（深圳）有限公司 Lane line data processing method, device, computer equipment and storage medium

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
MAHDI JAVANMARDI等: "Precise mobile laser scanning for urban mapping utilizing 3D aerial surveillance data" *
孔栋;孙亮;王建强;王晓原;: "基于3D激光雷达点云的道路边界识别算法" *
彭江帆: "基于车载激光扫描数据的高速公路道路要素提取方法研究" *

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021238283A1 (en) * 2020-05-25 2021-12-02 华为技术有限公司 Positioning method and positioning apparatus
CN111695486A (en) * 2020-06-08 2020-09-22 武汉中海庭数据技术有限公司 High-precision direction signboard target extraction method based on point cloud
CN111695486B (en) * 2020-06-08 2022-07-01 武汉中海庭数据技术有限公司 High-precision direction signboard target extraction method based on point cloud
CN112102409A (en) * 2020-09-21 2020-12-18 杭州海康威视数字技术股份有限公司 Target detection method, device, equipment and storage medium
CN112102409B (en) * 2020-09-21 2023-09-01 杭州海康威视数字技术股份有限公司 Target detection method, device, equipment and storage medium
CN112446884A (en) * 2020-11-27 2021-03-05 广东电网有限责任公司肇庆供电局 Method and device for positioning power transmission line in laser point cloud and terminal equipment
CN112446884B (en) * 2020-11-27 2024-03-26 广东电网有限责任公司肇庆供电局 Positioning method and device for power transmission line in laser point cloud and terminal equipment
CN112132853A (en) * 2020-11-30 2020-12-25 湖北亿咖通科技有限公司 Method and device for constructing ground guide arrow, electronic equipment and storage medium
CN112507891A (en) * 2020-12-12 2021-03-16 武汉中海庭数据技术有限公司 Method and device for automatically identifying high-speed intersection and constructing intersection vector
CN112683169A (en) * 2020-12-17 2021-04-20 深圳依时货拉拉科技有限公司 Object size measuring method, device, equipment and storage medium
CN112907746A (en) * 2021-03-25 2021-06-04 上海商汤临港智能科技有限公司 Method and device for generating electronic map, electronic equipment and storage medium
CN113205447A (en) * 2021-05-11 2021-08-03 北京车和家信息技术有限公司 Road picture marking method and device for lane line identification
CN113808142A (en) * 2021-08-19 2021-12-17 高德软件有限公司 Ground identifier identification method and device and electronic equipment
CN113808142B (en) * 2021-08-19 2024-04-26 高德软件有限公司 Ground identification recognition method and device and electronic equipment
CN114485671A (en) * 2022-01-24 2022-05-13 轮趣科技(东莞)有限公司 Automatic turning method and device for mobile equipment
CN114973910A (en) * 2022-07-27 2022-08-30 禾多科技（北京）有限公司 Map generation method and device, electronic equipment and computer readable medium
CN114973910B (en) * 2022-07-27 2022-11-11 禾多科技（北京）有限公司 Map generation method and device, electronic equipment and computer readable medium

Also Published As

Publication number Publication date
CN111179152B (en) 2023-04-28

Similar Documents

Publication Publication Date Title
CN111179152B (en) Road identification recognition method and device, medium and terminal
Dhiman et al. Pothole detection using computer vision and learning
US10217007B2 (en) Detecting method and device of obstacles based on disparity map and automobile driving assistance system
WO2018068653A1 (en) Point cloud data processing method and apparatus, and storage medium
CN109165549B (en) Road identification obtaining method based on three-dimensional point cloud data, terminal equipment and device
CN112257605B (en) Three-dimensional target detection method, system and device based on self-labeling training sample
CN111626277B (en) Vehicle tracking method and device based on over-station inter-modulation index analysis
GB2532948A (en) Objection recognition in a 3D scene
CN101842808A (en) Method of and apparatus for producing lane information
CN110197173B (en) Road edge detection method based on binocular vision
CN104239867A (en) License plate locating method and system
CN107808524B (en) Road intersection vehicle detection method based on unmanned aerial vehicle
US20230005278A1 (en) Lane extraction method using projection transformation of three-dimensional point cloud map
CN112070756B (en) Three-dimensional road surface disease measuring method based on unmanned aerial vehicle oblique photography
CN109635737A (en) Automobile navigation localization method is assisted based on pavement marker line visual identity
CN115049700A (en) Target detection method and device
CN112308913A (en) Vision-based vehicle positioning method and device and vehicle-mounted terminal
CN116168246A (en) Method, device, equipment and medium for identifying waste slag field for railway engineering
Hu Intelligent road sign inventory (IRSI) with image recognition and attribute computation from video log
Yao et al. Automatic extraction of road markings from mobile laser-point cloud using intensity data
JP2009217832A (en) Method and device for automatically recognizing road sign in video image, and storage medium which stores program of road sign automatic recognition
CN114299247A (en) Rapid detection and problem troubleshooting method for road traffic sign lines
CN111537954A (en) Real-time high-dynamic fusion positioning method and device
CN111611942B (en) Method for extracting and building database by perspective self-adaptive lane skeleton
CN116152761B (en) Lane line detection method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant