CN113297878B - Road intersection identification method, device, computer equipment and storage medium - Google Patents

Road intersection identification method, device, computer equipment and storage medium Download PDF

Info

Publication number
CN113297878B
CN113297878B CN202010110375.9A CN202010110375A CN113297878B CN 113297878 B CN113297878 B CN 113297878B CN 202010110375 A CN202010110375 A CN 202010110375A CN 113297878 B CN113297878 B CN 113297878B
Authority
CN
China
Prior art keywords
street view
view image
identification
boundary
road intersection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010110375.9A
Other languages
Chinese (zh)
Other versions
CN113297878A (en
Inventor
夏德国
张刘辉
杨建忠
曹雪卉
姜海林
李崎玮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202010110375.9A priority Critical patent/CN113297878B/en
Publication of CN113297878A publication Critical patent/CN113297878A/en
Application granted granted Critical
Publication of CN113297878B publication Critical patent/CN113297878B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/52Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/54Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/26Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/56Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)

Abstract

The application provides a road intersection identification method, a device, computer equipment and a storage medium, and relates to the technical field of image processing, wherein the method comprises the following steps: the method comprises the steps of obtaining a street view image, identifying the street view image by adopting a semantic segmentation model to determine a vehicle driving area from the street view image, and identifying whether the street view image contains a road intersection according to the boundary bending degree of the vehicle driving area so as to accurately identify the road intersection and the surrounding information of the road intersection, and simultaneously determining the position of the road intersection according to the GPS information of each image.

Description

Road intersection identification method, device, computer equipment and storage medium
Technical Field
The application relates to the technical field of artificial intelligence, in particular to the technical field of image processing, and specifically relates to a road intersection identification method, a device, computer equipment and a storage medium.
Background
With the increasing complexity of urban roads and the rapid popularization of smart phones, mobile phone navigation software has become a necessary tool for users to travel. Wherein, the map data is the basis and core of map navigation. As urban road construction increases and the accuracy requirements of users on map data become more stringent, it becomes important to collect urban road information more accurately and rapidly.
The roads in the map data use the intersections as traction points, and the trafficability of the roads around the intersections is described. The description of road intersections is an important component of map data and road navigation. Thus, in the process of collecting road information, detection of road intersections is very important.
The following ways are generally used in the related art to detect intersections: (1) The data collected by the laser range finder, the laser radar and the monocular camera are combined for detection, but the equipment such as the laser range finder is high in price, the detection system is complex, the cost is high, and a far-away road intersection cannot be detected. (2) The road intersection is detected by using the GPS aggregation information of the vehicles, but the road intersection detection mode needs to acquire a large amount of GPS data on the detected road, and the road with sparse GPS can not accurately detect the road intersection and the surrounding information of the road intersection.
Disclosure of Invention
The present application aims to solve at least one of the technical problems in the related art to some extent.
Therefore, a first object of the present application is to propose a road intersection recognition method, which performs semantic segmentation recognition on sequentially collected multi-frame street view images to determine a vehicle driving area, and recognizes whether the street view images contain intersections based on the boundary bending degree of the vehicle driving area, so as to accurately recognize the road intersections and the surrounding information of the intersections, and determine the positions of the road intersections according to the GPS information of each image.
A second object of the present application is to provide a road intersection recognition device.
A third object of the application is to propose a computer device.
A fourth object of the present application is to propose a non-transitory computer readable storage medium.
A fifth object of the application is to propose a computer programme product.
To achieve the above object, an embodiment of a first aspect of the present application provides a method for identifying a road intersection, including:
acquiring a street view image;
identifying the street view image by adopting a semantic segmentation model so as to determine a vehicle driving area from the street view image;
and identifying whether the street view image contains a road intersection or not according to the boundary bending degree of the vehicle driving area.
To achieve the above object, a second aspect of the present application provides a road intersection recognition device, including:
the acquisition module is used for acquiring street view images;
the segmentation module is used for identifying the street view image by adopting a semantic segmentation model so as to determine a vehicle driving area from the street view image;
and the identification module is used for identifying whether the street view image contains a road intersection or not according to the boundary bending degree of the vehicle driving area.
To achieve the above object, an embodiment of a third aspect of the present application provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to implement the method for identifying a road intersection according to the first aspect.
To achieve the above object, a fourth aspect of the present application provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the road intersection identification method according to the first aspect.
To achieve the above object, an embodiment of a fifth aspect of the present application proposes a computer program product comprising a computer program which, when executed by a processor, implements the road intersection identification method according to the first aspect.
The technical scheme provided by the embodiment of the application can comprise the following beneficial effects:
the method comprises the steps of obtaining a street view image, identifying the street view image by adopting a semantic segmentation model to determine a vehicle driving area from the street view image, and identifying whether the street view image contains a road intersection according to the boundary bending degree of the vehicle driving area so as to accurately identify the road intersection and the surrounding information of the road intersection, and simultaneously determining the position of the road intersection according to the GPS information of each image.
Additional aspects and advantages of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application.
Drawings
The foregoing and/or additional aspects and advantages of the application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
fig. 1 is a schematic flow chart of a road intersection recognition method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a continuous multi-frame street view image provided by the application;
FIG. 3 is a schematic diagram of a street view image after semantic segmentation according to an embodiment of the present application;
fig. 4 is a flow chart of another method for identifying a road intersection according to an embodiment of the present application;
FIG. 5 is a schematic diagram of determining a degree of bending of a boundary according to an embodiment of the present application;
FIG. 6 is a schematic diagram of another determination of the degree of bending of a boundary according to an embodiment of the present application;
FIG. 7 is a flowchart of another method for identifying a road intersection according to the present application;
FIG. 8 is a schematic diagram of the street view image screening according to the degree of boundary curvature provided by the application;
FIG. 9 is a schematic diagram of the result of identifying traffic objects at an intersection location provided by the present application;
Fig. 10 is a schematic structural diagram of a road intersection recognition device according to an embodiment of the present application; and
fig. 11 is a block diagram of an electronic device of a road intersection recognition method according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present application and should not be construed as limiting the application.
The following describes a road intersection recognition method, apparatus, computer device, and storage medium according to embodiments of the present application with reference to the accompanying drawings.
Fig. 1 is a flow chart of a road intersection recognition method according to an embodiment of the present application.
As shown in fig. 1, the method comprises the steps of:
step 101, obtaining street view images.
The street view images are sequentially collected multi-frame images.
Specifically, the street view image is acquired by a driver driving a road information acquisition vehicle through a camera arranged on the acquisition vehicle, and the road in the street view image contains various information such as a road intersection, a zebra crossing, a traffic light, an electronic eye and the like.
Fig. 2 is a schematic diagram of continuous multi-frame street view images provided by the application, and fig. 2 shows a continuous track diagram of street view images collected during vehicle-mounted driving, wherein 6 continuous track diagrams are numbered 1,2,3,4,5 and 6 according to the collection sequence. In the actual acquisition process, the number of the continuously acquired multi-frame images can be set according to specific situations, and the method is not limited in this embodiment.
And 102, identifying the street view image by adopting a semantic segmentation model so as to determine a vehicle driving area from the street view image.
As a possible implementation manner, the collected street view images are input into a trained semantic segmentation model, the semantic segmentation model performs semantic segmentation on each street view image by taking pixels as units, pixels representing the same category are combined, and buildings, vehicles, fences on roads, road edges, driving roads and the like contained in the street view images can be clearly identified in the finally output semantic segmented images so as to determine the vehicle driving areas in the street view images.
Fig. 3 is a schematic diagram of a street view image after semantic segmentation provided in an embodiment of the present application, fig. 3 is a diagram obtained by performing semantic segmentation on a continuous multi-frame street view image corresponding to fig. 2, as shown in fig. 3, a plurality of classifications identified after the semantic segmentation in the street view image are marked in the street view image with the number 5, including sky, building, lane lines, zebra stripes, ground arrows and driving roads, different gray scales in fig. 3 represent different classifications obtained by the semantic segmentation, and for a clearer explanation, the street view image marked with 5 in fig. 3 is taken as an example, and the arrows specifically indicate the different classifications obtained by the segmentation, wherein each arrow indicates one classification obtained by the semantic segmentation, however, in practical application, more classifications may be identified, which are only schematically listed here and do not constitute a limitation on the result obtained by the semantic segmentation in the embodiment.
And step 103, identifying whether the street view image contains a road intersection or not according to the boundary bending degree of the vehicle driving area.
Specifically, for each frame of street view image, an identification feature is extracted, wherein the identification feature comprises the boundary bending degree of a vehicle driving area in the corresponding street view image and comprises difference information of the boundary bending degree of the adjacent street view image, the extracted identification feature is input into a classification model, and the classification model is learned to obtain the mapping relation between the identification feature and the road intersection identification result, so that whether the corresponding street view image contains a road intersection or not can be identified according to the classification model.
Further, as the road continuous street view images acquired by the acquisition vehicle also have corresponding geographic position information, wherein the geographic position information is GPS information. Each street view image has corresponding GPS information, so that after the street view image containing the road intersection is obtained through identification, the GPS position of the street view image containing the road intersection is the GPS position of the road intersection, the position information of the road intersection is determined while the road intersection is determined, the determined road intersection and the position information can be added into a geographic information database, and the geographic information database is updated.
In the road intersection identification method provided by the embodiment of the application, the street view image is acquired through the camera, the street view image is identified by adopting the semantic segmentation model, so that the vehicle driving area is determined from the street view image, whether the street view image contains the road intersection or not is identified according to the boundary bending degree of the vehicle driving area, so that the road intersection and the surrounding information of the intersection can be accurately identified, and meanwhile, the position of the road intersection can be determined according to the GPS information of each image, and the cost for identifying the road intersection is reduced because expensive laser range finders, laser radars and other equipment are not adopted.
Based on the above embodiment, the present embodiment provides a possible implementation manner of the road intersection recognition method, which describes how to determine the degree of boundary curvature at each boundary point of the driving area after determining the driving area of the vehicle in the street view image by the semantic segmentation model. Fig. 4 is a flowchart of another road intersection recognition method according to an embodiment of the present application.
As shown in fig. 4, following step 102, the following steps may be further included:
step 401, determining a degree of boundary curvature of a vehicle running area.
The following two possible implementations are provided in this embodiment for determining the degree of bending at the boundary point.
As a possible implementation manner, for each boundary point in the vehicle driving area, an identification frame with the corresponding boundary point as an angular point is determined, and the degree of boundary bending at the corresponding boundary point is determined according to the number of pixels belonging to the vehicle driving area in the identification frame. This is because the greater the degree of curvature of the boundary of the vehicle running area, the greater the number of pixels of the running area contained in the identification frame having the corresponding boundary point as the corner point. After the identification is performed through the semantic segmentation model, boundary pixel points of all the driving areas in the image can be found, then an M-by-N identification frame is drawn for each boundary pixel point, M and N are natural numbers larger than 1, the number of the pixel points of the driving areas contained in each identification frame is counted, the number of the pixel points indicates the boundary bending degree at the corresponding boundary, and the greater the number of the pixel points of the driving areas contained in the corresponding identification frame is, the greater the boundary bending degree of the corresponding boundary points is. For example, as shown in fig. 5, the number of pixels of the traveling area included in the identification frame in the diagram denoted by 4 is smaller than the number of pixels of the traveling area included in the identification frame corresponding to the boundary point in the street view image denoted by 5, that is, the degree of curvature of the boundary corresponding to the identification frame in the diagram denoted by 4 is smaller than the degree of curvature of the boundary corresponding to the identification frame in the diagram denoted by 5.
As another possible implementation manner, a plurality of boundary points in a vehicle driving area are grouped, wherein each group comprises a fixed number of boundary points which are adjacently arranged, the boundary points in the same group are fitted to obtain a fitting straight line, and for each group of boundary points, the boundary bending degree of each boundary point in the group is determined according to an included angle between the corresponding fitting straight line and the adjacent fitting straight line. For example, as shown in fig. 6, the gray part of the ground corresponds to the vehicle running area, the plurality of boundary points in the vehicle running area are grouped, for example, 20 boundary points are grouped into a group, the plurality of groups are divided in the vertical direction, for example, from top to bottom, in the horizontal direction, for example, from left to right, and the boundary points in the same group are fitted to obtain corresponding fitting straight lines, for example, a plurality of fitting straight lines indicated by white arrows corresponding to letters a-d in the right boundary in fig. 6, and when the fitting straight lines a and b corresponding to the boundaries of the running area indicate directions substantially coincide, the corresponding boundaries are smoother, that is, the degree of bending of the boundary at each boundary point in the fitting straight lines a group and b group is smaller. When the fitting straight line c and the fitting straight line d are reached forward, the direction angle indicated by the fitting straight line c and the fitting straight line d is larger in difference, namely the included angle between the fitting straight line c and the fitting straight line d is larger, so that the degree of bending of the boundary at each boundary point in the c group and the d group is larger.
The identification frame of the boundary point in fig. 5 and the fitted straight line of each group in fig. 6 are shown only schematically, and do not limit the present embodiment.
In the road intersection identification method of the embodiment, the boundary bending degree at the boundary points in the vehicle driving area can be identified by carrying out semantic segmentation on the continuous street view images, the boundary bending degree at each boundary point can be determined, and the greater the boundary bending degree is, the higher the possibility that the boundary bending degree is the road intersection is indicated, so that the boundary with the boundary bending degree larger than the threshold value is determined as the road intersection contained in the street view images, the intersection identification based on the bending degree at the boundary is realized, and the identification accuracy is improved.
Based on the above embodiments, the present embodiment provides a possible implementation manner of a road intersection identification method, which illustrates that in order to improve accuracy of road intersection identification, according to a degree of bending a border of a continuous multi-frame street view image, traffic objects presented around an intersection, and difference information between adjacent street view images, through interaction between multiple identification features, accuracy of identifying whether the street view image includes a road intersection is improved. Fig. 7 is a flow chart of another method for identifying a road intersection according to the present application.
As shown in fig. 7, step 103 may comprise the following sub-steps:
step 1031, screening the sequentially collected multi-frame street view images to reserve part of the street view images with the maximum value of the boundary bending degree larger than the set threshold value.
In the above embodiment, the bending degree of each boundary point in each frame of street view image is determined, in this embodiment, the image with the maximum value of the bending degree of each boundary point in each frame of street view image larger than the set threshold value is screened out, which indicates that the greater the bending degree of the boundary is, the greater the possibility that the image contains the road intersection is, so that the part of street view image with the maximum value of the bending degree of the boundary larger than the set threshold value can be used as the candidate street view image possibly containing the road intersection, so as to reduce the operation amount of road intersection identification by using the classification model in the subsequent step, and improve the efficiency and accuracy of road intersection identification.
It is to be understood that the set threshold value of the bending degree is smaller, so that the street view image possibly containing the road intersection is recalled, and the street view image is used for primary identification of whether the road intersection is contained in the street view image. And in the subsequent step, further accurately identifying whether the street view image contains the road intersection or not according to the determined identification characteristics.
As shown in fig. 8, the degree of curvature of the boundary point is determined in such a manner that the number of the traveling area pixels included in the boundary point identification frame is the same. In the street view images numbered 1,2,3 and 6, the boundary of the driving area is smoother, that is, the maximum value of the boundary bending degree of each boundary point is not greater than a set threshold value, and in the street view images numbered 4 and 5, the boundary of the driving area is obviously bent, that is, the maximum value of the boundary bending degree of each boundary point is greater than the set threshold value, so that the street view images numbered 4 and 5 are screened out and used as street view images possibly containing a road intersection, and further identification is carried out through subsequent steps to accurately determine whether the screened street view images really contain the road intersection or not, thereby improving the identification efficiency of the road intersection.
Step 1032 extracts the identification feature for each frame of street view image.
Specifically, for each frame of the screened street view image, the road intersection position is determined from the vehicle running area of the corresponding street view image, wherein the road intersection position is a section in which the degree of boundary curvature is greater than the set threshold in the vehicle running area of the corresponding street view image, because the region in which the degree of boundary curvature is greater is more likely to be the region in which the intersection is located, the section in which the degree of boundary curvature is greater than the set threshold in the vehicle running area can be taken as the road intersection position.
Further, according to the semantic segmentation model, traffic objects presented around each intersection position are determined, wherein the traffic objects comprise traffic marks and traffic lane facilities, and can also comprise traffic signal lamps, electronic eyes and the like, wherein the traffic marks are, for example, zebra crossings, stop lines and the like, the traffic lane facilities are, for example, lane lines, mark marks and the like, as shown in fig. 9, by taking a street view image marked as fig. 5 in fig. 9 as an example, after the semantic segmentation model, the left side of the intersection is identified to comprise the zebra crossings, and the left bottom side of the intersection comprises the stop lines and mark line-ground traffic arrows. The traffic objects presented around the determined intersection position are used as identification features, so that the accuracy of the intersection identification can be improved, the corresponding relation between the identified intersection and the traffic objects presented around is also determined, and the information contained in the identified intersection is increased. And for each frame of street view image, taking the traffic objects presented at the periphery of the road intersection, the boundary bending degree of the vehicle driving area, the difference information between the traffic objects in the adjacent street view images and the boundary bending degree in the adjacent street view images as the identification characteristics of the corresponding street view images.
Step 1033, inputting the extracted identification features into the classification model to identify whether the corresponding street view image contains the road intersection.
Specifically, the extracted identification features are input into the classification model, and the classification model learns to obtain the mapping relation between the identification features and the identification results of the road intersections, so that the street view images containing the road intersections can be accurately identified, and meanwhile, the identification features also contain traffic objects presented around the road intersections, so that after the street view images containing the road intersections are identified, the traffic objects contained in the street view images containing the road intersections can be obtained, for example, the street view images corresponding to fig. 5 are images containing the road intersections, and meanwhile, the zebra stripes are contained at the left side of the road intersections, and the stop lines and the mark marks (ground passing arrows) are contained at the lower left side, so that the road intersections contained in the street view images are accurately identified, the traffic objects contained around the road intersections are also obtained, and the richness of the navigation map information created later is improved.
Further, in order to improve accuracy of classification model identification, optionally, the identification features further comprise road intersection probabilities identified by the convolutional neural network according to the input street view images, wherein the convolutional neural network learns to obtain the mapping relationship between the identification features of the street view images and the road intersection probabilities.
Further, for each street view image, based on the identification features, a classification model obtained through training is adopted to predict and score whether the corresponding street view image contains a road intersection, namely whether the street view image contains the road intersection is identified, wherein a calculation formula adopted during identification of the classification model is as follows:S i =f(R i ,D i ,V i ,T i ) Wherein C is a recognition frame with all boundary points of road intersection positions in corresponding street view images determined according to semantic segmentation models as corner points, R is a recognition frame with all boundary points of road intersection positions in corresponding street view images i For the traffic objects around the identification frame corresponding to each boundary point, and the degree of boundary bending, D i The difference information of traffic objects in the adjacent street view images and the difference information of the bending degree of the boundary in the adjacent street view images. V (V) i And the probability of the road intersection output by the convolutional neural network is represented. T (T) i As a property feature of a road intersection, this part of the feature can be extracted from the existing database if the road intersectionThe attribute characteristics of (c) are not present in the database and may be set to default values. f is a rank function, and is a classification model obtained through supervised training, such as a gradient lifting iterative decision tree (Gradient Boosting Decision Tree, GBDT) model and the like, and the accuracy of classification model identification is improved by adding identification features.
In the road intersection identification method of the embodiment, after the vehicle driving area in the street view image is determined, the sequentially collected multi-frame street view image is screened to reserve a part of street view images with the maximum value of the boundary bending degree being larger than the set threshold value, so that the street view image containing the road intersection is screened out preliminarily, further, identification features are extracted from the screened image possibly containing the road intersection, traffic objects presented around the road intersection and the boundary bending degree of the vehicle driving area contained in the identification features, and difference information of the traffic objects in the adjacent street view image and the boundary bending degree in the adjacent street view image are input into the classification model, the street view image containing the road intersection is accurately identified by utilizing the classification model, and meanwhile, the traffic objects presented around the road intersection are also contained in the identification features.
Based on the above embodiment, as a possible implementation manner, the identifying feature may further include one or more combinations of an area of the vehicle driving area, a number of lanes included in the vehicle driving area, and a road class of a road section where the vehicle driving area is located, that is, the identifying feature is added, so that the identifying feature includes not only a traffic object presented around the road intersection included in the above embodiment, a degree of boundary curvature of the vehicle driving area, and difference information with respect to the traffic object in the neighboring street view image, and difference information with respect to a degree of boundary curvature in the neighboring street view image, but also one or more combinations of an area of the vehicle driving area, a number of lanes included in the vehicle driving area, and a road class of a road section where the vehicle driving area is located, and by adding the identifying feature in the input classification model, accuracy of identifying the road intersection may be improved.
In order to realize the embodiment, the application also provides a road intersection identification device.
Fig. 10 is a schematic structural diagram of a road intersection recognition device according to an embodiment of the present application.
As shown in fig. 10, the apparatus includes: an acquisition module 91, a segmentation module 92 and an identification module 93.
An acquiring module 91, configured to acquire a street view image.
The segmentation module 92 is configured to identify the street view image by using a semantic segmentation model, so as to determine a vehicle driving area from the street view image.
The identifying module 93 is configured to identify whether the street view image includes a road intersection according to the degree of bending of the boundary of the vehicle driving area.
Further, in one possible implementation manner of the embodiment of the present application, as one possible implementation manner, the apparatus further includes: and a determining module.
As a first possible implementation manner, the determining module is configured to determine, for each boundary point in the vehicle driving area, an identification frame that uses the corresponding boundary point as a corner point; and determining the boundary bending degree at the corresponding boundary point according to the number of the pixel points belonging to the vehicle driving area in the identification frame.
As a second possible implementation manner, the determining module is further configured to group a plurality of boundary points in the vehicle driving area, where each group includes a fixed number of boundary points that are adjacently arranged; fitting boundary points in the same group to obtain a fitting straight line; and for each group of boundary points, determining the boundary bending degree of each boundary point in the group according to the included angle between the corresponding fitting straight line and the adjacent fitting straight line.
As a possible implementation manner, the identifying module 93 includes:
the extraction unit is used for extracting identification features aiming at each frame of street view image; the identification feature comprises the boundary bending degree of the vehicle driving area in the corresponding street view image and comprises difference information of the boundary bending degree of the adjacent street view image.
The identification unit is used for inputting the extracted identification features into the classification model so as to identify whether the corresponding street view image contains the road intersection, wherein the classification model is learned to obtain the mapping relation between the identification features and the road intersection identification result.
As a possible implementation manner, the above identification module 93 further includes:
and the screening unit is used for screening the sequentially acquired multi-frame street view images so as to keep part of street view images with the maximum value of the boundary bending degree larger than a set threshold value.
As a possible implementation manner, the extracting unit is specifically configured to:
for each frame of street view image, determining the position of a road intersection from the vehicle driving area of the corresponding street view image; the road intersection position is a section of which the boundary bending degree in a vehicle driving area of a corresponding street view image is larger than a set threshold value, traffic objects presented around the road intersection position are determined according to a semantic segmentation model, and for each frame of street view image, the traffic objects presented around the road intersection, the boundary bending degree of the vehicle driving area, the difference information between the traffic objects in adjacent street view images and the difference information between the boundary bending degree in adjacent street view images are used as identification features of the corresponding street view images.
As one possible implementation, the traffic object includes a traffic marking and a traffic lane facility.
As one possible implementation, the identification feature further includes one or more combinations of an area of a vehicle driving area, a number of lanes included in the vehicle driving area, and a road class of a road segment on which the vehicle driving area is located.
As a possible implementation manner, the identifying feature further includes a road intersection probability identified by the convolutional neural network according to the input street view image, wherein the convolutional neural network has learned a mapping relationship between the identifying feature of the street view image and the road intersection probability.
It should be noted that the foregoing explanation of the embodiments of the road intersection identifying method is also applicable to the road intersection identifying device of this embodiment, and the principle is the same, and will not be repeated here.
In the road intersection identification device provided by the embodiment of the application, the street view image is acquired through the camera, the street view image is identified by adopting the semantic segmentation model, so that the vehicle driving area is determined from the street view image, whether the street view image contains the road intersection or not is identified according to the boundary bending degree of the vehicle driving area, so that the road intersection and the surrounding information of the intersection can be accurately identified, and meanwhile, the position of the road intersection can be determined according to the GPS information of each image, and the cost for identifying the road intersection is reduced because no expensive laser range finder, laser radar and other equipment is adopted.
In order to implement the above embodiments, the embodiments of the present application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the method for identifying an intersection according to the foregoing method embodiments when executing the program.
In order to achieve the above-described embodiments, an embodiment of the present application proposes a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the road intersection identification method according to the foregoing method embodiment.
In order to achieve the above embodiments, an embodiment of the present application proposes a computer program product comprising a computer program which, when executed by a processor, implements a road intersection identification method as described in the foregoing method embodiments.
According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.
As shown in fig. 11, there is a block diagram of an electronic device of a road intersection recognition method according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the applications described and/or claimed herein.
As shown in fig. 11, the electronic device includes: one or more processors 1001, memory 1002, and interfaces for connecting the components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the electronic device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple electronic devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 1001 is illustrated in fig. 11.
Memory 1002 is a non-transitory computer-readable storage medium provided by the present application. The memory stores instructions executable by the at least one processor to cause the at least one processor to perform the road intersection identification method provided by the application. The non-transitory computer-readable storage medium of the present application stores computer instructions for causing a computer to execute the road intersection identification method provided by the present application.
The memory 1002 is used as a non-transitory computer readable storage medium, and may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules (e.g., the acquisition module 91, the segmentation module 92, and the recognition module 93 shown in fig. 10) corresponding to the road intersection recognition method in the embodiment of the present application. The processor 1001 executes various functional applications of the server and data processing, i.e., implements the road intersection identification method in the above-described method embodiment, by running non-transitory software programs, instructions, and modules stored in the memory 1002.
Memory 1002 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created according to the use of the electronic device of the road junction identification method, or the like. In addition, the memory 1002 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1002 may optionally include memory remotely located with respect to the processor 1001, which may be connected to the electronic device of the road junction identification method via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device of the road intersection identification method may further include: an input device 1003 and an output device 1004. The processor 1001, memory 1002, input device 1003, and output device 1004 may be connected by a bus or other means, for example by a bus connection in fig. 11.
The input device 1003 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device of the road junction identification method, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a joystick, one or more mouse buttons, a track ball, a joystick, and the like. The output means 1004 may include a display device, auxiliary lighting means (e.g., LEDs), tactile feedback means (e.g., vibration motors), and the like. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device may be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASIC (application specific integrated circuit), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computing programs (also referred to as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the technical scheme of the embodiment of the application, the street view image is acquired, the street view image is identified by adopting the semantic segmentation model, so that the vehicle driving area is determined from the street view image, whether the street view image contains the road intersection or not is identified according to the boundary bending degree of the vehicle driving area, so that the road intersection and the surrounding information of the intersection can be accurately identified, and meanwhile, the position of the road intersection can be determined according to the GPS information of each image, and the cost for identifying the road intersection is reduced because expensive equipment such as a laser range finder and a laser radar is not adopted.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed embodiments are achieved, and are not limited herein.
The above embodiments do not limit the scope of the present application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of the present application.

Claims (18)

1. A method of identifying a roadway intersection, the method comprising:
acquiring a street view image;
identifying the street view image by adopting a semantic segmentation model so as to determine a vehicle driving area from the street view image;
the street view images are sequentially acquired multi-frames, and identification features are extracted for each frame of street view image; the identification features comprise the boundary bending degree of the vehicle driving area in the corresponding street view image and the difference information of the boundary bending degree of the adjacent street view image;
Inputting the extracted identification features into a classification model to identify whether the corresponding street view image contains a road intersection or not; wherein, the classification model is learned to obtain the mapping relation between the identification characteristics and the road intersection identification result.
2. The method of claim 1, wherein before extracting the identification feature for each frame of the street view image, further comprising:
and screening the sequentially acquired multi-frame street view images to reserve part of street view images with the maximum value of the boundary bending degree larger than a set threshold value.
3. The method of claim 1, wherein extracting the identification feature for each frame of the street view image comprises:
for each frame of the street view image, determining the position of a road intersection from the vehicle driving area of the corresponding street view image; the road intersection position is a section of the corresponding street view image, wherein the boundary bending degree of the vehicle driving area is greater than a set threshold value;
determining traffic objects presented around the position of each road intersection according to the semantic segmentation model;
and for each frame of the street view image, taking the traffic objects presented at the periphery of the road intersection, the boundary bending degree of the vehicle driving area, the difference information between the traffic objects in the adjacent street view image and the boundary bending degree in the adjacent street view image as the identification characteristics of the corresponding street view image.
4. The method of claim 3, wherein the traffic object comprises a traffic marking and a traffic lane departure facility.
5. The method of claim 1, wherein the identification features further comprise one or more combinations of an area of the vehicle travel area, a number of lanes contained in the vehicle travel area, and a road class of a road segment on which the vehicle travel area is located.
6. The method of claim 1, wherein the identifying features further comprise a road intersection probability identified by the convolutional neural network based on the input street view image;
the convolutional neural network learns to obtain the mapping relation between the recognition features of the street view image and the probability of the road intersection.
7. The method for identifying a road intersection according to any one of claims 1 to 6, wherein the identifying the street view image using a semantic segmentation model to determine a vehicle driving area from the street view image comprises:
for each boundary point in the vehicle driving area, determining an identification frame taking the corresponding boundary point as a corner point;
And determining the boundary bending degree at the corresponding boundary point according to the number of the pixel points belonging to the vehicle driving area in the identification frame.
8. The method for identifying a road intersection according to any one of claims 1 to 6, wherein the identifying the street view image using a semantic segmentation model to determine a vehicle driving area from the street view image comprises:
grouping a plurality of boundary points in the vehicle driving area, wherein each group comprises a fixed number of boundary points which are adjacently arranged;
fitting boundary points in the same group to obtain a fitting straight line;
and for each group of boundary points, determining the boundary bending degree of each boundary point in the group according to the included angle between the corresponding fitting straight line and the adjacent fitting straight line.
9. A roadway intersection identification apparatus, the apparatus comprising:
the acquisition module is used for acquiring street view images;
the segmentation module is used for identifying the street view image by adopting a semantic segmentation model so as to determine a vehicle driving area from the street view image;
the identification module is used for identifying whether the street view image contains a road intersection or not according to the boundary bending degree of the vehicle driving area;
The street view image is a plurality of frames which are collected sequentially; the identification module comprises:
the extraction unit is used for extracting identification features for each frame of street view image; the identification features comprise the boundary bending degree of the vehicle driving area in the corresponding street view image and the difference information of the boundary bending degree of the adjacent street view image;
the identification unit is used for inputting the extracted identification features into a classification model so as to identify whether the corresponding street view image contains a road intersection or not; wherein, the classification model is learned to obtain the mapping relation between the identification characteristics and the road intersection identification result.
10. The pathway intersection identification device of claim 9, wherein the identification module further comprises:
and the screening unit is used for screening the sequentially acquired multi-frame street view images so as to keep part of street view images with the maximum value of the boundary bending degree larger than a set threshold value.
11. The road intersection identification device according to claim 9, wherein the extraction unit is specifically configured to:
for each frame of the street view image, determining the position of a road intersection from the vehicle driving area of the corresponding street view image; the road intersection position is a section of the corresponding street view image, wherein the boundary bending degree of the vehicle driving area is greater than a set threshold value;
Determining traffic objects presented around the position of each road intersection according to the semantic segmentation model;
and for each frame of the street view image, taking the traffic objects presented at the periphery of the road intersection, the boundary bending degree of the vehicle driving area, the difference information between the traffic objects in the adjacent street view image and the boundary bending degree in the adjacent street view image as the identification characteristics of the corresponding street view image.
12. The pathway intersection identification device of claim 11, wherein the traffic object comprises a traffic marking and a traffic diversion facility.
13. The road intersection identification device according to claim 9, characterized in that the identification feature further comprises one or more combinations of an area of the vehicle running area, a number of lanes contained in the vehicle running area, and a road class of a road segment on which the vehicle running area is located.
14. The apparatus according to claim 9, wherein the recognition feature further comprises a road intersection probability recognized by the convolutional neural network from the inputted street view image;
the convolutional neural network learns to obtain the mapping relation between the recognition features of the street view image and the probability of the road intersection.
15. The pathway intersection identification device of any one of claims 9-14, wherein the device further comprises:
the determining module is used for determining an identification frame taking the corresponding boundary point as a corner point for each boundary point in the vehicle driving area; and determining the boundary bending degree at the corresponding boundary point according to the number of the pixel points belonging to the vehicle driving area in the identification frame.
16. The pathway intersection identification device of any one of claims 9-14, wherein the device is further configured to:
grouping a plurality of boundary points in the vehicle driving area, wherein each group comprises a fixed number of boundary points which are adjacently arranged; fitting boundary points in the same group to obtain a fitting straight line; and for each group of boundary points, determining the boundary bending degree of each boundary point in the group according to the included angle between the corresponding fitting straight line and the adjacent fitting straight line.
17. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the road junction identification method of any one of claims 1-8 when the program is executed.
18. A non-transitory computer-readable storage medium, on which a computer program is stored, characterized in that the program, when executed by a processor, implements the road intersection identification method as claimed in any one of claims 1-8.
CN202010110375.9A 2020-02-21 2020-02-21 Road intersection identification method, device, computer equipment and storage medium Active CN113297878B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010110375.9A CN113297878B (en) 2020-02-21 2020-02-21 Road intersection identification method, device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010110375.9A CN113297878B (en) 2020-02-21 2020-02-21 Road intersection identification method, device, computer equipment and storage medium

Publications (2)

Publication Number Publication Date
CN113297878A CN113297878A (en) 2021-08-24
CN113297878B true CN113297878B (en) 2023-12-08

Family

ID=77317671

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010110375.9A Active CN113297878B (en) 2020-02-21 2020-02-21 Road intersection identification method, device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113297878B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113865610A (en) * 2021-09-30 2021-12-31 北京百度网讯科技有限公司 Method, apparatus, device, medium and product for generating navigation information
CN114694116A (en) * 2022-03-24 2022-07-01 商汤集团有限公司 Road boundary detection method and device, electronic equipment and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102908154A (en) * 2012-11-07 2013-02-06 北京工业大学 Method for detecting start reaction time of driver based on machine vision
CN103162691A (en) * 2011-12-13 2013-06-19 爱信艾达株式会社 Display system, display method and display program supporting a lane change in a vehicle
CN103954277A (en) * 2014-04-30 2014-07-30 百度在线网络技术(北京)有限公司 Method and device for detecting positions of interest points
CN104422458A (en) * 2013-08-30 2015-03-18 深圳富泰宏精密工业有限公司 Vehicle navigation system and vehicle navigation method
CN109147331A (en) * 2018-10-11 2019-01-04 青岛大学 A kind of congestion in road condition detection method based on computer vision
CN109582993A (en) * 2018-06-20 2019-04-05 长安大学 Urban transportation scene image understands and multi-angle of view gunz optimization method
CN110170148A (en) * 2019-05-13 2019-08-27 苏州米龙信息科技有限公司 A kind of intermittent training system and method with amusement interactive function

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9150220B2 (en) * 2013-12-04 2015-10-06 Mobileye Vision Technologies Ltd. Systems and methods for mimicking a leading vehicle

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103162691A (en) * 2011-12-13 2013-06-19 爱信艾达株式会社 Display system, display method and display program supporting a lane change in a vehicle
CN102908154A (en) * 2012-11-07 2013-02-06 北京工业大学 Method for detecting start reaction time of driver based on machine vision
CN104422458A (en) * 2013-08-30 2015-03-18 深圳富泰宏精密工业有限公司 Vehicle navigation system and vehicle navigation method
CN103954277A (en) * 2014-04-30 2014-07-30 百度在线网络技术(北京)有限公司 Method and device for detecting positions of interest points
CN109582993A (en) * 2018-06-20 2019-04-05 长安大学 Urban transportation scene image understands and multi-angle of view gunz optimization method
CN109147331A (en) * 2018-10-11 2019-01-04 青岛大学 A kind of congestion in road condition detection method based on computer vision
CN110170148A (en) * 2019-05-13 2019-08-27 苏州米龙信息科技有限公司 A kind of intermittent training system and method with amusement interactive function

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于MMS街景的导航数据采集方法研究;李观石 等;《现代测绘》;第36卷(第01期);第44-46页 *

Also Published As

Publication number Publication date
CN113297878A (en) 2021-08-24

Similar Documents

Publication Publication Date Title
CN111797187B (en) Map data updating method and device, electronic equipment and storage medium
US11887376B2 (en) Method and apparatus of estimating road condition, and method and apparatus of establishing road condition estimation model
CN111695488B (en) Method, device, equipment and storage medium for identifying interest surface
KR102177880B1 (en) Class labeling apparatus for autonomous driving
US20210209395A1 (en) Method, electronic device, and storage medium for recognizing license plate
CN111611955B (en) Method, device, equipment and storage medium for identifying passable construction road
CN106845487A (en) A kind of licence plate recognition method end to end
CN111275983B (en) Vehicle tracking method, device, electronic equipment and computer-readable storage medium
CN111104538A (en) Fine-grained vehicle image retrieval method and device based on multi-scale constraint
Mahaur et al. Road object detection: a comparative study of deep learning-based algorithms
CN111950537B (en) Zebra crossing information acquisition method, map updating method, device and system
CN111767831B (en) Method, apparatus, device and storage medium for processing image
CN114037966A (en) High-precision map feature extraction method, device, medium and electronic equipment
US11380035B2 (en) Method and apparatus for generating map
CN113297878B (en) Road intersection identification method, device, computer equipment and storage medium
CN111652112B (en) Lane flow direction identification method and device, electronic equipment and storage medium
CN114443794A (en) Data processing and map updating method, device, equipment and storage medium
CN115661522A (en) Vehicle guiding method, system, equipment and medium based on visual semantic vector
CN111950345A (en) Camera identification method and device, electronic equipment and storage medium
CN113011298B (en) Truncated object sample generation, target detection method, road side equipment and cloud control platform
CN114419603A (en) Automatic driving vehicle control method and system and automatic driving vehicle
CN114724113B (en) Road sign recognition method, automatic driving method, device and equipment
CN111325811B (en) Lane line data processing method and processing device
CN115761698A (en) Target detection method, device, equipment and storage medium
CN113221602B (en) Road surface condition determining method, device, equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant