WO2022110862A1 - 地面导向箭头的构建方法及其装置、电子设备及存储介质 - Google Patents
地面导向箭头的构建方法及其装置、电子设备及存储介质 Download PDFInfo
- Publication number
- WO2022110862A1 WO2022110862A1 PCT/CN2021/109249 CN2021109249W WO2022110862A1 WO 2022110862 A1 WO2022110862 A1 WO 2022110862A1 CN 2021109249 W CN2021109249 W CN 2021109249W WO 2022110862 A1 WO2022110862 A1 WO 2022110862A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- arrow
- ground
- mask
- image
- ground guidance
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 239000011159 matrix material Substances 0.000 claims description 21
- 230000000877 morphologic effect Effects 0.000 claims description 21
- 238000004891 communication Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 15
- 238000010276 construction Methods 0.000 claims description 13
- 230000033001 locomotion Effects 0.000 claims description 7
- 101150064138 MAP1 gene Proteins 0.000 description 4
- 238000013500 data storage Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000003709 image segmentation Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 101100400452 Caenorhabditis elegans map-2 gene Proteins 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/20—Drawing from basic elements, e.g. lines or circles
- G06T11/206—Drawing of charts or graphs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/149—Segmentation; Edge detection involving deformable models, e.g. active contour models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Definitions
- the present invention relates to the technical field of image processing, and in particular, to a method for constructing a ground guidance arrow, a device thereof, an electronic device and a storage medium.
- the ground guidance arrow is an important part of the 3D electronic map, and in the existing technical scheme of constructing the ground guidance arrow in the 3D electronic map, it is necessary to use the lidar to collect point cloud data, and then further collect the point cloud data through manual analysis.
- the arrow category, geometric shape and position of the ground guidance arrow are identified, and then the ground guidance arrow is constructed in the three-dimensional electronic map based on the above-identified information.
- ground guidance arrows can be constructed by applying the above methods, the efficiency of manually identifying the arrow types, geometric shapes and positions of the ground guidance arrows is low, resulting in a low construction efficiency of the ground guidance arrows.
- one or more embodiments of the present invention provide a method for constructing a ground guidance arrow, including:
- a ground guide arrow located in a three-dimensional electronic map is constructed.
- the generated mask map is tracked to determine a plurality of mask maps corresponding to the same ground guidance arrow, including:
- the matching cost matrix of each frame of the road image is solved based on the Hungarian algorithm to obtain the arrow regions in each frame of the road image and each of the arrow regions in the next frame of the road image.
- the matching result with the lowest matching cost in the arrow region, wherein the matching result is used to indicate the matching arrow region, and the ground guidance arrows represented by the matching arrow region are the same;
- the mask maps corresponding to the matched arrow regions are determined as the plurality of mask maps corresponding to the same ground guide arrow.
- the determining an arrow type and an arrow outline for each of the ground guidance arrows based on the arrow regions in the plurality of mask maps for each of the ground guidance arrows includes:
- the shape information of the arrow region in each of the mask images determines a set of pixel coordinates of the arrow outline of the ground guidance arrow.
- the shape information includes contour integrity, contour smoothness, and contour fit with the preset arrow contour.
- the contour integrity, the contour smoothness and the contour fit of the arrow region in each of the mask maps based on the ground guide arrow, as well as the preset contour integrity, contour smoothness and The scoring weight of the contour fit, calculate the morphological score of the arrow area in the mask image;
- the set of pixel coordinates of the arrow outline of the ground guidance arrow is extracted.
- the constructing the ground guidance arrow located in the three-dimensional electronic map according to the arrow category and the arrow outline of each of the ground guidance arrows includes:
- the arrow outline of each of the ground guidance arrows is converted from the image physical coordinate system Convert to camera coordinate system;
- the arrow outline is transformed from the camera coordinate system to the world coordinate system;
- a ground guide arrow corresponding to the arrow class of each of the ground guide arrows is constructed at the projected position of each of the ground guide arrows.
- the arrow region where the ground guidance arrow corresponding to each mask image is located in the road image in the next frame is predicted.
- one or more embodiments of the present invention also provide a device for constructing a ground guide arrow, comprising:
- an image acquisition module for acquiring continuously collected road images, and determining the area where the ground guidance arrows are located in each of the road images as the arrow area;
- a mask map generation module configured to generate, for each of the road images, a mask map corresponding to each of the arrow regions in the road image
- the mask map tracking module is used to track the generated mask map and determine multiple mask maps corresponding to the same ground guidance arrow;
- an information determination module configured to determine an arrow type and an arrow outline of each of the ground guidance arrows based on the arrow regions in the plurality of mask maps for each of the ground guidance arrows;
- An arrow building module configured to construct a ground guide arrow located in a three-dimensional electronic map according to the arrow category and the arrow outline of each of the ground guide arrows.
- the mask map tracking module is specifically configured to, for each frame of the road image, predict the mask map based on the position of the arrow region in the mask map of the road image The arrow area where the corresponding ground guidance arrow is located in the next frame of road image; calculate the distance between the predicted arrow area of each arrow area in each frame of the road image and each arrow area in the next frame of road image and based on the calculated intersection ratio, generate a matching cost matrix between each arrow area in each frame of the road image and each arrow area in the next frame of road image; The matching cost matrix of the road image, and the matching cost matrix is solved based on the Hungarian algorithm to obtain the lowest matching cost between each arrow area in each frame of the road image and each arrow area in the next frame of road image.
- the matching result is used to indicate the matched arrow area, and the ground guide arrows represented by the matched arrow area are the same; the mask map corresponding to the matched arrow area is determined to correspond to the same ground guide arrow. the plurality of mask images.
- the information determination module is specifically configured to: for each mask map of the plurality of mask maps of each of the ground guidance arrows, predict the ground surface represented by the arrow area The predicted arrow category of the guide arrow, and determine the shape information of the ground guide arrow represented by the arrow area in the mask map; for each of the ground guide arrows, from the predicted arrow categories of the ground guide arrow, determine The predicted arrow type with the most same type is used as the arrow type of the ground guidance arrow, and according to the shape information of the arrow area in each of the mask maps of the ground guidance arrow, the location of the ground guidance arrow is determined.
- the set of pixel coordinates of the outline of the arrow is specifically configured to: for each mask map of the plurality of mask maps of each of the ground guidance arrows, predict the ground surface represented by the arrow area The predicted arrow category of the guide arrow, and determine the shape information of the ground guide arrow represented by the arrow area in the mask map; for each of the ground guide arrows, from the predicted
- the shape information includes contour integrity, contour smoothness, and contour fit with the preset arrow contour.
- the information determination module When determining the set of pixel coordinates of the arrow outline of the ground guidance arrow according to the shape information of the arrow region in each of the mask maps of the ground guidance arrow, the information determination module specifically uses In: the contour integrity, the contour smoothness and the contour fit of the arrow region in each of the mask maps based on the ground guide arrow, as well as the preset contour integrity, contour smoothness
- the morphological score of the arrow region in the mask map is calculated; from the arrow regions in each of the mask maps of the ground-oriented arrow, select the morphological score with the highest morphological score
- the arrow area of the ground guide arrow is used as the target area of the ground guide arrow; the outline of the target area of the ground guide arrow is extracted, and the extracted outline is smoothed, and the smoothed outline is used as the ground guide arrow.
- the outline of the arrow extract the set of pixel coordinates of the outline of the arrow of the ground guide arrow.
- the arrow building module is specifically configured to: according to the device external parameter of the image acquisition device when collecting the road image of each of the ground guidance arrows, make all the ground guidance arrows
- the arrow outline is converted from the image pixel coordinate system to the image physical coordinate system; each The arrow outline of the ground guidance arrow is converted from the image physical coordinate system to the camera coordinate system; according to the device extrinsic parameter of the image acquisition device and the The world coordinate of the image acquisition device in the world coordinate system, the arrow outline of each of the ground guidance arrows is converted from the camera coordinate system to the world coordinate system; based on the location of each of the ground guidance arrows the outline of the arrow in the world coordinate system, calculate the projected position of each of the ground-oriented arrows; construct the arrow category corresponding to each of the ground-oriented arrows at the projected position of each of the ground-oriented arrows ground guide arrows.
- one or more embodiments of the present invention further provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus. communication.
- the memory is used to store computer programs.
- the processor is used for executing the program stored in the memory, so as to realize any one of the above-mentioned construction methods for ground guidance arrows.
- one or more embodiments of the present invention further provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any of the above-mentioned 1.
- one or more embodiments of the present invention also provide a computer program product comprising instructions, which, when executed on a computer, cause the computer to execute any of the above-mentioned methods for constructing a ground guidance arrow.
- FIG. 1 is a flowchart of a method for constructing a ground guidance arrow according to one or more embodiments of the present invention.
- FIG. 2 is a flowchart of a method for determining a mask map according to one or more embodiments of the present invention.
- FIG. 3 is a flowchart of a method for identifying ground guidance arrows according to one or more embodiments of the present invention.
- FIG. 4 is a flowchart of a method for constructing a ground guidance arrow according to one or more embodiments of the present invention.
- FIG. 5 is a schematic structural diagram of a device for constructing a ground guide arrow according to one or more embodiments of the present invention.
- FIG. 6 is a schematic structural diagram of an electronic device according to one or more embodiments of the present invention.
- One or more embodiments of the present invention provide a method for constructing a ground guidance arrow, as shown in FIG. 1 , the method includes:
- S101 Acquire continuously collected road images, and determine the area where the ground guidance arrows are located in each road image as the arrow area.
- S103 Track the generated mask map, and determine multiple mask maps corresponding to the same ground guidance arrow.
- S104 Determine the arrow type and arrow outline of each ground guidance arrow based on the arrow regions in the multiple mask maps of each ground guidance arrow.
- S105 Construct a ground guide arrow located in the three-dimensional electronic map according to the arrow type and arrow outline of each ground guide arrow.
- ground guidance arrows As shown in FIG. 1 , according to a method for constructing ground guidance arrows provided by one or more embodiments of the present invention, after acquiring a road image, it is possible to determine the arrow area in the road image, generate a mask map corresponding to the arrow area, etc. A series of operations are performed to obtain the arrow category and arrow outline of the ground-oriented arrow, and then construct the ground-oriented arrow located in the 3D electronic map according to the obtained arrow category and arrow outline. This process does not require manual identification of the arrow category and arrow outline, so that Improved construction efficiency of ground guide arrows.
- the road image is an image obtained by collecting a road.
- the collected road image it can be divided into a road image that does not include ground guidance arrows and a road image that includes ground guidance arrows.
- This solution only involves
- the road images including ground guidance arrows in order to explain the embodiments of the present invention more clearly, the road images in the embodiments of the present invention are all road graphics including ground guidance arrows.
- road images may be read from a database storing pre-collected road images, or the road images may be obtained from an image acquisition device.
- the road image collected by the image acquisition device is copied in the data storage unit of the device, and it can also be the road image acquired in real time from the image acquisition device, wherein the image acquisition device can be a camera, a camera, an image sensor, etc. installed on the acquisition vehicle.
- ground guide arrows Subject to the road environment, image acquisition equipment and acquisition angle when collecting road images, the number of ground guidance arrows contained in road images is not fixed. ground guide arrows.
- the road image contains only one ground guidance arrow, the area where the ground guidance arrow is located can be used as the arrow area.
- the road image contains multiple ground guidance arrows, independent arrow regions are determined for each ground guidance arrow.
- the above-mentioned arrow area may be the smallest area where the ground guidance arrow is located.
- an image segmentation model for each road image, can be used to perform pixel segmentation on each road image, and the pixel area of each ground guidance arrow in the road image is segmented, and the pixel area is Arrow area.
- the image segmentation model may be deeplabv3.
- the mask image corresponding to each arrow area is an image generated by setting the pixel value of the pixel in the arrow area as the first pixel value, and setting the pixel value of the pixel outside the arrow area as the second pixel value .
- the pixel value of the pixel in the arrow area is set to 1
- the pixel value of the pixel outside the arrow area is set to 0.
- only one mask map can be generated from the road image, and when the road image contains multiple ground guidance arrows, for each road image Each area where the ground guidance arrows are located generates a mask image, thereby obtaining multiple mask images.
- step S103 for the ground guidance arrows in the road, when the image acquisition device passes the ground guidance arrows, the ground guidance arrows may be continuously collected for multiple times. Therefore, for a ground guidance arrow in the road, it may be Recorded in multiple road images, which means that there can be multiple masks corresponding to the same ground guidance arrow in all the resulting masks.
- the arrow region in each mask map may be tracked to determine multiple mask maps corresponding to the same ground guidance arrow.
- the image acquisition device collects three road images through the ground guidance arrow T, namely road image 1, road image 2, and road image 3, wherein the ground guidance arrow T is located in road image 1, road image 1, and road image 3, respectively.
- the bottom area, the middle area and the top area of the image 2 and the road image 3, after step S102, the mask map 1 and the mask map corresponding to the area where the road image 1, the road image 2 and the road image 3 are located by the ground guidance arrow T can be obtained respectively.
- Figure 2 and Mask Figure 3 When the mask map 1 is first obtained, by tracking the mask map 1, it can be determined that the mask map 2 and the mask map 3 and the mask map 1 correspond to the same ground guide arrow.
- the arrow categories include straight arrows, left-turn arrows, right-turn arrows, etc., and the outline of the arrow may be described by a set of pixel coordinates of edge pixels of the arrow area.
- shape recognition may be performed on arrow regions in a plurality of mask maps of each ground guidance arrow, an arrow type of each ground guidance arrow may be determined according to the shape of the identified region, and the identified The area outline of the area is the arrow outline of each ground guide arrow.
- the arrow type of the ground guidance arrow may be determined according to the determined number of arrow types. For example, the arrow class with the largest proportion can be used as the arrow class for ground-directed arrows.
- each determined arrow outline may be fused, and the fused arrow outline may be used as the arrow outline of the ground guide arrow. It is also possible to first select the optimal arrow outline as the arrow outline of the ground guide arrow.
- step S105 after determining the arrow type and arrow outline of each ground-oriented arrow, the pixel coordinate set corresponding to the arrow outline can be converted into a three-dimensional coordinate set in the three-dimensional electronic map, so as to project the arrow outline in the three-dimensional electronic map, And the arrow outline in the 3D electronic map is associated with the determined arrow category to generate the ground guide arrow in the 3D electronic map.
- each mask image can be divided into multiple groups, each group is composed of a first type of mask image and a second type of mask image, wherein the first type of mask image is: The mask map corresponding to each arrow area in the first road image, the second type of mask map is: the mask map corresponding to each arrow area in the second road image, the first road image and the second road image collection time are adjacent, And the acquisition time of the first road image is earlier than that of the second road image.
- the mask map corresponding to each arrow region in road image A includes mask map A1, mask map A2, and mask map A3;
- the mask map corresponding to each arrow area in road image B includes mask map B1, mask map B2 and mask map B3;
- the mask map corresponding to each arrow area in road image C includes mask map C1, mask map C2 and mask image C3.
- the mask map corresponding to the road image A and the mask map corresponding to the road image B can be combined into a group. In this group, the mask map A1, the mask map A2 and the mask map A3 are the first type of mask maps.
- the mask map B1 , the mask map B2 and the mask map B3 are the second type of mask map.
- the mask map corresponding to the road image B and the mask map corresponding to the road image C can be combined into a group.
- the mask map B1, the mask map B2 and the mask map B3 are the first type of mask maps
- the mask map C1, the mask map C2 and the mask map C3 are the second type of mask map.
- the above grouping process can be carried out in real time.
- the time T-1 can be collected.
- the mask map corresponding to the road image is used as the first type of mask map
- the mask map corresponding to the road image collected at time T is used as the second type of mask map.
- the image acquisition device continues to collect images at time T+1, it can be The mask map corresponding to the road image collected at time T is used as the first type of mask map
- the mask map corresponding to the road image collected at time T+1 is used as the second type of mask map.
- the selection arrow area represents the mask map of the same ground guide arrow, including:
- the position of the arrow region in the mask map based on each road image can be combined to predict the arrow region where the ground guidance arrow corresponding to each mask map is located in the next frame of road image.
- the ground corresponding to each mask map is predicted according to the position of the arrow area in the mask map of each road image, according to the distance and direction of movement in one acquisition cycle represented by the motion information
- the motion information may be the position difference information of the ground guidance arrows in the area where the adjacent road images are located, indicated by the arrow area in the mask map of each road image, or may be based on the collection period of the image collection device and the location of the image collection device.
- the motion parameters determined by parameters such as the driving speed of the vehicle and the shooting angle when the image acquisition device collects road images.
- S202 Calculate the intersection ratio between the predicted arrow area of each arrow area in each frame of road image and each arrow area in the next frame of road image, and generate each frame of road image based on the calculated intersection ratio.
- the predicted arrow region of each arrow region and the next frame of road image based on the relative position of the predicted arrow region of each arrow region and each arrow region in the next frame of road image, the predicted arrow region of each arrow region and the next frame of road image The intersection ratio between each arrow area in .
- intersection ratio is: the area ratio of the intersection area obtained by the intersection of the two areas and the area ratio of the union area of the two areas.
- the intersection ratio of the two areas can reflect the overlap ratio of the two areas. When the intersection ratio is When it is 1, it means that the two regions completely overlap, and when the intersection ratio is 0, it means that the two regions do not overlap at all.
- the set of intersection ratio matrices may be generated based on the calculated intersection ratio. For example, if the number of predicted arrow regions is M, and the number of arrow regions in the next frame of road image is N, then an M ⁇ N intersection and complement matrix can be established as the relationship between each arrow region in each frame of road image and the next The matching cost matrix for each arrow region in the frame road image.
- the Hungarian Algorithm is a combinatorial optimization algorithm that solves the task assignment problem in polynomial time, and can calculate the matching cost of each matching result.
- the matching result is used to indicate the matched arrow area, and the ground guidance arrows represented by the matched arrow area are the same. Further, a matching result with the lowest matching cost between each arrow region in each frame of road image and each arrow region in the next frame of road image can be selected.
- arrow area 1 matches arrow area 2
- arrow area 3 matches arrow area 4
- arrow area 1 matches arrow area 2
- arrow area 3 matches arrow area 4
- arrow area 1 arrow area 2
- arrow area 3 matches arrow area 4
- arrow area 1 arrow area 2
- arrow area 3 matches arrow area 4
- arrow area 1 arrow area 2
- arrow area 3 matches arrow area 4
- arrow area 1 arrow area 2
- arrow area 3 matches arrow area 4
- the SORT Simple Online And Realtime Tracking, Simple Online And Realtime Tracking
- the SORT Simple Online And Realtime Tracking
- the ground guidance arrow can be used as a new target, and the above ground guidance arrow can be repeated. Operations, such as re-determining the arrow area represented by the ground guidance arrow, etc.
- the ground guidance arrow identification method shown in FIG. 3 may be used to determine the arrow of each ground guidance arrow Categories and arrow outlines, including:
- the above morphological information includes: contour integrity, contour smoothness, and contour fit with the preset arrow contour.
- each mask image can also be processed through a ResNet50 (Residual Network 50, Residual Network 50) model to obtain a prediction of the ground guidance arrow represented by the arrow area in each mask image Arrow category and morphological information.
- ResNet50 Residual Network 50, Residual Network 50
- the predicted arrow type with the most same type may be determined as the arrow type of the ground guidance arrow.
- each ground guide can be determined according to the morphological information of different arrow areas.
- the degree of pros and cons of the shape of the arrow so that the arrow region with the optimal shape of the ground-oriented arrow can be selected, and the region outline of the arrow region can be used as the arrow outline of the ground-oriented arrow corresponding to the mask map set, or the The area outline of the arrowed area is further processed.
- Step 1 Based on the contour integrity, contour smoothness, and contour fit of the arrow region in each mask map of the ground-guided arrow, and the preset scoring weights for contour integrity, contour smoothness, and contour fit , calculate the morphological score for the arrowed area in this mask.
- Step 2 From the arrow areas in each mask map of the ground guide arrow, select the arrow area with the highest morphological score as the target area of the ground guide arrow.
- the morphological information may be a morphological score determined according to the completeness of the regional outline of the arrow region, the degree of fit with the preset arrow outline, and the smoothness of the regional outline. High indicates that the shape of the ground guide arrow represented by the arrow area is better, and the arrow area with the highest shape score can be selected as the target area.
- Step 3 Extract the contour of the target area of the ground guide arrow, perform smooth processing on the extracted contour, and use the smoothed contour as the arrow contour of the ground guide arrow.
- the contour of the edge pixels of the target area may be extracted, and the extracted contour may be used as the contour of the ground guide arrow.
- the extracted contour can also be smoothed, and the smoothed contour can be used as the arrow contour of the ground guide arrow.
- the extracted contours may be smoothed using a Douglas-Peucker algorithm.
- Step 4 Extract the pixel coordinate set of the arrow outline of the ground-guided arrow.
- a set of pixel coordinates of the arrow outline of the ground guidance arrow can be extracted.
- an eight-neighbor algorithm can be used to extract the pixel coordinates of the arrow outline of the target area to form a pixel coordinate set.
- the ground corresponding to the mask map set can be determined according to the arrow area in each mask map in the mask map set
- the arrow categories and arrow outlines of the guiding arrows provide the basis for constructing the ground guiding arrows in the 3D electronic map based on the road image, which can improve the construction efficiency of the ground guiding arrows.
- a three-dimensional electronic map may be constructed according to the construction method for ground guidance arrows shown in FIG. 4 .
- Ground guidance arrows corresponding to each mask set including:
- S401 Convert the arrow outline of each ground guidance arrow from the image pixel coordinate system to the image physical coordinate system according to the device external parameters of the image acquisition device when the road image of each ground guidance arrow is collected.
- S402 Convert the arrow outline of each ground guidance arrow from the image physical coordinate system to the camera coordinate system according to the device external parameter and the device internal parameter of the image acquisition device when the road image of each ground guidance arrow is collected.
- S403 Convert the arrow outline of each ground guidance arrow from the image physical coordinate system to the camera coordinate system according to the device external parameter and the device internal parameter of the image acquisition device when the road image of each ground guidance arrow is collected.
- S405 Construct a ground guide arrow corresponding to the arrow type of each ground guide arrow at the projected position of each ground guide arrow.
- the device external parameters of the image acquisition device include parameters such as the height difference from the road when the road image is collected, the longitude and latitude where it is located, and the device internal parameters include parameters such as the focal length of the image acquisition device.
- the ground guidance arrows can be projected in the three-dimensional electronic map according to the following formula:
- (u, v) are the coordinates of the pixel in the pixel coordinates
- dx is the physical size of each pixel on the horizontal axis x
- dy is the size of each pixel on the physical vertical axis y
- u0 and v0 are the image
- the center of the plane f is the intersection of the image acquisition device
- R is the rotation matrix
- T is the translation vector
- (X w , Y w , Z w ) represents the projection coordinates in the three-dimensional electronic image.
- one or more embodiments of the present invention further provide a ground guide arrow construction device, and the device includes:
- the image acquisition module 501 is used for acquiring continuously collected road images, and determining the area where the ground guidance arrows are located in each road image as the arrow area;
- a mask map generation module 502 for generating a mask map corresponding to each arrow region in the road image for each road image;
- a mask map tracking module 503, configured to track the generated mask map, and determine a plurality of mask maps corresponding to the same ground guidance arrow;
- an information determination module 504 configured to determine the arrow type and arrow outline of each ground guidance arrow based on the arrow regions in the multiple mask maps of each ground guidance arrow;
- the arrow construction module 505 is used for constructing the ground guide arrows located in the three-dimensional electronic map according to the arrow type and arrow outline of each ground guide arrow.
- the mask map tracking module is specifically configured to predict the mask based on the position of the arrow region in the mask map of the road image for each frame of road image in the continuously collected road images.
- the arrow area where the ground guidance arrow corresponding to the figure is located in the next frame of road image calculate the intersection ratio between the predicted arrow area of each arrow area in each frame of road image and each arrow area in the next frame of road image, And based on the calculated intersection ratio, the matching cost matrix of each arrow area in each frame of road image and each arrow area in the next frame of road image is generated; for the matching cost matrix of each frame of road image, based on the Hungarian algorithm
- the matching cost matrix is solved to obtain the matching result with the lowest matching cost between each arrow area in each frame of road image and each arrow area in the next frame of road image, wherein the matching result is used to indicate the matching arrow area, the matching arrow area
- the represented ground guide arrows are the same; the mask maps corresponding to the matched arrow regions are determined as
- the information determination module is specifically configured to, for each mask map of the plurality of mask maps of each ground guidance arrow, predict the predicted arrow type of the ground guidance arrow represented by the arrow area, and determine The morphological information of the ground guidance arrow represented by the arrow area in the mask map; for each ground guidance arrow, from the predicted arrow categories of the ground guidance arrow, determine the predicted arrow type with the most same type as the ground guidance arrow and according to the shape information of the arrow area in each mask map of the ground guide arrow, the pixel coordinate set of the arrow outline of the ground guide arrow is determined.
- the morphological information includes: contour integrity, contour smoothness, and contour fit with the preset arrow contour;
- an information determination module which is specifically used for the contour integrity, contour smoothness and contour fit of the arrow region in each mask image based on the ground guide arrow, as well as the preset contour integrity, contour smoothness and contour fit Calculate the morphological score of the arrow area in the mask map; select the arrow area with the highest morphological score from the arrow areas in each mask map of the ground-oriented arrow as the target area of the ground-oriented arrow; extract The contour of the target area of the ground guide arrow is smoothed, and the smoothed contour is used as the arrow contour of the ground guide arrow; the pixel coordinate set of the arrow contour of the ground guide arrow is extracted.
- the arrow building module is specifically configured to convert the arrow outline of each ground guidance arrow from the image pixel coordinate system to the device extrinsic parameter of the image acquisition device when collecting the road image of each ground guidance arrow Image physical coordinate system; according to the device external parameters and device internal parameters of the image acquisition device when collecting the road image of each ground guidance arrow, the arrow outline of each ground guidance arrow is converted from the image physical coordinate system to the camera coordinate system; When a road image of a ground guidance arrow is the device external parameter of the image acquisition device and the world coordinate of the image acquisition device in the world coordinate system, the arrow outline of each ground guidance arrow is converted from the camera coordinate system to the world coordinate system; based on each The arrow outline of the ground guide arrow in the world coordinate system, the projection position of each ground guide arrow is calculated; the ground guide arrow corresponding to the arrow type of each ground guide arrow is constructed at the projected position of each ground to the arrow-like arrow.
- the ground can be obtained by a series of operations such as determining the arrow area in the road image and generating a mask map corresponding to the arrow area.
- the arrow category and arrow outline of the guiding arrow, and then the ground guiding arrow located in the 3D electronic map is constructed according to the obtained arrow category and arrow outline. This process does not require manual identification of the arrow category and arrow outline, thereby improving the ground guiding arrow. build efficiency.
- the electronic device includes a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602 , the memory 603 completes the mutual communication through the communication bus 604 .
- the memory 603 is used to store computer programs.
- the processor 601 is used for executing the program stored in the memory 603 to realize the following steps:
- each ground guide arrow located in the three-dimensional electronic map is constructed.
- the arrow of the ground guidance arrow can be obtained by a series of operations such as determining the arrow area in the road image and generating the mask map corresponding to the arrow area. According to the obtained arrow categories and arrow contours, the ground guide arrows located in the 3D electronic map are constructed. This process does not require manual identification of arrow categories and arrow contours, thus improving the construction efficiency of ground guide arrows.
- the communication bus mentioned in the above electronic device may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an Extended Industry Standard Architecture (Extended Industry Standard Architecture, EISA) bus or the like.
- PCI peripheral component interconnect standard
- EISA Extended Industry Standard Architecture
- the communication bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
- the communication interface is used for communication between the above electronic device and other devices.
- the memory may include random access memory (Random Access Memory, RAM), and may also include non-volatile memory (Non-Volatile Memory, NVM), such as at least one disk storage.
- RAM Random Access Memory
- NVM non-Volatile Memory
- the memory may also be at least one storage device located remotely from the aforementioned processor.
- the above-mentioned processor can be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; it can also be a digital signal processor (Digital Signal Processing, DSP), dedicated integrated Circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
- CPU Central Processing Unit
- NP Network Processor
- DSP Digital Signal Processing
- ASIC Application Specific Integrated Circuit
- FPGA Field-Programmable Gate Array
- FPGA Field-Programmable Gate Array
- a computer-readable storage medium is also provided, and a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any one of the above ground guidance arrows is implemented steps of the build method.
- a computer program product containing instructions, which, when executed on a computer, cause the computer to execute any method for constructing a ground guidance arrow in the above-mentioned embodiments.
- the above-mentioned embodiments it may be implemented in whole or in part by software, hardware, firmware or any combination thereof.
- software it can be implemented in whole or in part in the form of a computer program product.
- the computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated.
- the computer may be a general purpose computer, special purpose computer, computer network, or other programmable device.
- the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server, or data center Transmission to another website site, computer, server, or data center is by wire (eg, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.).
- the computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, or the like that includes an integration of one or more available media.
- the usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)), and the like.
- the arrow types and types of ground guidance arrows are acquired through a series of operations such as determining the arrow area in the road image and generating a mask map corresponding to the arrow area. Then, according to the obtained arrow category and arrow contour, the ground guide arrow located in the 3D electronic map is constructed. This process does not require manual identification of the arrow category and arrow contour, thus improving the construction efficiency of the ground guide arrow.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Remote Sensing (AREA)
- Computer Graphics (AREA)
- Processing Or Creating Images (AREA)
- Instructional Devices (AREA)
- Image Analysis (AREA)
Abstract
本公开提供了一种地面导向箭头的构建方法，包括: 获取连续采集的道路图像，并确定每张道路图像中地面导向箭头所在的区域，作为箭头区域; 针对每一张道路图像，生成该道路图像中每一箭头区域对应的掩膜图; 对生成的掩膜图进行追踪，确定对应于同一地面导向箭头的多个掩膜图; 基于每一地面导向箭头的多个掩膜图中箭头区域，确定每一地面导向箭头的箭头类别和箭头轮廓; 根据每一地面导向箭头的箭头类别和箭头轮廓，构建位于三维电子地图中的地面导向箭头。
Description
本发明涉及图像处理技术领域，特别是涉及地面导向箭头的构建方法及其装置、电子设备及存储介质。
地面导向箭头是三维电子地图的重要组成部分，而现有的在三维电子地图中构建地面导向箭头的技术方案中，需要使用激光雷达采集点云数据，再进一步通过人工对采集到的点云数据识别出地面导向箭头的箭头类别、几何形态和位置，然后基于识别出的上述信息在三维电子地图中构建地面导向箭头。
虽然应用上述方式能够构建出地面导向箭头，但是人工识别地面导向箭头的箭头类别、几何形态和位置效率较低，从而导致地面导向箭头的构建效率较低。
发明内容
根据第一方面，本发明的一个或多个实施例提供了一种地面导向箭头的构建方法，包括：
获取连续采集的道路图像，并确定每张所述道路图像中地面导向箭头所在的区域，作为箭头区域；
针对每一张所述道路图像，生成该道路图像中每一所述箭头区域对应的掩膜图；
对生成的掩膜图进行追踪，确定对应于同一地面导向箭头的多个掩膜图；
基于每一所述地面导向箭头的所述多个掩膜图中的所述箭头区域，确定每一所述地面导向箭头的箭头类别和箭头轮廓；
根据每一所述地面导向箭头的所述箭头类别和所述箭头轮廓，构建位于三维电子地图中的地面导向箭头。
根据一个或多个实施例，所述对生成的掩膜图进行追踪，确定对应于同一地面导向箭头的多个掩膜图，包括：
针对每帧所述道路图像，基于该道路图像的所述掩膜图中所述箭头区域的位置，预测该掩膜图所对应的地面导向箭头在下一帧所述道路图像中所在的箭头区域；
计算每帧所述道路图像中每一所述箭头区域的预测箭头区域与下一帧所述道路图像中每一所述箭头区域之间的交并比，并基于所计算的交并比，生成每帧所述道路图像中每一所述箭头区域与下一帧所述道路图像中每一所述箭头区域的匹配成本矩阵；
针对每帧所述道路图像的所述匹配成本矩阵，基于匈牙利算法对该匹配成本矩阵进行求解，得到每帧所述道路图像中各所述箭头区域与下一帧所述道路图像中各所述箭头区域匹配成本最低的匹配结果，其中，所述匹配结果用于指示相匹配的箭头区域，相匹配的箭头区域所表示的地面导向箭头相同；
将相匹配的箭头区域对应的所述掩膜图确定为对应于同一地面导向箭头的所述多个掩膜图。
根据一个或多个实施例，所述基于每一所述地面导向箭头的所述多个掩膜图中的所述箭头区域，确定每一所述地面导向箭头的箭头类别和箭头轮廓，包括：
针对每一所述地面导向箭头的所述多个掩膜图中每一掩膜图，预测该箭头区域表示的所述地面导向箭头的预测箭头类别，并确定该掩膜图中的所述箭头区域所表示的所述地面导向箭头的形态信息；
针对每一所述地面导向箭头，从该地面导向箭头的各所述预测箭头类别中，确定相同类别最多的预测箭头类别，作为该地面导向箭头的所述箭头类别，并且根据该地面导向箭头的各所述掩膜图中所述箭头区域的所述形态信息，确定该地面导向箭头的所述箭头轮廓的像素坐标集合。
根据一个或多个实施例，所述形态信息包括轮廓完整度、轮廓光滑度和与预设箭头轮廓的轮廓贴合度。
所述根据该地面导向箭头的各所述掩膜图中所述箭头区域的所述形态信 息，确定该地面导向箭头的所述箭头轮廓的像素坐标集合，包括：
基于该地面导向箭头的每一所述掩膜图中所述箭头区域的所述轮廓完整度、所述轮廓光滑度和所述轮廓贴合度，以及预设的轮廓完整度、轮廓光滑度和轮廓贴合度的评分权重，计算该掩膜图中所述箭头区域的形态评分；
从该地面导向箭头的各所述掩膜图中的所述箭头区域中，选取所述形态评分最高的箭头区域，作为该地面导向箭头的目标区域；
提取该地面导向箭头的所述目标区域的轮廓，并对所提取的轮廓进行平滑处理，将平滑处理后的轮廓，作为该地面导向箭头的所述箭头轮廓；
提取该地面导向箭头的所述箭头轮廓的所述像素坐标集合。
根据一个或多个实施例，所述根据每一所述地面导向箭头的所述箭头类别和所述箭头轮廓，构建位于三维电子地图中的地面导向箭头，包括：
根据采集每一所述地面导向箭头的所述道路图像时图像采集设备的设备外参，将每一所述地面导向箭头的所述箭头轮廓从图像像素坐标系转换到图像物理坐标系；
根据采集每一所述地面导向箭头的所述道路图像时所述图像采集设备的所述设备外参和设备内参，将每一所述地面导向箭头的所述箭头轮廓从所述图像物理坐标系转换到相机坐标系；
根据采集每一所述地面导向箭头的所述道路图像时所述图像采集设备的所述设备外参和所述图像采集设备在世界坐标系下的世界坐标，将每一所述地面导向箭头的所述箭头轮廓从所述相机坐标系转换到所述世界坐标系；
基于每一所述地面导向箭头的在所述世界坐标系下的箭头轮廓，计算每一所述地面导向箭头的投影位置；
在每一所述地面导向箭头的所述投影位置处构建对应于每一所述地面导向箭头的所述箭头类别的地面导向箭头。
根据一个或多个实施例，所述基于该道路图像的所述掩膜图中所述箭头区域的位置，预测该掩膜图所对应的地面导向箭头在下一帧所述道路图像中所在的箭头区域，包括：
基于运动信息预测每一所述掩膜图所对应的地面导向箭头在下一帧所述道路图像中所在的箭头区域；或
基于卡尔曼滤波算法预测每一所述掩膜图所对应的地面导向箭头在下一帧所述道路图像中所在的箭头区域。
根据第二方面，本发明的一个或多个实施例还提供了一种地面导向箭头的构建装置，包括：
图像获取模块，用于获取连续采集的道路图像，并确定每张所述道路图像中地面导向箭头所在的区域，作为箭头区域；
掩膜图生成模块，用于针对每一张所述道路图像，生成该道路图像中每一所述箭头区域对应的掩膜图；
掩膜图追踪模块，用于对生成的掩膜图进行追踪，确定对应于同一地面导向箭头的多个掩膜图；
信息确定模块，用于基于每一所述地面导向箭头的所述多个掩膜图中的所述箭头区域，确定每一所述地面导向箭头的箭头类别和箭头轮廓；
箭头构建模块，用于根据每一所述地面导向箭头的所述箭头类别和所述箭头轮廓，构建位于三维电子地图中的地面导向箭头。
根据一个或多个实施例，所述掩膜图追踪模块，具体用于针对每帧所述道路图像，基于该道路图像的所述掩膜图中所述箭头区域的位置，预测该掩膜图所对应的地面导向箭头在下一帧道路图像中所在的箭头区域；计算每帧所述道路图像中每一所述箭头区域的预测箭头区域与下一帧道路图像中每一所述箭头区域之间的交并比，并基于所计算的交并比，生成每帧所述道路图像中每一所述箭头区域与下一帧道路图像中每一所述箭头区域的匹配成本矩阵；针对每帧所述道路图像的所述匹配成本矩阵，基于匈牙利算法对该匹配成本矩阵进行求解，得到每帧所述道路图像中各所述箭头区域与下一帧道路图像中各所述箭头区域匹配成本最低的匹配结果，其中，所述匹配结果用于指示相匹配的箭头区域，相匹配的箭头区域所表示的地面导向箭头相同；将相匹配的箭头区域对应的掩膜图确定对应于同一地面导向箭头的所述多个掩膜图。
根据一个或多个实施例，所述信息确定模块具体用于：针对每一所述地面 导向箭头的所述多个掩膜图中的每一掩膜图，预测该箭头区域表示的所述地面导向箭头的预测箭头类别，并确定该掩膜图中的所述箭头区域所表示地面导向箭头的形态信息；针对每一所述地面导向箭头，从该地面导向箭头的各预测箭头类别中，确定相同类别最多的预测箭头类别，作为该地面导向箭头的所述箭头类别，并且根据该地面导向箭头的各所述掩膜图中所述箭头区域的所述形态信息，确定该地面导向箭头的所述箭头轮廓的像素坐标集合。
根据一个或多个实施例，所述形态信息包括轮廓完整度、轮廓光滑度和与预设箭头轮廓的轮廓贴合度。
在执行所述根据该地面导向箭头的各所述掩膜图中所述箭头区域的所述形态信息，确定该地面导向箭头的所述箭头轮廓的像素坐标集合时，所述信息确定模块具体用于：基于该地面导向箭头的每一所述掩膜图中所述箭头区域的所述轮廓完整度、所述轮廓光滑度和所述轮廓贴合度，以及预设的轮廓完整度、轮廓光滑度和轮廓贴合度的评分权重，计算该掩膜图中所述箭头区域的形态评分；从该地面导向箭头的各所述掩膜图中的所述箭头区域中，选取所述形态评分最高的箭头区域，作为该地面导向箭头的目标区域；提取该地面导向箭头的所述目标区域的轮廓，并对所提取的轮廓进行平滑处理，将平滑处理后的轮廓，作为该地面导向箭头的所述箭头轮廓；提取该地面导向箭头的所述箭头轮廓的所述像素坐标集合。
根据一个或多个实施例，所述箭头构建模块具体用于：根据采集每一所述地面导向箭头的所述道路图像时图像采集设备的设备外参，将每一所述地面导向箭头的所述箭头轮廓从图像像素坐标系转换到图像物理坐标系；根据采集每一所述地面导向箭头的所述道路图像时所述图像采集设备的所述设备外参和设备内参，将每一所述地面导向箭头的所述箭头轮廓从所述图像物理坐标系转换到相机坐标系；根据采集每一所述地面导向箭头的所述道路图像时所述图像采集设备的所述设备外参和所述图像采集设备在世界坐标系下的世界坐标，将每一所述地面导向箭头的所述箭头轮廓从所述相机坐标系转换到所述世界坐标系；基于每一所述地面导向箭头的在所述世界坐标系下的箭头轮廓，计算每一所述地面导向箭头的投影位置；在每一所述地面导向箭头的所述投影位置处构建对应于每一所述地面导向箭头的所述箭头类别的地面导向箭头。
根据第三方面，本发明的一个或多个实施例还提供了一种电子设备，包括 处理器、通信接口、存储器和通信总线，其中，处理器，通信接口，存储器通过通信总线完成相互间的通信。所述存储器用于存放计算机程序。所述处理器用于执行存储器上所存放的程序，以实现上述任一所述的地面导向箭头的构建方法。
根据第四方面，本发明的一个或多个实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现上述任一所述的地面导向箭头的构建方法。
根据第五方面，本发明的一个或多个实施例还提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述任一所述的地面导向箭头的构建方法。
为了更清楚地说明本发明的一个或多个实施例，下面将对实施例描述中所需要使用的附图作简单地介绍。显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的实施例。
图1为根据本发明一个或多个实施例的地面导向箭头的构建方法的流程图。
图2为根据本发明一个或多个实施例的掩膜图确定方法的流程图。
图3为根据本发明一个或多个实施例的地面导向箭头识别方法的流程图。
图4为根据本发明或多个一个实施例的地面导向箭头的构建方法的流程图。
图5为根据本发明一个或多个实施例的地面导向箭头的构建装置的结构示意图。
图6为根据本发明一个或多个实施例的电子设备的结构示意图。
下面将结合附图，对本发明的实施例进行清楚、完整地描述。显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于所描述的 实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明的范围。
本发明的一个或多个实施例提供了一种地面导向箭头的构建方法，如图1所示，该方法包括：
S101：获取连续采集的道路图像，并确定每张道路图像中地面导向箭头所在的区域，作为箭头区域。
S102：针对每一张道路图像，生成该道路图像中每一箭头区域对应的掩膜图。
S103：对生成的掩膜图进行追踪，确定对应于同一地面导向箭头的多个掩膜图。
S104：基于每一地面导向箭头的多个掩膜图中箭头区域，确定每一地面导向箭头的箭头类别和箭头轮廓。
S105：根据每一地面导向箭头的箭头类别和箭头轮廓，构建位于三维电子地图中的地面导向箭头。
如图1所示，根据本发明一个或多个实施例提供的地面导向箭头的构建方法，可以在获取道路图像后，通过确定道路图像中的箭头区域、生成箭头区域对应的掩膜图等一系列操作，获取地面导向箭头的箭头类别和箭头轮廓，进而根据获取到的箭头类别和箭头轮廓，构建位于三维电子地图中的地面导向箭头，这一过程不需要人工识别箭头类别和箭头轮廓，从而提高了地面导向箭头的构建效率。
为了清楚地阐述本发明的实施例，下面分步骤阐述本发明一个或多个实施例提供的如图1所示的地面导向箭头的构建方法。
针对步骤S101，道路图像为对道路进行采集得到的图像，针对采集的道路图像而言，其可以划分为不含包地面导向箭头的道路图像和包含有地面导向箭头的道路图像，本方案仅涉及到包含有地面导向箭头的道路图像，为了更清楚阐述本发明实施例，本发明实施例中道路图像均为包含有地面导向箭头的道路图形。
根据一个或多个实施例，针对不同的应用场景和应用需求采用不同的道路 图像的获取方式，如，可以从存储有预先采集的道路图像的数据库中读取道路图像，也可以从图像采集设备的数据存储单元中拷贝图像采集设备采集的道路图像，还可以为从图像采集设备实时获取的道路图像，其中，图像采集设备可以为安装在采集车上的相机、摄像头、图像传感器等。
受制于采集道路图像时的道路环境、图像采集设备和采集角度等，道路图像中包含的地面导向箭头的数量是不固定的，每张道路图像可以仅包含一个地面导向箭头，也可以包含有多个地面导向箭头。当道路图像中仅包含有一个地面导向箭头时，可以将该地面导向箭头所在的区域作为箭头区域。当道路图像中包含有多个地面导向箭头时，针对每个地面导向箭头确定出相互独立的箭头区域。在一个实施例中，上述箭头区域可以为地面导向箭头所在的最小区域。
根据本发明的一个或多个实施例，针对每张道路图像，可以使用图像分割模型对每张道路图像进行像素分割，分割出道路图像中每张地面导向箭头的像素区域，该像素区域即为箭头区域。在一个实施例中，图像分割模型可以为deeplabv3。
针对步骤S102，每一箭头区域对应的掩膜图为：将该箭头区域内像素的像素值设为第一像素值，该箭头区域外的像素的像素值设为第二像素值所生成的图像。如，将该箭头区域内像素的像素值设为1，该箭头区域外的像素的像素值设为0。
根据一个或多个实施例，当道路图像中仅包含有一个地面导向箭头时，则通过道路图像仅能生成一张掩膜图，当道路图像中包含有多个地面导向箭头时，则针对每个地面导向箭头所在的区域均生成一张掩膜图，从而得到多张掩膜图。
针对步骤S103，针对道路中的地面导向箭头，图像采集设备在经过该地面导向箭头时，可能会对该地面导向箭头进行连续多次的采集，因而，针对道路中的一个地面导向箭头可能会被记录在多张道路图像中，这也就意味着在所有所得到的掩膜图中，对应于同一地面导向箭头的掩膜图可以为多个。
根据一个或多个实施例，可以对每个掩膜图中的箭头区域进行追踪，从而确定对应于同一地面导向箭头的多个掩膜图。
如针对地面导向箭头T，图像采集设备在经过地面导向箭头T采集到了三张道路图像，分别为道路图像1、道路图像2和道路图像3，其中，地面导向箭头T 分别位于道路图像1、道路图像2和道路图像3的底部区域、中部区域和顶部区域，经过步骤S102，可以得到地面导向箭头T分别在道路图像1、道路图像2和道路图像3所在区域对应的掩膜图1、掩膜图2和掩膜图3。在最先得到掩膜图1时，通过对掩膜图1进行追踪，可以确定掩膜图2和掩膜图3与掩膜图1对应于同一个地面导向箭头。
针对步骤S104，其中，箭头类别包括：直行箭头、左转箭头、右转箭头等，而箭头轮廓可以为以箭头区域的边缘像素的像素坐标集合描述。
根据本发明的一个或多个实施例，可以对每一地面导向箭头的多个掩膜图中箭头区域进行形状识别，根据所识别的区域形状确定每一地面导向箭头的箭头类别，并且所识别区域的区域轮廓即为每一地面导向箭头的的箭头轮廓。在一个实施例中，可以根据确定出的箭头类别的数量确定地面导向箭头的箭头类别。例如，可以将占比最多的箭头类别作为地面导向箭头的箭头类别。
根据一个或多个实施例，可以对确定出各箭头轮廓进行融合，将融合后的箭头轮廓作为地面导向箭头的箭头轮廓。还可以先选择出最优的箭头轮廓作为地面导向箭头的箭头轮廓。
针对步骤S105，在确定每一地面导向箭头的箭头类别和箭头轮廓后，可以将箭头轮廓对应的像素坐标集合转换为三维电子地图中的三维坐标集合，以将箭头轮廓投影在三维电子地图中，并且将三维电子地图中箭头轮廓与确定的箭头类别关联，以生成三维电子地图中的地面导向箭头。
下面将结合具体实施例对本发明的技术方案进行阐述。
根据本发明的一个或多个实施例，可以将各掩膜图划分为多组，每组由第一类掩膜图和第二类掩膜图组成，其中，第一类掩膜图为：第一道路图像内各箭头区域对应的掩膜图，第二类掩膜图为：第二道路图像内各箭头区域对应的掩膜图，第一道路图像与第二道路图像采集时刻相邻，且第一道路图像的采集时刻早于第二道路图像。
示例性的，存在连续采集的道路图像A、道路图像B、道路图像C，其中，道路图像A内各箭头区域对应的掩膜图包括掩膜图A1、掩膜图A2和掩膜图A3；道路图像B内各箭头区域对应的掩膜图包括掩膜图B1、掩膜图B2和掩膜图B3；道路图像C内各箭头区域对应的掩膜图包括掩膜图C1、掩膜图C2和掩膜图C3。 其中，道路图像A对应的掩膜图和道路图像B对应的掩膜图可以组合成一组，该组中，掩膜图A1、掩膜图A2和掩膜图A3为第一类掩膜图，掩膜图B1、掩膜图B2和掩膜图B3为第二类掩膜图。同样的，道路图像B对应的掩膜图和道路图像C对应的掩膜图可以组合成一组，该组中，掩膜图B1、掩膜图B2和掩膜图B3为第一类掩膜图，掩膜图C1、掩膜图C2和掩膜图C3为第二类掩膜图。
需要说明的是，在实际使用的过程中，当道路图像为实时采集时，上述分组过程可以为实时进行的，举例而言，当在T时刻采集到道路图像，可以将T-1时刻采集到道路图像对应的掩膜图作为第一类掩膜图，将T时刻采集到道路图像对应的掩膜图作为第二类掩膜图，当图像采集设备在T+1时刻继续采集图像时，可以将T时刻采集到道路图像对应的掩膜图作为第一类掩膜图，将T+1时刻采集到道路图像对应的掩膜图作为第二类掩膜图。
针对每组中的第一类掩膜图和第二类掩膜图，可以按照如图2所示的掩膜图确定方法实现从各组第一类掩模图和第二类掩膜图中选择箭头区域表示同一地面导向箭头的掩膜图，包括：
S201：针对连续采集的多帧道路图像中的每帧道路图像，基于该道路图像的掩膜图中箭头区域的位置，预测该掩膜图所对应的地面导向箭头在下一帧道路图像中所在的箭头区域。
本步骤中，可以结合基于每一道路图像的掩膜图中箭头区域的位置，预测每一掩膜图所对应的地面导向箭头在下一帧道路图像中所在的箭头区域。
根据一个或多个实施例，以每一道路图像的掩膜图中箭头区域的位置，按照运动信息所表征的在一个采集周期所运行的距离、方向，预测每一掩膜图所对应的地面导向箭头在下一帧道路图像中所在的箭头区域。其中，运动信息可以为每一道路图像的掩膜图中箭头区域所表示地面导向箭头在相邻道路图像所处区域的位置相差信息，也可以为基于图像采集设备的采集周期、图像采集设备所在车辆的行驶速度、图像采集设备采集道路图像时的拍摄角度等参数所确定的运动参数。
根据本发明的一个或多个实施例，针对连续采集的多帧道路图像中的每帧道路图像，还可以基于卡尔曼滤波算法预测该掩膜图所对应的地面导向箭头在下一帧道路图像中所在的箭头区域。
S202：计算每帧道路图像中每一箭头区域的预测箭头区域与下一帧道路图像中每一箭头区域之间的交并比，并基于所计算的交并比，生成每帧道路图像中每一箭头区域与下一帧道路图像中每一箭头区域的匹配成本矩阵。
根据本发明的一个或多个实施例，可以基于每一箭头区域的预测箭头区域与下一帧道路图像中每一箭头区域的相对位置，每一箭头区域的预测箭头区域与下一帧道路图像中每一箭头区域之间的交并比。
其中，上述交并比为：两个区域的相交所得到的交集区域和两个区域并集区域的面积比值，两个区域的交并比可以反映两个区域的重叠率，当交并比为1时，表示两个区域完全重叠，当交并比为0时，表示两个区域完全不重叠。
根据本发明的一个或多个实施例，可以基于计算得到的交并比生成该组的交并比矩阵。举例而言，预测箭头区域的数量为M个，下一帧道路图像中箭头区域为N个，则可以建立M×N的交并补矩阵，作为每帧道路图像中每一箭头区域与下一帧道路图像中每一箭头区域的匹配成本矩阵。
S203：针对每帧道路图像的匹配成本矩阵，基于匈牙利算法对该匹配成本矩阵进行求解，得到每帧道路图像中各箭头区域与下一帧道路图像中各箭头区域匹配成本最低的匹配结果，其中，匹配结果用于指示相匹配的箭头区域，相匹配的箭头区域所表示的地面导向箭头相同。
其中，匈牙利算法(Hungarian Algorithm)是一种在多项式时间内求解任务分配问题的组合优化算法，可以计算每一种匹配结果的匹配成本。其中，匹配结果用于指示相匹配的箭头区域，而相匹配的箭头区域所表示的地面导向箭头相同。进而可以选择出每帧道路图像中各箭头区域与下一帧道路图像中各箭头区域匹配成本最低的匹配结果。
S204：将相匹配的箭头区域对应的掩膜图确定为对应于同一地面导向箭头的多个掩膜图。
本步骤中，示例性的，箭头区域1与箭头区域2相匹配、箭头区域2与箭头区域3相匹配、箭头区域3与箭头区域4相匹配，则箭头区域1、箭头区域2、箭头区域3和箭头区域4对应的掩膜图为对应于同一地面导向箭头的多个掩膜图。
根据本发明的一个或多个实施例，还可以基于SORT(Simple Online And Realtime Tracking，简单在线和实时跟踪)算法，以实现对表示同一地面导向箭头的掩膜图的追踪。
在使用SORT算法对地面导向箭头进行跟踪的过程中，当存在地面导向箭头未被追踪到的次数大于预设次数时，可以将该地面导向箭头作新目标，并对该地面导向箭头重复执行上述操作，如重新确定该地面导向箭头所表的箭头区域等。
根据一个或多个实施例，在图1所示的基础上，针对每一地面导向箭头的多个掩膜图，可以采用如图3所示地面导向箭头识别方法确定每一地面导向箭头的箭头类别和箭头轮廓，包括：
S301：针对每一地面导向箭头的多个掩膜图中每一掩膜图，预测该箭头区域表示的地面导向箭头的预测箭头类别，并确定该掩膜图中的箭头区域所表示地面导向箭头的形态信息。
本步骤中，针对每一箭头区域，可以根据该箭头区域的区域轮廓上像素之间的位置关系，与预设的、不同箭头类别的地面导向箭头的箭头区域上像素之间的位置关系做对比，将位置关系最相似的预设的地面导向箭头所对应的箭头类别作为对应箭头区域的预测箭头类别。
上述形态信息包括：轮廓完整度、轮廓光滑度和与预设箭头轮廓的轮廓贴合度。
根据本发明的一个或多个实施例，还可以通过ResNet50(Residual Network 50，残余网络50)模型对每一掩膜图进行处理，得到每一掩膜图中箭头区域所表示地面导向箭头的预测箭头类别和形态信息。
S302：针对每一地面导向箭头，从该地面导向箭头的各预测箭头类别中，确定相同类别最多的预测箭头类别，作为该地面导向箭头的箭头类别，并且根据该地面导向箭头的各掩膜图中箭头区域的形态信息，确定该地面导向箭头的箭头轮廓的像素坐标集合。
本步骤中，针对每一地面导向箭头，可以从该地面导向箭头的各预测箭头类别中，确定相同类别最多的预测箭头类别，作为该地面导向箭头的箭头类别。
根据本发明的一个或多个实施例，由于箭头区域的区域轮廓的完整度，与 预设箭头轮廓的贴合程度以及区域轮廓的光滑程度，进而可以根据不同箭头区域的形态信息确定各地面导向箭头的形态的优劣程度，从而可以选取出表示的地面导向箭头形态最优的箭头区域，并以该箭头区域的区域轮廓作为该掩膜图集合对应的地面导向箭头的箭头轮廓，或者对该箭头区域的区域轮廓进行进一步的处理。
此时，可以按照下述步骤实现：
步骤1：基于该地面导向箭头的每一掩膜图中箭头区域的轮廓完整度、轮廓光滑度和轮廓贴合度，以及预设的轮廓完整度、轮廓光滑度和轮廓贴合度的评分权重，计算该掩膜图中箭头区域的形态评分。
步骤2：从该地面导向箭头的各掩膜图中箭头区域中，选取形态评分最高的箭头区域，作为该地面导向箭头的目标区域。
本步骤中，根据一个或多个实施例，形态信息可以为根据箭头区域的区域轮廓的完整度，与预设箭头轮廓的贴合程度以及区域轮廓的光滑程度所确定的形态评分，形态评分越高表示箭头区域所表示的地面导向箭头的形态越好，可以选取形态评分最高的箭头区域，作为目标区域。
步骤3：提取该地面导向箭头的目标区域的轮廓，并对所提取的轮廓进行平滑处理，将平滑处理后的轮廓，作为该地面导向箭头的箭头轮廓。
本步骤中，可以提取目标区域的边缘像素的轮廓，可以将所提取的轮廓作为地面导向箭头箭头轮廓。还可以对所提取的轮廓进行平滑处理，将平滑处理后的轮廓，作为地面导向箭头的箭头轮廓。根据一个或多个实施例，可以使用Douglas-Peucker(道格拉斯-皮克)算法对所提取的轮廓进行平滑处理。
步骤4：提取该地面导向箭头的箭头轮廓的像素坐标集合。
本步骤中，在确定出地面导向箭头的箭头轮廓后，可以提取该地面导向箭头的箭头轮廓的像素坐标集合。根据一个或多个实施例，可以使用八邻域算法提取目标区域的箭头轮廓的的像素坐标，组成像素坐标集合。
如图3所示，根据本发明一个或多个实施例提供的地面导向箭头识别方法，由于可以根据掩膜图集合内每一掩膜图中的箭头区域确定出与掩膜图集合对应的地面导向箭头的箭头类别和箭头轮廓，从而为基于道路图像构建位于三维 电子地图中的地面导向箭头提供了基础，可以提高地面导向箭头的构建效率。
根据一个或多个实施例，在如图1所示地面导向箭头构建方法的基础上，针对每一掩膜图集合，可以按照如图4所示地面导向箭头的构建方法在三维电子地图中构建每一掩膜图集合对应的地面导向箭头，包括：
S401：根据采集每一地面导向箭头的道路图像时图像采集设备的设备外参，将每一地面导向箭头的箭头轮廓从图像像素坐标系转换到图像物理坐标系。
S402：根据采集每一地面导向箭头的道路图像时图像采集设备的设备外参和设备内参，将每一地面导向箭头的箭头轮廓从图像物理坐标系转换到相机坐标系。
S403：根据采集每一地面导向箭头的道路图像时图像采集设备的设备外参和设备内参，将每一地面导向箭头的箭头轮廓从图像物理坐标系转换到相机坐标系。
S404：基于每一地面导向箭头的在世界坐标系下的箭头轮廓，计算每一地面导向箭头的投影位置。
S405：在每一地面导向箭头的投影位置处构建对应于每一地面导向箭头的箭头类别的地面导向箭头。
其中，图像采集设备的设备外参包括采集道路图像时与道路的高度差、所处的经纬度等参数，而设备内参则包括该图像采集设备的焦距等参数。
在一个实施例中，可以按照以下公式将地面导向箭头投影在三维电子地图中：
其中，(u，v)为像素点在像素坐标下的坐标，dx为每个像素在横轴x上的物理尺寸，dy为每个像素在物理纵轴y上的尺寸，u0和v0是图像平面中心，f为 图像采集设备的交局，R为旋转矩阵、T为平移向量，(X
w，Y
w，Z
w)表示在三维电子图像中的投影坐标。
相应于上述地面导向箭头的构建装置方法，如图5所示，本发明一个或多个实施例还提供了一种地面导向箭头的构建装置，该装置包括：
图像获取模块501，用于获取连续采集的道路图像，并确定每张道路图像中地面导向箭头所在的区域，作为箭头区域；
掩膜图生成模块502，用于针对每一张道路图像，生成该道路图像中每一箭头区域对应的掩膜图；
掩膜图追踪模块503，用于对生成的掩膜图进行追踪，确定对应于同一地面导向箭头的多个掩膜图；
信息确定模块504，用于基于每一地面导向箭头的多个掩膜图中箭头区域，确定每一地面导向箭头的箭头类别和箭头轮廓；
箭头构建模块505，用于根据每一地面导向箭头的箭头类别和箭头轮廓，构建位于三维电子地图中的地面导向箭头。
根据一个或多个实施例，掩膜图追踪模块，具体用于针对连续采集的多帧道路图像中的每帧道路图像，基于该道路图像的掩膜图中箭头区域的位置，预测该掩膜图所对应的地面导向箭头在下一帧道路图像中所在的箭头区域；计算每帧道路图像中每一箭头区域的预测箭头区域与下一帧道路图像中每一箭头区域之间的交并比，并基于所计算的交并比，生成每帧道路图像中每一箭头区域与下一帧道路图像中每一箭头区域的匹配成本矩阵；针对每帧道路图像的匹配成本矩阵，基于匈牙利算法对该匹配成本矩阵进行求解，得到每帧道路图像中各箭头区域与下一帧道路图像中各箭头区域匹配成本最低的匹配结果，其中，匹配结果用于指示相匹配的箭头区域，相匹配的箭头区域所表示的地面导向箭头相同；将相匹配的箭头区域对应的掩膜图确定为对应于同一地面导向箭头的多个掩膜图。
根据一个或多个实施例，信息确定模块，具体用于针对每一地面导向箭头的多个掩膜图中每一掩膜图，预测该箭头区域表示的地面导向箭头的预测箭头类别，并确定该掩膜图中的箭头区域所表示地面导向箭头的形态信息；针对每 一地面导向箭头，从该地面导向箭头的各预测箭头类别中，确定相同类别最多的预测箭头类别，作为该地面导向箭头的箭头类别，并且根据该地面导向箭头的各掩膜图中箭头区域的形态信息，确定该地面导向箭头的箭头轮廓的像素坐标集合。
根据一个或多个实施例，形态信息包括：轮廓完整度、轮廓光滑度和与预设箭头轮廓的轮廓贴合度；
信息确定模块，具体用于基于该地面导向箭头的每一掩膜图中箭头区域的轮廓完整度、轮廓光滑度和轮廓贴合度，以及预设的轮廓完整度、轮廓光滑度和轮廓贴合度的评分权重，计算该掩膜图中箭头区域的形态评分；从该地面导向箭头的各掩膜图中箭头区域中，选取形态评分最高的箭头区域，作为该地面导向箭头的目标区域；提取该地面导向箭头的目标区域的轮廓，并对所提取的轮廓进行平滑处理，将平滑处理后的轮廓，作为该地面导向箭头的箭头轮廓；提取该地面导向箭头的箭头轮廓的像素坐标集合。
根据一个或多个实施例，箭头构建模块，具体用于根据采集每一地面导向箭头的道路图像时图像采集设备的设备外参，将每一地面导向箭头的箭头轮廓从图像像素坐标系转换到图像物理坐标系；根据采集每一地面导向箭头的道路图像时图像采集设备的设备外参和设备内参，将每一地面导向箭头的箭头轮廓从图像物理坐标系转换到相机坐标系；根据采集每一地面导向箭头的道路图像时图像采集设备的设备外参和图像采集设备在世界坐标系下的世界坐标，将每一地面导向箭头的箭头轮廓从相机坐标系转换到世界坐标系；基于每一地面导向箭头的在世界坐标系下的箭头轮廓，计算每一地面导向箭头的投影位置；在每一地面到像箭头的投影位置处构建对应于每一地面导向箭头的箭头类别的地面导向箭头。
如图5所示，根据本发明一个或多个实施例提供的地面导向箭头的构建装置，由于可以通过确定道路图像中的箭头区域、生成箭头区域对应的掩膜图等一系列操作，获取地面导向箭头的箭头类别和箭头轮廓，进而根据获取到的箭头类别和箭头轮廓，构建位于三维电子地图中的地面导向箭头，这一过程不需要人工识别箭头类别和箭头轮廓，从而提高了地面导向箭头的构建效率。
本发明的一个或多个实施例还提供了一种电子设备，如图6所示，该电子 设备包括处理器601、通信接口602、存储器603和通信总线604，其中，处理器601，通信接口602，存储器603通过通信总线604完成相互间的通信。存储器603用于存放计算机程序。处理器601用于执行存储器603上所存放的程序，以实现如下步骤：
获取连续采集的道路图像，并确定每张道路图像中地面导向箭头所在的区域，作为箭头区域；
针对每一张道路图像，生成该道路图像中每一箭头区域对应的掩膜图；
对生成的掩膜图进行追踪，确定对应于同一地面导向箭头的多个掩膜图；
基于每一地面导向箭头的多个掩膜图中箭头区域，确定每一地面导向箭头的箭头类别和箭头轮廓；
根据每一地面导向箭头的箭头类别和箭头轮廓，构建位于三维电子地图中的地面导向箭头。
如图6所示，根据本发明一个或多个实施例提供的电子设备，由于可以通过确定道路图像中的箭头区域、生成箭头区域对应的掩膜图等一系列操作，获取地面导向箭头的箭头类别和箭头轮廓，进而根据获取到的箭头类别和箭头轮廓，构建位于三维电子地图中的地面导向箭头，这一过程不需要人工识别箭头类别和箭头轮廓，从而提高了地面导向箭头的构建效率。
需要说明的是，上述电子设备实现地面导向箭头的构建的其他实施例，与前述方法实施例部分提及的地面导向箭头的构建方法相同，在此不再赘述。
上述电子设备提到的通信总线可以是外设部件互连标准(Peripheral Component Interconnect，PCI)总线或扩展工业标准结构(Extended Industry Standard Architecture，EISA)总线等。该通信总线可以分为地址总线、数据总线、控制总线等。为便于表示，图中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
通信接口用于上述电子设备与其他设备之间的通信。
存储器可以包括随机存取存储器(Random Access Memory，RAM)，也可以包括非易失性存储器(Non-Volatile Memory，NVM)，例如至少一个磁盘存储器。在一个实施例中，存储器还可以是至少一个位于远离前述处理器的存 储装置。
上述的处理器可以是通用处理器，包括中央处理器(Central Processing Unit，CPU)、网络处理器(Network Processor，NP)等；还可以是数字信号处理器(Digital Signal Processing，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。
在本发明提供的又一实施例中，还提供了一种计算机可读存储介质，该计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现上述任一地面导向箭头的构建方法的步骤。
在本发明提供的又一实施例中，还提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述实施例中任一地面导向箭头的构建方法。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本发明实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘Solid State Disk(SSD))等。
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含” 或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个......”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
本说明书中的各个实施例均采用相关的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于电子设备、计算机可读存储介质和计算机程序产品实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
根据本发明的一个或多个实施例，在获取连续采集的道路图像后，通过确定道路图像中的箭头区域、生成箭头区域对应的掩膜图等一系列操作，获取地面导向箭头的箭头类别和箭头轮廓，进而根据获取到的箭头类别和箭头轮廓，构建位于三维电子地图中的地面导向箭头，这一过程不需要人工识别箭头类别和箭头轮廓，从而提高了地面导向箭头的构建效率。
尽管已经针对有限数量的实施例描述了本发明，但是受益于本公开的本领域普通技术人员将理解，可以设计其他实施例而不脱离本文所公开的本发明的范围。因此，本发明的范围应仅由所附权利要求书限制。
Claims (14)
- 一种地面导向箭头的构建方法，包括：获取连续采集的道路图像，并确定每张所述道路图像中地面导向箭头所在的区域，作为箭头区域；针对每一张所述道路图像，生成该道路图像中每一所述箭头区域对应的掩膜图；对生成的掩膜图进行追踪，确定对应于同一地面导向箭头的多个掩膜图；基于每一所述地面导向箭头的所述多个掩膜图中的所述箭头区域，确定每一所述地面导向箭头的箭头类别和箭头轮廓；根据每一所述地面导向箭头的所述箭头类别和所述箭头轮廓，构建位于三维电子地图中的地面导向箭头。
- 根据权利要求1所述的方法，其中，所述对生成的掩膜图进行追踪，确定对应于同一地面导向箭头的多个掩膜图，包括：针对每帧所述道路图像，基于该道路图像的所述掩膜图中所述箭头区域的位置，预测该掩膜图所对应的地面导向箭头在下一帧所述道路图像中所在的箭头区域；计算每帧所述道路图像中每一所述箭头区域的预测箭头区域与下一帧所述道路图像中每一所述箭头区域之间的交并比，并基于所计算的交并比，生成每帧所述道路图像中每一所述箭头区域与下一帧所述道路图像中每一所述箭头区域的匹配成本矩阵；针对每帧所述道路图像的所述匹配成本矩阵，基于匈牙利算法对该匹配成本矩阵进行求解，得到每帧所述道路图像中各所述箭头区域与下一帧所述道路图像中各所述箭头区域匹配成本最低的匹配结果，其中，所述匹配结果用于指示相匹配的箭头区域，相匹配的箭头区域所表示的地面导向箭头相同；将相匹配的箭头区域对应的所述掩膜图确定为对应于同一地面导向箭头的所述多个掩膜图。
- 根据权利要求1所述的方法，其中，所述基于每一所述地面导向箭头的所述多个掩膜图中的所述箭头区域，确定每一所述地面导向箭头的箭头类别和 箭头轮廓，包括：针对每一所述地面导向箭头的所述多个掩膜图中每一掩膜图，预测该箭头区域表示的所述地面导向箭头的预测箭头类别，并确定该掩膜图中的所述箭头区域所表示的所述地面导向箭头的形态信息；针对每一所述地面导向箭头，从该地面导向箭头的各所述预测箭头类别中，确定相同类别最多的预测箭头类别，作为该地面导向箭头的所述箭头类别，并且根据该地面导向箭头的各所述掩膜图中所述箭头区域的所述形态信息，确定该地面导向箭头的所述箭头轮廓的像素坐标集合。
- 根据权利要求3所述的方法，其中，所述形态信息包括轮廓完整度、轮廓光滑度和与预设箭头轮廓的轮廓贴合度，所述根据该地面导向箭头的各所述掩膜图中所述箭头区域的所述形态信息，确定该地面导向箭头的所述箭头轮廓的像素坐标集合，包括：基于该地面导向箭头的每一所述掩膜图中所述箭头区域的所述轮廓完整度、所述轮廓光滑度和所述轮廓贴合度，以及预设的轮廓完整度、轮廓光滑度和轮廓贴合度的评分权重，计算该掩膜图中所述箭头区域的形态评分；从该地面导向箭头的各所述掩膜图中的所述箭头区域中，选取所述形态评分最高的箭头区域，作为该地面导向箭头的目标区域；提取该地面导向箭头的所述目标区域的轮廓，并对所提取的轮廓进行平滑处理，将平滑处理后的轮廓，作为该地面导向箭头的所述箭头轮廓；提取该地面导向箭头的所述箭头轮廓的所述像素坐标集合。
- 根据权利要求1所述的方法，其中，所述根据每一所述地面导向箭头的所述箭头类别和所述箭头轮廓，构建位于三维电子地图中的地面导向箭头，包括：根据采集每一所述地面导向箭头的所述道路图像时图像采集设备的设备外参，将每一所述地面导向箭头的所述箭头轮廓从图像像素坐标系转换到图像物理坐标系；根据采集每一所述地面导向箭头的所述道路图像时所述图像采集设备的所述设备外参和设备内参，将每一所述地面导向箭头的所述箭头轮廓从所述图像物理坐标系转换到相机坐标系；根据采集每一所述地面导向箭头的所述道路图像时所述图像采集设备的所述设备外参和所述图像采集设备在世界坐标系下的世界坐标，将每一所述地面导向箭头的所述箭头轮廓从所述相机坐标系转换到所述世界坐标系；基于每一所述地面导向箭头的在所述世界坐标系下的箭头轮廓，计算每一所述地面导向箭头的投影位置；在每一所述地面导向箭头的所述投影位置处构建对应于每一所述地面导向箭头的所述箭头类别的地面导向箭头。
- 根据权利要求2所述的方法，其中，所述基于该道路图像的所述掩膜图中所述箭头区域的位置，预测该掩膜图所对应的地面导向箭头在下一帧所述道路图像中所在的箭头区域，包括：基于运动信息预测每一所述掩膜图所对应的地面导向箭头在下一帧所述道路图像中所在的箭头区域；或基于卡尔曼滤波算法预测每一所述掩膜图所对应的地面导向箭头在下一帧所述道路图像中所在的箭头区域。
- 一种地面导向箭头的构建装置，包括：图像获取模块，用于获取连续采集的道路图像，并确定每张所述道路图像中地面导向箭头所在的区域，作为箭头区域；掩膜图生成模块，用于针对每一张所述道路图像，生成该道路图像中每一所述箭头区域对应的掩膜图；掩膜图追踪模块，用于对生成的掩膜图进行追踪，确定对应于同一地面导向箭头的多个掩膜图；信息确定模块，用于基于每一所述地面导向箭头的所述多个掩膜图中的所述箭头区域，确定每一所述地面导向箭头的箭头类别和箭头轮廓；箭头构建模块，用于根据每一所述地面导向箭头的所述箭头类别和所述箭 头轮廓，构建位于三维电子地图中的地面导向箭头。
- 根据权利要求7所述的装置，其中，所述掩膜图追踪模块，具体用于针对每帧所述道路图像，基于该道路图像的所述掩膜图中所述箭头区域的位置，预测该掩膜图所对应的地面导向箭头在下一帧道路图像中所在的箭头区域；计算每帧所述道路图像中每一所述箭头区域的预测箭头区域与下一帧道路图像中每一所述箭头区域之间的交并比，并基于所计算的交并比，生成每帧所述道路图像中每一所述箭头区域与下一帧道路图像中每一所述箭头区域的匹配成本矩阵；针对每帧所述道路图像的所述匹配成本矩阵，基于匈牙利算法对该匹配成本矩阵进行求解，得到每帧所述道路图像中各所述箭头区域与下一帧道路图像中各所述箭头区域匹配成本最低的匹配结果，其中，所述匹配结果用于指示相匹配的箭头区域，相匹配的箭头区域所表示的地面导向箭头相同；将相匹配的箭头区域对应的掩膜图确定为对应于同一地面导向箭头的所述多个掩膜图。
- 根据权利要求7所述的装置，其中，所述信息确定模块，具体用于针对每一所述地面导向箭头的所述多个掩膜图中的每一掩膜图，预测该箭头区域表示的所述地面导向箭头的预测箭头类别，并确定该掩膜图中的所述箭头区域所表示地面导向箭头的形态信息；针对每一所述地面导向箭头，从该地面导向箭头的各预测箭头类别中，确定相同类别最多的预测箭头类别，作为该地面导向箭头的所述箭头类别，并且根据该地面导向箭头的各所述掩膜图中所述箭头区域的所述形态信息，确定该地面导向箭头的所述箭头轮廓的像素坐标集合。
- 根据权利要求9所述的装置，其中，所述形态信息包括轮廓完整度、轮廓光滑度和与预设箭头轮廓的轮廓贴合度，在执行所述根据该地面导向箭头的各所述掩膜图中所述箭头区域的所述形态信息，确定该地面导向箭头的所述箭头轮廓的像素坐标集合时，所述信息确定模块具体用于：基于该地面导向箭头的每一所述掩膜图中所述箭头区域的所述轮廓完整度、所述轮廓光滑度和所述轮廓贴合度，以及预设的轮廓完整度、轮廓光滑度和轮廓贴合度的评分权重，计算该掩膜图中所述箭头区域的形态评分；从该地面导向箭头的各所述掩膜图中的所述箭头区域中，选取所述形态评 分最高的箭头区域，作为该地面导向箭头的目标区域；提取该地面导向箭头的所述目标区域的轮廓，并对所提取的轮廓进行平滑处理，将平滑处理后的轮廓，作为该地面导向箭头的所述箭头轮廓；提取该地面导向箭头的所述箭头轮廓的所述像素坐标集合。
- 根据权利要求7所述的装置，其中，所述箭头构建模块具体用于：根据采集每一所述地面导向箭头的所述道路图像时图像采集设备的设备外参，将每一所述地面导向箭头的所述箭头轮廓从图像像素坐标系转换到图像物理坐标系；根据采集每一所述地面导向箭头的所述道路图像时所述图像采集设备的所述设备外参和设备内参，将每一所述地面导向箭头的所述箭头轮廓从所述图像物理坐标系转换到相机坐标系；根据采集每一所述地面导向箭头的所述道路图像时所述图像采集设备的所述设备外参和所述图像采集设备在世界坐标系下的世界坐标，将每一所述地面导向箭头的所述箭头轮廓从所述相机坐标系转换到所述世界坐标系；基于每一所述地面导向箭头的在所述世界坐标系下的箭头轮廓，计算每一所述地面导向箭头的投影位置；在每一所述地面导向箭头的所述投影位置处构建对应于每一所述地面导向箭头的所述箭头类别的地面导向箭头。
- 根据权利要求8所述的装置，其中，在执行所述基于该道路图像的所述掩膜图中所述箭头区域的位置，预测该掩膜图所对应的地面导向箭头在下一帧所述道路图像中所在的箭头区域时，所述掩膜图追踪模块具体用于：基于运动信息预测每一所述掩膜图所对应的地面导向箭头在下一帧所述道路图像中所在的箭头区域；或基于卡尔曼滤波算法预测每一所述掩膜图所对应的地面导向箭头在下一帧所述道路图像中所在的箭头区域。
- 一种电子设备，包括：通信接口；通信总线；存储器，用于存放计算机程序；和处理器，用于执行所述存储器上所存放的程序，以实现权利要求1-6中任一项所述的方法，其中，所述处理器、所述通信接口和所述存储器通过所述通信总线完成相互间的通信。
- 一种计算机可读存储介质，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现权利要求1-6中任一项所述的方法。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011367386.1A CN112132853B (zh) | 2020-11-30 | 2020-11-30 | 地面导向箭头的构建方法及其装置、电子设备及存储介质 |
| CN202011367386.1 | 2020-11-30 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2022110862A1 true WO2022110862A1 (zh) | 2022-06-02 |
Family
ID=73852078
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2021/109249 WO2022110862A1 (zh) | 2020-11-30 | 2021-07-29 | 地面导向箭头的构建方法及其装置、电子设备及存储介质 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112132853B (zh) |
| WO (1) | WO2022110862A1 (zh) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116030047A (zh) * | 2023-03-24 | 2023-04-28 | 四川中星电子有限责任公司 | 一种电容器工艺过程中识别掩膜合格的方法 |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112132853B (zh) * | 2020-11-30 | 2021-02-05 | 湖北亿咖通科技有限公司 | 地面导向箭头的构建方法及其装置、电子设备及存储介质 |
| CN113742440B (zh) * | 2021-09-03 | 2023-09-26 | 北京百度网讯科技有限公司 | 道路图像数据处理方法、装置、电子设备及云计算平台 |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007119493A1 (ja) * | 2006-04-17 | 2007-10-25 | Pioneer Corporation | 地図表示装置及び地図表示方法 |
| CN108986122A (zh) * | 2018-08-01 | 2018-12-11 | 重庆大学 | 室内泊车引导地图智能重构方法 |
| CN111311710A (zh) * | 2020-03-20 | 2020-06-19 | 北京四维图新科技股份有限公司 | 一种高精地图的制作方法和装置、电子设备、存储介质 |
| CN111382625A (zh) * | 2018-12-28 | 2020-07-07 | 沈阳美行科技有限公司 | 道路标识识别方法、装置及电子设备 |
| CN112132853A (zh) * | 2020-11-30 | 2020-12-25 | 湖北亿咖通科技有限公司 | 地面导向箭头的构建方法及其装置、电子设备及存储介质 |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104008377A (zh) * | 2014-06-07 | 2014-08-27 | 北京联合大学 | 基于时空关联的地面交通标志实时检测识别方法 |
| WO2018200522A1 (en) * | 2017-04-24 | 2018-11-01 | Mobileye Vision Technologies Ltd. | Systems and methods for compression of lane data |
| CN108132054A (zh) * | 2017-12-20 | 2018-06-08 | 百度在线网络技术（北京）有限公司 | 用于生成信息的方法和装置 |
| EP3865822A1 (en) * | 2018-05-15 | 2021-08-18 | Mobileye Vision Technologies Ltd. | Systems and methods for autonomous vehicle navigation |
| CN109271937B (zh) * | 2018-09-19 | 2020-09-25 | 深圳市赢世体育科技有限公司 | 基于图像处理的运动场地标志物识别方法及系统 |
| CN111179152B (zh) * | 2018-11-12 | 2023-04-28 | 阿里巴巴集团控股有限公司 | 一种道路标识识别方法及装置、介质、终端 |
| CN110135485A (zh) * | 2019-05-05 | 2019-08-16 | 浙江大学 | 单目相机与毫米波雷达融合的物体识别与定位方法和系统 |
| CN110501018B (zh) * | 2019-08-13 | 2021-11-02 | 广东星舆科技有限公司 | 一种服务于高精度地图生产的交通标志牌信息采集方法 |
-
2020
- 2020-11-30 CN CN202011367386.1A patent/CN112132853B/zh active Active
-
2021
- 2021-07-29 WO PCT/CN2021/109249 patent/WO2022110862A1/zh active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007119493A1 (ja) * | 2006-04-17 | 2007-10-25 | Pioneer Corporation | 地図表示装置及び地図表示方法 |
| CN108986122A (zh) * | 2018-08-01 | 2018-12-11 | 重庆大学 | 室内泊车引导地图智能重构方法 |
| CN111382625A (zh) * | 2018-12-28 | 2020-07-07 | 沈阳美行科技有限公司 | 道路标识识别方法、装置及电子设备 |
| CN111311710A (zh) * | 2020-03-20 | 2020-06-19 | 北京四维图新科技股份有限公司 | 一种高精地图的制作方法和装置、电子设备、存储介质 |
| CN112132853A (zh) * | 2020-11-30 | 2020-12-25 | 湖北亿咖通科技有限公司 | 地面导向箭头的构建方法及其装置、电子设备及存储介质 |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116030047A (zh) * | 2023-03-24 | 2023-04-28 | 四川中星电子有限责任公司 | 一种电容器工艺过程中识别掩膜合格的方法 |
| CN116030047B (zh) * | 2023-03-24 | 2023-05-30 | 四川中星电子有限责任公司 | 一种电容器工艺过程中识别掩膜合格的方法 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112132853B (zh) | 2021-02-05 |
| CN112132853A (zh) | 2020-12-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2022110862A1 (zh) | 地面导向箭头的构建方法及其装置、电子设备及存储介质 | |
| KR102126724B1 (ko) | 포인트 클라우드 데이터를 복구하기 위한 방법 및 장치 | |
| CN111563442B (zh) | 基于激光雷达的点云和相机图像数据融合的slam方法及系统 | |
| CN109285220B (zh) | 一种三维场景地图的生成方法、装置、设备及存储介质 | |
| CN113168717B (zh) | 一种点云匹配方法及装置、导航方法及设备、定位方法、激光雷达 | |
| KR102068419B1 (ko) | 포인트 클라우드 데이터 수집 궤적을 조정하기 위한 방법, 장치 및 컴퓨터 판독 가능한 매체 | |
| EP3574450A1 (en) | Method and apparatus for optimizing scan data and method and apparatus for correcting trajectory | |
| CN111815707A (zh) | 点云确定方法、点云筛选方法、装置、计算机设备 | |
| WO2020107326A1 (zh) | 车道线检测方法、设备、计算机可读存储介质 | |
| CN112036359B (zh) | 一种车道线的拓扑信息获得方法、电子设备及存储介质 | |
| CN111915657A (zh) | 一种点云配准方法、装置、电子设备及存储介质 | |
| CN111986214A (zh) | 一种地图中人行横道的构建方法和电子设备 | |
| CN112613551A (zh) | 汽车零部件识别方法、存储介质及系统 | |
| CN110851978B (zh) | 一种基于可见性的摄像机位置优化方法 | |
| CN114332134A (zh) | 一种基于密集点云的建筑物立面提取方法和装置 | |
| CN111709269B (zh) | 一种深度图像中基于二维关节信息的人手分割方法和装置 | |
| CN112381873A (zh) | 一种数据标注方法及装置 | |
| CN112818866A (zh) | 车辆定位的方法、装置及电子设备 | |
| CN113096104A (zh) | 目标分割模型的训练方法及装置和目标分割方法及装置 | |
| Kang et al. | 3D urban reconstruction from wide area aerial surveillance video | |
| CN113255405A (zh) | 车位线识别方法及其系统、车位线识别设备、存储介质 | |
| CN112215036A (zh) | 跨镜追踪方法、装置、设备及存储介质 | |
| WO2018120932A1 (en) | Method and apparatus for optimizing scan data and method and apparatus for correcting trajectory | |
| CN116977572B (zh) | 一种多尺度动态图卷积的建筑物立面结构提取方法 | |
| WO2023201723A1 (zh) | 目标检测模型的训练方法、目标检测方法及装置 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21896369 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 21896369 Country of ref document: EP Kind code of ref document: A1 |