CN115128626A

CN115128626A - Traffic sign vectorization method, system, terminal and storage medium

Info

Publication number: CN115128626A
Application number: CN202110330754.3A
Authority: CN
Inventors: 郭永春; 张超
Original assignee: Xi'an Navinfo Information Technology Co ltd
Current assignee: Xi'an Navinfo Information Technology Co ltd
Priority date: 2021-03-26
Filing date: 2021-03-26
Publication date: 2022-09-30

Abstract

The application provides a traffic sign vectorization method, a system, a terminal and a storage medium, the method carries out vectorization on the traffic sign based on a visual imaging technology, wherein an image sent by a data acquisition device is used as a data source, data acquisition is convenient and fast, acquisition channels are more, the traffic sign vectorization cost is correspondingly reduced, the traffic sign vectorization efficiency is improved, and the requirement of real-time updating is met. Moreover, the traffic signs are subjected to matching grouping, and the sign scenes to which the traffic signs belong are determined, so that the traffic signs subjected to matching grouping are subjected to vectorization according to different sign scenes, the vectorized sign accuracy is improved, the robustness is enhanced, and the method is suitable for application.

Description

Traffic sign vectorization method, system, terminal and storage medium

Technical Field

The present application relates to the field of navigation electronic map technologies, and in particular, to a traffic sign vectorization method, system, terminal, and storage medium.

Background

With the development of technology, the requirements of people on maps are further improved, and high-precision maps are applied to multiple aspects of people's life, such as automatic driving, path planning, navigation positioning and the like. In the field of high-precision maps, traffic signs are one of the most important map elements. When drawing, the vectorization of the traffic sign is an important link of drawing.

In the related art, vectorization of a traffic sign mainly includes scanning a high-density point cloud along a driving direction through a high-precision laser radar device, a Global Navigation Satellite System (GNSS) device, and an Inertial Measurement Unit (IMU) device, which are mounted on a collection device, and then obtaining a vector graph of the traffic sign based on the high-density point cloud.

However, the vectorization technology of the traffic sign needs to acquire high-density point cloud, which is high in cost and low in efficiency, and is difficult to meet the requirement of real-time update.

Disclosure of Invention

In order to solve the problems in the prior art, the application provides a traffic sign vectorization method, a traffic sign vectorization system, a traffic sign vectorization terminal and a storage medium.

In a first aspect, an embodiment of the present application provides a traffic sign vectorization method, including the following steps:

receiving a plurality of images to be processed sent by data acquisition equipment, wherein each image to be processed comprises a plurality of traffic signs to be vectorized and the internal and external parameters of the image;

dividing the same traffic sign to be vectorized in the multiple images to be processed into one group according to the multiple traffic signs to be vectorized in each image to be processed, and obtaining multiple groups of traffic signs to be vectorized;

determining the sign scene to which the plurality of traffic signs to be vectorized belong in each image to be processed according to the image internal and external parameters in each image to be processed;

and performing vectorization processing on the plurality of groups of traffic signs to be vectorized according to the sign scenes to which the plurality of traffic signs to be vectorized belong in each image to be processed.

In one possible implementation manner, the signage scene to which the plurality of traffic signs to be vectorized belong in each image to be processed includes side-by-side signs.

The vectorizing processing of the plurality of groups of traffic signs to be vectorized according to the sign scenes to which the plurality of traffic signs to be vectorized belong in each image to be processed includes:

determining traffic signs to be vectorized of a first target group in the plurality of groups of traffic signs to be vectorized according to the side-by-side signs;

according to the distance between each group of traffic signs to be vectorized and the data acquisition equipment in the traffic signs to be vectorized of the first target group, sequencing the images to be processed where each group of traffic signs to be vectorized is located;

acquiring a plurality of first target images from the to-be-processed images of each ordered group of traffic signs to be vectorized;

and acquiring two images meeting a first preset image interval from the plurality of first target images, and performing vectorization processing on each group of traffic signs to be vectorized by using the two images.

In a possible implementation manner, the side-by-side signs are a plurality of traffic signs to be vectorized, where the internal and external parameters of an image in one image to be processed are distributed according to a preset requirement.

In a possible implementation manner, the performing vectorization processing on each group of traffic signs to be vectorized by using the two images includes:

constructing the label homonymy points of the two images based on multi-feature matching and a quadtree technology;

carrying out triangulation on the same name points of the signs of the two images to obtain a spatial point cloud;

performing space plane fitting on the space point cloud to obtain a space plane;

and obtaining a vector graph of each group of traffic signs to be vectorized according to the corner points of each group of traffic signs to be vectorized and the space plane.

In a possible implementation manner, before the performing the spatial plane fitting on the spatial point cloud to obtain a spatial plane, the method further includes:

removing outliers in the spatial point cloud;

the space plane fitting is carried out on the space point cloud to obtain a space plane, and the method comprises the following steps:

and performing space plane fitting on the space point cloud with the discrete points removed to obtain the space plane.

In a possible implementation manner, each image to be processed further includes GNSS positioning data and IMU measurement data.

After the obtaining the vector graphics of each group of traffic signs to be vectorized according to the corner points of each group of traffic signs to be vectorized and the spatial plane, the method further includes:

and converting a coordinate system of the vector graphics of each group of traffic signs to be vectorized according to the GNSS positioning data and the IMU measurement data.

In one possible implementation manner, the signage scene to which the plurality of traffic signs to be vectorized belong in each image to be processed includes a separate sign.

The vectorization processing is performed on the multiple groups of traffic signs to be vectorized according to the sign scenes to which the multiple traffic signs to be vectorized belong in each image to be processed, and the vectorization processing includes:

determining traffic signs to be vectorized of a second target group in the plurality of groups of traffic signs to be vectorized according to the independent signs;

according to the distance between each group of traffic signs to be vectorized and the data acquisition equipment in the traffic signs to be vectorized of the second target group, sequencing the images to be processed where each group of traffic signs to be vectorized is located;

acquiring a plurality of second target images from the to-be-processed images of each ordered group of traffic signs to be vectorized;

and acquiring a plurality of images meeting a second preset image interval from the plurality of second target images, and performing vectorization processing on each group of traffic signs to be vectorized by using the plurality of images.

In a possible implementation manner, the independent signs are traffic signs to be vectorized remaining in one image to be processed, except for a plurality of traffic signs to be vectorized, where the internal and external parameters of the image are distributed according to preset requirements.

In a possible implementation manner, the vectoring, by using the multiple images, on each group of traffic signs to be vectorized includes:

constructing the label homonymous points of the multiple images based on multi-feature matching and a quadtree technology;

carrying out triangulation on the same name points of the signs of the multiple images to obtain a spatial point cloud;

removing outliers in the spatial point cloud;

and performing space plane fitting on the space point cloud without the discrete points to obtain the space plane.

In a possible implementation manner, the dividing, according to the multiple traffic signs to be vectorized in each image to be processed, the same traffic sign to be vectorized in the multiple images to be processed into a group to obtain multiple groups of traffic signs to be vectorized includes:

according to the plurality of traffic signs to be vectorized in each image to be processed, tracking and predicting the same traffic sign to be vectorized in the plurality of images to be processed;

performing similarity calculation on the traffic signs according to the tracking prediction result of the same traffic sign to be vectorized in the multiple images to be processed;

and dividing the same traffic sign to be vectorized in the multiple images to be processed into one group according to the similarity calculation result of the same traffic sign to be vectorized in the multiple images to be processed, so as to obtain multiple groups of traffic signs to be vectorized.

In a possible implementation manner, the performing, according to a tracking prediction result of a traffic sign to be vectorized in the same image to be processed, similarity calculation of the traffic sign includes:

calculating the intersection and union of the circumscribed rectangles of the same traffic sign to be vectorized in the front and back images to be processed in the plurality of images to be processed according to the tracking prediction result of the same traffic sign to be vectorized in the plurality of images to be processed;

according to the intersection and the union set, determining the similarity of the same traffic sign to be vectorized in the front and back images to be processed in the multiple images to be processed;

or

Calculating the feature similarity of the same traffic sign to be vectorized in the front and back images to be processed in the plurality of images to be processed according to the tracking prediction result of the same traffic sign to be vectorized in the plurality of images to be processed;

and determining the similarity of the traffic sign to be vectorized in the same two front and back images to be processed in the plurality of images to be processed according to the characteristic similarity.

In a second aspect, an embodiment of the present application provides a traffic sign vectoring device, including:

the image receiving module is used for receiving a plurality of images to be processed sent by the data acquisition equipment, wherein each image to be processed comprises a plurality of traffic signs to be vectorized and the internal and external parameters of the image;

the sign grouping module is used for dividing the same traffic sign to be vectorized in the multiple images to be processed into one group according to the multiple traffic signs to be vectorized in each image to be processed to obtain multiple groups of traffic signs to be vectorized;

the type determining module is used for determining the sign scene to which the plurality of traffic signs to be vectorized belong in each image to be processed according to the image internal and external parameters in each image to be processed;

and the sign vectorization module is used for carrying out vectorization processing on the plurality of groups of traffic signs to be vectorized according to the sign scenes to which the plurality of traffic signs to be vectorized belong in each image to be processed.

The sign vectorization module is specifically configured to:

and acquiring two images meeting a first preset image space from the plurality of first target images, and performing vectorization processing on each group of traffic signs to be vectorized by using the two images.

In a possible implementation manner, the signage vectorization module is specifically configured to:

removing outliers in the spatial point cloud;

The sign vectorization module is specifically configured to:

In a possible implementation manner, the sign vectorization module is specifically configured to:

removing outliers in the spatial point cloud;

The sign vectorization module is specifically configured to:

In a possible implementation manner, the tag grouping module is specifically configured to:

performing similarity calculation on the traffic signs according to the tracking prediction result of the same traffic sign to be vectorized in the plurality of images to be processed;

determining the similarity of the same traffic sign to be vectorized in the front and back images to be processed in the multiple images to be processed according to the intersection and the union;

or

and determining the similarity of the traffic sign to be vectorized in the front and back images to be processed in the plurality of images to be processed according to the characteristic similarity.

In a third aspect, an embodiment of the present application provides a traffic sign vectorization system, including a data acquisition device, a terminal, and a server;

the data acquisition equipment acquires a plurality of images to be processed and sends the images to be processed to the terminal, wherein each image to be processed comprises a plurality of traffic signs to be vectorized and image internal and external parameters;

the terminal receives the multiple images to be processed, divides the same traffic sign to be vectorized in the multiple images to be processed into a group according to the multiple traffic signs to be vectorized in each image to be processed, obtains multiple groups of traffic signs to be vectorized, determines sign scenes to which the multiple traffic signs to be vectorized in each image to be processed belong according to the image internal and external parameters in each image to be processed, and performs vectorization processing on the multiple groups of traffic signs to be vectorized according to the sign scenes to which the multiple traffic signs to be vectorized in each image to be processed belong;

and the server makes a high-precision map based on the traffic sign subjected to vectorization processing.

In a fourth aspect, an embodiment of the present application provides a terminal, including:

a processor;

a memory; and

a computer program;

wherein the computer program is stored in the memory and configured to be executed by the processor, the computer program comprising instructions for performing the method of the first aspect.

In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored, and the computer program causes a server to execute the method according to the first aspect.

In a sixth aspect, the present application provides a computer program product, which includes computer instructions for executing the method of the first aspect by a processor.

The traffic sign vectorization method, system, terminal and storage medium provided in the embodiments of the present application, in which a plurality of images to be processed sent by a data acquisition device are received, and then, according to a plurality of traffic signs to be vectorized in each image to be processed, the same traffic sign to be vectorized in the plurality of images to be processed is divided into one group, a plurality of groups of traffic signs to be vectorized are obtained, and according to internal and external parameters of the image in each image to be processed, a sign scene to which a plurality of traffic signs to be vectorized in each image to be processed belong is determined, so that, according to the sign scene to which a plurality of traffic signs to be vectorized in each image to be processed belong, vectorization processing is performed on the plurality of groups of traffic signs to be vectorized, that is, traffic signs are vectorized based on a visual mapping technology in the embodiments of the present application, the images sent by the data acquisition equipment are used as data sources, data acquisition is convenient and rapid, acquisition channels are more, traffic sign vectorization cost is correspondingly reduced, traffic sign vectorization efficiency is improved, and the requirement of real-time updating is met. Moreover, the traffic signs are subjected to matching grouping, and the sign scenes to which the traffic signs belong are determined, so that the traffic signs subjected to matching grouping are subjected to vectorization according to different sign scenes, the vectorized sign accuracy is improved, the robustness is enhanced, and the method is suitable for application.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.

Fig. 1 is a schematic diagram of an architecture of a traffic sign vectoring system according to an embodiment of the present disclosure;

fig. 2 is a schematic flowchart of a traffic sign vectoring method according to an embodiment of the present disclosure;

fig. 3 is a schematic flow chart of another traffic sign vectoring method according to an embodiment of the present disclosure;

fig. 4 is a schematic flowchart of another traffic sign vectoring method according to an embodiment of the present disclosure;

fig. 5 is a schematic structural diagram of a traffic sign vectoring device according to an embodiment of the present disclosure;

fig. 6 is a schematic diagram of a basic hardware architecture of a terminal provided in the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The terms "first," "second," "third," and "fourth," etc., in the description and claims of this application and in the foregoing drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be implemented in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

In the related technology, the traffic sign vectorization mainly scans along the driving direction to obtain high-density point cloud through high-precision laser radar equipment, GNSS equipment and IMU equipment carried by collection equipment, and then obtains a vector graph of the traffic sign based on the high-density point cloud. For example, after obtaining a high-density point cloud by scanning, a series of operations such as absolute position correction, point cloud denoising, registration, classification, vectorization, and the like at a later stage are required to obtain a vector graph of each traffic sign.

However, the vectorization technology of the traffic sign needs to acquire high-density point cloud, which is high in cost and low in efficiency.

Therefore, the embodiment of the application provides a traffic sign vectorization method, images sent by data acquisition equipment are used as data sources, data acquisition is convenient and fast, acquisition channels are more, traffic sign vectorization cost is correspondingly reduced, traffic sign vectorization efficiency is improved, and the requirement of real-time updating is met. And matching and grouping the traffic signs in the images, and determining sign scenes to which the traffic signs in the images belong, so that the traffic signs subjected to matching and grouping are vectorized according to different sign scenes, the vectorized sign accuracy is improved, the robustness is enhanced, and the method is suitable for application.

Optionally, the traffic sign vectorization method provided in the embodiment of the present invention is applied to a traffic sign vectorization processing in a high-precision map mapping process, and specifically, may be applied to a system that generates a high-precision map by performing information interaction between a data acquisition device and a terminal and a server, where fig. 1 is an architecture diagram of the traffic sign vectorization system provided in the embodiment of the present invention, and as shown in fig. 1, the system includes a data acquisition device 11, a terminal 12, and a server 13. Here, the data collection device 11 may be a collection vehicle on which collection devices, such as a camera, and the like, are mounted, and the collection vehicle belongs to a special vehicle for collecting geographic information in the mapping industry, and is generally different from the vehicle on which the terminal 12 is located.

It is to be understood that the structure illustrated in the embodiment of the present application does not form a specific limitation to the architecture of the traffic sign vectoring system. In other possible embodiments of the present application, the foregoing architecture may include more or less components than those shown in the drawings, or combine some components, or split some components, or arrange different components, which may be determined according to practical application scenarios, and is not limited herein. The components shown in fig. 1 may be implemented in hardware, software, or a combination of software and hardware.

In the specific implementation process, the collection vehicle collects a plurality of images to be processed along the driving direction through a collection device carried on the vehicle. Illustratively, the collection vehicle can receive a collection instruction, and then, based on the collection instruction, collect a plurality of images to be processed through a collection device carried on the vehicle. The collection instruction may be sent to the collection vehicle by the terminal 12, for example, the terminal 12 acquires a position of the collection vehicle, and if it is determined that the position needs to be subjected to traffic sign vectorization according to the position, the collection instruction is sent to the collection vehicle. Here, the collection vehicle is also mounted with a positioning device, and the terminal 12 acquires the position of the collection vehicle by the positioning device.

After the collection vehicle collects the multiple images to be processed, the collection vehicle sends the multiple images to be processed to the terminal 12. The terminal 12 uses the image sent by the collection vehicle as a data source, so that data can be conveniently and quickly obtained, more channels are obtained, the vectorization cost of the traffic sign is correspondingly reduced, the vectorization efficiency of the traffic sign is improved, and the requirement of real-time updating is met. Moreover, the terminal 12 performs matching grouping on the traffic signs in the images, and determines sign scenes to which the traffic signs in the images belong, so that vectorization is performed on the traffic signs after matching grouping according to different sign scenes, so that the vectorized sign accuracy is improved, the robustness is enhanced, and the method is suitable for application.

The terminal 12 transmits the traffic sign after the vectorization processing to the server 13, and the server 13 creates a high-precision map based on the traffic sign after the vectorization processing, and can apply the created high-precision map to automatic driving, path planning, navigation positioning, and the like.

It should be noted that, in fig. 1, the server 13 may also send a location area where traffic sign vectorization needs to be performed to the terminal 12, so that the terminal 12 sends a collection instruction to the collection vehicle when determining that the collection vehicle is located in the location area. The acquisition vehicle acquires a plurality of images to be processed through an acquisition device carried on the vehicle based on the acquisition instruction.

In this embodiment of the application, each of the terminals may be a handheld device, a vehicle-mounted device, a wearable device, a computing device, and various forms of User Equipment (UE), and the like.

In addition, the system architecture and the service scenario described in the embodiment of the present application are for more clearly illustrating the technical solution of the embodiment of the present application, and do not constitute a limitation to the technical solution provided in the embodiment of the present application, and it can be known by a person skilled in the art that along with the evolution of the system architecture and the appearance of a new service scenario, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

The technical solutions of the present application are described below with several embodiments as examples, and the same or similar concepts or processes may not be described in detail in some embodiments.

Fig. 2 is a schematic flow chart of a traffic sign vectorization method provided in an embodiment of the present application, where an execution subject of the embodiment may be a terminal in the embodiment shown in fig. 1, as shown in fig. 2, the method may include:

s201: and receiving a plurality of images to be processed sent by the data acquisition equipment, wherein each image to be processed comprises a plurality of traffic signs to be vectorized and the internal and external parameters of the image.

Here, each of the images to be processed may also include a traffic sign to be vectorized. The image internal and external parameters may include parameters of an acquisition device acquiring the plurality of images to be processed, such as camera parameters and image position and posture parameters.

For example, after receiving a plurality of images to be processed sent by the data acquisition device, the terminal may further perform internal reference calibration on the plurality of images to be processed to perform image correction, so that traffic sign vectorization is performed subsequently based on the corrected images, and accuracy of a subsequent processing result is improved.

The terminal may perform internal reference calibration on the multiple images to be processed by using a formula given by a merchant of the data acquisition device or an algorithm of an open source library, for example, an internal reference calibration algorithm of the data acquisition device based on the bimirror, which may be determined specifically according to an actual situation, and this is not particularly limited in this embodiment of the present application.

S202: and according to the plurality of traffic signs to be vectorized in each image to be processed, dividing the same traffic sign to be vectorized in the plurality of images to be processed into a group, and obtaining a plurality of groups of traffic signs to be vectorized.

Here, the terminal divides the same traffic sign into the plurality of received images to be processed into the same group based on the plurality of traffic signs to be vectorized in each of the images to be processed. Further, the terminal may divide the same traffic sign into the same group of the received multiple images to be processed based on the multiple traffic signs to be vectorized in each image to be processed after the image correction, so as to improve the accuracy of the subsequent processing result.

For example, the terminal may perform tracking prediction on the same traffic sign to be vectorized in the multiple images to be processed according to the multiple traffic signs to be vectorized in each image to be processed, further perform similarity calculation on the traffic sign according to a tracking prediction result, and divide the same traffic sign to be vectorized in the multiple images to be processed into one group according to a similarity calculation result, so as to obtain multiple groups of traffic signs to be vectorized.

The terminal may calculate, according to the tracking prediction result, an intersection and a union of circumscribed rectangles of the same traffic sign to be vectorized in the two preceding and following images to be processed in the plurality of images to be processed, and thereby determine, according to the intersection and the union, similarity of the same traffic sign to be vectorized in the two preceding and following images to be processed in the plurality of images to be processed, that is, similarity calculation of the traffic sign is performed.

In addition, the terminal may further calculate a feature similarity of the same traffic sign to be vectorized in two preceding and following images of the plurality of images to be processed according to the tracking prediction result, so that the similarity of the same traffic sign to be vectorized in two preceding and following images of the plurality of images to be processed is determined according to the feature similarity, and the similarity calculation of the traffic sign is also performed.

In this embodiment of the present application, the terminal may perform tracking prediction on the same traffic sign to be vectorized in the multiple images to be processed, then perform similarity calculation on the traffic sign, and finally divide the same traffic sign into the same group in the multiple received images to be processed according to a result of the similarity calculation. For example, the terminal may first fix an image, determine a sign on the image, then perform image similarity calculation according to a continuous frame motion rule of the image to perform matching, give a code to the same sign, and add code storage if a new sign appears, thereby dividing the same traffic sign to be vectorized in the multiple images to be processed into one group, and obtaining multiple groups of traffic signs to be vectorized.

S203: and determining the sign scene to which the plurality of traffic signs to be vectorized belong in each image to be processed according to the image internal and external parameters in each image to be processed.

The sign scene to which the plurality of traffic signs to be vectorized belong in each image to be processed comprises a side-by-side sign, wherein the side-by-side sign is a plurality of traffic signs to be vectorized, and internal and external parameters of the image in one image to be processed are distributed according to preset requirements. For example, taking a portal frame as an example, a plurality of traffic signs are arranged on the portal frame. If the internal and external parameters of the images of the plurality of traffic signs are distributed according to the horizontal direction of the portal frame. That is, if the signs are sequentially distributed at intervals in the x-axis direction and the distribution is not changed much or is substantially the same in the y-axis direction, taking the direction horizontal to the portal frame as the x-axis and the direction vertical to the portal frame as the y-axis, the sign scenes to which the plurality of traffic signs belong are the side-by-side signs.

Besides the side-by-side signs, the sign scene to which the plurality of traffic signs to be vectorized belong in each image to be processed further includes independent signs, wherein the independent signs are the remaining traffic signs to be vectorized except for the plurality of traffic signs to be vectorized, in which the internal and external parameters of the image in one image to be processed are distributed according to the preset requirement. For example, five traffic signs are arranged on the portal frame, wherein four traffic signs are horizontally distributed on the portal frame, the sign scenes to which the four traffic signs belong are side-by-side signs, and the sign scenes to which the remaining one traffic sign belongs are independent signs.

In this embodiment, the terminal may determine whether the internal and external parameters of the image in each to-be-processed image are distributed according to a preset requirement, for example, on a traffic sign of the portal frame, and the terminal may determine whether the internal and external parameters of the image of the traffic sign are distributed horizontally on the portal frame, that is, whether the internal and external parameters of the image are distributed at intervals in the x-axis direction, and the distribution is not changed greatly or is substantially the same in the y-axis direction.

If the image internal and external parameters in each image to be processed are distributed according to the preset requirement, for example, are distributed horizontally on the portal frame, the terminal determines that the signage scene to which the plurality of traffic signs to be vectorized belong in each image to be processed is a side-by-side signage, otherwise, the signage scene is an independent signage.

S204: and performing vectorization processing on the plurality of groups of traffic signs to be vectorized according to the sign scenes to which the plurality of traffic signs to be vectorized belong in each image to be processed.

Here, the terminal may perform vectorization processing on the multiple groups of traffic signs to be vectorized according to different sign scenes, so that the accuracy of the vectorized signs is improved, the robustness is enhanced, and the method is suitable for application.

In the embodiment of the application, a plurality of images to be processed sent by a data acquisition device are received through a terminal, and then, according to a plurality of traffic signs to be vectorized in each image to be processed, the same traffic signs to be vectorized in the plurality of images to be processed are divided into one group to obtain a plurality of groups of traffic signs to be vectorized, and according to internal and external parameters of the image in each image to be processed, signage scenes to which the plurality of traffic signs to be vectorized in each image to be processed belong are determined, so that according to the signage scenes to which the plurality of traffic signs to be vectorized in each image to be processed belong, vectorization processing is performed on the plurality of groups of traffic signs to be vectorized, that is, traffic signs are vectorized based on a visual mapping technology in the embodiment of the application, wherein the image sent by the data acquisition device is used as a data source, the data acquisition is convenient and fast, the acquisition channels are more, the traffic sign vectorization cost is correspondingly reduced, the traffic sign vectorization efficiency is improved, and the requirement of real-time update is met. Moreover, the traffic signs are subjected to matching grouping, and the sign scenes to which the traffic signs belong are determined, so that the traffic signs subjected to matching grouping are subjected to vectorization according to different sign scenes, the vectorized sign accuracy is improved, the robustness is enhanced, and the method is suitable for application.

In addition, for the side-by-side signs, when the vectorization processing is performed on the plurality of groups of traffic signs to be vectorized, the vectorization calculation is performed by using a pair of images, that is, two images, so as to ensure the consistency of the position relationship between the signs. Fig. 3 is a schematic flow chart of another traffic sign vectorization method according to an embodiment of the present application. As shown in fig. 3, the method includes:

s301: and receiving a plurality of images to be processed sent by the data acquisition equipment, wherein each image to be processed comprises a plurality of traffic signs to be vectorized and the internal and external parameters of the image.

S302: and according to the plurality of traffic signs to be vectorized in each image to be processed, dividing the same traffic sign to be vectorized in the plurality of images to be processed into a group, and obtaining a plurality of groups of traffic signs to be vectorized.

S303: and determining the sign scene to which the plurality of traffic signs to be vectorized belong in each image to be processed according to the image internal and external parameters in each image to be processed.

The steps S301 to S303 are the same as the steps S201 to S203, and are not described herein again.

S304: and determining the traffic signs to be vectorized of the first target group in the plurality of groups of traffic signs to be vectorized according to the side-by-side signs.

As can be seen from the above, the terminal divides the same traffic sign into the multiple received images to be processed, and divides the multiple received images to be processed into the same group, so as to obtain multiple groups of traffic signs to be vectorized. Here, further, the terminal finds the traffic sign to be vectored of the first target group corresponding to the side-by-side sign in the plurality of groups of traffic signs to be vectored. For example, five traffic signs are arranged on the portal frame, wherein four traffic signs are horizontally distributed on the portal frame, and the sign scenes to which the four traffic signs belong are side-by-side signs. And the terminal finds the traffic signs to be vectorized of the first target group corresponding to the four traffic signs horizontally distributed on the portal frame, namely four groups of traffic signs to be vectorized corresponding to the four traffic signs horizontally distributed on the portal frame, in the multiple groups of traffic signs to be vectorized.

S305: and sequencing the images to be processed where each group of traffic signs to be vectorized is located according to the distance between each group of traffic signs to be vectorized and the data acquisition device in the traffic signs to be vectorized of the first target group.

For example, the terminal may sort, according to a distance between each group of traffic signs to be vectorized and the data acquisition device in the traffic signs to be vectorized of the first target group, from near to far, images to be processed where each group of traffic signs to be vectorized is located.

S306: and acquiring a plurality of first target images from the to-be-processed images of each ordered group of the to-be-vectorized traffic signs.

In this embodiment of the application, the terminal may obtain, as the first target image, an image in which the number of traffic signs in the image is greater than a preset number from the to-be-processed images in which each group of traffic signs to be vectorized is located after the ordering, so as to obtain a plurality of first target images.

The preset number may be determined according to actual conditions, and is, for example, equal to the number of all signs in the image.

S307: and acquiring two images meeting a first preset image interval from the plurality of first target images, and performing vectorization processing on each group of traffic signs to be vectorized by using the two images.

Here, the first preset image interval may be determined according to actual conditions, for example, two images with a baseline of about 3-5 meters.

The terminal may perform vectorization processing on each group of traffic signs to be vectorized by using the two images. That is, the terminal may construct the tag homologous points of the two images based on the multi-feature matching and quadtree technology, further perform triangulation on the tag homologous points of the two images to obtain a spatial point cloud, perform spatial plane fitting on the spatial point cloud to obtain a spatial plane, and obtain a vector graph of each group of traffic tags to be vectorized according to the corner points of each group of traffic tags to be vectorized and the spatial plane, for example, project the corner points of each group of traffic tags to be vectorized to the corresponding spatial plane to obtain the vector graph of each group of traffic tags to be vectorized.

The terminal can also remove outliers in the space point cloud before performing space plane fitting on the space point cloud to obtain a space plane, so that the accuracy of a subsequent processing result is improved.

Illustratively, the terminal may perform depth weighted average on the spatial point cloud, perform depth correction, and remove outliers in the spatial point cloud, so as to perform spatial plane fitting on the spatial point cloud from which the discrete points are removed, and obtain the spatial plane.

In addition, the embodiment of the application considers that the obtained vector graphics of each group of traffic signs to be vectorized are relative to the coordinate system of the data acquisition device, and are not universal in practical application. Therefore, after obtaining the vector graphics of each group of traffic signs to be vectorized, the terminal performs coordinate system conversion on the vector graphics of each group of traffic signs to be vectorized, that is, converts the vector graphics of each group of traffic signs to be vectorized into a coordinate system in practical application.

For example, the data sent by the data acquisition device to the terminal further includes GNSS positioning data and IMU measurement data, that is, each of the to-be-processed images further includes GNSS positioning data and IMU measurement data. And after the terminal obtains the vector graphics of each group of traffic signs to be vectorized, converting the coordinate system of the vector graphics of each group of traffic signs to be vectorized according to the GNSS positioning data and the IMU measurement data, so as to meet the actual application requirements.

In the embodiment of the present application, for the side-by-side signs, when the embodiment of the present application performs vectorization processing on the multiple groups of traffic signs to be vectorized, it is considered that a pair of images, that is, two images are used for vectorization calculation, so as to ensure consistency of the position relationship between the signs. In addition, the traffic sign is vectorized based on the vision-based imaging technology, wherein the image sent by the data acquisition device is used as a data source, the data acquisition is convenient and fast, the acquisition channels are more, the vectorization cost of the traffic sign is correspondingly reduced, the vectorization efficiency of the traffic sign is improved, and the requirement of real-time updating is met. Moreover, the traffic signs are subjected to matching grouping, and the sign scenes to which the traffic signs belong are determined, so that the traffic signs subjected to matching grouping are subjected to vectorization according to different sign scenes, the vectorized sign accuracy is improved, the robustness is enhanced, and the method is suitable for application.

In addition, for the independent signs, when vectorization processing is performed on the multiple groups of traffic signs to be vectorized, vectorization calculation is performed by using multiple images, that is, an inter-frame fusion technology is adopted to overcome accidental errors caused by matching points and poses. Fig. 4 is a schematic flow chart of another traffic sign vectorization method according to an embodiment of the present application. As shown in fig. 4, the method includes:

s401: and receiving a plurality of images to be processed sent by the data acquisition equipment, wherein each image to be processed comprises a plurality of traffic signs to be vectorized and the internal and external parameters of the image.

S402: and according to the plurality of traffic signs to be vectorized in each image to be processed, dividing the same traffic sign to be vectorized in the plurality of images to be processed into a group, and obtaining a plurality of groups of traffic signs to be vectorized.

S403: and determining the sign scene to which the plurality of traffic signs to be vectorized belong in each image to be processed according to the image internal and external parameters in each image to be processed.

The steps S401 to S403 are the same as the steps S201 to S203, and are not described herein again.

S404: and determining the traffic signs to be vectorized of the second target group in the plurality of groups of traffic signs to be vectorized according to the independent signs.

Similarly, as can be seen from the above, the terminal divides the same traffic sign into the plurality of received images to be processed, and divides the images into the same group, so as to obtain a plurality of groups of traffic signs to be vectorized. Here, further, the terminal finds the traffic sign to be vectorized of the second target group corresponding to the independent sign in the plurality of groups of traffic signs to be vectorized. For example, five traffic signs are arranged on the portal frame, wherein four traffic signs are horizontally distributed on the portal frame, the sign scenes to which the four traffic signs belong are side-by-side signs, and the sign scenes to which the remaining one traffic sign belongs are independent signs. And the terminal finds the traffic signs to be vectorized of the second target group corresponding to the independent signs in the plurality of groups of traffic signs to be vectorized.

S405: and sequencing the images to be processed where each group of traffic signs to be vectorized is located according to the distance between each group of traffic signs to be vectorized and the data acquisition device in the traffic signs to be vectorized of the second target group.

For example, the terminal may sort, according to a distance between each group of traffic signs to be vectorized and the data acquisition device in the traffic signs to be vectorized of the second target group, from near to far, images to be processed where each group of traffic signs to be vectorized is located.

S406: and acquiring a plurality of second target images from the to-be-processed images of each ordered group of the to-be-vectorized traffic signs.

In this embodiment of the application, the terminal may obtain, from the to-be-processed images in which each group of traffic signs to be vectorized is located after the ordering, an image in which the traffic signs in the images are complete and which is close to the data acquisition device as a second target image, so as to obtain a plurality of second target images.

S407: and acquiring a plurality of images meeting a second preset image interval from the plurality of second target images, and performing vectorization processing on each group of traffic signs to be vectorized by using the plurality of images.

Here, the second preset image interval may be determined according to actual conditions, for example, a plurality of images with a baseline of about 3-5 m.

The terminal may perform vectorization processing on each group of traffic signs to be vectorized by using the plurality of images. That is, the terminal may construct the signage homologous points of the multiple images based on the multiple feature matching and quadtree techniques, further perform triangulation on the signage homologous points of the multiple images to obtain a spatial point cloud, perform spatial plane fitting on the spatial point cloud to obtain a spatial plane, and obtain a vector graph of each group of traffic signs to be vectorized according to the corner points of each group of traffic signs to be vectorized and the spatial plane, for example, project the corner points of each group of traffic signs to be vectorized to the corresponding spatial plane to obtain the vector graph of each group of traffic signs to be vectorized.

Illustratively, the terminal may perform depth weighted average on the spatial point cloud, perform depth correction, and remove outliers in the spatial point cloud, thereby performing spatial plane fitting on the spatial point cloud from which the discrete points are removed to obtain the spatial plane.

In addition, the embodiment of the application considers that the obtained vector graphics of each group of traffic signs to be vectorized are relative to the coordinate system of the data acquisition equipment, and are not universal in practical application. Therefore, after obtaining the vector graphics of each group of traffic signs to be vectorized, the terminal performs coordinate system conversion on the vector graphics of each group of traffic signs to be vectorized, that is, converts the vector graphics of each group of traffic signs to be vectorized into a coordinate system in practical application.

In the embodiment of the application, for the independent sign, when the vectorization processing is performed on the plurality of groups of traffic signs to be vectorized, the vectorization calculation is performed by using a plurality of images, that is, an interframe fusion technology is used to overcome accidental errors caused by matching points and poses. In addition, the traffic sign is vectorized based on the vision-based imaging technology, wherein the image sent by the data acquisition device is used as a data source, the data acquisition is convenient and fast, the acquisition channels are more, the vectorization cost of the traffic sign is correspondingly reduced, the vectorization efficiency of the traffic sign is improved, and the requirement of real-time updating is met. Moreover, the traffic sign is subjected to matching grouping and the sign scene to which the traffic sign belongs is determined, so that the traffic sign subjected to matching grouping is subjected to vectorization according to different sign scenes, the vectorized sign accuracy is improved, the robustness is enhanced, and the method and the device are suitable for application.

Fig. 5 is a schematic structural diagram of a traffic sign vectoring device according to an embodiment of the present application, corresponding to the traffic sign vectoring method according to the foregoing embodiment. For convenience of explanation, only portions related to the embodiments of the present application are shown. Fig. 5 is a schematic structural diagram of a traffic sign vectoring device according to an embodiment of the present application, where the traffic sign vectoring device 50 includes: image receiving module 501, signage grouping module 502, type determination module 503, and signage vectorization module 504. The traffic sign vectoring device may be the terminal itself, or a chip or an integrated circuit that implements the functions of the terminal. It should be noted here that the division of the image receiving module, the sign grouping module, the type determining module, and the sign vectoring module is only a division of logical functions, and the two may be integrated or independent physically.

The image receiving module 501 is configured to receive multiple images to be processed sent by a data acquisition device, where each image to be processed includes multiple traffic signs to be vectorized and the internal and external parameters of the image.

The sign grouping module 502 is configured to divide the same traffic sign to be vectorized in the multiple images to be processed into one group according to the multiple traffic signs to be vectorized in each image to be processed, so as to obtain multiple groups of traffic signs to be vectorized.

A type determining module 503, configured to determine, according to the image internal and external parameters in each image to be processed, a signage scene to which the traffic signage to be vectorized belongs in each image to be processed.

And the sign vectorization module 504 is configured to perform vectorization processing on the plurality of groups of traffic signs to be vectorized according to the sign scenes to which the plurality of traffic signs to be vectorized belong in each image to be processed.

The sign vectorization module 504 is specifically configured to:

In a possible implementation manner, the sign vectorization module 504 is specifically configured to:

removing outliers in the spatial point cloud;

The sign vectorization module 504 is specifically configured to:

In a possible implementation manner, the independent signs are the remaining traffic signs to be vectorized in one image to be processed, except for a plurality of traffic signs to be vectorized, where the internal and external parameters of the image are distributed according to preset requirements.

constructing the label homonymy points of the multiple images based on multi-feature matching and a quadtree technology;

removing outliers in the spatial point cloud;

carrying out space plane fitting on the space point cloud to obtain a space plane, wherein the method comprises the following steps:

The sign vectorization module 504 is specifically configured to:

In a possible implementation manner, the tag grouping module 502 is specifically configured to:

In a possible implementation manner, the sign grouping module 502 is specifically configured to:

or

The apparatus provided in the embodiment of the present application may be configured to implement the technical solutions of the method embodiments, and the implementation principles and technical effects are similar, which are not described herein again.

Optionally, fig. 6 schematically provides a schematic diagram of a possible basic hardware architecture of the terminal described in the present application.

Referring to fig. 6, the terminal includes at least one processor 601 and a communication interface 603. Further optionally, a memory 602 and a bus 604 may also be included.

Wherein, in the terminal, the number of the processors 601 may be one or more, and fig. 6 only illustrates one of the processors 601. Alternatively, the processor 601 may be a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), or a Digital Signal Processor (DSP). If the terminal has multiple processors 601, the types of the multiple processors 601 may be different, or may be the same. Optionally, the plurality of processors 601 of the terminal may also be integrated as a multi-core processor.

Memory 602 stores computer instructions and data; the memory 602 may store computer instructions and data necessary to implement the above-described traffic sign vectoring method provided herein, e.g., the memory 602 stores instructions for implementing the steps of the above-described traffic sign vectoring method. The memory 602 may be any one or any combination of the following storage media: nonvolatile memory (e.g., Read Only Memory (ROM), Solid State Disk (SSD), hard disk (HDD), optical disk), volatile memory.

The communication interface 603 may provide information input/output for the plurality of processors. Any one or any combination of the following devices may also be included: a network interface (such as an ethernet interface), a wireless network card, and the like.

Optionally, the communication interface 603 may also be used for data communication between the terminal and other computing devices or terminals.

Further alternatively, fig. 6 shows the bus 604 as a thick line. The bus 604 may connect the processor 601 with the memory 602 and the communication interface 603. Thus, via bus 604, processor 601 may access memory 602 and may also interact with other computing devices or terminals using communication interface 603.

In the present application, the terminal executes computer instructions in the memory 602, so that the terminal implements the traffic sign vectoring method provided in the present application, or the terminal deploys the traffic sign vectoring device.

From a logical functional partitioning perspective, illustratively, as shown in fig. 6, an image receiving module 501, a signage grouping module 502, a type determining module 503, and a signage vectoring module 504 may be included in the memory 602. The inclusion herein merely refers to the functionality of the image receiving module, the signage grouping module, the type determination module, and the signage vectoring module, respectively, when executed, and is not limited to a physical structure.

In addition, the traffic sign vectoring device may be implemented by software as shown in fig. 6, or may be implemented by hardware as a hardware module or a circuit unit.

The present application provides a computer-readable storage medium storing a computer program that causes a server to execute the above-described traffic sign vectoring method provided by the present application.

A computer program product comprising computer instructions for executing the above traffic sign vectoring method provided herein by a processor is provided.

The present application provides a chip comprising at least one processor and a communication interface providing information input and/or output for the at least one processor. Further, the chip may also include at least one memory for storing computer instructions. The at least one processor is configured to invoke and execute the computer instructions to perform the traffic sign vectoring method provided herein.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or in the form of hardware plus a software functional unit.

Claims

1. A traffic sign vectorization method, comprising:

receiving a plurality of images to be processed sent by data acquisition equipment, wherein each image to be processed comprises a plurality of traffic signs to be vectorized and image internal and external parameters;

2. The method according to claim 1, wherein the signage scene to which the plurality of traffic signs to be vectorized belong in each image to be processed comprises side-by-side signs;

sequencing the images to be processed where each group of traffic signs to be vectorized are located according to the distance between each group of traffic signs to be vectorized and the data acquisition equipment in the traffic signs to be vectorized of the first target group;

3. The method according to claim 2, wherein the vectorizing, using the two images, of each group of traffic signs to be vectorized comprises:

4. The method according to claim 3, wherein each image to be processed further comprises Global Navigation Satellite System (GNSS) positioning data and Inertial Measurement Unit (IMU) measurement data;

5. The method according to any one of claims 1 to 4, wherein the signage scene to which the plurality of traffic signs to be vectorized belong in each image to be processed comprises a separate sign;

6. The method according to any one of claims 1 to 4, wherein the obtaining a plurality of groups of traffic signs to be vectorized by dividing the same traffic sign to be vectorized in the plurality of images to be processed into one group according to the plurality of traffic signs to be vectorized in each image to be processed comprises:

7. The method according to claim 6, wherein the performing the similarity calculation of the traffic sign according to the tracking prediction result of the same traffic sign to be vectorized in the plurality of images to be processed comprises:

or

8. A traffic sign vectorization system is characterized by comprising data acquisition equipment, a terminal and a server;

9. A terminal, comprising:

a processor;

a memory; and

a computer program;

wherein the computer program is stored in the memory and configured to be executed by the processor, the computer program comprising instructions for performing the method of any of claims 1-7.

10. A computer-readable storage medium, characterized in that it stores a computer program that causes a server to execute the method of any one of claims 1-7.