CN117351511A

CN117351511A - High-precision map detection method, device and equipment

Info

Publication number: CN117351511A
Application number: CN202311356891.XA
Authority: CN
Inventors: 田科
Original assignee: Hangzhou Langge Technology Co ltd; Zhejiang Geely Holding Group Co Ltd
Current assignee: Hangzhou Langge Technology Co ltd; Zhejiang Geely Holding Group Co Ltd
Priority date: 2023-10-19
Filing date: 2023-10-19
Publication date: 2024-01-05

Abstract

The application provides a method, a device and equipment for detecting a high-precision map. Comprising the following steps: acquiring a picture frame of a high-precision map to be detected in the driving process, and detecting the picture frame to obtain a picture set of the picture frame; the picture set comprises a plurality of pictures to be detected, wherein the pictures to be detected comprise map elements, and the pictures to be detected have element categories; performing problem detection processing on the picture to be detected to obtain a problem category of the picture to be detected; the problem category characterizes the problem of map elements in the picture to be detected; determining each picture to be detected corresponding to the picture to be detected under the corresponding position information on the picture frame, and determining the final element category of the picture to be detected based on each determined picture to be detected; according to the problem category of the picture to be detected and the final element category of the picture to be detected, determining problem information of the picture to be detected, wherein the problem information represents whether the map element in the picture to be detected is abnormal or not. The method and the device can improve the accuracy of abnormal problem detection.

Description

High-precision map detection method, device and equipment

Technical Field

The present disclosure relates to the field of high-precision maps, and in particular, to a method, an apparatus, and a device for detecting a high-precision map.

Background

With the continuous development of automobile intellectualization, the automatic driving technology and the high-precision map technology become research hotspots. The high-precision map technology is the basis of the automatic driving technology, and thus, the accuracy of data of the high-precision map is critical to the automatic driving technology.

At present, after a high-precision map is generated, map elements in the high-precision map need to be checked to determine whether the map elements of the generated high-precision map are abnormal; what is further needed is a method that can accurately and effectively determine whether the map elements of the generated high-precision map are abnormal.

Disclosure of Invention

The application provides a detection method, device and equipment for a high-precision map, which are used for improving the accuracy of high-precision map detection.

In a first aspect, the present application provides a method for detecting a high-precision map, including:

acquiring a picture frame of a high-precision map to be detected in a driving process, and detecting the picture frame to obtain a picture set of the picture frame; the picture set comprises a plurality of pictures to be detected, wherein the pictures to be detected comprise map elements, the pictures to be detected have element categories, and the element categories represent the categories of the map elements in the pictures to be detected;

Performing problem detection processing on the picture to be detected to obtain a problem category of the picture to be detected; the problem category characterizes the problem of map elements in the picture to be detected;

determining each picture to be detected corresponding to the picture to be detected under the corresponding position information on the picture frame, and determining the final element category of the picture to be detected based on each determined picture to be detected;

and determining problem information of the picture to be detected according to the problem category of the picture to be detected and the final element category of the picture to be detected, wherein the problem information represents whether the map element in the picture to be detected is abnormal or not.

Optionally, performing problem detection processing on the to-be-detected picture to obtain a problem category of the to-be-detected picture, including:

sequentially carrying out size normalization processing and image super-resolution reconstruction processing on the pictures to be detected which do not belong to the preset element category, and obtaining processed pictures to be detected;

and inputting the processed picture to be detected into a first detection model to obtain a problem category of the picture to be detected, which does not belong to the preset element category.

Optionally, for each picture to be detected, determining each picture to be detected corresponding to the picture to be detected under the corresponding position information on the picture frame, and determining a final element category of the picture to be detected based on each determined picture to be detected, including:

determining a picture to be detected which does not belong to a preset element category, and corresponding position information on a picture frame;

determining an information set corresponding to the picture to be detected which does not belong to the preset element category based on the position information; the information set comprises at least one picture to be detected, wherein the picture to be detected in the information set is the picture to be detected corresponding to the position information;

if the number of the pictures in the information set is determined to be multiple, determining the final element category of the pictures to be detected, which do not belong to the preset element category, according to the area of the pictures to be detected in the information set;

if the number of the pictures in the information set is determined to be one, determining the element category of the picture to be detected which does not belong to the preset element category as the final element category of the picture to be detected which does not belong to the preset element category.

Optionally, determining a final element category of the picture to be detected, which does not belong to the preset element category, according to the area of the picture to be detected in the information set includes:

Determining the area of a picture to be detected which does not belong to a preset element category and the intersection between the area of the ith picture to be detected in the information set as first area information corresponding to the ith picture to be detected in the information set aiming at the ith picture to be detected in the information set; determining the union set between the area of the picture to be detected which does not belong to the preset element category and the area of the ith picture to be detected in the information set, and taking the union set as second area information corresponding to the ith picture to be detected in the information set; wherein i is a positive integer greater than or equal to 1;

determining a probability value corresponding to an ith picture to be detected in the information set according to the confidence level of the ith picture to be detected in the information set, a preset weight value of the ith picture to be detected in the information set, first area information corresponding to the ith picture to be detected in the information set and second area information corresponding to the ith picture to be detected in the information set; the probability value characterizes the element category of the picture to be detected which does not belong to the preset element category, and is the probability of the element category of the picture to be detected in the information set;

And determining the element category of the picture to be detected corresponding to the maximum probability value as the final element category of the picture to be detected which does not belong to the preset element category.

Optionally, the probability value corresponding to the ith picture to be detected in the information set is p=c _i ×λ _i ×(I _i /U _i ) The method comprises the steps of carrying out a first treatment on the surface of the Wherein c _i For the confidence coefficient lambda of the ith picture to be detected in the information set _i A preset weight value of the ith picture to be detected in the information set is I _i U is first area information corresponding to the ith picture to be detected in the information set _i Is the second area information corresponding to the ith picture to be detected in the information set。

Optionally, determining the problem information of the to-be-detected picture according to the problem category of the to-be-detected picture and the final element category of the to-be-detected picture includes:

if the final element category of the picture to be detected is determined to be a first preset category, determining problem information corresponding to both the problem category of the picture to be detected and the final element category of the picture to be detected based on a first preset mapping relation, wherein the problem information is the problem information of the picture to be detected;

the first preset mapping relationship is a mapping relationship among a problem category of the picture to be detected, a final element category of the picture to be detected and problem information.

if the final element category of the picture to be detected is determined to be a second preset category, determining a first numerical value represented by a problem category of the picture to be detected;

if the final element category is the to-be-detected picture of the third preset category, determining a second numerical value represented by the problem category of the to-be-detected picture of the third preset category;

if the second value is smaller than or equal to the first value, determining that the problem information of the picture to be detected is that no abnormality exists in the map elements in the picture to be detected;

and if the second numerical value is larger than the first numerical value, determining that the problem information of the picture to be detected is that the map elements in the picture to be detected are abnormal.

Optionally, the method further comprises:

inputting a picture to be detected with abnormal map elements into a second detection model to obtain first description information corresponding to the picture to be detected with abnormal map elements; the first description information characterizes abnormal conditions of the picture to be detected;

Inputting a picture frame corresponding to a picture to be detected, of which the map element is abnormal, into the second detection model to obtain second description information; the second description information characterizes abnormal conditions of picture frames corresponding to the abnormal pictures to be detected in the map elements;

determining a text template corresponding to the problem category of the picture to be detected and the final element category of the picture to be detected according to a second preset mapping relation; the second preset mapping relation is a mapping relation among a problem category of the picture to be detected, a final element category of the picture to be detected and a text template;

filling the first description information and the second description information into the determined text template to obtain third description information; and generating and outputting report information according to the third description information, the picture to be detected with the abnormal map element and the picture frame corresponding to the picture to be detected with the abnormal map element.

Optionally, acquiring a picture frame of the high-precision map to be measured in the driving process includes:

acquiring a video recorded in a driving process, wherein the video comprises a map detected in the driving process; and performing segmentation processing on the video to obtain the picture frame.

Optionally, detecting the picture frame to obtain a picture set of the picture frame includes:

inputting the picture frame into a preset third detection model, and outputting map element information of the picture frame; wherein the map element information comprises a category of the map element, coordinates of the map element and a confidence level of the category of the map element;

cutting the picture frame according to the map element information to obtain a plurality of pictures to be detected, and forming a picture set of the picture frame according to the pictures to be detected; the category of the map element is an element category of the picture to be detected, and the confidence of the category of the map element is the confidence of the picture to be detected.

In a second aspect, the present application provides a detection apparatus for a high-precision map, including:

the acquisition unit is used for acquiring a picture frame of the high-precision map to be detected in the driving process, detecting the picture frame and obtaining a picture set of the picture frame; the picture set comprises a plurality of pictures to be detected, wherein the pictures to be detected comprise map elements, the pictures to be detected have element categories, and the element categories represent the categories of the map elements in the pictures to be detected;

The detection unit is used for carrying out problem detection processing on the picture to be detected to obtain the problem category of the picture to be detected; the problem category characterizes the problem of map elements in the picture to be detected;

the first determining unit is used for determining each picture to be detected corresponding to the picture to be detected under the corresponding position information on the picture frame, and determining the final element category of the picture to be detected based on each determined picture to be detected;

and the second determining unit is used for determining problem information of the picture to be detected according to the problem category of the picture to be detected and the final element category of the picture to be detected, wherein the problem information represents whether the map element in the picture to be detected is abnormal or not.

In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored in the memory to implement the method of detecting a high-precision map as set forth in any one of the first aspects.

In a fourth aspect, the present application provides a computer-readable storage medium having stored therein computer-executable instructions which, when executed by a processor, are adapted to carry out the method of detecting a high-precision map as set forth in any one of the first aspects.

In a fifth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the method of detecting a high-precision map according to any one of the first aspects.

In a sixth aspect, the present application provides a chip on which a computer program is stored, which when executed by the chip, implements the method for detecting a high-precision map according to any one of the first aspects.

According to the detection method, the device and the equipment for the high-precision map, the multi-frame to-be-detected picture is obtained by detecting the picture frame of the to-be-detected high-precision map, the to-be-detected picture is subjected to problem detection processing, the problem category of the to-be-detected picture is obtained, and whether the to-be-detected picture is abnormal or not is determined by combining the final element category of the to-be-detected picture. According to the method, element categories of a plurality of pictures to be detected, which are included in the picture frame, are taken as global information to be considered, and the global information of the picture frame is applied to the detection of the abnormal problem of the pictures to be detected by determining the final element category of the pictures to be detected, so that the accuracy of detecting the abnormal problem of the pictures to be detected, namely, the accuracy of detecting the abnormal problem of the map element is improved compared with the case that whether the map element is abnormal or not is directly identified by using a model.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.

FIG. 1 is a schematic illustration of a high-precision map;

fig. 2 is a flow chart of a method for detecting a high-precision map provided by the present application;

FIG. 3 is a schematic illustration of a high-precision map;

fig. 4 is a flow chart of another method for detecting a high-precision map provided in the present application;

fig. 5 is a schematic structural diagram of a detection device for a high-precision map provided by the application;

fig. 6 is a schematic structural diagram of a detection device for a high-precision map provided by the application;

fig. 7 is a schematic structural diagram of an electronic device provided in the present application.

Specific embodiments thereof have been shown by way of example in the drawings and will herein be described in more detail. These drawings and the written description are not intended to limit the scope of the inventive concepts in any way, but to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as detailed in the accompanying claims.

The terms referred to in this application are explained first:

high-precision map: is a high-precision map for automatic driving, comprising map elements such as road shapes, road marks, traffic signs and the like.

Currently, there are mainly two ways to detect data of a high-precision map:

mode one: before rendering map data, performing accuracy check on the map data in the GeoJSON format by using a preset rule, and after the accuracy check is passed, converting the map data in the GeoJSON format into the map data in the Opendrive format so as to facilitate rendering and obtain a high-precision map.

However, the accuracy of the map data in the GeoJSON format before rendering can only be guaranteed, and the accuracy of the map elements of the high-precision map after rendering cannot be guaranteed, because the conversion between the formats or errors can be introduced in the rendering process, and the map elements of the high-precision map after rendering are abnormal.

Mode two: and carrying out defect identification on the map elements in the rendered high-precision map by utilizing a pre-trained YOLOV5 model so as to determine whether the map elements of the high-precision map are abnormal.

In this embodiment, although the anomaly detection is performed on the map elements of the high-definition map after rendering, the defect recognition of the map elements is performed only by using the YOLOV5 model, and the result of whether or not there is an anomaly is inaccurate, for example, fig. 1 is a schematic diagram of a high-definition map, and as shown in fig. 1, the model recognizes that a lane line is inclined (a broken line frame portion in fig. 1), and considers that the map elements of the lane line are abnormal. But the lane line is a part of the diversion area, then the lane line is inclined without abnormal problems. That is, this method cannot determine problem abnormality based on global information, and thus, the problem abnormality determination is inaccurate.

In view of this, the present application provides a method for detecting a high-precision map, based on the abnormal problem of the map element of the obtained high-precision map, and confirms whether the map element has an abnormality in combination with global information, which improves the accuracy of detecting the abnormal problem of the map element compared with the above-mentioned abnormal problem of detecting the map element only by using a model.

The execution subject of the present application may be an electronic device with processing capability, such as a server, a computer, etc., which is not limited herein.

The following describes the technical solutions of the present application and how the technical solutions of the present application solve the above technical problems in detail with reference to specific embodiments. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.

Fig. 2 is a flow chart of a method for detecting a high-precision map provided by the application. As shown in fig. 2, the method includes:

s101, acquiring a picture frame of a high-precision map to be detected in a driving process, and detecting the picture frame to obtain a picture set of the picture frame; the picture set comprises a plurality of pictures to be detected, the pictures to be detected comprise map elements, the pictures to be detected have element categories, and the element categories represent the categories of the map elements in the pictures to be detected.

For example, the picture frame of the high-precision map to be measured refers to a frame of picture of the high-precision map to be measured in the automatic driving process based on the high-precision map to be measured, and it should be understood that the picture frame includes map elements. The element types may be, for example, map element types such as lane lines, guide areas, guard rails, arrows, and curbs.

In one example, the electronic device may communicate with other electronic devices to obtain a picture frame of the high-precision map to be measured in the driving process, or may communicate with other electronic devices to obtain a video of the high-precision map to be measured in the driving process, and perform segmentation processing on the video to obtain the picture frame of the high-precision map to be measured. It should be noted that, after the video of the high-precision map to be detected is cut, a plurality of frames of pictures of the high-precision map to be detected can be obtained, and the picture frame of the high-precision map to be detected is one frame of picture in the plurality of frames of pictures of the high-precision map to be detected.

In another example, the picture frame of the high-precision map to be measured may be imported by an external device, and the external device may be, for example, a usb disk.

After obtaining a picture frame of the high-precision map to be detected in the driving process, in one example, the picture frame may be input into a pre-trained third detection model to detect the picture frame, so as to obtain a picture set of the picture frame. The third detection model may be, for example, a convolutional neural network model, such as the YOLOV5 model, the YOLOV7 model, or the like. Specifically, the picture frame is input into the first detection model, information such as categories of a plurality of map elements included in the detected picture frame, coordinates of the map elements, confidence levels of the categories of the map elements and the like is output, the picture frame is cut according to the coordinates of the map elements, a plurality of pictures to be detected are obtained, and then a picture set of the picture frame is obtained.

S102, carrying out problem detection processing on a picture to be detected to obtain a problem category of the picture to be detected; the problem category characterizes the problem of map elements in the picture to be detected.

By way of example, the above-mentioned problem category may be, for example, lane line inclination, guard rail geometry, etc.

In one example, the picture to be detected may be input to a pre-trained first detection model, and the problem category of the picture to be detected may be output. In another example, a preset element category is used for representing a category of a map element without a problem, the preset element category is used for screening a plurality of pictures to be detected in the picture set to obtain pictures to be detected which do not belong to the preset element category, the pictures to be detected which do not belong to the preset element category are input into a pre-trained first detection model, and the problem category of the pictures to be detected is output.

It should be noted that if the picture to be detected has no abnormal problem, the output is normal, if the result output by the first detection model is normal, the picture does not belong to the problem category, the picture to be detected is ignored, and the problem detection processing is continuously performed on the next picture to be detected in the picture set.

S103, determining each picture to be detected corresponding to the picture to be detected under the corresponding position information on the picture frame, and determining the final element category of the picture to be detected based on each determined picture to be detected.

Illustratively, fig. 3 is a schematic diagram of a high-precision map. With fig. 3 as the picture frame, two pictures to be detected included in the picture set obtained through step S101 are a picture a (dashed box a) and a picture B (dashed box B). Referring to fig. 3, it can be seen that there are overlapping map elements in the picture a and the picture B, so when processing is performed on the picture a, each picture to be detected corresponding to the picture to be detected under the corresponding position information on the picture frame may be the picture a and the picture B.

For example, the element category of the picture a is a lane line, and the element category of the picture B is a diversion area. When the image a is processed, the map element at the position corresponding to the image a on the image frame may also correspond to the image B, that is, the image a and the image B include the same map element, so that the category of the map element of the same map element may be a lane line or a diversion area. The final element category of the picture to be detected refers to an element category with the highest probability value among a plurality of element categories to which the picture to be detected may belong.

In one example, the relationship between the coordinates of the map element and the coordinates of the map element of other to-be-detected pictures in the picture set of the picture frame may be determined according to the coordinates of the map element included in the to-be-detected picture, and if the coordinates of the map element of the other to-be-detected pictures have an intersection with the coordinates of the map element of the to-be-detected picture, the other to-be-detected picture is determined as the to-be-detected picture corresponding to the to-be-detected picture under the position information corresponding to the picture frame.

After determining each to-be-detected picture corresponding to the to-be-detected picture under the corresponding position information on the picture frame, in one example, a final element category of the to-be-detected picture may be determined according to a relationship between an area of coordinates of each map element included in each corresponding to-be-detected picture and an area of coordinates of map elements included in the to-be-detected picture.

The element category of the picture to be detected included in the picture frame is used as global information, the final element category of the picture to be detected is determined by combining the global information of the picture frame, the global information of the picture frame is taken into consideration, the accuracy of the final element category of the picture to be detected is improved, and the accuracy of detecting abnormal problems of the picture to be detected is further improved.

S104, determining problem information of the picture to be detected according to the problem category of the picture to be detected and the final element category of the picture to be detected, wherein the problem information represents whether the map element in the picture to be detected is abnormal or not.

In one example, the electronic device may determine the problem information of the picture to be detected according to the problem category of the picture to be detected, the final element category of the picture to be detected, and a plurality of preset detection rules, and traverse the plurality of detection rules. For example, the preset detection rule may be "if the problem class is C and the final element class is D, then the map element is considered to have a problem E. "problem E" is used as problem information of the picture to be detected.

In another example, a predictive model is pre-trained, a problem category of the picture to be detected and a final element category of the picture to be detected are input into the predictive model, and problem information of the picture to be detected is output.

The step is equivalent to combining the global information of the picture frame to confirm whether the picture to be detected is abnormal or not, and the accuracy of detecting the abnormal problem of the picture to be detected is improved.

In this embodiment, a multi-frame to-be-detected picture is obtained by detecting a picture frame of a to-be-detected precise map, and problem detection processing is performed on the to-be-detected picture, so that a problem category of the to-be-detected picture is obtained, and whether the to-be-detected picture is abnormal or not is determined by combining a final element category of the to-be-detected picture. According to the method, element categories of a plurality of pictures to be detected, which are included in the picture frame, are taken as global information to be considered, and the global information of the picture frame is applied to the detection of the abnormal problem of the pictures to be detected by determining the final element category of the pictures to be detected, so that the accuracy of detecting the abnormal problem of the pictures to be detected, namely, the accuracy of detecting the abnormal problem of the map element is improved compared with the case that whether the map element is abnormal or not is directly identified by using a model.

Fig. 4 is a flow chart of another method for detecting a high-precision map provided in the present application. As shown in fig. 4, the method includes:

s201, acquiring a video recorded in the driving process, wherein the video comprises a map detected in the driving process; and cutting the video to obtain the picture frame.

In one example, a high-precision map navigation application is installed in the electronic device, rendering data of a to-be-detected high-precision map is imported into the high-precision map navigation application to render the to-be-detected high-precision map, and the to-be-detected high-precision map and a preset to-be-detected route are utilized to simulate automatic driving and record videos to obtain videos recorded in a driving process.

After the video recorded in the driving process is obtained, in one example, the video is segmented according to a preset segmentation principle, so as to obtain a picture frame. The segmentation principle may be, for example, that the segmentation is performed according to the running speed of the vehicle, for example, that the running speed of the vehicle is 30 m/s, and that a picture is obtained every second. The segmentation principle is not limited, and can be set according to actual requirements.

It should be understood that the splitting process may be performed on the video to obtain a plurality of picture frames, and each picture frame may be processed according to the method of the embodiment of the present application, and only one picture frame is illustrated herein.

S202, inputting the picture frame into a preset third detection model, and outputting map element information of the picture frame; wherein the map element information includes a category of the map element, coordinates of the map element, and a confidence of the category of the map element.

The third detection model may be a YOLOV7 model.

S203, cutting the picture frame according to the map element information to obtain a plurality of pictures to be detected, and forming a picture set of the picture frame according to the plurality of pictures to be detected; the category of the map element is an element category of the picture to be detected, and the confidence coefficient of the category of the map element is the confidence coefficient of the picture to be detected.

The coordinates of the map element are, for example, coordinates of the picture to be detected.

In one example, the map elements may be cut according to the coordinates of each map element, so as to obtain a picture to be detected corresponding to the map element.

And S204, sequentially carrying out size normalization processing and image super-resolution reconstruction processing on the pictures to be detected which do not belong to the preset element categories, and obtaining the processed pictures to be detected.

In one example, the preset element category is used for representing the category of the map element without problems, and the preset element category is used for screening the plurality of pictures to be detected in the picture set to obtain pictures to be detected which do not belong to the preset element category.

After obtaining the to-be-detected picture not belonging to the preset element category, in one example, performing size normalization processing on the to-be-detected picture not belonging to the preset element category, and adjusting the size of the to-be-detected picture not belonging to the preset element category to the preset size to obtain the to-be-detected picture not belonging to the preset element category after size normalization. The size normalization process may take a bilinear difference processing manner, for example, and is not limited herein.

After obtaining the to-be-detected picture which does not belong to the preset element category after the size normalization, in one example, performing image super-resolution reconstruction processing on the to-be-detected picture which does not belong to the preset element category after the size normalization so as to improve the resolution of the to-be-detected picture which does not belong to the preset element category after the size normalization, thereby improving the accuracy of the subsequent problem detection processing. The image Super-resolution reconstruction processing may be, for example, inputting the image to be detected which does not belong to the preset element category after the size normalization to a pre-trained Super-resolution residual network (Super-Resolution Residual Network, srres net) model, and outputting the processed image to be detected which does not belong to the preset element category.

S205, inputting the processed picture to be detected into a first detection model to obtain a problem category of the picture to be detected, which does not belong to a preset element category.

The first detection model may be a YOLOV7 model, for example. It should be noted that the first detection model and the third detection model may be the same type of model, or may be different types of models, and may specifically be set according to actual requirements.

The step only carries out problem detection processing on the processed pictures to be detected, so that the accuracy of the problem detection processing is improved while the number of the processed pictures is reduced.

S206, determining the picture to be detected which does not belong to the preset element category, and corresponding position information on a picture frame.

For example, as described above, the picture to be detected that does not belong to the preset element category is obtained by clipping based on the map element information, and the coordinates of the picture to be detected correspond to the coordinates of the map element included in the picture to be detected, that is, the coordinates of the map element on the picture frame. Therefore, according to the coordinates of the to-be-detected picture not belonging to the preset element category, the position information corresponding to the to-be-detected picture not belonging to the preset element category on the picture frame can be determined. The position information may be, for example, coordinates of the picture to be detected that does not belong to a preset element category.

S207, determining an information set corresponding to a picture to be detected which does not belong to a preset element category based on the position information; the information set comprises at least one picture to be detected, and the picture to be detected in the information set is the picture to be detected corresponding to the position information.

Illustratively, referring to fig. 3, the above-described information sets include a picture a and a picture B.

S208, determining whether the number of pictures in the information set is one.

If the number of pictures in the information set is determined to be one, step S209 is executed; if it is determined that the number of pictures in the information set is plural, step S210 is performed.

S209, determining the element category of the picture to be detected which does not belong to the preset element category, and determining the element category of the picture to be detected which does not belong to the preset element category as the final element category of the picture to be detected.

And performs step S211.

S210, determining the final element category of the picture to be detected, which does not belong to the preset element category, according to the area of the picture to be detected in the information set.

In one example, this step may include the steps of:

s2101, aiming at an ith picture to be detected in an information set, determining an intersection between the area of the picture to be detected which does not belong to a preset element category and the area of the ith picture to be detected in the information set, wherein the intersection is first area information corresponding to the ith picture to be detected in the information set; determining the union set between the area of the picture to be detected which does not belong to the preset element category and the area of the ith picture to be detected in the information set, and taking the union set as second area information corresponding to the ith picture to be detected in the information set; wherein i is a positive integer greater than or equal to 1.

For example, the area of the to-be-detected picture not belonging to the preset element category may be obtained according to the coordinates of the to-be-detected picture not belonging to the preset element category. It should be understood that the coordinates of the picture to be detected, which do not belong to the preset element category, are obtained by the aforementioned third detection model, and the model may output the detected coordinates of four vertices of the rectangular frame including the map element. Therefore, the area of the picture to be detected, which does not belong to the preset element category, can be determined according to the coordinates of the picture to be detected, which does not belong to the preset element category. Similarly, the area of the ith picture to be detected in the information set can be determined, and will not be described herein.

S2102, aiming at an ith picture to be detected in the information set, determining a probability value corresponding to the ith picture to be detected in the information set according to the confidence coefficient of the ith picture to be detected in the information set, a preset weight value of the ith picture to be detected in the information set, first area information corresponding to the ith picture to be detected in the information set and second area information corresponding to the ith picture to be detected in the information set; the probability value characterizes the element category of the picture to be detected, which does not belong to the preset element category, and is the probability of the element category of the picture to be detected in the information set.

For example, the weight value of each element category is preset, and since each picture to be detected has an element category, the preset weight value of each picture to be detected can be determined.

In one example, the probability value corresponding to the ith picture to be detected in the information set may be determined according to a mapping relationship between a probability value corresponding to the ith picture to be detected in the information set, a confidence level of the ith picture to be detected in the information set, a preset weight value of the ith picture to be detected in the information set, first area information corresponding to the ith picture to be detected in the information set, and second area information corresponding to the ith picture to be detected in the information set. The form of the mapping relation is not limited in the present application.

Illustratively, the probability value corresponding to the ith picture to be detected in the information set is p=c _i ×λ _i ×(I _i /U _i ) The method comprises the steps of carrying out a first treatment on the surface of the Wherein c _i For the confidence coefficient of the ith picture to be detected in the information set, lambda _i The preset weight value of the ith picture to be detected in the information set is I _i U is first area information corresponding to the ith picture to be detected in the information set _i And the second area information corresponding to the ith picture to be detected in the information set.

S2103, determining the element category of the picture to be detected corresponding to the maximum probability value, wherein the element category is the final element category of the picture to be detected which does not belong to the preset element category.

For example, the probability values may be compared to obtain a maximum probability value, and the element category of the picture to be detected corresponding to the maximum probability value is used as the final element category of the picture to be detected which does not belong to the preset element category.

S211, determining problem information of the picture to be detected according to the problem category of the picture to be detected and the final element category of the picture to be detected, wherein the problem information represents whether the map element in the picture to be detected is abnormal or not.

Illustratively, this step can be divided into the following two cases according to element categories.

(1) First case:

if the final element category of the picture to be detected is determined to be a first preset category, determining problem information corresponding to both the problem category of the picture to be detected and the final element category of the picture to be detected based on a first preset mapping relation, wherein the problem information is the problem information of the picture to be detected; the first preset mapping relation is a mapping relation among a problem category of the picture to be detected, a final element category of the picture to be detected and problem information.

The first predetermined category may be, for example, a surface element of a road such as a lane line, a guard rail, a curb, or the like.

In an example one, the problem category of the picture to be detected is "lane line inclination", the final element category of the picture to be detected is "lane line", the first preset mapping relationship is "if the problem category is lane line inclination, and if the element category is lane line, the problem of lane line inclination is abnormal", and it can be determined that the problem information corresponding to the picture to be detected is "abnormal problem of lane line inclination exists" according to the first preset mapping relationship.

In the second example, the problem category of the picture to be detected is "lane line inclination", the final element category of the picture to be detected is "flow guiding area", the first preset mapping relationship is "if the problem category is lane line inclination and the element category is flow guiding area, no abnormal problem exists", and it can be determined that the problem information corresponding to the picture to be detected is "no abnormal problem" according to the first preset mapping relationship, that is, the picture to be detected does not have an abnormality.

(2) Second case:

s301, if the final element category of the picture to be detected is determined to be a second preset category, determining a first numerical value represented by the problem category of the picture to be detected.

The second preset category may be, for example, a speed limit card. Because the speed limit cards on the high-precision map are arranged at intervals, at least one picture frame comprises the speed limit cards in all picture frames obtained after the video in the running process of the high-precision map to be detected is segmented, and the element category of one picture to be detected in all pictures to be detected in the picture frames is the speed limit card. It should be noted that, if the element category of the picture to be detected is a speed limit board, that is, if the map element included in the picture to be detected is a speed limit board, when the picture to be detected is subjected to problem detection, the output problem category is a numerical value on the speed limit board, that is, the first numerical value.

In one example, if it is determined that the final element category of the picture to be detected is a speed limit, the value of the variable V1 is updated to a first value characterized by the problem category of the picture to be detected. It should be noted that, the variable V1 is a global variable for detecting an abnormal problem of the high-precision map to be detected in the driving process, that is, the variable may be applied to other picture frames in the driving process.

All the picture frames in the driving process are processed sequentially according to the sequence of the driving direction. If the final element category of the picture to be detected of the other picture frames in the driving process is the second preset category, which is not recognized before the picture frame, for the currently processed picture frame, the numerical value of the variable V1 is a preset data value.

S302, if the final element category is the to-be-detected picture of the third preset category, determining a second numerical value represented by the problem category of the to-be-detected picture of the third preset category.

The third preset category may be, for example, a vehicle running speed. It should be appreciated that when using a high-definition map for automatic driving, the running speed of the vehicle is typically displayed in real time on the left side of the display interface of the high-definition map. Therefore, the element category in which one picture to be detected exists in all the pictures to be detected included in each picture frame is the vehicle running speed. It should be noted that, if the element type of the to-be-detected picture is the vehicle running speed, that is, the map element included in the to-be-detected picture is the vehicle running speed, when the to-be-detected picture is subjected to the problem detection processing, the output problem type is the value of the vehicle running speed, that is, the second value.

In one example, if it is determined that the final element class of the picture to be detected is the vehicle driving speed, the value of the variable V2 is updated to a second value characterized by the problem class of the picture to be detected. Since the automatically driven vehicle is driven by selecting the driving speed of the vehicle according to the speed limit attribute of the road, the variable V2 can be used to characterize the speed limit attribute of the road on which the vehicle is driven.

S303, if the second value is smaller than or equal to the first value, determining that the problem information of the picture to be detected is that no abnormality exists in the map element in the picture to be detected; if the second value is larger than the first value, determining that the problem information of the picture to be detected is that the map elements in the picture to be detected are abnormal.

Illustratively, the speed limit attribute of the link is interpreted as abnormal when the second value is greater than the first value.

Optionally, after determining the problem information of the to-be-detected picture, if the to-be-detected picture is abnormal, the following steps may be performed:

s212, inputting a picture to be detected with abnormal map elements into a second detection model to obtain first description information corresponding to the picture to be detected with abnormal map elements; the first description information characterizes abnormal conditions of the picture to be detected.

The second detection model may be a deep learning model, for example, a contrast language-Image Pre-trained subtitle (Contrastive Language-Image Pre-Training Prefix for Image Captioning, clipCap) model, or may be another deep learning model, which is not limited herein, and may be set according to practical requirements. The second detection model is used for outputting explanatory words describing the picture based on the input picture.

Taking the second detection model as a ClipCap model for example, the ClipCap model may include an encoder and a decoder, wherein the encoder uses a contrast language-Image Pre-Training (CLIP model), and the decoder uses a Generative Pre-Training transform (GPT) model, for example, a GPT-2 model.

S213, inputting a picture frame corresponding to a picture to be detected with abnormal map elements into a second detection model to obtain second description information; the second description information characterizes abnormal conditions of the picture frames corresponding to the abnormal pictures to be detected in the map elements.

The explanation of this step may refer to the aforementioned step S212.

S214, determining a text template corresponding to the problem category of the picture to be detected and the final element category of the picture to be detected according to a second preset mapping relation; the second preset mapping relation is a mapping relation among a problem category of the picture to be detected, a final element category of the picture to be detected and a text template.

For example, the text template may be "if [ first description information ], and [ second description information ], an abnormal problem of the problem X exists. ". The problem X is used to characterize the abnormal problem that exists.

In one example, a plurality of text templates are preset in advance according to the problem category, the element category and the abnormal problem to obtain a second mapping relation. The second preset mapping relationship may be shown in table 1, for example. And then, according to the problem category of the detected picture and the final element category of the picture to be detected, determining a text template corresponding to the picture to be detected.

TABLE 1

S215, filling the first description information and the second description information into the determined text template to obtain third description information; and generating and outputting report information according to the third description information, the picture to be detected with the abnormal map element and the picture frame corresponding to the picture to be detected with the abnormal map element.

For example, the report information may be in PDF format and output to an operator to prompt the operator that the high-precision map to be tested has an abnormal problem.

The step of generating and outputting report information comprising the picture to be detected and the description information corresponding to the picture frame of the picture to be detected aims at generating the abnormal problem of the picture to be detected, which is convenient for operators to understand.

In this embodiment, a picture set included in the picture frame is determined based on a first detection model, and then the pictures to be detected in the picture set are subjected to screening, size normalization processing, image super-resolution reconstruction and other processing, and the processed pictures to be detected are subjected to problem detection processing, so that problem categories of the pictures to be detected, which do not belong to preset element categories, are obtained; and then, based on the position information of the pictures to be detected, which do not belong to the preset element category, on the picture frame, an information set corresponding to the pictures to be detected, which do not belong to the preset element category, is determined, based on the number of the pictures to be detected in the information set, the final element category of the pictures to be detected, which do not belong to the preset element category, is determined, and further, according to the problem category and the final element category of the pictures to be detected, which do not belong to the preset element category, the problem information of the pictures to be detected is determined. Before the problem detection processing is carried out on the picture to be detected, the picture to be detected in the picture frame is subjected to the processing such as screening, size normalization processing, image super-resolution reconstruction and the like, so that the number of pictures subjected to the problem detection processing is reduced, the resolution of the pictures subjected to the problem detection processing is improved, and the detection efficiency and the detection accuracy are further improved.

Fig. 5 is a schematic structural diagram of a detection device for a high-precision map provided by the application. As shown in fig. 5, the apparatus 40 includes:

an obtaining unit 41, configured to obtain a picture frame of a high-precision map to be tested in a driving process, and detect the picture frame to obtain a picture set of the picture frame; the picture set comprises a plurality of pictures to be detected, wherein the pictures to be detected comprise map elements, the pictures to be detected have element categories, and the element categories represent the categories of the map elements in the pictures to be detected;

the detecting unit 42 is configured to perform problem detection processing on the to-be-detected picture, so as to obtain a problem category of the to-be-detected picture; the problem category characterizes the problem of map elements in the picture to be detected;

a first determining unit 43, configured to determine, for each of the to-be-detected pictures, each to-be-detected picture corresponding to the to-be-detected picture under the corresponding position information on the picture frame, and determine a final element category of the to-be-detected picture based on each determined to-be-detected picture;

the second determining unit 44 is configured to determine problem information of the to-be-detected picture according to a problem category of the to-be-detected picture and a final element category of the to-be-detected picture, where the problem information characterizes whether an abnormality exists in a map element in the to-be-detected picture.

The detection device for the high-precision map can execute the detection method for the high-precision map in the method embodiment, and the implementation principle and the technical effect are similar and are not repeated here.

Fig. 6 is a schematic structural diagram of a detection device for a high-precision map provided by the application. As shown in fig. 6, the apparatus 50 includes:

an obtaining unit 51, configured to obtain a picture frame of a high-precision map to be tested in a driving process, and detect the picture frame to obtain a picture set of the picture frame; the picture set comprises a plurality of pictures to be detected, wherein the pictures to be detected comprise map elements, the pictures to be detected have element categories, and the element categories represent the categories of the map elements in the pictures to be detected;

the detecting unit 52 is configured to perform problem detection processing on the to-be-detected picture, so as to obtain a problem category of the to-be-detected picture; the problem category characterizes the problem of map elements in the picture to be detected;

a first determining unit 53, configured to determine, for each of the to-be-detected pictures, each to-be-detected picture corresponding to the to-be-detected picture under the corresponding position information on the picture frame, and determine a final element category of the to-be-detected picture based on each determined to-be-detected picture;

The second determining unit 54 is configured to determine problem information of the to-be-detected picture according to a problem category of the to-be-detected picture and a final element category of the to-be-detected picture, where the problem information characterizes whether an abnormality exists in a map element in the to-be-detected picture.

In one example, the detection unit 52 includes:

the processing module 521 is configured to sequentially perform size normalization processing and image super-resolution reconstruction processing on the to-be-detected pictures that do not belong to the preset element category, so as to obtain processed to-be-detected pictures;

the first obtaining module 522 is configured to input the processed picture to be detected into a first detection model, so as to obtain a problem category of the picture to be detected, which does not belong to a preset element category.

In one example, the first determining unit 53 includes:

the first determining module 531 is configured to determine a picture to be detected that does not belong to a preset element category, and corresponding position information on the picture frame;

a second determining module 532, configured to determine, based on the location information, an information set corresponding to a picture to be detected that does not belong to a preset element category; the information set comprises at least one picture to be detected, wherein the picture to be detected in the information set is the picture to be detected corresponding to the position information;

A third determining module 533, configured to determine, if it is determined that the number of pictures in the information set is multiple, a final element category of the to-be-detected picture that does not belong to the preset element category according to an area of the to-be-detected picture in the information set; if the number of the pictures in the information set is determined to be one, determining the element category of the picture to be detected which does not belong to the preset element category as the final element category of the picture to be detected which does not belong to the preset element category.

In one example, the third determining module 533 includes:

a first determining submodule 5331, configured to determine, for an i-th picture to be detected in the information set, an intersection between an area of the picture to be detected that does not belong to a preset element category and an area of the i-th picture to be detected in the information set, as first area information corresponding to the i-th picture to be detected in the information set; determining the union set between the area of the picture to be detected which does not belong to the preset element category and the area of the ith picture to be detected in the information set, and taking the union set as second area information corresponding to the ith picture to be detected in the information set; wherein i is a positive integer greater than or equal to 1;

A second determining submodule 5332, configured to determine, for an i-th picture to be detected in the information set, a probability value corresponding to the i-th picture to be detected in the information set according to a confidence level of the i-th picture to be detected in the information set, a preset weight value of the i-th picture to be detected in the information set, first area information corresponding to the i-th picture to be detected in the information set, and second area information corresponding to the i-th picture to be detected in the information set; the probability value characterizes the element category of the picture to be detected which does not belong to the preset element category, and is the probability of the element category of the picture to be detected in the information set;

the third determining submodule 5333 is configured to determine an element category of the to-be-detected picture corresponding to the maximum probability value, where the element category is a final element category of the to-be-detected picture that does not belong to the preset element category.

In one example, the probability value corresponding to the ith picture to be detected in the information set is p=c _i ×λ _i ×(I _i /U _i ) The method comprises the steps of carrying out a first treatment on the surface of the Wherein c _i For the confidence coefficient lambda of the ith picture to be detected in the information set _i A preset weight value of the ith picture to be detected in the information set is I _i U is first area information corresponding to the ith picture to be detected in the information set _i And the second area information is corresponding to the ith picture to be detected in the information set.

In one example, the second determining unit 54 includes:

a fourth determining module 541, configured to determine, based on a first preset mapping relationship, problem information corresponding to both a problem category of the to-be-detected picture and a final element category of the to-be-detected picture, as problem information of the to-be-detected picture, if it is determined that the final element category of the to-be-detected picture is the first preset category; the first preset mapping relationship is a mapping relationship among a problem category of the picture to be detected, a final element category of the picture to be detected and problem information.

In one example, the second determining unit 54 includes:

the fourth determining module 541 is further configured to determine, if it is determined that the final element class of the to-be-detected picture is the second preset class, a first value represented by a problem class of the to-be-detected picture; if the final element category is the to-be-detected picture of the third preset category, determining a second numerical value represented by the problem category of the to-be-detected picture of the third preset category; if the second value is smaller than or equal to the first value, determining that the problem information of the picture to be detected is that no abnormality exists in the map elements in the picture to be detected; and if the second numerical value is larger than the first numerical value, determining that the problem information of the picture to be detected is that the map elements in the picture to be detected are abnormal.

In one example, the apparatus may further include an output unit 55, configured to input a picture to be detected in which the map element is abnormal into the second detection model, to obtain first description information corresponding to the picture to be detected in which the map element is abnormal; the first description information characterizes abnormal conditions of the picture to be detected;

In one example, the acquiring unit 51 includes:

a second obtaining module 511, configured to obtain a video recorded during driving, where the video includes a map detected during driving; and performing segmentation processing on the video to obtain the picture frame.

In one example, the acquiring unit 51 may further include:

the output module 512 is configured to input the picture frame into a preset third detection model, and output map element information of the picture frame; wherein the map element information comprises a category of the map element, coordinates of the map element and a confidence level of the category of the map element;

the third obtaining module 513 is configured to perform a cropping process on the picture frame according to the map element information, obtain a plurality of to-be-detected pictures, and form a picture set of the picture frame according to the plurality of to-be-detected pictures; the category of the map element is an element category of the picture to be detected, and the confidence of the category of the map element is the confidence of the picture to be detected.

Fig. 7 is a schematic structural diagram of an electronic device provided in the present application. As shown in fig. 7, the electronic device 600 may include: at least one processor 601, a memory 602. The electronic device may be a server, a computer, or the like.

A memory 602 for storing a program. In particular, the program may include program code including computer-operating instructions.

The memory 602 may include high-speed RAM memory or may also include non-volatile memory (non-volatile memory).

The processor 601 is configured to execute computer-executable instructions stored in the memory 602 to implement the method for detecting a high-precision map described in the foregoing method embodiment. The processor 601 may be a central processing unit (Central Processing Unit, abbreviated as CPU), or an application specific integrated circuit (Application Specific Integrated Circuit, abbreviated as ASIC), or one or more integrated circuits configured to implement embodiments of the present application.

The electronic device 600 may also include a communication interface 603 such that communication interactions with external devices may be performed through the communication interface 603. The external device may be, for example, a computer, a tablet, or the like.

In a specific implementation, if the communication interface 603, the memory 602, and the processor 601 are implemented independently, the communication interface 603, the memory 602, and the processor 601 may be connected to each other through buses and perform communication with each other. The bus may be an industry standard architecture (Industry Standard Architecture, abbreviated ISA) bus, an external device interconnect (Peripheral Component, abbreviated PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated EISA) bus, among others. Buses may be divided into address buses, data buses, control buses, etc., but do not represent only one bus or one type of bus.

Alternatively, in a specific implementation, if the communication interface 603, the memory 602, and the processor 601 are integrated on a chip, the communication interface 603, the memory 602, and the processor 601 may complete communication through internal interfaces.

The present application also provides a computer-readable storage medium, which may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, specifically, a computer-readable storage medium storing therein computer-executable instructions for the detection method of a high-precision map in the above-described embodiment.

The present application also provides a computer program product comprising execution instructions stored in a readable storage medium. The at least one processor of the electronic device 600 may read the execution instructions from the readable storage medium, the execution instructions being executed by the at least one processor to cause the electronic device 600 to implement the methods provided by the various embodiments described above.

The present application also provides a chip having a computer program stored thereon, which when executed by the chip, implements the methods provided by the various embodiments.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It is to be understood that the present application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A method for detecting a high-precision map, the method comprising:

2. The method according to claim 1, wherein performing problem detection processing on the to-be-detected picture to obtain a problem category of the to-be-detected picture, includes:

3. The method according to claim 2, wherein for each of the pictures to be detected, determining each picture to be detected corresponding to the picture to be detected under the corresponding position information on the picture frame, and determining a final element category of the picture to be detected based on each determined picture to be detected, comprises:

4. A method according to claim 3, wherein determining a final element category of the picture to be detected that does not belong to a preset element category according to the area of the picture to be detected in the information set comprises:

5. The method of claim 4, wherein a probability value corresponding to an i-th picture to be detected in the information set is p = c _i ×λ _i ×(I _i /U _i ) The method comprises the steps of carrying out a first treatment on the surface of the Wherein c _i For the confidence coefficient lambda of the ith picture to be detected in the information set _i A preset weight value of the ith picture to be detected in the information set is I _i U is first area information corresponding to the ith picture to be detected in the information set _i And the second area information is corresponding to the ith picture to be detected in the information set.

6. The method of claim 1, wherein determining problem information for the picture to be detected based on the problem category of the picture to be detected and the final element category of the picture to be detected comprises:

7. The method of claim 1, wherein determining problem information for the picture to be detected based on the problem category of the picture to be detected and the final element category of the picture to be detected comprises:

8. The method according to any one of claims 1-7, further comprising:

9. The method according to any one of claims 1-7, wherein obtaining a picture frame of a high-precision map to be measured during driving comprises:

10. The method according to any one of claims 1-7, wherein detecting the picture frame to obtain a picture set of the picture frame comprises:

11. A high-precision map detection apparatus, characterized in that the apparatus comprises:

12. An electronic device, the electronic device comprising: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored in the memory to implement the method of detecting a high-precision map as claimed in any one of claims 1-10.

13. A computer-readable storage medium, wherein computer-executable instructions are stored in the computer-readable storage medium, which when executed by a processor, are configured to implement the method for detecting a high-precision map according to any one of claims 1 to 10.