CN115661796A

CN115661796A - Guideboard identification method and device and vehicle

Info

Publication number: CN115661796A
Application number: CN202211394445.3A
Authority: CN
Inventors: 孟鹏飞; 贾双成; 朱磊; 郭杏荣
Original assignee: Zhidao Network Technology Beijing Co Ltd
Current assignee: Zhidao Network Technology Beijing Co Ltd
Priority date: 2022-11-08
Filing date: 2022-11-08
Publication date: 2023-01-31

Abstract

The invention provides a method, a device and a vehicle for identifying a guideboard, wherein the method comprises the following steps: acquiring an image to be identified, wherein the image to be identified carries the shielded guideboard elements; and inputting the image to be recognized into a recognition model, and obtaining the guideboard corner point category which is output by the recognition model and corresponds to the image to be recognized. According to the method, the device and the vehicle for identifying the guideboard, provided by the invention, based on the fact that the image carrying the shielded guideboard element is used as the input of the identification model, the output result is that the angular points and the angular point categories thereof contained in the shielded part and the shielded part in the corresponding image are identified and calculated automatically through the multi-layer neural network, the logicality and the relevance in the picture containing the incomplete information are fully mined, the accurate identification of the shielded guideboard is realized, the fineness and the accuracy of the guideboard identification can be improved, and the generation efficiency of a high-precision map is further improved.

Description

Guideboard identification method and device and vehicle

Technical Field

The invention relates to the technical field of automatic driving, in particular to a method and a device for identifying a guideboard and a vehicle.

Background

With the development of technologies such as artificial intelligence and automatic driving, the construction of intelligent traffic becomes a research hotspot, and a high-precision map is an essential part in the construction of intelligent traffic data. The high-precision map can contain various traffic signs, for example, ground feature elements such as lane lines, driving stop lines, pedestrian crossing lines and the like in the real world and high-altitude feature elements such as guideboards, traffic lights and the like can be expressed through a detailed map so as to provide data support for navigation in an application scene such as automatic driving.

The guideboard in the traffic sign is used as an information bearing carrier of a city geographic entity, has information navigation functions such as place names, routes, distances and directions, is used as infrastructure distributed at urban road intersections, has specificity in space, and is a good carrier of a city basic internet of things. The generation of the traffic signpost is accurately and efficiently performed, and it is very important to draw a high-precision map.

However, when the guideboard is blocked, since sufficient usable information cannot be acquired from the image acquired by the camera of the vehicle, the guideboard cannot be recognized, thereby affecting the efficiency of high-precision mapping.

Disclosure of Invention

The invention provides a method and a device for identifying a guideboard and a vehicle, which are used for solving the defect that the shielded guideboard cannot be identified in the prior art.

The invention provides a method for identifying a guideboard, which comprises the following steps:

acquiring an image to be identified, wherein the image to be identified carries the shielded guideboard elements;

inputting the image to be recognized into a recognition model, and obtaining the guideboard corner point category which is output by the recognition model and corresponds to the image to be recognized;

the identification model is obtained based on a sample guideboard image and a category label labeled correspondingly to the sample guideboard image; the identification model comprises a feature extraction layer, a channel separation layer and a category identification layer;

the step of inputting the image to be recognized into a recognition model to obtain the guideboard corner point category output by the recognition model and corresponding to the image to be recognized specifically includes:

inputting the image to be identified into the feature extraction layer, and acquiring a fusion feature image output by the feature extraction layer;

inputting the fusion characteristic image into the channel separation layer, and acquiring a channel characteristic image output by the channel separation layer;

and inputting the channel characteristic image into the category identification layer, and acquiring the corner point category of the guideboard output by the category identification layer.

According to the method for identifying the guideboard, the step of inputting the image to be identified into the feature extraction layer to obtain the fused feature image output by the feature extraction layer comprises the following steps:

carrying out downsampling operation and convolution operation of different scales on the image to be identified to obtain characteristic images of different scales;

performing feature fusion based on the feature images of all scales to obtain a fusion feature image;

wherein, after each feature fusion, the next feature fusion is carried out after one multi-scale convolution.

According to the method for identifying the guideboard provided by the invention, the inputting the fused characteristic image into the channel separation layer and acquiring the channel characteristic image output by the channel separation layer comprises the following steps:

performing convolution calculation on the fusion characteristic image to obtain a channel characteristic image corresponding to each channel;

the number of the channels is determined according to the number of the corner points of the guideboard contained in the complete guideboard element.

According to the method for identifying the guideboard, the identification model is obtained based on the sample guideboard image, and the category label and the angular point coordinate information which are labeled correspondingly to the sample guideboard image;

the inputting the channel feature image into the category identification layer to obtain the guideboard corner point category output by the category identification layer specifically includes:

identifying and extracting categories of the channel characteristic images corresponding to the channels to obtain category probability sets corresponding to the channels;

and determining the guideboard corner point category and the guideboard corner point coordinate corresponding to the channel characteristic image by using the category probability set corresponding to the channel.

According to the method for identifying the guideboard, the step of acquiring the image to be identified comprises the following steps:

under the condition that the image to be recognized is determined not to carry the shielded guideboard element, intercepting a non-corner-point region image based on a target corner point in the image to be recognized;

covering the non-corner region image in a region corresponding to the target corner to generate a new image to be identified;

and inputting the new image to be recognized into the recognition model, and obtaining the guideboard corner point category which is output by the recognition model and corresponds to the new image to be recognized.

The present invention also provides a guideboard recognition apparatus, comprising:

the system comprises an image acquisition module, a road sign identification module and a road sign identification module, wherein the image acquisition module is used for acquiring an image to be identified, and the image to be identified carries shielded road sign elements;

the corner identification module is used for inputting the image to be identified to an identification model and obtaining the guideboard corner category which is output by the identification model and corresponds to the image to be identified;

the corner identification module specifically comprises a feature extraction unit, a channel separation unit and a category identification unit, wherein:

the feature extraction unit is used for inputting the image to be identified to the feature extraction layer and acquiring a fusion feature image output by the feature extraction layer;

the channel separation unit is used for inputting the fused characteristic image into the channel separation layer and acquiring a channel characteristic image output by the channel separation layer;

the category identification unit is used for inputting the channel characteristic image into the category identification layer and acquiring the category of the corner points of the guideboard output by the category identification layer.

The invention also provides a vehicle, which comprises a vehicle body and an identification device arranged on the vehicle body, wherein the identification device is used for executing the method for identifying the guideboard.

The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the method for identifying the guideboard.

The invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method of identifying a guideboard as described in any one of the above.

The invention also provides a computer program product comprising a computer program which, when executed by a processor, implements a method of identifying a guideboard as defined in any one of the above.

According to the method, the device and the vehicle for identifying the guideboard, provided by the invention, based on the fact that the image carrying the shielded guideboard elements is used as the input of the identification model, the output result is the angular points and the angular point categories of the shielded part and the shielded part in the corresponding image, the incomplete image information is automatically identified and calculated through a multi-layer neural network, the shielded guideboard is accurately identified, the fineness and the accuracy of guideboard identification can be improved, and the generation efficiency of a high-precision map is further improved.

Drawings

In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.

FIG. 1 is a schematic flow chart of a method for identifying a guideboard according to the present invention;

FIG. 2 is a second schematic flow chart of the method for identifying a guideboard according to the present invention;

FIG. 3 is a schematic structural diagram of a guideboard recognition device provided in the present invention;

FIG. 4 is a schematic structural view of a vehicle provided by the present invention;

fig. 5 is a schematic structural diagram of an electronic device provided in the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The terms "first," "second," and the like in the description of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that embodiments of the application may be practiced in sequences other than those illustrated or described herein, and that the terms "first," "second," and the like are generally used herein in a generic sense and do not limit the number of terms, e.g., the first term can be one or more than one.

It is to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

The terms "comprises" and "comprising" indicate the presence of the described features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Fig. 1 is a schematic flow chart of a method for identifying a guideboard according to the present invention. As shown in fig. 1, the method for identifying a guideboard according to an embodiment of the present invention includes: step 101, an image to be identified is obtained, and the image to be identified carries the blocked guideboard elements.

The execution subject of the method for identifying a guideboard according to the embodiment of the present invention is a guideboard identification device. The identification device of the guideboard may be a Central Processing Unit (CPU) built in the vehicle, or a development board integrated based on the CPU to perform information Processing and program operation.

The application scenario of the method for identifying the guideboard provided by the embodiment of the invention is that the image containing the blocked guideboard is identified, and all the angular points contained in the unblocked part and the blocked part in the guideboard and the categories of the angular points are obtained.

Specifically, in step 101, the identification device of the guideboard receives a picture taken by a camera mounted on a vehicle in real time as a picture to be identified, and the picture to be identified carries the blocked guideboard elements.

And the guideboard elements are graphic elements which are captured by a camera in the driving process of the vehicle and belong to traffic signs.

The image to be recognized is not limited to one type of guideboard element, and may be guideboards corresponding to various traffic signs. The guideboard corresponding to the traffic sign is usually a regular graph, which is not specifically limited in the embodiment of the present invention.

Exemplarily, the guideboard corresponding to the traffic sign may be square in shape, the square guideboard has four right-angle points, the right-angle point in each direction is the guideboard angle point of the corresponding category, and then in the identification process, the angle point category in the guideboard element may be obtained by identifying the shielded guideboard element.

And 102, inputting the image to be recognized into a recognition model, and obtaining the guideboard corner point category which is output by the recognition model and corresponds to the image to be recognized.

The identification model is obtained based on a sample guideboard image and a category label labeled correspondingly to the sample guideboard image. The recognition model comprises a feature extraction layer, a channel separation layer and a category recognition layer.

and inputting the image to be recognized into the feature extraction layer, and acquiring a fusion feature image output by the feature extraction layer.

And inputting the fusion characteristic image into the channel separation layer, and acquiring the channel characteristic image output by the channel separation layer.

It should be noted that the recognition model may be a neural network model, and the structure and parameters of the neural network include, but are not limited to, the number of input layers, hidden layers and output layers of the neural network, and the weight parameters of each layer. The type and structure of the neural network are not particularly limited in the embodiments of the present invention.

For example, the recognition model may be a feed-forward neural network, the model being composed of an input layer, a hidden layer, and an output layer, wherein:

the input layer receives image data carrying the occluded guideboard elements directly at the very front part of the overall network.

The hidden layer may have one or more layers, and the operation is performed by a weighted summation of its own neurons on the input vector, and the calculation formula may be expressed as:

z＝b+w1*x1+w2*x2+…+wm*xm

wherein z is a weight sum value output by the hidden layer, x1, x2 and x3 \8230, 8230, xm is m eigenvectors of each sample, b is offset, and w1, w2 \8230, 8230and wm are weights corresponding to each eigenvector.

The output layer is the last layer and is used to output the recognition result of the corner point category, and output the type of the recognition result according to different requirements, where the value may be a category vector value, or a continuous value generated like a linear regression, or another complex type value or vector, and the embodiment of the present invention is not particularly limited to this.

The excitation function is a function that runs on a neuron of the artificial neural network, and is responsible for mapping an input of the neuron to an output terminal, and performing logistic regression processing by using the activation function, that is, converting a weight sum value output by the hidden layer into a nonlinear recognition result.

Preferably, the output of a plurality of neurons is mapped into a (0, 1) interval by performing logistic regression processing by using a Softmax function, and the output can be understood as probability, so that multi-classification is performed.

The sample data includes a sample guideboard image corresponding to the sample data, and a category label for marking a corner in the sample guideboard image. Dividing the sample data into a training set and a test set according to a certain proportion.

Exemplary training sets and test sets include, but are not limited to, 9: 1. 8:2, etc., and the embodiment of the present invention is not particularly limited thereto.

Specifically, in step 102, the guideboard recognition device initializes the weight coefficients between the layers of the constructed recognition model, then inputs the labeled contents of a set of sample question data and sample answer data in the training set to the neural network under the current weight coefficient, and sequentially calculates the outputs of the nodes of the input layer, the hidden layer, and the output layer. And correcting the weight coefficient between each node of the input layer and the hidden layer according to a gradient descent method by the accumulated error between the final output result of the output layer and the actual connection position state type of the output layer. According to the process, the weight coefficients of the input layer and the hidden layer can be obtained until all samples in the training set are traversed.

The guideboard recognition device restores the recognition model in step 102 according to the weight coefficients of the neural network input layer and the hidden layer, and inputs each image to be recognized in the test set to the trained recognition model, so as to obtain a recognition result corresponding to the image.

The recognition result may be a probability value or a label result, and the form of the behavior recognition result is not particularly limited in the embodiment of the present invention.

If the recognition result can be a probability value, the probability that the corner points included in the image to be recognized respectively belong to the corner point categories corresponding to the orientations can be described through the probability value.

If the identification result can be a label result, an intermediate numerical value result can be obtained through the model, the numerical value result meets the preset target condition, and the corner point type label corresponding to the azimuth is distributed to the corner point corresponding to the numerical value result.

Preferably, the recognition model built in the recognition device of the guideboard is composed of an input layer, a hidden layer and an output layer. The hidden layer is used for extracting the characteristics of the input image with incomplete information through the neurons of the hidden layer, and the characteristic information which is favorable for identification is extracted.

The structure of the hidden layer is not particularly limited in the embodiments of the present invention.

Preferably, the hidden layer includes at least three layers, which are a feature extraction layer, a channel separation layer and a category identification layer, respectively, wherein:

the feature extraction layer can adopt a Convolutional Neural Network (CNN) to perform dimension reduction on an image to be recognized, and perform feature extraction and fusion while compressing a vector to obtain a fusion feature image.

In the convolution process, the cavity convolution or the expansion convolution can be adopted, so that the receptive field is enlarged under the condition of not performing posing loss information, and each convolution output contains information in a larger range.

The channel separation layer can set the channel number of the convolution kernel through the angular point number contained by the guideboard, and the convolution calculation is carried out on the fusion characteristic image again to obtain the two-dimensional channel characteristic image corresponding to each channel.

The category identification layer can map the two-dimensional channel characteristic images corresponding to the channels into two-dimensional vectors by adopting full connection processing and Softmax processing in sequence, and carry out classification processing according to the two-dimensional vectors to obtain the corner categories of the guideboard, so that all corners contained in the images to be identified and the corresponding corner categories can be obtained.

The embodiment of the invention is based on the fact that an image carrying blocked guideboard elements is used as input of a recognition model, the image to be recognized is subjected to feature extraction through a feature extraction layer, a channel separation layer is used for carrying out channel separation on a fusion feature image output by the feature extraction layer, then the channel feature images of all channels are subjected to full connection processing through a category recognition layer, angular points and angular point categories of the angular points contained in a blocked part and a blocked part in a corresponding image are output, incomplete image information is automatically recognized and calculated through a multi-layer neural network, the logicality and the relevance in the image containing incomplete information are fully mined, the accurate recognition of the blocked guideboard is realized, the fineness and the accuracy of guideboard recognition can be improved, and the generation efficiency with high accuracy is further improved.

On the basis of any of the above embodiments, inputting the image to be recognized to the feature extraction layer, and acquiring a fused feature image output by the feature extraction layer, includes: and performing downsampling operation and convolution operation of different scales on the image to be identified to obtain characteristic images of different scales.

And performing feature fusion based on the feature images of all scales to obtain the fusion feature image.

It should be noted that, the guideboard recognition device extracts a high-dimensional feature map from a matrix corresponding to a remote sensing image by using a large number of convolution kernels, so that a plurality of feature maps can be extracted, each feature map is a local perception extracted from a picture, and an interested part in the picture can be extracted by synthesizing the feature maps.

The original extracting effects of convolution kernels with different sizes on different types of guideboards are different, and the number and the size of the convolution kernels for carrying out multi-scale down sampling are not particularly limited in the embodiment of the invention.

And extracting relevant characteristic data for identifying the small guideboard by adopting the small convolution kernel 1x 1. And the related characteristic information of the identified guideboard can be extracted by adopting the middle convolution kernel 3 x 3. And the large convolution kernel 5 x 5 can be used for extracting relevant characteristic information for identifying the large guideboard. And then when feature data with different scales are fused, the related information with the guideboards with three different specifications can be reserved, so that the loss of the original effective data is prevented.

It should be noted that convolution calculation is also required after each fusion to prevent noise from being introduced.

Specifically, the guideboard recognition device inputs an image to be recognized into a cascaded convolutional layer, simultaneously enters convolutional kernels with different sizes for convolutional calculation to respectively obtain feature images with different dimensions, linearly adds the feature images with multiple dimensions obtained by the convolution at the time to obtain a fused feature image, then simultaneously enters a convolutional kernel with the next stage of size for repeating the above process, continuously performs feature extraction and feature fusion processes, and updates the fused feature image.

Illustratively, 5 convolution kernels with the size of 3 × 3 and the number of channels of 128, 64, 32, 16 and 8 may be sequentially set, and accordingly, 5 convolution kernels with the size of 5 × 5 and the number of channels of 128, 64, 32, 16 and 8 are correspondingly set to perform convolution processing, so that the height and the width of the feature map of the image after each convolution processing are both shortened by half, the number of channels is reduced by half, and after five times of convolution and linear addition are performed, a fused feature image with the number of channels of 8 can be obtained.

In order to linearly add the feature images of multiple scales, padding is required to ensure that the number of rows and columns of the feature images calculated by convolution is the same.

The embodiment of the invention is based on the convolution calculation of different scales of the image to be recognized, and the fusion characteristic image is obtained after the multi-scale characteristic image obtained by convolution is subjected to multiple times of fusion and convolution dimensionality reduction. The image can simultaneously contain low-level detail features and high-level semantic features, the fineness of image semantic segmentation can be improved, and the accuracy of image identification is further improved.

On the basis of any of the above embodiments, inputting the fused feature image to the channel separation layer, and acquiring a channel feature image output by the channel separation layer, includes: and performing convolution calculation on the fusion characteristic image to obtain a channel characteristic image corresponding to each channel.

It should be noted that, before step 102, the number of channels needs to be set according to the number of corner points included in the guideboard elements with different specifications, so that each channel corresponds to a feature corresponding to each corner point.

Specifically, the guideboard recognition device performs convolution calculation on the fusion feature image by successively adopting a convolution kernel with the channel number of 4 to obtain feature data of each channel, and performs full-connection dimensionality reduction by adopting a convolution kernel with the size of 1x1 to obtain channel feature images of a plurality of channels with the same width and height as the original image.

According to the embodiment of the invention, after the number of channels is determined based on the number of the corner points of the guideboard contained in the guideboard elements, the convolution calculation of the corresponding number of the channels is carried out on the fusion characteristic image so as to separate the channel characteristic image of each channel. The method can improve the fineness of image semantic segmentation and further improve the accuracy of image recognition.

On the basis of any one of the above embodiments, the identification model is obtained based on the sample guideboard image, and the category label and the corner point coordinate information labeled correspondingly to the sample guideboard image.

The inputting the channel feature image into the category identification layer to obtain the guideboard corner point category output by the category identification layer specifically includes: and identifying and extracting the category of the channel characteristic image corresponding to each channel to obtain a category probability set corresponding to each channel.

Specifically, in the process of training the recognition model, the recognition device of the guideboard trains the corner point categories corresponding to the channels and the corner point coordinate information corresponding to the corner points for identifying the categories as true values, recognizes the characteristic images of the channels, maps the pixel value of each pixel point in the characteristic images of the channels into the probability values of the corner point categories corresponding to different channels, and integrates the probability values into a category probability set.

For any channel feature image, the class probability corresponding to the channel should be the maximum value in the multiple class probability values corresponding to each pixel point.

Specifically, the guideboard identification device outputs a corner point category corresponding to a maximum value of the multiple probability values corresponding to each pixel point from the category probability set corresponding to each channel, and determines the guideboard corner point coordinates according to the corner point category corresponding to the corner point category.

For example, if the square guideboard of the image to be recognized is to be recognized, the three specifications of the square guideboards are 4 guideboard corner points, and the corner point categories corresponding to the upper left corner, the upper right corner, the lower right corner and the lower left corner of the guideboard element are set to be numbered 0,1, 2 and 3, regardless of whether the square guideboard is a small square guideboard, a medium square guideboard or a large square guideboard.

Therefore, the number of ordinary convolution channels for performing ordinary convolution calculation on the fusion feature image to be processed is equal to 4, and 4 channel feature images can be obtained. When the channel characteristic image corresponding to the channel with the class number of 0 is identified, the pixel point corresponding to the probability value which is greater than the preset threshold value in the class probability set is identified as the 0-number corner point class, and the corresponding pixel coordinate is output as the coordinate of the upper left corner point of the guideboard.

When the channel characteristic image corresponding to the channel with the class number of 1 is identified, the pixel points corresponding to the probability values which are greater than the preset threshold value in the class probability set are identified as the class of the corner point No. 1, and the corresponding pixel coordinates are output as the coordinates of the upper right corner point of the guideboard.

When the channel characteristic image corresponding to the channel with the class number of 2 is identified, the pixel points corresponding to the probability values which are greater than the preset threshold value in the class probability set are identified as the class of the No. 2 corner points, and the corresponding pixel coordinates are output as the coordinates of the lower right corner points of the guideboard.

When the channel characteristic image corresponding to the channel with the category number of 3 is identified, the pixel points corresponding to the probability values which are greater than the preset threshold value in the category probability set are identified as the category of the corner point 3, and the corresponding pixel coordinates are output as the coordinates of the lower left corner point of the guideboard.

The embodiment of the invention takes the image carrying the shielded guideboard elements as the input of the identification model, outputs the result of the angle points, the angle point categories and the angle point coordinates contained in the corresponding shielded part and the shielded part in the corresponding image, and automatically identifies and calculates incomplete image information through a multi-layer neural network, thereby realizing the accurate identification of the shielded guideboard, improving the fineness and the accuracy of guideboard identification and further improving the generation efficiency of high-precision maps.

On the basis of any one of the above embodiments, the acquiring an image to be recognized includes: and under the condition that the image to be recognized does not carry the shielded guideboard element, intercepting a non-corner-point area image based on the target corner point in the image to be recognized.

Specifically, in step 101, the identification device of the guideboard performs preliminary screening identification on an image to be identified, which is acquired by a camera, and if it is identified that the image to be identified does not contain a blocked guideboard element, one or more target angular points are extracted from the image, a pixel point which is a certain pixel distance away from a target angular point is used as a starting point of a non-angular point region, and any pixel point is selected in a direction away from the target angular point pixel as a terminal point of the non-angular point region, so as to capture a non-angular point region image in the defined non-angular point region.

And covering the non-corner region image in a region corresponding to the target corner to generate a new image to be identified.

Specifically, the identification device of the guideboard determines the size of the non-corner region image, and the non-corner region image with the corresponding size is overlapped with the starting point of the non-corner region by using the target corner point, so that the non-corner region image is covered in the region where the target corner point is located, and a new image to be identified is generated. For the recognition model to recognize the category of the target corner point from the covered region in the new image to be recognized.

The embodiment of the present invention does not specifically limit the specific implementation manner of the identification process.

Preferably, fig. 2 is a second schematic flow chart of the method for identifying the guideboard provided by the present invention. As shown in fig. 2, the entire recognition process includes a training process and a testing process.

And (I) in the training process, the sample guideboard image is sequentially input into the feature extraction layer, the channel separation layer and the class identification layer for processing, and the obtained sample guideboard angular point class and the class label labeled corresponding to the sample guideboard image are subjected to a gradient descent method to realize the training of the identification model.

In the test process, firstly, the image to be recognized acquired by the camera is preprocessed.

And if the image to be recognized contains the shielded corner part, directly inputting the image to be recognized into a trained recognition model, sequentially processing the image through a feature extraction layer, a channel separation layer and a category recognition layer, and finally recognizing the corner category corresponding to the shielded region in the image to be recognized.

If the image to be recognized does not contain the shielded corner part, the non-corner region in the image to be recognized needs to be covered in the corner region to form the shielded corner part, a new recognition image with the band is input into a trained recognition model, the new recognition image with the band is processed through a feature extraction layer, a channel separation layer and a category recognition layer in sequence, and finally the corresponding corner category is recognized from the covered region in the new image to be recognized.

The embodiment of the invention realizes the active corner shielding of the complete picture information based on the prejudgment of whether the image to be recognized contains the shielded corner or not, can improve the robustness of the recognition model and further improve the generation efficiency of the high-precision map.

Fig. 3 is a schematic structural diagram of a guideboard recognition device provided by the present invention. On the basis of any of the above embodiments, as shown in fig. 3, the apparatus includes an image acquisition module 310 and a corner point identification module 320, where:

the image obtaining module 310 is configured to obtain an image to be identified, where the image to be identified carries the blocked guideboard element.

The corner identification module 220 is configured to input the image to be identified to an identification model, and obtain a guideboard corner category output by the identification model and corresponding to the image to be identified.

The corner point identification module 320 specifically includes a feature extraction unit 321, a channel separation unit 322, and a category identification unit 323, where:

the feature extraction unit 321 is configured to input the image to be recognized to the feature extraction layer, and obtain a fused feature image output by the feature extraction layer.

The channel separation unit 322 is configured to input the fused feature image into the channel separation layer, and acquire a channel feature image output by the channel separation layer.

The category identifying unit 323 is configured to input the channel feature image to the category identifying layer, and acquire the category of the corner point of the guideboard output by the category identifying layer.

Specifically, the image acquisition module 310 and the corner point identification module 320 are electrically connected in sequence.

The image acquisition module 310 receives a picture taken by a camera mounted on a vehicle in real time as a picture to be recognized, and the picture to be recognized carries the blocked guideboard elements.

The corner point identification module 320 initializes the weight coefficients between the layers of the constructed identification model, and then inputs the labeled contents of a set of sample question data and sample answer data in the training set to the neural network under the current weight coefficients, and calculates the outputs of the nodes of the input layer, the hidden layer and the output layer in sequence. And correcting the weight coefficient between each node of the input layer and the hidden layer according to a gradient descent method by the accumulated error between the final output result of the output layer and the actual connection position state type of the output layer. According to the above process, the weight coefficients of the input layer and the hidden layer can be obtained until all samples in the training set are traversed.

The guideboard recognition device restores the recognition model according to the weight coefficients of the neural network input layer and the hidden layer, and inputs each image to be recognized in the test set into the trained recognition model, so as to obtain a recognition result corresponding to the image.

Optionally, the feature extraction unit 321 includes a multi-scale convolution sub-unit and a fusion sub-unit, where:

and the multi-scale convolution subunit is used for performing downsampling operation and convolution operation of different scales on the image to be identified to obtain the characteristic images of different scales.

And the fusion subunit is used for carrying out feature fusion based on the feature images of all scales to obtain the fusion feature image.

Optionally, the channel separation unit 322 is specifically configured to perform convolution calculation on the fusion feature image to obtain a channel feature image corresponding to each channel.

Optionally, the identification model is obtained based on the sample guideboard image, and the class label and the corner coordinate information labeled corresponding to the sample guideboard image.

Accordingly, the category identifying unit 323 comprises a probability set acquiring subunit and an identifying subunit, wherein:

and the probability set acquisition subunit is used for identifying and extracting the categories of the channel characteristic images corresponding to the channels to acquire the category probability sets corresponding to the channels.

And the identification subunit is used for determining the guideboard corner point category and the guideboard corner point coordinate corresponding to the channel characteristic image by using the category probability set corresponding to the channel.

Optionally, the image acquisition module 310 comprises a clipping unit and a covering unit, wherein:

and the intercepting unit is used for intercepting a non-corner region image based on the target corner in the image to be identified under the condition that the image to be identified is determined not to carry the blocked guideboard element.

And the covering unit is used for covering the non-corner region image in a region corresponding to the target corner to generate a new image to be identified.

The device for identifying a guideboard according to an embodiment of the present invention is configured to execute the method for identifying a guideboard according to the present invention, and an implementation manner of the device is consistent with that of the method for identifying a guideboard according to the present invention, and the same beneficial effects can be achieved, and details are not repeated here.

The embodiment of the invention is based on the fact that an image carrying blocked guideboard elements is used as input of a recognition model, the image to be recognized is subjected to feature extraction through a feature extraction layer, a channel separation layer is used for carrying out channel separation on a fusion feature image output by the feature extraction layer, then the channel feature images of all channels are subjected to full connection processing through a category recognition layer, angular points and angular point categories of the angular points contained in a blocked part and a blocked part in a corresponding image are output, incomplete image information is automatically recognized and calculated through a multi-layer neural network, the logicality and the relevance in the image containing incomplete information are fully mined, the accurate recognition of the blocked guideboard is realized, the fineness and the accuracy of guideboard recognition can be improved, and the generation efficiency with high accuracy is further improved. Fig. 4 is a schematic structural diagram of a vehicle provided by the present invention. On the basis of any of the above embodiments, as shown in fig. 4, the guideboard recognition device includes a vehicle body 410, and further includes a recognition device 420 disposed on the vehicle body 410, and the recognition device is configured to execute the above guideboard recognition method.

Specifically, the vehicle is composed of at least a vehicle body 410, and an identification device 420 embedded in a development board of the vehicle body 410.

The development board of the vehicle body 410 is connected to the identification device 420, and is used for remote transmission communication in a communication manner such as wireless communication technology (Wi-Fi), bluetooth, or a serial port, which is not specifically limited in this embodiment of the present invention.

The embodiment of the invention is based on the fact that an image carrying blocked guideboard elements is used as input of a recognition model, the image to be recognized is subjected to feature extraction through a feature extraction layer, a channel separation layer is used for carrying out channel separation on a fusion feature image output by the feature extraction layer, then the channel feature images of all channels are subjected to full connection processing through a category recognition layer, angular points and angular point categories of the angular points contained in a blocked part and a blocked part in a corresponding image are output, incomplete image information is automatically recognized and calculated through a multi-layer neural network, the logicality and the relevance in the image containing incomplete information are fully mined, the accurate recognition of the blocked guideboard is realized, the fineness and the accuracy of guideboard recognition can be improved, and the generation efficiency with high accuracy is further improved. Fig. 5 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 5: a processor (processor) 510, a communication Interface (Communications Interface) 520, a memory (memory) 530, and a communication bus 540, wherein the processor 510, the communication Interface 520, and the memory 530 communicate with each other via the communication bus 540. Processor 510 may invoke logic instructions in memory 530 to perform a method of guideboard identification, the method comprising: acquiring an image to be identified, wherein the image to be identified carries the shielded guideboard elements; inputting the image to be recognized into a recognition model, and obtaining the guideboard corner point category which is output by the recognition model and corresponds to the image to be recognized; the identification model is obtained based on a sample guideboard image and a category label labeled correspondingly to the sample guideboard image; the recognition model comprises a feature extraction layer, a channel separation layer and a category recognition layer; the step of inputting the image to be recognized into a recognition model to obtain the guideboard corner point category output by the recognition model and corresponding to the image to be recognized specifically includes: inputting the image to be identified into the feature extraction layer, and acquiring a fusion feature image output by the feature extraction layer; inputting the fusion characteristic image into the channel separation layer, and acquiring a channel characteristic image output by the channel separation layer; and inputting the channel characteristic image into the category identification layer, and acquiring the corner point category of the guideboard output by the category identification layer.

In addition, the logic instructions in the memory 530 may be implemented in the form of software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.

In another aspect, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being storable on a non-transitory computer-readable storage medium, the computer program, when executed by a processor, being capable of executing the method for identifying a guideboard provided by the above methods, the method comprising: acquiring an image to be identified, wherein the image to be identified carries the shielded guideboard elements; inputting the image to be recognized into a recognition model, and obtaining the guideboard corner point category which is output by the recognition model and corresponds to the image to be recognized; the identification model is obtained based on a sample guideboard image and a category label labeled correspondingly to the sample guideboard image; the identification model comprises a feature extraction layer, a channel separation layer and a category identification layer; the step of inputting the image to be recognized into a recognition model to obtain the guideboard corner point category output by the recognition model and corresponding to the image to be recognized specifically includes: inputting the image to be recognized to the feature extraction layer, and acquiring a fusion feature image output by the feature extraction layer; inputting the fusion characteristic image into the channel separation layer, and acquiring a channel characteristic image output by the channel separation layer; and inputting the channel characteristic image into the category identification layer, and acquiring the corner point category of the guideboard output by the category identification layer.

In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which when executed by a processor, implements a method for identifying a guideboard provided by the above methods, the method including: acquiring an image to be identified, wherein the image to be identified carries the shielded guideboard elements; inputting the image to be recognized into a recognition model, and obtaining the guideboard corner point category which is output by the recognition model and corresponds to the image to be recognized; the identification model is obtained based on a sample guideboard image and a category label correspondingly labeled with the sample guideboard image; the identification model comprises a feature extraction layer, a channel separation layer and a category identification layer; the step of inputting the image to be recognized into a recognition model to obtain the guideboard corner point category output by the recognition model and corresponding to the image to be recognized specifically includes: inputting the image to be identified into the feature extraction layer, and acquiring a fusion feature image output by the feature extraction layer; inputting the fused characteristic image into the channel separation layer, and acquiring a channel characteristic image output by the channel separation layer; and inputting the channel characteristic image into the category identification layer, and acquiring the corner point category of the guideboard output by the category identification layer.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment may be implemented by software plus a necessary general hardware platform, and may also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A method of identifying a guideboard, comprising:

the identification model is obtained based on a sample guideboard image and a category label correspondingly labeled with the sample guideboard image; the recognition model comprises a feature extraction layer, a channel separation layer and a category recognition layer;

2. The method for identifying the guideboard according to claim 1, wherein the inputting the image to be identified into the feature extraction layer and obtaining the fused feature image output by the feature extraction layer comprises:

3. The method for identifying a guideboard according to claim 1, wherein the inputting the fused feature image into the channel separation layer and acquiring the channel feature image output by the channel separation layer includes:

4. The method for identifying the guideboard according to claim 1, wherein the identification model is obtained based on a sample guideboard image, and a class label and corner point coordinate information labeled correspondingly to the sample guideboard image;

the inputting the channel characteristic image into the category identification layer and obtaining the corner point category of the guideboard output by the category identification layer specifically include:

5. The method for identifying a guideboard according to any one of claims 1-4, wherein the acquiring the image to be identified includes:

under the condition that the image to be recognized is determined not to carry the shielded guideboard element, intercepting a non-corner area image based on a target corner in the image to be recognized;

6. A device for recognizing a road sign, comprising:

7. A vehicle comprising a vehicle body, characterized by further comprising an identification device provided at the vehicle body for performing the method of identifying a guideboard according to any one of claims 1 to 5.

8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor, when executing the program, implements a method of identifying a guideboard according to any one of claims 1 to 5.

9. A non-transitory computer-readable storage medium on which a computer program is stored, wherein the computer program, when executed by a processor, implements the method of identifying a guideboard according to any one of claims 1 to 5.

10. A computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements a method of identifying a guideboard according to any one of claims 1 to 5.