CN115661796A - Guideboard identification method and device and vehicle - Google Patents

Abstract

The invention provides a method, a device and a vehicle for identifying a guideboard, wherein the method comprises the following steps: acquiring an image to be identified, wherein the image to be identified carries the shielded guideboard elements; and inputting the image to be recognized into a recognition model, and obtaining the guideboard corner point category which is output by the recognition model and corresponds to the image to be recognized. According to the method, the device and the vehicle for identifying the guideboard, provided by the invention, based on the fact that the image carrying the shielded guideboard element is used as the input of the identification model, the output result is that the angular points and the angular point categories thereof contained in the shielded part and the shielded part in the corresponding image are identified and calculated automatically through the multi-layer neural network, the logicality and the relevance in the picture containing the incomplete information are fully mined, the accurate identification of the shielded guideboard is realized, the fineness and the accuracy of the guideboard identification can be improved, and the generation efficiency of a high-precision map is further improved.

Description

Road sign recognition method, device and vehicle

technical field

The invention relates to the technical field of automatic driving, in particular to a road sign recognition method, device and vehicle.

Background technique

With the development of technologies such as artificial intelligence and autonomous driving, the construction of intelligent transportation has become a research hotspot, and high-precision maps are an indispensable part of the construction of intelligent transportation data. High-precision maps can contain a variety of traffic signs. For example, ground features such as lane lines, stop lines, and pedestrian crossing lines in the real world can be expressed through detailed maps, as well as high-altitude feature elements such as road signs and traffic lights, so as to provide information for autonomous driving, etc. Navigation in application scenarios provides data support.

As the information carrier of urban geographical entities, road signs in traffic signs have information navigation functions such as place names, routes, distances and directions. Good carrier for networking. Correct and efficient generation of traffic signs is critical to the drawing of high-precision maps.

However, when the street signs are blocked, the street signs cannot be recognized because sufficient information cannot be obtained from the images acquired by the vehicle's camera, which affects the efficiency of high-precision map drawing.

Contents of the invention

The invention provides a street sign recognition method, device and vehicle to solve the defect that the blocked street sign cannot be recognized in the prior art.

The invention provides a road sign recognition method, comprising:

Acquiring the image to be recognized, the image to be recognized carries the blocked street sign element;

inputting the image to be recognized into a recognition model, and obtaining the street sign corner category output by the recognition model corresponding to the image to be recognized;

Wherein, the recognition model is obtained based on a sample street sign image and a category label corresponding to the sample street sign image; the recognition model includes a feature extraction layer, a channel separation layer, and a category recognition layer;

The step of inputting the image to be recognized into the recognition model and obtaining the street sign corner category output by the recognition model corresponding to the image to be recognized specifically includes:

Inputting the image to be identified into the feature extraction layer, and obtaining a fusion feature image output by the feature extraction layer;

input the fusion feature image to the channel separation layer, and obtain the channel feature image output by the channel separation layer;

The channel feature image is input to the category recognition layer, and the street sign corner category output by the category recognition layer is obtained.

According to a street sign recognition method provided by the present invention, the inputting the image to be recognized into the feature extraction layer and obtaining the fused feature image output by the feature extraction layer includes:

Performing downsampling operations and convolution operations of different scales on the image to be identified to obtain feature images of different scales;

performing feature fusion based on the feature images of each scale, and obtaining the fused feature image;

Among them, after each feature fusion, a multi-scale convolution is performed before the next feature fusion.

According to a street sign recognition method provided by the present invention, the input of the fusion feature image to the channel separation layer, and the acquisition of the channel feature image output by the channel separation layer include:

Carrying out convolution calculation on the fusion feature image to obtain the channel feature image corresponding to each channel;

Wherein, the number of passages is determined according to the number of street sign corners contained in a complete street sign element.

According to a street sign recognition method provided by the present invention, the recognition model is obtained based on a sample street sign image, and category labels and corner point coordinate information corresponding to the sample street sign image;

The step of inputting the channel feature image into the category identification layer, and obtaining the street sign corner category output by the category identification layer specifically includes:

For the channel feature image corresponding to each channel, perform category identification and extraction, and obtain the category probability set corresponding to each channel;

Using the class probability set corresponding to the channel, the street sign corner category corresponding to the channel feature image and the street sign corner point coordinates are determined.

According to a recognition method of street signs provided by the present invention, the acquisition of an image to be recognized includes:

In the case where it is determined that the image to be recognized does not carry a blocked street sign element, based on the target corner point in the image to be recognized, intercept the non-corner area image;

Overlay the non-corner area image on the area corresponding to the target corner to generate a new image to be recognized;

The new image to be recognized is input into the recognition model, and the street sign corner category corresponding to the new image to be recognized output by the recognition model is obtained.

The present invention also provides a street sign recognition device, comprising:

An image acquisition module, configured to acquire an image to be recognized, the image to be recognized carries a blocked street sign element;

A corner point recognition module, configured to input the image to be recognized into a recognition model, and obtain the street sign corner category output by the recognition model corresponding to the image to be recognized;

Wherein, the recognition model is obtained based on a sample street sign image and a category label corresponding to the sample street sign image; the recognition model includes a feature extraction layer, a channel separation layer, and a category recognition layer;

The corner recognition module specifically includes a feature extraction unit, a channel separation unit and a category recognition unit, wherein:

The feature extraction unit is configured to input the image to be recognized to the feature extraction layer, and obtain a fusion feature image output by the feature extraction layer;

The channel separation unit is configured to input the fusion feature image to the channel separation layer, and obtain the channel feature image output by the channel separation layer;

The category identification unit is configured to input the channel feature image to the category identification layer, and obtain the street sign corner category output by the category identification layer.

The present invention also provides a vehicle, including a vehicle body, and a recognition device disposed on the vehicle body, the recognition device being used to execute the street sign recognition method as described above.

The present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory and operable on the processor. When the processor executes the program, it realizes the identification of any road sign described above. method.

The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the identification method of any one of the above-mentioned street signs is realized.

The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, any one of the street sign recognition methods described above is realized.

The road sign recognition method, device and vehicle provided by the present invention are based on the image carrying the occluded street sign elements as the input of the recognition model, and the output result is the corner points and the occluded parts contained in the corresponding image. Its corner point category automatically recognizes and calculates incomplete image information through a multi-layer neural network to realize accurate recognition of blocked street signs, which can improve the fineness and accuracy of street sign recognition, thereby improving the generation efficiency of high-precision maps.

Description of drawings

In order to more clearly illustrate the present invention or the technical solutions in the prior art, the accompanying drawings that need to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings in the following description are the present invention. For some embodiments of the invention, those skilled in the art can also obtain other drawings based on these drawings without creative effort.

Fig. 1 is one of the schematic flow charts of the recognition method of street signs provided by the present invention;

Fig. 2 is the second schematic flow diagram of the identification method of street signs provided by the present invention;

Fig. 3 is a structural schematic diagram of a street sign recognition device provided by the present invention;

Fig. 4 is a schematic structural view of a vehicle provided by the present invention;

Fig. 5 is a schematic structural diagram of an electronic device provided by the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention , but not all examples. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

The terms "first", "second" and the like in the description of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application can be practiced in sequences other than those illustrated or described herein, and that references to "first," "second," etc. distinguish Objects are generally of one type, and the number of objects is not limited. For example, there may be one or more first objects.

It should be understood that the terminology used in the description of the present invention is for the purpose of describing particular embodiments only and is not intended to limit the present invention. As used in the present specification, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly dictates otherwise.

The terms "comprising" and "comprising" indicate the presence of described features, integers, steps, operations, elements and/or components, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components and/or The presence or addition of its collection.

Fig. 1 is one of the schematic flowcharts of the street sign recognition method provided by the present invention. As shown in FIG. 1 , the street sign recognition method provided by the embodiment of the present invention includes: Step 101 , acquiring an image to be recognized, and the image to be recognized carries a blocked street sign element.

It should be noted that, the road sign recognition method provided by the embodiment of the present invention is executed by a street sign recognition device. The street sign recognition device may be a central processing unit (Central Processing Unit, CPU) built into the vehicle, or a development board integrated based on the CPU for information processing and program operation.

The application scenario of the street sign recognition method provided by the embodiment of the present invention is to identify an image containing an occluded street sign, and obtain all corner points contained in the unoccluded part and the occluded part of the street sign, and the types of the corner points.

Specifically, in step 101, the street sign recognition device receives in real time a picture taken by a camera installed in a vehicle as a picture to be recognized, and the picture to be recognized carries a blocked street sign element.

The street sign element is a graphic element belonging to the traffic sign captured by the camera during the driving process of the vehicle.

Wherein, the street sign element carried in the image to be recognized is not limited to one kind, and may be street signs corresponding to various traffic signs. The street sign corresponding to the traffic sign is usually a regular figure, which is not specifically limited in the embodiment of the present invention.

Exemplarily, the shape of the street sign corresponding to the traffic sign can be square, and the square street sign has four right-angle points, and the right-angle points on each orientation are the street sign corner points of the corresponding category, and then in the recognition process, it can be blocked by The identification of the street sign element to get the corner point category in the street sign element.

Step 102: Input the image to be recognized into a recognition model, and obtain the street sign corner category output by the recognition model corresponding to the image to be recognized.

Wherein, the recognition model is obtained based on a sample street sign image and a category label corresponding to the sample street sign image. The recognition model includes feature extraction layer, channel separation layer and category recognition layer.

The step of inputting the image to be recognized into the recognition model and obtaining the street sign corner category output by the recognition model corresponding to the image to be recognized specifically includes:

The image to be recognized is input to the feature extraction layer, and a fused feature image output by the feature extraction layer is obtained.

The fusion feature image is input to the channel separation layer, and the channel feature image output by the channel separation layer is obtained.

The channel feature image is input to the category recognition layer, and the street sign corner category output by the category recognition layer is obtained.

It should be noted that the recognition model may be a neural network model, and the structure and parameters of the neural network include but are not limited to the input layer, hidden layer and output layer of the neural network, and the weight parameters of each layer. The embodiment of the present invention does not specifically limit the type and structure of the neural network.

For example, a recognition model could be a feed-forward neural network consisting of an input layer, a hidden layer, and an output layer, where:

The input layer is at the front end of the entire network, and directly receives the image data carrying the occluded street sign elements.

The hidden layer can have one or more layers, and its own neurons perform weighted summation on the input vector. The calculation formula can be expressed as:

z=b+w1*x1+w2*x2+...+wm*xm

Among them, z is the sum of the weights output by the hidden layer, x1, x2, x3...xm are the m feature vectors of each sample, b is the bias, w1, w2...wm are the weights corresponding to each feature vector .

The output layer is the last layer, which is used to output the recognition result of the corner point category, and output the type of the recognition result according to different requirements. This value can be a category vector value, or a continuous value similar to linear regression. It may also be other complex types of values or vectors, which are not specifically limited in this embodiment of the present invention.

The activation function is a function that runs on the neurons of the artificial neural network. It is responsible for mapping the input of the neurons to the output. The activation function is used for logistic regression processing, that is, the sum of the weights output by the hidden layer is converted into a non-linear recognition result. , the embodiment of the present invention does not specifically limit the type of activation function.

Preferably, the Softmax function is used for logistic regression processing, and the outputs of multiple neurons are mapped to the (0,1) interval, which can be understood as a probability, so as to perform multi-classification.

It should be noted that the sample data includes a sample street sign image corresponding to the sample data, and category labels marked on diagonal points in the sample street sign image. Divide the sample data into a training set and a test set according to a certain ratio.

Exemplarily, the proportion of the training set and the test set in the sample data includes but is not limited to 9:1, 8:2, etc., which is not specifically limited in this embodiment of the present invention.

Specifically, in step 102, the road sign recognition device initializes the weight coefficients between the layers of the constructed recognition model, and then inputs a set of sample question data and sample answer data in the training set into the current weight coefficient The neural network below calculates the output of each node in the input layer, hidden layer and output layer in turn. According to the cumulative error between the final output result of the output layer and the actual connection position state type, the weight coefficient between each node of the input layer and the hidden layer is corrected according to the gradient descent method. According to the above process, until all the samples in the training set are traversed, the weight coefficients of the input layer and the hidden layer can be obtained.

The street sign recognition device restores the recognition model in step 102 according to the weight coefficients of the neural network input layer and the hidden layer, and inputs each image to be recognized in the test set into the trained recognition model to obtain the corresponding recognition of the image. result.

The recognition result may be a probability value or a label result, and the embodiment of the present invention does not specifically limit the form of the behavior recognition result.

If the recognition result can be a probability value, the probability value can be used to describe the probability that the corner points included in the image to be recognized belong to the corner point categories corresponding to each orientation.

If the recognition result can be a label result, an intermediate numerical result can be obtained through the model, and the numerical result meets the preset target conditions, and the corner point corresponding to the numerical result is assigned the corner point category label of the corresponding orientation.

Preferably, the recognition model built in the street sign recognition device consists of an input layer, a hidden layer and an output layer. The role of the hidden layer is to extract the features of the input image with incomplete information through its own neurons, and extract the feature information that is beneficial to recognition.

The embodiment of the present invention does not specifically limit the structure of the hidden layer.

Preferably, the hidden layer includes at least three layers, which are feature extraction layer, channel separation layer and category recognition layer, wherein:

The feature extraction layer can use Convolutional Neural Network (CNN) to reduce the dimension of the image to be recognized, compress the vector and perform feature extraction and fusion at the same time to obtain the fusion feature image.

Among them, in the convolution process, dilated convolution or dilated convolution can also be used to increase the receptive field without performing pooling loss information, so that each convolution output contains a larger range of information.

The channel separation layer can set the number of channels of the convolution kernel according to the number of corner points contained in the street signs, and perform convolution calculation on the fusion feature image again to obtain the two-dimensional channel feature image corresponding to each channel.

The category recognition layer can map the two-dimensional channel feature images corresponding to each channel into two-dimensional vectors by using full connection processing and Softmax processing in turn, and classify them according to the two-dimensional vectors to obtain the street sign corner category, and then we can know the Identify all corner points contained in the image, and the corresponding corner class.

In the embodiment of the present invention, based on the image carrying the occluded street sign elements as the input of the recognition model, feature extraction is performed on the image to be recognized through the feature extraction layer, and channel separation is performed on the fused feature image output by the feature extraction layer through the channel separation layer After that, the channel feature images of each channel are fully connected through the category recognition layer, and the corner points and their corner point categories contained in the occluded part and occluded part of the corresponding image are output. The automatic recognition and calculation of image information fully excavates the logic and relevance of pictures containing incomplete information, realizes the accurate recognition of blocked street signs, improves the fineness and accuracy of street sign recognition, and then improves the generation efficiency of high-precision maps.

On the basis of any of the above embodiments, inputting the image to be recognized into the feature extraction layer, and obtaining the fused feature image output by the feature extraction layer includes: downscaling the image to be recognized at different scales Sampling operation and convolution operation to obtain feature images of different scales.

Based on the feature images of each scale, feature fusion is performed to obtain the fused feature image.

Among them, after each feature fusion, a multi-scale convolution is performed before the next feature fusion.

It should be noted that the road sign recognition device uses a large number of convolution kernels to extract high-dimensional feature maps from the matrix corresponding to the remote sensing image, and can extract multiple feature maps. Each feature map is a local perception extracted from the picture. Combining these feature maps can extract the interesting parts of the picture.

Convolution kernels of different sizes have different effects on the original extraction of different types of road signs, and the number and size of convolution kernels for multi-scale downsampling in the embodiment of the present invention are not specifically limited.

Among them, the use of a small convolution kernel 1*1 can extract relevant feature data for identifying small road signs. Using the middle convolution kernel 3*3 can extract the relevant feature information for identifying middle road signs. Using a large convolution kernel 5*5 can extract the relevant feature information for identifying road signs. Then, when the feature data of different scales are fused, the relevant information of street signs with three different specifications can be retained, thereby preventing the loss of original valid data.

It should be noted that convolution calculation is required after each fusion to prevent the introduction of noise.

Specifically, the road sign recognition device inputs the image to be recognized into the cascaded convolutional layer, and first enters the convolution cores of different sizes to perform convolution calculations to obtain feature images of different dimensions respectively. By convolving this time The obtained multi-dimensional feature images are linearly added, and after the fusion feature image is obtained, it enters the next-level convolution kernel at the same time to repeat the above process, continuously performs feature extraction and feature fusion processes, and updates the fusion feature image.

Exemplarily, five convolution kernels with a size of 3*3 and a number of channels of 128, 64, 32, 16, and 8 can be set in sequence, and correspondingly, five convolution kernels with a size of 5*5 and a number of channels can be set correspondingly Perform convolution processing for convolution kernels of 128, 64, 32, 16, and 8, so that the height and width of the feature map of the image will be reduced by half after each convolution processing, and the number of channels will be reduced by half, performing five convolutions and linear After the addition, a fusion feature image with 8 channels can be obtained.

It should be noted that in order to linearly add feature images of multiple scales, it is necessary to use padding to ensure that the number of rows and columns of feature images calculated by convolution is the same.

The embodiment of the present invention is based on performing convolution calculations of different scales on the image to be recognized, and performing multiple fusion and convolution dimensionality reduction on multi-scale feature images obtained by convolution to obtain the fused feature image. It can make the image contain low-level detail features and high-level semantic features at the same time, which can improve the fineness of image semantic segmentation, and then improve the accuracy of image recognition.

On the basis of any of the above embodiments, inputting the fused feature image to the channel separation layer, and obtaining the channel feature image output by the channel separation layer includes: performing convolution calculation on the fused feature image, obtaining The channel feature image corresponding to each channel.

Wherein, the number of passages is determined according to the number of street sign corners contained in a complete street sign element.

It should be noted that before step 102, the number of channels needs to be set according to the number of corner points included in street sign elements of different specifications, so that each channel corresponds to the feature corresponding to each corner point.

Specifically, the road sign recognition device uses a convolution kernel with a channel number of 4 to perform convolution calculations on the fused feature image successively to obtain the feature data of each channel, and then uses a convolution kernel with a size of 1x1 to perform full-connection dimensionality reduction to obtain The channel feature image of multiple channels with the same width and height dimensions as the original image.

In the embodiment of the present invention, after the number of channels is determined based on the number of street sign corners included in the street sign elements, convolution calculation is performed on the fusion feature image corresponding to the number of channels to separate the channel feature images of each channel. It can improve the fineness of image semantic segmentation, and then improve the accuracy of image recognition.

On the basis of any of the above embodiments, the recognition model is obtained based on a sample street sign image, and category labels and corner point coordinate information marked corresponding to the sample street sign image.

The step of inputting the channel characteristic image into the category recognition layer and obtaining the street sign corner category output by the category recognition layer specifically includes: performing category identification and extraction on the channel characteristic image corresponding to each channel, and obtaining The set of category probabilities corresponding to each channel.

Specifically, in the process of training the recognition model, the street sign recognition device trains the corner point categories corresponding to each channel and the corner point coordinate information corresponding to the corner points of the identification category as true values, and performs training on the feature images of each channel Recognition, the pixel value of each pixel in the channel feature image is mapped to the probability value of the corner point category corresponding to different channels, and integrated into a category probability set.

Wherein, for any channel feature image, among various category probability values corresponding to each pixel, the category probability corresponding to its channel should be the maximum value.

Using the class probability set corresponding to the channel, the street sign corner category corresponding to the channel feature image and the street sign corner point coordinates are determined.

Specifically, the street sign recognition device outputs the corner point category corresponding to the maximum value among the multiple probability values corresponding to each pixel from the category probability set corresponding to each channel, and determines according to the corresponding corner point category The coordinates of the street sign corners.

For example, if you want to recognize a square street sign in an image to be recognized, whether it is a small square street sign, a medium square street sign or a large square street sign, these three sizes of square street signs all have four street sign corners, and define the upper left corner of the street sign element, The upper right corner, lower right corner, and lower left corner correspond to the corner category setting numbers 0, 1, 2, and 3.

Therefore, the number of ordinary convolution channels for performing ordinary convolution calculation on the fusion feature image to be processed is equal to 4, and 4 channel feature images can be obtained. When identifying the channel feature image corresponding to the channel whose category number is 0, the pixel point corresponding to the probability value greater than the preset threshold in the category probability set is identified as the corner point category 0, and the corresponding pixel coordinates are used as the upper left of the road sign Corner coordinate output.

When identifying the channel feature image corresponding to the channel with category number 1, identify the pixel corresponding to the probability value greater than the preset threshold in the category probability set as the No. 1 corner point category, and use the corresponding pixel coordinates as the upper right corner of the street sign Corner coordinate output.

When identifying the channel feature image corresponding to the channel whose category number is 2, the pixel corresponding to the probability value greater than the preset threshold in the category probability set is identified as the No. 2 corner point category, and the corresponding pixel coordinate is used as the street sign Output the coordinates of the lower corner point.

When identifying the channel feature image corresponding to the channel with category number 3, the pixel corresponding to the probability value greater than the preset threshold in the category probability set is identified as the No. 3 corner point category, and the corresponding pixel coordinates are used as the lower left of the street sign Corner coordinate output.

The embodiment of the present invention is based on the image carrying the occluded street sign elements as the input of the recognition model, and the output result is the corner points contained in the occluded part and the occluded part of the corresponding image, its corner point category and its corner point coordinates, through The multi-layer neural network automatically recognizes and calculates incomplete image information, realizes accurate recognition of blocked street signs, improves the fineness and accuracy of street sign recognition, and then improves the generation efficiency of high-precision maps.

On the basis of any of the above-mentioned embodiments, the acquiring the image to be recognized includes: when it is determined that the image to be recognized does not carry an occluded street sign element, based on the target corner point in the image to be recognized , intercept the non-corner area image.

Specifically, in step 101, the street sign recognition device conducts preliminary screening and recognition on the image to be recognized collected by the camera, and if it is recognized that the image to be recognized does not contain any blocked street sign elements, it extracts one or For multiple target corners, a pixel point that is a certain pixel distance from the target corner point is used as the starting point of the non-corner point area, and any pixel point in the direction away from the target corner point pixel is selected as the end point of the non-corner point area to intercept the limited A non-corner area image within a good non-corner area.

Overlay the non-corner region image on the region corresponding to the target corner to generate a new image to be recognized.

Specifically, the street sign recognition device first determines the size of the non-corner area image, and then overlaps the starting point of the target corner with the non-corner area, so that the non-corner area image of the corresponding size is covered in the area where the target corner is located , to generate a new image to be recognized. It is used for the recognition model to recognize the category of the target corner from the covered area in the new image to be recognized.

The embodiment of the present invention does not specifically limit the specific implementation manner of the identification process.

Preferably, FIG. 2 is the second schematic flow diagram of the street sign recognition method provided by the present invention. As shown in Figure 2, the entire recognition process includes a training process and a testing process.

(1) The training process is to sequentially input the sample street sign image to the feature extraction layer, channel separation layer and category recognition layer for processing, and perform the gradient descent method on the obtained sample street sign corner category and the category label corresponding to the sample street sign image , to realize the training of the recognition model.

(2) During the testing process, the image to be recognized collected by the camera should be preprocessed first.

If the image to be recognized contains occluded corners, the image to be recognized is directly input into the trained recognition model, and then processed through the feature extraction layer, channel separation layer and category recognition layer in turn, and finally from the image to be recognized The occluded regions identify their corresponding corner categories.

If the image to be recognized does not contain the occluded corner part, it is necessary to cover the non-corner area in the image to be recognized on the corner area to form the occluded corner part, and input the new image with recognition to the trained In the recognition model, it is processed sequentially through the feature extraction layer, channel separation layer and category recognition layer, and finally the corresponding corner category is identified from the covered area in the new image to be recognized.

The embodiment of the present invention is based on predicting whether the image to be recognized contains occluded corners, and realizes active corner occlusion of complete picture information, which can improve the robustness of the recognition model, and further improve the generation efficiency of high-precision maps.

Fig. 3 is a schematic structural diagram of a street sign recognition device provided by the present invention. On the basis of any of the above embodiments, as shown in FIG. 3 , the device includes an image acquisition module 310 and a corner recognition module 320, wherein:

The image acquiring module 310 is configured to acquire an image to be recognized, and the image to be recognized carries a blocked street sign element.

The corner recognition module 220 is configured to input the image to be recognized into a recognition model, and obtain the street sign corner category output by the recognition model corresponding to the image to be recognized.

Wherein, the recognition model is obtained based on a sample street sign image and a category label corresponding to the sample street sign image. The recognition model includes a feature extraction layer, a channel separation layer and a category recognition layer.

The corner point identification module 320 specifically includes a feature extraction unit 321, a channel separation unit 322 and a category identification unit 323, wherein:

The feature extraction unit 321 is configured to input the image to be recognized to the feature extraction layer, and obtain a fused feature image output by the feature extraction layer.

The channel separation unit 322 is configured to input the fused feature image to the channel separation layer, and obtain a channel feature image output by the channel separation layer.

The category identification unit 323 is configured to input the channel feature image to the category identification layer, and obtain the street sign corner category output by the category identification layer.

Specifically, the image acquisition module 310 and the corner recognition module 320 are electrically connected in sequence.

The image acquisition module 310 receives in real time the picture taken by the camera installed in the vehicle as the picture to be recognized, and the picture to be recognized carries the blocked street sign element.

The corner point recognition module 320 initializes the weight coefficients between the layers of the constructed recognition model, and then inputs a set of sample question data and sample answer data in the training set into the neural network under the current weight coefficient, and calculates The output of each node of the input layer, hidden layer, and output layer. According to the cumulative error between the final output result of the output layer and the actual connection position state type, the weight coefficient between each node of the input layer and the hidden layer is corrected according to the gradient descent method. According to the above process, until all the samples in the training set are traversed, the weight coefficients of the input layer and the hidden layer can be obtained.

The street sign recognition device restores the recognition model according to the weight coefficients of the input layer and the hidden layer of the neural network, and inputs each image to be recognized in the test set into the trained recognition model to obtain the recognition result corresponding to the image.

Optionally, the feature extraction unit 321 includes a multi-scale convolution subunit and a fusion subunit, wherein:

The multi-scale convolution subunit is used to perform down-sampling operations and convolution operations of different scales on the image to be recognized to obtain feature images of different scales.

The fusion subunit is configured to perform feature fusion based on feature images of various scales, and obtain the fused feature images.

Among them, after each feature fusion, a multi-scale convolution is performed before the next feature fusion.

Optionally, the channel separation unit 322 is specifically configured to perform convolution calculation on the fused feature image to obtain channel feature images corresponding to each channel.

Wherein, the number of passages is determined according to the number of street sign corners contained in a complete street sign element.

Optionally, the recognition model is obtained based on a sample street sign image, and category labels and corner coordinate information marked corresponding to the sample street sign image.

Correspondingly, the category identification unit 323 includes a probability set acquisition subunit and an identification subunit, wherein:

The probability set acquisition subunit is used to identify and extract the category of the channel feature image corresponding to each channel, and acquire the category probability set corresponding to each channel.

The identification subunit is configured to use the category probability set corresponding to the channel to determine the category of the corner point of the street sign corresponding to the feature image of the channel and the coordinates of the corner point of the street sign.

Optionally, the image acquisition module 310 includes an intercepting unit and a covering unit, wherein:

An intercepting unit, configured to intercept a non-corner region image based on a target corner point in the image to be recognized when it is determined that the image to be recognized does not carry an occluded street sign element.

A covering unit, configured to cover the non-corner region image on the region corresponding to the target corner to generate a new image to be recognized.

The street sign recognition device provided by the embodiment of the present invention is used to implement the above-mentioned street sign recognition method of the present invention, and its implementation mode is consistent with the implementation mode of the street sign recognition method provided by the present invention, and can achieve the same beneficial effect. Let me repeat.

In the embodiment of the present invention, based on the image carrying the occluded street sign elements as the input of the recognition model, feature extraction is performed on the image to be recognized through the feature extraction layer, and channel separation is performed on the fused feature image output by the feature extraction layer through the channel separation layer After that, the channel feature images of each channel are fully connected through the category recognition layer, and the corner points and their corner point categories contained in the occluded part and occluded part of the corresponding image are output. The automatic recognition and calculation of image information fully excavates the logic and relevance of pictures containing incomplete information, realizes the accurate recognition of blocked street signs, improves the fineness and accuracy of street sign recognition, and then improves the generation efficiency of high-precision maps. Fig. 4 is a structural schematic diagram of the vehicle provided by the present invention. On the basis of any of the above embodiments, as shown in FIG. 4 , it includes a vehicle body 410 and a recognition device 420 disposed on the vehicle body 410 , the recognition device is used to implement the above-mentioned street sign recognition method.

Specifically, the vehicle is composed of at least a vehicle body 410 and an identification device 420 embedded in a development board of the vehicle body 410 .

The development board of the vehicle body 410 is connected to the identification device 420 for long-distance transmission and communication through wireless communication technology (Wi-Fi), bluetooth or serial port and other communication methods, which is not specifically limited in the embodiment of the present invention.

In the embodiment of the present invention, based on the image carrying the occluded street sign elements as the input of the recognition model, feature extraction is performed on the image to be recognized through the feature extraction layer, and channel separation is performed on the fused feature image output by the feature extraction layer through the channel separation layer After that, the channel feature images of each channel are fully connected through the category recognition layer, and the corner points and their corner point categories contained in the occluded part and occluded part of the corresponding image are output. The automatic recognition and calculation of image information fully excavates the logic and relevance of pictures containing incomplete information, realizes the accurate recognition of blocked street signs, improves the fineness and accuracy of street sign recognition, and then improves the generation efficiency of high-precision maps. FIG. 5 illustrates a schematic diagram of the physical structure of an electronic device. As shown in FIG. 5, the electronic device may include: a processor (processor) 510, a communication interface (Communications Interface) 520, a memory (memory) 530 and a communication bus 540, Wherein, the processor 510 , the communication interface 520 , and the memory 530 communicate with each other through the communication bus 540 . The processor 510 can call the logic instruction in the memory 530 to execute the identification method of the road sign, the method includes: acquiring an image to be recognized, the image to be recognized carries a blocked street sign element; inputting the image to be recognized to A recognition model, obtaining the street sign corner category output by the recognition model and corresponding to the image to be recognized; wherein, the recognition model is obtained based on a sample street sign image and a category label corresponding to the sample street sign image; The recognition model includes a feature extraction layer, a channel separation layer, and a category recognition layer; the input of the image to be recognized to the recognition model is to obtain the street sign corner category output by the recognition model corresponding to the image to be recognized, It specifically includes: inputting the image to be recognized into the feature extraction layer, obtaining a fusion feature image output by the feature extraction layer; inputting the fusion feature image into the channel separation layer, and obtaining the output of the channel separation layer The channel feature image; input the channel feature image to the category identification layer, and obtain the street sign corner category output by the category identification layer.

In addition, the above logic instructions in the memory 530 may be implemented in the form of software function units and be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes. .

On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can Executing the street sign recognition method provided by each of the above methods, the method includes: acquiring an image to be recognized, the image to be recognized carries a blocked street sign element; inputting the image to be recognized to a recognition model to obtain the recognition The street sign corner category corresponding to the image to be recognized output by the model; wherein, the recognition model is obtained based on the sample street sign image and the category label corresponding to the sample street sign image; the recognition model includes feature extraction layer, channel separation layer, and category recognition layer; the input of the image to be recognized into the recognition model, and obtaining the street sign corner category output by the recognition model corresponding to the image to be recognized specifically includes: The recognition image is input to the feature extraction layer, and the fusion feature image output by the feature extraction layer is obtained; the fusion feature image is input to the channel separation layer, and the channel feature image output by the channel separation layer is obtained; The channel feature image is input to the category recognition layer, and the street sign corner category output by the category recognition layer is obtained.

In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it is implemented to perform the street sign recognition method provided by the above-mentioned methods, the method includes : Obtain the image to be recognized, the image to be recognized carries the blocked street sign element; input the image to be recognized to the recognition model, and obtain the street sign corner category output by the recognition model corresponding to the image to be recognized ; Wherein, the recognition model is obtained based on the sample street sign image and the category label corresponding to the sample street sign image; the recognition model includes a feature extraction layer, a channel separation layer and a category recognition layer; the described The image to be recognized is input to the recognition model, and the street sign corner category corresponding to the image to be recognized outputted by the recognition model is obtained, which specifically includes: inputting the image to be recognized to the feature extraction layer, and obtaining the feature extraction The fusion feature image output by the layer; the fusion feature image is input to the channel separation layer, and the channel feature image output by the channel separation layer is obtained; the channel feature image is input to the category identification layer, and the The street sign corner category output by the category recognition layer.

The device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without any creative effort.

Through the above description of the implementations, those skilled in the art can clearly understand that each implementation can be implemented by means of software plus a necessary general hardware platform, and of course also by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of software products, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic discs, optical discs, etc., including several instructions to make a computer device (which may be a personal computer, server, or network device, etc.) execute the methods described in various embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims (10)

1. A recognition method for street signs, comprising: Acquiring the image to be recognized, the image to be recognized carries the blocked street sign element; inputting the image to be recognized into a recognition model, and obtaining the street sign corner category output by the recognition model corresponding to the image to be recognized; Wherein, the recognition model is obtained based on a sample street sign image and a category label corresponding to the sample street sign image; the recognition model includes a feature extraction layer, a channel separation layer, and a category recognition layer; The step of inputting the image to be recognized into the recognition model and obtaining the street sign corner category output by the recognition model corresponding to the image to be recognized specifically includes: Inputting the image to be identified into the feature extraction layer, and obtaining a fusion feature image output by the feature extraction layer; input the fusion feature image to the channel separation layer, and obtain the channel feature image output by the channel separation layer; The channel feature image is input to the category recognition layer, and the street sign corner category output by the category recognition layer is obtained. 2. the recognition method of street sign according to claim 1, is characterized in that, described described image to be recognized is input to described feature extraction layer, obtains the fusion characteristic image of described feature extraction layer output, comprises: Performing downsampling operations and convolution operations of different scales on the image to be identified to obtain feature images of different scales; performing feature fusion based on the feature images of each scale, and obtaining the fused feature image; Among them, after each feature fusion, a multi-scale convolution is performed before the next feature fusion. 3. the identification method of street sign according to claim 1, is characterized in that, described fusion feature image is input to described channel separation layer, obtains the channel feature image of described channel separation layer output, comprising: Carrying out convolution calculation on the fusion feature image to obtain the channel feature image corresponding to each channel; Wherein, the number of passages is determined according to the number of street sign corners contained in a complete street sign element. 4. the identification method of street sign according to claim 1, is characterized in that, described recognition model is based on sample street sign image, and the category label and corner point coordinate information of label corresponding to described sample street sign image obtain; The step of inputting the channel feature image into the category identification layer, and obtaining the street sign corner category output by the category identification layer specifically includes: For the channel feature image corresponding to each channel, perform category identification and extraction, and obtain the category probability set corresponding to each channel; Using the class probability set corresponding to the channel, the street sign corner category corresponding to the channel feature image and the street sign corner point coordinates are determined. 5. The method for identifying street signs according to any one of claims 1-4, wherein said acquiring an image to be identified comprises: In the case where it is determined that the image to be recognized does not carry a blocked street sign element, based on the target corner point in the image to be recognized, intercept the non-corner area image; Overlay the non-corner region image on the region corresponding to the target corner to generate a new image to be recognized. 6. A street sign recognition device, characterized in that it comprises: An image acquisition module, configured to acquire an image to be recognized, the image to be recognized carries a blocked street sign element; A corner point recognition module, configured to input the image to be recognized into a recognition model, and obtain the street sign corner category output by the recognition model corresponding to the image to be recognized; Wherein, the recognition model is obtained based on a sample street sign image and a category label corresponding to the sample street sign image; the recognition model includes a feature extraction layer, a channel separation layer, and a category recognition layer; The corner recognition module specifically includes a feature extraction unit, a channel separation unit and a category recognition unit, wherein: The feature extraction unit is configured to input the image to be recognized to the feature extraction layer, and obtain a fusion feature image output by the feature extraction layer; The channel separation unit is configured to input the fusion feature image to the channel separation layer, and obtain the channel feature image output by the channel separation layer; The category identification unit is configured to input the channel feature image to the category identification layer, and obtain the street sign corner category output by the category identification layer. 7. A vehicle, comprising a vehicle body, characterized in that it further comprises a recognition device disposed on the vehicle body, the recognition device being used to execute the street sign recognition method according to any one of claims 1 to 5. 8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein the processor according to claim 1 is implemented when executing the program. The identification method of the street sign described in any one of to 5. 9. A non-transitory computer-readable storage medium, on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the street sign recognition method according to any one of claims 1 to 5 is implemented. 10. A computer program product, comprising a computer program, characterized in that, when the computer program is executed by a processor, the street sign recognition method according to any one of claims 1 to 5 is implemented.

Images