JP2017516197A - Method and apparatus for recognizing traffic signs - Google Patents

Abstract

The present application discloses a method and apparatus for recognizing traffic signs. One specific embodiment of the method includes the step of obtaining a feature value obtained by a predetermined feature algorithm in a predetermined integration channel of a scan window image obtained by dividing by a spherical image, and the scan Detecting a scan window image based on a feature value of the window image and a pretrained detection classifier model, obtaining a traffic sign window image to be confirmed, and a pretrained convolutional neural network model Recognizing a traffic sign window image to be confirmed and obtaining a traffic sign type, wherein the pre-trained detection classifier model comprises a sample scan window image and its The pre-trained convolution menu obtained by training based on the feature values. Le network model obtained by training on the basis of the sample and its traffic sign kinds of traffic signs window image was confirmed. According to this embodiment, the accuracy of detecting and recognizing traffic signs in a panoramic image is improved, and the efficiency of updating road network data is improved. [Selection] Figure 1

Description

Priority request

  This application claims priority, filed on Mar. 31, 2015, with a Chinese patent application number of “201510150525.8” and is incorporated herein by reference in its entirety.

  The present application relates to the field of computer technology, specifically to the field of computer target recognition, and more particularly to a method and apparatus for recognizing traffic signs.

  In order to provide accurate and complete navigation data, it is necessary to recognize traffic signs in the traffic road network.

  The conventional method of recognizing traffic signs mainly depends on artificial processing, placing specific program software and base map data in a collection vehicle in advance, and following the vehicle by a trained field worker and operating it. When looking at the sign, the worker manually enters the type and viewing distance into the software, and after field collection is complete, the indoor worker compares the pre- and post-work base maps to provide useful information to the road network. Update to the database. This process requires the support of a variety of software tools, but the accuracy of data outcomes is determined primarily by the individual ability and concentration of the operator, while at the same time the complex collection process increases the efficiency of updating road network data. Reduce.

  The present application aims to provide a method and apparatus for recognizing traffic signs in order to solve the technical problems described in the background art.

  In one aspect, the present application provides a method for recognizing traffic signs, which is obtained by a predetermined feature algorithm in a predetermined integration channel of a scan window image obtained by dividing by a omnidirectional image. Obtaining a feature value; detecting a scan window image based on the feature value of the scan window image and a pre-trained detection classifier model; obtaining a traffic sign window image to be confirmed; Recognizing a traffic sign window image to be confirmed based on a pretrained convolutional neural network model and acquiring a traffic sign type, wherein the pretrained detection classification The instrument model is obtained by training based on scan window image samples and their feature values. , Neural network model convolution said predetermined training is obtained by training on the basis of the sample and its traffic sign kinds of traffic signs window image was confirmed.

  In another aspect, the present application provides an apparatus for recognizing traffic signs, which is obtained by a predetermined feature algorithm in a predetermined integration channel of a scan window image obtained by segmenting with a spherical image. A traffic sign window for detecting and confirming a scan window image based on a feature value acquisition module for acquiring the acquired feature value, a feature value of the scan window image, and a pre-trained detection classifier model A traffic sign detection module for acquiring an image, a traffic sign recognition module for recognizing a traffic sign window image to be confirmed based on a pretrained convolutional neural network model, and acquiring a traffic sign type Where the pre-trained detection classifier model is scanned Indou obtained by training on the basis of the sample and its feature value of the image, the neural network model convolution pre training is obtained by training on the basis of the sample and its traffic sign kinds of traffic signs window image was confirmed.

  A method and apparatus for recognizing a traffic sign according to the present application obtains a feature value obtained by a predetermined feature algorithm in a predetermined integration channel of a scan window image obtained by dividing a spherical image, and then continues. Based on the feature value of the scan window image and a pre-trained detection classifier model, the scan window image is detected to obtain a traffic sign window image to be confirmed, and then pre-trained convolution Based on the neural network model, by recognizing the traffic sign window image to be confirmed and obtaining the traffic sign type, the traffic sign window image to be confirmed is detected by the pre-trained detection classifier model. Traffic sign by pre-trained convolutional neural network model It realized to recognize the type, improves the accuracy of recognizing and detecting the traffic sign in a panorama image, to improve the update efficiency of road network data.

Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the drawings.
4 is an exemplary flowchart illustrating a method for recognizing a traffic sign according to an embodiment of the present application. 4 is an exemplary flowchart illustrating a method for training a pre-trained detection model according to an embodiment of the present application. 6 is an exemplary flowchart illustrating a method for obtaining a traffic sign window image to be confirmed according to an embodiment of the present application. 4 is an exemplary flowchart illustrating a method for training a pretrained convolutional neural network model according to an embodiment of the present application. FIG. 3 is an exemplary structural diagram illustrating a predetermined convolutional neural network model according to an embodiment of the present application. It is a structure schematic diagram which shows the apparatus which recognizes the traffic sign by the Example of this application. FIG. 2 is an exemplary structural diagram illustrating a detection classifier model training device according to an embodiment of the present application; FIG. 2 is an exemplary structural diagram showing an apparatus for acquiring a traffic sign window image to be confirmed. FIG. 3 is an exemplary structural diagram illustrating an apparatus for training a pretrained convolutional neural network model according to an embodiment of the present application. 1 is a structural schematic diagram showing a computer system according to an embodiment of the present application.

  Hereinafter, the present invention will be described in more detail with reference to the drawings and examples. However, it should be understood that the specific embodiments described herein are merely for interpreting the invention and do not limit the scope of the invention. For convenience of explanation, only the parts related to the present invention are shown in the drawings.

  However, as long as there is no collision, the embodiments in the present application and the features in the embodiments may be combined with each other. Hereinafter, the present invention will be described in detail based on examples with reference to the drawings.

  FIG. 1 is an exemplary flowchart illustrating a method for recognizing traffic signs according to an embodiment of the present application.

  As shown in FIG. 1, the method 100 for recognizing traffic signs may include the following steps.

  Step 110: Obtain a feature value obtained by a predetermined feature algorithm in a predetermined integration channel of a scan window image obtained by dividing the omnidirectional image.

  Here, first, an omnidirectional image that needs to recognize a traffic sign is acquired, then, the omnidirectional image is scanned by a window of a predetermined size, thereby acquiring a scan window image, and then further A feature value obtained by a predetermined feature algorithm in a predetermined integration channel of the scan window image is acquired.

  The omnidirectional image is an image that can display a panorama formed by joining a plurality of fisheye images. The window of a predetermined size is generally a predetermined window based on the size of the target object.

  It should be noted that when acquiring a feature value obtained by a predetermined feature algorithm in a predetermined integration channel of a scan window image, the predetermined integration channel is any one type or various types of integration channels in the prior art in the image processing field. There may be any one or many integration channels that may be developed in future technology, and the predetermined feature algorithm is any one or many in the prior art in the image processing field. It may be a feature algorithm or any one or many feature algorithms that may develop in future technologies. In this application, it is not restricted to the system which acquires a specific feature value, It can select according to an actual use demand by a user. For example, a haar feature value can be obtained by a Haar algorithm for a grayscale image channel, and a histogram feature value can be obtained by a histogram feature value algorithm for a gradient image channel with different angular parameters, and at the same time, red, green, blue Random pair feature values and the like are acquired by a random pair feature value algorithm in the monochrome image channel.

  In order to further improve the speed of acquiring the feature value of the scan window image, the feature value of the scan window image can be acquired by the integral image. Optionally, in the above embodiment, step 110 is the omnidirectional image. Step 111 for acquiring an integrated image of the scan window image obtained by dividing the integrated image of the predetermined integration channel by step 111, and step 112 for acquiring the feature value of the scan window image based on the integrated image of the scan window image. May be included.

  By calculating the feature value of the scan window image using the integral image of the omnidirectional image described above, the calculation of the feature value of the scan window image is accelerated and the calculation efficiency of the feature value of the scan window image is improved. be able to.

  Optionally, depending on the type of traffic sign, depending on the type of traffic sign window image, in order to accelerate the comparison between the feature value of the scan window image and the feature value of the traffic sign window image sample in the pre-trained surveillance classifier model. A predetermined integration channel and a predetermined feature algorithm for acquiring the sample feature value and the scan window image feature value can be determined. The description in the red channel can be strengthened for forbidden indicators such as speed limits, for example. Preferably, the predetermined integration channel may include one or more terms such as a gray scale image channel, a red, green, blue monochrome image channel, a gradient image channel with different angular parameters, and an edge detection image channel, and The predetermined feature algorithm may include one or more terms such as a Haar feature algorithm, a local binary pattern feature algorithm, a histogram feature algorithm, and a random pair feature algorithm. It should be understood by those skilled in the art that the predetermined integration channel and the predetermined feature algorithm in the above embodiment may be combined as necessary to obtain the characteristic values necessary to detect the scan window image. .

  According to the predetermined integration channel and the predetermined feature algorithm for acquiring the characteristic value of the sample of the traffic sign window image and the characteristic value of the scan window image determined by the type of the traffic sign, the characteristic of the object from a plurality of different angles Can be explained, so that the object absorbs changes due to different angles and illumination.

  Step 120: Based on the feature value of the scan window image and the pre-trained detection classifier model, the scan window image is detected and the traffic sign window image to be confirmed is obtained, where the pre-trained The detection classifier model is obtained by training based on a sample of the scan window image and its feature value.

  Here, the pre-trained detection classifier model first receives the manual marking and determines the scan window image including the traffic sign and the scan window image not including the traffic sign as the sample of the traffic sign window image in the scan window image. Then, after obtaining the sample feature value, the sample and its feature value are used to train the classifier model parameters set according to actual demand, thereby pre-trained detection classification Get a vessel model.

  After the feature value of the scan window image is acquired in the above step 101, the scan window image is detected based on the feature value of the scan window image and the detection trainer model trained in advance, and the window that seems to be a traffic sign It can be a traffic sign window image to be acquired and confirmed.

  Step S130: Based on the pretrained convolutional neural network model, the traffic sign window image to be confirmed is recognized and the traffic sign type is obtained, where the pretrained convolutional neural network model is confirmed. It is obtained by training based on a sample of a traffic sign window image and its traffic sign type.

  Here, the pre-trained convolutional neural network model first receives the manual marking, classifies the sample of the traffic sign window image to be confirmed detected by the detection classifier model, and the specific traffic sign type Alternatively, a convolutional neural network model set according to actual demand is trained according to a sample of a traffic sign window image to be marked and then to be confirmed and its traffic sign type, and then trained in advance. Get a convolutional neural network model.

  After the traffic sign window image to be confirmed in step 102 is acquired, the traffic sign window image to be confirmed is determined based on the traffic sign window image to be confirmed and the pretrained convolutional neural network model. The traffic sign type with the highest probability corresponding to the traffic sign window image to be confirmed is set as the recognized traffic sign type.

  According to the method for recognizing a traffic sign in the above embodiment of the present application, the accuracy of detecting and recognizing the traffic sign in the panoramic image is improved and the efficiency of updating the road network data is improved.

  Hereinafter, a method of training a detection model trained in advance will be described with reference to FIG.

FIG. 2 is an exemplary flowchart illustrating a method for training a pre-trained detection model according to an embodiment of the present application.

  As shown in FIG. 2, a method 200 for training a pre-trained detection classifier model may include the following steps.

  In step 201, a positive sample and a negative sample in a sample of the scan window image are obtained, wherein the positive sample includes a window image of a traffic sign or a window image in which a predetermined pixel is expanded around the traffic sign and its surroundings. Including negative samples include scan window images excluding positive samples.

  Here, for the positive sample and the negative sample in the sample of the scan window image, first, the sample is obtained from the scan window image, and then the window image including the traffic sign in the sample of the scan window image according to the received manual marking or A window image in which a traffic sign and surrounding pixels are expanded is defined as a positive sample, and a scan window image obtained by excluding positive samples in the sample is defined as a negative sample.

  When the positive sample includes only a window image of a traffic sign, the efficiency of calculating the characteristic value of the positive sample can be improved. Considering that the pixels around the target object can also effectively describe the target object itself, the positive sample when the positive sample includes a traffic sign and a window image that extends a predetermined pixel around it It is possible to improve the accuracy of describing the feature value of.

  In step 202, feature values obtained by a predetermined feature algorithm for a positive sample and a negative sample in a predetermined integration channel are obtained.

  Here, when acquiring the characteristic values obtained by the predetermined feature algorithm in the predetermined integration channel of the positive sample and the negative sample, the predetermined integration channel is any one or various types in the conventional technology in the image processing field. It may be an integration channel, or any one or many types of integration channels that may develop in future technology, and the predetermined feature algorithm is any one in the prior art in the image processing field. Alternatively, it may be a variety of feature algorithms, or any one or a variety of feature algorithms that may develop in future technologies. In the present application, the present invention is not limited to a predetermined integration channel and a predetermined feature algorithm for obtaining a specific feature value, and can be selected according to an actual use demand by a user. For example, haar feature values obtained by the Haar algorithm for grayscale image channels, histogram feature values obtained by the histogram feature value algorithm for gradient image channels with different angular parameters, and random for red, green and blue monochrome image channels Random pair feature values obtained by the pair feature value algorithm.

  Optionally, when specifically calculating sample feature values, in order to quickly and effectively obtain the feature values obtained by a given feature algorithm in a given integration channel for positive and negative samples, Depending on the type of traffic sign, a predetermined integration channel and a predetermined feature algorithm of a sample of the traffic sign window image can be determined. The description for the red channel can be strengthened for prohibition indicators such as speed limits. Preferably, the predetermined integration channel may include one or more of a grayscale image channel, a red, green, blue monochrome image channel, a gradient image channel with different angular parameters, an edge detection image channel, etc. The predetermined feature algorithm may include one or more of a Haar feature algorithm, a local binary pattern feature algorithm, a histogram feature algorithm, a random pair feature algorithm, and the like. The predetermined integration channel and the predetermined feature algorithm in the above embodiment may be combined as necessary so as to obtain the characteristic values obtained by the predetermined feature algorithm in the predetermined integration channel of the positive sample and the negative sample. It should be appreciated by those skilled in the art.

  In step 203, training the detection classifier model with a boosting algorithm based on the positive and negative samples and the feature values obtained by the predetermined feature algorithm in a predetermined integration channel of the positive and negative samples; Obtain a pre-trained detection classifier model.

  Here, after obtaining the feature values obtained by the predetermined feature algorithm in the predetermined integration channel of the positive sample and the negative sample in the above step 202, the detection classifier model can be trained by the boosting algorithm, thereby A high accuracy result classifier is obtained as a pretrained detection classifier model.

  Hereinafter, a method for acquiring a traffic sign window image to be confirmed based on the above embodiment will be described with reference to FIG.

  FIG. 3 is an exemplary flowchart illustrating a method for obtaining a traffic sign window image to be confirmed according to an embodiment of the present application.

  As shown in FIG. 3, a method 300 for obtaining a traffic sign window image to be confirmed includes the following steps.

  Step 301: Continuously downsample the omnidirectional image to obtain an image pyramid.

  Here, pyramidal transformation, for example, Laplace pyramid transformation, can be performed on the omnidirectional image, and the omnidirectional image is converted from the original size to a different size, thereby obtaining an image pyramid including a multistage image. .

  Step 302: Acquire an integral image of a scan window image of each step image in the image pyramid obtained by dividing the integral image of a predetermined integration channel by the image pyramid.

  After the image pyramid is acquired in step 301, each stage image of the image pyramid can be first integrated in a predetermined integration channel to obtain an integrated image of the image pyramid, and then the image pyramid can be obtained by a window of a predetermined size. Are integrated to obtain an integrated image of the scan window image of each stage image.

  Step 303: Based on the integrated image of the scan window image of each stage image, the feature value of the scan window image of each stage image is acquired.

  After the integral image of the scan window image of each stage image is acquired in the above step 302, calculation is performed on the integral image of the scan window image of each stage image based on the integral image of the scan window image of each stage image. The feature value of the scan window image of the image can be acquired quickly.

Step 304: Based on the feature value of the scan window image of each stage image and the pre-trained detection classifier model, the scan window image of each stage image is detected to obtain the traffic sign window image to be confirmed. .

  After the feature value of the scan window image of each step image is acquired in the above step 303, the scan window image of each step image is detected by a pre-trained detection classifier model based on the feature value of the scan window image of each step image. , And a traffic sign window image to be confirmed by acquiring a window image that seems to be a traffic sign therein, thereby obtaining all the traffic sign window images to be confirmed in the image pyramid.

  According to the method of acquiring the traffic sign window image to be confirmed in the above embodiment of the present application, the scan window image is acquired from the pyramid image obtained by the pyramid transformation in the omnidirectional image, and then the user wants to confirm the scan window image. The traffic sign to be confirmed is detected in order to reduce the possibility of missing the traffic sign window to be confirmed in the omnidirectional image by detecting the scan window image of each step image so as to obtain the traffic sign window image. Improve the accuracy of getting the window. Further, since the feature value of the scan window image of each stage image is obtained by using the integral image, the feature value acquisition speed is improved.

  Hereinafter, a method for training a pretrained convolutional neural network model will be described with reference to FIG.

  FIG. 4 is an exemplary flowchart illustrating a method for training a pretrained convolutional neural network model according to an embodiment of the present application.

  As shown in FIG. 4, a method 400 for training a pre-trained convolutional neural network model includes the following steps.

  Step 401: Initialize the weights of both the convolutional layer and the fully connected layer of a predetermined convolutional neural network model according to a Gaussian distribution, and the predetermined convolutional neural network model is connected to a convolutional layer, an extraction layer, a fully connected layer, and Includes a normalization layer.

  Here, the convolution layer, the extraction layer, the total coupling layer, and the normalization layer connected in sequence may include one convolution layer, one extraction layer, one full coupling layer, and one normalization layer. And a convolution layer and an extraction layer corresponding to the convolution layer, one or more full coupling layers, and a normalization layer.

  Here, the installed convolution layer can reinforce the original signal characteristics and reduce noise by the convolution operation, and the installed extraction layer uses the principle of image local correlation to Can be subsampled, reducing data throughput and holding useful information.

  Hereinafter, a predetermined convolution neural network model will be described with reference to FIG.

  FIG. 5 is an exemplary structural diagram illustrating a predetermined convolutional neural network model according to an embodiment of the present application.

As shown in FIG. 5, a predetermined convolutional neural network model 500 includes a convolutional layer conv1, an extraction layer pool1, a convolutional layer conv2, an extraction layer pool2, a convolutional layer conv3, an extraction layer pool3, an all coupling layer fc1, which are sequentially connected. The entire coupling layer fc2 is included.

  Here, conv1 has 16 convolution kernels of size 5 * 5 * 3, conv2 has 32 convolution kernels of size 5 * 5 * 16, and conv3 has 64 sizes of 5 * Has a 5 * 32 convolution kernel. The two fully connected layers have 512 and 120 neurons, respectively.

  Returning to FIG. 4, in step 402, the weights of both the convolutional layer and the fully connected layer are iterated by the backpropagation BP algorithm based on the confirmed sample traffic sign window image and its traffic sign type.

  Here, the sample of the confirmed traffic sign window image is a sample of the window image including the traffic sign after confirmation based on the received manual marking in the sample extracted from the traffic sign window image to be confirmed. It is.

  Further, the confirmed traffic sign window image sample is an original sample in which the traffic sign window image is confirmed, and one or more of rotation, translation, and scaling with respect to the original sample in which the traffic sign window image is confirmed. An image that is normalized to a predetermined window size after the processing of the term is performed may be included. By expanding the sample, the accuracy rate of sample recognition can be improved.

  Thereafter, a convolutional neural network model having an optimum weight can be determined in step S403 or step S404.

  Step S403: If the difference between the currently repeated weight and the previously repeated weight is smaller than a predetermined value, the currently repeated weight is determined as the optimum weight.

  Step S404: If the repeated weight with the lowest error rate appears, determine the repeated weight with the lowest error rate as the optimum weight.

  Step S405: The convolutional neural network model including the optimum weight is set as a pretrained convolutional neural network model.

  After the pretrained convolutional neural network model is trained, in step 103 in FIG. 1, the traffic sign window image to be confirmed is input to the convolutional neural network model, and the weight output by the normalization layer is A step of obtaining the maximum traffic sign type and a step of obtaining the traffic sign type obtained by recognizing the traffic sign type having the maximum weight may be included.

  FIG. 6 is a structural schematic diagram showing an apparatus for recognizing a traffic sign according to an embodiment of the present application.

  As shown in FIG. 6, the traffic sign recognition apparatus 600 may include a feature value acquisition module 610, a traffic sign detection module 620, and a traffic sign recognition module 630.

  The feature value acquisition module 610 is used to acquire a feature value obtained by a predetermined feature algorithm in a predetermined integration channel of a scan window image obtained by dividing the omnidirectional image.

  Optionally, the feature value acquisition module 610 includes a first integrated image acquisition sub-module 611 for acquiring an integrated image of a scan window image obtained by dividing the whole celestial sphere image by an integrated image of a predetermined integration channel. And a first feature value acquisition sub-module 612 for acquiring a feature value of the scan window image based on the integral image of the scan window image.

  The traffic sign detection module 620 is used to detect a scan window image based on a feature value of the scan window image and a pre-trained detection classifier model and obtain a traffic sign window image to be confirmed. Here, the pre-trained detection classifier model is obtained by training based on the scan window image sample and its feature value.

  The traffic sign recognition module 630 is used for recognizing a traffic sign window image to be confirmed based on a pretrained convolutional neural network model and acquiring a traffic sign type, where the trained information is pretrained. A convolutional neural network model is obtained by training based on a sample of a confirmed traffic sign window image and its traffic sign type.

  According to the apparatus for recognizing a traffic sign of the above embodiment of the present application, the accuracy of detecting and recognizing the traffic sign in the panoramic image is improved and the updating efficiency of the road network data is improved.

  Alternatively, the pre-trained detection classifier model described above may be obtained by the following method: obtaining positive and negative samples in the sample of the scan window image, where the positive sample is a traffic sign Or a window image in which predetermined pixels are expanded around the traffic sign and its surroundings, the negative sample includes a scan window image that excludes positive samples, and a predetermined integration channel of positive samples and negative samples. And obtaining boost values based on the positive and negative samples and the positive and negative sample feature values obtained by the predetermined feature algorithm in the predetermined integration channel. Depending on the algorithm, the detection classifier module To train Le, obtains a detection classifier models previously trained.

  The above-described method for acquiring a pre-trained detection classifier model can be realized by a detection classifier model training apparatus as shown in FIG.

  FIG. 7 shows an exemplary structural diagram of a detection classifier model training apparatus according to an embodiment of the present application.

  As shown in FIG. 7, the detection classifier model training apparatus 700 may include a sample acquisition module 701, a sample feature value acquisition module 702, and a detection model training module 703.

  The sample acquisition module 701 is used to acquire a positive sample and a negative sample in a sample of a scan window image, where the positive sample includes a window image of a traffic sign or a predetermined pixel around the traffic sign and its surroundings. A negative sample includes a scan window image that includes an expanded window image and excludes positive samples.

  The sample feature value acquisition module 702 is used to acquire feature values obtained by a predetermined feature algorithm in a predetermined integration channel of positive samples and negative samples.

  The predetermined integration channel includes one or more of a gray scale image channel, a red, green, blue monochrome image channel, a gradient image channel with different angular parameters, and an edge detection image channel, and the predetermined feature algorithm is One or more of a Haar feature algorithm, a local binary pattern feature algorithm, a histogram feature algorithm, and a random pair feature algorithm.

  The detection model training module 703 trains the detection classifier model by the boosting algorithm based on the positive and negative samples and the feature values obtained by the predetermined feature algorithm in the predetermined integration channel of the positive and negative samples. And used to obtain a pre-trained detection classifier model.

  Hereinafter, an apparatus for acquiring a traffic sign window image to be confirmed will be described with reference to FIG.

  FIG. 8 shows an exemplary structural diagram of an apparatus for obtaining a traffic sign window image to be confirmed.

  As shown in FIG. 8, an apparatus 800 for acquiring a traffic sign window image to be confirmed includes a downsampling sub-module 801, a second integral image acquisition sub-module 802, a second feature value acquisition sub-module 803, and multistage detection. A submodule 804 may be included.

  The downsampling submodule 801 is used to continuously downsample the omnidirectional image to obtain an image pyramid.

  The second integrated image acquisition submodule 802 is used to acquire an integrated image of the scan window image of each stage image in the image pyramid obtained by dividing the integrated image of the predetermined integration channel by the image pyramid.

  The second feature value acquisition submodule 803 is used to acquire the feature value of the scan window image of each stage image based on the integral image of the scan window image of each stage image.

  The multi-stage detection sub-module 804 detects the scan window image of each stage image based on the feature value of the scan window image of each stage image and the pre-trained detection classifier model and confirms the traffic sign window to be confirmed. Used to acquire images.

According to the apparatus for acquiring the traffic sign window image to be confirmed described above in the embodiment of the present application, the accuracy rate for acquiring the traffic sign window image to be further confirmed is improved and the traffic sign window image to be confirmed is confirmed. Can be detected at a higher speed.

  Alternatively, the pretrained convolutional neural network model in FIG. 6 may be obtained by the following method, and the weights of both the convolutional layer and the fully connected layer of a given convolutional neural network model according to a Gaussian distribution. Where the predetermined convolutional neural network model comprises a convolutional layer, an extraction layer, a fully connected layer and a normalization layer connected in sequence, a sample of a confirmed traffic sign window image and its traffic sign type When the weights of both the convolutional layer and the total coupling layer are repeated by the error backpropagation BP algorithm, and the difference between the currently repeated weight and the previously repeated weight is smaller than a predetermined value, The currently repeated weight is determined as the optimal weight, and the convolutional neural network including the optimal weight is determined. The work model is the pretrained convolutional neural network model, or when the repeated weight with the lowest error rate appears, the lowest repeated weight with the error rate is determined as the optimal weight, and the optimal weight Is a pre-trained convolutional neural network model.

  The confirmed traffic sign window image sample is a sample of the window image including the traffic sign after confirmation based on the received manual marking in the sample extracted from the traffic sign window image to be confirmed.

  Further, the confirmed traffic sign window image sample is an original sample in which the traffic sign window image is confirmed, and one or more of rotation, translation, and scaling with respect to the original sample in which the traffic sign window image is confirmed. An image that is normalized to a predetermined window size after the processing of the term is performed may be included.

  Here, the convolution layer, the extraction layer, the total coupling layer, and the normalization layer that are sequentially connected are a plurality of convolution layers and an extraction layer corresponding to the convolution layer, one or more total coupling layers, and one normalization layer. Is provided.

  Furthermore, the predetermined convolution neural network model may further include a loss function layer.

  In the step of further including a loss function layer corresponding to a predetermined convolutional neural network model and repeating the weights of both the convolution layer and the all coupling layer by an error back-propagation BP algorithm, a convolution layer is obtained by the loss function and the BP algorithm. And repeating the weights of both the and all connected layers, respectively.

  The above-described method for obtaining the pretrained convolutional neural network model can be realized by the apparatus for training the pretrained convolutional neural network model shown in FIG.

  FIG. 9 is an exemplary structural diagram illustrating an apparatus for training a pretrained convolutional neural network model according to an embodiment of the present application.

  As shown in FIG. 9, an apparatus 900 for training a pre-trained convolutional neural network model comprises the following modules:

  The initialization module 901 is used to initialize the weights of both the convolutional layer and the fully connected layer of a predetermined convolutional neural network model according to a Gaussian distribution, where the predetermined convolutional neural network model is sequentially connected. A convolution layer, an extraction layer, a fully connected layer and a normalization layer.

  The convolutional layer, the extraction layer, the total coupling layer, and the normalization layer that are sequentially connected include a plurality of convolutional layers and an extraction layer corresponding to the convolutional layer, one or more total coupling layers, and one normalization layer.

  Optionally, the predetermined convolutional neural network model further comprises a loss function layer.

  The weight iteration module 902 is used to iterate the weights of both the convolutional layer and the fully connected layer by the backpropagation BP algorithm based on the confirmed traffic sign window image sample and its traffic sign type.

  The first optimum weight determination module 903 determines the currently repeated weight as the optimum weight when the difference between the currently repeated weight and the previously repeated weight is smaller than a predetermined value. Used.

  The second optimum weight determination module 904 is used to determine the lowest repeated weight with the lowest error rate as the optimum weight when the lowest repeated weight with the error rate appears.

  The model setting module 905 is used to make a convolutional neural network model including optimal weights into a pretrained convolutional neural network model.

  A loss function layer is further provided corresponding to a predetermined convolutional neural network model, and the weight determination module 902 is further used to iterate the weights of both the convolution layer and the fully connected layer by the loss function and the BP algorithm, respectively.

  After the pretrained convolutional neural network model is trained by the apparatus 900 for training the pretrained convolutional neural network model, the traffic sign recognition module 630 in FIG. 6 convolves the traffic sign window image to be verified. A maximum weight recognition module (not shown) for obtaining the traffic sign type with the maximum weight, which is input to the neural network model and output by the normalization layer, and recognizes the traffic sign type with the maximum weight You may provide the sign kind setting module (not shown) set as the obtained traffic signs.

  Each unit described in the apparatus 600 corresponds to each step in the method described with reference to FIG. Each unit described in the apparatus 700 corresponds to each step in the method described with reference to FIG. Each unit described in the apparatus 800 corresponds to each step in the method described with reference to FIG. It should be understood that each unit described in apparatus 900 corresponds to a step in the method described with reference to FIG. Thereby, the operations and features described above for the method of recognizing traffic signs apply to the apparatus 600 and the units contained therein as well, and to the method of training the detection model previously trained above. The operations and features described above are similarly applied to the apparatus 700 and the units included therein, and the operations and characteristics described above for the method of obtaining the traffic sign window image to be confirmed are similarly the apparatus 800. And the operations and features described above for the method of training a pretrained convolutional neural network model as applied to the units included therein and the units included therein are similarly applied to the apparatus 900 and the units included therein. It is omitted here. Corresponding units in the devices 600, 700, 800 and 900 may be matched to each other in the terminal equipment and / or server so as to realize the proposed embodiment of the present application.

  The modules described in the embodiments of the present invention may be realized by software or hardware. The described module may be set in the processor, and may be described as, for example, “a processor including a feature value acquisition module, a detection module, and a recognition module”. Among them, the names of these modules do not limit the module itself in some cases. For example, the feature value acquisition module is “a predetermined value of a scan window image obtained by dividing an omnidirectional image”. May be described as a module for obtaining feature values obtained by a predetermined feature algorithm in the integration channel of

  FIG. 10 is a structural schematic diagram showing a computer system according to an embodiment of the present application.

  Reference is made to FIG. 10, which is a structural schematic diagram of a computer system 1000 applied to the device of the embodiment of the present application.

  As shown in FIG. 10, the computer system 1000 performs various appropriate operations based on a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage unit 1008 to a random access memory (RAM) 1003. And a central processing unit (CPU) 1001 capable of executing processing. When a program stored in the ROM 1002 or a program loaded from the storage unit 1008 to the RAM 1003 is executed by the CPU 1001, the CPU 1001 executes the method described in the present application. The RAM 1003 further stores various programs and data necessary for operating the system 1000. The CPU 1001, ROM 1002, and RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

  An input unit 1006 including a keyboard and a mouse, a cathode ray tube (CRT), a liquid crystal display (LCD), and an output unit 1007 including a speaker, a storage unit 1008 including a hard disk, and a network interface including a LAN card and a modem The communication unit 1009 of the card is connected to the I / O interface 1005. The communication unit 1009 executes communication processing via a network such as the Internet. The driver 1010 is connected to the I / O interface 1005 as necessary. The removable medium 1011 is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, and is attached to the driver 1010 as necessary. Therefore, a computer program read from the driver 1010 is necessary. Is installed in the storage unit 1008 accordingly.

  In particular, according to an embodiment of the present invention, the process described with reference to the above flowchart may be implemented as a software program on a computer. For example, embodiments of the present invention include a computer program product, the computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program performing the method shown in the flowchart. Including program code. In such an embodiment, the computer program may be downloaded and installed from the network via the communication unit 1009 and / or installed from the removable medium 1011.

  The flowcharts and block diagrams in the Figures illustrate the architecture, functionality, and operation that can be implemented by systems, methods and computer program products according to embodiments of the present invention. Here, each frame in the flowchart or block diagram may represent one module, program segment, or part of code, and the module, program segment, or part of code may have a defined logical function. Contains one or more executable instructions to accomplish. Also, as some alternative embodiments, the functions shown in the boxes may be performed in a different order than the order shown in the drawings. For example, two frames shown in succession may actually be executed substantially in parallel, or may be executed in reverse order, depending on the function involved. Also, each frame in the block diagram and / or flowchart and the combination of the frame in the block diagram and / or flowchart may be realized by a hardware-based dedicated system that performs a specified function or operation, or It may be executed by a combination of dedicated hardware and computer instructions.

  On the other hand, the present invention further provides a computer-readable storage medium, and the computer-readable storage medium may be a computer-readable storage medium included in the apparatus of the above-described embodiment, and exists independently. Thus, it may be a computer-readable storage medium that is not assembled in the terminal. One or more programs are stored on the computer readable storage medium, and the one or more programs execute the method for recognizing traffic signs described in the present application by one or more processors. It is used.

  The foregoing is merely illustrative of the preferred embodiment of the invention and the technical principles used. The scope of the claims of the present invention is not limited to a technical proposal comprising a specific combination of the above-described technical features, and the above-described technical features or equivalent features without departing from the spirit of the present invention. It should be understood by those skilled in the art that other technical solutions consisting of any combination of the above should also be included. For example, there is a technical proposal in which the above features and technical features (not limited to these) having similar functions disclosed in the present invention are replaced with each other.

Priority request

  This application claims priority, filed on Mar. 31, 2015, with a Chinese patent application number of “201510150525.8” and is incorporated herein by reference in its entirety.

  The present application relates to the field of computer technology, specifically to the field of computer target recognition, and more particularly to a method and apparatus for recognizing traffic signs.

  In order to provide accurate and complete navigation data, it is necessary to recognize traffic signs in the traffic road network.

  The conventional method of recognizing traffic signs mainly depends on artificial processing, placing specific program software and base map data in a collection vehicle in advance, and following the vehicle by a trained field worker and operating it. When looking at the sign, the worker manually enters the type and viewing distance into the software, and after field collection is complete, the indoor worker compares the pre- and post-work base maps to provide useful information to the road network. Update to the database. This process requires the support of a variety of software tools, but the accuracy of data outcomes is determined primarily by the individual ability and concentration of the operator, while at the same time the complex collection process increases the efficiency of updating road network data. Reduce.

  The present application aims to provide a method and apparatus for recognizing traffic signs in order to solve the technical problems described in the background art.

  In one aspect, the present application provides a method for recognizing traffic signs, which is obtained by a predetermined feature algorithm in a predetermined integration channel of a scan window image obtained by dividing by a omnidirectional image. Obtaining a feature value; detecting a scan window image based on the feature value of the scan window image and a pre-trained detection classifier model; obtaining a traffic sign window image to be confirmed; Recognizing a traffic sign window image to be confirmed based on a pretrained convolutional neural network model and acquiring a traffic sign type, wherein the pretrained detection classification The instrument model is obtained by training based on scan window image samples and their feature values. , Neural network model convolution said predetermined training is obtained by training on the basis of the sample and its traffic sign kinds of traffic signs window image was confirmed.

  In another aspect, the present application provides an apparatus for recognizing traffic signs, which is obtained by a predetermined feature algorithm in a predetermined integration channel of a scan window image obtained by segmenting with a spherical image. A traffic sign window for detecting and confirming a scan window image based on a feature value acquisition module for acquiring the acquired feature value, a feature value of the scan window image, and a pre-trained detection classifier model A traffic sign detection module for acquiring an image, a traffic sign recognition module for recognizing a traffic sign window image to be confirmed based on a pretrained convolutional neural network model, and acquiring a traffic sign type Where the pre-trained detection classifier model is scanned Indou obtained by training on the basis of the sample and its feature value of the image, the neural network model convolution pre training is obtained by training on the basis of the sample and its traffic sign kinds of traffic signs window image was confirmed.

  A method and apparatus for recognizing a traffic sign according to the present application obtains a feature value obtained by a predetermined feature algorithm in a predetermined integration channel of a scan window image obtained by dividing a spherical image, and then continues. Based on the feature value of the scan window image and a pre-trained detection classifier model, the scan window image is detected to obtain a traffic sign window image to be confirmed, and then pre-trained convolution Based on the neural network model, by recognizing the traffic sign window image to be confirmed and obtaining the traffic sign type, the traffic sign window image to be confirmed is detected by the pre-trained detection classifier model. Traffic sign by pre-trained convolutional neural network model It realized to recognize the type, improves the accuracy of recognizing and detecting the traffic sign in a panorama image, to improve the update efficiency of road network data.

Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the drawings.
4 is an exemplary flowchart illustrating a method for recognizing a traffic sign according to an embodiment of the present application. 4 is an exemplary flowchart illustrating a method for training a pre-trained detection model according to an embodiment of the present application. 6 is an exemplary flowchart illustrating a method for obtaining a traffic sign window image to be confirmed according to an embodiment of the present application. 4 is an exemplary flowchart illustrating a method for training a pretrained convolutional neural network model according to an embodiment of the present application. FIG. 3 is an exemplary structural diagram illustrating a predetermined convolutional neural network model according to an embodiment of the present application. It is a structure schematic diagram which shows the apparatus which recognizes the traffic sign by the Example of this application. FIG. 2 is an exemplary structural diagram illustrating a detection classifier model training device according to an embodiment of the present application; FIG. 2 is an exemplary structural diagram showing an apparatus for acquiring a traffic sign window image to be confirmed. FIG. 3 is an exemplary structural diagram illustrating an apparatus for training a pretrained convolutional neural network model according to an embodiment of the present application. 1 is a structural schematic diagram showing a computer system according to an embodiment of the present application.

  Hereinafter, the present invention will be described in more detail with reference to the drawings and examples. However, it should be understood that the specific embodiments described herein are merely for interpreting the invention and do not limit the scope of the invention. For convenience of explanation, only the parts related to the present invention are shown in the drawings.

  However, as long as there is no collision, the embodiments in the present application and the features in the embodiments may be combined with each other. Hereinafter, the present invention will be described in detail based on examples with reference to the drawings.

  FIG. 1 is an exemplary flowchart illustrating a method for recognizing traffic signs according to an embodiment of the present application.

  As shown in FIG. 1, the method 100 for recognizing traffic signs may include the following steps.

  Step 110: Obtain a feature value obtained by a predetermined feature algorithm in a predetermined integration channel of a scan window image obtained by dividing the omnidirectional image.

  Here, first, an omnidirectional image that needs to recognize a traffic sign is acquired, then, the omnidirectional image is scanned by a window of a predetermined size, thereby acquiring a scan window image, and then further A feature value obtained by a predetermined feature algorithm in a predetermined integration channel of the scan window image is acquired.

  The omnidirectional image is an image that can display a panorama formed by joining a plurality of fisheye images. The window of a predetermined size is generally a predetermined window based on the size of the target object.

  It should be noted that when acquiring a feature value obtained by a predetermined feature algorithm in a predetermined integration channel of a scan window image, the predetermined integration channel is any one type or various types of integration channels in the prior art in the image processing field. There may be any one or many integration channels that may be developed in future technology, and the predetermined feature algorithm is any one or many in the prior art in the image processing field. It may be a feature algorithm or any one or many feature algorithms that may develop in future technologies. In this application, it is not restricted to the system which acquires a specific feature value, It can select according to an actual use demand by a user. For example, a haar feature value can be obtained by a Haar algorithm for a grayscale image channel, and a histogram feature value can be obtained by a histogram feature value algorithm for a gradient image channel with different angular parameters, and at the same time, red, green, blue Random pair feature values and the like are acquired by a random pair feature value algorithm in the monochrome image channel.

  In order to further improve the speed of acquiring the feature value of the scan window image, the feature value of the scan window image can be acquired by the integral image. Optionally, in the above embodiment, step 110 is the omnidirectional image. Step 111 for acquiring an integrated image of the scan window image obtained by dividing the integrated image of the predetermined integration channel by step 111, and step 112 for acquiring the feature value of the scan window image based on the integrated image of the scan window image. May be included.

  By calculating the feature value of the scan window image using the integral image of the omnidirectional image described above, the calculation of the feature value of the scan window image is accelerated and the calculation efficiency of the feature value of the scan window image is improved. be able to.

  Optionally, depending on the type of traffic sign, depending on the type of traffic sign window image, in order to accelerate the comparison between the feature value of the scan window image and the feature value of the traffic sign window image sample in the pre-trained surveillance classifier model. A predetermined integration channel and a predetermined feature algorithm for acquiring the sample feature value and the scan window image feature value can be determined. The description in the red channel can be strengthened for forbidden indicators such as speed limits, for example. Preferably, the predetermined integration channel may include one or more terms such as a gray scale image channel, a red, green, blue monochrome image channel, a gradient image channel with different angular parameters, and an edge detection image channel, and The predetermined feature algorithm may include one or more terms such as a Haar feature algorithm, a local binary pattern feature algorithm, a histogram feature algorithm, and a random pair feature algorithm. It should be understood by those skilled in the art that the predetermined integration channel and the predetermined feature algorithm in the above embodiment may be combined as necessary to obtain the characteristic values necessary to detect the scan window image. .

  According to the predetermined integration channel and the predetermined feature algorithm for acquiring the characteristic value of the sample of the traffic sign window image and the characteristic value of the scan window image determined by the type of the traffic sign, the characteristic of the object from a plurality of different angles Can be explained, so that the object absorbs changes due to different angles and illumination.

  Step 120: Based on the feature value of the scan window image and the pre-trained detection classifier model, the scan window image is detected and the traffic sign window image to be confirmed is obtained, where the pre-trained The detection classifier model is obtained by training based on a sample of the scan window image and its feature value.

  Here, the pre-trained detection classifier model first receives the manual marking and determines the scan window image including the traffic sign and the scan window image not including the traffic sign as the sample of the traffic sign window image in the scan window image. Then, after obtaining the sample feature value, the sample and its feature value are used to train the classifier model parameters set according to actual demand, thereby pre-trained detection classification Get a vessel model.

After the feature value of the scan window image is acquired in the step 110 , the scan window image is detected based on the feature value of the scan window image and the detection trainer model trained in advance, and the window that seems to be a traffic sign It can be a traffic sign window image to be acquired and confirmed.

  Step S130: Based on the pretrained convolutional neural network model, the traffic sign window image to be confirmed is recognized and the traffic sign type is obtained, where the pretrained convolutional neural network model is confirmed. It is obtained by training based on a sample of a traffic sign window image and its traffic sign type.

  Here, the pre-trained convolutional neural network model first receives the manual marking, classifies the sample of the traffic sign window image to be confirmed detected by the detection classifier model, and the specific traffic sign type Alternatively, a convolutional neural network model set according to actual demand is trained according to a sample of a traffic sign window image to be marked and then to be confirmed and its traffic sign type, and then trained in advance. Get a convolutional neural network model.

After the traffic sign window image to be confirmed in step 120 is acquired, the traffic sign window image to be confirmed is determined based on the traffic sign window image to be confirmed and the pretrained convolutional neural network model. The traffic sign type with the highest probability corresponding to the traffic sign window image to be confirmed is set as the recognized traffic sign type.

  According to the method for recognizing a traffic sign in the above embodiment of the present application, the accuracy of detecting and recognizing the traffic sign in the panoramic image is improved and the efficiency of updating the road network data is improved.

  Hereinafter, a method of training a detection model trained in advance will be described with reference to FIG.

  FIG. 2 is an exemplary flowchart illustrating a method for training a pre-trained detection model according to an embodiment of the present application.

  As shown in FIG. 2, a method 200 for training a pre-trained detection classifier model may include the following steps.

  In step 201, a positive sample and a negative sample in a sample of the scan window image are obtained, wherein the positive sample includes a window image of a traffic sign or a window image in which a predetermined pixel is expanded around the traffic sign and its surroundings. Including negative samples include scan window images excluding positive samples.

  Here, for the positive sample and the negative sample in the sample of the scan window image, first, the sample is obtained from the scan window image, and then the window image including the traffic sign in the sample of the scan window image according to the received manual marking or A window image in which a traffic sign and surrounding pixels are expanded is defined as a positive sample, and a scan window image obtained by excluding positive samples in the sample is defined as a negative sample.

  When the positive sample includes only a window image of a traffic sign, the efficiency of calculating the characteristic value of the positive sample can be improved. Considering that the pixels around the target object can also effectively describe the target object itself, the positive sample when the positive sample includes a traffic sign and a window image that extends a predetermined pixel around it It is possible to improve the accuracy of describing the feature value of.

  In step 202, feature values obtained by a predetermined feature algorithm for a positive sample and a negative sample in a predetermined integration channel are obtained.

  Here, when acquiring the characteristic values obtained by the predetermined feature algorithm in the predetermined integration channel of the positive sample and the negative sample, the predetermined integration channel is any one or various types in the conventional technology in the image processing field. It may be an integration channel, or any one or many types of integration channels that may develop in future technology, and the predetermined feature algorithm is any one in the prior art in the image processing field. Alternatively, it may be a variety of feature algorithms, or any one or a variety of feature algorithms that may develop in future technologies. In the present application, the present invention is not limited to a predetermined integration channel and a predetermined feature algorithm for obtaining a specific feature value, and can be selected according to an actual use demand by a user. For example, haar feature values obtained by the Haar algorithm for grayscale image channels, histogram feature values obtained by the histogram feature value algorithm for gradient image channels with different angular parameters, and random for red, green and blue monochrome image channels Random pair feature values obtained by the pair feature value algorithm.

  Optionally, when specifically calculating sample feature values, in order to quickly and effectively obtain the feature values obtained by a given feature algorithm in a given integration channel for positive and negative samples, Depending on the type of traffic sign, a predetermined integration channel and a predetermined feature algorithm of a sample of the traffic sign window image can be determined. The description for the red channel can be strengthened for prohibition indicators such as speed limits. Preferably, the predetermined integration channel may include one or more of a grayscale image channel, a red, green, blue monochrome image channel, a gradient image channel with different angular parameters, an edge detection image channel, etc. The predetermined feature algorithm may include one or more of a Haar feature algorithm, a local binary pattern feature algorithm, a histogram feature algorithm, a random pair feature algorithm, and the like. The predetermined integration channel and the predetermined feature algorithm in the above embodiment may be combined as necessary so as to obtain the characteristic values obtained by the predetermined feature algorithm in the predetermined integration channel of the positive sample and the negative sample. It should be appreciated by those skilled in the art.

  In step 203, training the detection classifier model with a boosting algorithm based on the positive and negative samples and the feature values obtained by the predetermined feature algorithm in a predetermined integration channel of the positive and negative samples; Obtain a pre-trained detection classifier model.

  Here, after obtaining the feature values obtained by the predetermined feature algorithm in the predetermined integration channel of the positive sample and the negative sample in the above step 202, the detection classifier model can be trained by the boosting algorithm, thereby A high accuracy result classifier is obtained as a pretrained detection classifier model.

  Hereinafter, a method for acquiring a traffic sign window image to be confirmed based on the above embodiment will be described with reference to FIG.

  FIG. 3 is an exemplary flowchart illustrating a method for obtaining a traffic sign window image to be confirmed according to an embodiment of the present application.

  As shown in FIG. 3, a method 300 for obtaining a traffic sign window image to be confirmed includes the following steps.

  Step 301: Continuously downsample the omnidirectional image to obtain an image pyramid.

  Here, pyramidal transformation, for example, Laplace pyramid transformation, can be performed on the omnidirectional image, and the omnidirectional image is converted from the original size to a different size, thereby obtaining an image pyramid including a multistage image. .

  Step 302: Acquire an integral image of a scan window image of each step image in the image pyramid obtained by dividing the integral image of a predetermined integration channel by the image pyramid.

  After the image pyramid is acquired in step 301, each stage image of the image pyramid can be first integrated in a predetermined integration channel to obtain an integrated image of the image pyramid, and then the image pyramid can be obtained by a window of a predetermined size. Are integrated to obtain an integrated image of the scan window image of each stage image.

  Step 303: Based on the integrated image of the scan window image of each stage image, the feature value of the scan window image of each stage image is acquired.

  After the integral image of the scan window image of each stage image is acquired in the above step 302, calculation is performed on the integral image of the scan window image of each stage image based on the integral image of the scan window image of each stage image. The feature value of the scan window image of the image can be acquired quickly.

  Step 304: Based on the feature value of the scan window image of each stage image and the pre-trained detection classifier model, the scan window image of each stage image is detected to obtain the traffic sign window image to be confirmed. .

  After the feature value of the scan window image of each step image is acquired in the above step 303, the scan window image of each step image is detected by a pre-trained detection classifier model based on the feature value of the scan window image of each step image. , And a traffic sign window image to be confirmed by acquiring a window image that seems to be a traffic sign therein, thereby obtaining all the traffic sign window images to be confirmed in the image pyramid.

  According to the method of acquiring the traffic sign window image to be confirmed in the above embodiment of the present application, the scan window image is acquired from the pyramid image obtained by the pyramid transformation in the omnidirectional image, and then the user wants to confirm the scan window image. The traffic sign to be confirmed is detected in order to reduce the possibility of missing the traffic sign window to be confirmed in the omnidirectional image by detecting the scan window image of each step image so as to obtain the traffic sign window image. Improve the accuracy of getting the window. Further, since the feature value of the scan window image of each stage image is obtained by using the integral image, the feature value acquisition speed is improved.

  Hereinafter, a method for training a pretrained convolutional neural network model will be described with reference to FIG.

  FIG. 4 is an exemplary flowchart illustrating a method for training a pretrained convolutional neural network model according to an embodiment of the present application.

  As shown in FIG. 4, a method 400 for training a pre-trained convolutional neural network model includes the following steps.

  Step 401: Initialize the weights of both the convolutional layer and the fully connected layer of a predetermined convolutional neural network model according to a Gaussian distribution, and the predetermined convolutional neural network model is connected to a convolutional layer, an extraction layer, a fully connected layer, and Includes a normalization layer.

  Here, the convolution layer, the extraction layer, the total coupling layer, and the normalization layer connected in sequence may include one convolution layer, one extraction layer, one full coupling layer, and one normalization layer. And a convolution layer and an extraction layer corresponding to the convolution layer, one or more full coupling layers, and a normalization layer.

  Here, the installed convolution layer can reinforce the original signal characteristics and reduce noise by the convolution operation, and the installed extraction layer uses the principle of image local correlation to Can be subsampled, reducing data throughput and holding useful information.

  Hereinafter, a predetermined convolution neural network model will be described with reference to FIG.

  FIG. 5 is an exemplary structural diagram illustrating a predetermined convolutional neural network model according to an embodiment of the present application.

  As shown in FIG. 5, a predetermined convolutional neural network model 500 includes a convolutional layer conv1, an extraction layer pool1, a convolutional layer conv2, an extraction layer pool2, a convolutional layer conv3, an extraction layer pool3, an all coupling layer fc1, which are sequentially connected. The entire coupling layer fc2 is included.

  Here, conv1 has 16 convolution kernels of size 5 * 5 * 3, conv2 has 32 convolution kernels of size 5 * 5 * 16, and conv3 has 64 sizes of 5 * Has a 5 * 32 convolution kernel. The two fully connected layers have 512 and 120 neurons, respectively.

  Returning to FIG. 4, in step 402, the weights of both the convolutional layer and the fully connected layer are iterated by the backpropagation BP algorithm based on the confirmed sample traffic sign window image and its traffic sign type.

  Here, the sample of the confirmed traffic sign window image is a sample of the window image including the traffic sign after confirmation based on the received manual marking in the sample extracted from the traffic sign window image to be confirmed. It is.

  Further, the confirmed traffic sign window image sample is an original sample in which the traffic sign window image is confirmed, and one or more of rotation, translation, and scaling with respect to the original sample in which the traffic sign window image is confirmed. An image that is normalized to a predetermined window size after the processing of the term is performed may be included. By expanding the sample, the accuracy rate of sample recognition can be improved.

  Thereafter, a convolutional neural network model having an optimum weight can be determined in step S403 or step S404.

  Step S403: If the difference between the currently repeated weight and the previously repeated weight is smaller than a predetermined value, the currently repeated weight is determined as the optimum weight.

  Step S404: If the repeated weight with the lowest error rate appears, determine the repeated weight with the lowest error rate as the optimum weight.

  Step S405: The convolutional neural network model including the optimum weight is set as a pretrained convolutional neural network model.

After the pretrained convolutional neural network model is trained, in step 130 in FIG. 1, the traffic sign window image to be confirmed is input to the convolutional neural network model, and the weight output by the normalization layer is A step of obtaining the maximum traffic sign type and a step of obtaining the traffic sign type obtained by recognizing the traffic sign type having the maximum weight may be included.

  FIG. 6 is a structural schematic diagram showing an apparatus for recognizing a traffic sign according to an embodiment of the present application.

  As shown in FIG. 6, the traffic sign recognition apparatus 600 may include a feature value acquisition module 610, a traffic sign detection module 620, and a traffic sign recognition module 630.

  The feature value acquisition module 610 is used to acquire a feature value obtained by a predetermined feature algorithm in a predetermined integration channel of a scan window image obtained by dividing the omnidirectional image.

  Optionally, the feature value acquisition module 610 includes a first integrated image acquisition sub-module 611 for acquiring an integrated image of a scan window image obtained by dividing the whole celestial sphere image by an integrated image of a predetermined integration channel. And a first feature value acquisition sub-module 612 for acquiring a feature value of the scan window image based on the integral image of the scan window image.

  The traffic sign detection module 620 is used to detect a scan window image based on a feature value of the scan window image and a pre-trained detection classifier model and obtain a traffic sign window image to be confirmed. Here, the pre-trained detection classifier model is obtained by training based on the scan window image sample and its feature value.

  The traffic sign recognition module 630 is used for recognizing a traffic sign window image to be confirmed based on a pretrained convolutional neural network model and acquiring a traffic sign type, where the trained information is pretrained. A convolutional neural network model is obtained by training based on a sample of a confirmed traffic sign window image and its traffic sign type.

  According to the apparatus for recognizing a traffic sign of the above embodiment of the present application, the accuracy of detecting and recognizing the traffic sign in the panoramic image is improved and the updating efficiency of the road network data is improved.

  Alternatively, the pre-trained detection classifier model described above may be obtained by the following method: obtaining positive and negative samples in the sample of the scan window image, where the positive sample is a traffic sign Or a window image in which predetermined pixels are expanded around the traffic sign and its surroundings, the negative sample includes a scan window image that excludes positive samples, and a predetermined integration channel of positive samples and negative samples. And obtaining boost values based on the positive and negative samples and the positive and negative sample feature values obtained by the predetermined feature algorithm in the predetermined integration channel. Depending on the algorithm, the detection classifier module To train Le, obtains a detection classifier models previously trained.

  The above-described method for acquiring a pre-trained detection classifier model can be realized by a detection classifier model training apparatus as shown in FIG.

  FIG. 7 shows an exemplary structural diagram of a detection classifier model training apparatus according to an embodiment of the present application.

  As shown in FIG. 7, the detection classifier model training apparatus 700 may include a sample acquisition module 701, a sample feature value acquisition module 702, and a detection model training module 703.

  The sample acquisition module 701 is used to acquire a positive sample and a negative sample in a sample of a scan window image, where the positive sample includes a window image of a traffic sign or a predetermined pixel around the traffic sign and its surroundings. A negative sample includes a scan window image that includes an expanded window image and excludes positive samples.

  The sample feature value acquisition module 702 is used to acquire feature values obtained by a predetermined feature algorithm in a predetermined integration channel of positive samples and negative samples.

  The predetermined integration channel includes one or more of a gray scale image channel, a red, green, blue monochrome image channel, a gradient image channel with different angular parameters, and an edge detection image channel, and the predetermined feature algorithm is One or more of a Haar feature algorithm, a local binary pattern feature algorithm, a histogram feature algorithm, and a random pair feature algorithm.

  The detection model training module 703 trains the detection classifier model by the boosting algorithm based on the positive and negative samples and the feature values obtained by the predetermined feature algorithm in the predetermined integration channel of the positive and negative samples. And used to obtain a pre-trained detection classifier model.

  Hereinafter, an apparatus for acquiring a traffic sign window image to be confirmed will be described with reference to FIG.

  FIG. 8 shows an exemplary structural diagram of an apparatus for obtaining a traffic sign window image to be confirmed.

  As shown in FIG. 8, an apparatus 800 for acquiring a traffic sign window image to be confirmed includes a downsampling sub-module 801, a second integral image acquisition sub-module 802, a second feature value acquisition sub-module 803, and multistage detection. A submodule 804 may be included.

  The downsampling submodule 801 is used to continuously downsample the omnidirectional image to obtain an image pyramid.

  The second integrated image acquisition submodule 802 is used to acquire an integrated image of the scan window image of each stage image in the image pyramid obtained by dividing the integrated image of the predetermined integration channel by the image pyramid.

  The second feature value acquisition submodule 803 is used to acquire the feature value of the scan window image of each stage image based on the integral image of the scan window image of each stage image.

  The multi-stage detection sub-module 804 detects the scan window image of each stage image based on the feature value of the scan window image of each stage image and the pre-trained detection classifier model and confirms the traffic sign window to be confirmed. Used to acquire images.

  According to the apparatus for acquiring the traffic sign window image to be confirmed described above in the embodiment of the present application, the accuracy rate for acquiring the traffic sign window image to be further confirmed is improved and the traffic sign window image to be confirmed is confirmed. Can be detected at a higher speed.

  Alternatively, the pretrained convolutional neural network model in FIG. 6 may be obtained by the following method, and the weights of both the convolutional layer and the fully connected layer of a given convolutional neural network model according to a Gaussian distribution. Where the predetermined convolutional neural network model comprises a convolutional layer, an extraction layer, a fully connected layer and a normalization layer connected in sequence, a sample of a confirmed traffic sign window image and its traffic sign type When the weights of both the convolutional layer and the total coupling layer are repeated by the error backpropagation BP algorithm, and the difference between the currently repeated weight and the previously repeated weight is smaller than a predetermined value, The currently repeated weight is determined as the optimal weight, and the convolutional neural network including the optimal weight is determined. The work model is the pretrained convolutional neural network model, or when the repeated weight with the lowest error rate appears, the lowest repeated weight with the error rate is determined as the optimal weight, and the optimal weight Is a pre-trained convolutional neural network model.

  The confirmed traffic sign window image sample is a sample of the window image including the traffic sign after confirmation based on the received manual marking in the sample extracted from the traffic sign window image to be confirmed.

  Further, the confirmed traffic sign window image sample is an original sample in which the traffic sign window image is confirmed, and one or more of rotation, translation, and scaling with respect to the original sample in which the traffic sign window image is confirmed. An image that is normalized to a predetermined window size after the processing of the term is performed may be included.

  Here, the convolution layer, the extraction layer, the total coupling layer, and the normalization layer that are sequentially connected are a plurality of convolution layers and an extraction layer corresponding to the convolution layer, one or more total coupling layers, and one normalization layer. Is provided.

  Furthermore, the predetermined convolution neural network model may further include a loss function layer.

  In the step of further including a loss function layer corresponding to a predetermined convolutional neural network model and repeating the weights of both the convolution layer and the all coupling layer by an error back-propagation BP algorithm, a convolution layer is obtained by the loss function and the BP algorithm. And repeating the weights of both the and all connected layers, respectively.

  The above-described method for obtaining the pretrained convolutional neural network model can be realized by the apparatus for training the pretrained convolutional neural network model shown in FIG.

  FIG. 9 is an exemplary structural diagram illustrating an apparatus for training a pretrained convolutional neural network model according to an embodiment of the present application.

  As shown in FIG. 9, an apparatus 900 for training a pre-trained convolutional neural network model comprises the following modules:

  The initialization module 901 is used to initialize the weights of both the convolutional layer and the fully connected layer of a predetermined convolutional neural network model according to a Gaussian distribution, where the predetermined convolutional neural network model is sequentially connected. A convolution layer, an extraction layer, a fully connected layer and a normalization layer.

  The convolutional layer, the extraction layer, the total coupling layer, and the normalization layer that are sequentially connected include a plurality of convolutional layers and an extraction layer corresponding to the convolutional layer, one or more total coupling layers, and one normalization layer.

  Optionally, the predetermined convolutional neural network model further comprises a loss function layer.

  The weight iteration module 902 is used to iterate the weights of both the convolutional layer and the fully connected layer by the backpropagation BP algorithm based on the confirmed traffic sign window image sample and its traffic sign type.

  The first optimum weight determination module 903 determines the currently repeated weight as the optimum weight when the difference between the currently repeated weight and the previously repeated weight is smaller than a predetermined value. Used.

  The second optimum weight determination module 904 is used to determine the lowest repeated weight with the lowest error rate as the optimum weight when the lowest repeated weight with the error rate appears.

  The model setting module 905 is used to make a convolutional neural network model including optimal weights into a pretrained convolutional neural network model.

  A loss function layer is further provided corresponding to a predetermined convolutional neural network model, and the weight determination module 902 is further used to iterate the weights of both the convolution layer and the fully connected layer by the loss function and the BP algorithm, respectively.

  After the pretrained convolutional neural network model is trained by the apparatus 900 for training the pretrained convolutional neural network model, the traffic sign recognition module 630 in FIG. 6 convolves the traffic sign window image to be verified. A maximum weight recognition module (not shown) for obtaining the traffic sign type with the maximum weight, which is input to the neural network model and output by the normalization layer, and recognizes the traffic sign type with the maximum weight You may provide the sign kind setting module (not shown) set as the obtained traffic signs.

  Each unit described in the apparatus 600 corresponds to each step in the method described with reference to FIG. Each unit described in the apparatus 700 corresponds to each step in the method described with reference to FIG. Each unit described in the apparatus 800 corresponds to each step in the method described with reference to FIG. It should be understood that each unit described in apparatus 900 corresponds to a step in the method described with reference to FIG. Thereby, the operations and features described above for the method of recognizing traffic signs apply to the apparatus 600 and the units contained therein as well, and to the method of training the detection model previously trained above. The operations and features described above are similarly applied to the apparatus 700 and the units included therein, and the operations and characteristics described above for the method of obtaining the traffic sign window image to be confirmed are similarly the apparatus 800. And the operations and features described above for the method of training a pretrained convolutional neural network model as applied to the units included therein and the units included therein are similarly applied to the apparatus 900 and the units included therein. It is omitted here. Corresponding units in the devices 600, 700, 800 and 900 may be matched to each other in the terminal equipment and / or server so as to realize the proposed embodiment of the present application.

  The modules described in the embodiments of the present invention may be realized by software or hardware. The described module may be set in the processor, and may be described as, for example, “a processor including a feature value acquisition module, a detection module, and a recognition module”. Among them, the names of these modules do not limit the module itself in some cases. For example, the feature value acquisition module is “a predetermined value of a scan window image obtained by dividing an omnidirectional image”. May be described as “a module for obtaining a feature value obtained by a predetermined feature algorithm”.

  FIG. 10 is a structural schematic diagram showing a computer system according to an embodiment of the present application.

Reference is made to FIG. 10, which is a structural schematic diagram of a computer system 1000 applied to the device of the embodiment of the present application.

  As shown in FIG. 10, the computer system 1000 performs various appropriate operations based on a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage unit 1008 to a random access memory (RAM) 1003. And a central processing unit (CPU) 1001 capable of executing processing. When a program stored in the ROM 1002 or a program loaded from the storage unit 1008 to the RAM 1003 is executed by the CPU 1001, the CPU 1001 executes the method described in the present application. The RAM 1003 further stores various programs and data necessary for operating the system 1000. The CPU 1001, ROM 1002, and RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

  An input unit 1006 including a keyboard and a mouse, a cathode ray tube (CRT), a liquid crystal display (LCD), and an output unit 1007 including a speaker, a storage unit 1008 including a hard disk, and a network interface including a LAN card and a modem The communication unit 1009 of the card is connected to the I / O interface 1005. The communication unit 1009 executes communication processing via a network such as the Internet. The driver 1010 is connected to the I / O interface 1005 as necessary. The removable medium 1011 is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, and is attached to the driver 1010 as necessary. Therefore, a computer program read from the driver 1010 is necessary. Is installed in the storage unit 1008 accordingly.

  In particular, according to an embodiment of the present invention, the process described with reference to the above flowchart may be implemented as a software program on a computer. For example, embodiments of the present invention include a computer program product, the computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program performing the method shown in the flowchart. Including program code. In such an embodiment, the computer program may be downloaded and installed from the network via the communication unit 1009 and / or installed from the removable medium 1011.

  The flowcharts and block diagrams in the Figures illustrate the architecture, functionality, and operation that can be implemented by systems, methods and computer program products according to embodiments of the present invention. Here, each frame in the flowchart or block diagram may represent one module, program segment, or part of code, and the module, program segment, or part of code may have a defined logical function. Contains one or more executable instructions to accomplish. Also, as some alternative embodiments, the functions shown in the boxes may be performed in a different order than the order shown in the drawings. For example, two frames shown in succession may actually be executed substantially in parallel, or may be executed in reverse order, depending on the function involved. Also, each frame in the block diagram and / or flowchart and the combination of the frame in the block diagram and / or flowchart may be realized by a hardware-based dedicated system that performs a specified function or operation, or It may be executed by a combination of dedicated hardware and computer instructions.

  On the other hand, the present invention further provides a computer-readable storage medium, and the computer-readable storage medium may be a computer-readable storage medium included in the apparatus of the above-described embodiment, and exists independently. Thus, it may be a computer-readable storage medium that is not assembled in the terminal. One or more programs are stored on the computer readable storage medium, and the one or more programs execute the method for recognizing traffic signs described in the present application by one or more processors. It is used.

  The foregoing is merely illustrative of the preferred embodiment of the invention and the technical principles used. The scope of the claims of the present invention is not limited to a technical proposal comprising a specific combination of the above-described technical features, and the above-described technical features or equivalent features without departing from the spirit of the present invention. It should be understood by those skilled in the art that other technical solutions consisting of any combination of the above should also be included. For example, there is a technical proposal in which the above features and technical features (not limited to these) having similar functions disclosed in the present invention are replaced with each other.

Claims (16)

Acquiring a feature value obtained by a predetermined feature algorithm in a predetermined integration channel of a scan window image obtained by dividing the omnidirectional image;
Detecting a scan window image based on a feature value of the scan window image and a pre-trained detection classifier model to obtain a traffic sign window image to be confirmed;
Recognizing a traffic sign window image to be confirmed based on a pretrained convolutional neural network model, and obtaining a traffic sign type,
Here, the pre-trained detection classifier model is obtained by training based on scan window image samples and their feature values, and the pre-trained convolutional neural network model is obtained from the confirmed traffic sign window image. A method for recognizing a traffic sign obtained by training based on a sample and its kind. Obtain the pre-trained detection classifier model by the following method:
Obtaining a positive sample and a negative sample in a sample of a scan window image, wherein the positive sample includes a window image of a traffic sign or a window image in which a predetermined pixel is expanded around the traffic sign and the surroundings, The negative sample includes a scan window image that excludes the positive sample,
Obtaining a characteristic value obtained by a predetermined characteristic algorithm in a predetermined integration channel of the positive sample and the negative sample;
Training a detection classifier model with a boosting algorithm based on the positive sample and the negative sample and the feature values of the positive sample and the negative sample obtained by a predetermined feature algorithm in a predetermined integration channel; The method of claim 1, wherein the pretrained detection classifier model is obtained. In the step of acquiring a feature value obtained by a predetermined feature algorithm in a predetermined integration channel of the scan window image obtained by dividing by the omnidirectional image,
Obtaining an integral image of the scan window image obtained by dividing the integral image of the predetermined integral channel by the omnidirectional image;
The method according to claim 1, further comprising: obtaining a feature value of the scan window image based on an integrated image of the scan window image. In the step of obtaining an integrated image of a scan window image obtained by dividing the omnidirectional image by an integrated image of a predetermined integration channel, the omnidirectional image is continuously down-sampled to obtain an image pyramid. And obtaining an integrated image of the scan window image of each stage image in the image pyramid obtained by dividing the integrated image of the predetermined integration channel by the image pyramid, and
In the step of obtaining the feature value of the scan window image based on the integral image of the scan window image, the scan window image of each stage image is obtained based on the integral image of the scan window image of each stage image. Obtaining a feature value, and detecting a scan window image based on the feature value of the scan window image and a pre-trained detection classifier model, and a traffic sign window image to be confirmed In the obtaining step, the traffic sign to be detected by detecting the scan window image of each stage image based on the feature value of the scan window image of each stage image and the pre-trained detection classifier model The method according to claim 3, further comprising: obtaining a window image. Method. Obtain the pretrained convolutional neural network model by the following method:
In accordance with a Gaussian distribution, the weights of both the convolutional layer and the fully connected layer of a predetermined convolutional neural network model are initialized, and the predetermined convolutional neural network model is sequentially connected to a convolutional layer, an extraction layer, a fully connected layer, and a normalization. With layers,
Based on the confirmed traffic sign window image sample and the type of the traffic sign, the weights of both the convolutional layer and the total coupling layer are iterated by the backpropagation BP algorithm, and the currently repeated weight and the previous weight are repeated. The currently repeated weight is determined as the optimal weight when the difference from the determined weight is smaller than a predetermined value, and the convolutional neural network model including the optimal weight is the pretrained convolutional neural network model, Or if the lowest repeated weight with the lowest error rate appears, determine the lowest repeated weight with the lowest error rate as the optimal weight and the convolutional neural network model containing the optimal weight is the pretrained convolutional neural network The method according to claim 1, wherein the method is a network model. . The predetermined convolutional neural network model further includes a loss function layer, and in the step of repeating the weights of both the convolution layer and the fully connected layer by the error back-propagation BP algorithm, the loss function and the BP algorithm 6. The method of claim 5, comprising repeating the weights of both the convolutional layer and the total coupling layer, respectively. In the step of recognizing the traffic sign window image to be confirmed based on the pretrained convolutional neural network model and obtaining the traffic sign type,
Inputting the traffic sign window image to be confirmed to the convolutional neural network model, obtaining the traffic sign type with the maximum weight output by the normalization layer; and recognizing the traffic sign type with the maximum weight. And setting as traffic signs obtained as described above. The method according to claim 5. A feature value acquisition module for acquiring a feature value obtained by a predetermined feature algorithm in a predetermined integration channel of a scan window image obtained by dividing the omnidirectional image;
A traffic sign detection module for detecting a scan window image based on the feature value of the scan window image and a pre-trained detection classifier model, and obtaining a traffic sign window image to be confirmed;
A traffic sign recognition module for recognizing a traffic sign window image to be confirmed and acquiring a traffic sign type based on a pretrained convolutional neural network model;
Here, the pre-trained detection classifier model is obtained by training based on a scan window image sample and its feature value, and the pre-trained convolutional neural network model is a confirmed traffic sign window image sample. And a device for recognizing a traffic sign obtained by training based on the type of the traffic sign. The pre-trained detection classifier model is obtained by the following method, that is, a positive sample and a negative sample are obtained in a scan window image sample, wherein the positive sample includes a traffic sign window image. Or a traffic sign and a window image in which predetermined pixels are expanded around the traffic sign, and the negative sample includes a scan window image excluding the positive sample,
Obtaining a characteristic value obtained by a predetermined characteristic algorithm in a predetermined integration channel of the positive sample and the negative sample;
Training a detection classifier model with a boosting algorithm based on the positive sample and the negative sample and the feature values of the positive sample and the negative sample obtained by a predetermined feature algorithm in a predetermined integration channel; 9. The apparatus of claim 8, wherein the pretrained detection classifier model is obtained. The feature value acquisition module includes:
A first integral image acquisition sub-module for acquiring an integral image of a scan window image obtained by dividing the whole celestial sphere image by an integral image of a predetermined integral channel;
The apparatus according to claim 8, further comprising: a first feature value acquisition submodule for acquiring a feature value of the scan window image based on an integral image of the scan window image. The first integral image acquisition submodule continuously downsamples the omnidirectional image to obtain an image pyramid, and divides the integrated image of a predetermined integration channel by the image pyramid. A second integral image acquisition submodule for acquiring an integral image of a scan window image of each stage image in the image pyramid obtained in the above,
The first feature value acquisition submodule includes a second feature value acquisition submodule for acquiring a feature value of a scan window image of each stage image based on an integral image of the scan window image of each stage image,
The detection module detects a scan window image of each stage image based on a feature value of the scan window image of each stage image and a pre-trained detection classifier model, and a traffic sign window to be confirmed The apparatus according to claim 10, comprising a multi-stage detection sub-module for acquiring an image. The pretrained convolutional neural network model is obtained by the following method:
In accordance with a Gaussian distribution, the weights of both the convolutional layer and the fully connected layer of a predetermined convolutional neural network model are initialized, and the predetermined convolutional neural network model is sequentially connected to a convolutional layer, an extraction layer, a fully connected layer, and a normalization. With layers,
Based on the confirmed traffic sign window image sample and the type of the traffic sign, the weights of both the convolutional layer and the total coupling layer are iterated by the backpropagation BP algorithm, and the currently repeated weight and the previous weight are repeated. The currently repeated weight is determined as the optimal weight when the difference from the determined weight is smaller than a predetermined value, and the convolutional neural network model including the optimal weight is the pretrained convolutional neural network model, Or if the lowest repeated weight with the lowest error rate appears, determine the lowest repeated weight with the lowest error rate as the optimal weight and the convolutional neural network model containing the optimal weight is the pretrained convolutional neural network The device according to any one of claims 8 to 11, wherein the device is a network model. Place. The predetermined convolutional neural network model further comprises a loss function layer;
The apparatus of claim 12, wherein the weight determination module is further used to iterate the weights of both the convolutional layer and the full coupling layer, respectively, with a loss function and a BP algorithm. The traffic sign recognition module includes:
A maximum weight recognition module for inputting the traffic sign window image to be confirmed to the convolutional neural network model and obtaining a traffic sign type with the maximum weight output by the normalization layer;
The apparatus according to claim 12, further comprising: a sign type setting module configured to set as a traffic sign obtained by recognizing the traffic sign type having the maximum weight. A processor and a memory,
A computer readable instruction executable by the processor is stored in the memory, and the processor performs the method of any one of claims 1 to 7 when the computer readable instruction is executed. A device characterized by Computer readable instructions executable by a processor are stored, and when the computer readable instructions are executed by the processor, the processor performs the method of any one of claims 1-7. A non-volatile computer storage medium.