KR20160132331A - Method and device for identifying traffic signs - Google Patents

Abstract

The present invention discloses a method and apparatus for recognizing traffic signs. A specific embodiment of the method comprises the steps of: obtaining a feature value of a scanning window image obtained with a panoramic spherical image segment obtained with a predefined feature algorithm in a predefined integral channel; Acquiring a traffic sign window image to detect and verify a scanning window image according to a pre-trained detection classifier model trained according to a characteristic value of the scanning window image, a sample of the scanning window image, and a characteristic value thereof; And acquiring a traffic sign type by recognizing a traffic sign window image to be verified according to a pre-trained convolution neural network model trained according to a sample of the traffic sign window image already confirmed and the traffic sign type thereof. The embodiment improves the accuracy of detection and recognition of traffic signs in a panoramic image and improves the efficiency of updating road network data.

Description

[0001] METHOD AND DEVICE FOR IDENTIFYING TRAFFIC SIGNS [0002]

The present invention claims the priority of a Chinese patent application having a filing date of Mar. 31, 2015 and the application number is 201510150525.8, the entire contents of which are incorporated herein by reference.

Field of the Invention [0002] The present invention relates to a computer technology field, and more particularly, to a computer object recognition technology, and more particularly, to a traffic sign recognition method and apparatus.

In order to provide accurate and complete navigation data, it is necessary to recognize the traffic signs in the traffic network.

Conventional traffic sign recognition methods rely on artificial processing mainly to install predetermined program software and map data in a collection vehicle and to allow a skilled external worker to work with the vehicle to observe traffic signs After the worker manually inputs the type of the traffic sign and the distance he / she has seen, the internal worker compares the basic diagrams before and after the work is finished and updates valid information to the road network database after the completion of the external work collection. Although this process requires support from various software, the accuracy of the data results is mainly determined by the individual ability and commitment of the worker, and the complicated collection process reduces the update efficiency of the road network data.

SUMMARY OF THE INVENTION The present invention has been made to solve the technical problems mentioned in the background section, and it is an object of the present invention to provide a traffic sign recognition method and apparatus.

In a first aspect, the present invention provides a traffic sign recognition method, wherein the traffic sign recognition method comprises the steps of: acquiring a feature of a scanning window image acquired with a panoramic spherical image segment acquired with a predetermined feature algorithm in a default integral channel Obtaining a value; Acquiring a traffic sign window image to detect and verify a scanning window image according to a pre-trained detection classifier model trained according to a characteristic value of the scanning window image, a sample of the scanning window image, and a characteristic value thereof; And acquiring a traffic sign type by recognizing the traffic sign window image to be verified according to a sample of the traffic sign window image already confirmed and a pre-trained convolution neural network model trained according to the traffic sign type thereof.

In a second aspect, the present invention provides a traffic sign recognition device, wherein the traffic sign recognition device comprises: a feature of a scanning window image obtained with a panoramic spherical image segment acquired with a predefined feature algorithm on a predefined integral channel; A feature value acquisition module configured to acquire a value; A traffic sign window configured to detect and verify a scanning window image according to a pre-trained detection classifier model trained according to a characteristic value of the scanning window image, a sample of the scanning window image, and a characteristic value thereof, Detection module; And a traffic sign recognition module configured to recognize a traffic sign window image to be verified according to a pre-trained convolution neural network model trained according to a sample of the traffic sign window image and a traffic sign type of the traffic sign window, .

A method and apparatus for recognizing a traffic sign provided in the present invention is a method for acquiring a feature value of a scanning window image obtained by a panoramic spherical image segment acquired with a predetermined feature algorithm in a predetermined integral channel, And a traffic window window image to be identified according to a pre-trained convolution neural network model, and recognizes a traffic sign window image to be confirmed according to the following pre-trained convolution neural network model, The traffic sign window image to be confirmed is detected by the pre-trained detection classifier model. By recognizing the traffic sign type with the pre-trained convolution neural network model, it is possible to detect the traffic sign in the panoramic image, Improves the accuracy of recognition, and improves the efficiency of updating the road network data.

Other features, objects and advantages of the present invention will become more apparent from the detailed description of the non-limiting embodiments which proceeds with reference to the following drawings.
1 is an exemplary flowchart of a method of recognizing a traffic sign according to an embodiment of the present invention.
2 is an exemplary flow chart of a method of training a pre-trained detection model in accordance with an embodiment of the present invention.
3 is an exemplary flowchart of a method for obtaining a traffic sign window image to be verified according to an embodiment of the present invention.
4 is an exemplary flow chart of a method of training a pre-trained convolution neural network model in accordance with an embodiment of the present invention.
5 is an exemplary schematic diagram of a predetermined convolution neural network model according to an embodiment of the present invention.
6 is a view illustrating a structure of a traffic sign recognition apparatus according to an embodiment of the present invention.
7 is an exemplary structural view of a training apparatus of a detection classifier model according to an embodiment of the present invention.
8 is an exemplary structure diagram of an apparatus for acquiring a traffic sign window image to be verified.
9 is an exemplary schematic diagram of an apparatus for training a pre-trained convolution neural network model in accordance with an embodiment of the present invention.
10 is a structural schematic diagram of a computer system provided in accordance with an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. It is to be understood that the specific embodiments described herein are for interpretation of the related invention only and are not intended to limit the invention. In addition, for convenience of explanation, it is explained in advance that only the parts related to the invention are shown in the attached drawings.

It is to be understood that the features of the embodiments and the embodiments of the present invention can be combined with each other unless they are mutually contradictory. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is an exemplary flowchart of a method of recognizing a traffic sign according to an embodiment of the present invention.

As shown in FIG. 1, the traffic sign recognition method 100 may include the following steps.

At step 110, feature values of the scanning window image obtained with the panoramic spherical image segment obtained with the feature algorithm of the default at the default integral channel are obtained.

Here, first, a panorama spherical image to be subjected to traffic sign recognition is acquired, a panorama spherical image is scanned through a window of a next predetermined size, a scanning window image is acquired, and then a predefined characteristic algorithm The characteristic value of the scanning window image acquired by the scanning window image acquiring unit can be obtained.

Here, the panoramic spherical image refers to an image formed by mosaicing a plurality of fisheye images capable of displaying a panorama. A window of a predetermined size is typically a predetermined window according to the size of the object object.

When acquiring a feature value of a scanning window image acquired with the feature algorithm of the default in the default integral channel, the default integral channel may be any one or more integral channels of the prior art in the field of image processing, The feature algorithm may be any one or more feature algorithms in the prior art in the field of image processing or any one or more feature algorithms I can explain that. The present invention does not proceed with the limitation on the concrete method of acquiring the feature value, so that the user can proceed to the selection according to the actual use demand. For example, it is possible to obtain a haar feature value from a gray-scale image channel in a gray-scale image channel, acquire a histogram feature value from a histogram feature value algorithm in a gradient image channel of different angle parameters, A method of obtaining a random feature value from a color image channel using a random feature value algorithm can be selected.

In order to further improve the speed at which the characteristic values of the scanning window image are acquired, the characteristic values of the scanning window image can be obtained through the integral image. Optionally, in this embodiment, step 110 comprises obtaining (111) an integral image of the scanning window image obtained with the integral image segment of the panoramic spherical image at the predefined integral channel, And acquiring (112) a feature value of the scanning window image according to the image.

By calculating the feature value of the scanning window image using the integral image of the panoramic spherical image, it is possible to accelerate the calculation of the feature value of the scanning window image and improve the efficiency of calculating the feature value of the scanning window image.

Optionally, in order to accelerate the comparison of the feature values of the scanning window image with the feature values of the samples of the traffic-indication window image in the pre-trained monitoring classifier model, the feature values of the samples of the traffic- It is possible to determine the default integral channel and feature algorithm for acquiring feature values of the scanning window image. For example, a prohibited indicator such as a speed limit or the like may enhance the description on the red channel. Preferably, the default integral channel may comprise one or more of a gray scale image channel, a single color image channel of red, green, indigo, a gradient image channel of different angular parameters and an edge detection image channel, The feature algorithm may include one or more of a Harsh feature algorithm, a local binary pattern feature algorithm, a histogram feature algorithm, and a random feature algorithm. Those skilled in the art will appreciate that it is possible to obtain the feature values necessary for the detection of the scanning window image by combining the default integral channel and the default feature algorithm in the above embodiments according to demand .

The default integral channel and the default feature algorithm for obtaining the feature value of the sample of the traffic sign window image and the feature value of the scanning window image determined according to the type of the traffic sign are different from each other by describing the characteristics of the object from a plurality of different angles It is possible to overcome the change of the object due to the angle and the irradiation.

In step 120, a scanning window image is detected and a traffic sign window image to be checked, according to a pre-trained detection classifier model trained according to the characteristic value of the scanning window image and the sample of the scanning window image, .

Here, the pre-trained detection classifier model first receives artificial tagging, determines a scanning window image including a traffic sign in a scanning window image and a scanning window image not including a traffic sign as a sample of a traffic sign window image, Obtained by acquiring characteristic values of a sample, and then training the parameters of the classifier model set according to actual demand using the samples and their characteristic values.

After obtaining the feature value of the scanning window image in step 101, the scanning window image is detected according to the feature value of the scanning window image and the pre-trained detection classifier model, To obtain a suspected window image.

In step S130, a traffic sign type is obtained by recognizing a traffic sign window image to be verified according to a sample of the traffic sign window image already confirmed and a pre-trained convolution neural network model trained according to the traffic sign type thereof.

Here, the pre-trained convolution neural network model first receives the artificial tagging, classifies the sample of the traffic sign window image to be verified detected by the detection classifier model, and performs tagging with a specific traffic sign type or non- After that, it is a pre-trained convolution neural network model obtained by training a convolution neural network model set according to actual demand using a sample of a traffic sign window image to be checked and its traffic sign type.

After obtaining the traffic sign window image to be checked in step (102), the traffic sign window image to be checked according to the traffic sign window image to be checked and the pre-trained convone neural network model are recognized The traffic sign type having the highest probability of being associated with the desired traffic sign window image is obtained as the recognized traffic sign type.

The method of recognizing a traffic sign of the embodiment of the present invention improves the accuracy of detection and recognition of panorama images on traffic signs and improves the efficiency of updating the road network data.

Hereinafter, a method of training a pre-trained detection model in conjunction with FIG. 2 will be described.

2 is an exemplary flow chart of a method of training a pre-trained detection model in accordance with an embodiment of the present invention.

As shown in FIG. 2, a method 200 for training a pre-trained detection classifier model may include the following steps.

In step 201, the remaining scanning window images except the positive sample and the positive sample, including the window image of the traffic sign in the sample of the scanning window image or the window of the preset pixel expanding around the traffic sign and surroundings thereof are included Lt; / RTI >

Here, in the positive sample and the negative sample in the sample of the scanning window image, first, a sample is acquired from the scanning window image, and then the window image of the traffic sign is included in the sample of the scanning window image according to the received artificial tagging, A window image including a predetermined pixel expanding the vicinity thereof is set as a positive sample, and a remaining scanning window image excluding a positive sample in a sample is set as a negative sample.

When the positive sample includes only the window image of the traffic sign, the efficiency of calculating the feature value of the positive sample can be improved. Since the pixels around the object object can also effectively describe the object itself, it is possible to improve the accuracy of the feature value describing the positive sample if the positive sample includes a window image of a predefined pixel that has expanded the traffic sign and its surroundings have.

In step 202, the feature values of the positive and negative samples obtained with the feature algorithm of the default in the default integral channel are obtained.

Here, when obtaining the feature values of the positive samples and the negative samples acquired with the feature algorithm of the default in the default integral channel, the default integral channel may be any one or more of the integral channels of the prior art in the field of image processing, And the predefined feature algorithm may be any one or more feature algorithms in the prior art in the field of image processing or any one or more Lt; / RTI > algorithm. The present invention does not proceed to the definition of the integral channel of the acquired acquired characteristic value and the characteristic algorithm of the predetermined characteristic so that the user can proceed with the selection according to the actual use demand. For example, a haar feature value obtained with a Haar feature algorithm in a gray scale image channel, a gradient of different angular parameters, a histogram feature value obtained with a histogram feature algorithm in an image channel, an image of a single color of red, green, A random feature value obtained by a random feature algorithm in a channel, and the like.

Alternatively, in order to quickly and effectively acquire the characteristic values of the positive sample and the negative sample acquired by the characteristic algorithm of the default in the default integral channel when proceeding to the concrete calculation of the characteristic value of the sample, You can determine the default integral channel and default feature algorithm of a sample of a window image. For example, the description on the red channel can be enhanced for a prohibitive indicator such as a speed limit. Preferably, the default integral channel may comprise one or more of a gray scale image channel, a single color image channel of red, green, indigo, a gradient image channel of different angle parameters and an edge detection image channel, The algorithm may include one or more of a Harsh feature algorithm, a local binary pattern feature algorithm, a histogram feature algorithm, and a random feature algorithm. Those skilled in the art will understand that the positive and negative samples obtained by using the default feature algorithm in the default integral channel by combining the predefined integral channel of the embodiment and the predefined feature algorithm according to demand, It is possible to obtain the feature value of "

In step 203, training is performed on the detected classifier model using a boosting algorithm, based on their positive and negative samples and their characteristic values obtained with the feature algorithm of the default in the predefined integral channel, Obtain a classifier model.

In step 202, the feature values of the positive and negative samples acquired by the feature algorithm of the default on the default integral channel are acquired. Then, training is performed on the detection classifier model using the boosting algorithm to obtain a single high-accuracy The result classifier can be obtained as a pre-trained detection classifier model.

Hereinafter, a method of acquiring a traffic sign window image to be verified based on the embodiment will be described with reference to FIG.

3 is an exemplary flowchart of a method for obtaining a traffic sign window image to be verified according to an embodiment of the present invention.

As shown in FIG. 3, a method 300 for acquiring a traffic sign window image to be verified includes the following steps.

In step 301, continuous downsampling is performed on the panoramic spherical image to obtain an image pyramid.

Here, the pyramid transformation, such as, for example, a Laplace pyramid transformation, may be performed on the panoramic spherical image to convert the panoramic spherical image from the original size to a different scale space to obtain the image pyramid including the multilevel image.

At step 302, an integral image of the scanning window image of each level image in the image pyramid acquired with the integral image segment of the image pyramid in the default integral channel is obtained.

After acquiring the image pyramid in step 301, the integrals are progressed for each level image of the image pyramid in the predefined integral channel to obtain an integral image of the image pyramid, and then the image pyramid It is possible to acquire an integral image of the scanning window image of each level image.

At step 303, feature values of the scanning window image of each level image are obtained according to the integral image of the scanning window image of each level image.

In operation 302, an integral image of the scanning window image of each level image is acquired. Then, an operation is performed using the integral image of the scanning window image of each level image according to the integral image of the scanning window image of each level image The feature value of the scanning window image of each level image can be obtained quickly.

In step 304, a scanning window image of each level image is acquired and a traffic sign window image to be verified is obtained according to the feature value of the scanning window image of each level image and the pre-trained detection classifier model.

After obtaining the feature values of the scanning window image of each level image in step 303, the scanning window image of each level image is detected through the pre-trained detection classifier model according to the feature value of the scanning window image of each level image And acquires the traffic sign window image as a traffic sign window image to be identified as a traffic sign, thereby obtaining all the traffic sign window images to be checked in the image pyramid.

A method of acquiring a traffic sign window image to be verified in the embodiment of the present invention includes acquiring a scanning window image in a pyramid image obtained through pyramid conversion in a panoramic spherical image, and then detecting a scanning window image of each level image By acquiring the traffic sign window image to be verified, it is possible to reduce the possibility of missing the traffic sign window to be confirmed in the panoramic spherical image, thereby improving the acquisition accuracy of the traffic sign window to be verified. In addition, since the feature value of the scanning window image of each level image is obtained using the integral image, the acquisition speed of the feature value is improved.

4, a method of training a pre-trained convolution neural network model is described.

4 is an exemplary flow chart of a method of training a pre-trained convolution neural network model in accordance with an embodiment of the present invention.

As shown in FIG. 4, a method 400 for training a pre-trained convolution neural network model includes the following steps.

In step 401, the weights of both the convolution layer and the fully connected layer of the predetermined convolution neural network model including the sequentially connected convolution layer, the extraction layer, the perfect connection layer and the normalization layer are initialized in accordance with the Gaussian distribution .

Here, the sequentially connected convolution layer, the extraction layer, the complete coupling layer and the normalization layer may include one convolution layer, one extraction layer, one complete coupling layer and one normalization layer, or a plurality of convolution layers and An extraction layer corresponding to the convolution layer, one or more complete connection layers and one normalization layer.

The convolution layer installed here can enhance original signal characteristics and reduce noise through convolution operation. The extracted layer can use the partial associativity principle of the image to perform sub-sampling on the image to reduce data throughput and to hold useful information.

Hereinafter, a predetermined convolution neural network model will be described with reference to FIG.

5 is an exemplary schematic diagram of a predetermined convolution neural network model according to an embodiment of the present invention.

5, the predetermined convolution neural network model 500 includes a convolution layer conv1, an extraction layer pool1, a convolution layer conv2, an extraction layer pool2, Layer conv3, an extraction layer pool3, a full coupling layer fc1 and a complete coupling layer fc2.

There are 16 convolution kernels of size 5 * 5 * 3 in conv1, 32 convolution kernels of size 5 * 5 * 16 in conv2, and convolution kernels of size 5 * 5 * 32 exist in conv3. There are 64 convolution kernels. There are 512 neurons and 120 neurons in each of the two perfectly connected layers.

Optionally, in order to increase the output difference of the similar object and the dissimilar object, the predetermined convolution neural network model may further include a loss function layer. For example, the loss function can be calculated as a function of various logarithmic losses, ie:

(a)이고,

(a),

는 네트워크에서 출력된 제i개 샘플이 제j류인 확율이며,

는 샘플의 진리 값이고, 제i개 샘플이 제j류에 속하면 1이고 속하지 않으면 0이다. Where L is the loss function, N is the number of training samples, M is the number of species,

Is the probability that the ith sample output from the network is the jth class,

Is the truth value of the sample, 1 if the ith sample belongs to class j and 0 if it is not.

Referring again to FIG. 4, in step 402, an iterative propagation BP algorithm is used to determine the weights of both the convolution layer and the fully connected layer according to the sample of the previously identified traffic sign window image and its traffic sign type, ).

Here, the sample of the traffic sign window image that has been confirmed is a sample of the window image that is confirmed according to the received artificial tagging including the traffic sign among the samples extracted from the traffic sign window image to be confirmed.

Further, a sample of the already verified traffic sign window image may be set after a sample of the already verified traffic sign window image and a sample of the originally verified traffic sign window image through one or more processes of rotation, translation and reduction And may include images that are aggregated into window sizes. It is possible to improve the recognition accuracy of the sample by expanding the sample.

)에 대한 경사도를 후진적으로 산출하며, 그 다음 각 층의 가중치(

)의 경사도에 따라 가중치(

)를 업데이트하는 바, 즉: The step of performing iteration with respect to the weights of both the convolution layer and the fully connected layer using the error back propagation BP algorithm in correspondence with the provision of the loss function layer in step 401 includes: And iterating each of the weights of the loop layer and the fully connected layer. For example, after initializing all the weights with a Gaussian distribution, iterates several times for the network model using the lanam slope descent algorithm, and every time it is iterated, a) to gradually calculate the loss function L of the network, and then conversely the function L is weighted by each layer

), And then the weight of each layer (

) ≪ / RTI >

), That is:

-

(b)

-

(b)

는 손실 함수(L)가 가중치(

)에 대한 경사도이다. Here,? Is a learning rate of a predetermined gradient lowering,

(L) is a weighting factor

).

Thereafter, the convolutional neural network model of the optimal weight can be determined in step S403 or step S404.

Step S403: If the difference between the current repeated weight and the immediately previous weight is less than the preset value, the currently repeated weight is determined as the optimum weight.

Step S404: If the repeated weight value with the lowest error rate is found, the repeated weight value with the lowest error rate is determined as the optimum weight value.

Step S405: The convolutional neural network model including the optimum weight is set as a pre-trained convolutional neural network model.

After training the pre-trained convolution neural network model, the step 103 in FIG. 1 inputs the traffic sign window image to be verified to the convolutional neural network model to determine the traffic sign type having the largest weight output from the normalization layer Obtaining; And setting the traffic mark type having the largest weight as the recognized traffic mark type.

6 is a view illustrating a structure of a traffic sign recognition apparatus according to an embodiment of the present invention.

6, the traffic sign recognition apparatus 600 may include a feature value acquisition module 610, a traffic sign detection module 620, and a traffic sign recognition module 630. [

The feature value acquisition module 610 is configured to obtain a feature value of the scanning window image obtained with the panoramic spherical image segment acquired with the feature algorithm of the default in the default integral channel.

Optionally, the feature acquisition module 610 may include an integral image acquisition first sub-module 611 configured to acquire an integral image of the scanning window image acquired with the integral image segment of the panoramic spherical image in the predefined integral channel; And a feature value acquisition first sub-module 612 configured to acquire feature values of the scanning window image according to an integral image of the scanning window image.

The traffic sign detecting module 620 detects a traffic sign window image according to a pre-trained detection classifier model trained according to a characteristic value of a scanning window image, a sample of a scanning window image, and a characteristic value thereof, Image.

The traffic sign recognition module 630 recognizes the traffic sign window image to be verified according to the pre-trained convolution neural network model trained according to the sample of the traffic sign window image already confirmed and its traffic sign type, .

The traffic sign recognition apparatus of the embodiment of the present invention improves the accuracy of detection and recognition of traffic signs in the panoramic image and improves the update efficiency of the road network data.

Optionally, the pre-trained detection classifier model includes a positive and a negative sample, including a window image of a traffic sign among the samples of the scanning window image or a positive sample comprising a window image of a predetermined pixel expanding around the traffic sign and its surroundings Acquiring a negative sample including the remaining scanning window image, acquiring the characteristic values of the positive sample and the negative sample acquired with the characteristic algorithm of the default in the default integral channel, and determining the positive and negative values of the positive and negative samples in the integral channel Training can be obtained by acquiring a pre-trained detection classifier model by tracing the detected classifier model using the boosting algorithm according to the feature values obtained by the feature algorithm

The method of acquiring the pre-trained detection classifier model is realized through the training device of the detection classifier model shown in FIG.

7 is an exemplary structural view of a training apparatus of a detection classifier model according to an embodiment of the present invention.

7, the training apparatus 700 of the detection classifier model may include a sample acquisition module 701, a sample feature value acquisition module 702, and a detection model training module 703.

The sample acquiring module 701 acquires a positive sample including a window image of a traffic sign among the samples of the scanning window image or a window image of a predetermined pixel that has expanded around the traffic sign and the vicinity thereof, ≪ / RTI >

The sample feature value acquisition module 702 is configured to acquire the feature values of the positive and negative samples acquired with the feature algorithm of the default in the default integral channel.

Here, the default integral channel may include one or more of a grayscale image channel, a single color image channel of red, green, indigo, a gradient image channel of different angle parameters, and an edge detection image channel. The default feature algorithm may include one or more of a Harsh feature algorithm, a local binary pattern feature algorithm, a histogram feature algorithm, and a random feature algorithm.

The detection model training module 703 trains the detection classifier model with the boosting algorithm according to the feature values obtained by the feature algorithm of the default in the positive sample and the negative sample and the default integral channel, To obtain a classifier model.

Hereinafter, with reference to FIG. 8, an apparatus for acquiring a traffic sign window image to be verified will be described on the basis of the above embodiment.

FIG. 8 shows an exemplary structure of an apparatus for acquiring a traffic sign window image to be verified.

8, an apparatus 800 for acquiring a traffic sign window image to be identified includes a downsampling sub-module 801, an integral image acquisition second sub-module 802, a feature value second acquisition sub- Module 803 and a multi-level detection submodule 804, as shown in FIG.

The downsampling submodule 801 is configured to successively downsample the panoramic spherical image to obtain an image pyramid.

The integral image acquisition second submodule 802 is configured to obtain an integral image of the scanning window image of each level image of the image pyramids obtained with the integral image segment of the image pyramid in the predefined integral channel.

The feature value second acquisition submodule 803 is configured to obtain a feature value of the scanning window image of each level image according to an integral image of the scanning window image of each level image.

The multi-level detection sub-module 804 detects the scanning window image of each level image according to the feature value of the scanning window image of each level image and the pre-trained detection classifier model to obtain a traffic sign window image to be checked .

The detection accuracy of the traffic sign window image to be verified can be further improved and the detection speed of the traffic sign window image to be verified can be improved through the traffic sign window image obtaining device of the embodiment of the present invention.

Alternatively, the pre-trained convolution neural network model of FIG. 6 may include a convolution layer of a predetermined convolution neural network model including a convolution layer, an extraction layer, a full connection layer and a normalization layer sequentially connected in accordance with a Gaussian distribution Iterates the weights of both the convolution layer and the fully connected layer by an error back propagation BP algorithm according to the sample of the traffic sign window image and the type of the traffic sign of the traffic window window, If the difference between the current repeated weight and the immediately previous weight is less than the preset value, the current repeated weight is determined as the optimum weight, and the convolution neural network model including the optimal weight is converted into the pre- If the neural network model is set or a repeated weight with the lowest error rate appears, By determining the lowest weight by repeating the optimum weight, and setting the convolutional neural network, including optimal weight by the pre-training a convolutional neural network can be obtained.

Here, the sample of the already-verified traffic sign window image is a sample of the window image including the traffic sign confirmed according to the artificial tagging received from the sample extracted from the traffic sign window image to be verified.

Further, a sample of the traffic sign window image that has already been verified may include a sample of the traffic sign window image that has already been verified, and a sample of the traffic sign window image that has already been verified, after one or more processes of rotation, You can include an image normalized to the setting window size.

Here, the sequentially connected convolution layer, the extracting layer, the complete connecting layer and the normalizing layer include an extraction layer corresponding to a plurality of convolution layers and a convolution layer, one or more perfect connection layers and one normalization layer.

Furthermore, the predetermined convolution neural network model may further include a loss function layer.

Wherein the step of iterating the weights of both the convolution layer and the fully-connected layer with an error back propagation BP algorithm, corresponding to the pre-determined convolution neural network model further comprising a lossy function layer, And a step of performing iteration on the weights of both the convolution layer and the fully connected layer by the BP algorithm.

The method of acquiring the pre-trained convolution neural network model can be realized through an apparatus for training the pre-trained convolution neural network model shown in FIG.

9 shows an exemplary structural view of an apparatus for training a pre-trained convolution neural network model according to an embodiment of the present invention.

As shown in FIG. 9, an apparatus 900 for training a pre-trained convolution neural network model includes the following modules.

The initialization module 901 initializes the weights of both the convolution layer and the fully connected layer of the predetermined convolution neural network model including the sequentially connected convolution layer, the extraction layer, the complete connection layer and the normalization layer according to the Gaussian distribution .

Here, the sequentially connected convolution layer, the extracting layer, the complete connecting layer and the normalizing layer include an extraction layer corresponding to a plurality of convolution layers and a convolution layer, one or more perfect connection layers and one normalization layer.

Optionally, the predetermined convolution neural network model further comprises a loss function layer.

The weighing repetition module 902 is configured to perform iteration with respect to the weights of both the convolution layer and the fully connected layer using the error back propagation BP algorithm according to the sample of the traffic sign window image already confirmed and its traffic sign type do.

The first optimal weight determination module 903 is configured to determine the currently repeated weight as the optimum weight if the difference between the current repeated weight and the immediately previous weight is less than the preset value.

The second optimal weight determining module 904 is configured to determine the repeated weight having the lowest error rate as the optimum weight when the repeated weight having the lowest error rate is displayed.

The model setting module 905 is configured to set the convolutional neural network model including optimal weights to a pre-trained convolutional neural network model.

The predetermined convolution neural network model corresponds to further including the loss function layer, and the weight determination module 902 performs iteration on the weight of both the convolution layer and the fully connected layer by the loss function and the BP algorithm Lt; / RTI >

After training the pre-trained convolution neural network model, the device 900 for training the pre-trained convolution neural network model then traverses the traffic sign window image to be verified, A maximum weight recognition module (not shown) configured to input the neural network model and obtain the traffic mark type having the largest weight output from the normalization layer; And a cover type setting module (not shown) configured to set the traffic mark type having the largest weight as the recognized traffic mark type.

The various units described in the apparatus 600 and the respective steps in the method described in Fig. 1 correspond to each other, and the various units described in the apparatus 700 and the respective steps in the method described in Fig. 2 correspond to each other. ) And the steps described in Fig. 3 correspond to each other, and it should be understood that the various units described in the apparatus 900 and the steps in the method described in Fig. 4 correspond to each other. Thus, in the above description, the operations and features described with respect to the recognition method of the traffic sign are equally applicable to the apparatus 600 and the units included therein, and in the above description, a method of training a pre- Operations and features are equally applied to the apparatus 700 and the units included therein, and the operations and features described with respect to the method of obtaining the traffic sign window image to be identified in the above description are equally applicable to the apparatus 800 and The operations and features described with respect to the method of training the convolutional neural network model applied in the above description to the previously described units apply equally to the apparatus 900 and the units included therein, It will be omitted. Corresponding units in devices 600, 700, 800, and 900 may cooperate with units in a terminal device and / or server to implement the means of an embodiment of the present invention.

The modules referred to in the embodiments of the present invention may be realized in the manner of software or in a hardware manner. The described module may be implemented in a processor, for example, a processor including a feature value acquisition module, a detection module and a recognition module. Here, the names of these modules do not constitute a limitation on the module itself in some cases, for example, the feature value acquisition module may acquire the panoramic spherical image segment acquired with the feature algorithm of the predefined integral channel " A module configured to acquire the feature value of the scanned window image. &Quot;

10 is a structural schematic diagram of a computer system 700 applied to the realization of the device of an embodiment of the present invention.

10, the computer system 1000 includes a central processing unit (CPU) 1001 that reads a program stored in a read only memory device (ROM) 1002 or a random access Various suitable operations and processes can be performed by the program loaded into the memory device (RAM) 1003. The RAM 1003 further stores various programs and data necessary for operating the system 1000. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus line 1004. An input / output (I / O) interface 1005 is also connected to the bus line 1004.

The input unit 1006 includes a keyboard, a mouse, and the like. The output unit 1006 includes a cathode ray tube (CRT), a liquid crystal display (LCD) A storage unit 1008 including a hard disk or the like, and a communication unit 1009 including a network interface card such as a LAN card or a modem. The communication unit 1009 executes communication processing through a network such as the Internet. The driving unit 1010 is also connected to the I / O interface 1005 according to demand. A removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory device or the like is connected to the driving unit 1010 in accordance with demand to install the computer program read from the driving unit 1010 in the storage unit 1008, Respectively.

In particular, and in accordance with the embodiments of the present disclosure, the processes described with reference to the flowcharts may be realized with computer software programs. For example, an embodiment of the present disclosure includes a computer program product and tangibly embodied in a machine-readable medium, the computer program comprising program code for executing the method shown in the flowchart . In this embodiment, the computer program may be downloaded and installed from the network via the communication unit 1009, and / or installed from the removable medium 1011. [

The flowcharts and block diagrams in the accompanying drawings illustrate the system structures, functions, and operations of the systems, methods, and computer program products according to the embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a portion of a code, and the module, program segment, or portion of code may include one or more Executable instructions. It should be noted that in some alternative implementations, the functions indicated in the blocks may occur in an order different from the order indicated in the accompanying drawings. For example, two sequentially displayed blocks may actually be executed at the same time on a basic basis, and in some cases in reverse order, depending on the function involved. Each block and / or block diagram in the block diagrams and / or flowchart diagrams and / or combinations of blocks in the flowchart illustrations may be embodied in a system based on hardware dedicated to a prescribed function or operation, or in a combination of dedicated hardware and computer instructions .

In another aspect, the present application also provides non-volatile computer storage media, which may be non-volatile computer storage media included in the apparatus of the embodiments described above, Non-volatile computer storage media. The non-volatile computer storage media stores one or more programs, and the one or more programs are executed by one or more processors to execute the method of recognizing the traffic signs described herein.

The foregoing description is only a description of the comparative preferred embodiments of the present application and the technical principles in operation. It will be understood by those skilled in the art that the scope of the present invention is not limited to the technical solutions made up of specific combinations of the technical features but can be applied to any combination of the technical features or their equivalent features without departing from the gist of the invention , And other technical measures, including, for example, technical features that have been replaced with technical features having similar features as those disclosed in this application, but which are not limited thereto.

Claims (16)

Obtaining a feature value of the scanning window image obtained with the panoramic spherical image segment acquired with the feature algorithm of the default in the default integral channel;
Detecting a scanning window image and acquiring a traffic sign window image to be verified according to a pre-trained detection classifier model trained according to a characteristic value of the scanning window image, a sample of the scanning window image, and a characteristic value thereof; And
And acquiring a traffic sign type by recognizing the traffic sign window image to be verified according to a sample of the traffic sign window image already confirmed and a pre-trained convolution neural network model trained according to the traffic sign type thereof. A method of recognizing traffic signs. The method according to claim 1,
Wherein the pre-trained detection classifier model comprises:
A negative sample including a positive sample including a window image of a preset pixel that includes a window image of a traffic sign among the samples of the scanning window image or a traffic window and a window of the predetermined pixel excluding the positive sample is obtained ,
Acquiring feature values of the positive sample and the negative sample acquired with a predetermined feature algorithm in the predefined integral channel,
Training is performed on the detected classifier model using a boosting algorithm according to the feature values obtained by the feature algorithm of the default in the positive sample and the negative sample and in the predefined integral channel of the predefined detection classifier model, Wherein the traffic information is obtained by acquiring the traffic sign. The method according to claim 1,
Obtaining the feature value of the scanning window image obtained with the panoramic spherical image segment obtained with the feature algorithm of the default in the default integral channel,
Obtaining an integral image of the scanning window image acquired with the integral image segment of the panoramic spherical image in the predefined integral channel; And
And acquiring a feature value of the scanning window image according to an integral image of the scanning window image. The method of claim 3,
Wherein acquiring an integral image of a scanning window image acquired with an integral image segment of a panoramic spherical image in the predefined integral channel comprises:
Continuing downsampling on the panoramic spherical image to obtain an image pyramid; And
Acquiring an integral image of a scanning window image of each level image in an image pyramid acquired with an integral image segment of the image pyramid in a predefined integral channel,
Wherein acquiring the feature value of the scanning window image according to an integral image of the scanning window image comprises:
Obtaining a feature value of a scanning window image of each level image according to an integral image of the scanning window image of each level image,
The step of acquiring a traffic sign window image to detect and confirm a scanning window image according to the characteristic values of the scanning window image and the pre-trained detection classifier model,
Detecting a scanning window image of each level image according to a feature value of the scanning window image of each level image and a pre-trained detection classifier model to obtain a traffic sign window image to be checked A method of recognizing traffic signs. 5. The method according to any one of claims 1 to 4,
The pre-trained convolution neural network model
The weights of both the convolution layer and the fully connected layer of the predetermined convolution neural network model including the sequentially connected convolution layer, the extraction layer, the perfect connection layer and the normalization layer are initialized according to the Gaussian distribution,
Iterating the weights of both the convolution layer and the fully connected layer with an error back propagation BP algorithm according to a sample of the traffic sign window image already confirmed and its traffic sign type, If the difference value of the previously repeated weights is smaller than the preset value, the current repeated weight is determined as the optimum weight, the convolution neural network model including the optimum weight is set to the pre-trained convolution neural network model, And a convergence neural network model including the optimal weight is set to the pre-trained convolution neural network model when the lowest iteration weight is found, the optimal weights are determined as the repeated weights having the lowest error rates, How to recognize the cover. 6. The method of claim 5,
Wherein the predetermined convolution neural network model further comprises a loss function layer,
The iteration of the weights of both the convolution layer and the fully-connected layer with the error backpropagation BP algorithm is performed by using a loss function and a BP algorithm with respect to the weights of the convolution layer and the fully- The method comprising the steps of: performing an iteration of each traffic mark. 6. The method of claim 5,
The step of acquiring a traffic sign type by recognizing a traffic sign window image to be verified according to the pre-trained convolution neural network model includes:
Inputting the traffic sign window image to be verified into the convolutional neural network model and obtaining a traffic sign type having the largest weight output from the normalization layer; And
And setting the traffic mark type having the largest weight as the recognized traffic mark type. A feature value acquisition module configured to acquire a feature value of a scanning window image obtained with a panoramic spherical image segment acquired with a feature algorithm of the default in the default integral channel;
A traffic sign window configured to detect and verify a scanning window image according to a pre-trained detection classifier model trained according to a characteristic value of the scanning window image, a sample of the scanning window image, and a characteristic value thereof, Detection module; And
A traffic sign recognition module configured to recognize a traffic sign window image to be verified according to a pre-trained convolution neural network model trained according to a sample of the traffic sign window image already confirmed and its traffic sign type, Wherein the traffic sign recognition apparatus comprises: 9. The method of claim 8,
Wherein the pre-trained detection classifier model comprises:
A negative sample including a positive sample including a window image of a preset pixel that includes a window image of a traffic sign among the samples of the scanning window image or a traffic window and a window of the predetermined pixel excluding the positive sample is obtained ,
Acquiring characteristic values of a positive sample and a negative sample acquired with a predetermined characteristic algorithm in the default integral channel,
Training is performed on the detected classifier model using a boosting algorithm according to the feature values obtained by the feature algorithm of the default in the positive sample and the negative sample and in the predefined integral channel of the predefined detection classifier model, Wherein the traffic information is acquired by acquiring the traffic sign. 9. The method of claim 8,
Wherein the feature value acquisition module comprises:
An integral image acquisition first sub-module configured to obtain an integral image of a scanning window image acquired with an integral image segment of a panoramic spherical image in a predetermined integral channel; And
And acquiring a feature value of the scanning window image according to an integral image of the scanning window image. 11. The method of claim 10,
The first submodule for obtaining an integral image includes:
A downsampling sub-module configured to perform successive downsampling on the panoramic spherical image to obtain an image pyramid; And
An integral image acquisition second submodule configured to acquire an integral image of a scanning window image of each level image in an image pyramid acquired with an integral image partition of the image pyramid in a predetermined integral channel,
Wherein the feature value acquisition first sub-
And a feature value second obtaining submodule configured to obtain a feature value of a scanning window image of each level image according to an integral image of the scanning window image of each level image,
Wherein the detection module comprises:
And a multi-level detection sub-module configured to detect and obtain a traffic sign window image to detect and scan the scanning window image of each level image according to the feature value of the scanning window image of each level image and the pre-trained detection classifier model Wherein the traffic sign recognition apparatus comprises: 12. The method according to any one of claims 8 to 11,
The pre-trained convolution neural network model comprises:
The weights of both the convolution layer and the fully connected layer of the predetermined convolution neural network model including the sequentially connected convolution layer, the extraction layer, the perfect connection layer and the normalization layer are initialized according to the Gaussian distribution,
Iterating the weights of both the convolution layer and the fully connected layer with an error back propagation BP algorithm according to a sample of the traffic sign window image already confirmed and its traffic sign type, If the difference value of the previously repeated weights is smaller than the preset value, the current repeated weight is determined as the optimum weight, the convolution neural network model including the optimum weight is set to the pre-trained convolution neural network model, And a convergence neural network model including the optimal weight is set to the pre-trained convolution neural network model when the lowest iteration weight is found, the optimal weights are determined as the repeated weights having the lowest error rates, Identification device of the cover. 13. The method of claim 12,
Wherein the predetermined convolution neural network model further comprises a loss function layer,
Wherein the weight determination module is further configured to perform iteration on the weights of both the convolution layer and the fully connected layer with a loss function and a BP algorithm. 13. The method of claim 12,
The traffic sign recognition module includes:
A maximum weight recognition module configured to input the traffic sign window image to be verified into the convolution neural network model and obtain a traffic sign type having the largest weight output from the normalization layer; And
And a cover type setting module configured to set the traffic mark type having the largest weight as the recognized traffic mark type. A processor; And
Memory,
Wherein the memory stores computer readable instructions that can be executed by the processor and, when the computer readable instructions are executed, the processor performs the method according to any one of claims 1 to 7 . A non-volatile computer storage medium having stored thereon computer-readable instructions that can be executed by a processor,
Wherein when the computer-readable instructions are executed by a processor, the processor performs the method according to any one of claims 1-7.

Images