WO2018028230A1

WO2018028230A1 - Deep learning-based method and device for segmenting vehicle license plate characters, and storage medium

Info

Publication number: WO2018028230A1
Application number: PCT/CN2017/080128
Authority: WO
Inventors: 谷爱国; 温炜; 许健; 李岩; 万定锐
Original assignee: 东方网力科技股份有限公司
Priority date: 2016-08-10
Filing date: 2017-04-11
Publication date: 2018-02-15
Also published as: CN106295646B; CN106295646A

Abstract

Provided in the present invention are a deep learning-based method and device for segmenting vehicle license plate characters, and a storage medium. The method comprises: acquiring the original image of a vehicle license plate, and labeling the original image to obtain a labeled image; constructing a neural network on the basis of the original image and the labeled image; on the basis of the neural network and loss function of softmax regression, classifying regions of the original image to obtain a classified image; comparing the classified image with the labeled image to determine whether the classified image and the labeled image are consistent; if not, performing training on the basis of the classified image to obtain the neural network model.

Description

License plate character segmentation method, device and storage medium based on deep learning

Cross-reference to related applications

The present application is based on a Chinese patent application filed on Apr. 10, 2016, the entire disclosure of which is hereby incorporated by reference.

Technical field

The present invention relates to the field of data identification, and in particular, to a method, device and storage medium for license plate character segmentation based on deep learning.

Background technique

License plate recognition is commonly used in bayonet, electric police, toll stations and parking lots. The traditional license plate recognition algorithm includes license plate character segmentation. The license plate character segmentation technology divides the license plate image area to obtain all the independent character regions on the license plate image, mainly for horizontal projection and vertical projection of the license plate, specifically: for the license plate The image is preprocessed to obtain a binarized image; the top-down progressive scan of the license plate image and the bottom-up progressive scan are performed to obtain the height range of the license plate characters; the height range of the license plate characters is scanned from left to right to determine each The range of widths of characters; the top-down and bottom-up progressive scans of each character's width range to obtain a more precise height range for each character.

However, with the development of society and the need to maintain social security, there are more and more surveillance cameras installed in the city. The installation height and angle of surveillance cameras have far exceeded the installation standards required by the intelligent transportation and security industries. The imaging quality of surveillance cameras is also One of the more prominent problems is that the license plate that needs to be identified is getting smaller and smaller, and because of the image quality problem, the license plate is ambiguous, the characters are stuck or contaminated by noise, and there will be interference between characters. The traditional segmentation algorithm The split position between characters cannot be found by horizontal projection and vertical projection, so it cannot Solve the above problems well.

Summary of the invention

An object of the present invention is to provide a method, a device and a storage medium for license plate character segmentation based on deep learning, which can perform effective character segmentation on a license plate with characters stuck, a license plate with a large noise pollution, and a defaced license plate, thereby obtaining more Accurate license plate character area to improve the accuracy of license plate character segmentation.

In a first aspect, an embodiment of the present invention provides a method for segmentation of a license plate character based on deep learning, the method comprising:

Obtaining an original image of the license plate, and marking the original image to obtain a label image;

Constructing a neural network according to the original image and the label image;

And classifying the regions in the original image to obtain a classified image based on the neural network and a softmax regression loss function;

Comparing the classified image with the label image to determine whether the classified image is consistent with the label image; if not, training based on the classified image to obtain a neural network model.

In the above solution, the method further includes: analyzing, according to the neural network model, an image of the license plate to be analyzed, to obtain a label image corresponding to the license plate to be analyzed.

In the above solution, the image of the license plate to be analyzed is analyzed based on the neural network model to obtain the label image corresponding to the license plate to be analyzed, including:

Passing the image of the license plate to be analyzed through the neural network model to obtain a probability of a first tag value corresponding to each pixel of the image of the license plate to be analyzed and a probability of a second tag value;

And processing, according to the probability of the first label value corresponding to each pixel and the probability of the second label value, the image of the license plate to be analyzed to obtain a mask image;

The mask image is processed to obtain a label image of the license plate to be analyzed.

In the above solution, the probability and the first label value corresponding to each pixel are The probability of the second tag value is processed by the image of the license plate to be analyzed to obtain a mask image, including:

The mask image is calculated according to the following formula:

The probability of the first tag value corresponding to the pixels of the i-th row and the j-th column, P ₀ (i, j) is the number corresponding to the pixels of the i-th row and the j-th column of the image of the license plate to be analyzed The probability of the two-tag value, i=1, 2, 3...M, j=1, 2, 3...N, the M is the height of the image of the license plate to be analyzed, and the N is the license plate to be analyzed The width of the image.

In the above solution, the mask image is processed to obtain a label image of the license plate to be analyzed, including:

Counting, in each column of the mask image, a pixel value of a pixel value of the first pixel value;

Determining, according to the number, a division position between characters on the license plate to be analyzed;

And obtaining a label image of the image of the license plate to be analyzed according to the division position between the characters on the license plate to be analyzed.

In the above solution, determining, according to the number, a division position between characters on the license plate to be analyzed includes:

Comparing the value corresponding to the number with a preset threshold;

If the value corresponding to the number is greater than a preset threshold, the value corresponding to the number is used as the split position of the currently adjacent character.

In the above solution, the marking the original image to obtain a label image comprises:

Marking an area between adjacent characters of the original image as a first tag value;

Marking other areas than the area between the adjacent characters as the second tag value;

An area marked as the first tag value and an area marked as the second tag value constitute the tag image.

In a second aspect, an embodiment of the present invention provides a license plate character segmentation device based on deep learning, and the device includes:

a marking unit, configured to acquire an original image of the license plate, and mark the original image to obtain a label image;

a constructing unit, configured to construct a neural network according to the original image and the label image;

a classified image generating unit, configured to classify the regions in the original image to obtain a classified image based on the neural network and a softmax regression loss function;

a determining unit, configured to compare the classified image with the label image, and determine whether the classified image is consistent with the label image;

a training unit, configured to perform a neural network model based on the classified image if not inconsistent.

In the above solution, the device further includes:

The label image generating unit is configured to analyze an image of the license plate to be analyzed based on the neural network model to obtain a label image corresponding to the license plate to be analyzed.

In the above solution, the label image generating unit includes:

a probability acquisition unit, configured to pass the image of the license plate to be analyzed to the neural network model, to obtain a probability of a first label value and a probability of a second label value corresponding to each pixel of the image of the license plate to be analyzed;

a first processing unit, configured to process, according to a probability of the first label value corresponding to each pixel and a probability of the second label value, an image of the license plate to be analyzed to obtain a mask image;

And a second processing unit, configured to process the mask image to obtain a label image of the license plate to be analyzed.

In the above solution, the first processing unit includes:

The mask image is calculated according to the following formula:

Wherein said

For the mask image, P ₁ (i, j) is the probability of the first tag value corresponding to the pixels of the i-th row and the j-th column of the image of the license plate to be analyzed, P ₀ (i, j) a probability of the second tag value corresponding to the pixels of the i-th row and the j-th column of the image of the license plate to be analyzed, i=1, 2, 3...M, j=1, 2, 3...N, M is the height of the image of the license plate to be analyzed, and N is the width of the image of the license plate to be analyzed.

In the above solution, the second processing unit includes:

a statistical unit, configured to count the number of pixels in each column of the mask image whose pixel value is the first pixel value;

a determining unit, configured to determine, according to the number, a split position between characters on the license plate to be analyzed;

And a label image acquiring unit, configured to obtain a label image of the image of the license plate to be analyzed according to the split position between the characters on the license plate to be analyzed.

In the above solution, the determining unit includes:

a comparing unit, configured to compare the value corresponding to the number with a preset threshold;

The segmentation position determining unit is configured to use the value corresponding to the number as the segmentation position of the currently adjacent character in a case where the value corresponding to the number is greater than a preset threshold.

In the above solution, the marking unit comprises:

a first tag value marking unit, configured to mark an area between adjacent characters of the original image as a first tag value;

a second tag value marking unit, configured to mark other areas than the area between the adjacent characters as a second tag value;

And a constituting unit configured to form the label image as an area marked as the first label value and an area marked as the second label value.

In a second aspect, an embodiment of the present invention provides a storage medium, where the storage medium includes a set of instructions, when executed, causing at least one processor to perform an operation including: obtaining Obtaining an original image of the license plate, marking the original image to obtain a label image;

The embodiment of the invention provides a method, a device and a storage medium for license plate character segmentation based on deep learning. The original image is marked to obtain a label image, and the neural network is constructed according to the original image and the label image, based on the neural network and softmax regression. a loss function, classifying the regions in the original image to obtain a classified image, and comparing the classified image with the label image. If the classified image is inconsistent with the label image, the classified image is trained to obtain a neural network. The network model obtains the label image from the original image through the neural network model, thereby obtaining a more accurate license plate character area and improving the accuracy of the license plate character segmentation.

DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly described below. It should be understood that the following drawings show only certain embodiments of the present invention, and therefore It should be seen as a limitation on the scope, and those skilled in the art can obtain other related drawings according to these drawings without any creative work.

FIG. 1 is a flowchart of a method for segmentation of a license plate character based on deep learning according to Embodiment 1 of the present invention; FIG.

2 is a schematic diagram of a structural neural network according to Embodiment 1 of the present invention;

3 is a schematic diagram of a neural network prediction network corresponding to FIG. 2 according to Embodiment 1 of the present invention;

FIG. 4 is a schematic diagram of a license plate character segmentation method based on deep learning according to Embodiment 1 of the present invention. The flowchart of step S106;

FIG. 5 is a flowchart of step S106 in another method for segmentation of a license plate character based on deep learning according to Embodiment 1 of the present invention;

FIG. 6 is a flowchart of step S101 in a method for segmenting license plate characters based on deep learning according to Embodiment 1 of the present invention;

FIG. 7 is a schematic diagram of a license plate character segmentation apparatus based on deep learning according to Embodiment 2 of the present invention.

Description of the reference signs:

10-label unit; 20-construction unit; 30-classified image generation unit;

40-judging unit; 50-training unit; 60-tag image generating unit.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is a partial embodiment of the invention, and not all of the embodiments. The components of the embodiments of the invention, which are generally described and illustrated in the figures herein, may be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention in the claims All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

For the existing license plate character segmentation technology, the horizontal projection and the vertical projection method are used to segment the characters, but the license plate for character sticking, staining and noise pollution cannot be divided. The embodiment of the invention provides a method, a device and a storage medium for license plate character segmentation based on deep learning. The original image is marked to obtain a label image, and the neural network is constructed according to the original image and the label image, based on the neural network and softmax regression. a loss function, classifying the regions in the original image to obtain a classified image, and classifying the image and the label map For comparison, if the classified image is inconsistent with the label image, the classified image is trained to obtain a neural network model, and the original image is obtained through the neural network model to obtain a label image, so that the license plate and noise pollution of the character adhesion can be greatly increased. The license plate and the defaced license plate are effectively character-divided to obtain a more accurate license plate character area and improve the accuracy of the license plate character segmentation. The details are described below by way of examples.

FIG. 1 is a flowchart of a method for segmentation of license plate characters based on deep learning according to an embodiment of the present invention.

Referring to FIG. 1, step S101, acquiring an original image of a license plate, and marking the original image to obtain a label image;

Specifically, the original image of the license plate is first acquired, the area between adjacent characters on the original image of the license plate is marked as a first label value, and the other area on the original image of the license plate is marked as a second label value, wherein A tag value is 1 and a second tag value is 0. Then, the area marked with 1 and the area marked with 0 constitute a label image.

In this way, each license plate image constitutes two corresponding images, which are the original image of the license plate and the label image. For example, the original image of the license plate is “Jing C·874”, and the adjacent characters “Beijing” and “C” The area between the areas is marked as 1, and so on, and the area other than the adjacent characters is marked as 0, thereby constituting the label image.

Step S102, constructing a neural network according to the original image and the label image;

For details, refer to the schematic diagram of constructing a neural network as shown in FIG. 2. The neural network is seven layers, each layer includes a convolution layer and an activation layer, and the original image and the label image are sequentially passed through the layers to form a neural network.

Step S103, classifying the regions in the original image to obtain a classified image based on the neural network and a softmax regression loss function;

Here, in the above step S102, the original image and the tag image construct a neural network, and then divide the region in the original image based on the neural network and the softmax regression loss function. The class is classified as an image. For details, refer to FIG. 3.

The area in the original image may be classified by: classifying at least one area in the original image, and classifying the area as 1 at the division position (that is, the label is 1), and other positions are classified. A value of 0, the classified image contains only two values of 0 and 1. Further, at least one position may exist in the original image, and each position may be measured by a pixel, for example, one pixel may correspond to one position, or one pixel may correspond to one position.

In step S104, the classified image is compared with the label image to determine whether the classified image and the label image are consistent. If not, step S105 is performed; if they are consistent, step S107 is performed.

Step S105, training the classified image to obtain a neural network model;

Here, the classified image is compared with the label image to determine whether the classified image matches the label image. If there is no match, the classified image needs to be trained to obtain a neural network model.

So far, the present embodiment has established a neural network model based on the original image. In addition, it can be understood that the number of original images is not limited in this embodiment. In actual processing, the original images of multiple license plates may be used for training to obtain a neural network model.

In addition, it can be understood that, after completing the foregoing step S105, the embodiment may further input the original image of the license plate into the neural network model to obtain the to-be-aligned label image calculated by the original image through the neural network model;

Comparing the tag image to be compared with the tag image of the original image of the license plate obtained in step S101, if the two match, the neural network model can be considered to be successful; if not, the original image of more license plates can be continued. The neural network model is trained until the to-be-aligned label image obtained by training the network model matches the label image.

In addition, the matching of the foregoing two label images can be measured by the degree of matching, for example, The preset values may be reached for the same value of the same region in the two label images. For example, if 99% of the same is present, the two label images may be considered to match. The preset value can be set according to the actual situation, and is not exhaustive in this embodiment.

Based on the foregoing steps to establish a neural network model, as shown in FIG. 1, the following processing is included:

Step S106, analyzing an image of the license plate to be analyzed based on the neural network model to obtain a label image corresponding to the license plate to be analyzed;

Here, the image of the license plate to be analyzed outputs the probability of the first label value corresponding to the pixel of the original image and the probability of the second label value through the neural network model, and then processes the mask image of the image to be analyzed, and finally performs the mask image. Post-processing obtains the label image of the license plate to be analyzed.

The foregoing is to use the neural network model established in step S105 to analyze other license plates to obtain a processing flow of the tag images of other license plates.

Step S107, ending.

Further, as shown in FIG. 4, in the method for segmentation of the license plate character based on the deep learning, the step S106 may be implemented by the following steps, including:

Step S201, the image of the license plate to be analyzed is passed through a neural network model, and the probability of the first label value corresponding to each pixel of the image of the license plate to be analyzed and the probability of the second label value are obtained;

Here, by obtaining the neural network model, and then inputting the image of the license plate to be analyzed into the neural network model, the probability of the first tag value corresponding to each pixel of the image of the license plate to be analyzed and the probability of the second tag value are finally output.

Wherein, the image of the license plate to be analyzed may be represented by H, and H(i, j) represents the i-th row of the image H of the license plate to be analyzed, and the pixel value of the j-th column, i=1, 2, 3, ..., M, j= 1, 2, 3, ..., N, M represents the height of the image H of the license plate to be analyzed, and N represents the width of the image H of the license plate to be analyzed.

The first tag value may be 1, and the second tag value may be 0, that is, the probability that the tag value corresponding to each pixel of the image of the license plate to be analyzed is 1 and the tag value is 0. Label value is 1 The probability is represented by P1, where P1(i,j) represents the i-th row of the image of the license plate to be analyzed, and the probability of the tag value corresponding to the pixel of the jth column is 1; the probability of the tag value of 0 is represented by P0, where P0 (i, j) represents the probability that the pixel value of the pixel of the jth column corresponds to the i-th row of the image of the license plate to be analyzed. Thus, the probability that the label value of each pixel of the image of the license plate to be analyzed corresponds to 1 and the probability that the label value is 0, and the sum of their probabilities is 1, as shown in formula (1):

P ₀ (i,j)+P ₁ (i,j)=1 (1)

Wherein, the label value of 1 represents a gap between adjacent characters of the license plate, and the label value of 0 represents other areas except the gap.

Step S202, processing the image of the license plate to be analyzed according to the probability of the first label value corresponding to each pixel and the probability of the second label value to obtain a mask image;

Here, according to the probability that the label value corresponding to each pixel of the image of the license plate to be analyzed is 1 and the probability that the label value is 0, the image of the license plate is processed to obtain a mask image.

Mask image

The width and height of the image H of the license plate to be analyzed are equal, and a mask image is obtained.

The method is as shown in formula (2):

among them,

For the mask image, P ₁ (i, j) is the probability of the first label value corresponding to the pixels of the i-th row and the j-th column of the image of the license plate to be analyzed, and P ₀ (i, j) is the image of the license plate to be analyzed. The probability of the second tag value corresponding to the pixels of the i-th row and the j-th column, i=1, 2, 3...M, j=1, 2, 3...N, M is the height of the image of the license plate to be analyzed, N The width of the image of the license plate to be analyzed.

Step S203, processing the mask image to obtain a label image of the license plate to be analyzed.

Specifically, as shown in FIG. 5, in the method for segmenting the license plate character based on the deep learning, the step S203 may be implemented by the following steps, including:

Step S301, counting the pixels in each column of the mask image whose pixel value is the first pixel value. number;

Here, the first pixel value is 1, through the statistical mask image

The number of pixels in each column is 1, and the number can be represented by T1(j), where j=1, 2, 3, ..., N, N represents the width of the original image H.

Step S302, determining, according to the number, a division position between characters on the license plate to be analyzed;

Specifically, the value corresponding to the number is compared with a preset threshold; if the value corresponding to the number is greater than the preset threshold, the value corresponding to the number is used as the segmentation position of the currently adjacent character.

Here, the preset threshold is represented by threshold, and the specific process is:

When T1(j)>threshold, j is taken as the division position of the current adjacent character, where threshold=a*M, M represents the mask image

The height, a takes 0.2.

Here, the value of a is determined by a large number of experiments, and can also be obtained by a statistical method. When a is determined to a certain value, the error between the segmentation position obtained by the above method and the actual segmentation position of the character on the license plate is minimized. It is possible to determine the optimal value of the current value a.

Step S303, obtaining a label image according to the division position between the characters on the license plate to be analyzed.

Further, as shown in FIG. 6 , in the method for segmentation of the license plate character based on the deep learning, the step S101 may be implemented by the following steps, including:

Step S401, marking an area between adjacent characters of the original image as a first label value;

Here, the first tag value is 1, and the area between adjacent characters in the original image is marked as 1.

Step S402, marking other areas except the area between adjacent characters as the second label value;

Here, the second tag value is 0, and in the original image, other areas are marked as 0 except for adjacent characters.

In step S403, the area marked as the first label value and the area marked as the second label value constitute a label image.

Here, the area between adjacent characters marked 1 and the area marked 0 constitute a label image.

Through this specific marking, the rules of the characters on the license plate can be automatically learned during the neural network learning process, such as the existence of gaps and character contours between characters, and the license plate for character sticking, character staining and noise pollution is effectively segmented.

The invention provides a license plate character segmentation method based on deep learning, which obtains a tag image by marking an original image, constructs a neural network according to the original image and the tag image, and based on the neural network and a softmax regression loss function, the original The regions in the image are classified to obtain a classified image, and the classified image is compared with the label image. If the classified image is inconsistent with the label image, the classified image is trained to obtain a neural network model, and the original image is passed. The neural network model obtains the label image to obtain a more accurate license plate character area and improve the accuracy of license plate character segmentation.

Referring to FIG. 7, the apparatus includes a marking unit 10, a construction unit 20, a classified image generation unit 30, a determination unit 40, a training unit 50, and a label image generation unit 60.

a marking unit 10, configured to acquire an original image of the license plate, and mark the original image to obtain a label image;

The constructing unit 20 is configured to construct a neural network according to the original image and the label image;

The classified image generating unit 30 is configured to classify the regions in the original image to obtain a classified image based on the neural network and the softmax regression loss function;

The determining unit 40 is configured to compare the classified image with the label image, and determine whether the classified image is consistent with the label image;

The training unit 50 is configured to perform a neural network model based on the classified image after the inconsistency.

So far, the present embodiment has established a neural network model based on the original image. In addition, It is understood that the number of original images is not limited in this embodiment. In actual processing, the original images of multiple license plates may be used for training to obtain a neural network model.

In addition, it can also be understood that the original image of the license plate can also be input into the neural network model to obtain the to-be-aligned label image calculated by the original image through the neural network model;

The comparison between the image of the label to be compared with the image of the original image of the obtained license plate is used. If the two match, the neural network model can be considered to be successful; if not, the original image of more license plates can be used to train the nerve. The network model until the to-be-aligned tag image obtained by training the network model matches the tag image.

In addition, the matching of the foregoing two label images may be measured by the degree of matching. For example, the ratio of the same label value corresponding to the same area in the two label images may reach a preset value, for example, 99% of the same, It can be considered that the two tag images match. The preset value can be set according to the actual situation, and is not exhaustive in this embodiment.

Based on the foregoing establishment of the neural network model, the method may further include:

The label image generating unit 60 is configured to analyze an image of the license plate to be analyzed based on the neural network model to obtain a label image corresponding to the license plate to be analyzed.

Further, the tag image generating unit 60 includes:

a probability acquisition unit (not shown), configured to pass an image of the license plate to be analyzed to a neural network model, and obtain a probability of a first tag value corresponding to each pixel of the image of the license plate to be analyzed and a probability of the second tag value;

a first processing unit (not shown), configured to process an image of the license plate to be analyzed according to a probability of a first label value corresponding to each pixel and a probability of a second label value to obtain a mask image;

A second processing unit (not shown) is configured to process the mask image to obtain a label image of the license plate to be analyzed.

Further, the first processing unit (not shown) includes:

Calculating a mask image according to formula (2), wherein

For the mask image, P ₁ (i, j) is the probability of the first tag value corresponding to the pixels of the i-th row and the j-th column of the image of the license plate to be analyzed, and P ₀ (i, j) is the license plate to be analyzed. The probability of the second tag value corresponding to the pixels of the i-th row and the j-th column of the image, i=1, 2, 3...M, j=1, 2, 3...N, M is the height of the image of the license plate to be analyzed , N is the width of the image of the license plate to be analyzed.

Further, the second processing unit (not shown) includes:

a statistical unit (not shown) for counting the number of pixels in each column of the mask image whose pixel value is the first pixel value;

Determining a unit (not shown) for determining a division position between characters on the license plate to be analyzed according to the number;

A label image acquisition unit (not shown) for obtaining a label image of the license plate to be analyzed according to the division position between the characters on the license plate to be analyzed.

Further, the determining unit (not shown) includes:

a comparison unit (not shown) for comparing the value corresponding to the number with a preset threshold;

The split position determining unit (not shown) is configured to use the value corresponding to the number as the split position of the currently adjacent character if the number corresponding to the number is greater than a preset threshold.

Further, the marking unit 10 comprises:

a first tag value marking unit (not shown) for marking an area between adjacent characters of the original image as the first tag value;

a second tag value marking unit (not shown) for marking other areas than the area between adjacent characters as the second tag value;

A constituent unit (not shown) for constituting the area marked as the first label value and the area marked as the second label value constitute the label image.

The invention provides a license plate character segmentation device based on deep learning, which obtains a tag image by marking an original image, constructs a neural network according to the original image and the tag image, and based on the neural network and a softmax regression loss function, the original Classification of areas in the image The classified image is obtained, and the classified image is compared with the label image. If the classified image is inconsistent with the label image, the classified image is trained to obtain a neural network model, and the original image is obtained through a neural network model to obtain a label image. In order to obtain a more accurate license plate character area, the accuracy of license plate character segmentation is improved.

The device provided by the embodiment of the present invention may be specific hardware on the device or software or firmware installed on the device. The implementation principle and the technical effects of the device provided by the embodiments of the present invention are the same as those of the foregoing method embodiments. For a brief description, where the device embodiment is not mentioned, reference may be made to the corresponding content in the foregoing method embodiments. A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the foregoing system, device and unit can refer to the corresponding processes in the foregoing method embodiments, and details are not described herein again.

In the embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. The device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. Alternatively, the mutual coupling or direct coupling or call connection shown or discussed may be an indirect coupling or a call connection through some of the communication interfaces, devices or units, and may be in electrical, mechanical or other form.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in the embodiment provided by the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

The function is implemented in the form of a software functional unit and sold or made as a standalone product When used, it can be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product, which is stored in a storage medium, including The instructions are used to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.

Embodiments of the present invention also provide a storage medium including a set of instructions that, when executed, cause at least one processor to perform an operation including: acquiring an original image of a license plate, and performing the original image Marking the label image;

The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

It should be noted that similar reference numerals and letters indicate similar items in the following figures. Therefore, once an item is defined in a drawing, it is not necessary to further define and explain it in the subsequent drawings. Moreover, the terms "first", "second", "third", and the like are used merely to distinguish a description, and are not to be construed as indicating or implying a relative importance.

Finally, it should be noted that the above-mentioned embodiments are merely specific embodiments of the present invention, and are used to explain the technical solutions of the present invention, and are not limited thereto, and the scope of protection of the present invention is not limited thereto, although reference is made to the foregoing. EXAMPLES The present invention has been described in detail, and the prior art It should be understood that any person skilled in the art can modify or modify the technical solutions described in the foregoing embodiments within the technical scope of the present invention, or replace some of the technical features. These modifications, variations, and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention. All should be covered by the scope of the present invention. Therefore, the scope of the invention should be determined by the scope of the claims.

Claims

A method for segmentation of license plate characters based on deep learning, the method comprising:

Obtaining an original image of the license plate, and marking the original image to obtain a label image;

Constructing a neural network according to the original image and the label image;

And classifying the regions in the original image to obtain a classified image based on the neural network and a softmax regression loss function;

Comparing the classified image with the label image to determine whether the classified image is consistent with the label image; if not, training based on the classified image to obtain a neural network model.
The method of claim 1 wherein the method further comprises:

Based on the neural network model, an image of the license plate to be analyzed is analyzed to obtain a label image corresponding to the license plate to be analyzed.
The method of claim 2, wherein the analyzing the image of the license plate to be analyzed based on the neural network model to obtain the label image corresponding to the license plate to be analyzed comprises:

Passing the image of the license plate to be analyzed through the neural network model to obtain a probability of a first tag value corresponding to each pixel of the image of the license plate to be analyzed and a probability of a second tag value;

And processing, according to the probability of the first label value corresponding to each pixel and the probability of the second label value, the image of the license plate to be analyzed to obtain a mask image;

The mask image is processed to obtain a label image of the license plate to be analyzed.
The method according to claim 3, wherein the image of the license plate to be analyzed is processed according to a probability of the first tag value corresponding to each pixel and a probability of the second tag value Mask images, including:

The mask image is calculated according to the following formula:

Wherein said
For the mask image, P 1 (i, j) is the probability of the first tag value corresponding to the pixels of the i-th row and the j-th column of the image of the license plate to be analyzed, P 0 (i, j) a probability of the second tag value corresponding to the pixels of the i-th row and the j-th column of the image of the license plate to be analyzed, i=1, 2, 3...M, j=1, 2, 3...N, M is the height of the image of the license plate to be analyzed, and N is the width of the image of the license plate to be analyzed.
The method according to claim 3, wherein the processing the mask image to obtain the label image of the license plate to be analyzed comprises:

Counting, in each column of the mask image, a pixel value of a pixel value of the first pixel value;

Determining, according to the number, a division position between characters on the license plate to be analyzed;

And obtaining a label image of the image of the license plate to be analyzed according to the division position between the characters on the license plate to be analyzed.
The method according to claim 5, wherein said determining, according to said number, a division position between characters on said license plate to be analyzed comprises:

Comparing the value corresponding to the number with a preset threshold;

If the value corresponding to the number is greater than a preset threshold, the value corresponding to the number is used as the split position of the currently adjacent character.
The method of claim 1 wherein said marking said original image to obtain a label image comprises:

Marking an area between adjacent characters of the original image as a first tag value;

Marking other areas than the area between the adjacent characters as the second tag value;

An area marked as the first tag value and an area marked as the second tag value constitute the tag image.
A license plate character segmentation device based on deep learning, the device comprising:

a marking unit, configured to acquire an original image of the license plate, and mark the original image to obtain a label image;

a constructing unit, configured to construct a neural network according to the original image and the label image;

a classified image generating unit, configured to classify the regions in the original image to obtain a classified image based on the neural network and a softmax regression loss function;

a determining unit, configured to compare the classified image with the label image, and determine whether the classified image is consistent with the label image;

a training unit, configured to perform a neural network model based on the classified image if not inconsistent.
The apparatus of claim 8 wherein said apparatus further comprises:

The label image generating unit is configured to analyze an image of the license plate to be analyzed based on the neural network model to obtain a label image corresponding to the license plate to be analyzed.
The apparatus according to claim 9, wherein the label image generating unit comprises:

a probability acquisition unit, configured to pass the image of the license plate to be analyzed to the neural network model, to obtain a probability of a first label value and a probability of a second label value corresponding to each pixel of the image of the license plate to be analyzed;

a first processing unit, configured to process, according to a probability of the first label value corresponding to each pixel and a probability of the second label value, an image of the license plate to be analyzed to obtain a mask image;

And a second processing unit, configured to process the mask image to obtain a label image of the license plate to be analyzed.
The apparatus of claim 10 wherein said first processing unit comprises:

The mask image is calculated according to the following formula:

The probability of the first tag value corresponding to the pixels of the i-th row and the j-th column, P 0 (i, j) is the number corresponding to the pixels of the i-th row and the j-th column of the image of the license plate to be analyzed The probability of the two-tag value, i=1, 2, 3...M, j=1, 2, 3...N, the M is the height of the image of the license plate to be analyzed, and the N is the license plate to be analyzed The width of the image.
The apparatus of claim 10 wherein said second processing unit comprises:

a statistical unit, configured to count the number of pixels in each column of the mask image whose pixel value is the first pixel value;

a determining unit, configured to determine, according to the number, a split position between characters on the license plate to be analyzed;

And a label image acquiring unit, configured to obtain a label image of the image of the license plate to be analyzed according to the split position between the characters on the license plate to be analyzed.
The apparatus of claim 12, wherein the determining unit comprises:

a comparing unit, configured to compare the value corresponding to the number with a preset threshold;

The segmentation position determining unit is configured to use the value corresponding to the number as the segmentation position of the currently adjacent character in a case where the value corresponding to the number is greater than a preset threshold.
The apparatus of claim 8 wherein said marking unit comprises:

a first tag value marking unit, configured to mark an area between adjacent characters of the original image as a first tag value;

a second tag value marking unit, configured to mark other areas than the area between the adjacent characters as a second tag value;

And a constituting unit configured to form the label image as an area marked as the first label value and an area marked as the second label value.
A storage medium comprising a set of instructions that, when executed, cause at least one processor to perform an operation comprising: acquiring an original image of a license plate, marking the original image to obtain a label image;

Constructing a neural network according to the original image and the label image;

An area in the original image based on the neural network and a softmax regression loss function Sorting to obtain a classified image;

Comparing the classified image with the label image to determine whether the classified image is consistent with the label image; if not, training based on the classified image to obtain a neural network model.