CN110503716A - A kind of automobile license plate generated data generation method - Google Patents
A kind of automobile license plate generated data generation method Download PDFInfo
- Publication number
- CN110503716A CN110503716A CN201910738871.6A CN201910738871A CN110503716A CN 110503716 A CN110503716 A CN 110503716A CN 201910738871 A CN201910738871 A CN 201910738871A CN 110503716 A CN110503716 A CN 110503716A
- Authority
- CN
- China
- Prior art keywords
- license plate
- data
- character
- model
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/50—Lighting effects
- G06T15/506—Illumination models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/63—Scene text, e.g. street names
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Geometry (AREA)
- Processing Or Creating Images (AREA)
Abstract
The present invention relates to a kind of automobile license plate generated data generation methods, firstly, obtain license plate font on the authority file disclosed in the official and modeled, so that the case where license plate data of synthesis are close to true license plate；Secondly, constructing scene using virtual engine and carrying out comprehensive rendering, mark is automatically generated, the acquisition formation efficiency of data set is improved；Again, diversified license plate formation condition is set from weather, illumination, distance, angle etc., obtains and covers more comprehensive data；Finally, carrying out Style Transfer to synthesis license plate data generated using Style Transfer technology, the difference between artificial synthesized image and true picture domain is reduced.
Description
Technical field
The invention belongs to field of intelligent transportation technology more particularly to a kind of automobile license plate generated data generation methods, can
Car plate detection and identification applied to intelligent transportation system.
Background technique
With the development of Video Supervision Technique and depth learning technology, intelligent transportation system becomes the hair of future transportation system
Phenomena such as opening up direction, the urban traffic blocking persistent ailment increasingly sharpened can be alleviated, reduce break in traffic rules and regulations and pernicious traffic accident,
Strong evidence is provided for all kinds of traffic accidents and the post-processing of personal safety as well as the property safety.And in this process, license plate
Number plays the role of vital as motor vehicle distinctive identity characterization, therefore the detection of license plate and identification are intelligent friendships
The important component of way system.
Research and practice have shown that, the performance of deep learning algorithm depends on the quality and scale of training data, therefore is based on
The Car license recognition of deep learning needs a large amount of high-quality data.In view of the demand of license plate application, license plate data set is wanted in addition to scale
Except sufficiently large, usually there are also claimed below: (1) characters on license plate type should be covered comprehensively, wherein first covering is each provincial
Administrative area, behind several should cover various situations and character distribution should as far as possible uniformly；(2) license plate data include various extreme feelings
Data under condition, such as dense fog, heavy snow weather, for the processing capacity of boosting algorithm in complex situations；(3) have accurate
Semantic tagger, and marked content is abundant as far as possible, such as marked content cover such as license plate area, character zone, Pixel-level semanteme, with
Support the design of all kinds of Recognition Algorithm of License Plate.
There are two main classes method constructs license plate data set at present: one is the method based on artificial acquisition mark, i.e., logical
It crosses candid photograph equipment and obtains true license plate contextual data.This method limitation is stronger, first is that covering all kinds of complexity because obtaining
The license plate data of situation are more difficult, secondly because the artificial mark consumption wealth to data takes time and effort, and are easy to appear mark
The situation of inaccuracy.Another kind is the method generated based on generated data, i.e., by modeling tool and virtual engine to motor vehicle
License plate and traffic scene carry out the rendering of modeling and material illumination, obtain a large amount of data, and computer is recycled to be labeled,
Its consumption relative moderate to manpower financial capacity.Studies have shown that the model by generated data collection training has on truthful data
Preferable migration, therefore license plate data are synthesized with very strong practical value.
In terms of traditional true license plate data acquisition, conventional method is carried out using traffic camera or mobile device
Acquisition proposes a kind of based on the camera shooting being mounted on cruiser in patent " a kind of movable type license plate data collection system "
Head carries out the acquisition systems of video mobile collection license plate data, is acquired to the license plate data in true license plate scene.This
Kind license plate data are authentic and valid, but coverage area is narrow, and collecting efficiency is lower, while there is still a need for artificial mark, time-consuming and laborious.
In terms of the generation of synthesis license plate data, there is presently no correlation techniques, but with the hair of computer graphics
Exhibition, using artificial synthesized data set carry out data extending, thus the training for completing deep learning model had it is very big into
Exhibition.Patent publication No. CN108596259A " a method of for object identification artificial intelligence training dataset generate "
In, article to be identified is generated in simulated environment, and according to Item Title and the label of environment generation object names, to obtain people
Work generated data.In patent publication No. CN109523629A, " a kind of object semanteme and pose data set based on physical simulation are raw
At method " in, point cloud data, which is obtained, using virtual engine rendering application scenarios environment goes forward side by side the marks of the markup informations such as line position appearance.
However there is no a kind of effective ways for license plate data in the generation method of above-mentioned generated data collection.Firstly, vehicle
Board data have the characteristics that itself and specification, and modeling pattern directly determines the using effect of generated data；Secondly, composite number
According to the difference between truthful data will affect learning model to truthful data transfer ability, therefore generated data needs to the greatest extent may be used
Data distribution difference between the reduction and true picture of energy.
Summary of the invention
The technology of the present invention solves the problems, such as: the data volume in order to solve the license plate data set currently used for deep learning training
Less, the problems such as data mark is inaccurate, coverage condition is not comprehensive, provides a kind of automobile license plate generated data generation method, obtains
Cover more comprehensive data；Reduce the difference between artificial synthesized image and true picture domain.
The technology of the present invention solution: a kind of automobile license plate generated data generation method, firstly, advising disclosed in the official
It obtains and license plate font and is modeled on model essay part, so that the case where license plate data of synthesis are close to true license plate；Secondly, utilizing
Virtual engine building scene simultaneously carries out comprehensive rendering, automatically generates mark, improves the acquisition formation efficiency of data set；Again
It is secondary, diversified license plate formation condition is set from weather, illumination, distance, angle etc., obtains the more comprehensive number of covering
According to；Finally, carrying out Style Transfer to synthesis license plate data generated using Style Transfer technology, artificial synthesized image is reduced
With the difference between true picture domain.
The present invention is directed to each type of license plate data, mainly comprises the steps that
Step 1: the corresponding automobile license plate normative document of selection license plate type, according in the specification in the normative document
Hold and three-dimensional modeling is carried out to license plate bottom parts, character portion and license plate ornament in modeling software, while constructing above-mentioned
Model is corresponding to make old model, obtains the threedimensional model of character required for license plate data, bottom plate and character ornament；
Step 2: constructing a variety of different traffic scenes using virtual engine, the environment generated as license plate data；It is described
A variety of different traffic scenes include common crossing, crossroad, parking lot, overhead scene, tunnel scene or High-speed Circumstance；It will
The threedimensional model that step 1 obtains is imported into traffic scene, is arranged according to traffic specification, target traffic required for obtaining
Scene；
Step 3: in the target traffic scene, the different illumination including daytime, noon and night are respectively set, with
And rainy day, fine day, snowy day, greasy weather and sandstorm these different weathers, and rendered using virtual engine, after being rendered
Multiple traffic scenes；
Step 4: in the traffic scene after each rendering, according to the distribution of satisfaction needed for the license plate data set of production, adopting
License plate data content is adjusted with corresponding data generation strategy, generates the license plate data being distributed needed for meeting；
Step 5: each traffic scene comprising license plate data being rendered using virtual engine, the friendship after acquisition rendering
Logical scene image generates corresponding traffic scene labeled data as license plate image data, while using automatic marking tool, with
Semantic segmentation figure indicates；
Step 6: it is executed by step 2-5, circulation generates license plate image data and corresponding traffic scene labeled data, until
The quantity of license plate image data meets the requirement of setting, and all license plate image data constitute the license plate image number converted without style
According to collection；
Step 7: marking number for the corresponding traffic scene of every license plate image data of license plate image data set in step 6
According to, the data (semantic segmentation figure), generation and the one-to-one license plate markup information of license plate image data are analyzed, further,
License plate markup information refers to that markup information corresponding to license plate image data can be used for the study of car plate detection and identification model；
Step 8: the conversion of license plate style being carried out to the license plate image data in license plate data set using Style Transfer method, most
The synthesis license plate data set with true license plate style is obtained eventually.
Further, three-dimensional modeling refers to, to character involved in automotive number plate standard, license plate bottom in modeling software
Plate and license plate ornament carry out equal proportion three-dimensional modeling, and modeling contents include shape, size, color, the material of above-mentioned model；
The abrasion condition occurred in use according to true license plate simultaneously constructs character, license plate bottom plate and license plate ornament
Old model is made, the content for making old model includes that surface abrasion, surface be dirty and slight deformation.
Further, data generation strategy refers to, the method that complete license plate model is constructed in scene, mainly includes following
Step:
1) a large amount of different types of auto models are collected from network, construct scene auto model library, according to license plate type from
The auto model to match with license plate type, the main body vehicle portion as license plate data content are chosen in scene auto model library
Point；
2) according to the angle and brightness of illumination condition and type of vehicle setting environment light source and vehicle light source, environment light source packet
Including street lamp, monitoring light compensating lamp and natural lighting, vehicle light source includes headlight, taillight and number plate lamp.As license plate data content
Illumination part；(each model is reflective in light source setting influence license plate data content, increases the validity of license plate data content
And diversity)
3) license plate bottom plate is randomly selected according to license plate type, then select probability presses vehicle mould according to requiring to preset
License plate bottom plate is placed into corresponding position by type；Backplane type part as license plate data content；
4) it is directed to each characters on license plate position, constructs corresponding character set according to the regulation of automobile license plate normative document,
For single characters on license plate position, one character of random selection is concentrated from respective symbols, is bonded with license plate bottom plate in step (3),
As the vehicle character of license plate data content, so far license plate data content is complete.
5) selection and setting of above-mentioned steps are carried out for each license plate data, combined vehicle, illumination, bottom plate and
Character generates complete license plate data content, while the distribution of the satisfaction according to required for the license plate data intensive data of production, adjusts
The probability that whole difference model, illumination occur, can be generated the license plate data content being distributed needed for meeting.
Further, scene labeled data refers to, classification corresponding to different pixels is semantic in traffic scene image, with language
Adopted segmentation figure indicates that wherein vehicle, license plate bottom plate, different characters on license plate use different semantic labels, fine to parse
Markup information.
Further, license plate markup information refers to, markup information corresponding to license plate image data, for car plate detection with
The study of identification model mainly includes the following contents: (1) position coordinates on license plate callout box and four vertex；(2) license plate
Character string；(3) the semantic tagger frame of each characters on license plate and the position coordinates on four vertex；(4) angle, distance, illumination, day
The environment markup information such as gas.
Further, license plate style conversion refers to, using the Style Transfer method based on deep learning, will synthesize license plate figure
As being merged with true license plate style, the high final license plate image data set of validity is obtained.Detailed process is as follows:
(1) the true license plate image data of acquisition a part are used as style image, believe obtained in abovementioned steps with mark
The synthesis license plate image data of breath are as content images；
(2) full convolutional neural networks model is established as Style Transfer network, wherein input is true license plate image data
With synthesis license plate image data, output is to generate image；
(3) using preparatory trained semantic segmentation network model, semantic segmentation is carried out to style image and content images,
Obtain the exposure mask of characters on license plate part and bottom parts；
(4) the feature extraction network model good using precondition, according to the character portion of style image and content images
With the exposure mask of bottom parts, the corresponding character feature and bottom plate feature for generating image of style image, content images is extracted respectively；
(5) character loss function and bottom plate loss function are separately designed for character feature and bottom plate feature, designed simultaneously
It indicates the loss function of content deltas between generation image and content images, passes through back-propagation algorithm and gradient declines strategy
Above-mentioned loss function is minimized, so that generating image in the case where keeping constant with the character content of content images, is minimized
The difference between the feature of style image finally obtains trained Style Transfer network；
(6) synthesis license plate image data are input in semantic segmentation model and trained Style Transfer network, are obtained
Synthesis license plate data set after migration, the mark including Pixel-level and the synthesis license plate data with true license plate style.
The advantages of the present invention over the prior art are that:
(1) it the invention discloses a kind of generation method of automobile license plate generated data, has the advantage that firstly, originally
Invention carries out the three-dimensional modeling of license plate, character and license plate ornament according to automotive number plate standard criterion, makes the license plate of synthesis
Data validity is higher；Secondly, rendering such as all kinds of weather of heavy rain, severe snow, dense fog and a variety of illumination, energy using virtual engine
Enough obtain the more comprehensive license plate data of scene；Again, independent wash with watercolours is carried out for character different in scene and license plate bottom plate
Dye and automatic marking, so that markup information is more fine and accurate while time saving and energy saving；Finally, using license plate Style Transfer mould
The license plate image data of type conversion synthesis, so that the license plate data of synthesis are closer to true license plate data.
(2) present invention considers to generate license plate data using modeling software and virtual engine and carries out Style Transfer, main
Advantage has: (a) obtaining the font and its requirement of different license plate types disclosed in the official on authority file, and is based on true license plate
Service condition is modeled, and the license plate model obtained in this way has authenticity, comprehensive and rich；(b) virtual engine is utilized
It renders such as heavy rain, severe snow and dense fog extreme weather and a variety of intensities of illumination, Overall Acquisition includes the license plate under extreme condition
Contextual data；(c) utilize Style Transfer method that the license plate generated data of generation to true license plate data conversion, is reduced composite diagram
Picture and the difference between true picture domain.
(3) the present invention is based on the generated data generation method of Style Transfer, the large size with Pixel-level mark can be generated
License plate data set is synthesized, artificial annotation process is eliminated, it is time saving and energy saving.
Detailed description of the invention
The overview flow chart of Fig. 1 automotive number plate generated data generation method；
Fig. 2 high-precision three-dimensional simulation modeling schematic diagram；
Fig. 3 traffic scene constructs schematic diagram；
The rendering effect schematic diagram of Fig. 4 different weather；
Fig. 5 scene labeled data is converted to license plate markup information schematic diagram；
The style transition diagram of Fig. 6 license plate image data.
Specific embodiment
Illustrate a specific embodiment of the invention with reference to the accompanying drawings of the specification.
First the noun occurred in the present invention is once explained:
License plate data: it indicates the component part of license plate data set, generally indicates there are the epoch to refer to the field comprising license plate with image
Scape；
License plate image data: indicate that emphasis is image with license plate data existing for image format；
License plate data content: the content in license plate data is indicated, including character, bottom plate, type of vehicle, the light source on license plate
Type；
Synthesis license plate data: the license plate data artificially synthesized are indicated.
As shown in Figure 1, the present invention provides a kind of automobile license plate generated data generation method is as follows: first according to motor-driven
Code requirement disclosed in number plate carries out high emulation three-dimensional modeling to license plate bottom plate and character, obtains the license plate bottom plate of high emulation
And character model.Then the building of typical traffic scene is carried out by traffic scene model (including vehicle, traffic lines etc.), and
The building of license plate data content is carried out according to data generation strategy.Later, scene is rendered using virtual emulation engine, it is raw
At traffic scene image data, meanwhile, scene labeled data is generated using the method for automatic marking, and be processed into license plate number
According to markup information.Finally, synthesis license plate data are merged with true license plate style, the license plate of high validity is finally obtained
Image data set.
Three-dimensional modeling
Three-dimensional modeling, which refers to, carries out three-dimensional modeling to model, makes model fidelity with higher, height emulation three-dimensional modeling
Process as shown in Fig. 2, according to disclosed license plate code requirement, to the negative of license plate and each characters on license plate model it is soft
Equal proportion modeling is carried out in part.In addition, referring to corresponding picture, old model is made in production, including surface abrasion, surface are dirty and slight
The bottom plate and character model of deformation.As an example, the operating path of each character is obtained from authority file using PS and in 3ds
It is opened in max, the operation such as is finely adjusted, squeezes out, is round and smooth to each character and bottom plate, to obtain equal proportion model.Together
When, the bottom plate and character model for obtaining the types such as dog-ear, bending are operated by mirror image switch.
Traffic scene building
Traffic scene selects the field of crossroad, common crossing, parking lot, overhead scene, tunnel scene or High-speed Circumstance
Scape is constructed, as an example, collecting a large amount of different types of auto models, buildings model, road model and vehicle from network
Road line model constructs model of place library, obtains corresponding a variety of auto models, road from model of place library according to license plate type
Model, building model, lane line model etc. are loaded into virtual engine together with the license plate model that three-dimensional modeling obtains, and building is not
Same traffic scene.The effect diagram of building in virtual engine as shown in figure 3, copy truth, by model construction
Crossroad, the scene at common crossing and parking lot.
Data generation strategy
According to data generation strategy, data content is chosen, complete license plate model is constructed in scene, first according to vehicle
Type chooses corresponding auto model from the model of place library built, and the mode of selection should enrich and at random as far as possible.Meanwhile root
Vehicle light source and environment light source are configured according to illumination condition and type of vehicle, including headlight, taillight, street lamp, light compensating lamp, vehicle
Board lamp.Then according to the negative of pre-set probability selection license plate, and the corresponding position of auto model is placed it in.Most
Afterwards, according to the license plate bottom plate of selection, the character model of corresponding types is selected, the character types of abrasion are chosen if wearing bottom plate.
For single characters on license plate position, one character of random selection is concentrated from respective symbols, is bonded with license plate bottom plate.So far, license plate
Data content is complete.
As an example, being distributed according to needed for data, volume of data content file has been made as data generation strategy, benefit
These files are read and according to file content to vehicle, illumination, environment, license plate negative and the word inside scene with virtual engine
Symbol carries out the corresponding reasonability chosen and building, arranged in pairs or groups when building between attention model.
License plate image data generate
Using virtual engine in scene image illumination and weather rendered and generate traffic scene image data.Make
For example, the blueprint of illumination and weather is made in illusory engine, carries out high image quality illumination building, effect signal using engine
Figure is as shown in Figure 4.Current traffic contextual data is finally saved as license plate image data.
License plate markup information generates
In one license plate image data of every generation, the scene of this license plate image data is obtained using automatic marking tool
Labeled data indicates that wherein vehicle, license plate bottom plate, different characters on license plate etc. use different semantic labels with semantic segmentation figure.
Then corresponding semantic segmentation figure is analyzed, is generated and the one-to-one license plate markup information of license plate image data.As an example, sharp
Use AirSim as automatic marking tool, while one license plate image data of every generation, to vehicle, license plate bottom in scene
The different models such as plate, different characters on license plate carry out the rendering of different colours, scene labeled data are obtained, with semantic segmentation chart
Show.Then the semantic segmentation figure for analyzing acquisition generates callout box according to the connected domain and pixel distribution of license plate and character, generates
Mark effect diagram as shown in figure 5, the traffic scene labeled data that the left side is in figure, different models is carried out
Different semantic renderings, according to the numerical value of the pixel value and bounding box is carried out to the pixel of single character expansion it is available
Annotation formatting in right figure, one character of a red collimation mark note, and obtain the specifying information of the character.
The conversion of license plate style
For each type of license plate data, some corresponding true license plate data are collected first, then use depth
Frame is practised, style switching network is built, for each type of license plate data, by the feature for synthesizing license plate data and true license plate
Loss function of the difference as training between the feature of data declines strategy, optimization by back-propagation algorithm and gradient
Loss function minimizes, and obtains trained style transformation model.Finally, the trained good style transformation model of license plate will be synthesized
Propagated forward obtains the high final license plate image data set of validity.Trained and propagated forward schematic diagram is as shown in fig. 6, elder generation
Train by synthesize license plate image data to true license plate data transformation model, then will synthesis license plate data by before network to
It propagates, the final license plate data obtained after Style Transfer.
Detailed process is as follows:
(1) the true license plate image data of acquisition a part are used as style image, believe obtained in abovementioned steps with mark
The synthesis license plate image data of breath are as content images；
(2) full convolutional neural networks model is established as Style Transfer network, wherein input is true license plate image data
With synthesis license plate image data, output is to generate image, as an example, the present invention selects the style that TransformerNet is used as
Migrate network；
(3) using preparatory trained semantic segmentation network model, semantic segmentation is carried out to style image and content images,
The exposure mask for obtaining characters on license plate part and bottom parts, as an example, the present invention selects deeplabv2 as semantic segmentation mould
Type；
(4) the feature extraction network model good using precondition, according to the character portion of style image and content images
With the exposure mask of bottom parts, the corresponding character feature and bottom plate feature for generating image of style image, content images is extracted respectively,
As an example, the present invention chooses VGG19 as Feature Selection Model；
(5) character loss function and bottom plate loss function are separately designed for character feature and bottom plate feature, designed simultaneously
It indicates the loss function of content deltas between generation image and content images, passes through back-propagation algorithm and gradient declines strategy
Above-mentioned loss function is minimized, so that generating image in the case where keeping constant with the character content of content images, is minimized
The difference between the feature of style image finally obtains trained Style Transfer network.As an example, the loss in training process
Function includes two parts, and first part is the difference for generating image and content images, i.e. content loss function, uses mean square error
(MSE) it is calculated.Wherein i, j are expressed as j-th of position in i-th of channel of image.Here x, y indicate to generate image and interior
Hold image.
Second part is the difference for generating the feature of image and style image, i.e. style loss function, with Gram matrix into
Row calculates, wherein i, and j indicates j-th of position in i-th of channel being characterized.
Wherein F, which is characterized, extracts the feature that network is extracted at l layers, then combines all layers of style loss function are as follows:
Wherein wlIndicate l layers of coefficient, ElIt is expressed as generating the mean square error of the Gram matrix of image and style image, A
For the Gram matrix for generating image, G is the Gram matrix of style image, and wherein i, j are expressed as the jth in i-th of channel of matrix
A position:
E is calculated separately according to the exposure mask of the exposure mask of character and bottom platelAddition obtains Lstyle(x,xs)。
The combination of this two parts is used as objective function
L(x,xc,xs)=α Lcontent(x,xc)+βLstyle(x,xs)
Wherein, x indicates to generate image, xcIndicate content images, xsIndicate style image, α, β are coefficient；
(6) synthesis license plate image data are input in semantic segmentation model and trained Style Transfer network, are obtained
Synthesis license plate data set after migration, the mark including Pixel-level and the synthesis license plate data with true license plate style.
Although describing specific implementation method of the invention above, it will be appreciated by those of skill in the art that these
It is merely illustrative of, under the premise of without departing substantially from the principle of the invention and realization, numerous variations can be made to these embodiments
Or modification, therefore, protection scope of the present invention is defined by the appended claims.
Claims (6)
1. a kind of automobile license plate generated data generation method, which comprises the following steps:
Step 1: the corresponding automobile license plate normative document of selection license plate type, the normative content according to normative document exist
In modeling software to license plate bottom plate, character and license plate ornament carry out three-dimensional modeling, while construct license plate bottom plate, character and
License plate ornament makees old model, obtains the threedimensional model of character required for license plate data, bottom plate and character ornament；
Step 2: constructing a variety of different traffic scenes using virtual engine, the environment generated as license plate data；It is described a variety of
Different traffic scenes includes: common crossing, crossroad, parking lot, overhead scene, tunnel scene or High-speed Circumstance；It will step
Rapid 1 obtained threedimensional model is imported into the traffic scene, is arranged according to traffic specification, target traffic required for obtaining
Scene；
Step 3: in the target traffic scene described in step 2, the different illumination including daytime, noon and night are respectively set, with
And rainy day, fine day, snowy day, greasy weather and sandstorm these different weathers, and rendered using virtual engine, after being rendered
Multiple traffic scenes；
Step 4: in the traffic scene after each rendering, according to the distribution of satisfaction needed for the license plate data set of production, using phase
The data generation strategy answered is adjusted license plate data content, generates the license plate data being distributed needed for meeting；
Step 5: each traffic scene comprising license plate data being rendered using virtual engine, the traffic field after acquisition rendering
Scape image generates corresponding traffic scene labeled data, traffic field as license plate image data, while using automatic marking tool
Scape labeled data is indicated with semantic segmentation figure；
Step 6: being executed by step 2-5, circulation generates license plate image data and corresponding traffic scene labeled data, until license plate
The quantity of image data meets the requirement of setting, and all license plate image data constitute license plate image data set；
Step 7: for the corresponding traffic scene labeled data of every license plate image data of license plate image data set in step 6,
The traffic scene labeled data is analyzed, is generated and the one-to-one license plate markup information of license plate image data, the license plate mark
Information is markup information corresponding to license plate image data, the study for car plate detection and identification model:
Step 8: license plate style being carried out to the license plate image data in original license plate image data set using Style Transfer method and is turned
It changes, it is final to obtain the synthesis license plate data set with true license plate style.
2. automobile license plate generated data generation method according to claim 1, it is characterised in that: in the step 1, build
The process of vertical threedimensional model are as follows: to character involved in automotive number plate standard, license plate bottom plate and license plate in modeling software
Ornament carry out equal proportion three-dimensional modeling, modeling contents include: character, the shape of license plate bottom plate and license plate ornament, size,
Color, material；The abrasion condition occurred in use according to true license plate simultaneously, building character, license plate bottom plate and vehicle
Board ornament makees old model, and doing old model content includes that surface abrasion, surface be dirty and slight deformation.
3. automobile license plate generated data generation method according to claim 1, it is characterised in that: in the step (4)
Data generation strategy the following steps are included:
(1) a large amount of different types of vehicles are collected from network, construct scene auto model library, according to license plate type from scene vehicle
The auto model to match with license plate type, the vehicle sections as license plate data content are chosen in model library；
(2) according to the angle and brightness of illumination condition and type of vehicle setting environment light source and vehicle light source, the environment light source
Including street lamp, monitoring light compensating lamp and natural lighting；The vehicle light source includes headlight, taillight and number plate lamp；This step indicates
Setting of the data generation strategy to light conditions, the illumination provided with license plate data content increase the authenticity of data and more
Sample；
(3) license plate bottom plate is randomly selected according to license plate type, for select probability according to requiring to preset, then pressing auto model will
License plate bottom plate is placed into corresponding position；Backplane type part as license plate data content；
(4) it is directed to each characters on license plate position, constructs corresponding character set according to the regulation of automobile license plate normative document, for
Single characters on license plate position is concentrated one character of random selection from respective symbols, is bonded with license plate bottom plate in step (3), as
The vehicle character of license plate data content, so far license plate data content is complete；
(5) for each license plate data, the selection and setting of above-mentioned steps, combined vehicle, illumination, bottom plate and character are carried out
Complete license plate data content, while the distribution of the satisfaction according to needed for the license plate data intensive data of production are generated, adjustment is different
The probability that model, different illumination occur, generates the license plate data content being distributed needed for meeting.
4. automobile license plate generated data generation method according to claim 1, it is characterised in that: in the step 5, hand over
Logical scene labeled data are as follows: classification corresponding to different pixels is semantic in traffic scene image, is indicated with semantic segmentation figure, wherein
Vehicle, license plate bottom plate, different characters on license plate use different semantic labels, to parse fine markup information.
5. automobile license plate generated data generation method according to claim 1, it is characterised in that: in the step 7, vehicle
Board markup information refers to markup information corresponding to license plate image data, for the study of car plate detection and identification model, mark
Information includes the following contents: (1) position coordinates on license plate callout box and four vertex；(2) character string of license plate；(3) every
The semantic tagger frame of a characters on license plate and the position coordinates on four vertex；(4) angle, distance, illumination, weather, class of vehicle ring
Border markup information.
6. automobile license plate generated data generation method according to claim 1, it is characterised in that: in the step 8, vehicle
Board style switch process is as follows:
(1) the true license plate image data of acquisition a part are used as style image, have markup information obtained in abovementioned steps
License plate image data are synthesized as content images；
(2) full convolutional neural networks model is established as Style Transfer network, wherein input is true license plate image data and conjunction
At license plate image data, output is to generate image；
(3) using preparatory trained semantic segmentation network model, semantic segmentation is carried out to style image and content images, is obtained
The exposure mask of characters on license plate part and bottom parts；
(4) the feature extraction network model good using precondition, according to the character portion and bottom of style image and content images
The exposure mask of plate part extracts the corresponding character feature and bottom plate feature for generating image of style image, content images respectively；
(5) character loss function and bottom plate loss function are separately designed for character feature and bottom plate feature, while designs expression
The loss function for generating content deltas between image and content images, it is minimum by back-propagation algorithm and gradient decline strategy
Change above-mentioned loss function, so that image is generated in the case where keeping constant with the character content of content images, minimum and wind
Difference between the feature of table images finally obtains trained Style Transfer network；
(6) synthesis license plate image data are input in semantic segmentation model and trained Style Transfer network, are migrated
Synthesis license plate data set afterwards, the mark including Pixel-level and the synthesis license plate data with true license plate style.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910738871.6A CN110503716B (en) | 2019-08-12 | 2019-08-12 | Method for generating motor vehicle license plate synthetic data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910738871.6A CN110503716B (en) | 2019-08-12 | 2019-08-12 | Method for generating motor vehicle license plate synthetic data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110503716A true CN110503716A (en) | 2019-11-26 |
| CN110503716B CN110503716B (en) | 2022-09-30 |
Family
ID=68587175
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910738871.6A Active CN110503716B (en) | 2019-08-12 | 2019-08-12 | Method for generating motor vehicle license plate synthetic data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110503716B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111259950A (en) * | 2020-01-13 | 2020-06-09 | 南京邮电大学 | Method for training YOLO neural network based on 3D model |
| CN111275796A (en) * | 2020-01-17 | 2020-06-12 | 北京迈格威科技有限公司 | License plate synthesis method and device, computer equipment and storage medium |
| CN111340964A (en) * | 2020-03-05 | 2020-06-26 | 长春中国光学科学技术馆 | 3D model image construction method based on transfer learning |
| CN111340720A (en) * | 2020-02-14 | 2020-06-26 | 云南大学 | Color register woodcut style conversion algorithm based on semantic segmentation |
| CN111582277A (en) * | 2020-06-15 | 2020-08-25 | 深圳天海宸光科技有限公司 | License plate recognition system and method based on transfer learning |
| CN112101349A (en) * | 2020-09-01 | 2020-12-18 | 北京智芯原动科技有限公司 | License plate sample generation method and device |
| WO2021027890A1 (en) * | 2019-08-15 | 2021-02-18 | 杭州海康威视数字技术股份有限公司 | License plate image generation method and device, and computer storage medium |
| CN112419186A (en) * | 2020-11-20 | 2021-02-26 | 北京易华录信息技术股份有限公司 | License plate image batch generation method and device and computer equipment |
| CN113971627A (en) * | 2020-07-23 | 2022-01-25 | 华为技术有限公司 | License plate picture generation method and device |
| CN115497084A (en) * | 2022-11-14 | 2022-12-20 | 深圳天海宸光科技有限公司 | Method for generating simulation license plate picture and character level label |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018028230A1 (en) * | 2016-08-10 | 2018-02-15 | 东方网力科技股份有限公司 | Deep learning-based method and device for segmenting vehicle license plate characters, and storage medium |
| CN109255772A (en) * | 2018-08-27 | 2019-01-22 | 平安科技（深圳）有限公司 | License plate image generation method, device, equipment and medium based on Style Transfer |
| CN109325989A (en) * | 2018-08-27 | 2019-02-12 | 平安科技（深圳）有限公司 | License plate image generation method, device, equipment and medium |
-
2019
- 2019-08-12 CN CN201910738871.6A patent/CN110503716B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018028230A1 (en) * | 2016-08-10 | 2018-02-15 | 东方网力科技股份有限公司 | Deep learning-based method and device for segmenting vehicle license plate characters, and storage medium |
| CN109255772A (en) * | 2018-08-27 | 2019-01-22 | 平安科技（深圳）有限公司 | License plate image generation method, device, equipment and medium based on Style Transfer |
| CN109325989A (en) * | 2018-08-27 | 2019-02-12 | 平安科技（深圳）有限公司 | License plate image generation method, device, equipment and medium |
Non-Patent Citations (1)
| Title |
|---|
| 曾超等: "车牌超分辨率重建与识别", 《计算机测量与控制》 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021027890A1 (en) * | 2019-08-15 | 2021-02-18 | 杭州海康威视数字技术股份有限公司 | License plate image generation method and device, and computer storage medium |
| CN111259950A (en) * | 2020-01-13 | 2020-06-09 | 南京邮电大学 | Method for training YOLO neural network based on 3D model |
| CN111275796A (en) * | 2020-01-17 | 2020-06-12 | 北京迈格威科技有限公司 | License plate synthesis method and device, computer equipment and storage medium |
| CN111275796B (en) * | 2020-01-17 | 2023-09-19 | 北京迈格威科技有限公司 | License plate synthesis method, license plate synthesis device, computer equipment and storage medium |
| CN111340720A (en) * | 2020-02-14 | 2020-06-26 | 云南大学 | Color register woodcut style conversion algorithm based on semantic segmentation |
| CN111340720B (en) * | 2020-02-14 | 2023-05-19 | 云南大学 | Color matching woodcut style conversion algorithm based on semantic segmentation |
| CN111340964B (en) * | 2020-03-05 | 2023-03-24 | 长春中国光学科学技术馆 | 3D model image construction method based on transfer learning |
| CN111340964A (en) * | 2020-03-05 | 2020-06-26 | 长春中国光学科学技术馆 | 3D model image construction method based on transfer learning |
| CN111582277A (en) * | 2020-06-15 | 2020-08-25 | 深圳天海宸光科技有限公司 | License plate recognition system and method based on transfer learning |
| CN113971627A (en) * | 2020-07-23 | 2022-01-25 | 华为技术有限公司 | License plate picture generation method and device |
| CN113971627B (en) * | 2020-07-23 | 2023-07-18 | 华为技术有限公司 | License plate picture generation method and device |
| CN112101349A (en) * | 2020-09-01 | 2020-12-18 | 北京智芯原动科技有限公司 | License plate sample generation method and device |
| CN112419186A (en) * | 2020-11-20 | 2021-02-26 | 北京易华录信息技术股份有限公司 | License plate image batch generation method and device and computer equipment |
| CN112419186B (en) * | 2020-11-20 | 2024-03-26 | 北京易华录信息技术股份有限公司 | Batch generation method and device for license plate images and computer equipment |
| CN115497084A (en) * | 2022-11-14 | 2022-12-20 | 深圳天海宸光科技有限公司 | Method for generating simulation license plate picture and character level label |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110503716B (en) | 2022-09-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110503716A (en) | A kind of automobile license plate generated data generation method | |
| CN111695448B (en) | Roadside vehicle identification method based on visual sensor | |
| CN108985194A (en) | A kind of intelligent vehicle based on image, semantic segmentation can travel the recognition methods in region | |
| CN110263786A (en) | A kind of road multi-targets recognition system and method based on characteristic dimension fusion | |
| CN110232316A (en) | A kind of vehicle detection and recognition method based on improved DSOD model | |
| CN108897313A (en) | A kind of end-to-end Vehicular automatic driving system construction method of layer-stepping | |
| CN105844257A (en) | Early warning system based on machine vision driving-in-fog road denoter missing and early warning method | |
| JPWO2020181685A5 (en) | ||
| CN110414418A (en) | A kind of Approach for road detection of image-lidar image data Multiscale Fusion | |
| CN105678315A (en) | Method of generating training image for automated vehicle object recognition system | |
| CN107016362A (en) | Vehicle based on vehicle front windshield sticking sign recognition methods and system again | |
| CN110032935A (en) | A kind of traffic signals label detection recognition methods based on deep learning cascade network | |
| CN108876805A (en) | The end-to-end unsupervised scene of one kind can traffic areas cognition and understanding method | |
| CN110991523A (en) | Interpretability evaluation method for unmanned vehicle detection algorithm performance | |
| Cheng et al. | Modeling weather and illuminations in driving views based on big-video mining | |
| CN114863441A (en) | Text image editing method and system based on character attribute guidance | |
| CN115273032A (en) | Traffic sign recognition method, apparatus, device and medium | |
| CN115424217A (en) | AI vision-based intelligent vehicle identification method and device and electronic equipment | |
| CN116597690B (en) | Highway test scene generation method, equipment and medium for intelligent network-connected automobile | |
| CN108734170A (en) | Registration number character dividing method based on machine learning and template | |
| CN114120246B (en) | Front vehicle detection algorithm based on complex environment | |
| Li et al. | Efficient residual neural network for semantic segmentation | |
| Zhuo et al. | A novel vehicle detection framework based on parallel vision | |
| Yang et al. | Intelligent intersection vehicle and pedestrian detection based on convolutional neural network | |
| CN113095207A (en) | Lightweight and efficient single-stage night vehicle detection algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |