Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant application and are not limiting of the application. It should be noted that, for the convenience of description, only the portions relevant to the application are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
It can be understood that in a logistics scene and other scenes, the license plate of the vehicle needs to be identified, for example, the license plate can be identified through a pre-constructed license plate identification model. In the construction of the recognition model, a large number of vivid license plate images are required to be used as training samples. In the embodiment of the application, in order to improve the generation efficiency and accuracy of the license plate image training samples and realize the rapid acquisition of the license plate image samples, a generation model (namely a generator) for generating the license plate image samples is provided through a cyclic countermeasure generation network, and then the generator and the artificial synthesis of the license plate images can be utilized to rapidly and efficiently acquire vivid license plate image samples.
For ease of understanding and explanation, the process of constructing the license plate image sample generation model is described in detail below with reference to fig. 1. Fig. 1 is a schematic flow chart of a license plate image generation method provided in an embodiment of the present application, and as shown in fig. 1, the method includes:
and S110, acquiring a training set.
And S120, generating a loop generation countermeasure network.
Specifically, a plurality of license plate images including license plate numbers and backgrounds can be synthesized by a manual synthesis method, and a plurality of real license plate images are obtained. And further, real license plate image frames equivalent to the number of the synthesized license plates can be generated through partial image amplification. And (4) manually synthesizing the license plate image and the real license plate image to form a training set.
Further, an initial loop-generated countermeasure network may be set up, as shown in fig. 3 and 4, that is, a first generator, a second generator, a first discriminator, and a second discriminator are generated.
It can be understood that the first generator comprises a mapping algorithm of the synthesized license plate image features to the real license plate image features, the input of the mapping algorithm is an artificially synthesized license plate image, so that after the artificially synthesized license plate image is input, the features of the real license plate image can be learned due to the mapping algorithm, and the real license plate image which is as vivid as possible can be output. The second generator comprises a mapping algorithm from the real license plate image characteristics to the synthesized license plate image characteristics, so that the real license plate image can learn the characteristics of the synthesized license plate image after being input, and a fake license plate image similar to the synthesized license plate is output.
The algorithm for mapping the artificially synthesized license plate image to the real license plate image may include a preliminary mapping relationship between the synthesized license plate image and the feature code of the real license plate image. For example, a mapping table of feature codes is established, and the mapping table comprises the feature codes of the synthesized license plate image, the feature codes of the real license plate image and the mapping relation between the two types of feature codes. When a license plate image is input into a first generator for training, namely the license plate image is divided into a plurality of pixel regions, then the feature codes of each pixel region are extracted, the features can comprise information such as the position, color distribution and gray distribution of the pixel regions, and the feature codes of each pixel region are used as the feature codes of the synthesized license plate image to be inquired in a mapping table, the feature codes of the corresponding real license plate image are inquired, and after the feature codes of the real license plate image corresponding to all pixels are obtained, decoding is carried out according to the feature codes, so that the real license plate image can be obtained.
The first discriminator may be used to identify the output of the first generator and the second discriminator may be used to identify the output of the second generator. It can be appreciated that in generating the first discriminator, the license plate images in the training set can be utilized such that they output a high score for the real license plate images and a low score for the composite license plate images. Similarly, for the second discriminator, the license plate in the training set can be used for preliminary training, so that when the second discriminator outputs a real license plate, a low score is output, and when the second discriminator inputs a synthesized license plate, a high score is output.
It can be understood that the mapping table established by the preliminary learning in the embodiment of the present application includes a small amount of information, so that the difference between the generated real license plate and the actual license plate is large, and therefore, the generator is continuously trained by the following method, the mapping relation table can be gradually enriched, an optimized generator model is obtained, and the generator model outputs a vivid license plate image after the synthesized license plate image is input.
S130, training the circularly generated countermeasure network based on the training set to obtain at least one trained first generator serving as a target generator.
Specifically, after the countermeasure network generated by the loop is generated, the acquired data in the training set may be input into the network, and two generators constructed preliminarily are trained to obtain a plurality of optimized first generators, that is, target generators, so that the license plate output by the target generators is as vivid as possible, that is, the target generators are used to generate a target license plate image.
When the generated cyclic generation countermeasure network is trained, the license plate image and the real license plate image are expected to be distributed and are similar as much as possible when the training is finished. Specifically, the acquired license plate images in the training set can be input into a preliminarily generated cyclic generation countermeasure network, and then the loss of the cyclic generation countermeasure network is determined according to the output results of the first generator, the second generator, the first discriminator and the second discriminator. Finally, parameters of the circularly generated countermeasure network can be replaced based on the loss of the output of the current round, and one-time training optimization of the circularly generated countermeasure network is completed.
It will be appreciated that the above steps of inputting, determining and replacing parameters are repeated for multiple rounds of model training. For each training, one optimized first generator may be saved, and multiple optimized first generators may be obtained after repeating multiple rounds.
For each training round, in particular, it can be combined with the description in fig. 2: firstly, the artificially synthesized license plate can be input into a first generator GA, so that the input synthesized license plate learns the characteristics of a real license plate, and a first license plate image GA(s) similar to the real license plate, namely a fake first license plate image, is output. The output may then be input to a first discriminator Dr to discriminatively score the result. According to the above-mentioned training of the first discriminator Dr, when GA(s) is input, Dr may output a high score if GA(s) is close to a real license plate, otherwise, output a low score.
It will be appreciated that for the above training, the first generator, to cater for the first discriminator, may map all synthesized license plates to license plates of the same pattern in the set of real license plate images, such that the first discriminator outputs a high score. Obviously, this will cause the first generator pattern to be missing.
To solve the above problem, GA(s) may be further input to the second generator GB, resulting in GB (GA (s)), i.e. a false third license plate image, i.e. such that GA(s) output by the first generator continue to revert to the synthesized license plate, to avoid training the first generator into a single mode.
Meanwhile, in order to successfully convert the unpaired license plate data, the real license plate image is input into a second generator GB, and GB (r), namely a fake second license plate image, is output, namely the second generator finishes the learning of the characteristic of the synthesized license plate so as to cooperate with and optimize the first generator. In order to ensure the training of the second generator, the output result of the second generator may be determined by the second determiner Ds of the training to determine whether the output result GB (r) is sufficiently imaginable with the composite license plate. Namely, the judgment result is that the synthesized license plate outputs a high score, and the judgment result is that the real license plate outputs a low score. Similarly, in order to avoid the second generator training to become a single mode in order to cater to the second discriminator, the output GB (r) of the second generator may be input to the first generator, so that it learns the features of the real license plate and reverts to the real license plate, i.e., the output GA (GB (r)), i.e., the fourth license plate image.
The input and output processes complete a round of training, and at this time, the loss of the loop generation countermeasure network can be calculated so as to adjust the generator parameters for optimization.
For example, the generation loss function of the first generator GA may be calculated:
the loss function enables the first discriminator Dr to output a higher score for the output result of the GA, namely, the output result of the GA and the feature distribution of the real license plate image are closer and closer along with multiple training.
The generation loss function of the second generator GB is calculated as:
similarly, the loss function enables the second discriminator Ds to output a higher score for the output result of GB, that is, the output result of GB and the feature distribution of the synthesized license plate image are closer and closer with multiple training.
Calculating the self-loop loss function of the GA:
the self-loop loss function makes the result of GB (GA (s)) as similar as possible to the synthetic license plate s.
Calculating the self-loop loss function of GB:
the self-loop loss function makes the GA (GB (r)) result as similar as possible to the real license plate r.
Further, the total generation loss is calculated by the above equations (1) to (4):
in the above formula, GA is the first generator, GB is the second generator, Dr is the first discriminator, Ds is the second discriminator, s is the synthesized license plate image, r is the real license plate image, GA(s) is the output result of the first generator,
representing an expectation of a composite license plate image sampled from a set of composite license plate images,
representing the expectation, alpha, of a real license plate image sampled from a set of real license plate images
BAnd alpha
AAre empirical values.
It will be appreciated that after the above-mentioned total generation loss is obtained, the cycle-generating countermeasure network can be optimized by replacing the parameters of the first generator and the second generator with the function values.
In addition, in the license plate image generation model construction method provided by the embodiment of the application, the weight of the circularly generated countermeasure network needs to be modified by calculating the discriminant loss function, so as to obtain the optimized target generator.
The discriminant loss function for Ds is:
this loss function enables Ds to distinguish artificially synthesized license plates from fake license plates generated by GB that resemble artificially synthesized license plates.
The discriminant loss function for Dr is:
this loss function enables Dr to distinguish between a real license plate and a fake license plate generated by GA that resembles a real license plate.
Further, after a discrimination loss function value is obtained, the weight of the countermeasure network generated circularly can be adjusted by a random gradient descent optimization method, and an optimized discriminator is obtained.
It will be appreciated that multiple rounds of input-output training may be performed while training the initial cycle generation countermeasure network. If the learning rate of the circularly generated confrontation network is set to be 0.0002, 30 rounds of circularly generated confrontation network training are carried out, the generation steps and the discrimination steps are alternately carried out, the generated model obtained by each round of training is saved, and 30 model results can be obtained in total, wherein the model results can comprise a first generator (namely a target generator) and a second generator.
Further, the saved target model can be verified by artificially synthesizing a test set, such as 100 license plates, by using the test set. Specifically, using the first generator in each saved model result, 100 license plates synthesized by artificial rules are used as input, and 30 sets of outputs are obtained. The similarity degree between the output result and the distribution of the real license plate can be compared by visually observing the image results output by each group, and one or more model results with the highest similarity degree are selected as the final result.
According to the license plate image generation model construction method provided by the embodiment of the application, after the confrontation network is generated in a circulating mode, the generator can be repeatedly trained and optimized by using the acquired training set, the loss function of the generator, the self-loop loss function and the discriminant loss function of the discriminator to obtain the optimal first generator, so that a large number of vivid license plates can be generated by using the first generator, and the efficiency and the reliability of license plate image generation are improved.
It can be understood that, in the embodiment of the application, after the target generators, that is, the license plate image generation models, are obtained by the above method, license plates of a required number can be synthesized by manual rules and input to one or more target generators selected finally, so that fake license plates of a required number, which are similar to the distribution of real license plate images, can be obtained. As shown in fig. 5, the method may include:
s510, acquiring a synthesized license plate image;
s520, inputting the synthesized license plate image into a pre-constructed target generator to generate a target license plate image.
According to the license plate image generation method provided by the embodiment of the application, after the required number of fake license plate images are acquired through artificial synthesis and other modes, the fake license plate images can be input into the optimal generator (generation model) which is trained in advance, a large number of vivid license plates are generated, and the efficiency and the reliability of license plate image generation are improved.
Fig. 6 is a schematic structural diagram of a license plate image generation model construction device provided in an embodiment of the present application, and as shown in the drawing, the device may include:
an obtaining module 610, configured to obtain a training set, where the training set includes a synthesized license plate image and a real license plate image;
a generating module 620, configured to generate a cyclic generation countermeasure network, where the cyclic generation countermeasure network includes a first generator, a second generator, a first discriminator, and a second discriminator, the first generator includes a mapping algorithm from a positive image to a negative image, the second generator includes a mapping algorithm from a negative image to a positive image, the first discriminator is configured to identify an output result of the first generator in the training set, and the second discriminator is configured to identify an output result of the second generator;
a training module 630, configured to train the cycle generation countermeasure network based on the training set, to obtain at least one trained first generator as a target generator, where the target generator is configured to generate a target license plate image.
Optionally, in the license plate image generation model construction device provided in the embodiment of the present application, the training module includes:
an input unit 631, configured to input the training set to the loop generation countermeasure network.
A determining unit 632 configured to determine that the loop generates a loss against the network.
An optimization unit 633 for replacing the parameters of the loop generation countermeasure network based on the loss.
Optionally, in the license plate image generation model construction device provided in this embodiment of the application, the determining unit is specifically configured to determine, based on output results of the first generator and the second generator, and the first discriminator and the second discriminator, a loss function of the first generator and the second generator, a self-loop loss function of the first generator and the second generator, and a discriminant loss function of the first discriminator and the second discriminator.
Fig. 7 is a schematic structural diagram of a license plate image generating device according to an embodiment of the present disclosure, and as shown in the drawing, the device 700 may include:
an obtaining module 710, configured to obtain a composite license plate image;
a generating module 720, configured to input the synthesized license plate image to the target generator according to the first aspect, and generate a target license plate image.
On the other hand, the embodiment of the present application further provides a server, where the server includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the computer program to implement the license plate image generation model construction method as described above.
Referring now to FIG. 8, shown is a block diagram of a computer system 800 suitable for use in implementing a server according to embodiments of the present application.
As shown in fig. 8, the computer system 800 includes a Central Processing Unit (CPU)801 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)802 or a program loaded from a storage section 803 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data necessary for the operation of the system 800 are also stored. The CPU 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
The following components are connected to the I/O interface 805: an input portion 806 including a keyboard, a mouse, and the like; an output section 807 including a signal such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 808 including a hard disk and the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. A drive 810 is also connected to the I/O interface 805 as necessary. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as necessary, so that a computer program read out therefrom is mounted on the storage section 808 as necessary.
In particular, according to embodiments of the license plate image generation model construction disclosed herein, the process described above with reference to fig. 1 may be implemented as a computer software program. For example, embodiments of license plate image generation model construction disclosed herein include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing the method of FIG. 1. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 809 and/or installed from the removable medium 811.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various license plate image generation model building embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules described in the embodiments of the present application may be implemented by software or hardware. The described units or modules may also be provided in a processor, and may be described as: a processor includes an acquisition model, a generation model, and a training module. Where the names of such units or modules do not in some cases constitute a limitation of the unit or module itself, for example, a training module may also be described as "training the cycle generation countermeasure network based on the training set, resulting in at least one trained first generator as a target generator for generating a target license plate image".
As another aspect, the present application also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the foregoing device in the foregoing embodiment; or it may be a separate computer readable storage medium not incorporated into the device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the method for determining a license plate image generation model construction described herein, and in particular:
acquiring a training set, wherein the training set comprises a synthesized license plate image and a real license plate image;
generating a cyclic generation countermeasure network, wherein the cyclic generation countermeasure network comprises a first generator, a second generator, a first discriminator and a second discriminator, the first generator comprises a mapping algorithm from a synthesized license plate image to a real license plate image, the second generator comprises a mapping algorithm from the real license plate image to the synthesized license plate image, the first discriminator is used for recognizing the output result of the first generator, and the second discriminator is used for recognizing the output result of the second generator;
and training the circularly generated countermeasure network based on the training set to obtain at least one trained first generator serving as a target generator, wherein the target generator is used for generating a target license plate image.
In summary, according to the license plate image generation model construction method and device provided by the embodiment of the application, after the cyclic generation countermeasure network is generated in advance, the acquired license plate images in the training set are input to the generator of the cyclic generation countermeasure network, and then the generator is optimized and tested by using the cyclic input and discriminator, so that an optimal target generator is obtained after repeated training for many times, and therefore a large number of vivid license plates can be generated by using the target generator, and the license plate image generation efficiency and reliability are improved.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by persons skilled in the art that the scope of the application referred to in the present application is not limited to the particular combination of features described above, but also covers other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the application. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.