CN114581302A - License plate sample image generation method, device, equipment and storage medium - Google Patents

License plate sample image generation method, device, equipment and storage medium Download PDF

Info

Publication number
CN114581302A
CN114581302A CN202210167280.XA CN202210167280A CN114581302A CN 114581302 A CN114581302 A CN 114581302A CN 202210167280 A CN202210167280 A CN 202210167280A CN 114581302 A CN114581302 A CN 114581302A
Authority
CN
China
Prior art keywords
image
license plate
simulated
region
sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210167280.XA
Other languages
Chinese (zh)
Inventor
孙尚云
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Elite Road Technology Co ltd
Original Assignee
Beijing Elite Road Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Elite Road Technology Co ltd filed Critical Beijing Elite Road Technology Co ltd
Priority to CN202210167280.XA priority Critical patent/CN114581302A/en
Publication of CN114581302A publication Critical patent/CN114581302A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/60—Rotation of whole images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20132—Image cropping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a license plate sample image generation method, a license plate sample image generation device, license plate sample image generation equipment and a storage medium, and relates to the technical field of deep learning and intelligent parking. The specific implementation scheme is as follows: obtaining a simulated license plate image; acquiring a first image which is acquired in an actual scene and contains a license plate area; fusing a license plate region in the simulated license plate image to a license plate region in the first image to obtain a second image; and carrying out harmony processing on the image style of the license plate region in the second image based on the image style of the background region in the second image to obtain a first sample image obtained based on the processing result. By applying the scheme provided by the embodiment of the disclosure, the sample image can be generated.

Description

License plate sample image generation method, device, equipment and storage medium
Technical Field
The utility model relates to an artificial intelligence technical field especially relates to degree of depth study, wisdom parking technical field.
Background
In daily life, license plates of vehicles need to be identified in many scenes. For example, intelligent parking charging scenes, illegal parking vehicle capturing scenes, and the like, all need to identify vehicles by recognizing license plates. In the related art, a license plate recognition model is often used for recognizing a license plate, and in order to improve the accuracy of the license plate recognition model in recognizing the license plate, a large number of license plate sample images are required to be used for pre-training the license plate recognition model.
In view of the above, it is desirable to provide a license plate sample image generation scheme for generating a license plate sample image.
Disclosure of Invention
The disclosure provides a license plate sample image generation method, a license plate sample image generation device, license plate sample image generation equipment and a storage medium.
According to an aspect of the present disclosure, a license plate sample image generation method is provided, including:
obtaining a simulated license plate image;
acquiring a first image which is acquired in an actual scene and contains a license plate area;
fusing a license plate region in the simulated license plate image to a license plate region in the first image to obtain a second image;
and on the basis of the image style of the background area in the second image, carrying out harmony processing on the image style of the license plate area in the second image to obtain a first sample image obtained on the basis of a processing result.
According to another aspect of the present disclosure, there is provided a license plate sample image generating apparatus including:
the simulated license plate image obtaining module is used for obtaining a simulated license plate image;
the first image acquisition module is used for acquiring a first image which is acquired in an actual scene and contains a license plate area;
the image fusion module is used for fusing the license plate region in the simulated license plate image to the license plate region in the first image to obtain a second image;
and the first sample image obtaining module is used for carrying out harmony processing on the image style of the license plate region in the second image based on the image style of the background region in the second image so as to obtain a first sample image based on the processing result.
According to another aspect of the present disclosure, there is provided an electronic device including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a license plate sample image generation method.
According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to execute a license plate sample image generating method.
According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements a license plate sample image generation method.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
fig. 1 is a schematic flowchart of a license plate sample image generation method according to an embodiment of the present disclosure;
fig. 2 is a schematic flowchart of another license plate sample image generation method provided in the embodiment of the present disclosure;
FIG. 3 is a schematic diagram of an image synthesis process provided by an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of a license plate sample image generation process provided by an embodiment of the present disclosure;
fig. 5 is a schematic flowchart of another license plate sample image generation method provided in the embodiment of the present disclosure;
FIG. 6 is a schematic diagram of an image style migration process provided by an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of a license plate sample image generation apparatus provided in an embodiment of the present disclosure;
fig. 8 is a schematic structural diagram of another license plate sample image generation apparatus provided in the embodiment of the present disclosure;
fig. 9 is a schematic structural diagram of another license plate sample image generating device according to an embodiment of the present disclosure;
fig. 10 is a block diagram of an electronic device for implementing a license plate sample image generation method according to an embodiment of the present disclosure.
Detailed Description
The following specifically describes a license plate sample image generation method provided by the embodiment of the present disclosure.
Referring to fig. 1, fig. 1 is a schematic flow chart diagram of a license plate sample image generation method provided in an embodiment of the present disclosure, and the method includes the following steps S101 to S104.
Step S101: and obtaining a simulated license plate image.
The simulated license plate image is an image for simulating a license plate in an actual scene, and can be a 2D image or a 3D image. From the perspective of the content contained in the image, the simulated license plate image may only contain a license plate region, or may contain both a background region and a license plate region.
In one embodiment, the simulated license plate image may be an image that is generated in advance and stored in a database, in which case the simulated license plate image may be read from the database.
In another embodiment, the simulated license plate image can be obtained first, and then the simulated license plate image is subjected to processing such as style migration, so as to obtain the simulated license plate image based on the processing result. Detailed description of the preferred embodimentsreferring to the embodiment shown in fig. 5, steps S501-S502 will not be described in detail here.
Step S102: a first image which is collected in an actual scene and contains a license plate area is obtained.
Since the license plate is usually disposed in the area of the head and the tail of the vehicle, the first image may be a head and/or tail image containing the license plate area. Of course, the first image may also be an image of the side of the vehicle containing the license plate region.
Specifically, the first image may be collected in advance and stored in a database, so that the first image can be read from the database. The first image stored in the database in advance can be obtained in the following manner.
In the first mode, the first image pre-stored in the database may be obtained by a worker shooting a vehicle in a real environment by using an image capturing device, such as a camera, a video camera, or the like.
In a second mode, the first image pre-stored in the database can be automatically acquired or captured by an image acquisition device included in the intelligent parking system or the illegal parking capturing system.
In a third way, the first image pre-stored in the database can be extracted from the web page using a web crawler engine.
Step S103: and fusing the license plate region in the simulated license plate image to the license plate region in the first image to obtain a second image.
In one embodiment, the area parameter of the license plate area in the first image may be determined, the license plate area in the simulated license plate image may be subjected to deformation processing according to the area parameter, and then the processed license plate area may be covered in the license plate area in the first image.
Specifically, the region parameter of the license plate region in the first image may be a vertex coordinate of the license plate region, so that the license plate region in the first image is subjected to deformation processing according to a relative position relationship between the vertex coordinates, and then the processed license plate region is covered in the license plate region in the first image.
For example, each side length of the license plate region can be calculated according to the distance between the coordinates of each vertex of the license plate region in the first image, and then the license plate region in the simulated license plate image is zoomed according to the ratio of each side length to each side length of the license plate region in the simulated license plate image; for another example, according to the relationship between the coordinates of each vertex of the license plate region in the first image, the included angle between each side of the license plate region and the horizontal line can be calculated, and then the license plate region in the simulated license plate image can be rotated according to the included angle. After the processing, the license plate area in the processed simulated license plate image can be covered in the license plate area in the first image according to the corresponding vertexes.
In another embodiment, a mask image of a license plate region in the first image may be generated, the simulated license plate image may be deformed according to a region parameter of the license plate region in the mask image, the deformed simulated license plate image may be fused to the license plate region in the mask image to obtain a third image, and then the third image and the first image may be synthesized to obtain a second image. Detailed description of the preferred embodimentsreferring to steps S203-S205 in the embodiment of fig. 2, detailed description thereof will be omitted.
Step S104: and carrying out harmony processing on the image style of the license plate region in the second image based on the image style of the background region in the second image to obtain a first sample image obtained based on the processing result.
The image style can be characterized by the color parameters of the image, such as brightness, contrast, saturation, etc., or by the parameters of the texture features of the image, etc., and the image styles of different images are different.
The steps can be seen that the second image is obtained by fusing the first image and the simulated license plate image, wherein the license plate region in the second image is the license plate region in the simulated image, and the background region in the second image is the background region in the first image.
The harmonious processing can enable the image style of the license plate region to approach the image style of the background region, and the background region is the background region in the first image collected in the actual scene, so that the image style of the license plate region of the generated first sample image approaches the image style of the actual scene, and the trueness degree of the license plate region of the generated first sample image is improved.
Specifically, the background region and the license plate region in the second image may be determined in the following manner.
In one embodiment, the target color parameter of the license plate region in the image may be preset, so that when the background region and the license plate region are determined, the region in the second image with the color parameter being the target color parameter may be identified, the identified region may be determined as the license plate region in the second image, and the remaining region may be determined as the background region. When the color difference between the background area and the license plate area in the second image is large, the background area and the license plate area in the second image can be accurately determined by using the method.
In another embodiment, the license plate region may be determined based on a mask image of the license plate region in the first sample image, which is mentioned below. Specifically, a region in the second image corresponding to a mask image with a pixel value of 255 may be determined as a license plate region in the second image, and a region in the second image corresponding to a mask image with a pixel value of 0 may be determined as a background region in the second image. The generation manner of the mask image is detailed in step S201 in the embodiment shown in fig. 2, and will not be detailed here.
As can be seen from the above, when the scheme provided by the embodiment of the present disclosure is applied to generate a sample image, after a simulated license plate image is obtained, a license plate region in the simulated license plate image is fused to a license plate region in a first image, so as to obtain a second image, and based on an image style of a background region in the second image, a harmony processing is performed on the image style of the license plate region in the second image, so that the image style of the license plate region in the second image approaches to the image style of the background region, and thus, based on a processing result, the first sample image can be generated.
Moreover, when the simulated license plate image is fused to the license plate region in the first image, the obtained background region of the second image is the background region in the real environment in the first image, and then the image style of the license plate region in the second image is harmoniously processed based on the image style of the background region in the second image, so that the image style of the license plate region of the generated first sample image approaches to the image style in the actual scene, the true degree of the generated first sample image is improved, and the accuracy of the license plate recognition model for recognizing the license plate can be improved by training the license plate recognition model by using the first sample image.
In an embodiment of the present disclosure, after obtaining the first sample image, a vertex of the license plate region in the first sample image may be further determined, and then the determined vertex is subjected to random perturbation processing to obtain a second sample image.
Specifically, a floating value may be preset, the position of the vertex of the license plate region in the first sample image is randomly changed within the range of the floating value, and the license plate region of the second sample image is deformed based on the vertex after the position is changed, so that the license plate region is adapted to the vertex after the position is changed. Therefore, by carrying out random disturbance processing on the vertex of the license plate region in the first sample image, a new second sample image can be obtained, and the number of the generated sample images is further increased.
In one embodiment of the present disclosure, after obtaining the first sample image, the license plate region in the first sample image may also be determined, and then a third sample image containing only the determined license plate region is obtained.
Specifically, a copy of the first sample image may be obtained, and the license plate region determined by the copy of the first image in the above manner is cropped, so that the obtained new image is the third sample image only including the determined license plate region. The method for determining the license plate region in the first sample image may refer to the embodiments provided above, and details are not repeated here.
Therefore, a third sample image only containing the license plate area can be obtained, and the number of the generated sample images is further increased.
On the basis of the embodiment shown in fig. 1, when the license plate region of the simulated license plate image is fused to the license plate region in the first image, a mask image of the license plate region in the first image may be generated, the simulated license plate image is deformed according to the region parameters of the license plate region in the mask image, the deformed simulated license plate image is fused to the license plate region in the mask image, a third image is obtained, and then the third image and the first image are synthesized to obtain a second image.
In view of the foregoing, the present disclosure provides another license plate sample image generation method.
Referring to fig. 2, fig. 2 is a schematic flow chart of another license plate sample image generation method provided in the embodiment of the present disclosure, where the method includes the following steps S201 to S206.
Step S201: and obtaining a simulated license plate image.
Step S202: a first image which is collected in an actual scene and contains a license plate area is obtained.
The steps S201 and S202 are the same as the steps S101 and S102 in the embodiment shown in fig. 1, and are not described again here.
Step S203: a mask image of the license plate region in the first image is generated.
The mask image is a binary image.
Specifically, the license plate region in the first image may be determined, and then the pixel value of the license plate region is set to 255, and the pixel value of the non-license plate region is set to 0, so as to generate a binary mask image of the license plate region in the first image.
Wherein the license plate region in the first image may be determined in the following manner.
In one embodiment, the edge image feature of the first image may be extracted, and then the license plate region in the first image may be identified according to the edge image feature. For example, in a region surrounded by continuous edge feature points, a region including character edge feature points in the region is selected as a license plate region.
In another embodiment, the target image feature of the license plate region may be preset, and in this case, when the license plate region in the first image is determined, the image feature of the first image may be extracted, and the image region corresponding to the feature, which is matched with the target image feature, in the image feature may be determined as the license plate region in the first image.
Step S204: and deforming the simulated license plate image according to the regional parameters of the license plate region in the mask image, and fusing the deformed simulated license plate image to the license plate region in the mask image to obtain a third image.
The region parameters of the license plate region in the mask image may include the length and width of the license plate region, a target included angle between a long side of the license plate region and a horizontal line, and the like.
The morphing may include zooming and rotating. Specifically, the simulated license plate image may be deformed according to the following steps a and B.
Step A: and determining a scaling parameter based on the length and the width of the license plate area in the mask image and the length and the width of the license plate area in the simulated license plate image, and scaling the simulated license plate image according to the scaling parameter. For example: the length and width of the region parameters of the license plate region are 440 pixels and 140 pixels, respectively, while the length and width of the license plate region in the simulated license plate image are 220 pixels and 70 pixels, respectively, so that it can be seen that the scaling is 2, and the length and width of the license plate region in the simulated license plate image need to be enlarged by 2 times.
And B: and rotating the zoomed simulated license plate image according to the target included angle.
Specifically, the rotating direction is determined according to whether the target included angle is larger than 0, and then the zoomed simulated license plate image is rotated by the target included angle according to the rotating direction.
After the simulated license plate image is deformed, the deformed simulated license plate image can be fused to the license plate area in the mask image in the following mode, and a third image is obtained.
Specifically, the vertex coordinates of the license plate region in the mask image can be determined, so that the license plate region in the simulation image can be covered in the mask image according to the vertex coordinates of the license plate region in the mask image, the vertex coordinates of the license plate region in the simulation image corresponding to the virtual license plate region are the same as the coordinates of the license plate region in the mask image, the license plate region in the simulation image is fused to the license plate region in the mask image, and the third image is generated.
Step S205: and carrying out image synthesis on the third image and the first image to obtain a second image.
Specifically, the second image may be obtained by image-synthesizing the third image and the first image in the following manner.
In one embodiment, the second image may be obtained by merging the contents of the license plate region in the third image into the first image. For convenience of expression, a pixel point in a background region in the third image is called a first pixel point, a pixel point in a license plate region in the third image is called a second pixel point, a pixel point in the background region in the first image is called a third pixel point, and a pixel point in the license plate region in the first image is called a fourth pixel point.
It can be known from the foregoing that the background region in the third image is the background region in the mask image, and the pixel value of the pixel point in the region is 0, so the binary representation after the complement of the pixel value is 11111111, and since the result of the and calculation of the 11111111 and the arbitrary value is the arbitrary value, the pixel value of the third pixel point cannot be changed after the complement of the pixel value of the first pixel point is inverted and the pixel value of the third pixel point is and calculated. Based on this, comparing the second image with the first image, the pixel values of the pixels in the background region of the two images are the same, but since the pixel value of the fourth pixel is set as the pixel value of the second pixel when the second image is generated, the pixel values of the pixels in the license plate region in the second image and the first image are different and are the same as the pixel value of the pixels in the license plate region in the third image.
For convenience of explanation, a specific process of synthesizing the two images a and b to obtain the image c will be described below with reference to fig. 3 as an example. Suppose that the pixel points at the lower right corner in the image a are the pixel points in the license plate region, and the other pixel points are the pixel points in the background region. As can be seen from fig. 3, the pixel value of each pixel in the background region in the image a is 0, and the complement of the pixel is negated and then anded with the pixel values of three pixels in the image b, so that the pixel values 96, 56, and 112 in the image b are not changed, and are retained in the image c. In addition, the pixel value of the lower right pixel point in the image c is set to be the pixel value 23 of the lower right pixel point in the image a. This generates image c.
In another embodiment, the license plate region can be removed from the first image, and then the first image and the third image after the license plate region is removed are subjected to image synthesis to obtain a second image. The method for combining the first image and the third image after the license plate region is removed is the same as the embodiment, and is not repeated here. Therefore, after the license plate region is removed from the first image, the first image and the third image are subjected to image synthesis, so that the influence of the original license plate region contained in the first image on a synthesis result can be reduced, and the first image and the third image are accurately subjected to image synthesis to obtain a second image.
Step S206: and carrying out harmony processing on the image style of the license plate region in the second image based on the image style of the background region in the second image to obtain a first sample image obtained based on the processing result.
Step S206 is the same as step S104 in the embodiment shown in fig. 1, and is not described here again.
A sample image generation process provided by this example is explained below with reference to fig. 4.
As can be seen from fig. 4, after the pseudo-real license plate image is fused to the license plate region in the mask image, the license plate region in the generated third image is the license plate region in the pseudo-real license plate image, and the background region is the background region in the mask image. And then when the third image is synthesized with the first image, the obtained second image not only contains the license plate region in the mask image, but also contains the background region in the actual scene in the first image. And then, carrying out image harmony processing on the license plate region in the second image according to the image style of the background region in the second image, so that the image style of the license plate region approaches the image style of the background region in the actual scene, and further a more real first sample image is generated.
Therefore, after the mask image of the license plate region in the first image is generated, as the mask image is a binary image, each pixel point in the license plate region corresponds to one pixel value, each pixel point in the background region corresponds to another pixel value, and only two pixel values enable the license plate region in the mask image to be more distinct, so that when the virtual license plate image is fused to the license plate region in the mask image, the license plate region in the mask image can be accurately determined, and errors generated in the fusion process are reduced.
On the basis of the embodiment shown in fig. 1, when the simulated license plate image is obtained, simulated license plate images in a plurality of preset image formats generated according to license plate rules may be obtained, and then for each simulated license plate image, a style migration network model corresponding to the image format of the simulated license plate image is adopted to perform style migration processing on the simulated license plate image, so as to obtain the simulated license plate image based on the processing result.
In view of the foregoing, the present disclosure provides another license plate sample image generation method.
Referring to fig. 5, fig. 5 is a schematic flowchart of a license plate sample image generation method provided in an embodiment of the present disclosure, where the method includes the following steps S501 to S505.
Step S501: and obtaining the simulated license plate images in various preset image formats generated according to the license plate rules.
The plurality of preset image formats may include an RGB format, a binary format, a gray scale format, a skeleton format, and the like.
In one embodiment, the simulated license plate images in the plurality of preset image formats may be generated in advance and stored in a database, in which case, the simulated license plate images may be read from the database.
In another embodiment, the license plate character sequence, the license plate character attribute and the license plate background attribute can be determined according to the license plate rule, and then the license plate character sequence, the license plate character attribute and the license plate background attribute are input into a simulated license plate generation model which is trained in advance to generate simulated license plate images in various preset image formats.
The license plate character attributes may include color, size, font, etc. of characters in the license plate.
The license plate background attributes may include a color of the license plate background, a gradient range, and the like.
The rules of the license plate are different according to different countries, different vehicle purposes and different vehicle types. For example: the number plate character sequence of the number plate of country a is: city identification + vehicle use identification + license plate number, the license plate character sequence of the license plate of country B is: city identification + license plate number + vehicle use identification; the background colors of the license plates of vehicles with different purposes in the same country are different; and the colors of the license plates of different types of vehicles can be different, for example, the license plate of a common small-sized vehicle is in a blue background and a white font, and the license plate of a new energy vehicle is in a gradually-changed green background and a black font.
Therefore, the license plate character sequence, the license plate character attribute and the license plate background attribute which accord with the preset license plate rules can be randomly generated according to the preset license plate rules, and then the license plate character sequence, the license plate character attribute and the license plate background attribute are input into a pre-trained simulated license plate generation model to generate simulated license plate images in various preset image formats.
Because the preset license plate rule is set according to the license plate rule in the actual scene, after the information generated according to the preset license plate rule is input into the pre-trained simulated license plate generation model, the model can generate the license plate image which accords with the license plate rule in the actual scene.
In an embodiment of the disclosure, when the pre-trained simulated license plate generation model generates the simulated license plate image according to the license plate character sequence, the license plate character attribute and the license plate background attribute, factors such as weather and illumination of an actual scene can be considered, so that the generated simulated license plate image can change along with changes of external factors such as weather and illumination as the actual scene, and the similarity between the generated simulated license plate image and the license plate image in the actual scene is higher.
Step S502: and aiming at each simulated license plate image, performing style migration processing on the simulated license plate image by adopting a style migration network model corresponding to the image format of the simulated license plate image to obtain a simulated license plate image based on the processing result.
Wherein, the style migration network model is as follows: the resulting network model is trained in advance by learning the image style of sample images acquired in an actual scene.
The style migration network model is obtained by pre-training through learning the image style of the sample image collected in the actual scene, and the style migration network model is used for carrying out style migration processing on the simulated license plate image, so that the image style of the simulated license plate image approaches to the actual scene.
Because the images in different formats have different image characteristics, in this embodiment, the style migration network model is respectively trained in advance for different image formats, so as to better perform style migration on the images in different formats.
Referring to fig. 6, which is a schematic diagram of an image style migration process provided in an embodiment of the present invention, it can be seen that, for different image formats, such as an RGB diagram, a grayscale diagram, a binary diagram, a skeleton diagram, and the like, the style migration is performed by the style migration network model corresponding to the image formats, so that after the style migration processing is performed on the image by using the corresponding style migration network, a simulated license plate image closer to an actual scene can be obtained.
Step S503: a first image which is collected in an actual scene and contains a license plate area is obtained.
Step S504: and fusing the license plate region in the simulated license plate image to the license plate region in the first image to obtain a second image.
Step S505: and carrying out harmony processing on the image style of the license plate region in the second image based on the image style of the background region in the second image to obtain a first sample image obtained based on the processing result.
The steps S503-S505 are the same as the steps S102-S104 in the embodiment shown in FIG. 1, and are not repeated here.
As can be seen from the above, in the embodiment, when the style migration network model is selected to perform the style migration on the simulated license plate image, the style migration network model for the image format is adopted according to the image format of the obtained simulated license plate image, so that after the style migration network model is used to perform the style migration on the simulated license plate image, the simulated license plate image closer to the actual scene can be obtained.
Corresponding to the license plate sample image generation method, the invention also provides a license plate sample image generation device.
Referring to fig. 7, fig. 7 is a schematic structural diagram of a license plate sample image generating device according to an embodiment of the present disclosure, where the device includes the following modules 701 and 704.
A simulated license plate image obtaining module 701, configured to obtain a simulated license plate image;
a first image obtaining module 702, configured to obtain a first image that is collected in an actual scene and includes a license plate region;
the image fusion module 703 is configured to fuse a license plate region in the simulated license plate image to a license plate region in the first image to obtain a second image;
a first sample image obtaining module 704, configured to perform a harmony processing on the image style of the license plate region in the second image based on the image style of the background region in the second image, and obtain a first sample image obtained based on the processing result.
As can be seen from the above, when the scheme provided by the embodiment of the present disclosure is applied to generate a sample image, after a simulated license plate image is obtained, a license plate region in the simulated license plate image is fused to a license plate region in a first image, so as to obtain a second image, and based on an image style of a background region in the second image, a harmony processing is performed on the image style of the license plate region in the second image, so that the image style of the license plate region in the second image approaches to the image style of the background region, and thus, based on a processing result, the first sample image can be generated.
Moreover, when the simulated license plate image is fused to the license plate region in the first image, the obtained background region of the second image is the background region in the real environment in the first image, and then the image style of the license plate region in the second image is harmoniously processed based on the image style of the background region in the second image, so that the image style of the license plate region of the generated first sample image approaches to the image style in the actual scene, the true degree of the generated first sample image is improved, and the accuracy of the license plate recognition model for recognizing the license plate can be improved by training the license plate recognition model by using the first sample image.
In one embodiment of the present disclosure, the apparatus further comprises:
a vertex determining module for determining a vertex of the license plate region in the first sample image;
and the second sample image obtaining module is used for carrying out random disturbance processing on the determined vertex to obtain a second sample image.
Therefore, by carrying out random disturbance processing on the vertex of the license plate region in the first sample image, a new second sample image can be obtained, and the number of the generated sample images is further increased.
In one embodiment of the present disclosure, the apparatus further comprises:
the license plate region determining module is used for determining a license plate region in the first sample image;
and the third sample image obtaining module is used for obtaining a third sample image only containing the determined license plate area.
Therefore, a third sample image only containing the license plate area can be obtained, and the number of the generated sample images is further increased.
Referring to fig. 8, fig. 8 is a schematic structural diagram of another license plate sample image generation apparatus provided in the embodiment of the present disclosure, where the apparatus includes the following modules 801 and 806.
A simulated license plate image obtaining module 801, configured to obtain a simulated license plate image;
a first image obtaining module 802, configured to obtain a first image that is collected in an actual scene and includes a license plate region;
a mask image generation sub-module 803 for generating a mask image of the license plate region in the first image;
a third image obtaining sub-module 804, configured to deform the simulated license plate image according to the regional parameter of the license plate region in the mask image, and fuse the deformed simulated license plate image to the license plate region in the mask image to obtain a third image;
and an image synthesis sub-module 805, configured to perform image synthesis on the third image and the first image to obtain a second image.
A first sample image obtaining module 806, configured to perform a harmony processing on an image style of a license plate region in the second image based on the image style of the background region in the second image, and obtain a first sample image obtained based on a processing result.
Therefore, after the mask image of the license plate region in the first image is generated, because the mask image is a binary image, each pixel point in the license plate region corresponds to one pixel value, each pixel point in the background region corresponds to another pixel value, and only two pixel values enable the license plate region in the mask image to be more distinct, so that when the virtual license plate image is fused to the license plate region in the mask image, the license plate region in the mask image can be accurately determined, and errors generated in the fusion process are reduced.
In one embodiment of the present disclosure, the image synthesis sub-module 805,
the method comprises the steps of specifically removing a license plate region from a first image; and carrying out image synthesis on the first image and the third image after the license plate region is scratched out to obtain a second image.
Therefore, after the license plate region is removed from the first image, the first image and the third image are subjected to image synthesis, so that the influence of the original license plate region contained in the first image on a synthesis result can be reduced, and the first image and the third image are accurately subjected to image synthesis to obtain a second image.
Referring to fig. 9, fig. 9 is a schematic structural diagram of another license plate sample image generation apparatus provided in the embodiment of the present disclosure, where the apparatus includes the following modules 901-905.
The simulated license plate image obtaining sub-module 901 is used for obtaining simulated license plate images in various preset image formats generated according to license plate rules;
the style migration submodule 902 is configured to, for each simulated license plate image, perform style migration processing on the simulated license plate image by using a style migration network model corresponding to an image format of the simulated license plate image, and obtain a simulated license plate image based on a processing result, where the style migration network model is: the resulting network model is trained in advance by learning the image style of sample images acquired in an actual scene.
A first image obtaining module 903, configured to obtain a first image that is collected in an actual scene and includes a license plate region;
an image fusion module 904, configured to fuse a license plate region in the simulated license plate image to a license plate region in the first image to obtain a second image;
a first sample image obtaining module 905, configured to perform harmony processing on the image style of the license plate region in the second image based on the image style of the background region in the second image, and obtain a first sample image obtained based on a processing result.
As can be seen from the above, in the embodiment, when the style migration network model is selected to perform the style migration on the simulated license plate image, the style migration network model for the image format is adopted according to the image format of the obtained simulated license plate image, so that after the style migration network model is used to perform the style migration on the simulated license plate image, the simulated license plate image closer to the actual scene can be obtained.
In one embodiment of the present disclosure, the simulated license plate obtaining sub-module 901,
the license plate recognition method is specifically used for determining a license plate character sequence, license plate character attributes and license plate background attributes according to license plate rules; and inputting the license plate character sequence, the license plate character attribute and the license plate background attribute into a pre-trained simulated license plate generation model to generate simulated license plate images in various preset image formats.
Because the preset license plate rule is set according to the license plate rule in the actual life, after the information generated according to the preset license plate rule is input into the pre-trained simulated license plate generation model, the model can generate the license plate image which accords with the license plate rule in the actual life.
In the technical scheme of the disclosure, the collection, storage, use, processing, transmission, provision, disclosure and other processing of the personal information of the related user are all in accordance with the regulations of related laws and regulations and do not violate the good customs of the public order.
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
In one embodiment of the present disclosure, there is provided an electronic device including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the license plate sample image generation method of the foregoing method embodiments.
In an embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, in which computer instructions are stored, where the computer instructions are configured to cause the computer to execute the license plate sample image generation method of the foregoing method embodiment.
In an embodiment of the present disclosure, a computer program product is provided, which includes a computer program, and when being executed by a processor, the computer program implements the license plate sample image generation method described in the foregoing method embodiment.
FIG. 10 illustrates a schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 10, the apparatus 1000 includes a computing unit 1001 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)1002 or a computer program loaded from a storage unit 1008 into a Random Access Memory (RAM) 1003. In the RAM 1003, various programs and data necessary for the operation of the device 1000 can also be stored. The calculation unit 1001, the ROM 1002, and the RAM 1003 are connected to each other by a bus 1004. An input/output (I/O) interface 1005 is also connected to bus 1004.
A number of components in device 1000 are connected to I/O interface 1005, including: an input unit 1006 such as a keyboard, a mouse, and the like; an output unit 1007 such as various types of displays, speakers, and the like; a storage unit 1008 such as a magnetic disk, optical disk, or the like; and a communication unit 1009 such as a network card, a modem, a wireless communication transceiver, or the like. The communication unit 1009 allows the device 1000 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.
Computing unit 1001 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 1001 performs the respective methods and processes described above, such as the license plate sample image generation method. For example, in some embodiments, the license plate sample image generation method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 1000 via ROM 1002 and/or communications unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the license plate sample image generation method described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform the license plate sample image generation method in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server combining a blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel or sequentially or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims (17)

1. A license plate sample image generation method comprises the following steps:
obtaining a simulated license plate image;
acquiring a first image which is acquired in an actual scene and contains a license plate area;
fusing the license plate region in the simulated license plate image to the license plate region in the first image to obtain a second image;
and on the basis of the image style of the background area in the second image, carrying out harmony processing on the image style of the license plate area in the second image to obtain a first sample image obtained on the basis of a processing result.
2. The method of claim 1, wherein fusing the license plate region in the simulated license plate image to the license plate region in the first image to obtain a second image comprises:
generating a mask image of a license plate region in the first image;
according to the regional parameters of the license plate region in the mask image, deforming the simulated license plate image, and fusing the deformed simulated license plate image to the license plate region in the mask image to obtain a third image;
and carrying out image synthesis on the third image and the first image to obtain a second image.
3. The method of claim 2, wherein the image-combining the third image and the first image to obtain a second image comprises:
removing a license plate region from the first image;
and carrying out image synthesis on the first image and the third image after the license plate region is scratched out to obtain a second image.
4. The method of any of claims 1-3, wherein after obtaining the first sample image, further comprising:
determining a vertex of a license plate region in the first sample image;
and carrying out random disturbance processing on the determined vertex to obtain a second sample image.
5. The method of claim 1, wherein the obtaining a simulated license plate image comprises:
obtaining simulated license plate images in a plurality of preset image formats generated according to license plate rules;
aiming at each simulated license plate image, adopting a style migration network model corresponding to the image format of the simulated license plate image to perform style migration processing on the simulated license plate image to obtain a simulated license plate image based on a processing result, wherein the style migration network model is as follows: the resulting network model is trained in advance by learning the image style of sample images acquired in an actual scene.
6. The method of claim 5, wherein the obtaining of the simulated license plate image in the plurality of preset image formats generated according to the license plate rules comprises:
determining a license plate character sequence, license plate character attributes and license plate background attributes according to license plate rules;
and inputting the license plate character sequence, the license plate character attribute and the license plate background attribute into a pre-trained simulated license plate generation model to generate simulated license plate images in various preset image formats.
7. The method of any of claims 1-3, wherein after obtaining the first sample image, further comprising:
determining a license plate region in the first sample image;
a third sample image is obtained that only contains the determined license plate region.
8. A license plate sample image generating device, comprising:
the simulated license plate image obtaining module is used for obtaining a simulated license plate image;
the first image acquisition module is used for acquiring a first image which is acquired in an actual scene and contains a license plate area;
the image fusion module is used for fusing the license plate region in the simulated license plate image to the license plate region in the first image to obtain a second image;
and the first sample image obtaining module is used for carrying out harmony processing on the image style of the license plate region in the second image based on the image style of the background region in the second image so as to obtain a first sample image based on the processing result.
9. The apparatus of claim 8, wherein the image fusion module comprises:
the mask image generation submodule is used for generating a mask image of the license plate area in the first image;
the third image obtaining sub-module is used for deforming the simulated license plate image according to the regional parameters of the license plate region in the mask image, and fusing the deformed simulated license plate image to the license plate region in the mask image to obtain a third image;
and the image synthesis submodule is used for carrying out image synthesis on the third image and the first image to obtain a second image.
10. The apparatus of claim 9, wherein,
the image synthesis submodule is specifically used for removing the license plate region from the first image; and carrying out image synthesis on the first image and the third image after the license plate region is scratched out to obtain a second image.
11. The apparatus of any of claims 8-10, further comprising:
a vertex determining module for determining a vertex of the license plate region in the first sample image;
and the second sample image obtaining module is used for carrying out random disturbance processing on the determined vertex to obtain a second sample image.
12. The apparatus of claim 8, wherein the simulated license plate image obtaining module comprises:
the simulated license plate image acquisition sub-module is used for acquiring simulated license plate images in various preset image formats generated according to license plate rules;
the style migration submodule is used for adopting a style migration network model corresponding to the image format of the simulated license plate image for each simulated license plate image, carrying out style migration processing on the simulated license plate image and obtaining a simulated license plate image based on the processing result, wherein the style migration network model is as follows: the resulting network model is trained in advance by learning the image style of sample images acquired in an actual scene.
13. The apparatus of claim 12, wherein,
the simulated license plate obtaining sub-module is specifically used for determining a license plate character sequence, license plate character attributes and license plate background attributes according to license plate rules; and inputting the license plate character sequence, the license plate character attribute and the license plate background attribute into a pre-trained simulated license plate generation model to generate simulated license plate images in various preset image formats.
14. The apparatus of any one of claims 8-10, the apparatus further comprising:
the license plate region determining module is used for determining a license plate region in the first sample image;
and the third sample image obtaining module is used for obtaining a third sample image only containing the determined license plate area.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
16. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-7.
17. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-7.
CN202210167280.XA 2022-02-23 2022-02-23 License plate sample image generation method, device, equipment and storage medium Pending CN114581302A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210167280.XA CN114581302A (en) 2022-02-23 2022-02-23 License plate sample image generation method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210167280.XA CN114581302A (en) 2022-02-23 2022-02-23 License plate sample image generation method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN114581302A true CN114581302A (en) 2022-06-03

Family

ID=81770635

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210167280.XA Pending CN114581302A (en) 2022-02-23 2022-02-23 License plate sample image generation method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN114581302A (en)

Similar Documents

Publication Publication Date Title
CN113657390B (en) Training method of text detection model and text detection method, device and equipment
EP4027299A2 (en) Method and apparatus for generating depth map, and storage medium
CN111739005B (en) Image detection method, device, electronic equipment and storage medium
CN114550177B (en) Image processing method, text recognition method and device
US8340433B2 (en) Image processing apparatus, electronic medium, and image processing method
CN113674421B (en) 3D target detection method, model training method, related device and electronic equipment
CN112560684B (en) Lane line detection method, lane line detection device, electronic equipment, storage medium and vehicle
CN113674287A (en) High-precision map drawing method, device, equipment and storage medium
CN114429637B (en) Document classification method, device, equipment and storage medium
CN113724388B (en) High-precision map generation method, device, equipment and storage medium
CN113362420A (en) Road marking generation method, device, equipment and storage medium
CN113657396B (en) Training method, translation display method, device, electronic equipment and storage medium
CN117315406B (en) Sample image processing method, device and equipment
CN113901998A (en) Model training method, device, equipment, storage medium and detection method
CN113409340A (en) Semantic segmentation model training method, semantic segmentation device and electronic equipment
CN113326766A (en) Training method and device of text detection model and text detection method and device
CN117745944A (en) Pre-training model determining method, device, equipment and storage medium
CN114882313B (en) Method, device, electronic equipment and storage medium for generating image annotation information
CN114998897B (en) Method for generating sample image and training method of character recognition model
CN114511862B (en) Form identification method and device and electronic equipment
CN113781653B (en) Object model generation method and device, electronic equipment and storage medium
CN115375847A (en) Material recovery method, three-dimensional model generation method and model training method
CN115311403A (en) Deep learning network training method, virtual image generation method and device
CN114581302A (en) License plate sample image generation method, device, equipment and storage medium
CN114510996A (en) Video-based vehicle matching method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination