CN117725614A

CN117725614A - License plate desensitizing method and device, electronic equipment and storage medium

Info

Publication number: CN117725614A
Application number: CN202311790306.7A
Authority: CN
Inventors: 杨娇娇; 孙辉; 戴一凡
Original assignee: Tsinghua University; Suzhou Automotive Research Institute of Tsinghua University
Current assignee: Tsinghua University; Suzhou Automotive Research Institute of Tsinghua University
Priority date: 2023-12-25
Filing date: 2023-12-25
Publication date: 2024-03-19

Abstract

The invention discloses a license plate desensitizing method, a license plate desensitizing device, electronic equipment and a storage medium. The method comprises the following steps: acquiring at least one target image corresponding to a target vehicle included in a target video; inputting the target image into a pre-trained first model to obtain license plate key point information, wherein the license plate key point information corresponds to the target image one by one, the first model is a model which is obtained by training by using a deep reinforcement learning method based on a training set and taking the optimized recognition accuracy as an index, and the training set at least comprises a distortion training image and/or a shielding training image; and according to license plate key point information, desensitizing the license plate of the target vehicle in the target video. The technical scheme of the invention can realize accurate identification and desensitization of the license plate to be desensitized with shielding and distortion, and improves the accuracy of license plate desensitization.

Description

License plate desensitizing method and device, electronic equipment and storage medium

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to a license plate desensitizing method, device, electronic device, and storage medium.

Background

In order to ensure personal information safety of a vehicle owner, when a monitoring camera is used for monitoring a vehicle, desensitization treatment is required to be carried out on the vehicle.

In order to make the monitoring range larger, in monitoring a vehicle, a camera sensor such as a fish-eye camera is often used to monitor the vehicle. However, although the coverage of the photo collected by the fisheye camera is wide, there are problems that the collected image is mostly a distorted image, and the quality of the collected image is low, and the like. Therefore, when the image acquired by the fisheye camera is desensitized, if the acquired image is not corrected, the detection accuracy of the target to be desensitized is reduced due to higher distortion degree of the image or shielding of the image, so that the conditions of missed detection and false detection exist.

Disclosure of Invention

The invention provides a license plate desensitization method, device, electronic equipment and storage medium, which are used for realizing accurate identification and desensitization of a license plate to be desensitized with shielding and distortion, and improving the accuracy of license plate desensitization.

According to an aspect of the present invention, there is provided a license plate desensitizing method comprising:

acquiring at least one target image corresponding to a target vehicle included in a target video;

inputting the target image into a pre-trained first model to obtain license plate key point information, wherein the license plate key point information corresponds to the target image one by one, the first model is a model which is obtained by training by using a deep reinforcement learning method based on a training set and taking the optimized recognition accuracy as an index, and the training set at least comprises a distortion training image and/or a shielding training image;

And according to license plate key point information, desensitizing the license plate of the target vehicle in the target video.

According to another aspect of the present invention, there is provided a license plate desensitizing apparatus comprising:

the acquisition module is used for acquiring at least one target image corresponding to a target vehicle included in the target video;

the first training module is used for inputting the target image into a pre-trained first model to obtain license plate key point information, wherein the license plate key point information corresponds to the target image one by one, the first model is a model obtained by training by using a deep reinforcement learning method based on a training set and taking the optimized recognition accuracy as an index, and the training set at least comprises a distortion training image and/or a shielding training image;

and the desensitization module is used for desensitizing the license plate of the target vehicle in the target video according to the license plate key point information.

According to another aspect of the present invention, there is provided an electronic apparatus including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the license plate desensitization method of any of the embodiments of the present invention.

According to another aspect of the present invention, there is provided a computer readable storage medium having stored thereon computer instructions for causing a processor to perform the license plate desensitization method of any of the embodiments of the present invention when executed.

According to the license plate desensitization method, at least one target image corresponding to a target vehicle in a target video is acquired, the target image is input into a first model trained in advance, license plate key point information is obtained, and the license plate of the target vehicle is desensitized in the target video according to the license plate key point information. According to the technical scheme, on one hand, the target vehicle is extracted from the target video, and the image of the corresponding area of the target vehicle is used as the target image, so that the image frames are extracted from the video to a certain extent, the target image is accurately extracted from a large number of image frames, and the accuracy of desensitizing the license plate of the target vehicle from the target video is improved. On the other hand, the target image is input into a first model trained in advance to obtain license plate key point information, so that when distortion and/or shielding exists in the current target image, the license plate key point information can be determined directly according to the license plate in the target image corresponding to the target vehicle. Finally, the license plate key point information of the target image is obtained, and the license plate top angle position of the target image corresponding to the target vehicle is found in the target video, so that the license plate is desensitized, the problem that the license plate to be desensitized cannot be accurately identified due to poor image quality when distortion and/or shielding exists in the target video is solved, the conditions of missing detection and false detection are reduced, and the license plate desensitization accuracy is improved.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic flow chart of a license plate desensitizing method provided by the invention;

FIG. 2 is a schematic flow chart of a license plate desensitizing method according to the present invention;

FIG. 3 is a diagram illustrating a second model training method according to the present invention;

FIG. 4 is an exemplary diagram of a target image frame provided by the present invention;

FIG. 5 is a schematic diagram of a license plate desensitizing device according to the present invention;

fig. 6 is a schematic structural diagram of an electronic device provided by the present invention.

Detailed Description

In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.

It should be noted that the terms "first," "second," "target," "original," and the like in the description and claims of the present invention and in the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Fig. 1 is a schematic flow chart of a license plate desensitizing method provided by the invention, where the embodiment is applicable to the condition of desensitizing a license plate to be desensitized in a picture or a video, and optionally, shielding and/or distortion may exist in the license plate to be desensitized. As shown in fig. 1, the method includes:

S101, at least one target image corresponding to a target vehicle included in the target video is acquired.

The target video is a video of a license plate or a vehicle to be desensitized. The target vehicle is a vehicle corresponding to the license plate to be desensitized. The target image may be an image corresponding to the target vehicle in an image frame of the target video.

Specifically, when a target video is obtained, a target image corresponding to a target vehicle in the target video is obtained. Illustratively, it is assumed that the target video is processed frame by frame to obtain a target image frame for each frame. In the target image frame, a vehicle and a non-vehicle may be included. Therefore, it is possible to extract the target vehicle from each target image frame by means such as image recognition software, a deep neural network, or the like, and take the vehicle image of the corresponding region in each target image frame as the target image with the region in which the extracted target vehicle is located.

In the embodiment, the target vehicle is extracted from the target video, and the image of the corresponding area of the target vehicle is used as the target image, so that the image frames are extracted from the video to a certain extent, the target image is accurately extracted from a large number of image frames, and the accuracy of desensitizing the license plate of the target vehicle from the target video is improved.

S102, inputting a target image into a pre-trained first model to obtain license plate key point information, wherein the license plate key point information corresponds to the target image one by one, the first model is a model obtained by training by using a deep reinforcement learning method based on a training set and taking optimized recognition accuracy as an index, and the training set at least comprises a distortion training image and/or a shielding training image.

The license plate key point information can be four vertex angle positions of the license plate.

Specifically, the first model needs to be trained before the target image is input into the first model trained in advance.

For example, the training method of the first model may be trained as follows: a training set is obtained that includes distorted training images and/or occlusion training images. In each training image, there is a piece of standard labeling information. The standard labeling information can be information for manually labeling the training image, or standard labeling information preset by a user. After the training set is acquired, inputting the current training image into an original model, and determining actual labeling information of the current training image; if the error between the actual labeling information of the current training image and the standard labeling information of the current training image is smaller than or equal to a second preset threshold value, determining the original model as a first model; if the error between the actual labeling information of the current training image and the standard labeling information of the current training image is larger than a second preset threshold value, adjusting parameters of the original model, taking the next training image as the current training image, and returning to execute the steps of inputting the current training image into the original model and determining the actual labeling information of the current training image. The second preset threshold is a preset training error. Adjusting parameters of the original model may include adjusting a loss function, an optimizer, etc. of the training model.

It should be noted that, in this embodiment, the labeling information is license plate key point information, and in each training image, the standard labeling information is license plate key point information of the standard of each training sample vehicle in each training image. And similarly, inputting the current training image into the original model to obtain actual labeling information, namely actual license plate key point information of each training sample vehicle generated by the original model. The consistency of standard license plate key point information and actual license plate key point information depends on the training degree of the original model.

After the first model is trained in advance, when a target image is obtained, the target image can be input into the first model, and license plate key point information is obtained.

In this embodiment, the target image is input into the first model trained in advance to obtain license plate key point information, so that when distortion and/or shielding exists in the current target image, the license plate key point information can be determined directly according to the license plate in the target image corresponding to the target vehicle.

S103, desensitizing the license plate of the target vehicle in the target video according to the license plate key point information.

Specifically, after license plate key point information in the target image is determined by using the first model, a corresponding area of the target image in the target video can be determined according to the license plate key point information and the target vehicle. Meanwhile, when the corresponding area of the target image in the target video is determined, the license plate of the target vehicle can be desensitized in the target video.

Illustratively, license plate keypoint information in the target image is determined using the first model; determining a corresponding image frame of a target vehicle corresponding to the target image in the target video and a corresponding region in the image frame; and determining four vertex angle positions of a license plate of the target vehicle in the target video according to the license plate key point information, and desensitizing an area obtained by sequentially connecting the vertex angle positions.

In the embodiment, the license plate key point information of the target image is obtained, and the license plate top angle position of the target image corresponding to the target vehicle is found in the target video, so that the license plate is desensitized, the problem that the license plate to be desensitized cannot be accurately identified due to poor image quality when distortion and/or shielding exists in the target video is solved, the conditions of missing detection and false detection are reduced, and the license plate desensitization accuracy is improved.

Fig. 2 is a schematic flow chart of a license plate desensitizing method according to the present invention, and the embodiment is based on the above embodiment, to describe in detail the step of acquiring a target image and the step of desensitizing a license plate of a target vehicle in a target video according to license plate key point information. As shown in fig. 2, the method includes:

s201, determining a target data set according to the target video.

Wherein the target data set includes M consecutive first image frames, M being a positive integer.

In particular, from the target video, a target data set may be determined. The target data set can be used for splitting a target video according to the dimension of the frame to obtain M continuous image frames serving as a first image frame.

S202, inputting M pieces of first image frames into a pre-trained second model to obtain M Zhang Dier image frames.

The first image frame corresponds to the second image frame one by one, the second image frame comprises at least one vehicle image of a vehicle, and one vehicle corresponds to one vehicle number. The second image frame may also include a vehicle type and a vehicle identification indicating whether the vehicle is occluded and/or distorted. The second model may be a model having feature information of the extracted input image, such as a deep neural network model including a feature extraction network, a region extraction network, a classification and location regression network, and the like.

Specifically, the second model needs to be trained before M first image frames are input into the pre-trained second model. The training method of the second model may be implemented as follows, for example.

Fig. 3 is a diagram illustrating an example of a second model training method according to the present invention. Fig. 3 shows a basic network schematic of the second model. First, a sample set 301 is obtained, the sample set 301 comprising a number of distorted sample images and/or occlusion sample images. Inputting the sample set 301 to a feature extraction network, and extracting feature information of each image in the sample set 301 by a feature extraction network 302; in the present embodiment, the feature information may be vehicle position information in the sample image, information of the type of the vehicle, whether the vehicle is blocked, and the like. After the feature extraction network 302 extracts the feature information of each image in the sample set 301, the region extraction network 303 is utilized to perform region extraction on the feature information in each sample image; in this embodiment, the extracted area may be the position of each vehicle in the sample image, for example, if there are 3 vehicles in one sample image, the extracted area is the area where the three vehicles in the sample image are respectively located. After the feature information in each image is subjected to region extraction by the region extraction network 303, the feature information is classified in all sample images by the classification and position regression network 304, and the position of each feature information in each sample image is predicted; in this embodiment, the classification and location regression network 304 may be used to determine whether there is a vehicle with the same feature information in all the sample images, i.e., whether there is the same vehicle in all the sample images; when the same vehicle exists in the different sample images, a unique vehicle number is allocated to the vehicle, and the position of the vehicle in the different sample images is determined by using the classification and position regression network 304; if there are vehicles which are different and not assigned with numbers in all the sample images, unique vehicle numbers are assigned to the vehicles respectively.

After the second model is successfully trained according to the method steps, M pieces of first image frames can be input into the second model, and M Zhang Dier image frames are obtained. Each second image frame comprises at least one vehicle image of the vehicle and a corresponding vehicle number.

In the embodiment, training a plurality of first image frames included in a target data set by using a pre-trained second model to obtain vehicle images of all vehicles in each first image frame and vehicle numbers corresponding to the vehicles, thereby realizing acquisition of the vehicles in a large number of image frames; meanwhile, the same vehicle in the image frame can be tracked by utilizing the type of the vehicle and the number of the vehicle in the second image frame, the possibility of missing detection and false detection of the vehicle when the vehicle in some images is blocked or distorted is reduced, and the accuracy of detecting and tracking the vehicle in the images is improved.

S203, obtaining the confidence information of the vehicle image of the target vehicle according to the vehicle number of the target vehicle.

The confidence information is the probability that the overall parameter value falls in a certain area of the sample statistical value, and in this embodiment, the confidence information is the confidence level that the vehicle image of the target vehicle is the real vehicle. That is, the vehicle image of the target vehicle obtained as described above may not be a real vehicle.

Specifically, when the vehicle number corresponding to the vehicle is obtained by using the second model, the confidence information of the vehicle image of the target vehicle needs to be obtained according to the vehicle number of the target vehicle. Notably, due to the fact that the target video is distorted and/or the target is occluded, i.e. the vehicle identification in the second image frame indicates whether the vehicle is occluded and/or distorted. Thus, after training the first image frame in the target dataset with the second model, the target vehicle in the resulting second image frame is not a real vehicle, i.e. there may be a "vehicle" in the target vehicle that is misdetected because the image is occluded and/or distorted. Therefore, it is necessary to determine whether the vehicle of the same vehicle number in each image frame is a real vehicle or a "vehicle" due to false detection using the confidence information of the vehicle image of the target vehicle.

S204, executing S205 if the confidence information of the vehicle images of the target vehicles are smaller than a first preset threshold value; if at least one is greater than or equal to the first preset threshold, then S206 is performed.

The first preset threshold is a preset threshold of confidence information. The first preset threshold may be defined according to user requirements.

Specifically, since there may be a target vehicle having the same number in a plurality of second image frames among the obtained second image frames. Therefore, among a plurality of target vehicles having the same number, there may be a possibility that the confidence of the vehicle image thereof is small due to the target vehicle being blocked or the degree of distortion of the image itself being large. Therefore, the confidence information of the vehicle images of all the target vehicles in the second image frame needs to be compared to determine whether the confidence information of the vehicle images of the target vehicles of the same number are smaller than the first preset threshold value. If the target images are smaller than the first preset threshold value, setting the target images to be empty; and if the confidence information of at least one vehicle image is greater than or equal to a first preset threshold value, taking the vehicle images of all the target vehicles with the vehicle numbers as target images.

S205, setting the target image to be empty.

Specifically, if the confidence information of the vehicle images of all the target vehicles with the same vehicle number is smaller than the first preset threshold, the target vehicle with the current vehicle number may be considered as a false detection, and the target vehicle may not be a real vehicle. Thus, the target image can be cleared to reduce subsequent processing of the target image.

S206, taking vehicle images of all target vehicles as target images.

Specifically, if the confidence information of the vehicle image of at least one target vehicle of the same vehicle number is greater than or equal to the first preset threshold value, the vehicle image of the target vehicle of the vehicle number may be regarded as the target image.

In this embodiment, whether the confidence information of the vehicle images of the target vehicles with the same number is smaller than the first preset threshold value or not is determined to determine the real vehicle in the second image frame, so as to reduce false detection and missing detection of the target vehicles caused by distortion of the images and/or target shielding, and improve detection accuracy of the target vehicles.

S207, inputting the target image into a pre-trained first model to obtain license plate key point information.

Specifically, after a first model is trained in advance, when a target image is obtained, the target image can be input into the first model to obtain license plate key point information.

Illustratively, in the obtained license plate key point information, besides the license plate key point of the normal vehicle, the embodiment can also obtain the license plate sprayed by the vehicle itself, for example, the key point of the sprayed license plate of the vehicle on the large truck.

S208, determining a target image frame.

The target image frames are image frames of target images corresponding to license plate key point information in a target video.

Specifically, after the target image is input into the first model and license plate key point information is obtained, the image frame of the target image in the target video needs to be determined. Thus, a target image frame needs to be determined. For example, assuming that license plate key point information of a current target image is the 3 rd frame in the target video, the target image frame is the 3 rd frame.

S209, determining four vertex angle positions of a license plate of the target vehicle in the target image frame according to the license plate key point information.

Specifically, after the target image is determined to be the target image frame in the target video, four vertex angle positions corresponding to the license plate of the target vehicle in the target image frame can be determined according to the license plate key point information.

Illustratively, as shown in fig. 4, fig. 4 is an exemplary diagram of a target image frame provided by the present invention. It is assumed that, as shown in fig. 4, in one target image frame, there are two license plates to which the target vehicles respectively correspond. Wherein the license plate 1 of the target vehicle 1 is not blocked, and the license plate 2 of the target vehicle 2 is blocked. For license plate 1, according to the license plate key point information, four vertex angle positions corresponding to license plate 1 of target vehicle 1 in the target image frame, namely A1, A2, A3 and A4, can be determined. For the license plate 2, because it is blocked by the blocking object, four vertex angle positions corresponding to the license plate 2 of the target vehicle 2 in the determined target image frame are B1, B2, B3 and B4 according to the key point information of the license plate, i.e. for the license plate in the blocked condition, the vertex angle positions are the intersection positions of the license plate and the blocking object, as shown in B3 and B4 in fig. 4. In order to show a distorted image, the license plate is therefore shown in a distorted shape, for example.

In this embodiment, license plate key point information in the target vehicles with the same number is determined, and image frames of target images corresponding to the license plate key point information in the target video are determined to obtain target image frames, so that the target vehicles in the corresponding target image frames are found directly according to the target images of the target vehicles.

And S210, sequentially connecting the four vertex angle positions to obtain a target area.

Wherein the target area is an area where desensitization is required.

Specifically, after four vertex angle positions are obtained, the four vertex angle positions are sequentially connected to obtain a region to be desensitized. For example, as shown in FIG. 4, for license plate 1, after A1, A2, A3 and A4 are obtained, A1, A2, A3 and A4, such as A1-A2-A3-A4, may be sequentially connected. For license plate 2, after B1, B2, B3 and B4 are obtained, B1, B2, B3 and B4, such as B1-B2-B3-B4, may be sequentially connected. And after the four vertex angle positions are connected in sequence, the target area can be obtained.

S211, desensitizing the target area.

Specifically, after the target area is obtained according to the vertex angle position, desensitization treatment can be performed on the target area. For example, the target area may be coded to achieve desensitization of the license plate.

In the embodiment, the license plate vertex angle position of the target vehicle in the target image frame is directly determined through license plate key point information in the target vehicle, and the target area obtained after the vertex angle position is connected is subjected to desensitization treatment, so that the rapid desensitization treatment of the license plate is realized; meanwhile, the accurate identification and desensitization of the license plate to be desensitized with shielding and distortion are realized, and the accuracy of the desensitization of the license plate is improved.

According to the license plate desensitization method provided by the invention, a target data set is determined according to a target video; inputting the first image frames into a pre-trained second model to obtain a corresponding number of second image frames; acquiring the confidence information of the vehicle image of the target vehicle according to the vehicle number of the target vehicle, and determining whether the confidence information of the vehicle image of the target vehicle is smaller than a first preset threshold value; when the images are smaller than each other, setting the target image to be empty; when at least one of the images is not satisfied, taking the vehicle images of all the target vehicles as target images, and inputting the target images into a first model trained in advance to obtain license plate key point information; and determining a target image frame, determining four vertex angle positions of a license plate of the target vehicle in the target image frame according to the key point information of the license plate, sequentially connecting the four vertex angle positions to obtain a target area, and performing desensitization treatment on the target area. According to the technical scheme, on one hand, the first image frames included in the target data set are trained by using the pre-trained second model, so that the vehicle images of all vehicles in each first image frame and the vehicle numbers corresponding to the vehicles are obtained, and the vehicles in a large number of image frames are obtained; meanwhile, the same vehicle in the image frame can be tracked by utilizing the type of the vehicle and the number of the vehicle in the second image frame, the possibility of missing detection and false detection of the vehicle when the vehicle in some images is blocked or distorted is reduced, and the accuracy of detecting and tracking the vehicle in the images is improved. On the other hand, whether the confidence information of the vehicle images of the target vehicles with the same number is smaller than the first preset threshold value or not is determined, so that the situation of false detection and omission of the target vehicles caused by distortion of the images and/or target shielding is reduced, and the detection precision of the target vehicles is improved. In still another aspect, the target image is input into a first model trained in advance to obtain license plate key point information, so that when distortion and/or shielding exists in the current target image, the license plate key point information can be determined directly according to the license plate in the target image corresponding to the target vehicle. In still another aspect, license plate key point information in the target vehicles with the same number is determined, and image frames of target images corresponding to the license plate key point information in the target video are determined to obtain target image frames, so that the target vehicles in the corresponding target image frames can be found directly according to the target images of the target vehicles. Finally, directly determining the license plate vertex angle position of the target vehicle in the target image frame through license plate key point information in the target vehicle, and performing desensitization treatment on a target area obtained after the vertex angle positions are connected so as to realize rapid desensitization treatment on the license plate; meanwhile, the accurate identification and desensitization of the license plate to be desensitized with shielding and distortion are realized, and the accuracy of the desensitization of the license plate is improved.

Fig. 5 is a schematic structural diagram of a license plate desensitizing device provided by the invention. As shown in fig. 5, the apparatus includes:

an obtaining module 501, configured to obtain at least one target image corresponding to a target vehicle included in a target video;

the first training module 502 is configured to input a target image into a first model trained in advance to obtain license plate key point information, where the license plate key point information corresponds to the target image one by one, the first model is a model obtained by training with a deep reinforcement learning method based on a training set and with optimized recognition accuracy as an index, and the training set at least includes a distortion training image and/or a shielding training image;

and the desensitizing module 503 is configured to desensitize the license plate of the target vehicle in the target video according to the license plate key point information.

Optionally, the obtaining module 501 is specifically configured to:

determining a target data set according to the target video; inputting M first image frames into a pre-trained second model to obtain M Zhang Dier image frames; and taking the vehicle image of the target vehicle in the M Zhang Dier image frame as a target image according to the vehicle number of the target vehicle.

Alternatively, when the vehicle image of the target vehicle in the M Zhang Dier image frame is taken as the target image according to the vehicle number of the target vehicle, the obtaining module 501 is specifically configured to:

Acquiring confidence information of a vehicle image of the target vehicle according to the vehicle number of the target vehicle; if the confidence degree information of the vehicle images of all the target vehicles is smaller than a first preset threshold value, setting the target images to be empty; and if the confidence information of the vehicle images of the at least one target vehicle is greater than or equal to a first preset threshold value, taking the vehicle images of all the target vehicles as target images.

Optionally, the desensitizing module 503 is specifically configured to:

determining a target image frame; determining a target area in a target image frame according to license plate key point information; and (5) desensitizing the target area.

Optionally, when determining the target area in the target image frame according to the license plate key point information, the desensitizing module 503 is specifically configured to:

determining four vertex angle positions of a license plate of a target vehicle in the target image frame according to the license plate key point information; and sequentially connecting the four vertex angle positions to obtain a target area.

Optionally, the apparatus further comprises a first model training device, and the first model training device is specifically configured to:

acquiring a training set; inputting the current training image into an original model, and determining actual labeling information of the current training image; if the error between the actual labeling information of the current training image and the standard labeling information of the current training image is smaller than or equal to a second preset threshold value, determining the original model as a first model; if the error between the actual labeling information of the current training image and the standard labeling information of the current training image is larger than a second preset threshold value, adjusting parameters of the original model, taking the next training image as the current training image, and returning to execute the steps of inputting the current training image into the original model and determining the actual labeling information of the current training image.

The license plate desensitizing device provided by the embodiment of the invention can execute the license plate desensitizing method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the executing method.

Fig. 6 is a schematic structural diagram of the electronic device 6 provided by the present invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 6, the electronic device 6 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 6 can also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.

The various components in the electronic device 6 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 5 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as license plate desensitization methods.

In some embodiments, the license plate desensitization method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 6 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the license plate desensitization method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the license plate desensitization method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.

The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.

The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims

1. A method of desensitizing a license plate, comprising:

inputting the target image into a pre-trained first model to obtain license plate key point information, wherein the license plate key point information corresponds to the target image one by one, the first model is a model obtained by training by using a deep reinforcement learning method based on a training set and taking optimized recognition accuracy as an index, and the training set at least comprises a distortion training image and/or a shielding training image;

And according to the license plate key point information, desensitizing the license plate of the target vehicle in the target video.

2. The license plate desensitizing method according to claim 1, wherein said obtaining at least one target image corresponding to a target vehicle included in a target video, comprises:

determining a target data set according to the target video, wherein the target data set comprises M continuous first image frames, and M is a positive integer;

inputting M pieces of first image frames into a pre-trained second model to obtain M Zhang Dier image frames, wherein the first image frames correspond to the second image frames one by one, the second image frames comprise vehicle images of at least one vehicle, and one vehicle corresponds to one vehicle number;

and taking the vehicle images of the target vehicle in the M second image frames as the target images according to the vehicle numbers of the target vehicles.

3. The license plate desensitization method according to claim 2, wherein said vehicle image has a confidence information;

the step of using the vehicle image of the target vehicle in the M second image frames as the target image according to the vehicle number of the target vehicle includes:

Acquiring confidence information of a vehicle image of the target vehicle according to the vehicle number of the target vehicle;

if the confidence information of all the vehicle images of the target vehicle is smaller than a first preset threshold value, setting the target image to be empty;

and if the confidence information of the vehicle image of at least one target vehicle is greater than or equal to a first preset threshold value, taking all the vehicle images of the target vehicles as the target images.

4. The license plate desensitizing method according to claim 1, wherein said desensitizing the license plate of the target vehicle in the target video according to the license plate keypoint information comprises:

determining a target image frame, wherein the target image frame is an image frame of the target image corresponding to the license plate key point information in the target video;

determining a target area in the target image frame according to the license plate key point information;

and desensitizing the target area.

5. The license plate desensitization method according to claim 4, wherein said determining a target area in said target image frame according to said license plate keypoint information comprises:

Determining four vertex angle positions of a license plate of the target vehicle in the target image frame according to the license plate key point information;

and sequentially connecting the four vertex angle positions to obtain the target area.

6. The license plate desensitization method according to claim 1, wherein the method of training the first model comprises:

the training set is obtained, wherein the training set comprises distorted training images and/or shielding training images, and one training image is provided with standard marking information;

inputting the current training image into an original model, and determining actual labeling information of the current training image;

if the error between the actual labeling information of the current training image and the standard labeling information of the current training image is smaller than or equal to a second preset threshold value, determining the original model as the first model;

and if the error between the actual labeling information of the current training image and the standard labeling information of the current training image is larger than a second preset threshold value, adjusting parameters of the original model, taking the next training image as the current training image, and returning to execute the step of inputting the current training image into the original model to determine the actual labeling information of the current training image.

7. The license plate desensitizing method according to claim 2, wherein said second image frames further comprise a vehicle type and a vehicle identification, said vehicle identification indicating whether the vehicle is occluded and/or distorted.

8. A license plate desensitizing device, comprising:

the first training module is used for inputting the target image into a pre-trained first model to obtain license plate key point information, wherein the license plate key point information corresponds to the target image one by one, the first model is a model obtained by training by using a deep reinforcement learning method based on a training set and taking optimized recognition accuracy as an index, and the training set at least comprises a distortion training image and/or a shielding training image;

9. An electronic device, the electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the license plate desensitization method according to any one of claims 1-7.

10. A computer readable storage medium storing computer instructions for causing a processor to perform the license plate desensitization method according to any one of claims 1-7.