CN111126286A

CN111126286A - Vehicle dynamic detection method and device, computer equipment and storage medium

Info

Publication number: CN111126286A
Application number: CN201911354855.3A
Authority: CN
Inventors: 周康明; 潘柳华
Original assignee: Shanghai Eye Control Technology Co Ltd
Current assignee: Shanghai Eye Control Technology Co Ltd
Priority date: 2019-12-22
Filing date: 2019-12-22
Publication date: 2020-05-08

Abstract

The application provides a vehicle dynamic detection method, a device, computer equipment and a storage medium, and obtains a license plate starting image and a license plate ending image; comparing the license plate starting image with the license plate ending image, and entering the next step if the comparison is successful; calculating the image displacement of the license plate starting image and the license plate ending image; and if the image displacement is greater than or equal to a preset value, judging that the vehicle moves. According to the method, whether the vehicle has dynamic displacement can be judged quickly, manual judgment is replaced, and the working efficiency is improved.

Description

Vehicle dynamic detection method and device, computer equipment and storage medium

Technical Field

The patent relates to the field of computer vision and artificial intelligence image processing, in particular to a vehicle dynamic detection method, a device, computer equipment and a storage medium.

Background

With the continuous development of social economy and the continuous improvement of the living standard of people, the quantity of motor vehicles in cities is rapidly increased. The workload of annual inspection of motor vehicles is also rapidly increased. The dynamic detection of the vehicle in the traditional annual detection of the vehicle is mainly realized through manual detection, the method is high in labor cost and low in efficiency, and long-time repeated detection operation easily causes the detection personnel to generate bad states such as fatigue and inattention, so that the detection accuracy is influenced. How to accurately and quickly carry out dynamic detection on the vehicle, and simultaneously, the defects that the manual detection cost is high, and detection personnel are easy to fatigue and neglect errors are overcome, so that the technical problem which needs to be solved urgently is solved.

Disclosure of Invention

The application provides a vehicle dynamic detection method, a vehicle dynamic detection device, computer equipment and a storage medium, overcomes the defect of manual detection in the prior art, realizes the detection of vehicle dynamic by using the artificial intelligence of a deep learning model, replaces manual detection, provides working efficiency, and reduces the defect of misjudgment.

In order to achieve the above object, in one aspect, the present application provides a vehicle dynamics detection method, the method including: obtaining a license plate starting image and a license plate ending image; identifying a first license plate in the license plate starting image and a second license plate in the license plate ending image; comparing the first license plate with the second license plate, and entering the next step if the comparison is successful; calculating the image displacement of the license plate starting image and the license plate ending image; and if the image displacement is greater than or equal to a preset value, judging that the vehicle moves.

Further, the recognizing a first license plate in the license plate starting image and a second license plate in the license plate ending image includes: acquiring a first license plate character number of the first license plate and a second license plate character number of the second license plate; the comparing the first license plate and the second license plate comprises: comparing the first license plate character number with the second license plate character number, and recording the character displacement amount; determining the single movement amount and the character movement times of the second license plate character according to the character displacement amount, and determining the comparison times N of the first license plate character and the second license plate character according to the single movement amount and the character movement times of the character; and recording comparison results of the N times of comparison, comparing the comparison results with a set threshold value, and judging whether the comparison is successful.

Further, recording comparison results of N comparisons, comparing the comparison results with a set threshold, and determining whether the comparison is successful, including: sequentially comparing the characters at the same positions of the first license plate character and the second license plate character according to the current position of the character, recording the comparison result every time, executing N times, and recording the comparison result N times; obtaining a comparison result with the minimum error number according to the N comparison results; and taking the comparison result with the minimum error number as a final comparison result, comparing the final comparison result with a set threshold, and if the final result is less than or equal to the set threshold, successfully comparing.

Further, the method comprises: the number of character movements of the second signboard character is equal to a ratio of the character displacement amount to the single movement amount; the comparison times N are equal to the character movement times plus one.

Further, the calculating the image displacement of the license plate starting image and the license plate ending image includes: obtaining the size of a license plate, wherein the size of the license plate comprises the following steps: a first license plate size of the license plate start image and a second license plate size of the license plate end image; wherein the license plate size comprises the height and width of a license plate image; or the license plate size comprises the diagonal length of the license plate image; calculating a license plate size difference value between the first license plate size and the second license plate size; and the license plate size difference value represents the image displacement.

Further, the calculating the image displacement of the license plate starting image and the license plate ending image includes: acquiring a first license plate center position and a second license plate center position; calculating a license plate center distance between the first license plate center position and the second license plate center position; the license plate center distance represents the image displacement.

Further, obtaining a first license plate center position and a second license plate center position includes: acquiring a vehicle starting image and a vehicle ending image before acquiring a vehicle starting image and a vehicle ending image; acquiring a first vehicle area image in the vehicle start image and a second vehicle area image in the vehicle end image; acquiring a first center position of the license plate starting image in the first vehicle area image, and acquiring a second center position of the license plate ending image in the second vehicle area image; acquiring a first position of the first vehicle area image in the vehicle start image, and acquiring a second position of the second vehicle area image in the vehicle end image; calculating a first license plate center position of the first license plate image in the vehicle start image according to the first center position and the first position; calculating a second card center position of the second card image in the vehicle end image according to the second center position and the second position.

Further, a recognition model is adopted to recognize a first license plate in the license plate starting image and a second license plate in the license plate ending image, and the recognition model obtaining process comprises the following steps: establishing a license plate number training data set; training a license plate number recognition model by using a license plate number training data set to obtain a license plate number recognition model; training the marked data by using an LSTM network, firstly adjusting the ratio of the concentrated training data and the test data of the license plate image to be 10: 1; secondly, according to the training effect of the model, adjusting the parameters of model training, wherein the basic learning rate is set to be 0.0001, weight _ decay is set to be 0.005, the learning rate strategy is set to be 'inv', gamma is set to be 0.0001, momentum is set to be 0.9, power is set to be 0.75, and the maximum 250000 iterations are carried out to obtain the trained LSTM model for license plate recognition.

In another aspect, the present application provides a vehicle dynamics detection apparatus, the apparatus comprising: the acquisition module is used for acquiring a license plate starting image and a license plate ending image; the recognition module is used for recognizing a first license plate in the license plate starting image and a second license plate in the license plate ending image; the comparison module is used for comparing the first license plate starting image license plate with the second license plate ending image license plate, and entering the next step if the comparison is successful; the calculation module is used for calculating the image displacement of the license plate starting image and the license plate ending image; and the judging module is used for judging that the vehicle moves if the image displacement is greater than or equal to a preset value.

In yet another aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

In yet another aspect, the present application also provides a computer readable storage medium having a computer program stored thereon, which when executed by a processor, performs the steps of the above method.

Compared with the prior art, the vehicle dynamic detection method, the vehicle dynamic detection device, the computer equipment and the storage medium obtain the license plate starting image and the license plate ending image; identifying a first license plate in the license plate starting image and a second license plate in the license plate ending image; comparing the first license plate with the second license plate, and entering the next step if the comparison is successful; calculating the image displacement of the license plate starting image and the license plate ending image; and if the image displacement is greater than or equal to a preset value, judging that the vehicle moves. The invention mainly applies a vehicle dynamic detection method based on deep learning, which realizes vehicle dynamic intelligent detection and saves manpower. The working efficiency is effectively improved, and the defects that detection personnel are easy to fatigue and neglect errors are effectively avoided.

Drawings

Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:

fig. 1 is a flowchart of a dynamic vehicle detection method according to an embodiment of the present disclosure;

FIG. 2 is a flow chart of another method for dynamic vehicle detection provided by the embodiments of the present application;

FIG. 3 is a flow chart of another method for dynamic vehicle detection provided by the embodiments of the present application;

fig. 4 is a schematic diagram of a license plate character comparison method provided in an embodiment of the present application;

FIG. 5 is a flow chart of another method for dynamic vehicle detection provided by the embodiments of the present application;

FIG. 6 is a schematic diagram illustrating a vehicle target detection model according to an embodiment of the present disclosure;

fig. 7 is a schematic diagram of detecting a license plate target detection model according to an embodiment of the present disclosure;

FIG. 8 is a flow chart of another method for dynamic vehicle detection provided by the embodiments of the present application;

FIG. 9 is a flowchart of another dynamic vehicle detection method provided in the embodiments of the present application;

fig. 10 is a schematic diagram illustrating a license plate center position comparison method according to an embodiment of the present disclosure;

fig. 11 is a block diagram of a vehicle dynamic detection device according to an embodiment of the present application;

FIG. 12 is a block diagram of another vehicle dynamics detection apparatus provided in the embodiments of the present application;

fig. 13 is a block diagram of another vehicle dynamic detection device provided in the embodiment of the present application;

the same or similar reference numbers in the drawings identify the same or similar elements.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

In a typical configuration of the present application, the terminal, the device serving the network, and the trusted party each include one or more processors (e.g., Central Processing Units (CPUs)), input/output interfaces, network interfaces, and memory. The Memory may include volatile Memory in a computer readable medium, Random Access Memory (RAM), and/or nonvolatile Memory such as Read Only Memory (ROM) or flash Memory (flash RAM). Memory is an example of a computer-readable medium.

The vehicle dynamic detection method provided by the application can be applied to the following application environments. The application environment comprises a terminal, a server and an image acquisition device. A terminal may refer to an electronic device with greater data storage and computing capabilities. Wherein, the terminal communicates with the server through the network. The image acquisition device can be a device in the terminal or a separate device. The terminal can be provided with a deep learning model which is trained. Specifically, a license plate starting image and a license plate ending image are acquired through the image acquisition device, and the terminal acquires the license plate starting image and the license plate ending image from the image acquisition device. The terminal compares the license plate starting image with the license plate ending image to obtain a comparison result, and if the comparison is successful, the next step is carried out; the terminal calculates the image displacement of the license plate starting image and the license plate ending image; and the terminal compares the value of the image displacement with a preset value, and if the image displacement is greater than or equal to the preset value, the vehicle is judged to move.

In other embodiments, the vehicle dynamic detection method provided by the application can also be applied to a terminal side and a server side, the image acquisition device acquires a license plate starting image and a license plate ending image, the license plate starting image and the license plate ending image are sent to the server through the terminal in a network connection mode and the like, and the server detects and judges the vehicle dynamic according to the license plate starting image and the license plate ending image. The terminal can be, but is not limited to, various portable mobile devices, and the server can be a live server or a remote server.

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. It should be noted that, in the vehicle dynamic detection method provided by the present application, the execution main body in fig. 1 to fig. 13 is a computer device, wherein the execution main body may also be a vehicle dynamic loading device, and the device may be implemented as part or all of vehicle dynamic detection by software, hardware, or a combination of software and hardware.

In one embodiment, as shown in fig. 1, the present application provides a vehicle dynamics detection method, the method comprising:

and step S100, obtaining a license plate starting image and a license plate ending image.

In one embodiment, the license plate starting image and the license plate ending image can be obtained from terminals such as image acquisition equipment or a server of a vehicle inspection station and the like, the vehicle appearance and the license plate in the vehicle image are clear and clean, and system and model identification is facilitated. The license plate starting image is the license plate in the initial state during dynamic detection, and the license plate ending image is the license plate in the ending state during dynamic detection. For example, whether the vehicle can be started and operated normally is detected, the vehicle start image is a still image before the vehicle is started, and the vehicle end image is an image in the process of running the vehicle or an image at the end of the vehicle starting and running.

S200, identifying a first license plate in the license plate starting image and a second license plate in the license plate ending image.

In one embodiment, the license plate in the license plate starting image is identified, and each character of the first license plate is obtained. And similarly, identifying the license plate in the license plate ending image to obtain each character of the second license plate. The license plate can be identified by adopting an identification model, which is not limited herein.

S300, comparing the first license plate with the second license plate, and entering the next step if the comparison is successful.

In one embodiment, the first license plate and the second license plate are compared. And determining whether the vehicles of the two images are the same vehicle or not by comparing the two license plates, if so, continuing to compare, and if not, needing no comparison. By comparing the license plates, wrong pictures can be effectively eliminated, and the time and power consumption of dynamic detection are saved. In an actual situation, the number plate may be shielded by an individual number, or the individual number cannot be identified due to dirt, and the method provided by the patent can also be used for comparing the number plate with the incomplete number.

S400, calculating the image displacement of the license plate starting image and the license plate ending image.

In one embodiment, if there is movement of the vehicle, there may be a change in the license plate in the two images, such as a change in size, a change in center position, a change in sharpness, etc. Therefore, the positions of the license plate starting image and the license plate ending image in the coordinate system of the shot picture are different. Therefore, the image displacement can be calculated by calculating the size or position difference value of the two images before and after. Based on the method of calculating the image displacement, it is possible to detect whether the vehicle has a displacement in the dynamic detection.

And S500, if the image displacement is larger than or equal to a preset value, judging that the vehicle moves.

In one embodiment, the image displacement calculated in the above step is determined, the image displacement is compared with a preset value, and if the image displacement is greater than or equal to the preset value, it is determined that the vehicle has moved. The preset value can be set according to the actual requirement of the mechanism detected by the vehicle inspection, and can be set in a system or a server before the detection is started.

The vehicle dynamic detection and judgment method provided by the embodiment of the application can be used for detecting the vehicle dynamic more quickly and more simply, the heavy workload of manual examination and verification is reduced, and the working efficiency is properly improved.

In an embodiment, referring to fig. 2, S200 in the vehicle dynamic detection method identifies a first license plate in the license plate start image and a second license plate in the license plate end image, including:

s202, acquiring a first license plate character number of the first license plate and a second license plate character number of the second license plate.

In an embodiment, referring to fig. 2, S300 in the vehicle dynamic detection method compares the first license plate with the second license plate, and if the comparison is successful, the next step includes:

s302, comparing the first license plate character number with the second license plate character number, and recording the character displacement.

In step S202, the number of characters of the first license plate and the number of characters of the second license plate are obtained. In the process of recognizing a license plate, the following situations may exist: for example, the image is not clear enough, the characters of the license plate are not completely recognized, and the characters are missed and incomplete. For example, for the same vehicle, the number of characters recognized by the first license plate is 5, and the number of characters recognized by the second license plate is 4, at this time, the number of characters recognized by the second license plate is not complete, but the vehicles in the two images are the same vehicle, and in order to avoid false recognition, the character difference of the license plates is calculated first and then compared. The difference in the number of characters may be calculated by comparing the first number of license plate characters with the second number of license plate characters, such that the amount of character displacement is equal to the difference in the number of characters.

S304, determining the single movement amount and the character movement times of the second license plate character according to the character displacement amount, and determining the comparison times N of the first license plate character and the second license plate character according to the single movement amount and the character movement times of the character.

In one embodiment, in order to compare license plates, the position of the license plate characters when the image of the license plate is finished can be moved in the comparison process, and the moved characters are compared with the license plate characters when the image of the license plate is started. In this embodiment, the character displacement is set to be equal to the difference of the number of characters, and the character displacement may also be set according to actual conditions.

S306, recording comparison results of the N times of comparison, comparing the comparison results with a set threshold value, and judging whether the comparison is successful.

In an embodiment, referring to fig. 3, in the method for detecting a vehicle dynamics, step S306 records a comparison result of N comparisons, compares the comparison result with a set threshold, and determines whether the comparison is successful, including:

s308, sequentially comparing the characters at the same positions of the first license plate character and the second license plate character according to the current position of the character, recording the comparison result every time, executing N times, and recording the comparison result N times.

In one embodiment, when comparing license plate characters, the characters at each position are compared one by one, the comparison result at each position is recorded, the comparison result at each position is counted, the comparison result at each comparison is obtained, the comparison frequency is N, the operation is executed for N times, and the comparison results for N times are recorded.

S310, obtaining a comparison result with the minimum error number according to the N comparison results.

In one embodiment, for convenience of statistics and subsequent judgment and analysis, a correct number and an error number are set, the comparison of each character of the license plate has correct or wrong conditions, the comparison result of each time comprises the correct number and the error number, and the correct number and the error number of the comparison result of N times are counted. Here, the alignment result with the smallest number of errors among the N alignment results is obtained.

S312, taking the comparison result with the minimum error number as a final comparison result, comparing the final comparison result with a set threshold, and if the final comparison result is less than or equal to the set threshold, the comparison is successful.

In an embodiment, the comparison result with the smallest number of errors is selected as the final comparison result, which is not limited herein, and the comparison result with the largest number of correct errors may also be selected as the final comparison result.

In one embodiment, the method comprises: the number of character movements of the second signboard character is equal to a ratio of the character displacement amount to the single movement amount; the comparison times N are equal to the character movement times plus one.

In one embodiment, the number of character movements of the second license plate may be obtained by comparing the number of character movements of the second license plate character with a ratio of the character displacement amount to the single movement amount, where the number of comparison is equal to the number of character movements plus one. The comparison times comprise: the number of comparison times is one, wherein the comparison times are carried out according to the number of character moving times, and the comparison times are two, the comparison times are carried out before the character does not move, and the number of the comparison times is one. When the license plate characters are compared, the characters on all the positions are compared one by one, the comparison result on each position is recorded, the comparison result on each position is counted, and the comparison result of each comparison is obtained. Referring to fig. 4, it is assumed that one license plate number recognition result has m characters and the other license plate number recognition result has n characters, and the number of comparison times is m-n + 1. The number of character movements is m-n.

In order to better and more clearly understand the method for comparing license plate characters mentioned in the above embodiments, the following description is provided by way of an example, which is merely exemplary and not limited to this example.

Specifically, if the detected license plate numbers are a123BC and 123ADE, the number of characters of the detected license plate numbers is the same, the difference is 0, and the character displacement is 0 because the character displacement is equal to the character number difference. And according to the comparison times which are equal to the character moving times plus one and are 1, only one comparison is needed. As shown in the following table, the characters at each position are compared in sequence, and after comparison, the comparison result is that the correct number is 0 and the error number is 6.

A	1	2	3	B	C
							1	2	3	A	D	E

According to the comparison results of N times, the error number with the minimum number is used as the final comparison result, in the example, the error number is one, so that the minimum error number is 6, the final comparison result is compared with the set threshold, the set threshold is assumed to be 3, and the final comparison result is larger than the set threshold, so that the comparison is unsuccessful, the vehicles in the two pictures are not the same vehicle, the next comparison is not needed, and the comparison time and the calculation amount are saved.

If the detected license plate numbers are A123BC and 123BC, the number of characters of the detected license plate numbers is different, the character difference value is 1, and the character displacement is 1 at this time according to the character displacement equal to the character number difference value. The single movement amount of the second token character may be set according to an actual comparison requirement, in this example, the single movement amount of the second token character is set to be 1, the character movement time of the second token character is 1/1 ═ 1, the comparison time is equal to the character movement time plus one, and the comparison time is 1+1 ═ 2, so that the comparison is performed twice. For two license plates with different numbers in the embodiment, the comparison needs to be performed twice, that is, the comparison is performed once when the displacement is 0, and the comparison is performed once when the displacement is 1.

The method comprises the following specific steps: the displacement is 0, the comparison result is that the correct number is 0 and the error number is 6.

A	1	2	3	B	C
							1	2	3	B	C

Shifts are 1 aligned once: the correct number is 5 and the error number is 1.

A	1	2	3	B	C
								1	2	3	B	C

In the two comparison results, the number of errors in character comparison is 1 as the minimum, so that the final comparison result is 1, the set threshold is 3, the final result is smaller than the set threshold by comparing 1<3, the comparison is successful, and the vehicles in the two pictures are proved to be the same vehicle and enter the next comparison process.

In an embodiment, referring to fig. 5, in the vehicle dynamic detection method, S400 calculates an image displacement of the license plate start image and the license plate end image, and includes:

s410, obtaining the size of a license plate, wherein the size of the license plate comprises the following steps: a first license plate size of the license plate start image and a second license plate size of the license plate end image; wherein the license plate size comprises the height and width of a license plate image; or the license plate size comprises the diagonal length of the license plate image.

And S420, calculating a license plate size difference value between the first license plate size and the second license plate size.

And S430, representing the image displacement by the license plate size difference value.

In one embodiment, the image displacement may be represented by a difference value of the license plate size. If the vehicle has dynamic movement, the size of the license plate in the license plate starting image and the size of the license plate in the license plate ending image are different. And (4) judging whether the vehicle has displacement or not by calculating the size of the license plate.

Specifically, the size of the license plate image can be characterized by the height and width of the license plate image. If the width and height of the license plate change, the vehicle moves. The size of the license plate can be represented by calculating diagonal lines, and if the length of the diagonal line of the pixel point of the outer contour of the license plate image in the license plate starting image is different from the length of the diagonal line of the pixel point of the outer contour of the license plate image in the license plate ending image, the vehicle moves.

Optionally, obtaining a license plate size comprises: acquiring a vehicle start image and a vehicle end image; acquiring a first vehicle area image in the vehicle start image and a second vehicle area image in the vehicle end image; acquiring a first height and a first width of the license plate starting image in the first vehicle area image, and acquiring a second height and a second width of the license plate ending image in the second vehicle area image; comparing the first height with the second height, and comparing the first width with the second width; or, calculating a first diagonal using the first height and the first width, and calculating a second diagonal using the second height and the second width; and comparing the first diagonal line with the second diagonal line.

In one embodiment, a vehicle target detection model based on deep learning can be adopted to detect a vehicle, whether the vehicle exists in an image is judged, if yes, the mark is recorded as 1, and a vehicle image is extracted; if not, recording the mark as 0, storing the related picture, and entering a statistical analysis process. Specifically, whether a vehicle exists in a vehicle start image is detected by using an object detection model, such as an SSD model, and if there is a record of this flag as 1, and a first vehicle area image is acquired, and similarly, whether a vehicle exists in a vehicle end image is detected, and if there is a record of this flag as 1, and a second vehicle area image is acquired.

In one embodiment, the target detection model is obtained as follows:

1. training data preparation: acquiring complete vehicle images of different vehicle types under the conditions of different illumination and different shooting angles;

2. data annotation: marking the vehicle area image in the original image by adopting a rectangular frame, wherein a vehicle target in the rectangular frame area needs to be complete and comprises a license plate and a vehicle mark, and each complete vehicle image corresponds to one rectangular frame to obtain a vehicle area image;

3. and training a target detection depth neural network model by using the vehicle area image to obtain a vehicle target detection model.

Specifically, the method for training the target detection deep neural network model by using the vehicle image to obtain the vehicle target detection model comprises the following steps: and (3) taking the VGG model trained by using the ImageNet data set as a basic model, inputting the marked vehicle image into the SSD frame, and then re-training the vehicle target detection model on the basic model. Firstly, modifying the final output of an SSD frame, and setting the final output to be a vehicle and a background; secondly, adjusting the ratio of training data to test data in the vehicle image set to 10: 1; and finally, adjusting the training parameters of the model according to the training effect of the model, wherein the basic learning rate is set to be 0.001, the weight _ decay is set to be 0.0005, the learning rate strategy is set to be 'multistep', the gamma is set to be 0.1, the momentum is set to be 0.9, and the maximum iteration is 120000 times. The SSD model trained by the above setting is suitable for vehicle detection.

Further, the detection process of the trained vehicle target detection model is as follows: as shown in fig. 6, an original input image is first input into a vehicle target detection model, N one-dimensional arrays [ class, x, y, width, height ] are first obtained, the first element of the array represents the object class, the vehicle is 1, the vehicle is not 0, the four elements after the array represent the rectangular area where the target object is located, x and y represent the abscissa and ordinate of the upper left corner of the rectangle respectively, width represents the width of the rectangle frame, and height represents the height of the rectangle frame. Each array corresponds to a vehicle target, vehicle distance information is constructed by using the area size of the rectangular frame, the array with the largest area of the rectangular frame is used as detection output, and then a vehicle area image is extracted from an original image through the position information of the rectangular frame. The method can effectively remove other non-annual inspection target vehicles in the background.

In one embodiment, specifically, a license plate target detection model based on deep learning is adopted to detect a license plate in a first vehicle area image, whether the license plate exists is judged, if yes, the mark is recorded as 1, and a license plate starting image is extracted; if the mark does not exist, recording the mark as 0, storing the related picture, and entering a statistical analysis process; similarly, detecting a license plate in a second vehicle region image by adopting a target detection model based on deep learning, judging whether the license plate exists, recording the mark as 1 if the license plate exists, and extracting a license plate ending image; if not, recording the mark as 0, storing the related picture, and entering a statistical analysis process.

Further, the license plate target detection model can adopt an SSD model, and the license plate target detection model based on the deep learning network is obtained by the following steps:

1. training data preparation: acquiring original images of different vehicle types, different brands and a specified shooting angle range (including a complete license plate), and processing the images in batch by using a vehicle target detection model to acquire a vehicle area image;

2. data annotation: marking a license plate in a vehicle region image by adopting a rectangular frame, wherein license plate targets in the rectangular frame need to completely comprise license plate frames, each license plate region image corresponds to one rectangular frame, and the frame comprises the license plate targets to obtain a license plate region image;

3. and training a target detection depth neural network model by using the license plate region image to obtain a license plate target detection model.

Specifically, the method for training a target detection depth neural network model by using a license plate region image to obtain a license plate target detection model is as follows: and (3) taking the VGG model trained by using the ImageNet data set as a basic model, inputting the marked license plate image into the SSD frame, and then re-training a license plate target detection model on the basic model. Firstly, modifying the final output of an SSD frame into a license plate and a background; secondly, adjusting the ratio of the concentrated training data and the concentrated testing data of the license plate image to be 10: 1; and finally, adjusting parameters of model training according to the training effect of the model, wherein the basic learning rate is set to be 0.0003, weight _ decay is set to be 0.005, the learning rate strategy is set to be 'multistep', gamma is set to be 0.1, momentum is set to be 0.9, and the maximum number of iterations is 125000. The SSD model which completes training through the setting is suitable for license plate detection.

Further, the detection process of the trained license plate target detection model is as follows: as shown in fig. 7, the detection module firstly inputs the vehicle region image into the license plate target detection model, and firstly obtains N one-dimensional arrays [ class, x, y, width, height ], the first element of the array represents the object type, the license plate is 1, the license plate is not 0, the four elements after the array represent the rectangular region where the target object is located, x and y represent the abscissa and ordinate of the upper left corner of the rectangle respectively, width represents the width of the rectangular frame, and height represents the height of the rectangular frame. Each array corresponds to a license plate target, and a license plate area image is extracted from the vehicle area image by using the position information of the rectangular frame. And after extracting the license plate region image, acquiring the height and width dimensions of the license plate image.

In one embodiment, the vehicle target is detected by using the vehicle target detection model, and then the license plate target is detected by using the license plate target detection model in the vehicle target image.

In an embodiment, referring to fig. 8, in the vehicle dynamic detection method, S400 calculates an image displacement of the license plate start image and the license plate end image, and includes:

s440, acquiring a first license plate center position and a second license plate center position;

s450, calculating the license plate center distance between the first license plate center position and the second license plate center position;

and S460, representing the image displacement by the license plate center distance.

Specifically, the license plate image size may be characterized by a center position of the license plate image. If the central position of the license plate changes, the vehicle moves.

The above-mentioned embodiments can be applied according to actual situations. The calculation method for the image displacement is not limited to the above method.

In an embodiment, referring to fig. 9, the obtaining the first license plate center position and the second license plate center position at S440 in the vehicle dynamic detection method includes:

s401, a vehicle starting image and a vehicle ending image are obtained before a vehicle starting image and a vehicle ending image are obtained.

S402, acquiring a first vehicle area image in the vehicle starting image and acquiring a second vehicle area image in the vehicle ending image.

In one embodiment, the target detection model is obtained as follows:

S403, acquiring a first center position of the license plate starting image in the first vehicle area image, and acquiring a second center position of the license plate ending image in the second vehicle area image.

Alternatively, before the central position is acquired, whether a license plate start image exists in the first vehicle area image and whether a license plate end image exists in the second vehicle area image may be detected.

Further, the detection process of the trained license plate target detection model is as follows: as shown in fig. 7, the detection module firstly inputs the vehicle region image into the license plate target detection model, and firstly obtains N one-dimensional arrays [ class, x, y, width, height ], the first element of the array represents the object type, the license plate is 1, the license plate is not 0, the four elements after the array represent the rectangular region where the target object is located, x and y represent the abscissa and ordinate of the upper left corner of the rectangle respectively, width represents the width of the rectangular frame, and height represents the height of the rectangular frame. Each array corresponds to a license plate target, and a license plate area image is extracted from the vehicle area image by using the position information of the rectangular frame. And after the license plate region image is extracted, acquiring the central position of the license plate image.

S404, acquiring a first position of the first vehicle area image in the vehicle starting image, and acquiring a second position of the second vehicle area image in the vehicle ending image.

S405, calculating a first license plate center position of the first license plate image in the vehicle starting image according to the first center position and the first position.

S406, calculating a second license plate center position of the second license plate image in the vehicle end image according to the second center position and the second position.

In one embodiment, in the vehicle region image, a license plate target detection model is used for obtaining the position of a rectangular frame of a license plate mark, and the height and the width of the license plate and the position of the center of the license plate in the vehicle region image are obtained through the pixel positions of four vertex angles of the rectangle. In the vehicle image, a rectangular frame of the vehicle mark is obtained by using a vehicle target detection model, and the position of the license plate center in the vehicle image is calculated according to the pixel positions of four vertex angles of the rectangular frame. And calculating the distance between the centers of the two license plates and the size difference value of the two license plates according to the position information of the license plate centers in the vehicle image.

In an application example, as shown in fig. 10, in a vehicle region image, a license plate target detection model is used to obtain a rectangular frame position of a license plate mark, and the heights and widths of a license plate 1 and a license plate 2 and positions of the center of the license plate 1 and the center of the license plate 2 in the vehicle region image 1 and the vehicle region image 2 are obtained through pixel positions of four vertex angles of the rectangular frame; obtaining a rectangular frame of a vehicle mark by using a vehicle target detection model, and calculating the positions of the center of a license plate 1 and the center of a license plate 2 in an original image 1 and an original image 2 respectively according to the pixel positions of four vertex angles of the rectangular frame; because the original images are consistent in size, the distance between the centers of the two license plates can be calculated according to the position information of the centers of the two license plates in the original images; and simultaneously, calculating the diagonal length of the two license plates and the difference value of the diagonal length according to the height and the width of the two license plates, and taking the difference value as the difference value of the sizes of the two license plates. And taking 50% of the height value of the license plate with smaller height as a threshold, and when the calculated license plate center distance or the calculated license plate size difference value is larger than or equal to the automatically generated threshold, indicating that the vehicle moves, otherwise, indicating that the vehicle does not move.

When the method is applied, the difference value of the image sizes can be independently selected to judge whether the vehicle moves, the change of the central position of the image can be independently selected to judge whether the vehicle moves, and a scheme combining the two schemes can be adopted to judge whether the vehicle moves.

In one embodiment, in the vehicle dynamic detection method, a recognition model is used to recognize a first license plate in the license plate start image and a second license plate in the license plate end image, and the license plate recognition model obtaining process includes:

1. and establishing a license plate number training data set.

Specifically, vehicle region images of different vehicle types, different brands and a specified shooting angle range (including a complete license plate) are obtained, and the vehicle region images are processed in batch by using a license plate target detection model to obtain license plate region images. In one embodiment, the license plate region image may be standardized by first generating a pure white background template image with a specified width w and a specified height h, and then enlarging the license plate region image in equal proportion and pasting the enlarged license plate region image to the template image.

2. And (6) data annotation.

Specifically, the number plate number is encoded by using 69 characters including 10 Arabic numerals, 24 English letters (except i and o), 34 provinces and cities in China and a null label, the number plate data is marked by using a 7-bit character label, the label 69 represents a blank background, the length of the label is counted, and if the number is less than 7, the label 69 is used for supplementing the number to 7.

3. And training the license plate number recognition model by using the marked license plate number training data set to obtain the license plate number recognition model.

Training the marked data by using an LSTM network, firstly adjusting the ratio of the concentrated training data and the test data of the license plate image to be 10: 1; secondly, according to the training effect of the model, adjusting the parameters of model training, wherein the basic learning rate is set to be 0.0001, weight _ decay is set to be 0.005, the learning rate strategy is set to be 'inv', gamma is set to be 0.0001, momentum is set to be 0.9, power is set to be 0.75, and the maximum 250000 iterations are carried out to obtain the trained LSTM model for license plate recognition.

In one embodiment, the image of the license plate area can be processed to realize data expansion, corresponding class labels are recorded, and the expanded data is used for training the recognition model.

Further, in an embodiment, as shown in fig. 11, the detection process of the trained license plate recognition model is as follows: inputting the obtained license plate region image into a recognition model, recognizing character strings in the license plate number by the recognition model to obtain a group of character strings consisting of spaces and corresponding prediction categories, processing the recognized result character strings, deleting space characters, reserving only one same label character among the space characters, and obtaining the processed character as a recognition result.

The invention provides a vehicle dynamic detection method, which comprises the steps of firstly obtaining license plate images at the beginning and the end; identifying a license plate in the license plate image; comparing the license plates at the beginning and the end, and entering the next step if the comparison is successful; calculating the displacement of the license plate image at the beginning and the end; and if the displacement is greater than or equal to the preset value, judging that the vehicle moves. The method can replace the existing manual auditing mode, save manpower, accelerate auditing speed and ensure the openness and justice of auditing work.

It should be understood that, although the steps in the above-described flowcharts are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in the above-described flowcharts may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or the stages is not necessarily sequential, but may be performed alternately or alternatingly with other steps or at least a portion of the sub-steps or stages of other steps.

In another embodiment, as shown in fig. 12, the present invention further provides a vehicle dynamics detection apparatus 800, comprising:

the obtaining module 810 obtains a license plate starting image and a license plate ending image;

the recognition module 820 is used for recognizing a first license plate in the license plate starting image and a second license plate in the license plate ending image;

the comparison module 830 is configured to compare the first license plate start image license plate with the second license plate end image license plate, and enter the next step if the comparison is successful;

the calculating module 840 is used for calculating the image displacement of the license plate starting image and the license plate ending image;

and the judging module 850 judges that the vehicle moves if the image displacement is greater than or equal to a preset value.

In an embodiment, referring to fig. 13, the comparing module 830 in the vehicle dynamics detecting apparatus 800 includes the following modules:

a character acquisition unit 831 that acquires a first license plate character number of the first license plate and a second license plate character number of the second license plate; and acquiring a first license plate character number in the license plate starting image and a second license plate character number in the license plate ending image.

And a character comparing unit 832 for comparing the first license plate character number and the second license plate character number and recording the character number difference.

A displacement statistics unit 833 for determining the single movement amount and the number of character movements of the second signboard character according to the character displacement amount, and determining the number of times N of comparison between the first signboard character and the second signboard character according to the single movement amount and the number of character movements, wherein the character displacement amount is equal to the character number difference value;

the result comparing unit 834 records comparison results of the N comparisons, compares the comparison results with a set threshold, and determines whether the comparison is successful.

In an embodiment, the result comparing unit 834 sequentially compares the characters at the same positions of the first license plate character and the second license plate character according to the current position of the character, records the comparison result each time, executes N times, and records the comparison result N times; the comparison result comparison unit 834 acquires a comparison result with the smallest error number according to the N comparison results; the result comparing unit 834 takes the comparison result with the minimum number of errors as a final comparison result, compares the final comparison result with a set threshold, and if the final comparison result is less than or equal to the set threshold, the comparison is successful.

In one embodiment, the displacement statistic unit 833 counts the number of character movements of a second car character, the number of character movements of the second car character being equal to the ratio of the character displacement amount to the single movement amount; the comparison times N are equal to the character movement times plus one.

In one embodiment, the calculation module 840 of the vehicle dynamics detection apparatus 800 includes:

the size acquisition unit acquires the size of a license plate, wherein the size of the license plate comprises: a first license plate size of the license plate start image and a second license plate size of the license plate end image; wherein the license plate size comprises the height and width of a license plate image; or the license plate size comprises the diagonal length of the license plate image;

a size calculation unit that calculates a license plate size difference value between the first license plate size and the second license plate size; and the license plate size difference value represents the image displacement.

a position acquisition unit that acquires a first license plate center position and a second license plate center position;

a position calculation unit that calculates a license plate center distance between the first license plate center position and the second license plate center position; the license plate center distance represents the image displacement.

a vehicle image acquisition unit that acquires a vehicle start image and a vehicle end image;

a vehicle area acquisition unit that acquires a first vehicle area image in the vehicle start image and a second vehicle area image in the vehicle end image;

a position acquiring unit acquires a first center position of the license plate starting image in the first vehicle area image and a second center position of the license plate ending image in the second vehicle area image; the position acquisition unit acquires a first position of the first vehicle area image in the vehicle start image and acquires a second position of the second vehicle area image in the vehicle end image; the position calculating unit calculates a first license plate center position of the first license plate image in the vehicle start image according to the first center position and the first position; a position calculation unit calculates a second card center position of the second card image in the vehicle end image based on the second center position and the second position.

In one embodiment, the identification module 820 in the vehicle dynamics detection apparatus 800 includes:

the generating unit is used for generating a recognition model, and the recognition model acquiring process comprises the following steps:

establishing a license plate number training data set; labeling data of a license plate number training data set; training a license plate number recognition model by using a license plate number training data set to obtain a license plate number recognition model; training the marked data by using an LSTM network, firstly adjusting the ratio of the concentrated training data and the test data of the license plate image to be 10: 1; secondly, according to the training effect of the model, adjusting the parameters of model training, wherein the basic learning rate is set to be 0.0001, weight _ decay is set to be 0.005, the learning rate strategy is set to be 'inv', gamma is set to be 0.0001, momentum is set to be 0.9, power is set to be 0.75, and the maximum 250000 iterations are carried out to obtain the trained LSTM model for license plate recognition.

For specific limitations of the vehicle dynamics detection apparatus, reference may be made to the above limitations of the vehicle dynamics method, which are not described herein again. The various modules in the vehicle dynamic detection device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

The invention also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method when executing the computer program. The computer device may be an end product, the processor of which is used to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a vehicle dynamics detection method.

The invention also provides a computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method.

Those skilled in the art will appreciate that all or part of the processes of the methods of the above embodiments may be implemented by a computer program that instructs associated hardware to perform the processes of the embodiments of the methods described above. Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, Phase-Change RAM (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), flash Memory or other Memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, magnetic cassette tape, tape-Disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A vehicle dynamics detection method, characterized in that the method comprises:

obtaining a license plate starting image and a license plate ending image;

identifying a first license plate in the license plate starting image and a second license plate in the license plate ending image;

comparing the first license plate with the second license plate, and entering the next step if the comparison is successful;

calculating the image displacement of the license plate starting image and the license plate ending image;

and if the image displacement is greater than or equal to a preset value, judging that the vehicle moves.

2. The vehicle dynamics detection method according to claim 1,

the identifying a first license plate in the license plate starting image and a second license plate in the license plate ending image comprises:

acquiring a first license plate character number of the first license plate and a second license plate character number of the second license plate;

the comparing the first license plate and the second license plate comprises:

comparing the first license plate character number with the second license plate character number, and recording the character displacement amount;

determining the single movement amount and the character movement times of the second license plate character according to the character displacement amount, and determining the comparison times N of the first license plate character and the second license plate character according to the single movement amount and the character movement times of the character;

and recording comparison results of the N times of comparison, comparing the comparison results with a set threshold value, and judging whether the comparison is successful.

3. The vehicle dynamic detection method according to claim 2, wherein the step of recording the comparison results of N comparisons, comparing the comparison results with a set threshold, and determining whether the comparison is successful comprises:

sequentially comparing the characters at the same positions of the first license plate character and the second license plate character according to the current position of the character, recording the comparison result every time, executing N times, and recording the comparison result N times;

obtaining a comparison result with the minimum error number according to the N comparison results;

and taking the comparison result with the minimum error number as a final comparison result, comparing the final comparison result with a set threshold, and if the final result is less than or equal to the set threshold, successfully comparing.

4. The vehicle dynamics detection method according to claim 2, comprising:

the number of character movements of the second signboard character is equal to a ratio of the character displacement amount to the single movement amount;

the comparison times N are equal to the character movement times plus one.

5. The vehicle dynamic detection method of claim 1, wherein the calculating the image displacement of the license plate start image and the license plate end image comprises:

obtaining the size of a license plate, wherein the size of the license plate comprises the following steps: a first license plate size of the license plate start image and a second license plate size of the license plate end image; wherein the license plate size comprises the height and width of a license plate image; or the license plate size comprises the diagonal length of the license plate image;

calculating a license plate size difference value between the first license plate size and the second license plate size;

and the license plate size difference value represents the image displacement.

6. The vehicle dynamic detection method of claim 1, wherein the calculating the image displacement of the license plate start image and the license plate end image comprises:

acquiring a first license plate center position and a second license plate center position;

calculating a license plate center distance between the first license plate center position and the second license plate center position;

the license plate center distance represents the image displacement.

7. The vehicle dynamics detection method of claim 6, wherein obtaining the first and second license plate center positions comprises:

acquiring a vehicle start image and a vehicle end image;

acquiring a first vehicle area image in the vehicle start image and a second vehicle area image in the vehicle end image;

acquiring a first center position of the license plate starting image in the first vehicle area image, and acquiring a second center position of the license plate ending image in the second vehicle area image;

acquiring a first position of the first vehicle area image in the vehicle start image, and acquiring a second position of the second vehicle area image in the vehicle end image;

calculating a first license plate center position of the first license plate image in the vehicle start image according to the first center position and the first position;

calculating a second card center position of the second card image in the vehicle end image according to the second center position and the second position.

8. The vehicle dynamic detection method of claim 1, wherein a recognition model is used to recognize a first license plate in the license plate start image and a second license plate in the license plate end image, and the recognition model obtaining process comprises:

establishing a license plate number training data set;

labeling data of a license plate number training data set;

training a license plate number recognition model by using a license plate number training data set to obtain a license plate number recognition model;

9. A vehicle dynamics detection apparatus, characterized in that the apparatus comprises:

the acquisition module is used for acquiring a license plate starting image and a license plate ending image;

the recognition module is used for recognizing a first license plate in the license plate starting image and a second license plate in the license plate ending image;

the comparison module is used for comparing the first license plate starting image license plate with the second license plate ending image license plate, and entering the next step if the comparison is successful;

the calculation module is used for calculating the image displacement of the license plate starting image and the license plate ending image;

and the judging module is used for judging that the vehicle moves if the image displacement is greater than or equal to a preset value.

10. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 8 when executing the computer program.

11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 8.