CN109493608B - Method and system for recognizing illegal operating vehicle and computer readable storage medium - Google Patents

Abstract

The invention discloses an illegal operating vehicle identification method, which comprises the following steps: s1: collecting vehicle data to establish a vehicle number plate database, a vehicle running track database and a card port longitude and latitude database; s2: cleaning the collected vehicle data; s3: deep learning is carried out based on the cleaned vehicle data, and an intelligent identification model is established; and S4: and identifying all vehicles by using the intelligent identification model. The method and the system for identifying the illegal operating vehicle and the computer readable storage medium can realize the efficient and accurate identification of the illegal operating vehicle.

Description

Method and system for recognizing illegal operating vehicle and computer readable storage medium

Technical Field

The present invention relates to the field of intelligent transportation technologies, and in particular, to a method and a system for identifying an illegal operating vehicle, and a computer-readable storage medium.

Background

The illegal operation vehicle is commonly called as a black vehicle, and refers to a vehicle which does not transact any related procedures in a transportation management department and does not receive operation license so as to implement illegal operation by paid service. In recent years, along with the development of economy, illegal operating vehicles tend to rise year by year, are not uniformly managed and added at will, seriously disturb the normal transportation market order and damage the legal rights and interests of passengers. Most illegal operation vehicles lack daily maintenance, mechanical accidents easily occur, forced scrapping limit is loose, safety factor is extremely low, and traffic safety cannot be guaranteed; in addition, many drivers of illegal operating vehicles do not have the basic requirements of drivers of operating passenger cars, the safety awareness is relatively thin, manual accident accidents are easy to happen, and high-quality service cannot be provided without passenger transport service training; even some drivers are low in quality, behaviors of strongly contracting passengers, knocking and strolling, and carrying out financial and pest activities frequently occur, the traveling safety of the passengers is seriously influenced, and unstable factors are brought to the society; particularly, due to the continuous development of the internet, various taxi taking apps are generated, a riding opportunity is provided for illegal operation, and the illegal operation is more concealed.

Because of the characteristics of huge number of illegal operating vehicles, strong concealment, difficult investigation and evidence collection and the like, no effective method or system for automatically identifying the vehicles exists at present, so that how to accurately identify the illegal operating vehicles becomes a problem which needs to be solved urgently by the administrative department of operation management, and becomes the key point for managing the illegal operating vehicles.

Disclosure of Invention

The invention provides an illegal operating vehicle identification method and system and a computer readable storage medium, which aim to solve the technical problem that the existing illegal operating vehicle cannot be accurately identified.

According to an aspect of the present invention, there is provided an illegal working vehicle identification method, including the steps of:

s1: collecting vehicle data to establish a vehicle number plate database, a vehicle running track database and a card port longitude and latitude database;

s2: cleaning the collected vehicle data;

s3: deep learning is carried out based on the cleaned vehicle data, and an intelligent identification model is established; and

s4: and identifying all vehicles by using the intelligent identification model.

Further, step S2 further includes the steps of:

s21: cleaning time disorder and repeated data in a vehicle running track database;

s22: selecting illegal operating vehicle types needing to be identified, extracting vehicle running track data of vehicles of corresponding types according to the identified illegal operating vehicle types, and acquiring a legal operating vehicle number plate set and a non-operating vehicle number plate set according to vehicle operating categories; and

s23: and carrying out classification exploratory analysis on the extracted vehicle running track data to remove serious distortion data.

Further, the deep learning in step S3 is performed by one or more of one-dimensional time trajectory deep learning, two-dimensional space-time trajectory deep learning, two-dimensional thermodynamic diagram deep learning, three-dimensional space-time trajectory graph deep learning, and three-dimensional thermodynamic diagram deep learning.

Further, the deep learning in step S3 is performed by one-dimensional time trajectory deep learning, and step S3 specifically includes the following steps:

s31 a: selecting samples, including positive samples and negative samples;

s32 a: constructing a bayonet pair, sequencing vehicle running track data according to vehicle license plates and time sequence to obtain a bayonet track sequence of each license plate, numbering the sequence from i to 1, and forming the bayonet pair by adjacent passing bayonets: k is a radical of_ik_i+1Time △ t elapsed between adjacent gates_i+1＝t_i+1-t_iAnd constructing vehicle gate pair passing time data in the following format:

license plate number

Bayonet pair kiki+1

Passage time △ ti+1

；

S33 a: setting the passing time granularity of the bayonet pairs, eliminating overlength delta t for rest and rest at night, applying normal distribution, solving an interval t of the delta t in u +/-3 sigma, wherein the interval t is also a value interval of the passing time granularity of the bayonet pairs, setting the passing time granularity of the bayonet pairs as lambda, lambda is a time segment, and lambda belongs to t;

s34 a: performing one-dimensional time running track matrix modeling;

s35 a: normalization processing; and

s36 a: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model.

Further, step S34a is specifically:

processing the time running track of each number plate vehicle into a one-dimensional matrix R required by a convolution neural network_mTaking the length of one-dimensional time running track as T24 60/lambda, T represents the selected sampleIn the date range, λ represents the pass time granularity of the gate pair, j × λ represents the total time of j time segments, and if a certain vehicle appears in a certain gate in the jth segment, the date range includes:

R_m[j]not, otherwise, R_m[j]＝0，

First, to R_mCarry out initialization, R_m[j]＝0，j∈(0，length)

Then, the start time t selected from the sample is calculated from the time when each number plate vehicle passes the f-th gate₀Time △ t used for f-th bayonet_ft₀＝t_f-t₀Then:

R_m[△t_ft₀/λ]＝1

according to the above, a time running track matrix R of each number plate is obtained_m(ii) a A positive sample matrix P is also obtained_mAnd negative sample matrix N_m。

Further, the deep learning in step S3 is performed by two-dimensional time trajectory deep learning, and step S3 specifically includes the following steps:

s31 b: selecting samples, including positive samples and negative samples;

s32 b: constructing a bayonet pair, sequencing vehicle running track data according to vehicle license plates and time sequence to obtain a bayonet track sequence of each license plate, numbering the sequence from i to 1, and forming the bayonet pair by adjacent passing bayonets: k is a radical of_ik_i+1Time △ t elapsed between adjacent gates_i+1＝t_i+1-t_iAnd constructing vehicle gate pair passing time data in the following format:

license plate number

Bayonet pair kiki+1

Passage time △ ti+1

；

S33 b: setting bayonet pair time granularity, eliminating overlength delta t for rest and rest at night, applying normal distribution to obtain an interval t of the delta t in u +/-3 sigma, wherein the interval t is also a value interval of the bayonet pair passing time granularity, setting the bayonet pair passing time granularity as lambda, and setting the lambda as a time segment, wherein the lambda belongs to the t;

s34 b: performing two-dimensional time running track matrix modeling;

s35 b: normalization processing; and

s36 b: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model.

Further, step S34b is embodied as

Processing the time running track of each number plate vehicle into a two-dimensional matrix R required by a convolution neural network_mWhere the Column indicates the number of days, Column width, Column _ width, T, date range, row indicates the transit time, row _ height, 24, 60/λ, λ indicates the traffic time granularity for the bayonet pair, and then for R_mAnd (3) initializing:

R_m[m][n-1]＝0，m∈(0，24*60/λ)，n∈(1，T)，max(m)*λ＝24*60

calculating the nth day of each vehicle in (1, T) according to the date of each vehicle passing the f-th gate, and selecting the starting time T from the sample₀Time △ t used for f-th bayonet_ft₀＝t_f-t₀Then m is △ t_ft₀λ - (n-1) × 24 × 60/λ, then:

R_m[△t_ft₀/λ-(n-1)*24*60/λ][n-1]＝1

according to the above, a time running track matrix R of each number plate is obtained_m(ii) a A positive sample matrix P is also obtained_mAnd negative sample matrix N_m。

Further, the deep learning in step S3 is performed by two-dimensional spatiotemporal trajectory deep learning, and step S3 specifically includes the following steps:

s31 c: selecting samples, including positive samples and negative samples;

s32 c: constructing a bayonet pair, sequencing vehicle running track data according to vehicle license plates and time sequence to obtain a bayonet track sequence of each license plate, numbering the sequence from i to 1, and forming the bayonet pair by adjacent passing bayonets: k is a radical of_ik_i+1Time △ t elapsed between adjacent gates_i+1＝t_i+1-t_iAnd constructing vehicle gate pair passing time data in the following format:

license plate number

Bayonet pair kiki+1

Passage time △ ti+1

；

S33 c: setting bayonet pair time granularity, eliminating overlength delta t for rest and rest at night, applying normal distribution to obtain an interval t of the delta t in u +/-3 sigma, wherein the interval t is also a value interval of the bayonet pair passing time granularity, setting the bayonet pair passing time granularity as lambda, and setting the lambda as a time segment, wherein the lambda belongs to the t;

s34 c: modeling a two-dimensional bayonet space matrix;

s35 c: performing two-dimensional space-time movement track matrix modeling;

s36 c: normalization processing; and

s37 c: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model.

Further, step S34c is embodied as

Dividing the operation area into L-size side lengths according to the longitude and latitude data of the bayonetThe square lattices in size are numbered according to the number of the lattices in the longitude and latitude directions and the bayonet number to obtain a bayonet SPACE matrix SPACE_m，SPACE_m[u][v]The grids are numbered as grids, and the grids belonging to the bayonets are the space positions represented by the bayonets.

Further, step S35c is embodied as

Processing the time running track of each number plate vehicle into a two-dimensional matrix R required by a convolution neural network_mWhere the Column indicates the number of days, Column width, Column _ width, T, date range, row indicates the transit time, row _ height, 24, 60/λ, λ indicates the traffic time granularity for the bayonet pair, and then for R_mAnd (3) initializing:

R_m[m][n-1]＝0，m∈(0，24*60/λ)，n∈(1，T)，max(m)*λ＝24*60

calculating the nth day of each vehicle in (1, T) according to the date of each vehicle passing the f-th gate, and selecting the starting time T from the sample₀Time △ t used for f-th bayonet_ft₀＝t_f-t₀Then m is △ t_ft₀The/lambda- (n-1) 24-60/lambda searches the SPACE matrix SPACE of the bayonet according to the serial number of the f-th bayonet_mObtaining the corresponding SPACE number SPACE_m[u][v]Then:

R_m[△t_ft₀/λ-(n-1)*24*60/λ][n-1]＝SPACE_m[u][v]

accordingly, a space-time running track matrix R of each number plate is obtained_m(ii) a A positive sample matrix P is also obtained_mAnd negative sample matrix N_m。

Further, the deep learning in step S3 is performed by two-dimensional thermodynamic diagram deep learning, and step S3 specifically includes the following steps:

s31 d: selecting samples, including positive samples and negative samples;

s32 d: forming a foreground thermodynamic diagram based on the vehicle running track data, sequencing the vehicle running track data according to the vehicle number plate and the time sequence, marking the times of passing the vehicle through the gates on a map in the form of the thermodynamic diagram until the last gate is reached, and then, generating a background base map to form the foreground thermodynamic diagram;

s33 d: modeling a two-dimensional bayonet space matrix;

s34 d: normalization processing; and

s35 d: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model.

Further, step S33d is embodied as

Dividing an operation area into square lattices with the side length of L size according to the longitude and latitude data of the bayonet, numbering each lattice according to the number of the lattices in the longitude and latitude direction and the number of the bayonet, and obtaining a spatial matrix SPACE of the bayonet_mThe grid to which the bayonet belongs is the SPACE position represented by the bayonet, and finally, the thermodynamic value of the corresponding square in the thermodynamic diagram is filled to obtain the SPACE matrix SPACE of the bayonet_m[u][v]And according to the above steps, obtaining thermodynamic diagram running track matrix R-SPACE of each number plate_m(ii) a Simultaneously, a positive sample matrix P-SPACE is obtained_mAnd negative sample matrix N-SPACE_m。

Further, the deep learning in step S3 is performed by a three-dimensional spatiotemporal trajectory graph, and step S3 specifically includes the following steps:

s31 e: selecting samples, including positive samples and negative samples;

s32 e: constructing a bayonet pair, sequencing vehicle running track data according to vehicle license plates and time sequence to obtain a bayonet track sequence of each license plate, numbering the sequence from i to 1, and forming the bayonet pair by adjacent passing bayonets: k is a radical of_ik_i+1Time △ t elapsed between adjacent gates_i+1＝t_i+1-t_iAnd constructing vehicle gate pair passing time data in the following format:

license plate number

Bayonet pair kiki+1

Passage time △ ti+1

；

S33 e: setting bayonet pair time granularity, eliminating overlength delta t for rest and rest at night, applying normal distribution to obtain an interval t of the delta t in u +/-3 sigma, wherein the interval t is also a value interval of the bayonet pair passing time granularity, setting the bayonet pair passing time granularity as lambda, and setting the lambda as a time segment, wherein the lambda belongs to the t;

s34 e: carrying out three-dimensional time running track matrix modeling;

s35 e: normalization processing; and

s36 e: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model.

Further, step S34e is embodied as

Dividing an operation area into square lattices with the side length of L size according to the longitude and latitude data of the bayonet, numbering each lattice according to the number of the lattices in the longitude and latitude direction and the number of the bayonet, and obtaining a spatial matrix SPACE of the bayonet_m，SPACE_m[u][v]The grid number is the grid number, the grid to which the bayonet belongs is the space position represented by the bayonet, the number of channels is the maximum value of the one-dimensional time running track sequence, the channel is T24 60/lambda, lambda represents the granularity of the bayonet to the passing time, and T represents the days of the selected sample; initializing the three-dimensional matrix:

Channel-SPACE_m[u][v][c]＝0，c∈(0，channel)

calculating the starting time t selected from the sample according to the time of each number plate vehicle passing through the f-th gate₀Time △ t used for f-th bayonet_ft₀＝t_f-t₀Then c is △ t_ft₀λ, directly determining SPACE according to the number of card number_m[u][v]Then, there are:

Channel-SPACE_m[u][v][△t_ft₀/λ]＝SPACE_m[u][v]

if the two-dimensional space-time trajectory curves pass through the adjacent bayonets and are directly connected, a spirally rising three-dimensional space-time trajectory curve is formed;

accordingly, a three-dimensional space-time trajectory curve diagram R of each number plate is obtained_m(ii) a At the same time, a positive sample P is obtained_mAnd negative sample N_m。

Further, the deep learning in step S3 is performed by three-dimensional thermodynamic diagram deep learning, and step S3 specifically includes the following steps:

s31 f: selecting samples, including positive samples and negative samples;

s32 f: constructing a bayonet pair, sequencing vehicle running track data according to vehicle license plates and time sequence to obtain a bayonet track sequence of each license plate, numbering the sequence from i to 1, and forming the bayonet pair by adjacent passing bayonets: k is a radical of_ik_i+1Time △ t elapsed between adjacent gates_i+1＝t_i+1-t_iAnd constructing vehicle gate pair passing time data in the following format:

license plate number

Bayonet pair kiki+1

Passage time △ ti+1

；

S33 f: setting bayonet pair time granularity, eliminating overlength delta t for rest and rest at night, applying normal distribution to obtain an interval t of the delta t in u +/-3 sigma, wherein the interval t is also a value interval of the bayonet pair passing time granularity, setting the bayonet pair passing time granularity as lambda, and setting the lambda as a time segment, wherein the lambda belongs to the t;

s34 f: carrying out three-dimensional thermodynamic diagram modeling;

s35 f: normalization processing; and

s36 f: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model.

Further, step S34f is embodied as

Dividing an operation area into square lattices with the side length of L size according to the longitude and latitude data of the bayonet, numbering each lattice according to the number of the lattices in the longitude and latitude direction and the number of the bayonet, and obtaining a spatial matrix SPACE of the bayonet_mThe grid to which the bayonet belongs is the space position represented by the bayonet, the number of channels is the maximum value of the one-dimensional time running track sequence, the channel is T24 60/lambda, lambda represents the granularity of the bayonet to the passing time, and T represents the number of days of the selected sample; initializing the three-dimensional matrix:

Channel-SPACE_m[u][v][c]＝0，c∈(0，channel)

sequencing vehicle running track data according to vehicle license plate number and time sequence, drawing the times of vehicles passing through a gate in a thermodynamic diagram mode in a layered mode by taking lambda as a unit or drawing the vehicles in a layered accumulation mode, thus obtaining a three-dimensional layered gradual change thermodynamic diagram or a hierarchical change thermodynamic diagram by taking lambda as a unit until the last gate is reached, and then generating a bottom diagram to form a foreground thermodynamic diagram;

calculating the starting time t selected from the sample according to the time of each number plate vehicle passing through the f-th gate₀To the first_fTime △ t for one bayonet_ft₀＝t_f-t₀Then c is △ t_ft₀/λ；

Filling the thermal values of the corresponding squares in the layered thermodynamic diagram into a three-dimensional thermodynamic diagram matrix Channel-SPACE_m[u][v][c]；

Accordingly, the three-dimensional thermodynamic diagram R-SPACE of each number plate is obtained_m(ii) a At the same time, a positive sample P-SPACE is obtained_mAnd negative sample N-SPACE_m。

Further, the vehicle running track data of a specific period of time may be extracted in step S22 to specifically identify the vehicle illegally operated during the selected period of time.

Further, step S1 includes acquiring video camera data of a law enforcement point to build a law enforcement video database;

the illegal operating vehicle identification method further comprises the following steps:

s5: and obtaining the number plate of the vehicle in illegal operation based on the identification result, and correspondingly calling the video evidence or the picture evidence in illegal operation from the law enforcement video database.

The invention also provides an illegal operating vehicle identification system, which comprises

The data acquisition module is used for acquiring vehicle data to establish a vehicle number plate database, a vehicle running track database and a card port longitude and latitude database;

the data cleaning module is used for cleaning the acquired vehicle data;

and the deep learning module is used for carrying out deep learning based on the cleaned vehicle running track database so as to establish an intelligent recognition model and recognizing all vehicles by utilizing the intelligent recognition model.

The present invention also provides a computer-readable storage medium for storing a computer program for performing illegal operation vehicle recognition, the computer program, when executed on a computer, performing the steps of:

s1: collecting vehicle data to establish a vehicle number plate database, a vehicle running track database and a card port longitude and latitude database;

s2: cleaning the collected vehicle data;

s3: deep learning is carried out based on the cleaned vehicle data, and an intelligent identification model is established; and

s4: and identifying all vehicles by using the intelligent identification model.

The invention has the following beneficial effects:

the invention relates to a method for identifying illegal operating vehicles, which establishes a vehicle number plate database, a vehicle running track database and a card port longitude and latitude database by collecting vehicle data. The collected vehicle data are cleaned, and the vehicle running track data corresponding to the vehicle types can be extracted according to the selected illegal operating vehicle types in the cleaning process, so that the illegal operating vehicles of different vehicle types can be efficiently and accurately and quickly identified; and according to the characteristic that the periodicity of the illegal operating vehicle operation time period is strong, only a specific time period is correspondingly extracted when the vehicle operation track data is extracted so as to identify the illegal operating vehicle in the selected time period. And classification training is carried out by adopting a deep learning convolution neural network, so that efficient and accurate illegal operating vehicle identification is realized.

In addition, according to different dimensions of input data, multiple deep learning modes including one-dimensional time track deep learning, two-dimensional space-time track deep learning, two-dimensional thermodynamic diagram deep learning, three-dimensional space-time track graph deep learning and three-dimensional thermodynamic diagram deep learning are available, the deep learning modes can be started from different dimensions at the same time, identification results are mutually supplemented and proved, and the identification precision is high.

In addition to the objects, features and advantages described above, other objects, features and advantages of the present invention are also provided. The present invention will be described in further detail below with reference to the drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the invention and, together with the description, serve to explain the invention and not to limit the invention. In the drawings:

fig. 1 is a flowchart illustrating an illegal operating vehicle recognition method according to a preferred embodiment of the present invention.

Fig. 2 is a sub-flowchart of step S2 in fig. 1 according to the preferred embodiment of the present invention.

Fig. 3 is a sub-flowchart of step S3 in fig. 1 according to the preferred embodiment of the present invention.

Fig. 4 is another sub-flowchart of step S3 in fig. 1 according to the preferred embodiment of the present invention.

Fig. 5 is another sub-flowchart of step S3 in fig. 1 according to the preferred embodiment of the present invention.

Fig. 6 is another sub-flowchart of step S3 in fig. 1 according to the preferred embodiment of the present invention.

Fig. 7 is another sub-flowchart of step S3 in fig. 1 according to the preferred embodiment of the present invention.

Fig. 8 is another sub-flowchart of step S3 in fig. 1 according to the preferred embodiment of the present invention.

Fig. 9 is a schematic block diagram of an illegal operating vehicle identification system according to another embodiment of the present invention.

Fig. 10 is a schematic view of a sub-module structure of the data acquisition module in fig. 9 according to another embodiment of the present invention.

FIG. 11 is a sub-module structure diagram of the data cleansing module in FIG. 9 according to another embodiment of the present invention.

Fig. 12 is a sub-module structure diagram of the deep learning module in fig. 9 according to another embodiment of the present invention.

Illustration of the drawings:

11. a bayonet collector; 12. a data acquisition module; 13. a data cleaning module; 14. a controller; 15. a deep learning module; 16. an evidence collection module; 17. a video capture device; 121. a number plate collecting unit; 122. a card port data acquisition unit; 123. a card port longitude and latitude acquisition unit; 124. a video data acquisition unit; 125. a non-aperture data acquisition unit; 131. a data cleaning unit; 132. a data selection unit; 133. a data truncation unit; 151. a sample selection unit; 152. a data modeling unit; 153. a deep learning unit; 154. and an identification unit.

Detailed Description

The embodiments of the invention will be described in detail below with reference to the accompanying drawings, but the invention can be embodied in many different forms, which are defined and covered by the following description.

As shown in fig. 1, a preferred embodiment of the present invention provides an illegal operating vehicle identification method, which can realize efficient and accurate identification of an illegal operating vehicle, and the illegal operating vehicle identification method includes the following steps:

step S1: collecting vehicle data to establish a vehicle number plate database, a vehicle running track database and a card port longitude and latitude database;

step S2: cleaning the collected vehicle data;

step S3: deep learning is carried out based on the cleaned vehicle data, and an intelligent identification model is established;

step S4: recognizing all vehicles by using an intelligent recognition model; and

step S5: and obtaining the number plate of the vehicle in illegal operation based on the identification result, and correspondingly calling the video evidence or the picture evidence in illegal operation from the law enforcement video database.

It is understood that the step S5 may be omitted.

It is understood that in the step S1, vehicle license plate data may be obtained from the transportation vehicle management data system, the vehicle license plate data at least includes a vehicle license plate, a vehicle type, an operation type, and the like, and a vehicle license plate database is established based on the obtained vehicle license plate data; acquiring vehicle gate data from a traffic gate data system, wherein the vehicle gate data at least comprises a vehicle number plate, a gate number, elapsed time, a gate photo and the like, and establishing a vehicle running track database on the basis of the acquired vehicle gate data; the method comprises the steps of obtaining geographic position data of a gate from a traffic gate data system, wherein the geographic position data of the gate at least comprise a gate number, a location area where the gate belongs, the longitude and latitude of the gate and the like, and establishing a gate longitude and latitude database on the basis of the obtained gate data. Preferably, in step S1, the video camera data of the law enforcement point may be collected by a camera of the law enforcement point to build a law enforcement video database. Preferably, in step S1, the data amount in the vehicle operation track database may be supplemented by collecting non-bayonet data.

It can be understood that, as shown in fig. 2, the step S2 specifically includes the following steps:

s21: cleaning the time-disorderly and repeated data in the vehicle running track database;

s22: selecting illegal operating vehicle types needing to be identified, extracting vehicle running track data of vehicles of corresponding types according to the identified illegal operating vehicle types, and acquiring a legal operating vehicle number plate set and a non-operating vehicle number plate set according to vehicle operating categories; and

s23: and carrying out classification exploratory analysis on the extracted vehicle running track data to remove serious distortion data.

In the step S21, the time-disorderly and repeated data can be cleaned, so as to ensure the accuracy of the acquired vehicle gate data.

In the step S22, the illegal operating vehicle types include cars, vans, commercial vehicles, or buses, etc., and the vehicle operation trajectory data of all cars may be extracted by selecting the illegal operating vehicle types that need to be identified, for example, selecting only cars, while the operation trajectory data of other types of vehicles, such as vans, commercial vehicles, and buses, etc., are hidden, and the legal operating vehicle number plate set a and the non-operating vehicle number plate set B are obtained according to the vehicle operation categories. It is understood that the vehicle operation category includes non-operation vehicles and operation vehicles. The invention can extract the vehicle running track data corresponding to the vehicle type by selecting the illegal operating vehicle type, and can realize the rapid identification of the illegal operating vehicles of different vehicle types efficiently and accurately. It can be understood that, as a further preferable aspect, according to the feature that the operating time period of the illegally operated vehicle is strong in periodicity, there are holidays, early peaks, late peaks, and nights, and accordingly, only a specific time period is extracted when the vehicle operation trajectory data is extracted to identify the vehicle illegally operated in the selected time period.

In step S23, the vehicle trajectory data extracted in step S22 is subjected to a classification exploratory analysis, and the checkpoint data required for the classification exploratory analysis at least includes: counting the average daily passing bayonet number, the average time passing bayonet number, the average passing bayonet number, the working day and the average weekend passing bayonet number, intercepting data in u +/-3 sigma by applying a normal distribution model so as to clean the serious distortion data, and directly selecting the vehicle running track data within a certain percentage range according to the statistical characteristic value so as to clean the serious distortion data. According to the invention, through carrying out classification exploratory analysis on the extracted vehicle running track data, the data with serious distortion can be cleaned, and the identification accuracy is further ensured.

It is to be understood that, in the step S3, the deep learning is performed by one-dimensional time trajectory deep learning, two-dimensional space-time trajectory deep learning, two-dimensional thermodynamic diagram deep learning, three-dimensional space-time trajectory graph deep learning, and three-dimensional thermodynamic diagram deep learning. The method can be used for carrying out deep learning based on the multidimensional running track, carrying out multidimensional modeling on vehicle running track data and carrying out classification training by adopting a deep learning convolution neural network, thereby realizing the efficient and accurate identification of illegal operating vehicles.

As shown in fig. 3, when the one-dimensional time trajectory deep learning is adopted in step S3, the step S3 includes the steps of:

s31 a: selecting samples, including positive samples and negative samples;

s32 a: constructing a bayonet pair;

s33 a: setting the pass time granularity of the bayonet pairs;

s34 a: performing one-dimensional time running track matrix modeling;

s35 a: normalization processing; and

s36 a: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model.

It is to be understood that, in the step S31a, vehicle trajectory data R within a certain date range T days is selected from the vehicle trajectory database cleaned in the step S2, positive samples P and negative samples N of the same number plate number x, that is, legal service vehicle trajectory data and non-service vehicle trajectory data, are selected in R, and a set P of positive sample number plates is obtained_xNegative sample number plate set N_x。

It can be understood that, in step S32a, the vehicle running trajectory data R is sorted according to the vehicle number plate and the chronological order, a bayonet trajectory sequence of each number plate is obtained, the bayonet trajectory sequence is numbered from i-1, and adjacent passing bayonets form a bayonet pair: k is a radical of_ik_i+1Time △ t elapsed between adjacent gates_i+1＝t_i+1-t_iAnd constructing vehicle gate pair passing time data in the following format:

license plate number

Bayonet pair kiki+1

Passage time △ ti+1

It can be understood that in the step S33a, the overlength Δ t of rest and rest at night is removed, the Δ t is not less than 4 hours, normal distribution is applied, an interval t where the Δ t falls within u ± 3 σ is obtained, the interval t is also a value interval of the bayonet-pair passage time granularity, the bayonet-pair passage time granularity is set to be λ, and λ belongs to t if λ is a time segment.

It is understood that in the step S34a, the one-dimensional time trajectory matrix modeling is performed on the selected samples, and the time trajectory of each of the number plate vehicles is processed into the one-dimensional matrix R required by the convolutional neural network_mTaking the length of the one-dimensional time running track, which is T × 24 × 60/λ, j × λ represents the total time of j time segments, and if a certain vehicle appears at a certain gate in the j-th segment, then:

R_m[j]not, otherwise, R_m[j]＝0，

First, to R_mCarry out initialization, R_m[j]＝0，j∈(0，length)

Then, the start time t selected from the sample is calculated from the time when each number plate vehicle passes the f-th gate₀Time △ t used for f-th bayonet_ft₀＝t_f-t₀Then:

R_m[△t_ft₀/λ]＝1

according to the above, a time running track matrix R of each number plate is obtained_m(ii) a A positive sample matrix P is also obtained_mAnd negative sample matrix N_m。

It is understood that the time travel path matrix R for each number plate in the step S35a_mPositive sample matrix P_mAnd negative sample matrix N_mNormalization processing is performed to simplify the data throughput to ensure efficient recognition.

It is to be understood that, in the step S36a, the positive sample P and the positive sample matrix P of the one-dimensional time trajectory matrix corresponding to the responsible sample N are used_mAnd negative sample matrix N_mInputting a convolutional neural network, performing feature learning, and classifying the features learned by the convolutional neural network by using a softmax classifier to obtain a vehicle intelligent identification model PN based on one-dimensional deep learning_AIThen, using the verification data to continuously adjust T, x lambda and convolution neural network parameters for training to obtain the optimal intelligent recognition model PN_AI. The structure of the convolutional neural network is 4 Conv2d convolutional layers, 4 max-firing pooling layers, 2 Dense fully-connected layers and an output layer.

For example, the vehicle running track data R within 30 days is selected from the cleaned vehicle running track database, the positive sample P and the negative sample N with the same number plate number x of 20000 are selected from R, namely the taxi running track data and the family car running track data, and the positive sample number plate set P is obtained at the same time_xNegative sample number plate set N_x。

And sequencing the vehicle running track data according to the vehicle number plate and the time sequence to obtain a bayonet track sequence of each number plate, numbering the sequence from i to 1, and forming a bayonet pair by adjacent passing bayonets: k is a radical of_ik_i+1Time △ t elapsed between adjacent gates_i+1＝t_i+1-t_iAnd constructing vehicle gate pair passing time data in the following format:

license plate number

Bayonet pair kiki+1

Passage time △ ti+1

And then eliminating overlength delta t of rest and rest at night for not less than 4 hours, applying normal distribution, solving an interval t (3,71) of the delta t within u +/-3 sigma, wherein the interval t is also a value interval of the passing time granularity of the bayonet, setting the passing time granularity of the bayonet as lambda, determining lambda belongs to t, and initially taking the lambda as 5 minutes.

Then one-dimensional time running track matrix modeling is carried out, and the time running track of each license plate vehicle is processed into a one-dimensional matrix R required by a convolutional neural network_mTaking the length of the one-dimensional time running track, length 30 x 24 x 60/5 x 8640, (i-1) x lambda represents the time taken by the vehicle to pass through the ith gate, and for R_mAnd (3) initializing:

R_m[i]＝0，i∈(1，8640)

calculating the starting time t selected from the sample according to the time of each number plate vehicle passing through the f-th gate₀Time t used for f-th bayonet_ft₀＝t_f-t₀Then:

R_m[t_ft₀/5+1]＝1

according to the above, a time running track matrix R of each number plate is obtained_m(ii) a A positive sample matrix P is also obtained_mAnd negative sample matrix N_m；

Then to R_m、P_m、N_mNormalization processing;

then, the positive sample P and a positive sample matrix P of a one-dimensional time running track matrix corresponding to the responsible sample N are used_mAnd negative sample matrix N_mInputting a convolutional neural network, performing feature learning, and classifying the features learned by the convolutional neural network by using a softmax classifier to obtain a vehicle intelligent identification model PN based on one-dimensional deep learning_AIThen, using the validation data, the convolutional neural network parameters are continually adjusted T, x lambda,training is carried out to obtain the optimal intelligent recognition model PN_AI。

It is to be understood that, as shown in fig. 4, when the two-dimensional time trajectory deep learning is adopted in step S3, the step S3 includes the steps of:

s31 b: selecting samples, including positive samples and negative samples;

s32 b: constructing a bayonet pair;

s33 b: setting the time granularity of the bayonet;

s34 b: performing two-dimensional time running track matrix modeling;

s35 b: normalization processing; and

s36 b: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model.

It is understood that the step S31b is the same as S31a, S32b is the same as S32a, S33b is the same as S33a, and step S36b is the same as step S36 a.

It is understood that in the step S34b, the two-dimensional time trajectory matrix modeling is performed on the selected samples, and the time trajectory of each of the number plate vehicles is processed into the two-dimensional matrix R required by the convolutional neural network_mWhere the Column indicates the number of days, Column _ width is T, the row indicates the transit time, and row _ height is 24 60/λ, then R is the pair_mAnd (3) initializing:

R_m[m][n-1]＝0，m∈(0，24*60/λ)，n∈(1，T)，max(m)*λ＝24*60；

calculating the nth day of each vehicle in (1, T) according to the date of each vehicle passing the f-th gate, and selecting the starting time T from the sample₀Time △ t used for f-th bayonet_ft₀＝t_f-t₀Then m is △ t_ft₀λ - (n-1) × 24 × 60/λ, then:

R_m[△t_ft₀/λ-(n-1)*24*60/λ][n-1]＝1；

according to the above, a time running track matrix R of each number plate is obtained_m(ii) a A positive sample matrix P is also obtained_mAnd negative sample matrix N_m。

It is understood that in said stepTime travel locus matrix R for each number plate in step S35b_mPositive sample matrix P_mAnd negative sample matrix N_mNormalization processing is performed to simplify the data throughput to ensure efficient recognition.

It is to be understood that, as shown in fig. 5, when the two-dimensional spatiotemporal trajectory deep learning is adopted in step S3, the step S3 includes the steps of:

s31 c: selecting samples, including positive samples and negative samples;

s32 c: constructing a bayonet pair;

s33 c: setting the time granularity of the bayonet;

s34 c: continuing modeling the two-dimensional bayonet space matrix;

s35 c: performing two-dimensional space-time movement track matrix modeling;

s36 c: normalization processing; and

s37 c: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model.

It is understood that the step S31c is the same as S31a, S32c is the same as S32a, S33c is the same as S33a, S36c is the same as S35a, and S37c is the same as S36 a.

It can be further understood that, in the step S34c, the operation area is divided into square lattices with an L size according to the longitude and latitude data of the bayonet, and each lattice is numbered according to the number of the lattices in the longitude and latitude direction and the bayonet number, so as to obtain the bayonet SPACE matrix SPACE_m，SPACE_m[u][v]The grids are numbered as grids, and the grids belonging to the bayonets are the space positions represented by the bayonets.

In the step S35c, two-dimensional space-time movement trajectory matrix modeling is performed on the selected samples, and the time movement trajectory of each number plate vehicle is processed into a two-dimensional matrix R required by a convolutional neural network_mWhere the Column indicates the number of days, Column _ width is T, the row indicates the transit time, and row _ height is 24 60/λ, then R is the pair_mAnd (3) initializing:

R_m[m][n-1]＝0，m∈(0，24*60/λ)，n∈(1，T)，max(m)*λ＝24*60

according to the day that each number plate vehicle passes through the f-th bayonetPeriod, which is calculated on the nth day of the (1, T) row, starting time T selected from the sample₀Time △ t used for f-th bayonet_ft₀＝t_f-t₀Then m is △ t_ft₀The/lambda- (n-1) 24-60/lambda searches the SPACE matrix SPACE of the bayonet according to the serial number of the f-th bayonet_mObtaining the corresponding SPACE number SPACE_m[u][v]Then:

R_m[△t_ft₀/λ-(n-1)*24*60/λ][n-1]＝SPACE_m[u][v]

accordingly, a space-time running track matrix R of each number plate is obtained_m(ii) a A positive sample matrix P is also obtained_mAnd negative sample matrix N_m。

It is to be understood that, as shown in fig. 6, when the two-dimensional thermodynamic diagram deep learning is adopted in step S3, the step S3 includes the steps of:

s31 d: selecting samples, including positive samples and negative samples;

s32 d: forming a foreground thermodynamic diagram based on the vehicle running track data;

s33 d: modeling a two-dimensional bayonet space matrix;

s34 d: normalization processing; and

s35 d: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model.

It is understood that the step S31d is the same as S31a, S34d is the same as S35a, and S35d is the same as S36 a.

In the step S32d, the vehicle running track data is sorted according to the vehicle number plate and the time sequence, the times of the vehicle passing through the gate are marked on the map in the form of thermodynamic diagrams until the last gate, and then the background base diagram is generated to form the foreground thermodynamic diagram.

In the step S33d, two-dimensional bayonet SPACE matrix modeling is performed on the selected sample, the operation area is divided into square lattices with the side length of L size according to the data of the longitude and latitude of the bayonet, and each lattice is numbered according to the number of the lattices in the longitude and latitude direction and the number of the bayonet, so as to obtain a bayonet SPACE matrix SPACE_mThe lattices to which the bayonets belongThe SPACE position represented by the bayonet is obtained, and finally, the thermodynamic value of the corresponding square in the thermodynamic diagram is filled in to obtain a bayonet SPACE matrix SPACE_m[u][v]. According to the method, a thermodynamic diagram running track matrix R-SPACE of each number plate is obtained_m(ii) a Simultaneously, a positive sample matrix P-SPACE is obtained_mAnd negative sample matrix N-SPACE_m。

As shown in fig. 7, when the depth learning in step S3 is performed by depth learning of a three-dimensional spatiotemporal trajectory graph, the step S3 specifically includes the following steps:

s31 e: selecting samples, including positive samples and negative samples;

s32 e: constructing a bayonet pair;

s33 e: setting the time granularity of the bayonet;

s34 e: modeling a three-dimensional time running track matrix;

s35 e: normalization processing; and

s36 e: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model.

It is understood that the step S31e is the same as S31a, S32e is the same as S32a, S33e is the same as S33a, S35e is the same as S35a, and S36e is the same as S36 a.

It can be understood that, in the step S34e, the three-dimensional SPACE-time trajectory graph is modeled on the selected sample, the operation area is divided into square lattices with the side length of L size according to the longitude and latitude data of the bayonet, and each lattice is numbered according to the number of the lattices in the longitude and latitude direction and the number of the bayonet, so as to obtain the SPACE matrix SPACE of the bayonet_m，SPACE_m[u][v]The grids are numbered as grids, and the grids belonging to the bayonets are the space positions represented by the bayonets. The channel number is the maximum value of the one-dimensional time running track sequence, and the channel is T24 60/lambda; initializing the three-dimensional matrix:

Channel-SPACE_m[u][v][c]＝0，c∈(0，channel)

calculating the starting time t selected from the sample according to the time of each number plate vehicle passing through the f-th gate₀Time △ t used for f-th bayonet_ft₀＝t_f-t₀Then c is △ t_ft₀λ, directly determining SPACE according to the number of card number_m[u][v]Then, there are:

Channel-SPACE_m[u][v][△t_ft₀/λ]＝SPACE_m[u][v]

if the two-dimensional space-time trajectory curves pass through the adjacent bayonets and are directly connected, a spirally rising three-dimensional space-time trajectory curve is formed;

accordingly, a three-dimensional space-time trajectory curve diagram R of each number plate is obtained_m(ii) a At the same time, a positive sample P is obtained_mAnd negative sample N_m。

As shown in fig. 8, when the deep learning in step S3 is performed by the three-dimensional thermodynamic diagram deep learning, step S3 specifically includes the steps of:

s31 f: selecting samples, including positive samples and negative samples;

s32 f: constructing a bayonet pair;

s33 f: setting the time granularity of the bayonet;

s34 f: carrying out three-dimensional thermodynamic diagram modeling;

s35 f: normalization processing; and

s36 f: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model.

It is understood that the step S31f is the same as S31a, S32f is the same as S32a, S33f is the same as S33a, S35f is the same as S35a, and S36f is the same as S36 a.

It can be understood that, in the step S34f, the selected sample is subjected to three-dimensional thermodynamic modeling, the operation area is divided into square lattices with side lengths of L size according to the longitude and latitude data of the bayonet, and each lattice is numbered according to the number of the lattices in the longitude and latitude direction and the bayonet number, so as to obtain the SPACE matrix SPACE of the bayonet_mThe lattice to which the bayonet belongs is the spatial position represented by the bayonet. The channel number is the maximum value of the one-dimensional time running track sequence, and the channel is T24 60/lambda; initializing the three-dimensional matrix:

Channel-SPACE_m[u][v][c]＝0，c∈(0，channel)

sequencing the vehicle running track data according to the vehicle number plate and the time sequence, drawing the times of the vehicles passing through the gates in a thermodynamic diagram mode in a layered mode by taking lambda as a unit or drawing the vehicles in a layered accumulation mode, thus obtaining a three-dimensional layered gradual change thermodynamic diagram or a hierarchical change thermodynamic diagram by taking lambda as a unit until the last gate, and then generating a bottom diagram to form the foreground thermodynamic diagram.

Calculating the starting time t selected from the sample according to the time of each number plate vehicle passing through the f-th gate₀Time △ t used for f-th bayonet_ft₀＝t_f-t₀Then c is △ t_ft₀/λ；

Filling the thermal values of the corresponding squares in the layered thermodynamic diagram into a three-dimensional thermodynamic diagram matrix Channel-SPACE_m[u][v][c]。

Accordingly, the three-dimensional thermodynamic diagram R-SPACE of each number plate is obtained_m(ii) a At the same time, a positive sample P-SPACE is obtained_mAnd negative sample N-SPACE_m。

It can be understood that, in the step S3, according to the characteristics of the convolutional neural network, since the detail degree of the information of the acquired samples is different, the recognition accuracy of the three-dimensional space-time trajectory graph deep learning and the three-dimensional thermodynamic diagram deep learning is higher than that of the two-dimensional time trajectory deep learning, the two-dimensional space-time trajectory deep learning and the two-dimensional thermodynamic diagram deep learning, and the recognition accuracy of the two-dimensional time trajectory deep learning, the two-dimensional space-time trajectory deep learning and the two-dimensional thermodynamic diagram deep learning is higher than that of the one-dimensional time trajectory deep learning. During identification, the method can be started from different dimensions at the same time, various identification results are mutually supplemented and proved, and the inspection is carried out layer by layer, so that the identification accuracy is ensured.

It can be understood that, in the step S4, the intelligent recognition model is applied to recognize all sample data, and an operating vehicle set a 'and a non-operating vehicle set B' are obtained, where an illegal operating vehicle exists in the operating vehicle, that is, the non-operating vehicle participates in the operation, then: a ' > A, B ' < B, then A ' -A is the illegal operation vehicle number plate set. A ≈ B ≈ A '+ B' when the recognition rate is 100%. A '. u (A '. sub.B ' -A. sub.B) is the license plate of vehicle for illegal local operation of vehicle in different place. The invention can carry out intelligent identification on the illegal operating vehicle through the intelligent identification model, and quickly and accurately troubleshoot the illegal operating vehicle.

It is understood that, in the step S5, the vehicle number plate of the illegal operation vehicle is obtained based on the recognition result in the step S4, and the video evidence or the picture evidence of the illegal operation is correspondingly called from the law enforcement video database, or the video evidence or the picture evidence of the illegal operation is directly obtained through the camera on the site of the law enforcement point.

The method for identifying the illegal operating vehicles obtains intelligent identification models under each dimension model by performing multi-dimension modeling on vehicle running track data and learning and training by using a convolutional neural network, obtains illegal operating vehicle sets under corresponding models, supplements and proves each other, ensures the identification precision, particularly combines video data acquisition, powerfully improves the legality and efficiency of checking illegal operating vehicles, beautifies vehicle operating environment, and ensures the safety of people's life and property. Meanwhile, a convolutional neural network is adopted, so that errors introduced in explicit feature extraction and threshold setting are avoided, and learning is performed from training data implicitly; the convolutional neural network has unique superiority in the aspect of image processing, and particularly, the complexity of data reconstruction in the processes of feature extraction and classification is avoided due to the fact that images of multi-dimensional input vectors can be directly input into the network.

As shown in fig. 9, another embodiment of the present invention further provides an illegal operating vehicle identification system, which can realize efficient and accurate identification of an illegal operating vehicle, and is preferably applied to the above-mentioned illegal operating vehicle identification method. The illegal operating vehicle identification system comprises a data acquisition module 12, a data cleaning module 13, a controller 14 and a deep learning module 15, wherein the data acquisition module 12 is used for acquiring vehicle data to establish a vehicle number plate database, a vehicle running track database and a bayonet longitude and latitude database, the data cleaning module 13 is used for cleaning the acquired vehicle data, the deep learning module 15 is used for deep learning based on the cleaned vehicle running track database to establish an intelligent identification model and identifying all vehicles by utilizing the intelligent identification model, the data acquisition module 12, the data cleaning module 13 and the deep learning module 15 are connected with the controller 14, and the data cleaning module 13 is also connected with the data acquisition module 12.

It can be understood that, as an optimal choice, the illegal operation vehicle identification system further includes a bayonet collector 11, the bayonet collector 11 is connected with a data acquisition module 12, and the data acquisition module 12 acquires vehicle bayonet data through the bayonet collector 11 to establish a vehicle operation track database.

Preferably, the illegal operating vehicle identification system further comprises a video acquisition device 17 for acquiring video camera data of law enforcement points, the video acquisition device 17 is connected with the data acquisition module 12, and the data acquisition module 12 acquires the video camera data of the law enforcement points through the video acquisition device 17 to establish a law enforcement video database. It will be appreciated that the video capture device 17 is a video camera.

As shown in fig. 10, the data acquisition module 12 includes a number plate acquisition unit 121 for acquiring vehicle number plate data to establish a vehicle number plate database, a gate data acquisition unit 122 for acquiring vehicle gate data to establish a vehicle running track database, and a gate longitude and latitude acquisition unit 123 for acquiring geographic position data of a gate to establish a gate longitude and latitude database; the number plate acquisition unit 121, the card port data acquisition unit 122 and the card port longitude and latitude acquisition unit 123 are all connected with the controller 14. The number plate acquisition unit 121 may acquire vehicle number plate data from a traffic management data system, the vehicle number plate data at least includes a vehicle number plate, a vehicle type, an operation type, and the like, and a vehicle number plate database is established based on the acquired vehicle number plate data. The vehicle access data acquisition unit 122 can acquire vehicle access data from a traffic access data system, or the access data acquisition unit 122 is connected with the access collector 11 and directly acquires the vehicle access data through the access collector 11, wherein the vehicle access data at least comprises a vehicle number plate, an access serial number, transit time, an access photo and the like, and a vehicle running track database is established on the basis of the acquired vehicle access data. The card slot longitude and latitude acquisition unit 123 may acquire geographic position data of each card slot from a traffic card slot data system, where the card slot geographic position data at least includes a card slot number, a card slot belonging position area, card slot longitude and latitude, and the like, and establish a card slot longitude and latitude database based on the acquired vehicle card slot data. Preferably, the data acquisition module 12 further comprises a video data acquisition unit 124 for acquiring video camera data of law enforcement points to establish a law enforcement video database, and the video data acquisition unit 124 is connected with the controller 14 and the video acquisition device 17. Preferably, the data acquisition module 12 further includes a non-bayonet data acquisition unit 125 for acquiring non-bayonet data to supplement the data amount in the vehicle trajectory database, and the non-bayonet data acquisition unit 125 is connected to the controller 14.

As shown in fig. 11, the data cleaning module 13 includes a data cleaning unit 131 for cleaning time-disorderly and repeated data in the vehicle operation trajectory database, a data selecting unit 132 for selecting illegal vehicle types to be identified, extracting vehicle operation trajectory data of vehicles of corresponding types according to the identified illegal vehicle types, and acquiring a legal operation vehicle license plate set and a non-operation vehicle license plate set according to vehicle operation categories, and a data truncating unit 133 for performing classification exploratory analysis on the extracted vehicle operation trajectory data to remove severely distorted data, wherein the data cleaning unit 131, the data selecting unit 132, and the data truncating unit 133 are all connected to the controller 14. It is understood that the data selection unit 132 is also connected to the data cleansing unit 131 and the data truncation unit 133.

It can be understood that the data cleaning module 13 cleans the time-disorderly and repeated data, and ensures the accuracy of the collected vehicle gate data. The data selecting unit 132 selects an illegal operating vehicle type to be identified, for example, only a car, so that vehicle travel track data of all cars can be extracted, while travel track data of other types of vehicles, such as a minibus, a commercial vehicle, a passenger car, and the like, are hidden, and a legal operating vehicle number plate set a and a non-operating vehicle number plate set B are obtained according to vehicle operation categories. Preferably, in order to perform illegal operation on the self-cloning fake plate of the legal operation vehicle, the data selection unit 132 can randomly pair the legal operation vehicle track databases, and the track data of the vehicles with the same number plate is doubled, so that the data selection unit is specially used for identifying the cloning illegal operation vehicle. It can be understood that, as a further preferable, according to the feature that the operating time period of the illegally operated vehicle is periodically strong, there are holidays, early peaks, late peaks, and nights, and only a specific time period is extracted by the data selecting unit 132 when extracting the vehicle operation trajectory data accordingly to identify the vehicle illegally operated in the selected time period. The data truncation unit 133 may perform classification exploratory analysis on the vehicle running track data extracted by the data selection unit 132, where the checkpoint data required by the classification exploratory analysis at least includes: counting the average daily passing bayonet number, the average time passing bayonet number, the average passing bayonet number, the working day and the average weekend passing bayonet number, intercepting data in u +/-3 sigma by applying a normal distribution model so as to clean the serious distortion data, and directly selecting the vehicle running track data within a certain percentage range according to the statistical characteristic value so as to clean the serious distortion data. According to the invention, the extracted vehicle running track data is subjected to classified exploratory analysis, so that seriously distorted data can be clear, and the identification accuracy is further ensured.

As shown in fig. 12, the deep learning module 15 includes a sample selecting unit 151 for selecting a sample from the washed vehicle running track database, a data modeling unit 152 for performing modeling processing on the sample, a deep learning unit 153 for performing deep learning on the modeled sample by using a convolutional neural network to establish an intelligent recognition model, and an identifying unit 154 for identifying an illegal operating vehicle by using the intelligent recognition model, and the sample selecting unit 151, the data modeling unit 152, the deep learning unit 153, and the identifying unit 154 are all connected to the controller 14. It is understood that the data modeling unit 152 is connected to the sample selection unit 151 and the deep learning unit 153, respectively, and the deep learning unit 153 and the recognition unit 154. It is understood that the deep learning in the deep learning unit 153 is performed by one-dimensional time trajectory deep learning, two-dimensional space-time trajectory deep learning, two-dimensional thermodynamic diagram deep learning, three-dimensional space-time trajectory graph deep learning, and three-dimensional thermodynamic diagram deep learning. The convolutional neural network included in the deep learning unit 153 has a structure of 4 Conv2d convolutional layers, 4 max-posing pooling layers, 2 sense fully-connected layers and an output layer. Preferably, the deep learning unit 153 classifies the features learned by the convolutional neural network by using a softmax classifier to obtain an intelligent recognition model.

Preferably, the illegal operation vehicle identification system further comprises an evidence collection module 16 for calling corresponding illegal operation video evidence or picture evidence based on the identification result, and the evidence collection module 16 is connected with the controller 14 and the data collection module 12. After the deep learning module 15 identifies the illegal operating vehicle, the feedback information is sent to the controller 14, the controller 14 receives the feedback information and then controls the evidence collection module 16 to execute an evidence collection instruction, the evidence collection module 16 extracts the number plate of the illegal operating vehicle from the vehicle number plate database of the data collection module 12, correspondingly calls the video evidence and/or the picture evidence of the illegal operating vehicle for illegal operation from the law enforcement video database, and sends the warning information to the controller 14 after the evidence is collected. It is understood that, as a variation, the evidence collection module 16 may also be connected to the video collection device 17, and directly collect the video evidence and/or the picture evidence of illegal operations through the video collection device 17.

The controller 14 may control the data acquisition module 12 to acquire vehicle data by sending an acquisition data instruction to the data acquisition module 12, and after the data acquisition module 12 finishes acquiring, send completion information back to the controller 14, and wait for a next data acquisition instruction of the controller 14. The controller 14 receives the acquisition completion feedback information fed back by the data acquisition module 12 and then sends a data cleaning instruction to the data cleaning module 13 to control the data cleaning module 13 to clean the vehicle data acquired by the data acquisition module 12. After the data cleaning module 13 finishes cleaning, sending completion information to be fed back to the controller 14 and waiting for a next data cleaning instruction of the controller 14, the controller 14 receiving the feedback information and sending a deep learning instruction to the deep learning module 15, the deep learning module 15 executing the deep learning instruction to identify an illegal operating vehicle, and after the identification is finished, sending the completion information to be fed back to the controller 14 and waiting for the next deep learning instruction of the controller 14. The controller 14 receives the information fed back by the deep learning module 15 and then sends an evidence collection instruction to the evidence collection module 16, the evidence collection module 16 executes the evidence collection instruction, extracts the number plate of the illegal operating vehicle from the vehicle number plate database of the data collection module 12 and correspondingly calls the video evidence and/or the picture evidence of the illegal operating vehicle for illegal operation from the law enforcement video database, and after the evidence collection is completed, the evidence collection module 16 sends alarm information to the controller 14.

The identification system of the illegal operating vehicle performs multidimensional modeling on the vehicle running track data, utilizes the convolutional neural network to learn and train, obtains intelligent identification models under all dimensional models, obtains illegal operating vehicle sets under corresponding models, supplements and proves each other, ensures the identification precision, particularly combines with video data acquisition, powerfully improves the legality and efficiency of checking illegal operating vehicles, beautifies the vehicle operating environment, and ensures the safety of people's life and property. Meanwhile, a convolutional neural network is adopted, so that errors introduced in explicit feature extraction and threshold setting are avoided, and learning is performed from training data implicitly; the convolutional neural network has unique superiority in the aspect of image processing, and particularly, the complexity of data reconstruction in the processes of feature extraction and classification is avoided due to the fact that images of multi-dimensional input vectors can be directly input into the network.

Another embodiment of the present invention also provides a computer-readable storage medium for storing a computer program for performing illegal operation vehicle recognition, the computer program performing the following steps when running on a computer:

s1: collecting vehicle data to establish a vehicle number plate database, a vehicle running track database and a card port longitude and latitude database;

s2: cleaning the collected vehicle data;

s3: deep learning is carried out based on the cleaned vehicle data, and an intelligent identification model is established; and

s4: and identifying all vehicles by using the intelligent identification model.

The general form of computer readable media includes: floppy disk (floppy disk), flexible disk (flexible disk), hard disk, magnetic tape, any of its magnetic media, CD-ROM, any of the other optical media, punch cards (punchcards), paper tape (paper tape), any of the other physical media with patterns of holes, Random Access Memory (RAM), Programmable Read Only Memory (PROM), Erasable Programmable Read Only Memory (EPROM), FLASH erasable programmable read only memory (FLASH-EPROM), any of the other memory chips or cartridges, or any of the other media from which a computer can read. The instructions may further be transmitted or received by a transmission medium. The term transmission medium may include any tangible or intangible medium that is operable to store, encode, or carry instructions for execution by the machine, and includes digital or analog communications signals or intangible medium that facilitates communication of the instructions. Transmission media include coaxial cables, copper wire and fiber optics, including the wires that comprise a bus for transmitting a computer data signal.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (6)

1. An illegal operating vehicle identification method is characterized in that:

which comprises the following steps:

s1: collecting vehicle data to establish a vehicle number plate database, a vehicle running track database and a card port longitude and latitude database;

s2: cleaning the collected vehicle data;

s3: deep learning is carried out based on the cleaned vehicle data, and an intelligent identification model is established; and

s4: recognizing all vehicles by using an intelligent recognition model to obtain an operating vehicle set A ' and a non-operating vehicle set B ', wherein the operating vehicle set A ' is greater than a legal operating vehicle number plate set A because illegal operating vehicles exist in the operating vehicles, and the non-operating vehicle set B ' is less than a non-operating vehicle number plate set B, so that A ' -A is the illegal operating vehicle number plate set;

the deep learning in the step S3 is performed in one-dimensional time trajectory deep learning, two-dimensional space-time trajectory deep learning, two-dimensional thermodynamic diagram deep learning, three-dimensional space-time trajectory graph deep learning and three-dimensional thermodynamic diagram deep learning;

when the one-dimensional time trajectory deep learning is adopted in the step S3, the step S3 includes the steps of:

s31 a: selecting samples, including positive samples and negative samples;

s32 a: constructing a bayonet pair, sequencing vehicle running track data according to vehicle license plates and time sequence to obtain a bayonet track sequence of each license plate, numbering the sequence from i to 1, and forming the bayonet pair by adjacent passing bayonets: k is a radical of_ik_i+1Time △ t elapsed between adjacent gates_i+1＝t_i+1-t_iAnd constructing vehicle gate pair passing time data in the following format:

license plate number Bayonet pair k_ik_i+1 Passage time △ t_i+1

；

S33 a: setting the passing time granularity of the bayonet pairs, eliminating overlength delta t for rest and rest at night, applying normal distribution, solving an interval t of the delta t in u +/-3 sigma, wherein the interval t is also a value interval of the passing time granularity of the bayonet pairs, setting the passing time granularity of the bayonet pairs as lambda, lambda is a time segment, and lambda belongs to t;

s34 a: performing one-dimensional time running track matrix modeling;

s35 a: normalization processing; and

s36 a: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model;

wherein, step S34a specifically includes:

processing the time running track of each number plate vehicle into a one-dimensional matrix R required by a convolution neural network_mTaking the length of the one-dimensional time running track, where T is T24 60/λ, T represents the number of days of the date range of the selected sample, λ represents the bayonet-to-transit time granularity, j λ represents the total time of j time segments, and if a certain vehicle appears in a certain bayonet in a jth segment, then:

R_m[j]not, otherwise, R_m[j]＝0，

First, to R_mCarry out initialization, R_m[j]＝0，j∈(0，length)

Then, the start time t selected from the sample is calculated from the time when each number plate vehicle passes the f-th gate₀Time △ t used for f-th bayonet_ft₀＝t_f-t₀Then:

R_m[△t_ft₀/λ]＝1

according to the above, a time running track matrix R of each number plate is obtained_m(ii) a A positive sample matrix P is also obtained_mAnd negative sample matrix N_m；

When the two-dimensional time trajectory deep learning is adopted in step S3, the step S3 includes the steps of:

s31 b: selecting samples, including positive samples and negative samples;

s32 b: constructing a bayonet pair, sequencing the vehicle running track data according to the vehicle number plate and the time sequence to obtainObtaining a bayonet track sequence of each number plate, numbering the sequence from i to 1, and forming a bayonet pair by adjacent passing bayonets: k is a radical of_ik_i+1Time △ t elapsed between adjacent gates_i+1＝t_i+1-t_iAnd constructing vehicle gate pair passing time data in the following format:

license plate number Bayonet pair k_ik_i+1 Passage time △ t_i+1

；

S33 b: setting bayonet pair time granularity, eliminating overlength delta t for rest and rest at night, applying normal distribution to obtain an interval t of the delta t in u +/-3 sigma, wherein the interval t is also a value interval of the bayonet pair passing time granularity, setting the bayonet pair passing time granularity as lambda, and setting the lambda as a time segment, wherein the lambda belongs to the t;

s34 b: performing two-dimensional time running track matrix modeling;

s35 b: normalization processing; and

s36 b: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model;

wherein, the step S34b is specifically

Processing the time running track of each number plate vehicle into a two-dimensional matrix R required by a convolution neural network_mWhere the Column indicates the number of days, Column width, Column _ width, T, date range, row indicates the transit time, row _ height, 24, 60/λ, λ indicates the traffic time granularity for the bayonet pair, and then for R_mAnd (3) initializing:

R_m[m][n-1]＝0，m∈(0，24*60/λ)，n∈(1，T)，max(m)*λ＝24*60

calculating the nth day of each vehicle in (1, T) according to the date of each vehicle passing the f-th gate, and selecting the starting time T from the sample₀Time △ t used for f-th bayonet_ft₀＝t_f-t₀Then m is △ t_ft₀λ - (n-1) × 24 × 60/λ, then:

R_m[△t_ft₀/λ-(n-1)*24*60/λ][n-1]＝1

according to the above, a time running track matrix R of each number plate is obtained_m(ii) a A positive sample matrix P is also obtained_mAnd negative sample matrix N_m；

When the two-dimensional spatiotemporal trajectory deep learning is adopted in the step S3, the step S3 includes the steps of:

s31 c: selecting samples, including positive samples and negative samples;

s32 c: constructing a bayonet pair, sequencing vehicle running track data according to vehicle license plates and time sequence to obtain a bayonet track sequence of each license plate, numbering the sequence from i to 1, and forming the bayonet pair by adjacent passing bayonets: k is a radical of_ik_i+1Time △ t elapsed between adjacent gates_i+1＝t_i+1-t_iConstructing vehicle gate-to-pass time data in the format

License plate number Bayonet pair k_ik_i+1 Passage time △ t_i+1

；

S33 c: setting bayonet pair time granularity, eliminating overlength delta t for rest and rest at night, applying normal distribution to obtain an interval t of the delta t in u +/-3 sigma, wherein the interval t is also a value interval of the bayonet pair passing time granularity, setting the bayonet pair passing time granularity as lambda, and setting the lambda as a time segment, wherein the lambda belongs to the t;

s34 c: modeling a two-dimensional bayonet space matrix;

s35 c: performing two-dimensional space-time movement track matrix modeling;

s36 c: normalization processing; and

s37 c: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model;

wherein, the step S34c is specifically

Dividing an operation area into square lattices with the side length of L size according to the longitude and latitude data of the bayonet, numbering each lattice according to the number of the lattices in the longitude and latitude direction and the number of the bayonet, and obtaining a spatial matrix SPACE of the bayonet_m，SPACE_m[u][v]The grids are numbered as grids, and the grids belonging to the bayonets are the space positions represented by the bayonets;

the step S35c specifically includes:

processing the time running track of each number plate vehicle into a two-dimensional matrix R required by a convolution neural network_mWhere the Column indicates the number of days, Column width, Column _ width, T, date range, row indicates the transit time, row _ height, 24, 60/λ, λ indicates the traffic time granularity for the bayonet pair, and then for R_mAnd (3) initializing:

R_m[m][n-1]＝0，m∈(0，24*60/λ)，n∈(1，T)，max(m)*λ＝24*60；

calculating the nth day of each vehicle in (1, T) according to the date of each vehicle passing the f-th gate, and selecting the starting time T from the sample₀Time △ t used for f-th bayonet_ft₀＝t_f-t₀Then m is △ t_ft₀The/lambda- (n-1) 24-60/lambda searches the SPACE matrix SPACE of the bayonet according to the serial number of the f-th bayonet_mObtaining the corresponding SPACE number SPACE_m[u][v]Then:

R_m[△t_ft₀/λ-(n-1)*24*60/λ][n-1]＝SPACE_m[u][v]；

accordingly, a space-time running track matrix R of each number plate is obtained_m(ii) a A positive sample matrix P is also obtained_mAnd negative sample matrix N_m；

When the two-dimensional thermodynamic diagram deep learning is adopted in the step S3, the step S3 specifically includes the following steps:

s31 d: selecting samples, including positive samples and negative samples;

s32 d: forming a foreground thermodynamic diagram based on the vehicle running track data, sequencing the vehicle running track data according to the vehicle number plate and the time sequence, marking the times of passing the vehicle through the gates on a map in the form of the thermodynamic diagram until the last gate is reached, and then, generating a background base map to form the foreground thermodynamic diagram;

s33 d: modeling a two-dimensional bayonet space matrix;

s34 d: normalization processing; and

s35 d: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model;

wherein, the step S33d is specifically

Dividing an operation area into square lattices with the side length of L size according to the longitude and latitude data of the bayonet, numbering each lattice according to the number of the lattices in the longitude and latitude direction and the number of the bayonet, and obtaining a spatial matrix SPACE of the bayonet_mThe grid to which the bayonet belongs is the SPACE position represented by the bayonet, and finally, the thermodynamic value of the corresponding square in the thermodynamic diagram is filled to obtain the SPACE matrix SPACE of the bayonet_m[u][v]And according to the above steps, obtaining thermodynamic diagram running track matrix R-SPACE of each number plate_m(ii) a Simultaneously, a positive sample matrix P-SPACE is obtained_mAnd negative sample matrix N-SPACE_m；

When the step S3 adopts the three-dimensional spatiotemporal trajectory graph for deep learning, the step S3 specifically includes the following steps:

s31 e: selecting samples, including positive samples and negative samples;

s32 e: constructing a bayonet pair, sequencing the vehicle running track data according to the vehicle number plate and time sequence to obtain a bayonet track sequence of each number plate, and sequencing the sequenceif i is 1, the adjacent passing bayonets form a bayonet pair: k is a radical of_ik_i+1Time △ t elapsed between adjacent gates_i+1＝t_i+1-t_iAnd constructing vehicle gate pair passing time data in the following format:

license plate number Bayonet pair k_ik_i+1 Passage time △ t_i+1

；

S33 e: setting bayonet pair time granularity, eliminating overlength delta t for rest and rest at night, applying normal distribution to obtain an interval t of the delta t in u +/-3 sigma, wherein the interval t is also a value interval of the bayonet pair passing time granularity, setting the bayonet pair passing time granularity as lambda, and setting the lambda as a time segment, wherein the lambda belongs to the t;

s34 e: carrying out three-dimensional time running track matrix modeling;

s35 e: normalization processing; and

s36 e: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model;

wherein, the step S34e is specifically

Dividing an operation area into square lattices with the side length of L size according to the longitude and latitude data of the bayonet, numbering each lattice according to the number of the lattices in the longitude and latitude direction and the number of the bayonet, and obtaining a spatial matrix SPACE of the bayonet_m，SPACE_m[u][v]The number of the channels is the maximum value of the one-dimensional time running track sequence, the channel is T24 60/lambda, lambda represents the granularity of the passing time of the bayonet, and T represents the granularity of the passing time of the bayonetThe number of days of the selected sample; initializing the three-dimensional matrix:

Channel-SPACE_m[u][v][c]＝0，c∈(0，channel)；

calculating the starting time t selected from the sample according to the time of each number plate vehicle passing through the f-th gate₀Time △ t used for f-th bayonet_ft₀＝t_f-t₀Then c is △ t_ft₀λ, directly determining SPACE according to the number of card number_m[u][v]Then, there are:

Channel-SPACE_m[u][v][△t_ft₀/λ]＝SPACE_m[u][v]；

if the two-dimensional space-time trajectory curves pass through the adjacent bayonets and are directly connected, a spirally rising three-dimensional space-time trajectory curve is formed;

accordingly, a three-dimensional space-time trajectory curve diagram R of each number plate is obtained_m(ii) a At the same time, a positive sample P is obtained_mAnd negative sample N_m；

When the three-dimensional thermodynamic diagram deep learning is adopted in the step S3, the step S3 specifically includes the following steps:

s31 f: selecting samples, including positive samples and negative samples;

s32 f: constructing a bayonet pair, sequencing vehicle running track data according to vehicle license plates and time sequence to obtain a bayonet track sequence of each license plate, numbering the sequence from i to 1, and forming the bayonet pair by adjacent passing bayonets: k is a radical of_ik_i+1Time △ t elapsed between adjacent gates_i+1＝t_i+1-t_iAnd constructing vehicle gate pair passing time data in the following format:

license plate number Bayonet pair k_ik_i+1 Passage time △ t_i+1

；

S33 f: setting bayonet pair time granularity, eliminating overlength delta t for rest and rest at night, applying normal distribution to obtain an interval t of the delta t in u +/-3 sigma, wherein the interval t is also a value interval of the bayonet pair passing time granularity, setting the bayonet pair passing time granularity as lambda, and setting the lambda as a time segment, wherein the lambda belongs to the t;

s34 f: carrying out three-dimensional thermodynamic diagram modeling;

s35 f: normalization processing; and

s36 f: deep learning is carried out by utilizing a convolutional neural network to establish an intelligent recognition model;

wherein, the step S34f is specifically

Dividing an operation area into square lattices with L sizes according to the longitude and latitude data of the bayonets, numbering each lattice according to the number of the lattices in the longitude and latitude directions and the bayonets, and obtaining a bayonet SPACE matrix SPACE_mThe grid to which the bayonet belongs is the space position represented by the bayonet, the number of channels is the maximum value of the one-dimensional time running track sequence, the channel is T24 60/lambda, lambda represents the granularity of the bayonet to the passing time, and T represents the number of days of the selected sample; initializing the three-dimensional matrix:

Channel-SPACE_m[u][v][c]＝0，c∈(0，channel)；

sequencing vehicle running track data according to vehicle license plate number and time sequence, drawing the times of vehicles passing through a gate in a thermodynamic diagram mode in a layered mode by taking lambda as a unit or drawing the vehicles in a layered accumulation mode, thus obtaining a three-dimensional layered gradual change thermodynamic diagram or a hierarchical change thermodynamic diagram by taking lambda as a unit until the last gate is reached, and then generating a bottom diagram to form a foreground thermodynamic diagram;

filling the thermal values of the corresponding squares in the layered thermodynamic diagram into a three-dimensional thermodynamic diagram matrix Channel-SPACE_m[u][v][c]；

Calculating the opening selected from the sample according to the time of each number plate vehicle passing through the f-th gateStarting time t₀Time △ t used for f-th bayonet_ft₀＝t_f-t₀Then c is △ t_ft₀/λ；

Accordingly, the three-dimensional thermodynamic diagram R-SPACE of each number plate is obtained_m(ii) a At the same time, a positive sample P-SPACE is obtained_mAnd negative sample N-SPACE_m。

2. The illegal operating vehicle recognition method according to claim 1, characterized in that:

step S2 further includes the steps of:

s21: cleaning time disorder and repeated data in a vehicle running track database;

s22: selecting illegal operating vehicle types needing to be identified, extracting vehicle running track data of vehicles of corresponding types according to the identified illegal operating vehicle types, and acquiring a legal operating vehicle number plate set and a non-operating vehicle number plate set according to vehicle operating categories; and

s23: and carrying out classification exploratory analysis on the extracted vehicle running track data to remove serious distortion data.

3. The illegal operating vehicle recognition method according to claim 2, characterized in that:

in step S22, the vehicle travel track data for a specific time period is extracted to specifically identify the vehicle operated illegally for the selected time period.

4. The illegal operating vehicle recognition method according to claim 1, characterized in that:

step S1 also includes acquiring video camera data of law enforcement points to build a law enforcement video database;

the illegal operating vehicle identification method further comprises the following steps:

s5: and obtaining the number plate of the vehicle in illegal operation based on the identification result, and correspondingly calling the video evidence or the picture evidence in illegal operation from the law enforcement video database.

5. An illegal operation vehicle recognition system using the illegal operation vehicle recognition method according to claim 1, characterized in that:

the identification system comprises

The data acquisition module (12) is used for acquiring vehicle data to establish a vehicle number plate database, a vehicle running track database and a card port longitude and latitude database;

the data cleaning module (13) is used for cleaning the acquired vehicle data;

and the deep learning module (15) is used for carrying out deep learning on the basis of the cleaned vehicle running track database so as to establish an intelligent recognition model and recognizing all vehicles by using the intelligent recognition model.

6. A computer-readable storage medium for storing a computer program for performing illegal operating vehicle recognition, characterized in that: the computer program, when running on a computer, performs the steps of the illegal working vehicle identification method according to claim 1.

Images