CN113221791A

CN113221791A - Vehicle parking violation detection method and device, electronic equipment and storage medium

Info

Publication number: CN113221791A
Application number: CN202110558597.1A
Authority: CN
Inventors: 倪卿元; 孙晓凯; 陈文强; 朱谌轶; 李梦媛; 张驰; 周翔
Original assignee: China Telecom Corp Ltd
Current assignee: China Telecom Corp Ltd
Priority date: 2021-05-21
Filing date: 2021-05-21
Publication date: 2021-08-06

Abstract

The invention relates to the technical field of video processing, and provides a vehicle parking violation detection method and device, electronic equipment and a storage medium. The vehicle parking violation detection method comprises the following steps: obtaining a plurality of paths of video streams to be detected; the method comprises the steps that multiple paths of video streams are detected in a round-robin manner, and a target image which corresponds to each video stream in each round-robin period and contains a vehicle target is obtained; when the time span of a target image of a video stream is larger than a violation time threshold, clustering all target images of the video stream according to the position coordinates of the included vehicle targets to obtain at least one target image group; and comparing the similarity of each target image group, and judging that the corresponding target image group has the illegal vehicle target when the number of the similar target images in the same group is greater than the quantity threshold value. According to the invention, through the round-robin detection, the image grouping and the similarity comparison based on the position coordinate clustering, the accuracy rate of vehicle illegal parking detection is improved while the computing resources are saved, and cost reduction and efficiency improvement are realized.

Description

Vehicle parking violation detection method and device, electronic equipment and storage medium

Technical Field

The invention relates to the technical field of video processing, in particular to a vehicle parking violation detection method and device, electronic equipment and a storage medium.

Background

Illegal parking threatens road traffic, urban public management, even life and property of citizens and the like. Especially in the place with dense traffic flow, the illegal parking can seriously obstruct the traffic and influence the public safety.

With the rapid development of science and technology and the gradual improvement of video image processing technology, video monitoring is widely applied to vehicle illegal parking detection. At present, the process of detecting whether a vehicle is parked illegally is shown in fig. 1: s101, obtaining a vehicle detection result, specifically finding out all vehicles in a camera picture through a target detection algorithm; and tracking each vehicle through a target tracking algorithm, respectively calculating the stay time of each vehicle, and judging whether the vehicle is in violation of parking, wherein the method specifically comprises the following steps: s102, carrying out similarity comparison with the previous vehicle detection result; s103, judging whether the similarity exceeds a corresponding threshold value; if so, executing S104, accumulating the stay time of the current vehicle, and if not, returning to obtain the next vehicle detection result (namely, the current vehicle is not tracked any more); and (5) continuing to execute S105 after accumulating the stay time, judging whether the stay time exceeds a corresponding threshold value, if so, executing S106, giving an alarm, otherwise, continuing to acquire a next vehicle detection result, and repeating the process to track the current vehicle.

Therefore, in the process of tracking the vehicle by the target tracking algorithm, as long as the two detection results are dissimilar, the stay time is not accumulated any more, so that the current vehicle is lost; corresponding to the situation that the vehicle is temporarily shielded by other objects in an actual scene, for example, the target tracking algorithm can be caused to lose track of the current vehicle, so that the stay time of the current vehicle is recalculated, and the stay time is inaccurate to calculate; in addition, in order to guarantee the detection precision of the target tracking algorithm, real-time tracking calculation is needed, one path of algorithm can only support one path of video image, the calculation cost is large, and the resource consumption is high.

It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the invention and therefore may include information that does not constitute prior art that is already known to a person of ordinary skill in the art.

Disclosure of Invention

In view of this, the invention provides a vehicle parking violation detection method, a vehicle parking violation detection device, an electronic device and a storage medium, which can solve the problems that a target tracking algorithm is high in calculation overhead and inaccurate in calculation of stay time.

One aspect of the present invention provides a vehicle parking detection method, including: obtaining a plurality of paths of video streams to be detected; the method comprises the steps that multiple paths of video streams are detected in a round-robin manner, and a target image which corresponds to each video stream in each round-robin period and contains a vehicle target is obtained; when the time span of a target image of a video stream is larger than a violation time threshold, clustering all target images of the video stream according to the position coordinates of the included vehicle targets to obtain at least one target image group; and comparing the similarity of each target image group, and judging that the corresponding target image group has the illegal vehicle target when the number of the similar target images in the same group is greater than the quantity threshold value.

In some embodiments, when the polling detects multiple video streams, storing a timestamp of a target image of a corresponding video stream and a position coordinate of a vehicle target in a polling period as an element in a fixed-length queue of the corresponding video stream; the determination condition of the time span of the target image of a video stream being greater than the pause-time long threshold is: the fixed-length queue of the video stream is full.

In some embodiments, the vehicle parking violation detection method further comprises: after the fact that the corresponding target image group has the illegal vehicle target is judged, the full fixed-length queue is emptied; and if it is determined that no illegal vehicle target exists in each target image group, performing clustering on all target images of the video stream according to the position coordinates of the included vehicle targets according to the updated full fixed length queue along with the process of the round-robin cycle when the full fixed length queue is updated.

In some embodiments, the vehicle parking violation detection method further comprises: monitoring a fixed-length queue of each path of video stream; and when the time interval for storing the elements in the fixed-length queue exceeds a violation interval threshold, emptying the fixed-length queue.

In some embodiments, the fixed capacity LQ of the fixed-length queue satisfies:

wherein T is the time length of violation threshold, W is an expansion coefficient, P is the polling period,

is a ceiling sign.

In some embodiments, the obtaining a target image containing a vehicle target of a corresponding video stream in each round trip cycle includes: in a current round of patrol period, obtaining a current video segment of a corresponding current video stream; extracting frames of the current video segment, and screening out a detection result image with a vehicle target in a target picture area through target detection; and generating a target image containing the time stamp of the frame and the position coordinates of the vehicle target according to the detection result image.

In some embodiments, said clustering all target images of said video stream by location coordinates of included vehicle targets comprises: obtaining position coordinates of vehicle targets of all target images of the video stream; clustering the obtained position coordinates; and according to the position coordinate clustering result, grouping all target images of the video stream, generating a plurality of target images respectively comprising a plurality of vehicle targets when one target image comprises the plurality of vehicle targets, and respectively grouping the plurality of target images into corresponding target image groups.

In some embodiments, after obtaining the at least one target image group, the method further includes: screening out a target image group with the number of target images smaller than a preset threshold value L, wherein the preset threshold value L meets the following requirements:

wherein T is the time length of violation threshold, P is the polling cycle,

is a rounded-down symbol.

In some embodiments, the performing similarity comparison on each target image group includes: arranging the target images of each target image group into a time sequence according to the time stamps; and comparing the similarity of the first target image of each time sequence with the subsequent target image of the time sequence in sequence, and accumulating the number of the similar target images of which the comparison results exceed the similarity threshold.

In some embodiments, the quantity threshold satisfies: r ═ n × E; wherein, R is the number threshold, n is the number of the target images of the currently compared target image group, and E is an error coefficient.

In some embodiments, after determining that there is an illegal vehicle target in the corresponding target image group, the method further includes: and outputting an illegal parking detection result containing a picture screenshot and illegal parking time length of the illegal parking vehicle target according to the similar target image of the target image group, wherein the illegal parking time length is obtained by calculation according to the timestamp of the similar target image.

Yet another aspect of the present invention provides a vehicle parking violation detection apparatus, comprising: the video reading module is used for obtaining a plurality of paths of video streams to be detected; the round-robin detection module is used for round-robin detection of the plurality of paths of video streams to obtain a target image containing a vehicle target of the corresponding video stream in each round-robin period; the image clustering module is used for clustering all target images of a video stream according to the position coordinates of the vehicle targets contained in the target images when the time span of the target images of the video stream is greater than the time-length-of-violation threshold value to obtain at least one target image group; and the illegal parking judging module is used for comparing the similarity of each target image group and judging that the corresponding target image group has illegal vehicle targets when the number of similar target images in the same group is greater than the quantity threshold value.

Yet another aspect of the present invention provides an electronic device, comprising: a processor; a memory having executable instructions stored therein; wherein the executable instructions, when executed by the processor, implement any of the vehicle violation detection methods described above.

Yet another aspect of the present invention provides a computer-readable storage medium storing a program that when executed by a processor implements any of the above-described vehicle violation detection methods.

Compared with the prior art, the invention has the beneficial effects that:

through round-robin detection, one path of algorithm is realized to support multiple paths of video streams, real-time tracking calculation is not needed, calculation overhead is reduced, and calculation resources are saved; moving objects and non-moving objects in the target image are effectively separated through image grouping based on position coordinate clustering of the vehicle target; through carrying out the similarity to the picture of the group after clustering grouping and comparing, effectively solve the vehicle target and be sheltered from by the short time and the long inaccurate problem of calculation of dwell time that causes, promote the vehicle and break down the precision rate that detects, avoid lou examining and the false positive, realize the cost reduction and improve effect.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.

FIG. 1 illustrates a schematic flow chart of vehicle violation detection in the prior art;

FIG. 2 is a schematic diagram illustrating the steps of a vehicle parking violation detection method according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a scene for obtaining a target image of a current video stream in a current polling period according to an embodiment of the present invention;

FIG. 4 is a schematic diagram illustrating the steps of a vehicle violation detection method according to yet another embodiment of the present invention;

FIG. 5 is a block diagram of a vehicle violation detection device in accordance with an embodiment of the present invention;

FIG. 6 is a schematic diagram of an electronic device according to an embodiment of the invention;

fig. 7 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.

The drawings are merely schematic illustrations of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

In addition, the flow shown in the drawings is only an exemplary illustration, and not necessarily includes all the steps. For example, some steps may be divided, some steps may be combined or partially combined, and the actual execution sequence may be changed according to the actual situation. The use of "first," "second," and similar terms in the detailed description is not intended to imply any order, quantity, or importance, but rather is used to distinguish one element from another. It should be noted that features of the embodiments of the invention and of the different embodiments may be combined with each other without conflict.

Fig. 2 shows the main steps of the vehicle parking detection method in one embodiment, and referring to fig. 2, the vehicle parking detection method includes: step S210, obtaining a plurality of paths of video streams to be detected; step S220, detecting multiple paths of video streams in a round-robin manner, and obtaining a target image which corresponds to the video streams and contains a vehicle target in each round-robin period; step S230, when the time span of the target image of a video stream is larger than a time-length-of-violation threshold, clustering all the target images of the video stream according to the position coordinates of the included vehicle targets to obtain at least one target image group; and step S240, comparing the similarity of each target image group, and judging that the corresponding target image group has an illegal vehicle target when the number of similar target images in the same group is greater than a quantity threshold value.

One path of video stream is shot by one camera, the camera can upload the shot video stream to a server in real time or at intervals, and the server can be a physical server or a cloud server. The multiple paths of video streams to be detected may belong to the same area, for example, video streams captured by cameras at three different positions arranged on a main road of a park, or may belong to different areas, for example, video streams captured by cameras at five different positions arranged on two different main roads of the park. In addition, the violation detection can be performed only on the multiple video streams within a certain period of historical time, or the violation detection can be performed by continuously reading the multiple video streams continuously received in the server without real-time reading.

Therefore, the vehicle illegal parking detection method can be suitable for any places needing illegal parking detection, real-time tracking calculation is not needed, one-way algorithm is supported for multiple paths of video streams through round-trip detection, calculation overhead is reduced, and calculation resources are saved. Particularly, in some places with intensive traffic flows, such as main roads of communities, parks and the like, moving objects and non-moving objects in target images can be effectively separated through image grouping based on position coordinate clustering of vehicle targets, and the problem that the stay time is inaccurate in calculation due to the fact that the vehicle targets are temporarily shielded often in the places with intensive traffic flows is effectively solved through similarity comparison of the images in groups after clustering and grouping. In some places with high requirements on installation of the internet of things equipment and high actual deployment difficulty, the vehicle illegal parking detection method is adopted, the original camera terminal is not required to be changed, the vehicle illegal parking detection can be carried out only by utilizing the video stream uploaded to the server by the camera, and the practicability is high. Even in some places with high network deployment difficulty, the camera terminal can return video streams to the cloud server through communication modes such as 5G and the like, and cloud network fusion is achieved.

The round-robin detection of the multiple video streams refers to sequentially detecting the video streams according to a set round-robin period. In one embodiment, a fixed-length queue is introduced, and the related information of the acquired target image is stored. Specifically, when a round-robin detection is performed on multiple video streams, a timestamp of a target image corresponding to the video stream and a position coordinate of a vehicle target in a round-robin period are stored as an element in a fixed-length queue corresponding to the video stream (that is, in a round-robin period, no matter how many pieces of object information are detected, only one element is stored in the corresponding fixed-length queue); accordingly, the determination condition that the time span of the target image of a video stream is greater than the violation duration threshold is: the fixed-length queue for the video stream is full.

The time stamp of the target image and the position coordinates of the vehicle target therein (hereinafter referred to as spatiotemporal information) can be obtained when the target image is acquired from the video stream, and the specific acquisition mode will be described below in conjunction with target detection. One path of video stream corresponds to a fixed-length queue, so that when the fixed-length queue is full, the corresponding video stream can acquire a target image with a proper time span for illegal parking judgment.

In this embodiment, five views are used for detectionFrequency stream (first video stream I)₁To a fifth video stream I₅) For example, the process of polling detection includes: in the 1 st round-robin period, a first video stream I is acquired₁The method comprises the steps of including a target image of a vehicle target, and taking space-time information of the acquired target image as an element i₁₁Into a first video stream I₁Queue of fixed length Q₁Performing the following steps; in the 2 nd round-robin period, the second path of video stream I is obtained₂The method comprises the steps of including a target image of a vehicle target, and taking space-time information of the acquired target image as an element i₂₂Into a second video stream I₂Queue of fixed length Q₂Performing the following steps; .., and so on, obtaining the target image of the corresponding video stream in each round-robin period, and storing the space-time information into the corresponding fixed-length queue.

When the first video stream I₁To a fifth video stream I₅After one round of inspection, the next round of inspection is continued, i.e., in the 6 th round of inspection period P₆Go back to the first video stream I₁Acquiring a target image including a vehicle target, and taking spatiotemporal information of the acquired target image as an element i₁₆Into its fixed-length queue Q₁Performing the following steps; .., repeating the process, getting the target image of the corresponding video stream and storing the space-time information into the corresponding fixed length queue. In the process, if a vehicle target does not exist in the corresponding video stream in a certain round of patrol period, the target image of the video stream cannot be acquired, and an element cannot be stored in the corresponding fixed-length queue.

Further, in the process of round-robin detection, a fixed-length queue of each path of video stream is also monitored; when the time interval for storing the elements in the fixed-length queue exceeds the violation interval threshold, the fact that the vehicle target does not exist in the corresponding video stream within a continuous period of sufficient length indicates that the violation condition is more unlikely to occur, and therefore the fixed-length queue is emptied to reduce the calculation amount of subsequent violation judgment. The violation interval threshold may be set as desired, for example, 5 minutes. In addition, the illegal parking time length threshold is a time length basis for judging the illegal parking of the vehicle target, and if the staying time length of a vehicle in a certain forbidden parking area is greater than the illegal parking time length threshold, the illegal parking is judged; the time-to-break threshold may also be set as desired or in accordance with the traffic regulations in the relevant place, for example 10 minutes.

In one embodiment, the fixed capacity LQ of the fixed-length queue specifically satisfies:

wherein T is the time length value of the time length threshold of violation, W is the expansion coefficient, P is the time length value of the same unit (in minutes in the embodiment) of the polling period and the time length threshold of violation,

is a ceiling sign. For example, the time duration of the violation threshold is 10 minutes (then T ═ 10); the expansion coefficient is 1.2 (then W is 1.2), the expansion coefficient is introduced for guaranteeing the accuracy of the algorithm, the threshold value of the pause time length is properly lengthened, and the value can be a proper value which is slightly larger than 1; the polling period is 30s (i.e., 0.5 min, P ═ 0.5); thus, the fixed capacity LQ, i.e., the length of the fixed-length queue, is 10 × 1.2/0.5 — 24. In other embodiments, the above values can be set according to requirements, and are not limited to those listed herein.

With the progress of the round-robin period, when a certain fixed-length queue is full, which indicates that the video stream corresponding to the fixed-length queue has acquired a sufficient number of target images suitable for performing the parking violation determination, all elements in the fixed-length queue are read, and the parking violation determination is performed in combination with all the target images of the corresponding video stream. The violation determination specifically adopts a method of combining position coordinate clustering and image similarity comparison, which will be described in detail below.

If the fixed-length queues are full at the same time, a multithreading parallel processing mode can be adopted to respectively carry out the violation judgment on the video streams.

In one embodiment, according to the result of the illegal parking determination of a video stream, if it is determined that one or more target image groups of the video stream have illegal vehicle targets, the full fixed-length queue of the video stream is emptied, so as to save memory space and reduce the calculation amount of the illegal parking determination of the video stream at the next time; subsequently, as the polling period progresses, spatiotemporal information of the target image obtained from the video stream is continuously stored in the corresponding polling period.

In one embodiment, according to the result of the illegal parking determination of a video stream, if it is determined that no illegal vehicle target exists in each target image group of the video stream, the process of round-robin cycle is performed, and when the full fixed-length queue of the video stream is updated, the step of clustering all target images of the video stream (all target images are target images corresponding to all elements in the updated full fixed-length queue) according to the position coordinates of the included vehicle targets is performed according to the updated full fixed-length queue.

By utilizing the principle that after a fixed-length queue is full, an old element stored firstly is extruded out every time a new element is stored, if an illegal vehicle target is not identified in illegal parking judgment, all elements in the corresponding full fixed-length queue are reserved, and illegal parking judgment is continuously carried out when the full fixed-length queue is updated next time, so that the accuracy of illegal parking judgment is ensured through the continuity of time.

In one embodiment, a target detection algorithm is employed to achieve the acquisition of a target image from a video stream. Specifically, obtaining a target image containing a vehicle target for a corresponding video stream in each round robin cycle includes: in a current round of patrol period, obtaining a current video segment of a corresponding current video stream; performing frame extraction on the current video segment, and screening out a detection result image with a vehicle target in a target picture area through target detection; a target image including the time stamp of the frame and the position coordinates of the vehicle target is generated from the detection result image.

Fig. 3 shows a scene of obtaining a target image of a current video stream in a current round trip cycle in an embodiment, and referring to fig. 3, a current video segment 310 of the current video stream corresponding to the current round trip cycle is obtained, and frame extraction detection is performed on the current video segment 310, that is, an extracted image is sent to a target detection algorithm, so as to detect whether a vehicle target exists in a target picture area of the image. The target detection algorithm is trained in advance, and detection frame and corner point coordinates of all vehicle targets in the target picture area can be output according to the input image. The specific principle of the target detection algorithm is the prior art, and therefore, the description is not provided.

Taking fig. 3 as an example, an image 320 (from the perspective of an actual scene, a main road area of the image 320 indicated by a black bold dashed line, including a vehicle indicated by a black bold frame) is extracted from the current video segment 310, a target picture area (i.e., the main road area) of the image 320 is identified to have a vehicle target (i.e., the vehicle) through target detection, and a detection result image is output, where the detection result image is actually a detection frame corresponding to the vehicle target 320' and corner coordinates of the detection frame, as shown in the figure, coordinates (x) of an upper left corner point are indicated₁,y₁) And coordinates of lower right corner point (x)₂,y₂). Further, according to the detection result image, a target image 330 is generated, which includes the timestamp of the frame corresponding to the image 320 and the position coordinates of the vehicle target 320 '(the position coordinates (x, y) of the center point of the vehicle target 320' can be calculated according to the corner point coordinates of the detection frame, which is convenient for subsequent position coordinate clustering).

And continuing to perform frame extraction detection, wherein the vehicle target is not detected in the target picture area of the image 350, and a detection result image is not generated. Two vehicle targets are detected in the target picture area of the image 360, and a detection frame and corner coordinates (not specifically labeled) of the detection frame are generated, which correspond to the two vehicle targets 360' and 360 ", respectively, so as to generate a corresponding target image 370.

In one embodiment, after acquiring a suitable number of target images of a video stream, clustering all target images of the video stream according to the position coordinates of included vehicle targets includes: obtaining position coordinates of vehicle targets of all target images of the video stream (in combination with the embodiment of the fixed-length queue, that is, all elements are taken out from the full fixed-length queue of the video stream, and timestamps in the fixed-length queue correspond to the position coordinates one to one, for example, the timestamps can be stored in a key value mode); clustering the obtained position coordinates; and according to the position coordinate clustering result, grouping all target images of the video stream, generating a plurality of target images respectively comprising a plurality of vehicle targets when one target image comprises the plurality of vehicle targets, and respectively grouping the plurality of target images into corresponding target image groups.

For example, as shown in fig. 3, when two vehicle targets 360 ' and 360 ″ are included in the target image 370, two target images are generated according to the target image 370 (for example, a screen region corresponding to the vehicle targets 360 '/360 ″ may be cut from the target image 370), one target image includes the vehicle targets 360 ', and the other target image includes the vehicle targets 360 ″, and the two generated target images are grouped into corresponding target image groups according to the position coordinate clustering result.

In addition, when the target images are grouped, all the target images can be intercepted, namely, the picture areas corresponding to the vehicle targets are intercepted, and the intercepted picture images are grouped, so that the subsequent picture similarity comparison is facilitated.

Alternatively, in one embodiment, when the target image is generated according to the detection result image, the screen region corresponding to the vehicle target may be cut out from the original image as the target image.

Further, in an embodiment, after at least one target image group is obtained through image grouping clustered based on the position coordinates of the vehicle target, the method further includes: screening out target image groups with the number of target images smaller than a preset threshold value L, wherein the preset threshold value L specifically meets the following requirements:

the above-mentioned embodiments can be referred to for the fetch manner of the time-length-of-violation threshold T and the polling period P,

is a rounded-down symbol.

If the number of the pictures in the target image group is insufficient, it is indicated that no long-time static object exists in the group, namely no illegal vehicle target exists. Therefore, the target image groups with insufficient number of pictures in the groups are filtered by screening the target image groups subjected to clustering grouping, the calculated amount of subsequent similarity comparison is reduced, and the process of judging violation is accelerated.

For the reserved target image group, performing intra-group image similarity comparison, specifically comprising: arranging the target images of each target image group into a time sequence according to the time stamps; and comparing the similarity of the first target image of each time sequence with the subsequent target image of the time sequence in sequence, and accumulating the number of the similar target images of which the comparison results exceed the similarity threshold.

For example, a certain target image group includes 10 target images (by way of example only, not as a limitation on the number of target images in the group), after the target images are arranged in a time sequence according to timestamps, the first target image is sequentially compared with the second target image, the third target image, the ninth target image and the tenth target image for the similarity of the vehicle target, if the comparison result exceeds the similarity threshold, the number of similar target images is accumulated, and finally, the number of similar target images is compared with the number threshold, so as to determine whether the target image group has an illegal vehicle target.

Through comparison of similarity of pictures in the group, under the condition that the moving object and the static object are divided into one group (corresponding to an actual scene, for example, a certain illegal parking vehicle is temporarily blocked by a passing vehicle), the influence caused by temporary blocking of the moving object is ignored, and the illegal parking vehicle target can be accurately identified. If the number of similar target images in the group exceeds the number threshold R, it indicates that there is an illegal vehicle in the group. The number threshold R specifically satisfies: r ═ n × E; wherein n is the number of target images in the currently compared target image group, E is an error coefficient, the value range of E is [0,1], and the identification result is more accurate when the value is larger.

The image similarity comparison specifically adopts the existing method, can compare the similarity of the given images, and returns the similarity degree value, and the description is not expanded here.

Further, in one embodiment, after determining that there is an illegal vehicle target in the corresponding target image group, the method further includes: and outputting an illegal parking detection result containing the picture screenshot and illegal parking time length of the illegal parking vehicle target according to the similar target image of the target image group, wherein the illegal parking time length is obtained by calculation according to the timestamp of the similar target image. Therefore, the illegal parking time length of the vehicle is further acquired on the basis of judging that the vehicle is illegal.

Fig. 4 shows a step flow of a vehicle parking violation detection method in a preferred embodiment, and referring to fig. 4, the process of performing parking violation detection on a vehicle in this embodiment specifically includes, in combination with the image polling, target detection, spatial position clustering, and image similarity comparison described in the foregoing embodiment:

in the first step S410, a frame round: the picture configuration of all the video streams needing to detect the violation is read in advance (for example, the video streams of the embodiment having I cameras in total need to be detected), and then one video stream is read according to the polling period and the video stream sequence.

Second step S420, target detection: performing frame extraction detection on the video stream, generating a target image according to a target detection result, and storing corresponding space-time information into a fixed-length queue of the video stream; and when the reading operation of one video stream is finished, storing the space-time information in the detection result into the corresponding fixed-length queue, and continuing to read the next video stream.

The third step S430: judging whether a fixed-length queue is full; if not, continuing to read the next video stream along with the round of patrol; if yes, reading all elements in the full fixed-length queue, performing subsequent violation judgment, and meanwhile continuing the polling reading of each path of video stream, before the next reading operation of the video stream corresponding to the full fixed-length queue, usually completing the violation judgment of the video stream, and if the next reading operation of the video stream can not be completed in the polling of the picture, suspending waiting for the violation judgment to be completed, and then continuing the reading operation of the video stream.

In fig. 4, the continuation of the screen round is shown by a dotted arrow.

Fourth step S440, spatial position clustering: the method comprises the steps of firstly clustering position coordinates of vehicle targets by combining elements in a full fixed-length queue and corresponding target images, and then grouping the target images according to position coordinate clustering results to obtain target image groups corresponding to different picture areas of a video stream, so that a static object and a moving object in a shot picture of the video stream can be effectively separated; and after clustering and grouping, counting the number of target images in each group, if the number of the target images in each group does not reach a preset threshold value, indicating that no long-time static object exists in the group, filtering, and keeping the target image group with the number of the target images in the group exceeding the preset threshold value.

Fifth step S450, image similarity comparison: and comparing the similarity of the target images in the group, judging whether the number of the similar target images in the group exceeds a quantity threshold value, if so, outputting an alarm, and thus, neglecting the influence caused by temporary blocking of the moving object under the condition that the moving object and the static object are possibly divided into one group, and accurately identifying the vehicle which is illegally parked.

It should be noted that fig. 4 only shows a brief flow of image polling, target detection, spatial position clustering, and image similarity comparison in the vehicle parking violation detection process, and specific methods and principles can refer to the description of the above embodiments, which is not repeated here.

In summary, compared with the problems that real-time calculation is needed, one path of algorithm can only support one path of video image, and vehicle target loss and inaccurate residence time calculation are caused by shielding due to the adoption of a target tracking algorithm in the prior art, the vehicle illegal parking detection method combining image polling, target detection, spatial position clustering and image similarity comparison is adopted in the invention, and the following advantages and effects are achieved:

the method based on spatial position coordinate clustering and picture similarity comparison is adopted to replace tracking, so that a static object and a moving object in a video picture can be well separated, the problem of inaccurate calculation of stay time caused by the fact that a vehicle target is blocked is effectively solved, the detection accuracy is improved, the false alarm rate and the missed detection rate are reduced, and the method can be applied to any intensive places of traffic flow, wherein the detection accuracy is specifically the result proportion of the blocking condition in the detected illegal parking result; meanwhile, because a tracking algorithm is not used, real-time detection is not needed, the calculation cost is low, and the frame polling can realize polling detection of a plurality of video stream frames, so that the method plays a supporting role of one-path algorithm on a plurality of paths of video streams, reduces cost and improves efficiency; in addition, the vehicle illegal parking detection method is strong in robustness, applicable to various scenes, less interfered by weather factors and capable of realizing accurate illegal parking detection.

The embodiment of the invention also provides a vehicle illegal parking detection device which can be used for realizing the vehicle illegal parking detection method described in any embodiment. The features and principles of the vehicle violation detection method described in any of the above embodiments can be applied to the following vehicle violation detection device embodiments. In the following embodiments of the vehicle violation detection device, the features and principles already set forth regarding the detection of vehicle violation are not repeated.

Fig. 5 shows the main blocks of the vehicle parking violation detecting device in one embodiment, and referring to fig. 5, the vehicle parking violation detecting device 500 includes: a video reading module 510, configured to obtain a multi-channel video stream to be detected; a round-robin detection module 520, configured to perform round-robin detection on multiple video streams to obtain a target image including a vehicle target of each round-robin period corresponding to a video stream; an image clustering module 530, configured to cluster all target images of a video stream according to position coordinates of included vehicle targets to obtain at least one target image group when a time span of the target images of the video stream is greater than a violation duration threshold; and the parking violation judging module 540 is configured to compare similarity of each target image group, and determine that there is a vehicle target in the corresponding target image group that violates when the number of similar target images in the same group is greater than the quantity threshold.

Further, the vehicle illegal parking detection device 500 may further include modules for implementing other processes of the above-described embodiments of the vehicle illegal parking detection method, for example, modules for implementing processes of various steps of the embodiment of the vehicle illegal parking detection method shown in fig. 4, and specific principles of the various modules may refer to the description of the above-described embodiments of the vehicle illegal parking detection method, and will not be repeated here.

As described above, the vehicle illegal parking detection device can realize that one path of algorithm supports multiple paths of video streams through wheel patrol detection, does not need real-time tracking calculation, and reduces calculation overhead; moving objects and non-moving objects in the target image are effectively separated through image grouping based on position coordinate clustering of the vehicle target; through carrying out the similarity to the picture in group after grouping the cluster and comparing, effectively solve the vehicle target and be sheltered from by the short time and cause the inaccurate problem of dwell time calculation, promote the vehicle and break the accurate rate that detects, realize cost reduction and benefit.

The embodiment of the present invention further provides an electronic device, which includes a processor and a memory, where the memory stores executable instructions, and when the executable instructions are executed by the processor, the vehicle parking violation detection method described in any of the above embodiments is implemented.

As described above, the electronic device of the present invention can realize that one path of algorithm supports multiple paths of video streams through polling detection, without real-time tracking calculation, thereby reducing calculation overhead; moving objects and non-moving objects in the target image are effectively separated through image grouping based on position coordinate clustering of the vehicle target; through carrying out the similarity to the picture in group after grouping the cluster and comparing, effectively solve the vehicle target and be sheltered from by the short time and cause the inaccurate problem of dwell time calculation, promote the vehicle and break the accurate rate that detects, realize cost reduction and benefit.

Fig. 6 is a schematic structural diagram of an electronic device in an embodiment of the present invention, and it should be understood that fig. 6 only schematically illustrates various modules, and these modules may be virtual software modules or actual hardware modules, and the combination, the splitting, and the addition of the remaining modules of these modules are within the scope of the present invention.

As shown in fig. 6, the electronic device 600 is embodied in the form of a general purpose computing device. The components of the electronic device 600 include, but are not limited to: at least one processing unit 610, at least one memory unit 620, a bus 630 connecting the different platform components (including the memory unit 620 and the processing unit 610), a display unit 640, etc.

Wherein the storage unit stores program code executable by the processing unit 610 to cause the processing unit 610 to perform the steps of the vehicle violation detection method described in any of the embodiments above. For example, processing unit 610 may perform the steps shown in fig. 2 and 4.

The storage unit 620 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)6201 and/or a cache memory unit 6202, and may further include a read-only memory unit (ROM) 6203.

The memory unit 620 may also include programs/utilities 6204 including one or more program modules 6205, such program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

Bus 630 may be one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

The electronic device 600 may also communicate with one or more external devices 700, and the external devices 700 may be one or more of a keyboard, a pointing device, a bluetooth device, and the like. The external devices 700 enable a user to interactively communicate with the electronic device 600. The electronic device 600 may also be capable of communicating with one or more other computing devices, including routers, modems. Such communication may occur via an input/output (I/O) interface 650. Also, the electronic device 600 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via the bus 630. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, to name a few.

Embodiments of the present invention further provide a computer-readable storage medium for storing a program, and the program, when executed, implements the vehicle violation detection method described in any of the above embodiments. In some possible embodiments, the various aspects of the invention may also be implemented in the form of a program product comprising program code for causing a terminal device to perform the vehicle violation detection method described in any of the embodiments above, when the program product is run on the terminal device.

As described above, the computer-readable storage medium of the present invention can implement a path algorithm to support multiple paths of video streams through polling detection, without real-time tracking calculation, thereby reducing calculation overhead; moving objects and non-moving objects in the target image are effectively separated through image grouping based on position coordinate clustering of the vehicle target; through carrying out the similarity to the picture in group after grouping the cluster and comparing, effectively solve the vehicle target and be sheltered from by the short time and cause the inaccurate problem of dwell time calculation, promote the vehicle and break the accurate rate that detects, realize cost reduction and benefit.

Fig. 7 is a schematic structural diagram of a computer-readable storage medium of the present invention. Referring to fig. 7, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of readable storage media include, but are not limited to: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

A computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device, such as through the internet using an internet service provider.

The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all shall be considered as belonging to the protection scope of the invention.

Claims

1. A vehicle parking violation detection method, comprising:

obtaining a plurality of paths of video streams to be detected;

the method comprises the steps that multiple paths of video streams are detected in a round-robin manner, and a target image which corresponds to each video stream in each round-robin period and contains a vehicle target is obtained;

when the time span of a target image of a video stream is larger than a violation time threshold, clustering all target images of the video stream according to the position coordinates of the included vehicle targets to obtain at least one target image group;

and comparing the similarity of each target image group, and judging that the corresponding target image group has the illegal vehicle target when the number of the similar target images in the same group is greater than the quantity threshold value.

2. The vehicle parking violation detecting method according to claim 1, wherein when the round-robin detects multiple video streams, a timestamp of a target image corresponding to a video stream in a round-robin period and a position coordinate of a vehicle target are stored as an element in a fixed-length queue corresponding to the video stream;

the determination condition of the time span of the target image of a video stream being greater than the pause-time long threshold is: the fixed-length queue of the video stream is full.

3. The vehicle parking detection method as recited in claim 2, further comprising:

after the fact that the corresponding target image group has the illegal vehicle target is judged, the full fixed-length queue is emptied; and

and if it is judged that no illegal vehicle target exists in each target image group, the step of clustering all target images of the video stream according to the position coordinates of the included vehicle targets is executed according to the updated full fixed length queue along with the process of the round-robin cycle when the full fixed length queue is updated.

4. The vehicle parking detection method as recited in claim 2, further comprising:

monitoring a fixed-length queue of each path of video stream;

and when the time interval for storing the elements in the fixed-length queue exceeds a violation interval threshold, emptying the fixed-length queue.

5. The vehicle of claim 2The vehicle parking violation detection method is characterized in that the fixed capacity LQ of the fixed-length queue satisfies the following conditions:

is a ceiling sign.

6. The vehicle parking violation detection method of claim 1, wherein said obtaining a target image containing a vehicle target for a corresponding video stream in each round trip cycle comprises:

in a current round of patrol period, obtaining a current video segment of a corresponding current video stream;

extracting frames of the current video segment, and screening out a detection result image with a vehicle target in a target picture area through target detection;

and generating a target image containing the time stamp of the frame and the position coordinates of the vehicle target according to the detection result image.

7. The vehicle parking violation detection method of claim 1, wherein said clustering all target images of said video stream by location coordinates of included vehicle targets comprises:

obtaining position coordinates of vehicle targets of all target images of the video stream;

clustering the obtained position coordinates;

and according to the position coordinate clustering result, grouping all target images of the video stream, generating a plurality of target images respectively comprising a plurality of vehicle targets when one target image comprises the plurality of vehicle targets, and respectively grouping the plurality of target images into corresponding target image groups.

8. The vehicle parking violation detection method of claim 1, wherein after obtaining at least one target image group, further comprising:

screening out a target image group with the number of target images smaller than a preset threshold value L, wherein the preset threshold value L meets the following requirements:

wherein T is the time length of violation threshold, P is the polling cycle,

is a rounded-down symbol.

9. The vehicle parking violation detection method of claim 1, wherein said comparing the similarity of each target image group comprises:

arranging the target images of each target image group into a time sequence according to the time stamps;

and comparing the similarity of the first target image of each time sequence with the subsequent target image of the time sequence in sequence, and accumulating the number of the similar target images of which the comparison results exceed the similarity threshold.

10. The vehicle parking detection method as recited in claim 1, wherein the quantity threshold satisfies: r ═ n × E;

wherein, R is the number threshold, n is the number of the target images of the currently compared target image group, and E is an error coefficient.

11. The vehicle parking violation detection method of claim 1, wherein after determining that there is a parked vehicle target in the corresponding target image group, further comprising:

and outputting an illegal parking detection result containing a picture screenshot and illegal parking time length of the illegal parking vehicle target according to the similar target image of the target image group, wherein the illegal parking time length is obtained by calculation according to the timestamp of the similar target image.

12. A vehicle parking violation detection device, comprising:

the video reading module is used for obtaining a plurality of paths of video streams to be detected;

the round-robin detection module is used for round-robin detection of the plurality of paths of video streams to obtain a target image containing a vehicle target of the corresponding video stream in each round-robin period;

the image clustering module is used for clustering all target images of a video stream according to the position coordinates of the vehicle targets contained in the target images when the time span of the target images of the video stream is greater than the time-length-of-violation threshold value to obtain at least one target image group;

and the illegal parking judging module is used for comparing the similarity of each target image group and judging that the corresponding target image group has illegal vehicle targets when the number of similar target images in the same group is greater than the quantity threshold value.

13. An electronic device, comprising:

a processor;

a memory having executable instructions stored therein;

wherein the executable instructions, when executed by the processor, implement the vehicle violation detection method of any of claims 1-11.

14. A computer-readable storage medium storing a program, wherein the program, when executed by a processor, implements the vehicle violation detection method according to any of claims 1-11.