CN115331151A

CN115331151A - Video speed measuring method and device, electronic equipment and storage medium

Info

Publication number: CN115331151A
Application number: CN202211042370.2A
Authority: CN
Inventors: 张松林; 严雪飞; 程亮
Original assignee: Shanghai Fuya Intelligent Technology Co ltd
Current assignee: Shanghai Fuya Intelligent Technology Co ltd
Priority date: 2022-08-29
Filing date: 2022-08-29
Publication date: 2022-11-11

Abstract

The embodiment of the invention discloses a video speed measuring method and device, electronic equipment and a storage medium. The method comprises the following steps: the method comprises the steps of obtaining a first image and a second image of a speed measuring time period, determining vehicle position information corresponding to a target vehicle in the first image and the second image respectively, obtaining key points corresponding to the first image and the second image respectively, determining matched background key points according to matching results of the key points, and determining the movement speed of the target vehicle based on imaging parameter information, position information of preset collecting equipment, the background key points and the vehicle position information. According to the embodiment of the invention, the speed of the vehicle is determined based on the imaging parameter information, the position information of the preset acquisition equipment, the background key points and the vehicle position information by combining the background key points matched in the first image and the second image and the vehicle positions respectively corresponding to the vehicle in the first image and the second image, so that the problems of large deviation and insufficient precision in vehicle speed measurement can be solved, and the accuracy of vehicle speed measurement is improved.

Description

Video speed measuring method and device, electronic equipment and storage medium

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to a video speed measurement method and apparatus, an electronic device, and a storage medium.

Background

In the speed detection of a moving target, the traditional methods such as laser speed measurement, radar speed measurement, induction devices and the like have the defects of complex installation procedures, high maintenance cost, high requirement on measurement environment, large measurement error and the like; with the rapid development of computer vision algorithms, video speed measurement based on target tracking becomes a hotspot researched in recent years and is mature to be applied in some scenes, and the existing video speed measurement technology has many defects, one is that imaging equipment can cause great influence on the effect of the traditional target tracking algorithm under the conditions of overall frame deviation, video frame skipping and the like caused by signal transmission influence, such as target loss or tracking of wrong targets and the like, and the target speed derived by the target tracking algorithm can generate great deviation, loss or serious error at the moment, so that great difficulty is caused in the scene of speed measurement by the traditional video technical means; the other is that when the view angle is in a moving state (such as being installed on an unmanned aerial vehicle), the target speed derived by the target tracking algorithm and the moving speed of the view angle per se need to be comprehensively considered, and then the relatively accurate target actual speed can be calculated, so that the scene task is effectively completed (such as judging whether the vehicle speed exceeds a specified range or not in a traffic control scene), however, because the accuracy of a sensor related to the speed of the device per se is insufficient or the deviation is large, and the imaging device shakes and other objective reasons caused by the device in the moving state, the deviation between the real speed of the moving view angle and the current speed output by the device is large, and further the calculated and obtained target real speed has large deviation, so that the scene task effect is poor.

Disclosure of Invention

In view of this, the invention provides a video speed measurement method and apparatus, an electronic device, and a storage medium, which can perform speed measurement of a target vehicle in a video by combining background key points and vehicle position information, solve the problems of large deviation and insufficient precision in speed measurement of the target vehicle, and improve the precision of speed measurement of the target vehicle.

According to an aspect of the present invention, an embodiment of the present invention provides a video speed measurement method, including:

acquiring a first image and a second image in a speed measuring time period;

determining vehicle position information respectively corresponding to the target vehicle in the first image and the second image;

acquiring a first key point corresponding to a first image and a second key point corresponding to a second image, and determining a matched background key point in a target area according to a matching result of the first key point and the second key point;

determining the movement speed of the target vehicle based on imaging parameter information of preset acquisition equipment, position information of the preset acquisition equipment, the background key point and the vehicle position information

According to another aspect of the present invention, an embodiment of the present invention further provides a video speed measuring device, where the device includes:

the image acquisition module is used for acquiring a first image and a second image within a speed measurement time period;

the position determining module is used for determining vehicle position information corresponding to the target vehicle in the first image and the second image respectively;

the key point determining module is used for acquiring a first key point corresponding to the first image and a second key point corresponding to the second image, and determining a matched background key point in the target area according to a matching result of the first key point and the second key point;

the speed determining module is used for determining the movement speed of the target vehicle based on imaging parameter information of preset acquisition equipment, position information of the preset acquisition equipment, the background key point and the vehicle position information.

According to another aspect of the present invention, an embodiment of the present invention further provides an electronic device, including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,

the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the video velocimetry method according to any of the embodiments of the present invention.

According to another aspect of the present invention, a computer-readable storage medium is further provided, where the computer-readable storage medium stores computer instructions, and the computer instructions are configured to enable a processor to implement the video speed measuring method according to any embodiment of the present invention when the processor executes the computer instructions.

According to the technical scheme of the embodiment of the invention, the matched background key points in the target area are determined according to the matching results of the first key points corresponding to the first image and the second key points corresponding to the second image, the optimally matched background key points can be found, the matching accuracy is improved, the movement speed of the target vehicle is determined according to the imaging parameter information of the preset acquisition equipment, the position information of the preset acquisition equipment, the background key points and the vehicle position information by determining the vehicle position information corresponding to the target vehicle in the first image and the second image respectively, the speed measurement of the target vehicle in the video can be comprehensively carried out according to the background key points and the vehicle position information, the problems of large speed measurement deviation and insufficient accuracy of the target vehicle are solved, and the speed measurement accuracy of the target vehicle is improved.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present invention, nor do they necessarily limit the scope of the invention. Other features of the present invention will become apparent from the following description.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a flowchart of a video speed measuring method according to an embodiment of the present invention;

fig. 2 is a flowchart of another video speed measurement method according to an embodiment of the present invention;

fig. 3 is a flowchart of another video speed measurement method according to an embodiment of the present invention;

fig. 4 is a block diagram of a video speed measuring device according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

In order to make those skilled in the art better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

In an embodiment, fig. 1 is a flowchart of a video speed measuring method according to an embodiment of the present invention, where this embodiment is applicable to a situation where a video speed is measured by combining a visual moving speed and a moving speed of a target vehicle, and the method can be executed by a video speed measuring device, where the video speed measuring device can be implemented in a form of hardware and/or software, and the video speed measuring device can be configured in an electronic device.

S110, acquiring a first image and a second image in a speed measuring time period.

The speed measurement time period refers to a range of start time and end time of speed measurement in the video stream. The speed measuring time period can be set by a user according to personal requirements, and any two time periods can be selected from the video stream according to time intervals to serve as the speed measuring time period. The first image may be understood as a first frame image acquired during the velocimetry period. The second image can be understood as a second frame image acquired in the speed measuring time period, and can be understood as determining the speed measuring time period of the first image and the second image, that is, determining the first image and the second image. In some embodiments, the first image is a first frame image corresponding to a start time in a speed measurement time period in the video stream; the second image is a second frame image corresponding to the end time in the speed measuring time period.

In this embodiment, a first frame image and a last frame image corresponding to the speed measurement start time and the speed measurement end time, respectively, may be acquired in the video stream. Specifically, the collected images or videos can be transmitted in real time in a real-time collection mode through a video or image collection device, so that the rear end can obtain a first frame image and a last frame image in a speed measurement time period in real time; the first frame image and the last frame image in the corresponding speed measurement time may also be selected for a certain time period of the video by acquiring a previously acquired video, and this embodiment is not limited herein.

And S120, determining vehicle position information corresponding to the target vehicle in the first image and the second image respectively.

The target vehicle refers to a vehicle which measures speed in a speed measuring period. The vehicle position information may be understood as the relevant coordinate information of the vehicle, including the relevant coordinate information of the target vehicle in the first image, and the relevant coordinate information of the target vehicle in the second image.

In this embodiment, vehicle position information corresponding to the target vehicle in the first image and the second image respectively may be determined in a variety of ways, in some embodiments, the target vehicle may be detected in the first image and the second image through a target detection neural network model, target detection frames corresponding to the target vehicle respectively are determined according to detection results of the target vehicle in the first image and the second image, and vehicle position information corresponding to the target vehicle in the first image and the second image respectively is determined through the obtained target detection frames; in other embodiments, the vehicle may also be detected through the point cloud data corresponding to the target vehicle in the first image and the second image, and then the point cloud data corresponding to the first image is compared with the point cloud data corresponding to the second image to determine the vehicle position information corresponding to the target vehicle in the first image and the second image, respectively.

S130, acquiring a first key point corresponding to the first image and a second key point corresponding to the second image, and determining a matched background key point in the target area according to a matching result of the first key point and the second key point.

Wherein the first keypoint refers to a keypoint of the related background in the first image. The second keypoints refer to keypoints of the relevant background in the second image. The number of the first key points and the second key points is at least one. The first and second keypoints may include roads, plants, trees, wastelands, and the like in the corresponding image background. The matching result may be that the first keypoint matches the second keypoint, or that the first keypoint does not match the second keypoint. The target area refers to the target area selected in the first image and the second image, and may be an entire road area corresponding to the road area range, or an area of a certain lane corresponding to the road area range, which is not limited in this embodiment. The background key points refer to key points of an image background in the first image and key points of an image background in the second image, and the obtained matched background key points are key points of road areas in the image background respectively corresponding to the first image and the second image, or key points of trees in the image background respectively corresponding to the first image and the second image.

It should be noted that the first and second key points may include areas with relatively large changes in color, illumination, gray value, and the like in the background of the first and second images.

In this embodiment, the key points in the first image and the second image may be respectively detected through a related image feature algorithm to obtain pixel coordinates and feature descriptors corresponding to the key points, in some embodiments, a semantic segmentation neural network model is used to respectively segment target regions of the first image and the second image to determine corresponding segmentation results, the first key points and the second key points are respectively filtered according to the segmentation results and detection results of corresponding target vehicles in the first image and the second image by using a target detection neural network model to obtain first background key points and second background key points corresponding to the first image and the second image in the target regions, and then a Fast approximation neighbor algorithm Library (Fast Library for approximation neighbor Neighbors, flags) and a homography matching method are used to match the key points of the feature descriptors in the first background point and the feature descriptors in the second background point to obtain a key point matching in the first background image and the second background image; in other embodiments, the matching background key points may also be obtained by extracting image features in the first image, screening out key points that do not belong to the background from the image features, and then matching the image features in the second image with the image features in the first image by a threshold matching method, a nearest neighbor distance ratio matching method, or the like.

S140, determining the movement speed of the target vehicle based on the imaging parameter information of the preset acquisition equipment, the position information of the preset acquisition equipment, the background key point and the vehicle position information.

The preset acquisition device may be understood as a device for acquiring images or videos, and may be a camera, a radar acquisition device, a video acquisition card, or the like. The imaging parameter information refers to imaging parameter information corresponding to a preset acquisition device, and may include a focal length, a shooting angle, a size of the preset acquisition device, a resolution at the time of shooting, and the like. The position information of the preset acquisition device can be understood as coordinate information of the preset acquisition device.

In this embodiment, first imaging parameter information corresponding to a preset acquisition device and first coordinate information corresponding to the preset acquisition device in a world coordinate system may be acquired at a speed measurement start time corresponding to a first image; and at the speed measurement ending time corresponding to the second image, acquiring second imaging parameter information corresponding to preset acquisition equipment, and second coordinate information corresponding to the preset acquisition equipment in a world coordinate system, so as to determine a first speed of the target vehicle in the world coordinate system through the first imaging parameter information, the second imaging parameter information, the first coordinate information, the second coordinate information, vehicle position information corresponding to the first image starting time and vehicle position information corresponding to the second image starting time, so as to determine a second speed of a background key point of the target area in the world coordinate system through the first imaging parameter information, the second imaging parameter information, the first coordinate information, the second coordinate information and position information corresponding to the background key point, and then determining the movement speed of the target vehicle in the world coordinate system according to the difference value of the first speed and the second speed.

According to the technical scheme, the background key points matched in the target area are determined according to the matching results of the first key points corresponding to the first image and the second key points corresponding to the second image, the optimally matched background key points can be found, the matching accuracy is improved, the vehicle position information corresponding to the target vehicle in the first image and the second image respectively is determined, the movement speed of the target vehicle is determined based on the imaging parameter information of the preset acquisition equipment, the position information of the preset acquisition equipment, the background key points and the vehicle position information, the speed of the target vehicle in the video can be comprehensively measured by combining the background key points and the vehicle position information, the problems of large deviation and insufficient accuracy in the speed measurement of the target vehicle are solved, and the speed measurement accuracy of the target vehicle is improved.

In an embodiment, fig. 2 is a flowchart of another video speed measurement method according to an embodiment of the present invention, and on the basis of the foregoing embodiments, the present embodiment further refines a movement speed of a target vehicle by acquiring a first key point corresponding to a first image and a second key point corresponding to a second image, determining a background key point matching in a target area according to a matching result of the first key point and the second key point, determining vehicle position information corresponding to the target vehicle in the first image and the second image, and determining the movement speed of the target vehicle based on imaging parameter information of a preset acquisition device, position information of the preset acquisition device, the background key point, and the vehicle position information.

As shown in fig. 2, the video speed measuring method in this embodiment may specifically include the following steps:

s210, acquiring a first image and a second image in a speed measuring time period.

S220, detecting target vehicles in the corresponding target areas in the first image and the second image respectively according to the target detection neural network model, wherein the first detection result corresponds to the first image, and the second detection result corresponds to the second image.

In this embodiment, a target vehicle is detected in a target area corresponding to a first image according to a target detection neural network model, so as to obtain a first detection result corresponding to the first image; and detecting the target vehicle in the target area corresponding to the second image according to the target detection neural network model to obtain a second detection result corresponding to the second image.

And S230, determining a first target detection frame corresponding to the target vehicle according to the first detection result of the target vehicle, and determining a second target detection frame corresponding to the target vehicle according to the second detection result of the target vehicle.

The first target detection frame refers to a detection frame corresponding to a target vehicle in the first map. The second target detection frame refers to a detection frame corresponding to the target vehicle in the second map.

In this embodiment, according to the detection results of the target vehicle in the first image and the second image by the target detection neural network model, the target detection frames corresponding to the target vehicle in the first image and the second image respectively can be determined. It should be noted that the first target detection frame and the second target detection frame may represent coordinate information of the target vehicle in the first image and the second image.

S240, determining first vehicle position information and second vehicle position information corresponding to the target vehicle in the first image and the second image according to the first target detection frame and the second target detection frame.

The first vehicle position information can be understood as vehicle coordinate information of the target vehicle in the first image, and the second vehicle position information can be understood as vehicle coordinate information of the target vehicle in the second image.

In this embodiment, through the target detection frame of the target vehicle in the first image and the target detection frame of the target vehicle in the second image, the vehicle coordinate information corresponding to the target vehicle in the first image and the second image, respectively, may be determined, and the vehicle coordinate information may be coordinate information in pixel coordinates.

And S250, calling a preset image feature algorithm to respectively determine a first key point in the first image and a second key point in the second image.

The preset image feature algorithm may be understood as a correlation algorithm for performing keypoint detection on the first image and the second image. The default image feature algorithm may be one of Scale-invariant feature transform (SIFT), speeded Up Robust Features algorithm (SURF), and feature detection algorithm (ordered FAST and ordered BRIEF, ORB). The SIFT may be used to detect and describe local features in the first image and the second image, search for key points (feature points) in different scale spaces, and calculate the direction of the key points. The key points searched by SIFT are some points which are quite prominent and can not be changed by factors such as illumination, affine transformation, noise and the like, such as angular points, edge points, bright points in a dark area, dark points in a bright area and the like; SURF is a robust image identification and description algorithm, and the SURF can maliciously use determinant values of Hesseian matrix as characteristic point detection and use an integral graph to accelerate operation; ORB can be applied for real-time feature detection. ORB feature detection has scale and rotation invariance, as well as invariance to noise and its perspective transformation.

In this embodiment, corresponding keypoint detection is performed on the acquired first image and second image through a preset image feature algorithm to determine corresponding keypoints in 2 images. It should be noted that each keypoint in the first image and the second image has a corresponding pixel coordinate at the pixel coordinate and a corresponding feature descriptor. The feature descriptor may be understood as feature description of a pixel value, a gray value, a color, illumination brightness, a gradient of a change of the gray value, and the like of a key point in an image. A feature descriptor can be understood as a vector of 1xN _dim _des; the value of N _ dim _ des depends on the keypoint extraction algorithm used. Exemplarily, SIFT corresponds to N _ dim _ des =128; when the envelope frame of the first frame image or the last frame image contains N _ pt feature points, the extracted feature description sub-array is a digital array of N _ pt x N _ dim _ des.

And S260, segmenting the target areas of the first image and the second image respectively according to the semantic segmentation neural network model, and determining the target areas corresponding to the first image and the second image respectively.

The semantic segmentation neural network model can be used for classifying each pixel point in the first image and each pixel point in the second image. Illustratively, the first image and the second image are scene images containing roads, plants and vehicles, and after the semantic segmentation neural network model is performed, images containing a plurality of colors can be obtained, wherein each color can be represented as a type of object, namely, the road is a type, the vehicle is a type, and the plants are a type.

In this embodiment, the target areas of the first image and the second image may be segmented by a semantic segmentation neural network model to determine the target areas corresponding to the first image and the second image, respectively, where the target areas are areas around the vehicle, and may be areas of a road.

S270, determining a first mask corresponding to the target area in the first image and a second mask corresponding to the target area in the second image.

In this embodiment, the first mask refers to a mask corresponding to a road region divided in the first image. The second mask refers to a mask corresponding to the road region divided in the second image. The mask may be understood as an 8-bit single-channel image (gray scale image/binary image), and 0 or 1 may be used to represent the mask corresponding to the target area. For example, in the road region after the division, a mask corresponding to the target region is represented by 1.

In this embodiment, a semantic segmentation neural network model is used to determine target regions corresponding to the first image and the second image, and then the target regions of the first image and the second image are segmented respectively to obtain segmentation results of the first image and the second image, that is, a first mask corresponding to the target region in the first image and a second mask corresponding to the target region in the second image.

S280, filtering the first key point and the second key point according to the first mask code, the second mask code, the first vehicle position information and the second vehicle position information to obtain a first background key point and a second background key point which correspond to the first image and the second image in the target area respectively.

The first background key point refers to a background key point obtained after filtering key points in the first image, and the background key point may be a road in the target area in the first image. The second background key point refers to a background key point obtained by filtering key points in the second image, and the background key point is a road in the target area in the second image.

In this embodiment, when detecting the key points in the first image and the second image, the key points include not only the roads of the image backgrounds in the first image and the second image, but also the trees, the grasses, the wastelands, the plants, the signboard, and the like in the image backgrounds, and at this time, the trees, the grasses, the wastelands, the plants, the signboard, and the like of the image backgrounds in the first image and the second image, and the first vehicle position information and the second vehicle position information in the first image and the second image need to be filtered out, so as to obtain the first background key points and the second background key points corresponding to the first image and the second image in the target area, respectively. Specifically, the key points corresponding to the target areas in the first image and the second image can be detected, and if the detection result is within the mask range and is outside the first vehicle position information and the second vehicle position information, the key points are reserved; and if the key points are not in the mask range and/or in the first vehicle position information and the second vehicle position information, filtering the key points.

In an embodiment, filtering the first keypoints and the second keypoints according to the first mask and the second mask respectively includes:

respectively carrying out key point detection on a first key point and a second key point of a target area in the first image and the second image; if the key point detection result is in the mask range and is outside the first vehicle position information and the second vehicle position information, reserving the first key point and the second key point; and if the detection result of the key points is not in the mask range and/or in the first vehicle position information and the second vehicle position information, filtering the first key points and the second key points.

In this embodiment, the mask range may be determined by the result of the segmentation in the first image and the second image. For example, if the non-target area is represented by 0, the corresponding pixel value is also 0, the target area is represented by 1, and then the mask range is searched through the target area represented by 1.

In the embodiment, corresponding key point detection is respectively carried out on a first key point and a second key point of a target area in a first image and a second image through one image feature algorithm of SIFT, SURF and ORB, and the first key point and the second key point are reserved under the condition that the key point detection result is in a mask range and the first vehicle position information and the second vehicle position information are not included; and filtering the first key points and the second key points under the condition that the detection result of the key points is not in the mask range and/or is in the first vehicle position information and the second vehicle position information.

And S290, calling a fast approximate neighbor algorithm library to perform key point matching on the feature descriptors in the first background key points and the feature descriptors in the second background key points so as to obtain similar background key points.

In this embodiment, a fast approximate neighbor algorithm library is adopted to perform key point matching on a feature descriptor of a first background key point corresponding to a target region in a first image and a feature descriptor of a second background key point corresponding to the target region in a second image, and a clustering method is used to obtain background key points similar to the matched feature descriptors. It should be noted that the similar background keypoints refer to similar background keypoints obtained by performing preliminary screening on keypoints in the first image and the second image. The similar background key points in the first image and the second image may be that one background key point in the first image is similar to the background key points in the plurality of second images; it may also be that a background key point in the first image is similar to a background key point in the second image, and this embodiment is not limited herein.

S2100, performing secondary matching on the similar background key points by adopting a homography matching method to obtain matched background key points in the first image and the second image.

Wherein homography matching means a projection mapping method from one plane to another, homography matching is a method of finding each other in two images when one image is a perspective transformation of the other image.

In this embodiment, after similar background key points are obtained in the first image and the second image, the obtained similar background key points in the first image and the second image are subjected to secondary matching by using a homography matching method, so that only the first image and the second image have the optimal condition of single matching, and noise background key points which cannot meet geometric mapping between the first image and the second image can be filtered by using the homography matching method, so that background key points in a target region with higher reliability in the first image and the second image are obtained. It should be noted that the information corresponding to the background key point at least includes: the pixel coordinate information of the background key points in the pixel coordinate system and the feature descriptors corresponding to the background key points.

And S2110, acquiring first imaging parameter information corresponding to the preset acquisition equipment and first coordinate information of the preset acquisition equipment in a world coordinate system at the corresponding moment of the first image.

The first imaging parameter information refers to imaging parameter information corresponding to a preset acquisition device at a start time corresponding to a first frame of image, and the imaging parameter information may include a focal length, a shooting angle, a size of the preset acquisition device, a resolution during shooting, and the like. The first coordinate information refers to relative coordinate information of the preset acquisition equipment in a world coordinate system at the corresponding starting time of the first frame image.

In this embodiment, at a start time corresponding to a first frame of image within a speed measurement time period in a video stream, first imaging parameter information corresponding to a preset acquisition device and first coordinate information of the preset acquisition device in a world coordinate system are obtained.

S2120, at the moment of correspondence of the second image, second imaging parameter information of the preset acquisition device and second coordinate information of the preset acquisition device in a world coordinate system are obtained.

The second imaging parameter information refers to imaging parameter information corresponding to the preset acquisition device at the end time corresponding to the last frame of image, and the imaging parameter information may also include a focal length, a shooting angle, a size of the preset acquisition device, a resolution during shooting, and the like. The second coordinate information refers to relative coordinate information of the preset acquisition device in a world coordinate system at the end time corresponding to the last frame of image.

In this embodiment, in a video stream, second imaging parameter information corresponding to a preset acquisition device and second coordinate information of the preset acquisition device in a world coordinate system are obtained at an end time corresponding to a last frame of image within a speed measurement time period.

S2130, determining a first speed of the target vehicle in a world coordinate system according to the first imaging parameter information, the second imaging parameter information, first coordinate information and second coordinate information of a preset acquisition device in the world coordinate system, and the first vehicle position information and the second vehicle position information.

Wherein the first speed refers to a speed of the target vehicle in a world coordinate system.

In this embodiment, the first speed of the target vehicle in the world coordinate system may be determined according to the first imaging parameter information, the second imaging parameter information, the first coordinate information and the second coordinate information of the preset acquisition device in the world coordinate system, and the first vehicle position information and the second vehicle position information. Specifically, a first pixel coordinate information and a second pixel coordinate information of a target vehicle in a pixel coordinate system in a first vehicle position information and a second vehicle position information are determined, a first coordinate mapping matrix of the first pixel coordinate information to a preset acquisition device in the world coordinate system is determined according to the first imaging parameter information and the first coordinate information of the preset acquisition device in the world coordinate system, a second coordinate mapping matrix of the second pixel coordinate information to the preset acquisition device in the world coordinate system is determined according to the second imaging parameter information and the second coordinate information of the preset acquisition device in the world coordinate system, then the first pixel coordinate information and the second pixel coordinate information corresponding to the target vehicle are mapped to the world coordinate system according to the first coordinate mapping matrix and the second mapping matrix respectively to obtain a first imaging device coordinate and a second imaging device coordinate which are opposite, and a first imaging device coordinate and a second imaging device coordinate in the first image and the second image and the total time of the target vehicle in a speed measuring time period are determined according to the coordinate change information of the first imaging device coordinate and the second imaging device coordinate in the world coordinate system.

In an embodiment, determining a first speed of the target vehicle in the world coordinate system according to the first imaging parameter information, the second imaging parameter information, the first coordinate information and the second coordinate information of the preset acquisition device in the world coordinate system, and the first vehicle position information and the second vehicle position information includes:

determining first pixel coordinate information and second pixel coordinate information of the target vehicle in a pixel coordinate system in the first vehicle position information and the second vehicle position information;

determining a first coordinate mapping matrix of the first pixel coordinate information to the preset collection equipment under the world coordinate system according to the first imaging parameter information and first coordinate information of the preset collection equipment under the world coordinate system;

determining a second coordinate mapping matrix of the second pixel coordinate information to the world coordinate system relative to the preset acquisition equipment according to the second imaging parameter information and second coordinate information of the preset acquisition equipment in the world coordinate system;

respectively mapping first pixel coordinate information and second pixel coordinate information corresponding to the target vehicle to a world coordinate system according to the first coordinate mapping matrix and the second mapping matrix so as to obtain relative first imaging equipment coordinates and second imaging equipment coordinates;

and determining the first speed of the target vehicle under the world coordinate system according to the coordinate change information of the first imaging equipment coordinate and the second imaging equipment coordinate in the first image and the second image and the total time in the speed measurement time period.

The first pixel coordinate information refers to pixel coordinate information of the target vehicle in a pixel coordinate system at a starting time corresponding to the first frame image. The second pixel coordinate information refers to the pixel coordinate information of the target vehicle in the pixel coordinate system at the ending time corresponding to the last frame of image. The first coordinate mapping matrix refers to a coordinate mapping matrix of the first pixel coordinate information to the relative preset acquisition equipment in the world coordinate system. The second coordinate mapping matrix refers to a coordinate mapping matrix of the second pixel coordinate information to the corresponding preset acquisition equipment in the world coordinate system.

In this embodiment, the coordinate change information may be understood as pixel coordinate information corresponding to the target vehicle in the first image and the second image, respectively, and is mapped to a difference change of coordinate information formed in the world coordinate system. The total time in the speed measurement time period may be understood as a total time corresponding to a starting time corresponding to the first frame image to an ending time corresponding to the last frame image, and the total time may change with the selection change of the speed measurement time period.

In the embodiment, first pixel coordinate information and second pixel coordinate information of a target vehicle in a pixel coordinate system in first vehicle position information and second vehicle position information are determined, a first coordinate mapping matrix of the first pixel coordinate information to a preset acquisition device in the world coordinate system is determined according to the first imaging parameter information and the first coordinate information of the preset acquisition device in the world coordinate system, a second coordinate mapping matrix of the second pixel coordinate information to the preset acquisition device in the world coordinate system is determined according to the second imaging parameter information and the second coordinate information of the preset acquisition device in the world coordinate system, then the first pixel coordinate information and the second pixel coordinate information corresponding to the target vehicle are mapped to the world coordinate system according to the first coordinate mapping matrix and the second mapping matrix respectively, so that relative first imaging device coordinates and second imaging device coordinates are obtained, and a first imaging device coordinate and second imaging device coordinate change information of the target vehicle in the first image and the second image and total time of the target vehicle in the world coordinate system are determined according to total time of the target vehicle in the world coordinate system.

For example, in a two-dimensional physical space coordinate system, the coordinate information formed by mapping the pixel coordinate information corresponding to the target vehicle in the first frame image to the world coordinate system is (1, 1), the pixel coordinate information corresponding to the target vehicle in the last frame image is (2, 2), and the change of the coordinate information is the coordinate difference between the last frame image and the first frame image, so that the first speed may be determined by the ratio of the coordinate difference between the last frame image and the first frame image to the total time in the speed measurement period.

S2140, determining a second speed of the background key point of the target area in the world coordinate system according to the first imaging parameter information, the second imaging parameter information, the first coordinate information and the second coordinate information of the preset acquisition equipment in the world coordinate system, and the pixel coordinate information of the background key point.

Wherein, the information corresponding to the background key points at least comprises: the pixel coordinate information of the background key points and the feature descriptors corresponding to the background key points.

Wherein the second speed refers to the speed of the background key points of the target area in a world coordinate system.

In this embodiment, pixel coordinate information corresponding to a matched background key point in a first image and a second image is respectively recorded as third pixel coordinate information and fourth pixel coordinate information, a third coordinate mapping matrix of the third pixel coordinate information to a preset acquisition device under a world coordinate system is determined according to the first imaging parameter information and first coordinate information of the preset acquisition device under the world coordinate system, fourth pixel coordinate information is determined according to the second imaging parameter information and second coordinate information of the preset acquisition device under the world coordinate system to a fourth coordinate mapping matrix of the preset acquisition device under the world coordinate system, the third pixel coordinate information and the fourth pixel coordinate information corresponding to the background key point are respectively mapped under the world coordinate system according to the third coordinate mapping matrix and the fourth mapping matrix to obtain a third imaging device coordinate and a fourth imaging device coordinate which are opposite, and a second imaging device coordinate in the world coordinate system is determined according to coordinate change information of the third imaging device coordinate and the fourth imaging device coordinate in the first image and the second image and total time of a speed measuring key point in a target area under the world coordinate system.

In an embodiment, determining a second speed of the background key points of the target area in the world coordinate system according to the first imaging parameter information, the second imaging parameter information, the first coordinate information and the second coordinate information of the preset acquisition device in the world coordinate system, and the corresponding information of the background key points includes:

respectively recording corresponding pixel coordinate information of the matched background key points in the first image and the second image as third pixel coordinate information and fourth pixel coordinate information;

determining a third coordinate mapping matrix of third pixel coordinate information to the preset acquisition equipment under the world coordinate system according to the first imaging parameter information and first coordinate information of the preset acquisition equipment under the world coordinate system;

determining a fourth coordinate mapping matrix of the fourth pixel coordinate information to the preset acquisition equipment under the world coordinate system according to the second imaging parameter information and second coordinate information of the preset acquisition equipment under the world coordinate system;

respectively mapping third pixel coordinate information and fourth pixel coordinate information corresponding to the background key points to a world coordinate system according to a third coordinate mapping matrix and a fourth mapping matrix so as to obtain a third imaging device coordinate and a fourth imaging device coordinate which are opposite;

and determining a second speed of the background key point in the target area under a world coordinate system according to the coordinate change information of the coordinates of the third imaging equipment and the fourth imaging equipment in the first image and the second image and the total time in the speed measuring time period.

The third pixel coordinate information refers to pixel coordinate information of the background key points matched with the target area in a world coordinate system at the corresponding start time of the first frame image. The fourth pixel coordinate information refers to the pixel coordinate information of the background key points matched with the target area in the world coordinate system at the end time corresponding to the last frame of image. The third coordinate mapping matrix refers to a coordinate mapping matrix of third pixel coordinate information to the world coordinate system relative to the preset acquisition equipment. The fourth coordinate mapping matrix refers to a coordinate mapping matrix of fourth pixel coordinate information to a world coordinate system relative to a preset acquisition device.

In this embodiment, the coordinate change information may be understood as pixel coordinate information corresponding to a background key point matching the target area in the first image and the second image, and is mapped to a difference change of coordinate information formed in a world coordinate system.

In this embodiment, pixel coordinate information corresponding to a matched background key point in a first image and a second image is respectively recorded as third pixel coordinate information and fourth pixel coordinate information, a third coordinate mapping matrix of the third pixel coordinate information to a preset acquisition device under a world coordinate system is determined according to the first imaging parameter information and first coordinate information of the preset acquisition device under the world coordinate system, fourth pixel coordinate information is determined to a fourth coordinate mapping matrix of the preset acquisition device under the world coordinate system according to the second imaging parameter information and second coordinate information of the preset acquisition device under the world coordinate system, then the third pixel coordinate information and the fourth pixel coordinate information corresponding to the background key point are respectively mapped to the world coordinate system according to the third coordinate mapping matrix and the fourth mapping matrix, so as to obtain a third imaging device coordinate and a fourth imaging device coordinate which are opposite, and a second imaging device coordinate in the target coordinate system is determined according to coordinate change information of the third imaging device coordinate and the fourth imaging device coordinate in the first image and the second image and total time measurement time in the target time period.

For example, in a two-dimensional physical space coordinate system, the coordinate information formed by mapping the pixel coordinate information corresponding to the background key point matched with the target area in the first frame image to the world coordinate system is (3, 3), the coordinate information formed by mapping the pixel coordinate information corresponding to the background key point matched with the target area in the last frame image to the world coordinate system is (2, 2), and it is known that the change of the coordinate information is the coordinate difference between the last frame image and the first frame image, the second speed may be determined in a manner of calculating the second speed as the ratio of the coordinate difference between the last frame image and the first frame image to the total time in the speed measurement period.

And S2150, determining the movement speed of the target vehicle in the world coordinate system according to the difference value of the first speed and the second speed.

In this embodiment, the real moving speed of the target vehicle in the world coordinate system may be determined according to the difference between the speed of the target vehicle in the world coordinate system and the speed of the background key point of the target area in the world coordinate system.

According to the technical scheme of the embodiment, target areas corresponding to a first image and a second image are determined according to a semantic segmentation neural network model, the target areas of the first image and the second image are segmented respectively to determine a first mask corresponding to the target area in the first image and a second mask corresponding to the target area in the second image, first key points and second key points are filtered respectively according to the first mask and the second mask to obtain first background key points and second background key points corresponding to the first image and the second image in the target areas respectively, a fast approximate neighbor algorithm library and a homography matching method are adopted to perform key point matching on feature descriptors in the first background key points and feature descriptors in the second background key points to obtain matched background key points in the first image and the second image, the optimally matched background key points can be found, and the matching accuracy is improved; acquiring first imaging parameter information corresponding to preset acquisition equipment and first coordinate information of the preset acquisition equipment in a world coordinate system at the corresponding moment of the first image; acquiring second imaging parameter information of preset acquisition equipment and second coordinate information of the preset acquisition equipment in a world coordinate system at the corresponding moment of a second image; determining a first speed of a target vehicle in a world coordinate system according to the first imaging parameter information, the second imaging parameter information, first coordinate information and second coordinate information of a preset acquisition device in the world coordinate system, and first vehicle position information and second vehicle position information; determining a second speed of a background key point of a target area in a world coordinate system according to first imaging parameter information, second imaging parameter information, first coordinate information and second coordinate information of preset acquisition equipment in the world coordinate system and pixel coordinate information of the background key point; the moving speed of the target vehicle under the world coordinate system is determined according to the difference value of the first speed and the second speed, the speed of the background key point matched with the first image and the second image and the speed of the target vehicle can be comprehensively considered, the speed measurement of the tracked target under each scene is effectively completed, the problem that the real speed of the target vehicle has large deviation when the speed measurement is carried out is solved, and the speed measurement precision of the target vehicle is improved.

In an embodiment, to facilitate better understanding of the video speed measurement method, fig. 3 is a flowchart of another video speed measurement method provided in an embodiment of the present invention, in this embodiment, homography matching is performed on features of a target area background key point between two frames, a target area background key point with high reliability and a coordinate thereof in a pixel coordinate system are obtained, a pixel coordinate of a tracking target can be obtained according to a target tracking result, and the pixel coordinate and a mapping matrix of the pixel coordinate to a coordinate of a relative imaging device in a world coordinate system are combined, so as to calculate a calculation speed of the tracking target and the target area background key point in the world coordinate system; therefore, the relative speed between the tracking target and the background of the target area is obtained through calculation, namely the real speed of the tracking target under the real world coordinate, and the accuracy of target speed measurement through a video technical means is improved.

As shown in fig. 3, taking an example that a target area scene is a road range in a scene of judging whether the driving speed of a road vehicle exceeds a specified range under a traffic control scene, the video speed measurement method specifically includes the following steps:

s310, obtaining a first frame image and a last frame image in the speed measuring time period.

And S320, detecting the target vehicle in the target areas respectively corresponding to the first frame image and the last frame image by using a target detection neural network model.

And S330, performing key point detection on the first frame image and the last frame image.

S340, segmenting the target areas corresponding to the first frame image and the last frame image respectively by using a semantic segmentation neural network model to determine a first mask corresponding to the target area in the first frame image and a second mask corresponding to the target area in the last frame image, and filtering key points according to the first mask, the second mask and the detection result of the target vehicle to obtain background key points.

And S350, matching the background key points by adopting a fast approximate neighbor algorithm and a library homography matching method to obtain the matched background key points in the first frame image and the last frame image, and first pixel coordinate information corresponding to the matched background key points in the first frame image under the pixel coordinate and second pixel coordinate information corresponding to the matched background key points in the last frame image under the pixel coordinate.

And S360, determining third pixel coordinate information and fourth pixel coordinate information, corresponding to the target vehicle in the target area in the first frame image and the last frame image, in a pixel coordinate system according to the target detection frame corresponding to the first frame image and the target detection frame corresponding to the last frame image.

And S370, acquiring shooting parameter information corresponding to the imaging devices respectively at the corresponding time of the first frame image and the corresponding time of the last frame image, and relative coordinate information corresponding to the imaging devices respectively in a world coordinate system, wherein the relative coordinate information corresponding to the first frame image is defined as first coordinate information, and the relative coordinate information corresponding to the last frame image is defined as second coordinate information.

And S380, according to the shooting parameter information respectively corresponding to the imaging devices, the first coordinate information and the second coordinate information, under the corresponding time of the first frame image and the corresponding time of the last frame image, determining pixel coordinate information respectively corresponding to the target vehicles in the first frame image and the last frame image, and mapping matrixes of the pixel coordinate information respectively corresponding to the target area background key points to the coordinates of the imaging devices in the world coordinate system, wherein the mapping matrixes are respectively marked as a first mapping matrix, a second mapping matrix, a third mapping matrix and a fourth mapping matrix.

The first mapping matrix and the second mapping matrix respectively represent mapping matrixes of pixel coordinate information respectively corresponding to the target vehicle at the first image corresponding time and the second image corresponding time; the third mapping matrix and the fourth mapping matrix respectively represent mapping matrixes of pixel coordinate information corresponding to the background key points of the target area respectively under the corresponding time of the first image and the corresponding time of the second image.

And S390, determining a first speed of the target vehicle in the world coordinate according to the first mapping matrix, the second mapping matrix, the third pixel coordinate information and the fourth pixel coordinate information of the target vehicle in the first frame image and the last frame image in the pixel coordinate system, and the total time corresponding to the first frame image and the last frame image.

And S3100, determining a second speed of the matched background key points in the first frame image and the last frame image in the world coordinate system according to the third mapping matrix, the fourth mapping matrix, first pixel coordinate information corresponding to the matched background key points in the first frame image under the pixel coordinates, second pixel coordinate information corresponding to the matched background key points in the last frame image under the pixel coordinates, and total time corresponding to the first frame image and the last frame image.

S3110, determining the real movement speed of the target vehicle in the world coordinate system according to the first speed of the target vehicle in the world coordinate system and the second speed of the matched background key points in the first frame image and the last frame image in the world coordinate system.

In an embodiment, fig. 4 is a block diagram of a video speed measuring device according to an embodiment of the present invention, the video speed measuring device is suitable for use in a situation where a video speed is measured by combining a visual moving speed and a moving speed of a target vehicle, and the video speed measuring device can be implemented by hardware/software. The method for measuring the video speed can be configured in the electronic equipment to realize the method for measuring the video speed in the embodiment of the invention. As shown in fig. 4, the apparatus includes: an image acquisition module 410, a location determination module 420, a keypoint determination module 430, and a velocity determination module 440.

The image obtaining module 410 obtains a first image and a second image within a speed measuring time period;

a position determination module 420 that determines a vehicle position location of the target vehicle in the first image and the second image;

the keypoint determining module 430 is configured to obtain a first keypoint corresponding to a first image and a second keypoint corresponding to a second image, and determine background keypoint information matched in a target region according to a matching result of the first keypoint and the second keypoint;

the speed determination module 440 determines a moving speed of the target vehicle based on imaging parameter information of a preset collection device, location information of the preset collection device, the background key point, and the vehicle location information.

According to the embodiment of the invention, a key point determining module determines matched background key points in a target area through matching results of a first key point corresponding to a first image and a second key corresponding to a second image, can find the optimally matched background key points, improves matching accuracy, determines vehicle position information corresponding to a target vehicle in the first image and the second image respectively through a position determining module, and a speed determining module determines the moving speed of the target vehicle through imaging parameter information of preset acquisition equipment, position information of preset acquisition equipment, the background key points and the vehicle position information, can comprehensively measure the speed of the target vehicle in a video by combining the background key points and the vehicle position information, solves the problems of large deviation and insufficient accuracy in measuring the speed of the target vehicle, and improves the accuracy of measuring the speed of the target vehicle.

In one embodiment, the position determination module 420 includes:

a vehicle detection unit, configured to perform detection on the target vehicle for target areas respectively corresponding to the first image and the second image according to a target detection neural network model, where a first detection result of the detection corresponds to the first image, and a second detection result of the detection corresponds to the second image;

the detection frame determining unit is used for determining a first target detection frame corresponding to the target vehicle according to a first detection result of the target vehicle and determining a second target detection frame corresponding to the target vehicle according to a second detection result of the target vehicle;

an information determining unit, configured to determine first vehicle position information and second vehicle position information corresponding to the target vehicle in the first image and the second image according to the first target detection frame and the second target detection frame.

In one embodiment, the keypoint determination module 430 comprises:

a first key point determining unit, configured to invoke a preset image feature algorithm to determine the first key point in the first image and the second key point in the second image, respectively;

the region determining unit is used for determining target regions corresponding to the first image and the second image respectively according to a semantic segmentation neural network model;

a mask determining unit, configured to segment the target regions of the first image and the second image, respectively, to determine a first mask corresponding to the target region in the first image and a second mask corresponding to the target region in the second image;

a second key point determining unit, configured to filter the first key point and the second key point, and the first vehicle position information and the second vehicle position information according to the first mask code and the second mask code, respectively, so as to obtain a first background key point and a second background key point, where the first background key point and the second background key point correspond to the first image and the second image in a target area, respectively;

a third key point determining unit, configured to invoke a fast approximate neighbor algorithm library to perform key point matching on the feature descriptors in the first background key points and the feature descriptors in the second background key points, so as to obtain similar background key points;

and the fourth key point determining unit is used for performing secondary matching on the similar background key points by adopting a homography matching method to obtain matched background key points in the first image and the second image.

In an embodiment, the second keypoint determining unit includes:

a detection subunit, configured to perform keypoint detection on the first keypoint and the second keypoint of the target region in the first image and the second image, respectively;

a reservation subunit, configured to reserve the first key point and the second key point if a result of the key point detection is within the mask range and outside the first vehicle position information and the second vehicle position information;

and the filtering subunit is configured to filter the first key point and the second key point if the result of the key point detection is not within the mask range and/or is within the first vehicle position information and the second vehicle position information.

In one embodiment, the speed determination module 440 includes:

the first coordinate determining unit is used for acquiring first imaging parameter information corresponding to the preset acquisition equipment and first coordinate information of the preset acquisition equipment in a world coordinate system at the corresponding time of the first image;

the second coordinate determination unit is used for acquiring second imaging parameter information of the preset acquisition equipment and second coordinate information of the preset acquisition equipment in a world coordinate system at the corresponding time of the second image;

a first speed determination unit, configured to determine a first speed of the target vehicle in a world coordinate system according to the first imaging parameter information, the second imaging parameter information, the first coordinate information of the preset acquisition device in the world coordinate system, the second coordinate information, the first vehicle position information, and the second vehicle position information;

a second speed determination unit, configured to determine, according to the first imaging parameter information, the second imaging parameter information, the first coordinate information of the preset acquisition device in a world coordinate system, the second coordinate information, and the pixel coordinate information of the background keypoint, a second speed of the background keypoint of a target area in the world coordinate system;

and the movement speed determining unit is used for determining the movement speed of the target vehicle under the world coordinate system according to the difference value of the first speed and the second speed.

In an embodiment, the first speed determination unit comprises:

a first coordinate information determining subunit, configured to determine first pixel coordinate information and second pixel coordinate information of the target vehicle in a pixel coordinate system in the first vehicle position information and the second vehicle position information;

the first matrix determining subunit is used for determining a first coordinate mapping matrix from the first pixel coordinate information to the preset acquisition equipment under the world coordinate system according to the first imaging parameter information and first coordinate information of the preset acquisition equipment under the world coordinate system;

the second matrix determining subunit is configured to determine, according to the second imaging parameter information and second coordinate information of the preset acquisition device in a world coordinate system, a second coordinate mapping matrix from the second pixel coordinate information to the preset acquisition device in the world coordinate system, where the second coordinate mapping matrix is relative to the preset acquisition device;

the device coordinate determining subunit is configured to map first pixel coordinate information and second pixel coordinate information corresponding to the target vehicle to the world coordinate system according to the first coordinate mapping matrix and the second mapping matrix, so as to obtain a first imaging device coordinate and a second imaging device coordinate which are opposite to each other;

and the first speed determining subunit is used for determining the first speed of the target vehicle under the world coordinate system according to the coordinate change information of the first imaging device coordinate and the second imaging device coordinate in the first image and the second image and the total time in the speed measuring time period.

In an embodiment, the second speed determination unit comprises:

a second coordinate determination subunit, configured to record corresponding pixel coordinate information of the matched background keypoint in the first image and the second image as third pixel coordinate information and fourth pixel coordinate information, respectively;

the third matrix determining subunit is used for determining a third coordinate mapping matrix from third pixel coordinate information to the preset acquisition equipment under the world coordinate system according to the first imaging parameter information and the first coordinate information of the preset acquisition equipment under the world coordinate system;

a fourth matrix determining subunit, configured to determine, according to the second imaging parameter information and second coordinate information of the preset acquisition device in a world coordinate system, a fourth coordinate mapping matrix from the fourth pixel coordinate information to the preset acquisition device in the world coordinate system;

the device coordinate determining subunit is configured to map the third pixel coordinate information and the fourth pixel coordinate information corresponding to the background key point to the world coordinate system according to the third coordinate mapping matrix and the fourth mapping matrix, so as to obtain a third imaging device coordinate and a fourth imaging device coordinate which are opposite to each other;

and the second speed determining subunit is configured to determine, according to the coordinate change information of the coordinates of the third imaging device and the coordinates of the fourth imaging device in the first image and the second image, and the total time in the speed measurement time period, a second speed of the background key point in the target area in a world coordinate system.

The video speed measuring device provided by the embodiment of the invention can execute the video speed measuring method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

In an embodiment, fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 5, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, and the like, wherein the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various suitable actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data necessary for the operation of the electronic apparatus 10 may also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.

A number of components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, or the like; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

Processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, or the like. The processor 11 performs the various methods and processes described above, such as the video velocimetry method.

In some embodiments, the video velocimetry method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the video velocimetry method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the video velocimetry method in any other suitable way (e.g. by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on a machine, as a stand-alone software package partly on a machine and partly on a remote machine or entirely on a remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the Internet.

The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.

The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A video speed measurement method is characterized by comprising the following steps:

acquiring a first image and a second image in a speed measuring time period;

determining vehicle position information respectively corresponding to the target vehicles in the first image and the second image;

and determining the movement speed of the target vehicle based on imaging parameter information of preset acquisition equipment, position information of the preset acquisition equipment, the background key point and the vehicle position information.

2. The method of claim 1, wherein the determining vehicle position information corresponding to the target vehicle in the first image and the second image respectively comprises:

detecting the target vehicle according to target detection neural network models corresponding to target areas in the first image and the second image respectively, wherein a first detection result of the detection corresponds to the first image, and a second detection result of the detection corresponds to the second image;

determining a first target detection frame corresponding to the target vehicle according to a first detection result of the target vehicle, and determining a second target detection frame corresponding to the target vehicle according to a second detection result of the target vehicle;

and determining first vehicle position information and second vehicle position information corresponding to the target vehicle in the first image and the second image according to the first target detection frame and the second target detection frame.

3. The method according to claim 1 or 2, wherein the obtaining a first key point corresponding to a first image and a second key point corresponding to a second image, and matching the first key point and the second key point to determine a matched background key point in a target region comprises:

calling a preset image feature algorithm to respectively determine the first key point in the first image and the second key point in the second image;

determining target areas corresponding to the first image and the second image respectively according to a semantic segmentation neural network model;

segmenting the target regions of the first image and the second image, respectively, to determine a first mask corresponding to the target region in the first image and a second mask corresponding to the target region in the second image;

respectively filtering the first key point and the second key point according to the first mask code, the second mask code and the first vehicle position information and the second vehicle position information to obtain a first background key point and a second background key point which respectively correspond to the first image and the second image in the target area;

calling a fast approximate neighbor algorithm library to perform key point matching on the feature descriptors in the first background key points and the feature descriptors in the second background key points to obtain similar background key points;

and performing secondary matching on the similar background key points by adopting a homography matching method to obtain matched background key points in the first image and the second image.

4. The method of claim 3, wherein the filtering the first and second keypoints according to the first and second masks, and the first and second vehicle location information, respectively, comprises:

performing key point detection on the first key point and the second key point of the target area in the first image and the second image respectively;

if the result of the key point detection is within the mask range and is outside the first vehicle position information and the second vehicle position information, reserving the first key point and the second key point;

and if the detection result of the key points is not in the mask range and/or in the first vehicle position information and the second vehicle position information, filtering the first key points and the second key points.

5. The method of claim 1, wherein the determining the moving speed of the target vehicle based on imaging parameter information of a preset acquisition device, position information of the preset acquisition device, the background key point and the vehicle position information comprises:

acquiring first imaging parameter information corresponding to the preset acquisition equipment and first coordinate information of the preset acquisition equipment in a world coordinate system at the corresponding time of the first image;

at the corresponding time of the second image, acquiring second imaging parameter information of the preset acquisition equipment and second coordinate information of the preset acquisition equipment in a world coordinate system;

determining a first speed of the target vehicle in a world coordinate system according to the first imaging parameter information, the second imaging parameter information, the first coordinate information and the second coordinate information of the preset acquisition equipment in the world coordinate system, and the first vehicle position information and the second vehicle position information;

determining a second speed of the background key points of the target area in a world coordinate system according to the first imaging parameter information, the second imaging parameter information, the first coordinate information and the second coordinate information of the preset acquisition equipment in the world coordinate system, and the pixel coordinate information of the background key points;

and determining the movement speed of the target vehicle under the world coordinate system according to the difference value of the first speed and the second speed.

6. The method of claim 5, wherein the determining the first speed of the target vehicle in the world coordinate system according to the first imaging parameter information, the second imaging parameter information, the first coordinate information, the second coordinate information, and the first vehicle position information and the second vehicle position information of the preset acquisition device in the world coordinate system comprises:

determining a first coordinate mapping matrix of the first pixel coordinate information to the preset acquisition equipment under a world coordinate system according to the first imaging parameter information and first coordinate information of the preset acquisition equipment under the world coordinate system;

determining a second coordinate mapping matrix of the second pixel coordinate information to the preset acquisition equipment under the world coordinate system according to the second imaging parameter information and second coordinate information of the preset acquisition equipment under the world coordinate system;

respectively mapping first pixel coordinate information and second pixel coordinate information corresponding to the target vehicle to the world coordinate system according to the first coordinate mapping matrix and the second mapping matrix so as to obtain a first imaging device coordinate and a second imaging device coordinate which are opposite;

and determining the first speed of the target vehicle under the world coordinate system according to the coordinate change information of the first imaging equipment coordinate and the second imaging equipment coordinate in the first image and the second image and the total time in the speed measuring time period.

7. The method of claim 5, wherein the determining a second velocity of the background keypoints of the target region in the world coordinate system according to the first imaging parameter information, the second imaging parameter information, the first coordinate information and the second coordinate information of the preset acquisition device in the world coordinate system, and the corresponding information of the background keypoints comprises:

recording corresponding pixel coordinate information of the matched background key points in the first image and the second image as third pixel coordinate information and fourth pixel coordinate information respectively;

mapping the third pixel coordinate information and the fourth pixel coordinate information corresponding to the background key points to the world coordinate system according to the third coordinate mapping matrix and the fourth mapping matrix, so as to obtain a third imaging device coordinate and a fourth imaging device coordinate which are opposite;

and determining a second speed of the background key point in the target area in a world coordinate system according to coordinate change information of the coordinates of the third imaging device and the fourth imaging device in the first image and the second image and the total time in the speed measurement time period.

8. A vehicle speed measurement device, comprising:

9. An electronic device, characterized in that the electronic device comprises:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the video velocimetry method of any of claims 1 to 7.

10. A computer readable storage medium having stored thereon computer instructions for causing a processor to implement the video velocimetry method of any of claims 1-7 when executed.