CN115761563A - River surface flow velocity calculation method and system based on optical flow measurement and calculation - Google Patents
River surface flow velocity calculation method and system based on optical flow measurement and calculation Download PDFInfo
- Publication number
- CN115761563A CN115761563A CN202211182996.3A CN202211182996A CN115761563A CN 115761563 A CN115761563 A CN 115761563A CN 202211182996 A CN202211182996 A CN 202211182996A CN 115761563 A CN115761563 A CN 115761563A
- Authority
- CN
- China
- Prior art keywords
- flow
- river
- optical flow
- image matrix
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/30—Assessment of water resources
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a river surface flow velocity calculation method and system based on optical flow measurement, wherein the method comprises the following steps: calculating an optical flow value of the image by adopting an optical flow algorithm, calculating the characteristic point offset in the flow measurement image matrix based on the optical flow value, and filtering the offset value; filtering the optical flow value of the flow measurement image matrix; calculating to obtain the river surface speed under a pixel coordinate system, and converting the pixel coordinate of each video frame image in the river flow video collected by the camera into a world coordinate; calculating the spatial resolution of the image according to the pixel coordinates of two continuous frames of images and the world coordinates of two continuous frames of images in the flow measurement image matrix after the optical flow value is filtered; and calculating the river surface flow velocity in the world coordinate system by using the spatial resolution of the image and the river velocity of the filtered characteristic points in the pixel coordinate system. The invention can improve the calculation efficiency and the calculation accuracy.
Description
Technical Field
The invention relates to the technical field of flow field measurement, in particular to a method and a system for calculating river surface flow velocity based on optical flow measurement.
Background
Flow monitoring plays a crucial role in hydrological modeling and hydrological engineering. Traditional flow rate measurement relies on instrumental methods, most of which are expensive, require a lot of time for relevant personnel to measure, and can only perform point measurement, and cannot perform comprehensive measurement.
With the development of the field of computer vision, a method of extracting hydraulic information from image data is increasingly applied to actually measure river surface discharge. The existing image velocimetry methods include large-scale particle image velocimetry (LSPIV), particle Tracking Velocimetry (PTV) and optical flow methods. The existing methods for measuring the surface flow of river water by using an optical flow method are divided into two methods, one method is the direct application of various optical flow methods in flow measurement, and the other method is the combination of a deep learning method and the optical flow method.
In the process of implementing the invention, the inventor of the present application finds that the prior art method has at least the following technical problems:
due to the influence of vegetation, sediment and river surface floating objects, a great deal of turbulence exists in the river, the movement of each part in the river is shown by an optical flow method, and the calculation amount is large due to the existence of a great deal of turbulence. Although the L-K optical flow method is independently used, the optical flow can be obtained, but the method is not sensitive to noise, when the movement scale of the target is too large or shielding and the like occur, a large calculation error is generated due to the too large movement scale, the accuracy of the algorithm is influenced, and the overall calculation speed is reduced; the method of adding deep learning can improve the algorithm calculation accuracy to a certain extent, but has extremely high requirements on equipment, and the running time can be increased again on the original basis, so that the effect of monitoring the flow in real time cannot be achieved.
Disclosure of Invention
The invention provides a method and a system for calculating river surface flow velocity based on optical flow measurement and calculation, which are used for solving or at least partially solving the technical problem of low calculation efficiency in the prior art.
In order to solve the above technical problem, a first aspect of the present invention provides a method for calculating a river surface flow velocity based on optical flow estimation, comprising:
s1: collecting a river flow video by using a camera;
s2: extracting video frames from the collected river flow video according to a preset time interval and a frame interval, wherein all the extracted video frames form a flow measurement image matrix;
s3: preprocessing a flow measurement image matrix;
s4: performing feature detection and light flow value calculation on the image in the preprocessed flow measurement image matrix by adopting a light flow algorithm and combining a feature detection algorithm to obtain a light flow value g of the measurement image matrix L ＝[l x ,l y ] T The optical flow values of the measured image matrix represent the movement of feature points between image frames in the flow-measuring image matrix, wherein,
is a component in a direction parallel to the cross section of the river,
is a component in the direction vertical to the cross section of the river;
s5: obtaining a flow measurement image matrix based on the optical flow value of the flow measurement image matrixOffset of middle feature point
Then calculating the offset value I of the characteristic point in the flow measurement image matrix according to the Euclidean distance,
sorting the offset values from high to low, and filtering the offset values of the preset proportion after sorting, wherein the offset value of the characteristic point in the flow measurement image matrix is the moving distance of the characteristic point between image frames in the flow measurement image matrix;
s6: filtering the optical flow value of the flow measurement image matrix;
s7: calculating to obtain the river speed of the filtered characteristic points under the pixel coordinate system according to the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix, and selecting the river speed of the filtered characteristic points of a preset part under the pixel coordinate system to obtain an average value so as to obtain the river surface speed under the pixel coordinate system;
s8: converting the pixel coordinates of each frame of image in the flow measurement image matrix into world coordinates;
s9: calculating the spatial resolution of the image according to the pixel coordinates of two continuous frames of images and the world coordinates of two continuous frames of images in the flow measurement image matrix after the optical flow value is filtered;
s10: and calculating the river surface flow velocity under the world coordinate system by using the spatial resolution of the image and the river surface velocity under the pixel coordinate system.
In one embodiment, the preprocessing of the side-stream image matrix at S3 includes removing hue and saturation information, adjusting hue of the picture, and cropping the image boundary.
In one embodiment, performing feature point detection and optical flow value calculation on an image in a preprocessed flow-measuring image matrix by using an optical flow algorithm in combination with a feature detection algorithm includes:
zooming each frame of image in the flow measurement image matrix layer by layer in a pyramid layering mode, wherein the resolution of the image at the bottom layer is the largest, and the resolution of the image at the top layer is the smallest; starting from the topmost image, performing feature detection by using a feature detection algorithm to obtain feature points of a kth frame and a (k + 1) th frame, and obtaining an optical flow value of each feature point in the topmost image through feature matching so as to obtain the optical flow value of the topmost image; and estimating the optical flow value of the next layer according to the optical flow value in the top layer image until the optical flow value of the bottommost layer is calculated and is used as the optical flow value of the flow measuring image matrix.
In one embodiment, S6 comprises: and filtering the optical flow value by adopting a statistical histogram mode.
In one embodiment, when the river flow direction is perpendicular to the cross section, S6 includes: filtering the light flow value according to the included angle range between the initial offset and the final offset of the characteristic point in the measured image matrix,
wherein
For the initial offset of the feature points in the lateral flow image matrix,
the final offset of the characteristic point in the selected flow measurement image matrix is obtained.
In one embodiment, step S7 comprises:
calculating the river speed of each filtered characteristic point in a pixel coordinate system according to the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix
Wherein v is the river velocity of each filtered feature point under a pixel coordinate system, w is the time of a video corresponding to the flow measurement image matrix, and | l | | | is an offset value;
selecting river speeds of the filtered characteristic points of the preset part under the pixel coordinate system to obtain an average value, and obtaining the river surface speed under the pixel coordinate system:
wherein N is the number of the filtered characteristic points of the selected preset part, V is the river surface speed under the pixel coordinate system,
and summing the river speed representing each selected characteristic point.
In one embodiment, the calculation manner of S9 is:
wherein (u) k ,v k ) Is the pixel coordinate of the k frame image, (u) k+1 ,v k+1 ) Is the pixel coordinate of the (k + 1) th frame image, (X) k ,Y k ) Coordinates in X and y directions which are world coordinates of the k frame image, (X) k+1 ,Y k+1 ) Coordinates in x and y directions, λ, of world coordinates of the k +1 th frame image x And λ y Spatial resolution in the x-direction and y-direction, respectively.
In one embodiment, the calculation of S10 is:
wherein V is the river surface velocity in the pixel coordinate system, lambda x And λ y The spatial resolution is respectively in the x direction and the y direction, V' is the river surface flow velocity in the world coordinate system, and fps is the frame rate.
Based on the same inventive concept, the second aspect of the present invention provides a system for calculating river surface flow velocity based on optical flow estimation, comprising:
the video acquisition module is used for acquiring a river flow video by using a camera;
the video frame extraction module is used for extracting video frames from the collected river flowing video according to a preset time interval and a frame interval, and all the extracted video frames form a flow measurement image matrix;
the preprocessing module is used for preprocessing the flow measurement image matrix;
an optical flow value calculation module for performing feature detection and optical flow value calculation on the image in the preprocessed flow measurement image matrix by adopting an optical flow algorithm and combining a feature detection algorithm to obtain an optical flow value g of the measurement image matrix L ＝[l x ,l y ] T An optical flow value of the measured image matrix represents a movement of the feature points between image frames in the flow-measuring image matrix, wherein,
is a component in a direction parallel to the cross section of the river,
is a component in the direction vertical to the cross section of the river;
an offset calculating module for obtaining the offset of the characteristic point in the flow measurement image matrix based on the optical flow value of the flow measurement image matrix
Then calculating the offset value l of the characteristic point in the flow measurement image matrix according to the Euclidean distance,
sorting the offset values from high to low, and filtering the offset values of the preset proportion after sorting, wherein the offset value of the characteristic point in the flow measurement image matrix is the moving distance of the characteristic point between image frames in the flow measurement image matrix;
the optical flow value filtering module is used for filtering the optical flow values of the flow measurement image matrix;
the river surface speed calculating module under the pixel coordinate system is used for calculating the river speed of each filtered characteristic point under the pixel coordinate system according to the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix, and selecting the river speed of a preset part of the filtered characteristic points under the pixel coordinate system to obtain an average value so as to obtain the river surface speed under the pixel coordinate system;
the coordinate conversion module is used for converting the pixel coordinate of each frame of image in the flow measurement image matrix into a world coordinate;
the spatial resolution calculation module is used for calculating the spatial resolution of the image according to the pixel coordinates of two continuous frames of images and the world coordinates of the two continuous frames of images in the flow measurement image matrix after the optical flow value is filtered;
and the river surface flow velocity calculating module under the world coordinate system is used for calculating the river surface flow velocity under the world coordinate system by utilizing the spatial resolution of the image and the river surface velocity under the pixel coordinate system.
Based on the same inventive concept, a third aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored, which when executed, performs the method of the first aspect.
Compared with the prior art, the invention has the advantages and beneficial technical effects as follows:
in the prior art, an optical flow method is adopted to calculate optical flow values of all captured feature points on a picture, the calculation amount is very large, the calculation is also very complex, but the river has turbulence or influence of bank vegetation, so that the water flow does not flow according to the main flow direction, and the calculation of the optical flow values in the non-main flow direction influences the accuracy of data on one hand and also has large calculation amount on the other hand. The method provided by the invention processes and calculates the image in the preprocessed flow measurement image matrix in a mode of combining an optical flow algorithm and a characteristic detection algorithm to obtain the optical flow value of the measurement image matrix, calculates the offset value and filters the offset value, further filters the optical flow value of the flow measurement image matrix, and can filter the optical flow value in a non-main flow direction, thereby reducing the calculation time and improving the calculation accuracy.
Drawings
In order to more clearly illustrate the embodiments or technical solutions of the present invention, the drawings used in the embodiments or technical solutions in the prior art are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a flow chart of a method for calculating river surface flow velocity based on optical flow estimation according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of optical flow meshing provided by an embodiment of the present invention;
FIG. 3 is a histogram statistics diagram provided by an embodiment of the present invention;
FIG. 4 is a graph showing the test results of the first embodiment of the present invention.
FIG. 5 is a graph showing the test results of the second embodiment of the present invention.
FIG. 6 is a graph showing the test results of the third embodiment of the present invention.
Fig. 7 is a block diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
The inventor of the application finds out through a great deal of research and practice that: because the light streams on the same moving target are similar and the distribution of the light streams is in a certain regularity, the light streams on different targets are different, the deviation of the angle and the cross section of the river is overlarge, and the light stream with the too short length can be considered as the light stream formed by the turbulence caused by silt, vegetation and floating branches, the method for calculating the speed of the surface of the river based on the light stream measurement and calculation is provided.
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example one
The embodiment of the invention provides a river surface flow velocity calculation method based on optical flow measurement, which comprises the following steps:
s1: collecting a river flow video by using a camera;
s2: extracting video frames from the collected river flow video according to a preset time interval and a frame interval, wherein all the extracted video frames form a flow measurement image matrix;
s3: preprocessing a flow measurement image matrix;
s4: performing feature detection and light flow value calculation on the image in the preprocessed flow measurement image matrix by adopting a light flow algorithm and combining a feature detection algorithm to obtain a light flow value g of the measurement image matrix L ＝[l x ,l y ] T An optical flow value of the measured image matrix represents a movement of the feature points between image frames in the flow-measuring image matrix, wherein,
is a component in a direction parallel to the cross section of the river,
is in river connection withComponent of cross section vertical direction;
s5: obtaining the offset of the characteristic point in the flow measurement image matrix based on the optical flow value of the flow measurement image matrix
Then calculating the offset value l of the characteristic point in the flow measurement image matrix according to the Euclidean distance,
sorting the offset values from high to low, and filtering the offset values of the preset proportion after sorting, wherein the offset value of the characteristic point in the flow measurement image matrix is the moving distance of the characteristic point between image frames in the flow measurement image matrix;
s6: filtering the optical flow value of the flow measurement image matrix;
s7: calculating to obtain the river speed of the filtered characteristic points under the pixel coordinate system according to the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix, and selecting the river speed of the filtered characteristic points of a preset part under the pixel coordinate system to obtain an average value to obtain the river surface speed under the pixel coordinate system;
s8: converting the pixel coordinates of each frame of image in the flow measurement image matrix into world coordinates;
s9: calculating the spatial resolution of the image according to the pixel coordinates of two continuous frames of images and the world coordinates of two continuous frames of images in the flow measurement image matrix after the optical flow value is filtered;
s10: and calculating the river surface flow velocity under the world coordinate system by using the spatial resolution of the image and the river surface velocity under the pixel coordinate system.
Fig. 1 is a flowchart of a method for calculating a river surface flow velocity based on optical flow estimation according to an embodiment of the present invention.
In the specific implementation process, the RTK and the camera can be carried by the unmanned aerial vehicle to realize video acquisition, and the camera is perpendicular to the river surface.
In step S2, the preset interval time Δ t and the frame interval S may be set according to actual conditions, and 1 frame is extracted at regular intervals, so as to obtain a flow measurement image matrix I composed of all extracted video frames i (i =1, 2.. Once, n), i denotes the sequence number of the video frame (image), and n is the total number of video frames.
The feature detection algorithm is combined with the optical flow algorithm for calculation, the feature detection algorithm is mainly used for feature point detection, the feature points are points containing image information, and feature matching is used in the combination process of the two algorithms. The optical flow algorithm in S4 may be the LK optical flow algorithm or another optical flow algorithm. The main steps of the LK optical flow method can be divided into three steps: the method comprises the following steps of establishing a pyramid, tracking the pyramid and iterating, so that the process of feature matching is carried out in the step of tracking the pyramid, feature points are detected in a first-layer picture of a first frame by using a feature detection algorithm, and because the first-layer picture is reduced and the range of the selected feature points is actually very large, the feature points need to be expanded through iterative calculation, and the method specifically comprises the following steps of: the result of the first layer is used as the initial optical flow value of the next layer, the size of the picture is enlarged, and therefore, the feature points are detected again to make the result more detailed, that is, each layer selects the feature points in a preset range, and the gradually-refined optical flow value is a vector value and is a numerical value with a direction. The calculation process of the feature vector of the feature point in the two frames (the k frame and the (k + 1) th frame) is actually a feature matching process, and the displacement condition of the feature point in the two frames can be obtained through the process, wherein the displacement condition comprises the distance, the direction and the component (l) x 、l y ) Is data of feature points including distance and direction.
The preset proportion of the ranking at the back can be set according to actual conditions, for example, set to 10%.
In one embodiment, the preprocessing of the side-stream image matrix at S3 includes removing hue and saturation information, adjusting the hue of the picture, and cropping the image boundaries.
In particular, for images in a flow-measuring image matrixEliminating hue and saturation information, converting the original color image into a gray scale image, adjusting the hue of the image by using a log correction method (or other correction methods such as a Gamma correction method), and cutting the image boundary to obtain a preprocessed flow measuring image matrix I' i (i＝1,2,...,n)。
In one embodiment, performing feature detection and optical flow value calculation on an image in a preprocessed flow-measuring image matrix by using an optical flow algorithm in combination with a feature detection algorithm includes:
zooming each frame of image in the flow measurement image matrix layer by layer in a pyramid layering mode, wherein the resolution of the image at the bottom layer is the largest, and the resolution of the image at the top layer is the smallest; starting from the topmost image, performing feature detection by using a feature detection algorithm to obtain feature points of a kth frame and a (k + 1) th frame, and obtaining an optical flow value of each feature point in the top image through feature matching so as to obtain the optical flow value of the top image; and estimating the optical flow value of the next layer according to the optical flow value in the top layer image until the optical flow value of the bottommost layer is calculated and is used as the optical flow value of the flow measuring image matrix.
Specifically, the optical flow algorithm may be a Lucas-Kanade optical flow algorithm or other optical flow algorithm (Farneback algorithm).
Estimating the optical flow value of the next layer according to the optical flow value in the top layer image, using the estimated optical flow value as the initial optical flow value of the next layer, and then calculating the actual optical flow value of the next layer by using the method for calculating the optical flow value of the top layer image according to the initial optical flow value of the next layer, thereby repeating the steps until the optical flow value of the lowest layer (namely the original image optical flow value) is calculated.
When calculating the optical flow value of the top layer image, the optical flow value g of each point in the top layer image is obtained by minimizing the matching error sum in the neighborhood range of each point L Scaling the image size to half each time, scaling L m Layer, layer 0 is the original image, and assuming the angular point displacement of the original image is d, the displacement of each layer is
According toThe optical flow value in the top layer image is used as the estimated value g of the optical flow value at the initial time of the next layer L-1 ＝2(g L +d L )。
In one embodiment, the feature detection algorithm employed includes, but is not limited to, the FAST algorithm.
Specifically, the feature detection algorithm is used for obtaining feature points in the optical flow value calculation process. In order to improve the detection efficiency, the characteristic points of the front frame and the rear frame are obtained by adopting a FAST algorithm, and then the optical flow value is obtained according to the pixel coordinates of the characteristic points of the front frame and the rear frame.
The FAST algorithm comprises the following specific steps: comparing the gray value of the pixel in a certain area in the picture with the central point, and the specific method comprises the following steps: comparing gray values of the central point and 12 continuous points in a neighborhood through which a circle with the radius =3.4 passes, if the gray values of the points are all larger or smaller than the central point, the central point can be regarded as a candidate corner point, and after the candidate corner point is obtained, an ID3 classifier is used for judging whether the candidate corner point is a corner point.
In one embodiment, one filtering manner in S6 is: and filtering the optical flow value by adopting a statistical histogram mode.
Filtering the light flow values by means of a statistical histogram can be applied to various river flow directions, and the method uses the histogram to count the directions of the light flow values in the picture. Referring to fig. 2, the optical flow diagram is divided into a grid by using a mesh division method, each optical flow is contained in one grid, the optical flow direction in the grid is counted, and the gradient histogram divides the direction range of 0-360 degrees into 12 bins (bins), wherein each bin is 30 degrees. As shown in fig. 3, the direction of the peak of the histogram represents the principal direction of the keypoint. The peak value of the direction histogram represents the direction of the key point, and the maximum value in the histogram is taken as the main direction of the corresponding key point. The peak value may be considered as a numerical value of each acquired optical flow direction, and the maximum value may be a value having the largest value among all optical flow directions. In order to enhance the robustness of matching, the direction with the peak value larger than 80% of the peak value of the main direction is reserved as the auxiliary direction of the key point, and other parts are filtered out.
When the river flow direction is vertical to the section, one filtering mode in the S6 is as follows: filtering the light flow value according to the included angle range between the initial offset and the final offset of the characteristic point in the measured image matrix,
wherein
For the initial offset of the feature points in the lateral flow image matrix,
the final offset of the characteristic points in the selected lateral flow image matrix is obtained.
Wherein the initial offset represents the state (including position and direction) of the feature point in the first frame image, and the final offset represents the state of the feature point in the last frame image, which is the optical flow.
In the specific implementation process, if the river flow is stable, the river surface has no floating branches, leaves and the like which affect the river, and the river bottom roughness is low and no sediment exists, the probability of turbulence of the river is low, in order to reduce the running time of the program, the two methods can select 80% of the filtered optical flow values to calculate the surface flow rate of the river after filtering the optical flow, if not, but to achieve a faster running speed, or can select 80% of the filtered optical flow values to calculate the surface flow rate of the river after filtering the optical flow.
In one embodiment, step S7 comprises:
and calculating the river speed of each filtered characteristic point in a pixel coordinate system according to the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix:
wherein v is the river speed of each offset, w is the time of the video corresponding to the flow measurement image matrix, and | l | is the offset value;
selecting river speeds of the filtered characteristic points of the preset part under the pixel coordinate system to obtain an average value, and obtaining the river surface speed under the pixel coordinate system:
wherein N is the number of the filtered characteristic points of the selected preset part, V is the river surface speed under the pixel coordinate system,
the river velocities representing each offset value selected are summed.
Specifically, the offset value is a moving distance of the feature point between image frames in the flow measurement image matrix, and the river speed in the pixel coordinate system is obtained through the moving distance. Referring to fig. 2, the arrow included in the rectangular frame is the offset corresponding to the feature point of the selected predetermined portion.
In one embodiment, the calculation of S9 is:
wherein (u) k ,v k ) Is the pixel coordinate of the k frame image, (u) k+1 ,v k+1 ) Is the pixel coordinate of the (k + 1) th frame image, (X) k ,Y k ) Coordinates in X and y directions which are world coordinates of the k frame image, (X) k+1 ,Y k+1 ) Coordinates in x-direction and y-direction of world coordinates of the (k + 1) th frame image, λ x And λ y Spatial resolution in the x-direction and y-direction, respectively.
In a specific implementation process, the coordinate transformation of the S8 is realized by the following method:
for the ith frame pixel coordinate (u) i ,v i ) The corresponding world coordinate is (X) i ,Y i ,Z i ) External reference K of video camera 1 Internal reference K, K of camera 2 Focal length f, pixel width dx, pixel length dy, center point of pixel coordinate system (u) 0 ,z 0 ) Height Z of camera wi Attitude angle of camera
θ, the rotation matrix R, and the translation vector T are known parameters. Wherein the content of the first and second substances,
one point (u) on the pixel coordinate i ,v i ) Conversion to image coordinates of (x) i ,y i )，
Z c Is a camera scale variable.
Step S9 by flow measurementPixel coordinate (u) of two consecutive frames of images in an image k ,v k ) And (u) k+1 ,v k+1 ) And world coordinates G (X) of two consecutive frames of pictures k ,Y k ,Z k ) And G (X) k+1 ,Y k+1 ,Z k+1 ) The picture spatial resolution is calculated.
In one embodiment, the calculation manner of S10 is:
wherein V is the river surface velocity in the pixel coordinate system, lambda x And λ y The spatial resolution in the x direction and the spatial resolution in the y direction are respectively, V' is the river surface flow velocity in the world coordinate system, and fps is the frame rate.
The prior art has methods for directly applying an optical flow algorithm to flow velocity measurement, and the present invention is different from these methods in that: the LK optical flow method is only used as a basic algorithm, other optical flow algorithms are feasible, the key point of the invention is to filter the optical flow value, all steps of the existing method are how to apply the LK optical flow method to flow velocity measurement, the LK optical flow method only exists in step S3, the invention can solve the problems of high calculation complexity and large calculation amount of the optical flow method in the prior art, because the optical flow method can calculate the optical flow value for all captured feature points on a picture, the calculation amount is very large, the calculation is also very complex, but a river can have turbulence condition or bank vegetation influence, so that the water flow does not flow according to the main flow direction, the calculation of the optical flow value of the non-main flow direction influences the accuracy of data, on the one hand, the calculation amount is also large, the optical flow value of the non-main flow direction is filtered, the calculation time can be reduced, and the calculation accuracy can be improved.
In order to illustrate the effectiveness of the method provided by the invention, the river object is tested at the upstream of the brood-oriented ditch, 3 seconds, 5 seconds and 7 seconds of the video are respectively selected and tested at intervals of 1 frame, the number of the video frames is 24 frames/s, and the real speed of the river is 0.8m/s. The test results are shown in fig. 4 to fig. 6, where fig. 4 is a schematic diagram of the test results performed at the time of 3 seconds of the selected video, fig. 5 is a schematic diagram of the test results performed at the time of 5 seconds of the selected video, and fig. 6 is a schematic diagram of the test results performed at the time of 7 seconds of the selected video. The filtering algorithm (namely LK +) adding the optical flow value on the basis of the LK optical flow algorithm is the algorithm of the invention, the LK and Farneback algorithms are the algorithms adopted in the prior art, and the results of figures 4-6 show that the speed and the time of the invention are superior to those of the existing algorithms.
Example two
Based on the same inventive concept, the present embodiment provides a system for calculating river surface flow velocity based on optical flow estimation, including:
the video acquisition module is used for acquiring a river flow video by using a camera;
the video frame extraction module is used for extracting video frames from the collected river flow video according to a preset time interval and a frame interval, and all the extracted video frames form a flow measurement image matrix;
the preprocessing module is used for preprocessing the flow measurement image matrix;
an optical flow value calculation module for performing feature detection and optical flow value calculation on the image in the preprocessed flow measurement image matrix by adopting an optical flow algorithm and combining a feature detection algorithm to obtain an optical flow value g of the measurement image matrix L ＝[l x ,l y ] T An optical flow value of the measured image matrix represents a movement of the feature points between image frames in the flow-measuring image matrix, wherein,
is a component in a direction parallel to the cross section of the river,
is a component in the direction vertical to the cross section of the river;
an offset calculation module for obtaining the offset of the characteristic point in the flow measurement image matrix based on the optical flow value of the flow measurement image matrix
Then calculating the offset value l of the characteristic point in the flow measurement image matrix according to the Euclidean distance,
sorting the offset values from high to low, and filtering the offset values of the preset proportion after sorting, wherein the offset value of the characteristic point in the flow measurement image matrix is the moving distance of the characteristic point between image frames in the flow measurement image matrix;
the optical flow value filtering module is used for filtering the optical flow values of the flow measurement image matrix;
the river surface speed calculating module under the pixel coordinate system is used for calculating and obtaining the river speed of each filtered characteristic point under the pixel coordinate system according to the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix, and selecting the river speed of the filtered characteristic points of a preset part under the pixel coordinate system to obtain an average value so as to obtain the river surface speed under the pixel coordinate system:
the coordinate conversion module is used for converting the pixel coordinate of each frame of image in the flow measurement image matrix into a world coordinate;
the spatial resolution calculation module is used for calculating the spatial resolution of the image according to the pixel coordinates of two continuous frames of images and the world coordinates of the two continuous frames of images in the flow measurement image matrix after the optical flow value is filtered;
and the river surface flow velocity calculating module under the world coordinate system is used for calculating the river surface flow velocity under the world coordinate system by utilizing the spatial resolution of the image and the river surface velocity under the pixel coordinate system.
Since the system described in the second embodiment of the present invention is a system used for implementing the method for calculating the river surface flow velocity based on optical flow measurement and calculation in the first embodiment of the present invention, a person skilled in the art can understand the specific structure and deformation of the system based on the method described in the first embodiment of the present invention, and thus, details thereof are not described herein again. All systems adopted by the method in the first embodiment of the invention belong to the protection scope of the invention.
EXAMPLE III
Based on the same inventive concept, please refer to fig. 7, the present invention further provides a computer readable storage medium 300, on which a computer program 311 is stored, which when executed implements the method as described in the first embodiment.
Since the computer-readable storage medium introduced in the third embodiment of the present invention is a computer-readable storage medium used for implementing the method for calculating the river surface flow velocity based on optical flow measurement and calculation in the first embodiment of the present invention, based on the method introduced in the first embodiment of the present invention, persons skilled in the art can understand the specific structure and deformation of the computer-readable storage medium, and therefore, details are not described here. Any computer readable storage medium used in the method of the first embodiment of the present invention is within the protection scope of the present invention.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made in the embodiments of the present invention without departing from the spirit or scope of the embodiments of the invention. Thus, if such modifications and variations of the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to encompass these modifications and variations.
Claims (10)
1. A river surface flow velocity calculation method based on optical flow measurement and calculation is characterized by comprising the following steps:
s1: collecting a river flow video by using a camera;
s2: extracting video frames from the collected river flow video according to a preset time interval and a frame interval, wherein all the extracted video frames form a flow measurement image matrix;
s3: preprocessing a flow measurement image matrix;
s4: performing feature detection and light flow value calculation on the image in the preprocessed flow measurement image matrix by adopting a light flow algorithm and combining a feature detection algorithm to obtain a light flow value g of the measurement image matrix L ＝[l x ,l y ] T An optical flow value of the measured image matrix represents a movement of the feature points between image frames in the flow-measuring image matrix, wherein,
is a component in a direction parallel to the cross section of the river,
is a component in the direction vertical to the cross section of the river;
s5: based on the moment of the flow-measuring imageObtaining the offset of the characteristic point in the flow measurement image matrix by the optical flow value of the matrix
Then calculating the offset value l of the characteristic point in the flow measurement image matrix according to the Euclidean distance,
sorting the offset values from high to low, and filtering the offset values of a preset proportion at the back of the sorting, wherein the offset value of the characteristic point in the flow measurement image matrix is the moving distance of the characteristic point between image frames in the flow measurement image matrix;
s6: filtering the optical flow value of the flow measurement image matrix;
s7: calculating to obtain the river speed of the filtered characteristic points under the pixel coordinate system according to the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix, and selecting the river speed of the filtered characteristic points of a preset part under the pixel coordinate system to obtain an average value to obtain the river surface speed under the pixel coordinate system;
s8: converting the pixel coordinates of each frame of image in the flow measurement image matrix into world coordinates;
s9: calculating the spatial resolution of the image according to the pixel coordinates of two continuous frames of images and the world coordinates of two continuous frames of images in the flow measurement image matrix after the optical flow value is filtered;
s10: and calculating the river surface flow velocity in the world coordinate system by using the spatial resolution of the image and the river surface velocity in the pixel coordinate system.
2. The method for calculating river surface flow velocity based on optical flow estimation according to claim 1, wherein the preprocessing of the flow image matrix by S3 comprises removing hue and saturation information, adjusting hue of the picture, and clipping the image boundary.
3. The method for calculating river surface flow velocity based on optical flow estimation according to claim 1, wherein the method for performing feature point detection and optical flow value calculation on the image in the preprocessed flow measurement image matrix by using an optical flow algorithm in combination with a feature detection algorithm comprises:
zooming each frame of image in the flow measurement image matrix layer by layer in a pyramid layering mode, wherein the resolution of the image at the bottom layer is the largest, and the resolution of the image at the top layer is the smallest; starting from the topmost image, performing feature detection by using a feature detection algorithm to obtain feature points of a kth frame and a (k + 1) th frame, and obtaining an optical flow value of each feature point in the top image through feature matching so as to obtain the optical flow value of the top image; and estimating the optical flow value of the next layer according to the optical flow value in the top layer image until the optical flow value of the bottommost layer is calculated, and taking the optical flow value as the optical flow value of the flow measurement image matrix.
4. The method for calculating the river surface flow velocity based on optical flow estimation according to claim 1, wherein S6 includes: and filtering the optical flow value by adopting a statistical histogram mode.
5. The method for calculating the river surface flow velocity based on optical flow estimation according to claim 1, wherein, when the river flow direction is perpendicular to the cross section, S6 includes: filtering the light flow value according to the included angle range between the initial offset and the final offset of the characteristic points in the measured image matrix,
wherein
For the initial offset of the feature points in the side-stream image matrix,
the final offset of the characteristic point in the selected flow measurement image matrix is obtained.
6. The method for calculating the surface flow velocity of a river based on optical flow estimation according to claim 1, wherein the step S7 comprises:
calculating the river speed of each filtered characteristic point in a pixel coordinate system according to the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix
Wherein v is the river speed of each filtered feature point under the pixel coordinate system, w is the time of the video corresponding to the flow measurement image matrix, and | l | is the offset value;
selecting river speeds of the filtered characteristic points of the preset part under the pixel coordinate system to obtain an average value, and obtaining the river surface speed under the pixel coordinate system:
wherein N is the number of the filtered characteristic points of the selected preset part, V is the river surface speed under the pixel coordinate system,
and summing the river speed of each selected characteristic point.
7. The method for calculating the river surface flow velocity based on the optical flow estimation as claimed in claim 1, wherein the calculation manner of S9 is:
wherein (u) k ,v k ) (u) is the pixel coordinate of the k frame image k+1 ,v k+1 ) Is the pixel coordinate of the (k + 1) th frame image, (X) k ,Y k ) Coordinates in X and y directions which are world coordinates of the k frame image, (X) k+1 ,Y k+1 ) Coordinates in x-direction and y-direction of world coordinates of the (k + 1) th frame image, λ x And λ y Spatial resolution in the x-direction and y-direction, respectively.
8. The method for calculating the river surface flow velocity based on the optical flow estimation as claimed in claim 1, wherein the calculation manner of S10 is:
wherein V is the river surface velocity in the pixel coordinate system, lambda x And λ y The spatial resolution in the x direction and the spatial resolution in the y direction are respectively, V' is the river surface flow velocity in the world coordinate system, and fps is the frame rate.
9. A system for calculating river surface flow velocity based on optical flow estimation, comprising:
the video acquisition module is used for acquiring a river flow video by using a camera;
the video frame extraction module is used for extracting video frames from the collected river flow video according to a preset time interval and a frame interval, and all the extracted video frames form a flow measurement image matrix;
the preprocessing module is used for preprocessing the flow measurement image matrix;
an optical flow value calculation module for calculating an optical flow value,the method is used for carrying out feature detection and optical flow value calculation on the image in the preprocessed flow measurement image matrix by adopting an optical flow algorithm and combining a feature detection algorithm to obtain an optical flow value g of the measurement image matrix L ＝[l x ,l y ] T An optical flow value of the measured image matrix represents a movement of the feature points between image frames in the flow-measuring image matrix, wherein,
is a component in a direction parallel to the cross section of the river,
is a component in the direction vertical to the cross section of the river;
an offset calculating module for obtaining the offset of the characteristic point in the flow measurement image matrix based on the optical flow value of the flow measurement image matrix
Then calculating the offset value l of the characteristic point in the flow measurement image matrix according to the Euclidean distance,
sorting the offset values from high to low, and filtering the offset values of a preset proportion at the back of the sorting, wherein the offset value of the characteristic point in the flow measurement image matrix is the moving distance of the characteristic point between image frames in the flow measurement image matrix;
the optical flow value filtering module is used for filtering the optical flow values of the flow measurement image matrix;
the river surface speed calculating module under the pixel coordinate system is used for calculating the river speed of each filtered characteristic point under the pixel coordinate system according to the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix, and selecting the river speed of a preset part of the filtered characteristic points under the pixel coordinate system to obtain an average value so as to obtain the river surface speed under the pixel coordinate system;
the coordinate conversion module is used for converting the pixel coordinate of each frame of image in the flow measurement image matrix into a world coordinate;
the spatial resolution calculation module is used for calculating the spatial resolution of the image according to the pixel coordinates of two continuous frames of images and the world coordinates of the two continuous frames of images in the flow measurement image matrix after the optical flow value is filtered;
and the river surface flow velocity calculating module under the world coordinate system is used for calculating the river surface flow velocity under the world coordinate system by utilizing the spatial resolution of the image and the river surface velocity under the pixel coordinate system.
10. A computer-readable storage medium, on which a computer program is stored, which program, when executed, carries out the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211182996.3A CN115761563A (en) | 2022-09-27 | 2022-09-27 | River surface flow velocity calculation method and system based on optical flow measurement and calculation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211182996.3A CN115761563A (en) | 2022-09-27 | 2022-09-27 | River surface flow velocity calculation method and system based on optical flow measurement and calculation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115761563A true CN115761563A (en) | 2023-03-07 |
Family
ID=85350291
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211182996.3A Pending CN115761563A (en) | 2022-09-27 | 2022-09-27 | River surface flow velocity calculation method and system based on optical flow measurement and calculation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115761563A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116342656A (en) * | 2023-03-29 | 2023-06-27 | 华北电力大学 | Space-time image speed measurement method and device based on self-adaptive edge detection |
| CN116843725A (en) * | 2023-08-30 | 2023-10-03 | 武汉大水云科技有限公司 | River surface flow velocity measurement method and system based on deep learning optical flow method |
-
2022
- 2022-09-27 CN CN202211182996.3A patent/CN115761563A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116342656A (en) * | 2023-03-29 | 2023-06-27 | 华北电力大学 | Space-time image speed measurement method and device based on self-adaptive edge detection |
| CN116342656B (en) * | 2023-03-29 | 2024-04-26 | 华北电力大学 | Space-time image speed measurement method and device based on self-adaptive edge detection |
| CN116843725A (en) * | 2023-08-30 | 2023-10-03 | 武汉大水云科技有限公司 | River surface flow velocity measurement method and system based on deep learning optical flow method |
| CN116843725B (en) * | 2023-08-30 | 2024-01-05 | 武汉大水云科技有限公司 | River surface flow velocity measurement method and system based on deep learning optical flow method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107256225B (en) | Method and device for generating heat map based on video analysis | |
| CN115761563A (en) | River surface flow velocity calculation method and system based on optical flow measurement and calculation | |
| CN108416789A (en) | Method for detecting image edge and system | |
| CN108334847A (en) | A kind of face identification method based on deep learning under real scene | |
| CN111738995B (en) | RGBD image-based target detection method and device and computer equipment | |
| US20120328161A1 (en) | Method and multi-scale attention system for spatiotemporal change determination and object detection | |
| CN108596169B (en) | Block signal conversion and target detection method and device based on video stream image | |
| CN106600625A (en) | Image processing method and device for detecting small-sized living thing | |
| CN108537782B (en) | Building image matching and fusing method based on contour extraction | |
| CN107330373A (en) | A kind of parking offense monitoring system based on video | |
| CN110210350A (en) | A kind of quick parking space detection method based on deep learning | |
| CN103996049B (en) | Ship overlength and overwidth detection method based on video image | |
| CN110097536A (en) | Hexagon bolt looseness detection method based on deep learning and Hough transformation | |
| CN113610778B (en) | Bridge surface crack detection method and system based on semantic segmentation | |
| CN109376740A (en) | A kind of water gauge reading detection method based on video | |
| CN106127145A (en) | Pupil diameter and tracking | |
| CN106530281A (en) | Edge feature-based unmanned aerial vehicle image blur judgment method and system | |
| CN109685045A (en) | A kind of Moving Targets Based on Video Streams tracking and system | |
| CN110189375A (en) | A kind of images steganalysis method based on monocular vision measurement | |
| CN110135312B (en) | Rapid small target detection method based on hierarchical LCM | |
| CN110059634A (en) | A kind of large scene face snap method | |
| CN106530328A (en) | Method for detecting and smoothly following moving object based on video images | |
| CN110111352A (en) | One kind detecting false-alarm elimination method based on feature cascade SAR remote sensing images waters | |
| CN106651923A (en) | Method and system for video image target detection and segmentation | |
| CN115272876A (en) | Remote sensing image ship target detection method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |