JP4574657B2 - Method and apparatus for measuring water flow - Google Patents

Description

  The present invention relates to a method and apparatus for measuring the velocity and direction of ocean currents and other water currents.

Currently, a method using a radar device using the Doppler effect is used as a method of grasping the flow velocity and direction on the sea surface near the coastline (see FIG. 1). In these methods, facilities and the like are very expensive, and the purpose cannot be used otherwise.
Due to the Doppler effect, the surface flow velocity becomes V = C / fd (2f + fd) / cos θ.

C: Radio wave or ultrasonic velocity f: Transmission wave frequency fd: Doppler frequency V: Surface flow velocity θ: Angle between transmission / reception wave and water surface

In addition to the technique using the Doppler effect, a technique called a particle image velocimetry is proposed.
In this technique, fine particles that move at the same speed as the fluid are mixed in the fluid, and the velocity distribution of the fluid itself is obtained by measuring the velocity of the particles using an image measurement technique.

For example, in the invention of Japanese Patent Application Laid-Open No. 11-173889, a hot water drop is dropped on water flowing through a water channel, and the temperature information of the surface of the fluid on which the hot water drop is dropped is photographed at a predetermined time interval by an infrared camera. In the invention of Japanese Patent Laid-Open No. 11-173888, a water droplet is dropped on flowing water to cause disturbance on the water surface, and thereby a flow direction pattern generated on the water surface is photographed with a line sensor camera, and the signal is obtained. An image arranged two-dimensionally in time order is generated, and the flow velocity is obtained based on the inclination of the two-dimensional pattern information in the image.
Japanese Patent Laid-Open No. 11-173889 JP-A-11-173888

In the measurement method using the Doppler effect, it is possible to measure the flow velocity of the ocean current, but it is necessary to install a radar device on the coastline, and the equipment is very expensive. For this reason, it is only installed on some coasts, which prevents its spread.
In the particle image velocimetry, it is necessary to mix the particles chased by the camera into the water area to be measured. Therefore, even if it is possible to measure the flow velocity of a water channel that can drop particles from the bridge, it is not possible to measure the flow velocity of the ocean current.

This invention makes it a subject to obtain the measuring method and apparatus of the flow velocity which can measure the flow velocity of an ocean current, restraining installation cost.
More specifically, it is based on the following goals.
(A) Reduce equipment costs.
(B) The measurement algorithm is clear.
(C) System operation is not particularly difficult.
(D) The maintenance cost is not high.
(E) It should be possible to deal with disabilities without being an expert.

In the method of the present invention, the undulation pattern of the water surface generated on the water surface by the water flow or wave is photographed at a certain time interval, and the moving amount and moving direction of the undulation pattern in a certain region are measured to measure the flow velocity and the water flow. One or both of the directions are measured. As the video camera, a commercially available digital video camera can be used.
The method of the present invention does not track individual water particles. The average velocity of particles within a certain region (correlation window) is obtained. A correlation window image centered on the measurement point is cut out from two images (captured images) taken at a certain time interval, and the correlation that maximizes the similarity of the average pixel color value between the correlation window images This is a method in which the positional relationship between the windows is used as the average amount of movement of particles at that position (that is, the flow velocity).
For the evaluation of the similarity between images, it is preferable to use the following statistic called area correlation function. This calculation method calculates the area correlation value using image values that are similar for each frame that is adjacent to the front, back, left, and right, and uses the difference between the center positions of the frames that have obtained the highest correlation value as the amount of movement. Calculate the velocity and direction.
The area correlation value is calculated using an image in which low frequency components in the image are cut and the high frequency components are left.

The low frequency component represents a rough color plane, and it can be said that the closer the frequency is, the lower the frequency. The image in FIG. 2 is called an image with many low frequency components. That is, it means a solid image with few edges. It can be said that there are many artifacts and the town has many low-frequency components.
A high-frequency component refers to an image having a lot of high-frequency components that has many fine figures and outlines in the image (see FIG. 3). In a general photograph, when a dead tree or the like is photographed, it can be said that the image has a very high frequency component. The following images are images with a lot of high-frequency components as compared with the above-mentioned images.

ｆｍ：f画像の相関窓における平均画素色値（式1（数１）を参照）
gm：g画像の相関窓における平均画素色値（式2（数２）を参照）
δf：f画像の相関窓に対する標準偏差（式3（数３）を参照）
δg：g画像の相関窓に対する標準偏差（式4（数４）を参照）
δfg：f画像、g画像の相関窓に対する共分散値（式5（数５）を参照）
ρ：面積相関値（式6（数６）を参照）
正規化された相関係数ρは、-1から＋1の値となり、画像パッチが完全に一致する場合には＋1の相関値となり、全く相関の無い場合は0となる。ρ=-1は、同一画像内においてネガ・ポジの関係のような逆相関を表す場合である。（式6を参照） The exploration method in ocean current measurement is performed by the following calculation.
The f image (see FIG. 4) is a captured image of t seconds, and the g image (see FIG. 5) is a captured image of t seconds + 1/30 seconds. Also, in the following calculation formula:
f (x, y): f color value of the pixel at the location in the horizontal direction x and vertical direction y of the image
g (x, y): a color value of a pixel at a location in the horizontal direction x and the vertical direction y of the g image.

fm: Average pixel color value in the correlation window of f image (see Equation 1 (Equation 1))
gm: Average pixel color value in the correlation window of the g image (see Equation 2 (Equation 2))
δf: standard deviation for correlation window of f image (see Equation 3 (Equation 3))
δg: Standard deviation for correlation window of g image (see Equation 4 (Equation 4))
δfg: Covariance value for the correlation window of f and g images (see Equation 5 (Equation 5))
ρ: Area correlation value (see Equation 6 (Equation 6))
The normalized correlation coefficient ρ is a value from −1 to +1, and is a correlation value of +1 when the image patches completely match, and is 0 when there is no correlation. ρ = −1 represents a case where an inverse correlation such as a negative / positive relationship is represented in the same image. (See Equation 6)

The actual calculation is performed by the following method.
1. Fixing the image patch on the F image, g image patches on the image (N × N), is moved up and down and right and left by probing width D min 1 step, calculating the area correlation value 4D ² pieces of the above formula Then, the image patch having the highest correlation value is selected. It is determined that the fixed image patch in the f image has moved to the position of the selected image patch, and the movement amount is obtained by comparing the middle of both image patches.
2. One or a plurality of arbitrary points to be measured are designated on the captured image, and the flow velocity movement amount is obtained by the above method for each designated place.

The apparatus of the present invention includes a video camera for photographing a relief pattern on a water surface, an image processing program for performing processing for cutting out low frequency components from an image photographed by the video camera and leaving high frequency components, and similarity between images. It is composed of an arithmetic program for calculating a degree (area correlation value) and calculating a moving distance and a moving direction between images having high similarity.
Since the video camera is usually installed on the coast and shoots the water surface with a lens positioned obliquely downward, it is desirable to have a program for converting image data photographed obliquely into image data from the vertical direction.
In addition, since the sea surface changes its height due to tides, the lens tilt is automatically controlled to always shoot under the same conditions, and a zoom lens is used to keep the water surface shooting scale constant. It is preferable to do.

  Since this invention measures the undulation pattern of the water surface with a video camera and processes the image to measure either or both of the flow velocity and direction of the water flow, like the method using the topler effect It is not necessary to prepare expensive equipment such as a radar device, and the flow velocity and the flow direction can be measured at low cost. Further, since it is not necessary to mix fine particles and water droplets in the measurement water area, it is possible to measure the current velocity and direction of the ocean current.

  The method and apparatus of the present invention will be specifically described below.

(Image shooting)
Fix the video camera toward the surface of the water area to be measured, and shoot the relief pattern on the water surface. There is no limit to the shooting time. In extreme terms, it is possible to measure the flow velocity even with one-second shooting. However, when measuring ocean currents, it is preferable to always shoot because the ocean current changes in flow velocity and direction over time during the day.
A video camera having about 2 million pixels commercially available for home use can be used. The applicant used a digital video camera that was photographed at 30 frames per second during the experiment and recorded it in the file format DV format.
Next, in order to convert the DV format video into an image for each frame, image extraction is performed for each frame using video software, and the images (capture images) shown in FIGS. 6 and 7 are obtained. The image in FIG. 7 is the water surface after 1/30 second of the image in FIG. 6, and the images of each frame are similarly extracted.

(Image processing)
Next, the image of each frame is subjected to Fourier transform (FFT), and a high frequency component filter is set on the FFT image to cut low frequency components and leave high frequency components. The method is as follows.
For all the pixels of the captured image cut out every 1/30 seconds from the captured video, create a distribution of how many pixels of R, G, B color values (0 to 255), R, G, For each B, find the most color values. This is called a histogram. For the MAX_R value, MAX_G value, and MAX_B value of this histogram, threshold values are set under the following conditions to cut low frequency information.

(Threshold setting conditions)
In each image (t seconds and t + 1 seconds), in the range where nothing other than the water surface (sea surface) is shown, the histogram of the color values in that range (R, G, B values of each pixel, the most R, G, B distribution). The following processing is performed based on this histogram.
If the color value 255 occupies the maximum number of pixels in any of the histograms of R, G, and B, no processing is performed on the image. This is because when the most common color value is 255 in an entire image, it means that there are many places with no information at all.
In other cases, the following processing is performed.
1) Calculate RG and GB.
2) The average value (S) of R, G, B color values, which is the most common, is obtained by R + G + B / 3.
3) The larger absolute value of RG and GB obtained in 1) above is defined as (L).
4) Give the percentage in L / S x 100. This percentage is the spread of R, G, and B (distribution in the color value direction).
If the result of 4) is within 10%, the following processing is performed, and if it is 10% or more, processing is not performed.
If the average of the absolute values of RG and GB is 10 or less (color value), low frequency information for 32 pixels is cut, and if it is 10 or more (color value), low frequency information for 64 pixels is cut.

(Background technology of image processing)
Specifically, the Fourier transform uses “discrete cosine transform (DCT)”. Using a discrete cosine transform, the image in the block divided into 8x8 dots is decomposed into a total of 64 cosine curves and replaced with "how much are high-frequency and low-frequency components included?" . Here, “discrete” means “not continuous”, that is, digital. As shown in FIG. 3, an image in which “brightness” fluctuates when white and black are alternately present, which is the nature of the wave itself, and the concept of a wave can be applied to an image. . The frequency of this wave is called “spatial frequency”.
The spatial frequency is a numerical value representing the fineness of the periodic structure of the image. In the image of FIG. 2, white and black are distributed at wide intervals, which means that “the spatial frequency is low”. In the image of FIG. 3, on the contrary, white and black appear finely and alternately, and it can be said that “the spatial frequency is high”. If the entire image has a uniform color, the period is infinite, that is, the spatial frequency is zero. FIG. 2 shows an image with a low spatial frequency, and FIG. 3 shows an image with a high spatial frequency. Actually, since the image is spread in two dimensions, the graph is a three-dimensional graph with spatial coordinates on the X and Y axes and signal intensity on the Z axis.

  Next, the image data from which the low frequency component is cut is subjected to inverse Fourier transform (IFFT) and then binarized. By this processing, an image in which the contours of the relief pattern on the water surface are emphasized is obtained (FIGS. 8 and 9).

(Correlation window setting)
Next, in one image (capture image) that serves as a basis for measuring the flow velocity and direction, a certain region for searching for a region having a high similarity (area correlation value) after the next image is set. Hereinafter, this region is referred to as a “correlation window”. When setting the correlation window, the amount of noise present in the captured image is estimated from the signal-to-noise ratio (S / N ratio), and the optimum size of the correlation window is set.
The search width when searching for a region with a high area correlation value to be compared in the second image is set using the following calculation formula as a guide.
(Number of columns-Area correlation window size) / 2 <Number of columns / 3
(Number of lines-Area correlation window size) / 2 <Number of lines / 3
Area correlation window size: 2 ⁿ , but the correlation window size at sea level is considered to be optimal when n = 5-7 (32 × 32, 64 × 64, 128 × 128 is the optimal size) Based on 128x128.

(Calculation method of flow velocity)
On each frame file taken every 1/30 seconds obtained with a digital video camera, a correlation window is placed at the same location on the first frame (t seconds) and second frame (t + 1 seconds) images. The correlation window is set around the same (x1, y1) position on the image after t + 1 seconds with respect to the correlation window set in the column and line (x1, y1) on the t second image (FIG. 4). FIG. 5). Both the size of the selected correlation window and the search width are moved one pixel up, down, left, and right to calculate the area correlation value in the correlation window of the t second image and the t + 1 second image at each position (according to Equation 1). The difference between the center column / line (x2, y2) of the correlation window having the largest correlation value and the column / line (x1, y1) on the t-second image is the flow velocity direction and length. Divide this length by 1/30 second to get a flow rate per second.
In the calculation of the flow velocity, the moving distance on the computer is calculated in units of pixels, and the number of pixels is converted into units of meters in consideration of shooting conditions. Next, the correlation window at t + 2 seconds is searched with reference to the correlation window at t + 1 seconds, and the same processing is sequentially performed.

(Direction measurement)
This process is performed with a plurality of images (for example, 60) for each image by setting a plurality of correlation windows, and the results are displayed in an overlapping manner, and the center of the correlation window is connected to obtain the image of FIG. The vector connecting the centers of the correlation windows will display the flow direction.
The maximum search width in the above is set based on the resolution of one photographed pixel on the water surface, the maximum flow velocity to be observed, and the minimum flow velocity.

(Installation and calibration of video camera)
In the present invention, in order to accurately calculate the area correlation value of the correlation window, data obtained by photographing the water surface from a vertical position is required. If it is a waterway such as a river, it is possible to install the video camera lens at a position perpendicular to the surface of the water, but it is extremely difficult at sea, and usually a video camera must be installed on the coast. Therefore, the camera lens is at an oblique position with respect to the water surface (sea surface). Therefore, it is necessary to perform calculation processing so that an image photographed from an oblique position becomes an image at a vertical position.
Also, since the sea level changes due to tidal fluctuations, it is necessary to cope with the change.
Further, in order to measure a wide water surface with one camera, it is necessary to rotate the camera lens in the vertical and horizontal directions.
Therefore, when measuring ocean currents, calibration based on the installation conditions of the video camera is required.

In the present invention, the most important thing to note about the relationship between the flow velocity to be obtained and the camera specifications, sensor resolution, sensor size, shooting distance, camera focal length, Along, Cross direction angle is the angle in the Along direction. Yes (see Figure 11). Depending on this angle, it can be said that the flow rate with the required accuracy cannot be obtained. When it is applied in the actual field, it is necessary to perform precise ground survey and observe the camera position. In particular, it is necessary to obtain the distance to the sea surface with high accuracy using the tidal data of the sea surface from the imaging time. This data can be used as Z for creating a digital ortho of the sea surface.
The Along angle is preferably 90 ° to 65 °.

値として、Ｚ_０に補正する。
５．固定焦点距離のレンズを用いる場合は、その焦点距離に応じたレンズのラジアルディストーション補正を行なう。これは、既存の正確な座標値をもったフィールドを撮影して求める。或いは、ズームレンズの場合は、最大焦点と最小焦点の係数を求め適用する。そして、焦点距離の変化を記録する可能とする。 In order to determine the positional relationship between the digital video camera and the sea surface P (see FIG. 12), ground survey is performed to determine the center position and direction of the camera lens. The method is as follows.
1. The position (especially height) in the space of the installed digital video camera is accurately measured by ground survey. (Japanese geodetic system)
2. At the lens center position (X ₀ , Y ₀ , Z ₀ ) of the installed digital video camera, set (X ₀ , Y ₀ ) to 0, and for the height Z ₀ from the sea level, Ask and give.
3. When the direction of the digital video camera can be changed by the rotary head, the horizontal rotation angle φ and the vertical rotation angle ω are acquired from the encoder and counted from the encoder. There is no change in the height direction of the camera platform. It is necessary to be able to acquire the rotation angle of the pan head on which the video camera is installed with a personal computer.
Since the relationship between the image plane (positive surface), the lens center and the real space is considered in terms of projective transformation, the lens center is the origin in the camera coordinate system, and changes in the height direction must be considered. Absent.
4). The specific position (X, Y) on the sea surface is obtained by three-dimensional rotation conversion using the angle reflecting the rotation angle. Regarding the height direction, the difference in the height direction is reflected to reflect the tidal range at that time.

As the value is corrected to _{Z 0.}
5). When a lens having a fixed focal length is used, radial distortion correction of the lens is performed according to the focal length. This is obtained by photographing an existing field having an accurate coordinate value. Alternatively, in the case of a zoom lens, the coefficients of maximum focus and minimum focus are obtained and applied. Then, the change in focal length can be recorded.

Ｘ,Ｙ,Ｚ：水面上における実空間座標値（Ｚ値は、実際の海面高）
Ｘ_０，Ｙ_０，Ｚ_０：カメラレンズの実空間座標値で、今回はここを実空間の原点座標系とする。
ｘ、ｙ：撮像画像上での点Ｐの値（カラム、ライン）
ｃ：カメラレンズ焦点距離
ａ_１１〜ａ_３３:κ、φ、ωを用いた３行×３行の回転行列
回転行列に関しては、式９（数９）にて詳細を記述する。

X, Y, Z: Real space coordinate value on the water surface (Z value is the actual sea level)
X ₀ , Y ₀ , Z ₀ : real space coordinate values of the camera lens, this time this is the origin coordinate system of the real space.
x, y: value of the point P on the captured image (column, line)
c: Camera lens focal lengths a _{11 to} a ₃₃ : Details of the rotation matrix rotation matrix of 3 rows × 3 rows using κ, φ, ω are described in Expression 9 (Equation 9).

Expression 7 (Equation 7) is a collinear conditional expression describing the relationship between the spatial coordinates X, Y, Z and the coordinates x, y on the image three-dimensionally. This formula is constructed based on collinear conditions in aerial photogrammetry, and is widely used in aerial photogrammetry. In this ocean current survey, most of the camera installation locations are taken from an oblique direction, and it can be said that there are few things that can be taken with a direct view. The pixel interval of an image taken from an oblique direction is severer as it is farther away, and the resolution is lower. Therefore, in Equation 7, X ₀ , Y ₀ = 0 is set with X ₀ , Y ₀ , Z ₀ as the camera origin. Z ₀ is the height from the center of the camera lens to the sea surface. The inclination of a _{11 to} a ₃₃ is divided into a case where the Along direction is considered and a case where the three directions are considered. In the case of 3 directions, the rotation angle of the camera head is measured with an encoder and loaded into the PC (see Fig. 13). X and Y corresponding to the inclination element and x and y at the four corners are obtained. A maximum value and a minimum value X, Y are obtained from X and Y at the four corners to constitute an arbitrary XY plane. Next, a digital orthographic projection image of the sea surface on the XY plane is created.

The image obtained by the digital video camera is a projective transformation image obtained by mapping a three-dimensional space into a two-dimensional space by central projection using a lens. In the image of the sea surface, there are elevation differences due to waves, but there are few discontinuous vertical bulges (vertical objects such as urban buildings), and the sea surface is composed of continuous surfaces. Suppose that Although it depends on the imaging position, even if a concealed part due to the wave bulge occurs, the effect is statistically small if the image is considered as a plane rather than a point. It is considered that there is almost no need to consider a discontinuous image. Therefore, in order to specify the amount of movement of the sea surface, a digital orthographic projection map of the sea surface is created from an image taken obliquely.

The apparatus of the present invention will be described below.
A video camera unit 1 having a video camera for photographing the undulating pattern on the water surface at regular time intervals, a video processing unit 2 for processing and analyzing an image photographed by the video camera, a site server 4 and a management server 5 Configure (FIG. 14).

(Site server)
The site server 4 is the center of the system of the present invention and has the following functions (FIG. 15).
1) Interface function with management server 5 and user terminal.
2) Data storage function Records various data generated by the video processing unit, camera position information input from the camera control unit, and the like.
3) Control Function of Video Processing Unit The camera position information, ROI selection signal, and calculation setting signal are output to the video processing subsystem 21 of the video processing unit 2, and the calculation status and calculation result output from the video processing subsystem 21 Record video files, all still image files, and live video.
4) Control function of the camera control unit The camera control signal is output to the camera control subsystem 21 of the video camera unit 1 and the camera position information output from the camera control subsystem 21 is recorded.
This camera control signal performs control in accordance with the conditions described in paragraph number 0024 and thereafter.

(Video processing part)
The video processing unit has the following functions.
The image data output from the video signal transfer unit of the video camera unit 1 is input, cut out into a still image, the low-frequency component is cut out from the image to leave the high-frequency component, and a certain area is added to the first image. Identify and calculate an average pixel color value of the first specific area, calculate an average pixel color value in a plurality of areas in the vicinity of the identified area in the second image, and identify the first image The area correlation value between the average pixel color value in the selected area and the plurality of average pixel color values in the second image is calculated, and the area having the maximum area correlation value in the second image is specified as the second specific area The comparison area to be identified is specified, and the direction and speed of the water flow are calculated based on the position data of the first to nth specific areas. Then, the video data, calculation results, etc. are output to the site server 4.

(Camera control unit)
The camera control unit has the following functions.
1) A camera control signal is output to the video signal transfer unit based on an instruction from the site server.
2) Output the camera position information input from the video signal transfer unit to the site server.

(Video camera part)
The video camera unit 1 includes a video camera, a pan head that can be remotely controlled and that can output angle information, and a data exchange device with a video signal transfer unit.
When the direction of the video camera can be changed by the rotating pan head, the horizontal rotation angle φ and the vertical rotation angle ω are obtained from the encoder and counted from the encoder, and the rotation angle of the pan head on which the video camera is installed is counted as a personal computer. Need to be able to get in.
Specifically, it is linked to a program that performs the processing described in paragraph 0024.

The present invention makes it possible to measure the direction and flow velocity of ocean currents, which has been difficult to measure with a simple device, and has industrial applicability.

Illustration of the Doppler effect Illustration of low frequency image and wavelength Illustration of high frequency image and wavelength Illustration of the area correlation method Explanatory drawing which similarly shows the search range after t + 1/30 seconds Video image of the first frame Video image of the second frame (after t + 1/30 seconds) Reverse FET processed image Binary image after reverse FET Diagram showing measurement results of flow velocity and flow direction Diagram showing relationship between camera and shooting resolution Diagram showing the relationship between camera lens center, positive photographic surface and real space The figure which shows the inclination in the three-dimensional space of the camera The block diagram which shows the structure of the apparatus of this invention Block diagram showing relationship between video processing unit, camera control unit and site server

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Video camera part 2 Video processing part 3 Camera control part 4 Site server 5 Management server 21 Video processing subsystem 31 Camera control subsystem

Claims (3)

Video camera undulations on the water surface at regular intervals,
Cut low-frequency components in the captured image, leave the high-frequency components,
Identify a certain area in the first image and calculate the average pixel color value of the first specific area,
Calculating an average pixel color value in a plurality of regions in the vicinity of the identified region in the second image;
Calculating an area correlation value between an average pixel color value in the specified region of the first image and a plurality of average pixel color values in the second image;
In the second image, the region having the maximum area correlation value is set as a second specific region,
It is characterized by detecting the direction and / or velocity of the water flow by repeating this and tracking the first to nth specific areas,
Measuring method of water flow A video camera for shooting undulations on the water surface at regular intervals;
Image processing means for performing a process of cutting a low frequency component from an image photographed by a video camera and leaving a high frequency component;
A certain area is specified in the first image, the average pixel color value of the first specific area is calculated, and the average pixel color value in a plurality of areas in the vicinity of the specified area in the second image is calculated. Calculating an area correlation value between an average pixel color value in the specified region of the first image and a plurality of average pixel color values in the second image, and the area correlation value in the second image is maximized. Comparison area specifying means for specifying the area as the second specific area;
A water flow measuring device comprising water flow calculating means for calculating the direction and / or speed of the water flow based on the position data of the first to nth specific regions. The video camera is attached to a pan head with an angle control device that controls the tilt angle of the camera lens according to the change in water surface height.
The water flow measuring device according to claim 2.

Images