KR102193108B1 - Observation method for two-dimensional river mixing using RGB image acquired by the unmanned aerial vehicle - Google Patents

Abstract

The present invention relates to a method for measuring a two-dimensional river mixing behavior using an RGB image obtained from an unmanned aerial vehicle, which comprises the steps of: (a) obtaining RGB image data of a river using an unmanned aerial vehicle while performing a tracer experiment; (b) extracting, by an image coordinate tracking module, an image coordinate of a ground reference point for a subsequent frame with respect to the image coordinate of the ground reference point for a first frame of the RGB image data; (c) constructing, by a coordinate conversing module, a certain number of pairs of coordinates using the image coordinate of the ground reference point at each frame extracted in the step (b) and a UTM coordinate of the corresponding ground reference point, and projecting the pixel intensity of all the image coordinates of the RGB image data onto the pixel intensity of a UTM coordinate system; and (d) constructing, by an artificial neural network constructing module, an artificial neural network using a concentration value of a tracer material measured by a concentration measuring device and the pixel intensity of the RGB image, and converting the pixel intensity of a point where the concentration measuring device is not installed at each frame of the RGB image data into the concentration value of the tracer material through the constructed artificial neural network. According to the present invention, a relationship between the concentration value measured by the contact-type concentration measuring device at a small number of points and the pixel intensity of the RGB image for the corresponding concentration value is constructed through an artificial neural network, thereby temporally and spatially extracting concentration distribution for an area where the contact-type concentration measuring device is not installed from high-resolution RGB video image data.

Description

{Observation method for two-dimensional river mixing using RGB image acquired by the unmanned aerial vehicle}

The present invention relates to a two-dimensional stream mixing behavior measurement method using an RGB image acquired from an unmanned aerial vehicle, and more particularly, a concentration value measured by a contact-type concentration measuring device at a few points and RGB for the corresponding concentration value. A two-dimensional stream mixing behavior measurement method that allows the spatial and temporal extraction of the density distribution for the area where the contact density measuring device is not installed by establishing the relationship between the pixel intensity of the image through an artificial neural network. will be.

As accidents of hazardous chemicals are frequently leaked from urban rivers or rivers located near industrial complexes, rapid prediction of the stream mixing process of pollutants is required for effective disaster prevention plans and initial actions. In the event of a water pollution accident in which dissolved hazardous substances are introduced from a river, the pollutants show a mixing process of various characteristics according to the hydraulic conditions and topographic shape of the river, and for effective analysis of the mixing pattern for each characteristic, It is necessary to construct a vast amount of observational data on the star mixture characteristics.

As a conventional experimental method for observing the mixing process of dissolved pollutants in a river, trace substances such as fluorescent substances, salt water mixed solutions, or radioactive isotopes are injected into the stream and then contact-type measuring instruments are used. There is a tracer test method that measures the concentration of tracer substances.

However, in the conventional tracer test method using a contact-type measuring device, the concentration of the tracer substance can be acquired only as time series data at a fixed location where the measuring equipment is installed, and there is a limit to measuring the spatial distribution of tracer substance concentration. Therefore, studies that analyze the mixing process of streams using data obtained from the conventional tracer test method have also been mainly conducted through time-series analysis of concentration data.

Meanwhile, as a method of measuring the spatial distribution of the concentration of pollutants introduced into the aquatic environment, there is a method of extracting the concentration and distribution of a target substance from satellite images acquired from satellites, which are actively used in coastal and marine areas. However, the method of using satellite imagery has limitations in observing the mixing process of pollutants in the river environment due to the spatial resolution of the image data of about several hundred meters and the orbital period of the satellite that takes about tens of days.

The prior art Republic of Korea Patent Publication No. 10-1866239 (announced on June 12, 2018) relates to a method of monitoring the water quality environment with an image taken in real time at a set time and location at a river outlet where the water quality environment is suspected, (a ) When points for monitoring the water quality environment of rivers or outlets are collected and selected by the manager and information on the monitoring points is input to the manager terminal of the water quality environment monitoring system, the flight control system program installed on the manager terminal is executed. Transmitting flight information to a drone prepared at a remote monitoring point site through a wireless communication network; (b) the drone is a step of collecting image data by photographing a monitoring target point with a thermal imaging camera while flying at a set monitoring target point after starting a flight with flight information of a flight control system program; (c) the drone transmitting the collected image data to a water quality environment monitoring system through a wireless communication network in real time, and receiving and storing the image data from a central DB server of the water quality environment monitoring system; (d) The analysis server of the water quality environment monitoring system compares and analyzes the thermal image temperature of the image data of the monitoring target point stored in the central DB server and the image data of the previous monitoring target point that has already been photographed and stored, and changes in the set ratio Determining whether it has occurred; (e) If there is no change as a result of the analysis of the analysis server, the drone is allowed to fly to the next set monitoring target point or return to the point before flight by the control of the flight control system program of the manager terminal, and analysis of the analysis server If a change has occurred as a result, a configuration including the step of transmitting alarm information to the manager terminal and the monitoring station terminal in the field through a wireless communication network is disclosed. However, there is a limit to quantitatively calculating the spatiotemporal concentration data of the target solute from the change in the pixel intensity of the image.

Republic of Korea Patent Publication No. 10-1866239 (announced on June 12, 2018, title of invention: water quality environment monitoring method using a drone)

The present invention was conceived to solve the above problems, and an object of the present invention is to artificially analyze the relationship between the density value measured by the contact density measuring device at a few points and the pixel intensity of the RGB image for the corresponding density value. By constructing through a neural network, it is possible to extract the concentration distribution in a region where a contact-type concentration measuring device is not installed, from high-resolution RGB video image data, to obtain the spatial distribution of tracer material, which was difficult to observe in the conventional tracer experiment method. It is to provide a two-dimensional stream mixing behavior measurement method using an RGB image acquired from an unmanned aerial vehicle that enables more precise analysis of the mixing behavior of pollutants in natural rivers.

In order to achieve the above object, the present invention includes the steps of: (a) performing a tracer experiment and acquiring RGB image data of a river using an unmanned aerial vehicle; (b) extracting, by an image coordinate tracking module, the image coordinates of the ground reference point for subsequent frames based on the image coordinates of the ground reference point for the first frame of the RGB image data; (c) The coordinate conversion module constructs a certain number of coordinate pairs from the image coordinates of the ground reference point in each frame extracted in step (b) and the UTM coordinates of the corresponding ground reference point, and the pixel intensity of all the image coordinates of the RGB image data. Projecting the pixel intensity on the UTM coordinate system, and (d) the artificial neural network building module constructs an artificial neural network using the concentration value of the tracer material measured by the concentration measuring device and the pixel intensity of the RGB image, and It characterized in that it comprises the step of converting the pixel intensity at a point where the density measuring device is not installed in each frame of the RGB image data through a neural network into a concentration value of the tracer material.

In addition, in the present invention, the step (a) is a polytetrafluoroethylene (PTFE) panel for installing a certain number of ground reference points around the measurement section of a river, measuring UTM coordinates of the ground reference point, and obtaining reference pixel intensity for sunlight. It includes installation, installation of a certain number of concentration measuring devices to measure the concentration of the tracer material, rhodamine WT, and injection of the tracer material, rhodamine WT, and simultaneous RGB imaging.

In addition, in the present invention, in the step (b), the image coordinate tracking module extracts a partial image of a certain area around the image coordinates corresponding to each ground reference point from the first frame of the image, and then also in the image for each frame of the image. By extracting the partial image for the same region from which the partial image was extracted in the first frame, the following equation,

는 첫번째 프레임에서 추출된 부분영상의 평균 화소강도,

는 계산하고자 하는 프레임에서 추출된 부분영상의 평균 화소강도)으로부터 첫번째 프레임에서 추출한 부분영상과 각 프레임에서 추출된 부분영상 간의 정규상호상관계수가 가장 높은 지점의 이동거리를 탐색함으로써 각 지상기준점의 영상좌표 이동거리를 계산하는 것이다.(Here, NCC is the normal correlation coefficient, I is the pixel intensity of the partial image extracted from the first frame, i and j are the image coordinates of the partial image extracted from the first frame, and I _tmp is the extracted part from the frame to be calculated. Pixel intensity of the image, △X and △Y are the moving distance between two partial images,

Is the average pixel intensity of the partial image extracted from the first frame,

Is the image of each ground reference point by searching for the moving distance of the point with the highest normal correlation between the partial image extracted from the first frame and the partial image extracted from each frame from the average pixel intensity of the partial image extracted from the frame to be calculated. It is to calculate the coordinate travel distance.

In addition, in the present invention, in the step (c), in order for the coordinate conversion module to project the pixel intensity of all image coordinates of the RGB image data onto the pixel intensity of the UTM coordinate system, the established coordinate pairs and the following equations,

(Where x and y are UTM coordinates, X and Y are image coordinates, P ₁ ~ P ₈ are conversion factors) to calculate the transformation coefficient, and the image coordinates (X, By substituting Y) into the above equation, UTM coordinates (x,y) in all frames are calculated.

In addition, in the present invention, the relationship between the pixel intensity extracted after extracting the pixel intensity from the UTM coordinate corresponding to the installation location of the concentration measuring device and the concentration value (target value) of the tracer material measured by the concentration measuring device at the same time is indicated, and the concentration measuring device is Equation for the artificial neural network of step (d) that can calculate the concentration of the tracer material at the point where it is not installed through the pixel intensity of the image is,

은 추적자물질의 농도 추정치(출력값), W_jk는 은닉층과 출력층 간의 가중인자, b_k는 출력층의 편향인자)이되, 인공신경망의 입력자료는 영상 내 PTFE 패널에서 취득된 태양광 참조값으로 나누어 정규화시킨다.(Where x _i is the input data of the artificial neural network, DN is the pixel intensity of the image acquired with the rhodamine WT introduced, DN ₀ is the base pixel intensity of the image acquired before the rhodamine WT was introduced, and DN _s is Pixel intensity in the PTFE panel in the image, R, G, B are the bands of digital images, H _j is the value of the hidden neuron, W _ij is the weighting factor between the input layer and the hidden layer, b _j is the deflection factor of the hidden neuron, δ _j is the activation function of the hidden layer,

Is the estimated concentration of tracer material (output value), W _jk is the weighting factor between the hidden layer and the output layer, b _k is the deflection factor of the output layer), and the input data of the artificial neural network is normalized by dividing it by the solar reference value obtained from the PTFE panel in the image. .

In addition, in the present invention, in order to train the artificial neural network, a certain number of data pairs consisting of input data (normalized pixel intensity of the image coordinate at the concentration measuring device location) and a target value are constructed, and some of the data pairs are used as training data. Divide and input, but the error that occurs when the artificial neural network operation is performed by setting a random value as an initial value for the weighting factor and the bias factor of each neuron is the following equation, which is the cumulative square error for all training data,

은 m번째 훈련자료에 대한 추적자물질의 농도 추정값, C_0(m)은 농도측정기기에 의해 측정된 m번째 추적자물질의 농도값)으로 평가하고, 경사하강법을 이용하여 누적제곱오차가 최소가 되도록 가중인자와 편향인자를 보정한다.(Where E is the cumulative square error, M is the total number of training data,

Is the estimated value of the tracer substance concentration for the m-th training data, C _{0 (m)} is the concentration value of the m-th tracer substance measured by the concentration measuring device), and the cumulative square error is minimum using the gradient descent method. Correct the weighting factor and bias factor as much as possible.

The two-dimensional stream mixing behavior measurement method using the RGB image acquired from the unmanned aerial vehicle of the present invention, as discussed above, is the density value measured by the contact density measuring device at a few points and the pixel intensity of the RGB image for the density value. By establishing the relationship between them through an artificial neural network, it is possible to extract the concentration distribution in the area where the contact-type concentration measuring device is not installed from high-resolution RGB video image data, so that the spatial distribution of tracer material, which was difficult to observe in the conventional tracer test method. It is possible to obtain a more precise analysis on the mixing behavior of pollutants in natural rivers.

1 is a view showing an overall flowchart of a method for measuring a two-dimensional river mixing behavior according to the present invention.
2 is a view showing an experimental channel, which is an embodiment in which a method for measuring a two-dimensional river mixing behavior according to the present invention is performed.
Figure 3 is a schematic diagram showing the concept of a two-dimensional river mixing behavior measurement method according to the present invention.
4 is a view showing a process of tracking the image coordinates of the ground reference point in the method for measuring the two-dimensional river mixing behavior according to the present invention.
5 is a view showing a result of converting an image coordinate system to a UTM coordinate system in a method for measuring a two-dimensional stream mixing behavior according to the present invention.
6 is a diagram showing the structure of an artificial neural network used in the present invention.
7 is a view showing a graph comparing a concentration value converted from a pixel intensity of an image and a measured concentration value in a method for measuring a two-dimensional stream mixing behavior according to the present invention.
8 is a diagram showing a spatial distribution of concentration converted from pixel intensity of an image in a method for measuring a two-dimensional stream mixing behavior according to the present invention.
9 is a block diagram showing an embodiment of a system related to a method for measuring a two-dimensional river mixing behavior according to the present invention.

A preferred embodiment of the present invention configured as described above will be described in detail with reference to the accompanying drawings. Note that the accompanying drawings and the description with reference thereto are illustrated in order to be easily understood by those of ordinary skill in the art with respect to the present invention, and are not intended to limit the spirit and scope of the present invention. You will have to do it.

The use of commercial unmanned aerial vehicles is becoming popular among the general public through continuous technology development and dissemination for unmanned aerial vehicles, and applications are being actively sought in various industrial fields. Commercial unmanned aerial vehicles in recent years are equipped with high-resolution RGB digital cameras, and RGB digital cameras record visible light that can be perceived by human visual ability as pixel intensity divided into three bands according to the wavelength range of light. The use of unmanned aerial vehicles has the advantage of being able to acquire high-resolution digital images in space and time at a desired time and place at low cost. Hereinafter, the present invention acquires an RGB digital image using an unmanned aerial vehicle while performing a tracer experiment method using a fluorescent substance as a tracer substance, and converts the image data on the tracer substance into high-resolution spatiotemporal density data using an artificial neural network technique. This is how to convert.

9 is a block diagram showing an embodiment of a system related to a method for measuring a two-dimensional stream mixing behavior according to the present invention. The stream mixing behavior measuring apparatus 10 converts the acquired RGB digital image into a digital image using an artificial neural network technique. By converting the pixel intensity into a density value, it measures the spatiotemporal and high-resolution river mixing process, and searches the moving distance of the point with the highest normal correlation coefficient between the partial image extracted from the first frame of the image and the partial image extracted from each frame. The image coordinate tracking module 11 extracts the image coordinates of the ground reference point for subsequent frames based on the image coordinates of the ground reference point for the first frame of the image, and a coordinate conversion module that projects all pixel intensities in the image onto the UTM coordinate system. (12), an artificial neural network construction module that constructs an artificial neural network using the concentration value of the tracer substance measured by the concentration measuring device and the pixel intensity of the RGB image, and converts the pixel intensity of the RGB image into the concentration value of the tracer substance (13) ). That is, the image coordinate tracking module 11, the coordinate transformation module 12, and the artificial neural network construction module 13 are technical means for implementing the present invention on a computer, and construct an image coordinate tracking unit, a coordinate conversion unit, and an artificial neural network. You can also name each by wealth.

The stream mixing behavior measurement device 10 includes a server, a desktop, a laptop computer, a portable terminal, or the like, and stores software for measuring a two-dimensional stream mixing behavior using an RGB image acquired from an unmanned aerial vehicle.

In addition, it is preferable to store the data calculated or input/output by the river mixing behavior measurement device 10 in a separate storage device 20. The stream mixing behavior measurement device 10 may include a storage device 20.

A method for measuring a two-dimensional river mixing behavior using an RGB image acquired from an unmanned aerial vehicle according to the present invention made as described above will be described with reference to the flowchart of FIG. 1.

First, performing a conventional tracer test method and acquiring RGB image data using an unmanned aerial vehicle. A ground reference point at the periphery of the measurement section of the river while performing the conventional contact-based tracer test method using rhodamine WT fluorescent material. And a polytetrafluoroethylene (PTFE) panel to obtain reference pixel intensity for sunlight and UTM coordinate measurement of the ground reference point (S10).

FIG. 2 shows an experimental channel in which a two-dimensional stream mixing behavior measurement method according to the present invention is performed, in which a 20% solution of rhodamine WT, a fluorescent material, is used as a tracer material, and of rhodamine WT introduced into the flow. The concentration is measured at a total of 15 points by installing 5 concentration measuring devices on each of the 3 cross-sections, and one polytetrafluoroethylene (PTFE) panel and the actual coordinate system of the pixel intensity to obtain a reference value of the pixel intensity against sunlight. Twelve ground reference points are additionally installed for image projection, and the actual coordinates of each ground reference point are surveyed with a UTM coordinate system using RTK-GPS.

For reference, the experimental waterway in Fig. 2 is the river test center A3 test waterway of the Korea Institute of Construction Technology, located in Andong-si, Gyeongsangbuk-do. It is a connected full-scale outdoor experiment open channel, of which the experiment of one embodiment of the present invention was performed in the section corresponding to the meandering degree 1.7, and the repair conditions during the experiment were flow rate 2.02 ㎥/s, average flow rate 0.63 m/s, and average depth It was measured to be 0.61 m and an average lower width of 6.29 m, and the experimental data performed in the experimental channel were used in the measurement process of the two-dimensional river mixing behavior described later.

In the present invention, RGB digital video images are recorded using an unmanned aerial vehicle at the same time as performing the conventional tracer test method using a tracer material, and the unmanned aerial vehicle is approximately 65 m from the ground in order to record the entire range of the experimental channel section in the image. Fly in the air. In addition, in the present invention, the shooting setting of the RGB digital camera mounted on the unmanned aerial vehicle is preferably set manually to prevent the pixel intensity from being automatically corrected from the amount of ambient light, and polytetrafluoroethylene (PTFE) in the image The ISO value of 100 and the shutter speed of 1/2000 second are set so that the pixel intensity of the panel does not exceed the range of the maximum pixel intensity that can be recorded in an RGB digital image. The RGB digital image is photographed as a video image of 24 frames per second with 3,840×2,160 pixels, and the RGB video image photographed by the unmanned aerial vehicle is photographed for 987 seconds, and the total number of frames is 23,688. 3 is a schematic diagram showing the general concept of RGB image acquisition using an unmanned aerial vehicle at the same time as the conventional tracer test method.

Next, the pixel intensity of the RGB image is projected onto the UTM coordinate system. Using an RGB digital camera mounted on an unmanned aerial vehicle, an RGB video image is acquired so that the fluorescent material in the experiment section and the ground reference point exist in the image together. Then, the image coordinates for each ground reference point are tracked in each frame of the RGB video image using the normal cross-correlation method (S20), and the image coordinates in each frame of the ground reference point tracked above and the actual measured ground reference point are The perspective projection transformation equation is constructed using the coordinate pairs in each frame composed of coordinates, and the pixel intensity of the digital image for the image coordinates in each frame is projected to the pixel intensity of the actual coordinate system using the constructed perspective projection transformation equation ( S30).

In the video image captured by the unmanned aerial vehicle, the experimental waterway and surrounding objects are randomly moved within each frame of the image due to the vibration, wind, and GPS position error of the unmanned aerial vehicle. In order to easily extract the pixel intensity value, it is necessary to fix the arbitrary movement of the object in the image at a specific position. In order to calculate the spatial scale and feed rate of the tracer distribution in the mixing process, the image coordinates are converted to UTM according to the metric system. It needs to be converted to coordinates.

Accordingly, the present invention automates the extraction of the image coordinates of the ground reference point for subsequent frames based on the image coordinates of the ground reference point for the first frame. In the first frame of the image, that is, the frame of the image corresponding to the injection time of the rhodamine WT, the image coordinates corresponding to each ground reference point (the number of ground reference points in an embodiment of the present invention: 12) (for reference, the image coordinate is a pixel A partial image of 101×101 pixels is extracted centering on (may correspond to), and then a partial image is extracted from the first frame in the image for each frame of the image (total number of frames: 23,688 in an embodiment of the present invention). A partial image is extracted for the same image coordinate area. The image coordinate movement amount of each ground reference point can be calculated by searching the moving distance of the point with the highest normal correlation correlation between the partial image extracted from the first frame and the partial image extracted from each frame through the 2D normal correlation method. The equation of the dimensional normal intercorrelation method is shown in Equation 1 below.

는 첫번째 프레임에서 추출된 부분영상의 평균 화소강도,

는 계산하고자 하는 프레임에서 추출된 부분영상의 평균 화소강도이다.Here, NCC is the normal correlation coefficient, I is the pixel intensity of the partial image extracted from the first frame, i and j are the image coordinates for the partial image (101×101 in the embodiment of the present invention) acquired in the first frame, I _tmp is the pixel intensity of the partial image extracted from the frame to be calculated, △X and △Y are the moving distance between the two partial images,

Is the average pixel intensity of the partial image extracted from the first frame,

Is the average pixel intensity of the partial image extracted from the frame to be calculated.

The result of the two-dimensional normal correlation coefficient indicates the degree of cross-correlation for the moving distance between the two partial images being compared, and the moving distance indicating the highest mutual correlation coefficient corresponds to the moving distance in the image coordinates of the ground reference point between the two frames being compared. . Since the image coordinates for each ground reference point in the first frame are known, the image coordinates for the ground reference point in the frame to be calculated can be obtained by adding the moving distance searched by the normal cross-correlation method from the ground reference point image coordinates in the first frame. have.

Accordingly, the image coordinate tracking module 11 uses Equation 1 to search the moving distance of the point with the highest normal correlation coefficient between the partial image extracted from the first frame and the partial image extracted from each frame, and search for such a moving distance. Is a program coded with an algorithm directly through a programming language to be finally executed by a computer. In other words, such a program includes the above equation (1) and is stored in the river mixing behavior measurement device 10 or the storage device 20 The coordinate tracking module 11 searches for a moving distance of a point with the highest normal correlation coefficient using the program input and stored in the storage device 20.

As described above, by tracking the image coordinates for a total of 12 ground reference points in each frame of the RGB digital video image, 12 coordinate pairs (UTM coordinates (x, measured from 12 ground reference points) together with the UTM coordinate system of the corresponding ground reference point) are tracked. y), the image coordinates (X,Y) corresponding to each ground reference point) are constructed. In the present invention, perspective projection transformation is used to project all pixel intensities in an image (pixel intensity is included as attribute data in each image coordinate) onto the UTM coordinate system, and the equation for this is as Equation 2 below.

Here, x and y are UTM coordinates, X and Y are image coordinates, and P ₁ to P ₈ are transformation coefficients. The transformation coefficient in each frame is 12 pairs of coordinates in each frame collected in the above process (the image coordinates of each frame of the ground reference point can be known through the normal correlation coefficient, and the UTM coordinates of the ground reference point are actually measured coordinates. Is known) can be obtained by substituting in Equation 2 above. Here, the UTM coordinate values for each of the 12 ground reference points are constant values for all frames, but as described above, the image coordinates corresponding to each ground reference point in the image acquired by the arbitrary movement of the unmanned aerial vehicle are each frame. It becomes a variable value.

In the end, since the transformation coefficients in each frame as well as the ground reference point were obtained, substituting the image coordinates (X,Y) in all frames together with the transformation coefficients into Equation 2, the UTM coordinates (x,y) in all frames are known. You will be able to. That is, using Equation 2, the pixel intensity of a specific image coordinate is projected to the pixel intensity of the UTM coordinate system. 4 shows a process of tracking the image coordinates of the ground reference point in each frame, and FIG. 5 shows the result of projecting the image coordinate system of the RGB digital image captured by the unmanned aerial vehicle onto the UTM coordinate system.

That is, in the present invention, the coordinate conversion module 12 projects all the pixel intensities in the image onto the UTM coordinate system using Equation 2, and this projection process is carried out by an algorithm directly through a programming language in order to be finally performed by a computer. It is a coded program, in other words, such a program is stored in the river mixed behavior measurement device 10 or the storage device 20 while including the equation (2), and the coordinate conversion module 12 is input and stored in the storage device 20. All pixel intensities in the image are projected onto the UTM coordinate system using a program.

Next, the step of constructing an artificial neural network model using the density value measured by the contact density measuring device and the pixel intensity of the RGB image, R, G, and B at the actual coordinate points of the point where the contact density measuring device is installed. After extracting the intensity of each pixel of the band and normalizing it to the reference pixel intensity of sunlight, constructing and training an artificial neural network with the concentration value measured by a contact density measuring device at the same time, and then using the trained artificial neural network in each frame. The pixel intensity of the entire image of is converted into a density value (S40).

In one embodiment of the present invention, the RGB digital video image captured during the conventional tracer test method is recorded in each frame of the image at a time resolution of 1/24 sec. Each pixel intensity of each R, G, B band for a specific point in the image has a spatially different value depending on the basal water quality, water depth and bed material of the experimental section. The inflow of rhodamine WT causes a spatial change in the quality of the basal water, and as a result, the pixel intensity at the point where the rhodamine WT is introduced is different from the pixel intensity before the inflow of rhodamine WT, and the difference in the changed pixel intensity is rhodamine. Correlated with the concentration of WT. However, since the relationship between the concentration of rhodamine WT and the pixel intensity is nonlinear and spatially non-homogeneous, it is difficult to express it with a simple regression equation.

Therefore, in the present invention, an artificial neural network model is used to establish a relationship between the concentration of rhodamine WT and the pixel intensity. After extracting the pixel intensity from the UTM coordinates corresponding to the installation location of the contact-type density measuring device, an artificial neural network capable of representing the relationship between the extracted pixel intensity and the concentration value of the tracer substance measured by the contact-type density measuring device at the same time is constructed. It is possible to measure the concentration of the tracer material at the point where the concentration measuring device is not installed within the image through the pixel intensity of the image.

The structure of the artificial neural network for converting the pixel intensity into the concentration value of the tracer material for each frame consists of an input layer, a single hidden layer, and an output layer, and the input factor (input data) of the input layer is each in the state where rhodamine WT is introduced. The pixel intensity of the R, G, B bands for the point and the baseline pixel intensity of the R, G, and B bands for each point before the rhodamine WT is introduced are used for a total of six, and here, before the influx of rhodamine WT. The base pixel intensity is used to reflect the effects of water depth and bed material, which can be spatially different even in the absence of rhodamine WT, to the artificial neural network. The above six input data are normalized by dividing by the solar reference value obtained from the PTFE panel in the image.

The target value is the concentration value of the tracer material measured by the contact-type concentration measuring device at the point where the pixel intensity is extracted, and the equation for the artificial neural network is shown in Equation 3 below.

은 추적자물질의 농도 추정치(출력값), W_jk는 은닉층과 출력층 간의 가중인자, b_k는 출력층의 편향인자이다. 구축되는 인공신경망의 일반적인 구조는 도 6과 같다.Where x _i is the input data of the artificial neural network, DN is the pixel intensity of the image acquired with the rhodamine WT introduced, DN ₀ is the base pixel intensity of the image acquired before the rhodamine WT was introduced, and DN _s is the image In the PTFE panel, R, G, B are the bands of digital images, H _j is the value of the hidden neuron, W _ij is the weighting factor between the input layer and the hidden layer, b _j is the deflection factor of the hidden neuron, δ _j Is the activation function of the hidden layer,

Is the estimated concentration of tracer material (output value), W _jk is the weighting factor between the hidden layer and the output layer, and b _k is the deflection factor of the output layer. The general structure of the constructed artificial neural network is shown in FIG. 6.

The number of hidden neurons in the hidden layer should be determined to an appropriate value by the user, and as a result of evaluation using k-fold cross-validation in one embodiment of the present invention, it was found that the minimum error was shown when 15 hidden neurons were used. The total number of photographed frames was 23,688, and to reduce the amount of calculation and noise in the image, an average pixel intensity value of 1 second was used by averaging every 24 frames. Therefore, in an embodiment of the present invention, the measured concentration values of the tracer material at all concentration measurement points (15) in the experimental channel section are collected, and input factors (normalized pixel intensity of the image coordinate at the concentration measuring device location) and When a data pair consisting of target values is created, the number of data pairs for training the artificial neural network is 14,805 (15 (concentration measuring devices) × 23,688/24), of which the actual measured value is 0bbp (Rodamine WT is each corresponding Before reaching the concentration measuring device and after passing each corresponding concentration measuring device), a total of 2,472 data are included. Of the total data, 70% is divided into training data, and 30% is divided into verification data. In general, training data is data used in the process of training an artificial neural network so that the artificial neural network can present a set target value for the corresponding input factor, and the verification data shows good results even for new data not included in the training data. This is the data used for verification.

At the beginning of learning using training data, since the weighting factor and the bias factor of each neuron that exist between each layer are unknown, a random value is set as the initial value, and the data pair divided by the training data is input to build a random value. It performs the operation of the artificial neural network. The concentration value of the tracer material estimated by the artificial neural network shows a large error due to the arbitrary weighting and biasing factors, and this error is evaluated as the cumulative square error for all training data, and the equation is shown in Equation 4 below. same.

은 m번째 훈련자료에 대한 추적자물질의 농도 추정값, C_0(m)은 접촉식 농도측정기기에 의해 측정된 m번째 추적자물질의 농도값이다.Where E is the cumulative square error, M is the total number of training data,

Is the estimated value of the tracer substance concentration for the m-th training data, and C _0(m) is the concentration value of the m-th tracer substance measured by the contact-type concentration measuring device.

The learning process of the artificial neural network corresponds to the process of minimizing the cumulative square error, and the weighting factor and the bias factor are corrected so that the cumulative square error is minimized using gradient descent. The learning process is performed using only training data, and the final performance of the learned artificial neural network is evaluated using verification data corresponding to 30% of the total data. 7 is a graph showing a comparison between the measured concentration value for the verification data and the estimated concentration value of the artificial neural network in an embodiment of the present invention, and the artificial neural network finally learned in an embodiment of the present invention has a coefficient of determination (R ² ) Was evaluated as 0.93, and the root mean square error (RMSE) was evaluated as 3.43 ppb.

The input factor when learning from the training data uses the normalized pixel intensity of the image coordinate at the location of the concentration measuring device, but in one embodiment of the present invention, the concentration measuring device is not installed in each frame in the learned artificial neural network after learning is completed. When the normalized pixel intensities of all image coordinates of the non-points are arranged and input in the form of an input factor, the image for each frame projected on the UTM coordinate system is converted into a density field for each time as shown in FIG. 8.

That is, in the present invention, the artificial neural network building module 13 converts the pixel intensity of the RGB image to the concentration value of the tracer material using Equations 3 and 4, and this conversion process is finally performed by a computer. It is a program coded with an algorithm directly through, in other words, this program is stored in the river mixing behavior measurement device 10 or the storage device 20 while including the above equations 3 and 4, so that the artificial nerve only building module 13 is By using the program input and stored in the storage device 20, the pixel intensity of the image is converted into a concentration value of the tracer material.

10: Stream mixing behavior measurement device
11: Image coordinate tracking module
12: coordinate transformation module
13: Artificial Neural Network Construction Module
20: storage device

Claims (6)

(a) performing a tracer experiment and acquiring RGB image data of a river using an unmanned aerial vehicle (S10);
(b) an image coordinate tracking module 11 extracting an image coordinate of a ground reference point for a subsequent frame based on the image coordinate of a ground reference point for the first frame of the RGB image data (S20);
(c) The coordinate conversion module 12 constructs a certain number of coordinate pairs from the image coordinates of the ground reference point in each frame extracted in step (b) and the UTM coordinates of the corresponding ground reference point, and all image coordinates of the RGB image data. Projecting the pixel intensity of the UTM coordinate system to the pixel intensity (S30), and
(d) The artificial neural network construction module 13 constructs an artificial neural network using the concentration value of the tracer substance measured by the concentration measuring device and the pixel intensity of the RGB image, and each frame of RGB image data through the constructed artificial neural network. In the step of converting the pixel intensity at the point where the concentration measuring device is not installed into the concentration value of the tracer substance (S40),
After extracting the pixel intensity from the UTM coordinates corresponding to the installation location of the concentration measuring device, it shows the relationship between the extracted pixel intensity and the concentration value (target value) of the tracer substance measured by the concentration measuring device at the same time. Equation for the artificial neural network of step (d), which can calculate the concentration of the tracer material through the pixel intensity of the image,

(Where x _i is the input data of the artificial neural network, DN is the pixel intensity of the image acquired with the rhodamine WT introduced, DN ₀ is the base pixel intensity of the image acquired before the rhodamine WT was introduced, and DN _s is Pixel intensity in the PTFE panel in the image, R, G, B are the bands of digital images, H _j is the value of the hidden neuron, W _ij is the weighting factor between the input layer and the hidden layer, b _j is the deflection factor of the hidden neuron, δ _j is the activation function of the hidden layer,

Is the estimated concentration of tracer material (output value), W _jk is the weighting factor between the hidden layer and the output layer, b _k is the deflection factor of the output layer), and the input data of the artificial neural network is normalized by dividing it by the solar reference value obtained from the PTFE panel in the image. A method for measuring two-dimensional river mixing behavior using an RGB image obtained from an unmanned aerial vehicle.
The method of claim 1,
The step (a) is to install a certain number of ground reference points in the periphery of the measurement section of the river, measure the UTM coordinates of the ground reference point, install a polytetrafluoroethylene (PTFE) panel to obtain reference pixel intensity for sunlight, and the tracer material. Two-dimensional using RGB images acquired from unmanned aerial vehicles, characterized by including installation of a certain number of concentration measuring devices to measure the concentration of rhodamine WT and injection of rhodamine WT, a tracer material, and simultaneous RGB imaging. Method of measuring river mixing behavior.
The method of claim 1,
In the step (b), the image coordinate tracking module 11 extracts a partial image of a certain area centering on the image coordinates corresponding to each ground reference point from the first frame of the image, and the first frame in the image for each frame of the image thereafter. By extracting the partial image for the same region from which the partial image was extracted, the following equation,

(Here, NCC is the normal correlation coefficient, I is the pixel intensity of the partial image extracted from the first frame, i and j are the image coordinates of the partial image extracted from the first frame, and I _tmp is the extracted part from the frame to be calculated. Pixel intensity of the image, △X and △Y are the moving distance between two partial images,

Is the average pixel intensity of the partial image extracted from the first frame,

Is the image of each ground reference point by searching for the moving distance of the point with the highest normal correlation between the partial image extracted from the first frame and the partial image extracted from each frame from the average pixel intensity of the partial image extracted from the frame to be calculated. Two-dimensional stream mixing behavior measurement method using an RGB image acquired from an unmanned aerial vehicle, which calculates the coordinate movement distance.
The method of claim 1,
In the step (c), in order for the coordinate conversion module 12 to project the pixel intensity of all image coordinates of the RGB image data onto the pixel intensity of the UTM coordinate system, the established coordinate pairs and the following equations,

(Where x and y are UTM coordinates, X and Y are image coordinates, P ₁ ~ P ₈ are conversion factors) to calculate the transformation coefficient, and the image coordinates (X, Y) is substituted into the above equation to calculate UTM coordinates (x,y) in every frame. 2D river mixing behavior measurement method using an RGB image acquired from an unmanned aerial vehicle.
delete The method of claim 1,
In order to train the artificial neural network, a certain number of data pairs consisting of input data (normalized pixel intensity of the image coordinate at the concentration measuring device location) and a target value are constructed, and some of the data pairs are divided into training data and input, The error that occurs when the artificial neural network operation is performed by setting a random value for the weighting weight and the bias factor of each neuron as an initial value is the following equation, which is the cumulative square error for all training data,

(Where E is the cumulative square error, M is the total number of training data,

Is the estimated value of the tracer substance concentration for the m-th training data, C _{0 (m)} is the concentration value of the m-th tracer substance measured by the concentration measuring device), and the cumulative square error is minimum using the gradient descent method. A method of measuring a two-dimensional river mixing behavior using an RGB image acquired from an unmanned aerial vehicle, characterized in that correcting the weighting factor and the deflection factor as much as possible.

Images