CN113538580A - Vibration measurement method and system based on visual processing - Google Patents

Abstract

The application discloses vibration measurement method and system based on visual processing, which comprises the following steps: extracting brightness information of each pixel point in a first monitoring area; filtering the extracted brightness information; calculating to obtain the phase of each pixel point in the current frame vibration image in the first monitoring area; calculating the phase difference of each pixel point in the first monitoring area; weighting the obtained phase difference of each pixel point in the first monitoring area; summing the weighted phase differences of all pixel points in the first monitoring area in the current frame vibration image to obtain the object vibration quantity of the first monitoring area in the current frame vibration image; and generating a vibration signal of the target detection object image in the first monitoring area according to the object vibration quantity of the first monitoring area in each obtained frame of vibration image. The image filter of using design in this application carries out filtering process to the image, directly uses the pixel coordinate information in the image, need not to put up or spray the characteristic target point of artificial settlement on target detection object surface.

Description

A vibration measurement method and system based on vision processing

technical field

The present application belongs to the technical field of vision processing, and in particular relates to a vibration measurement method and system based on vision processing.

Background technique

High-speed rail platform canopy will vibrate due to air flow and other reasons when the high-speed train runs at high speed, which makes the canopy in a state of frequent vibration, which is a major test for the safety of the canopy structure. Once fatigue damage occurs, it will bring harm to the safe operation of the railway. , causing serious economic and property losses and threats to personal safety.

In the prior art, a contact measurement method is mainly used to detect the vibration of the canopy. Specifically, the canopy is detected by arranging a large number of contact measurement sensors on the canopy.

However, such a contact measurement method in the prior art requires a lot of workload and requires the cooperation of relevant departments, which is extremely inconvenient.

SUMMARY OF THE INVENTION

In order to solve the above technical problems in the prior art, the present application provides a vibration measurement method and system based on vision processing.

In a first aspect, the present application provides a vibration measurement method based on visual processing, comprising:

Obtain a vibration image sequence in which the target detection object is in a vibrating state, and the vibration image sequence includes multiple frames of vibration images sorted in time series;

On the image processing display interface, frame selection of at least one monitoring area, wherein each monitoring area covers at least part of the target detection object image in the vibration image;

Perform the following steps for each of the monitoring areas in each frame of the vibration image:

extracting the brightness information of each pixel in the first monitoring area, wherein the first monitoring area is any monitoring area in the at least one monitoring area;

filtering and processing the brightness information of each pixel in the extracted first monitoring area;

According to the filtered brightness information of each pixel in the first monitoring area, calculate the phase of each pixel in the current frame vibration image in the first monitoring area;

Respectively calculate the phase difference between the phase of the vibration image in the current frame and the phase of the vibration image in the first frame of each pixel in the first monitoring area;

Perform weighting processing on the obtained phase difference of each pixel in the first monitoring area to obtain the weighted phase difference of each pixel in the first monitoring area in the current frame vibration image;

Summing the weighted phase differences of all pixels in the first monitoring area in the current frame vibration image to obtain the vibration amount of the object in the first monitoring area in the current frame vibration image;

According to the vibration amount of the object in the first monitoring area in the obtained vibration images of each frame, a vibration signal of the image of the target detection object in the first monitoring area is generated.

Optionally, filtering and processing the brightness information flow of each pixel in the extracted first monitoring area, including:

Fourier transform is performed on the brightness information of each pixel in the first monitoring area to obtain a Fourier spectral response function of the brightness information of each pixel in the first monitoring area;

The Fourier spectral response function is filtered by the image filter H(u, v) to obtain the filter response function of each pixel in the first monitoring area, wherein the image filter H(u, v) satisfies The first relational formula, the first relational formula is:

Wherein, W represents the passband bandwidth of the image filter, D(u, v) represents the distance from (u, v) to the origin of the frequency plane, and D ₀ represents the cutoff frequency;

所述滤波后的亮度信息

满足第二关系关系式，所述第二关系关系式为：Inverse Fourier transform of the filter response function to obtain the filtered brightness information of each pixel in the first monitoring area

the filtered luminance information

Satisfy the second relational formula, the second relational formula is:

表示傅里叶频谱响应函数，x表示像素点的像素横坐标，y表示像素点的像素纵坐标，M、N表示振动图像的尺寸，j表示虚数单位，u表示x方向的频率变量，v表示y方向的频率变量。in,

Represents the Fourier spectral response function, x represents the pixel abscissa of the pixel, y represents the pixel ordinate of the pixel, M and N represent the size of the vibration image, j represents the imaginary unit, u represents the frequency variable in the x direction, and v represents Frequency variable in the y direction.

Optionally perform weighting processing on the obtained phase difference of each pixel in the first monitoring area to obtain the weighted phase difference of each pixel in the first monitoring area in the current frame vibration image, including:

Obtaining the brightness information of preset adjacent pixels corresponding to each pixel in the first monitoring area in the current frame vibration image;

According to the third relational formula, the phase difference of each pixel in the first monitoring area in the current frame of vibration image is weighted to obtain the weighted phase difference of each pixel in the first monitoring area in the current frame of vibration image. The three relations are:

表示第i帧振动图像中第一监测区域内每个像素点的加权相位差，

表示第i帧振动图像中像素坐标为(x,y)的像素点对应的预设邻近像素点亮度信息，

表示第i帧振动图像中像素坐标为(x,y)的像素点对应的相位差，m<k<n,m<l<n，其中，m和n表示与像素坐标为(x,y)的像素点对应的预设邻近像素点的像素坐标值。in,

represents the weighted phase difference of each pixel in the first monitoring area in the ith frame of vibration image,

Indicates the brightness information of the preset adjacent pixels corresponding to the pixels whose pixel coordinates are (x, y) in the i-th vibration image,

Represents the phase difference corresponding to the pixel with the pixel coordinate (x, y) in the i-th vibration image, m<k<n, m<l<n, where m and n represent the pixel coordinate (x, y) The pixel coordinate value of the preset adjacent pixel corresponding to the pixel.

Optionally, the method further includes:

Converting the generated vibration signal of the target detection object image in each monitoring area into a vibration signal of the solid part corresponding to the target detection object image in each monitoring area;

According to the vibration signal of the solid part corresponding to the image of the target detection object in each monitoring area, the vibration frequency information of each solid part of the target detection object is determined.

Optionally, if the target detection object is a canopy roof, the method includes:

According to the fourth relational expression, the vibration signal of the solid part corresponding to the canopy ceiling image in the first monitoring area is determined, and the fourth relational expression is:

R=S×γ×1/cos(α)

Wherein, R represents the vibration signal of the solid part corresponding to the canopy ceiling image in the first monitoring area, S represents the vibration signal of the canopy ceiling image in the first monitoring area, γ represents the pixel equivalent ratio, α represents the included angle between the physical vibration direction of the canopy ceiling and the vibration direction of the collected canopy ceiling image;

Fourier transform is performed on the vibration signal of the solid part corresponding to the canopy ceiling image in the first monitoring area to obtain vibration frequency information of the solid part corresponding to the canopy ceiling image in the first monitoring area.

In a second aspect, the present application also provides a vibration measurement system based on visual processing, comprising:

an acquisition module, configured to acquire a vibration image sequence in which the target detection object is in a vibrating state, where the vibration image sequence includes multiple frames of vibration images sorted in time series;

a frame selection module, configured to frame at least one monitoring area on the image processing display interface, wherein each of the monitoring areas covers at least part of the target detection object image in the vibration image;

an extraction module, configured to extract the brightness information of each pixel in a first monitoring area, wherein the first monitoring area is any monitoring area in the at least one monitoring area;

a filtering processing module, used for filtering and processing the brightness information of each pixel in the extracted first monitoring area;

The first calculation module is used to calculate the phase of each pixel in the current frame vibration image in the first monitoring area according to the filtered brightness information of each pixel in the first monitoring area;

The second calculation module is used to respectively calculate the phase difference between the phase of the vibration image of the current frame and the phase of the vibration image of the first frame of each pixel in the first monitoring area;

a weighted processing module, configured to perform weighting processing on the obtained phase difference of each pixel in the first monitoring area, to obtain the weighted phase difference of each pixel in the first monitoring area in the current frame vibration image;

The third calculation module is used for summing the weighted phase differences of all the pixels in the first monitoring area in the current frame vibration image to obtain the vibration amount of the object in the first monitoring area in the current frame vibration image;

The generating module is configured to generate the vibration signal of the image of the target detection object in the first monitoring area according to the vibration amount of the object in the first monitoring area in each frame of the obtained vibration image.

Optionally, the filtering processing module includes a Fourier transform module, an image filter and an inverse Fourier transform module;

a Fourier transform module, configured to perform Fourier transform on the brightness information of each pixel in the first monitoring area to obtain a Fourier spectral response function of the brightness information of each pixel in the first monitoring area;

The image filter is used to filter the Fourier spectral response function to obtain the filter response function of each pixel in the first monitoring area, wherein the image filter H(u, v) satisfies the first relational expression, The first relational formula is:

Wherein, W represents the passband bandwidth of the image filter, D(u, v) represents the distance from (u, v) to the origin of the frequency plane, and D ₀ represents the cutoff frequency;

所述滤波后的亮度信息

满足第二关系关系式，所述第二关系关系式为：an inverse Fourier transform module, configured to perform an inverse Fourier transform on the filtering response function to obtain filtered brightness information of each pixel in the first monitoring area

the filtered luminance information

Satisfy the second relational formula, the second relational formula is:

表示傅里叶频谱响应函数，x表示像素点的像素横坐标，y表示像素点的像素纵坐标，M、N表示振动图像的尺寸，j表示虚数单位，u表示x方向的频率变量，v表示y方向的频率变量。in,

Represents the Fourier spectral response function, x represents the pixel abscissa of the pixel, y represents the pixel ordinate of the pixel, M and N represent the size of the vibration image, j represents the imaginary unit, u represents the frequency variable in the x direction, and v represents Frequency variable in the y direction.

Optionally, the weighted processing module includes an acquisition submodule and a weighted processing submodule;

an acquisition sub-module, used for acquiring the preset adjacent pixel brightness information corresponding to each pixel in the first monitoring area in the current frame vibration image;

The weighting processing sub-module is used to weight the phase difference of each pixel in the first monitoring area in the current frame vibration image according to the third relational expression, and obtain the phase difference of each pixel in the first monitoring area in the current frame vibration image. Weighted phase difference, the third relationship is:

表示第i帧振动图像中第一监测区域内每个像素点的加权相位差，

表示第i帧振动图像中像素坐标为(x,y)的像素点对应的预设邻近像素点亮度信息，

表示第i帧振动图像中像素坐标为(x,y)的像素点对应的相位差，m<k<n,m<l<n，其中，m和n表示与像素坐标为(x,y)的像素点对应的预设邻近像素点的像素坐标值。in,

represents the weighted phase difference of each pixel in the first monitoring area in the ith frame of vibration image,

Indicates the brightness information of the preset adjacent pixels corresponding to the pixels whose pixel coordinates are (x, y) in the i-th vibration image,

Represents the phase difference corresponding to the pixel with the pixel coordinate (x, y) in the i-th vibration image, m<k<n, m<l<n, where m and n represent the pixel coordinate (x, y) The pixel coordinate value of the preset adjacent pixel corresponding to the pixel.

Optionally, the system also includes a conversion module and a determination module:

a conversion module, configured to convert the generated vibration signal of the target detection object image in each monitoring area into a vibration signal of the solid part corresponding to the target detection object image in each monitoring area;

The determining module is configured to determine the vibration frequency information of each entity part of the target detection object according to the vibration signal of the entity part corresponding to the target detection object image in each monitoring area.

Optionally, if the target detection object is a canopy ceiling, the conversion module is configured to determine the vibration signal of the solid part corresponding to the canopy ceiling image in the first monitoring area according to the fourth relational expression. , the fourth relational formula is:

R=S×γ×1/cos(α)

Wherein, R represents the vibration signal of the solid part corresponding to the canopy ceiling image in the first monitoring area, S represents the vibration signal of the canopy ceiling image in the first monitoring area, γ represents the pixel equivalent ratio, α represents the included angle between the physical vibration direction of the canopy ceiling and the vibration direction of the collected canopy ceiling image;

The determining module is configured to perform Fourier transform on the vibration signal of the solid part corresponding to the canopy ceiling image in the first monitoring area, and obtain the corresponding vibration signal of the canopy ceiling image in the first monitoring area. Vibration frequency information for the body part.

In summary, a vibration measurement method and system based on visual processing provided by this application uses a designed image filter to filter the image, directly uses the pixel coordinate information in the image, and does not need to identify any special features contained in the image in advance. , such as artificially set characteristic target points, that is to say, the characteristic information of the target detection object itself can be directly used in the present application, and there is no need to post or spray the artificially set characteristic target points on the surface of the target detection object. In addition, the vibration measurement method based on visual processing provided by the embodiment of the present application can analyze the local vibration of the target detection object by selecting a plurality of monitoring areas in a frame.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

1 is a schematic workflow diagram of a vibration measurement method based on visual processing provided by an embodiment of the application;

2 is a schematic diagram of on-site video stream or vibration image sequence acquisition of a target detection object provided by an embodiment of the present application;

3 is a schematic diagram of frame selection monitoring area in a vibration measurement method based on visual processing provided by an embodiment of the present application;

4 is a schematic workflow diagram of another vibration measurement method based on visual processing provided by an embodiment of the application;

FIG. 5 is a schematic diagram of vibration measurement of a canopy ceiling by using a vibration measurement method based on visual processing provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

The present application provides a vibration measurement method based on visual processing, as shown in Figure 1, comprising the following steps:

Step 100: Acquire a vibration image sequence in which the target detection object is in a vibrating state, where the vibration image sequence includes multiple frames of vibration images sorted in time series.

First of all, it should be noted that if the video stream in which the target detection object is in a vibrating state is initially collected, it is also necessary to perform decoding on the video stream to obtain a sequence of vibrating images.

Secondly, the present application needs to use a collection device to collect a video stream or a vibration image sequence in which the target detection object is in a vibrating state. However, since the vibration measurement of the target detection object needs to be carried out on the spot, that is to say, the acquisition equipment also collects video streams or vibration image sequences in the vibration state. Therefore, in order to ensure the accuracy of the vibration measurement results, the same vibration state In this case, the vibration frequency of the acquisition device itself is much lower than the vibration frequency of the target detection object.

Thirdly, the present application does not limit the acquisition equipment used to collect the video stream of the target detection object in a vibrating state. For example, the acquisition equipment may include a camera and a camera fixing device. High-quality vibration image of the target detection object, using the camera fixture to fix the camera, can provide a stable measurement environment.

In a specific example, as shown in Figure 2, the target detection object is a high-speed rail platform canopy, and it is necessary to authorize the detection personnel to set up the acquisition equipment on the platform (set up tools such as tripods and cameras within a safe range). After the acquisition equipment is set up, Waiting for the train to enter the station, start collecting data when it is about to enter the station, suspend the collection after an appropriate time after passing the station, save the data and record the passing time, model and heading direction. In the process of data processing (ie, the process of executing steps 100 to 900 ), you can choose to process and analyze the data in real time, or firstly photograph and store image data, and then perform data analysis after the photographing is over.

Step 200: On the image processing display interface, frame at least one monitoring area, wherein each monitoring area covers at least part of the target detection object image in the vibration image.

On the image processing display interface, the entire collected image of the target detection object can be displayed, but if the image in the entire display interface is directly processed, on the one hand, the amount of data is large, and on the other hand, it can only reflect the overall vibration of the target detection object. , which cannot reflect the local vibration of the target detection object. Based on this, in the embodiment of the present application, one or more monitoring areas may be selected in a frame on the image processing display interface, wherein each monitoring area covers at least part of the target detection object image in the vibration image, so as to separate Data processing and analysis are performed on each part of the target detection object in the vibration image, and then corresponding vibration signals are respectively output to reflect the local vibration of the target detection object. For example, as shown in Figures 3 and 4, the monitoring area A, monitoring area B and monitoring area C are selected in the collected complete target detection object image, and then the monitoring area A, monitoring area B and monitoring area C are respectively checked. Through data processing and analysis, vibration signals corresponding to the above three monitoring areas can be obtained respectively, which can reflect the local vibration of the target detection objects A, B, and C.

It should be noted that, after acquiring the vibration image sequence, a time period for data processing should be selected, so as to determine the start frame vibration image and the end frame vibration image for data processing. Since each frame of vibration image needs to be processed, the monitoring area can be framed on the display interface displaying the vibration image of the initial frame. When processing the vibration image of the subsequent frame, the pre-framed monitoring area can be displayed on the image processing display interface. position remains unchanged.

After the above step 200 is completed, the following steps 300 to 800 are performed for each monitoring area in each frame of the vibration image, so as to obtain the vibration status of the target detection object image in each monitoring area.

The following takes the data processing process of the first monitoring area in a frame of vibration image as an example for detailed description, wherein the first monitoring area may be any monitoring area in the at least one monitoring area.

Step 300: Extract the brightness information of each pixel in the first monitoring area.

其中，(x,y)表示像素点的像素坐标，t_i表示第i时刻，选取进行数据处理的时间段包括T个时刻t₁,t₂，t₃,……，t_T。也就是说，

表示第一监测区域在t₁时刻(即第一帧振动图像)，像素坐标为(x，y)的像素点的亮度信息，

表示第一监测区域在t₂时刻(即第二帧振动图像)，像素坐标为(x，y)的像素点的亮度信息，以此类推，得到每帧振动图像中第一监测区域内每个像素点的亮度信息。The brightness information described in the embodiments of the present application mainly includes brightness values, and the brightness information of each pixel in the first monitoring area can be expressed as

Among them, (x, y) represents the pixel coordinates of the pixel point, t _i represents the ith moment, and the time period selected for data processing includes T moments t ₁ , t ₂ , t ₃ ,..., t _T . That is,

Indicates the brightness information of the pixel point whose pixel coordinates are (x, y) in the first monitoring area at time t ₁ (that is, the first frame of vibration image),

Indicates that the first monitoring area is at time t ₂ (that is, the second frame of the vibration image), the brightness information of the pixel whose pixel coordinates are (x, y), and so on, to obtain each frame of vibration image in the first monitoring area. Brightness information of pixels.

Step 400 , filter and process the brightness information of each pixel in the extracted first monitoring area.

The present application does not limit the method for filtering the brightness information of each pixel in the extracted first monitoring area. In a specific example, it can be implemented through the following steps 410 to 430 .

其中，t_i表示第i时刻，(u，v)表示与像素坐标(x，y)对应的频率变量，

表示第i帧振动图像中像素坐标为(x，y)的像素点对应的傅里叶频谱响应函数，其中，傅里叶频谱响应函数

满足如下关系式(1)：Step 410: Perform Fourier transform on the brightness information of each pixel in the first monitoring area to obtain the Fourier spectral response function of the brightness information of each pixel in the first monitoring area

Among them, t _i represents the i-th time, (u, v) represents the frequency variable corresponding to the pixel coordinates (x, y),

Indicates the Fourier spectral response function corresponding to the pixel coordinates (x, y) in the i-th vibration image, where the Fourier spectral response function

It satisfies the following relation (1):

Among them, M and N represent the size of the vibration image, j represents the imaginary unit, u represents the frequency variable in the x direction, and v represents the frequency variable in the y direction.

进行滤波处理，得到第一监测区域内每个像素点的滤波响应函数

其中，所述图像滤波器H(u，v)满足第一关系式(2)，所述第一关系式(2)为：Step 420, pass the image filter H(u, v) to the Fourier spectral response function

Perform filtering processing to obtain the filtering response function of each pixel in the first monitoring area

Wherein, the image filter H(u, v) satisfies the first relational expression (2), and the first relational expression (2) is:

Wherein, W represents the passband bandwidth of the image filter, D(u, v) represents the distance from (u, v) to the origin of the frequency plane, and D ₀ represents the cutoff frequency.

傅里叶逆变换，得到第一监测区域内每个像素点滤波后的亮度信息

所述滤波后的亮度信息

满足第二关系关系式(3)，所述第二关系关系式(3)为：Step 430, to the filter response function

Inverse Fourier transform to obtain the filtered brightness information of each pixel in the first monitoring area

the filtered luminance information

Satisfy the second relational formula (3), the second relational formula (3) is:

Among them, x represents the pixel abscissa of the pixel point, y represents the pixel ordinate of the pixel point, M and N represent the size of the vibration image, j represents the imaginary unit, u represents the frequency variable in the x direction, and v represents the frequency variable in the y direction.

的值为复数形式，为振动图像经过上述滤波处理后的亮度信息图。通过相机拍摄目标检测物的振动，目标检测物的振动会投影到相机平面内形成不同的亮度值，目标检测物随时间振动，振动图像上目标检测物的相应位置的亮度值也会发生改变，即振动图像上的结构信息发生变化，而相位

的变化同步反映了振动图像上的结构信息发生变化，相位

的变化量等于振动图像结构信息的变化量，即可求得振动图像上目标检测物的振动量。因此，需要进一步根据

计算相位

相位

包含了振动图像上像素的结构信息。

The value of is a complex number, which is the brightness information map of the vibration image after the above filtering process. The vibration of the target detection object is captured by the camera, and the vibration of the target detection object will be projected into the camera plane to form different brightness values. The target detection object vibrates with time, and the brightness value of the corresponding position of the target detection object on the vibration image will also change. That is, the structural information on the vibration image changes, and the phase

The synchronization of the changes reflects the changes in the structural information on the vibration image, the phase

The change amount of is equal to the change amount of the vibration image structure information, and the vibration amount of the target detection object on the vibration image can be obtained. Therefore, it is necessary to further

Calculate the phase

phase

Contains structural information about the pixels on the vibrating image.

Step 500: Calculate the phase of each pixel in the first monitoring area in the current frame of the vibration image according to the filtered brightness information of each pixel in the first monitoring area.

然后可以按照如下关系式(4)，计算得到当前帧振动图像中第一监测区域内每个像素点的相位，关系式(4)为：According to the above step 400, the filtered brightness information of each pixel in the first monitoring area in the vibration image of the current frame can be obtained

Then, the phase of each pixel in the first monitoring area in the current frame vibration image can be calculated according to the following relational formula (4). The relational formula (4) is:

Step 600: Calculate the phase difference between the phase of the vibration image in the current frame and the phase of the vibration image in the first frame of each pixel in the first monitoring area.

After obtaining the phase of each pixel in the first monitoring area in the vibration image of the current frame, the phase difference between the vibration image of the current frame and the vibration image of the first frame of each pixel in the first monitoring area in the vibration image of the current frame is obtained. The phase difference is equal to the variation of the structural information of the vibration image. The first frame of vibration image refers to the vibration image corresponding to time t ₁ .

Similarly, the phase difference corresponding to each pixel in the first monitoring area in other frames of vibration images can be obtained. For example, the phase difference corresponding to each pixel in the first monitoring area in the fifth frame of vibration image refers to the fifth frame. The phase difference of each pixel in the first monitoring area in the frame vibration image in the fifth frame vibration image and the first frame vibration image.

Step 700: Perform weighting processing on the obtained phase difference of each pixel in the first monitoring area to obtain the weighted phase difference of each pixel in the first monitoring area in the vibration image of the current frame.

In order to obtain a more accurate vibration of the object, in this application, the obtained phase difference of each pixel in the first monitoring area is subjected to weighting processing, but the method for weighting each phase difference is not limited in this application. .

In a feasible manner, the embodiment of the present application first obtains the brightness information of the preset adjacent pixels corresponding to each pixel in the first monitoring area in the current frame vibration image, wherein the preset adjacent pixels corresponding to each pixel The number and position of the vibration images can be set by themselves, which is not limited in this application; then according to the third relational formula, the phase difference corresponding to each pixel in the first monitoring area in the current frame vibration image is weighted to obtain the current frame The weighted phase difference of each pixel in the first monitoring area in the vibration image, where the third relational formula (5) is:

表示第i帧振动图像中第一监测区域内每个像素点的加权相位差，

表示第i帧振动图像中像素坐标为(x，y)的像素点对应的预设邻近像素点亮度信息，

表示第i帧振动图像中像素坐标为(x，y)的像素点对应的相位差，m<k<n，m<l<n，其中，m和n表示与像素坐标为(x，y)的像素点对应的预设邻近像素点的像素坐标值。in,

represents the weighted phase difference of each pixel in the first monitoring area in the ith frame of vibration image,

Indicates the brightness information of the preset adjacent pixels corresponding to the pixels whose pixel coordinates are (x, y) in the i-th vibration image,

Represents the phase difference corresponding to the pixel with the pixel coordinate (x, y) in the i-th vibration image, m<k<n, m<l<n, where m and n represent the pixel coordinate (x, y) The pixel coordinate value of the preset adjacent pixel corresponding to the pixel.

(m<k<n，m<l<n)对

进行加权，得到该像素坐标(x,y)对应像素点的加权相位差

通过该步骤可以增大亮度值对比较大的区域的相位差在最后求取的物体振动量中的权重，即振动图像上目标检测物轮廓处的相位差的权重，使得最后求取的物体振动量

更为准确。Use luminance information around pixel coordinates (x, y)

(m<k<n, m<l<n) pairs

Perform weighting to obtain the weighted phase difference of the pixel corresponding to the pixel coordinate (x, y)

This step can increase the weight of the phase difference of the area with relatively large brightness value in the final obtained object vibration amount, that is, the weight of the phase difference at the contour of the target detection object on the vibration image, so that the finally obtained object vibrates quantity

more accurate.

Step 800: Sum the weighted phase differences of all the pixels in the first monitoring area in the current frame of vibration image to obtain the vibration amount of the object in the first monitoring area in the current frame of vibration image.

The vibration amount of the object in the first monitoring area in the vibration image of the current frame can reflect the vibration deviation of the first monitoring area in the vibration image of the current frame relative to the first monitoring area in the vibration image of the first frame.

其中，

According to the above steps 300-800, the vibration amount of the object in the first monitoring area in each frame of vibration image can be obtained

in,

Step 900: Generate a vibration signal of the image of the target detection object in the first monitoring area according to the vibration amount of the object in the first monitoring area in each frame of the obtained vibration image.

由此可知，振动信号能够反映监测区域内所述目标检测物图像在不同时刻的振动情况。It should be understood that, according to the above steps 300 to 900, the vibration amount of the object in each monitoring area in each frame of the vibration image can be obtained, so that the vibration signal of the target detection object image in each monitoring area can be obtained, that is, the vibration image can be obtained. The vibration signal corresponding to the sequence

It can be seen from this that the vibration signal can reflect the vibration of the image of the target detection object in the monitoring area at different times.

Further, the generated vibration signal of the target detection object image in each monitoring area can be converted into the vibration signal of the corresponding solid part of the target detection object image in each monitoring area; then, according to each monitoring area. The vibration signal of the solid part corresponding to the image of the target detection object is used to determine the vibration frequency information of each solid part of the target detection object.

As shown in Figure 5, taking the target detection object as the canopy roof as an example, the camera uses the elevation angle α to collect the vibration image of the canopy canopy. Assuming that the real vibration of the canopy canopy is in the vertical direction, the The vibration direction is actually the projection of the real vibration of the ceiling. Therefore, the real vibration signal of the canopy ceiling satisfies the fourth relational expression (6), and the fourth relational expression (6) is:

R=S×γ×1/cos(α) Relational formula (6)

Wherein, R represents the vibration signal of the solid part corresponding to the canopy ceiling image in the first monitoring area, S represents the vibration signal of the canopy ceiling image in the first monitoring area, γ represents the pixel equivalent ratio, α represents the camera elevation angle, that is, the angle between the physical vibration direction of the canopy ceiling and the vibration direction of the collected canopy ceiling image, wherein the pixel equivalent scale γ (unit: mm/ pixel); then, Fourier transform is performed on the vibration signal of the solid part corresponding to the canopy ceiling image in the first monitoring area to obtain a spectrogram of the signal, and directly read out the peak value of the signal on the spectrogram The frequency of each order of the signal can be obtained, thereby obtaining the vibration frequency information of the solid part corresponding to the canopy ceiling in the first monitoring area.

To sum up, the vibration measurement method based on visual processing provided by the embodiment of the present application uses a designed image filter to filter the image, directly uses the pixel coordinate information in the image, and does not need to identify any special features contained in the image in advance, such as The artificially set characteristic target points, etc., that is to say, the characteristic information of the target detection object itself can be directly used in this application, and there is no need to post or spray the artificially set characteristic target points on the surface of the target detection object. In addition, the vibration measurement method based on visual processing provided by the embodiment of the present application can analyze the local vibration of the target detection object by selecting a plurality of monitoring areas in a frame.

The embodiment of the present application also provides a vibration measurement system based on visual processing, including:

an acquisition module, configured to acquire a vibration image sequence in which the target detection object is in a vibrating state, where the vibration image sequence includes multiple frames of vibration images sorted in time series;

a frame selection module, configured to frame at least one monitoring area on the image processing display interface, wherein each of the monitoring areas covers at least part of the target detection object image in the vibration image;

an extraction module, configured to extract the brightness information of each pixel in a first monitoring area, wherein the first monitoring area is any monitoring area in the at least one monitoring area;

a filtering processing module, used for filtering and processing the brightness information of each pixel in the extracted first monitoring area;

The first calculation module is used to calculate the phase of each pixel in the current frame vibration image in the first monitoring area according to the filtered brightness information of each pixel in the first monitoring area;

The second calculation module is used to respectively calculate the phase difference between the phase of the vibration image of the current frame and the phase of the vibration image of the first frame of each pixel in the first monitoring area;

a weighted processing module, configured to perform weighting processing on the obtained phase difference of each pixel in the first monitoring area, to obtain the weighted phase difference of each pixel in the first monitoring area in the current frame vibration image;

The third calculation module is used for summing the weighted phase differences of all the pixels in the first monitoring area in the current frame vibration image to obtain the vibration amount of the object in the first monitoring area in the current frame vibration image;

The generating module is configured to generate the vibration signal of the image of the target detection object in the first monitoring area according to the vibration amount of the object in the first monitoring area in each frame of the obtained vibration image.

The filtering processing module includes a Fourier transform module, an image filter and an inverse Fourier transform module;

a Fourier transform module, configured to perform Fourier transform on the brightness information of each pixel in the first monitoring area to obtain a Fourier spectral response function of the brightness information of each pixel in the first monitoring area;

The image filter is used to filter the Fourier spectral response function to obtain the filter response function of each pixel in the first monitoring area, wherein the image filter H(u, v) satisfies the first relational expression, The first relational formula is:

Wherein, W represents the passband bandwidth of the image filter, D(u, v) represents the distance from (u, v) to the origin of the frequency plane, and D ₀ represents the cutoff frequency;

所述滤波后的亮度信息

满足第二关系关系式，所述第二关系关系式为：an inverse Fourier transform module, configured to perform an inverse Fourier transform on the filtering response function to obtain filtered brightness information of each pixel in the first monitoring area

the filtered luminance information

Satisfy the second relational formula, the second relational formula is:

表示傅里叶频谱响应函数，x表示像素点的像素横坐标，y表示像素点的像素纵坐标，M、N表示振动图像的尺寸，j表示虚数单位，u表示x方向的频率变量，v表示y方向的频率变量。in,

Represents the Fourier spectral response function, x represents the pixel abscissa of the pixel, y represents the pixel ordinate of the pixel, M and N represent the size of the vibration image, j represents the imaginary unit, u represents the frequency variable in the x direction, and v represents Frequency variable in the y direction.

The weighted processing module includes an acquisition submodule and a weighted processing submodule;

an acquisition sub-module, used for acquiring the preset adjacent pixel brightness information corresponding to each pixel in the first monitoring area in the current frame vibration image;

The weighting processing sub-module is used to weight the phase difference of each pixel in the first monitoring area in the current frame vibration image according to the third relational expression, and obtain the phase difference of each pixel in the first monitoring area in the current frame vibration image. Weighted phase difference, the third relationship is:

表示第i帧振动图像中第一监测区域内每个像素点的加权相位差，

表示第i帧振动图像中像素坐标为(x,y)的像素点对应的预设邻近像素点亮度信息，

表示第i帧振动图像中像素坐标为(x,y)的像素点对应的相位差，m<k<n,m<l<n，其中，m和n表示与像素坐标为(x,y)的像素点对应的预设邻近像素点的像素坐标值。in,

represents the weighted phase difference of each pixel in the first monitoring area in the ith frame of vibration image,

Indicates the brightness information of the preset adjacent pixels corresponding to the pixels whose pixel coordinates are (x, y) in the i-th vibration image,

Represents the phase difference corresponding to the pixel with the pixel coordinate (x, y) in the i-th vibration image, m<k<n, m<l<n, where m and n represent the pixel coordinate (x, y) The pixel coordinate value of the preset adjacent pixel corresponding to the pixel.

The system also includes a conversion module and a determination module:

a conversion module, configured to convert the generated vibration signal of the target detection object image in each monitoring area into a vibration signal of the solid part corresponding to the target detection object image in each monitoring area;

The determining module is configured to determine the vibration frequency information of each entity part of the target detection object according to the vibration signal of the entity part corresponding to the target detection object image in each monitoring area.

If the target detection object is a canopy ceiling, the conversion module is configured to determine the vibration signal of the solid part corresponding to the canopy ceiling image in the first monitoring area according to the fourth relational expression, and the first monitoring area The four relations are:

R=S×γ×1/cos(α)

Wherein, R represents the vibration signal of the solid part corresponding to the canopy ceiling image in the first monitoring area, S represents the vibration signal of the canopy ceiling image in the first monitoring area, γ represents the pixel equivalent ratio, α represents the included angle between the physical vibration direction of the canopy ceiling and the vibration direction of the collected canopy ceiling image;

The determining module is configured to perform Fourier transform on the vibration signal of the solid part corresponding to the canopy ceiling image in the first monitoring area, and obtain the corresponding vibration signal of the canopy ceiling image in the first monitoring area. Vibration frequency information for the body part.

It is sufficient to refer to each other for the same and similar parts among the various embodiments in this specification. In particular, as for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for related parts, please refer to the descriptions in the method embodiments.

The present application has been described in detail above with reference to specific embodiments and exemplary examples, but these descriptions should not be construed as a limitation on the present application. Those skilled in the art understand that, without departing from the spirit and scope of the present application, various equivalent replacements, modifications or improvements can be made to the technical solutions and embodiments of the present application, which all fall within the scope of the present application. The scope of protection of the present application is determined by the appended claims.

In a specific implementation, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium can store a program, and when the program is executed, the method and system for vibration measurement based on visual processing provided by the present application can be included in the program. some or all of the steps in each of the embodiments. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), and the like.

Those skilled in the art can clearly understand that the technology in the embodiments of the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solutions in the embodiments of the present application can be embodied in the form of software products in essence or in the parts that make contributions to the prior art, and the computer software products can be stored in a storage medium, such as ROM/RAM , magnetic disk, optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present application.

The above-described embodiments of the present application do not limit the protection scope of the present application.

Claims (10)

1. A vibration measurement method based on visual processing, comprising:

acquiring a vibration image sequence of a target detection object in a vibration state, wherein the vibration image sequence comprises a plurality of frames of vibration images which are sequenced according to a time sequence;

at least one monitoring area is selected in a frame mode on an image processing display interface, wherein each monitoring area at least covers part of the target detection object image in the vibration image;

executing the following steps for each monitoring area in each frame of vibration image:

extracting brightness information of each pixel point in a first monitoring area, wherein the first monitoring area is any one of the at least one monitoring area;

filtering the extracted brightness information of each pixel point in the first monitoring area;

respectively calculating the phase of each pixel point in the first monitoring area in the current frame vibration image according to the filtered brightness information of each pixel point in the first monitoring area;

respectively calculating the phase difference between the phase of each pixel point in the first monitoring area in the current frame vibration image and the phase of each pixel point in the first monitoring area in the first frame vibration image;

weighting the obtained phase difference of each pixel point in the first monitoring area to obtain the weighted phase difference of each pixel point in the first monitoring area in the current frame vibration image;

summing the weighted phase differences of all pixel points in the first monitoring area in the current frame vibration image to obtain the object vibration quantity of the first monitoring area in the current frame vibration image;

and generating a vibration signal of the target detection object image in the first monitoring area according to the object vibration quantity of the first monitoring area in each obtained frame of vibration image.

2. The method of claim 1, wherein filtering the extracted luminance information stream for each pixel in the first monitored region comprises:

performing Fourier transform on the brightness information of each pixel point in the first monitoring area to obtain a Fourier spectrum response function of the brightness information of each pixel point in the first monitoring area;

filtering the Fourier spectrum response function through an image filter H (u, v) to obtain a filtering response function of each pixel point in the first monitoring area, wherein the image filter H (u, v) meets a first relational expression, and the first relational expression is as follows:

wherein W represents the passband bandwidth of the image filter, D: (u, v) denotes the distance from (u, v) to the origin of the frequency plane, D₀Represents the cut-off frequency;

performing Fourier inverse transformation on the filter response function to obtain the brightness information of each pixel point in the first monitoring area after filtering

The filtered luminance information

Satisfying a second relational relationship:

wherein ,

the method comprises the steps of representing a Fourier spectrum response function, wherein x represents the pixel abscissa of a pixel point, y represents the pixel ordinate of the pixel point, M, N represents the size of a vibration image, j represents an imaginary unit, u represents a frequency variable in the x direction, and v represents a frequency variable in the y direction.

3. The method of claim 1, wherein weighting the obtained phase difference of each pixel point in the first monitoring area to obtain the weighted phase difference of each pixel point in the first monitoring area in the current frame vibration image comprises:

acquiring brightness information of preset adjacent pixel points corresponding to each pixel point in a first monitoring area in a current frame vibration image;

weighting the phase difference of each pixel point in the first monitoring area in the current frame vibration image according to a third relational expression, so as to obtain the weighted phase difference of each pixel point in the first monitoring area in the current frame vibration image, wherein the third relational expression is as follows:

wherein ,

representing the weighted phase difference of each pixel point in the first monitoring area in the ith frame of vibration image,

indicating the brightness information of the preset adjacent pixel points corresponding to the pixel point with the pixel coordinate (x, y) in the ith frame of vibration image,

m represents the phase difference corresponding to the pixel point with the pixel coordinate (x, y) in the vibration image of the ith frame<k<n,m<l<And n, wherein m and n represent pixel coordinate values of preset adjacent pixel points corresponding to the pixel point with the pixel coordinate of (x, y).

4. The method of claim 1, further comprising:

converting the generated vibration signal of the target detection object image in each monitoring area into a vibration signal of an entity part corresponding to the target detection object image in each monitoring area;

and determining the vibration frequency information of each entity part of the target detection object according to the vibration signal of the entity part corresponding to the target detection object image in each monitoring area.

5. The method of claim 4, wherein if the target detection object is a canopy, the method comprises:

determining a vibration signal of an entity part corresponding to the canopy ceiling image in the first monitoring area according to a fourth relational expression, wherein the fourth relational expression is as follows:

R＝S×γ×1/cos(α)

wherein, R represents a vibration signal of an entity portion corresponding to the canopy roof image in the first monitoring area, S represents a vibration signal of the canopy roof image in the first monitoring area, γ represents a pixel equivalent ratio, and α represents an included angle between an entity vibration direction of the canopy roof and a vibration direction of the acquired canopy roof image;

and carrying out Fourier transform on the vibration signal of the entity part corresponding to the canopy ceiling image in the first monitoring area to obtain the vibration frequency information of the entity part corresponding to the canopy ceiling image in the first monitoring area.

6. A vibration measurement system based on visual processing, comprising:

the system comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a vibration image sequence of a target detection object in a vibration state, and a plurality of frames of vibration images which are sequenced according to a time sequence are included in the vibration image sequence;

the frame selection module is used for selecting at least one monitoring area in a frame mode on an image processing display interface, wherein each monitoring area at least covers part of the target detection object image in the vibration image;

the extraction module is used for extracting the brightness information of each pixel point in a first monitoring area, wherein the first monitoring area is any monitoring area in the at least one monitoring area;

the filtering processing module is used for filtering the extracted brightness information of each pixel point in the first monitoring area;

the first calculation module is used for respectively calculating the phase of each pixel point in the first monitoring area in the current frame vibration image according to the brightness information of each pixel point in the first monitoring area after filtering;

the second calculation module is used for respectively calculating the phase difference between the phase of each pixel point in the first monitoring area in the current frame vibration image and the phase of each pixel point in the first monitoring area in the first frame vibration image;

the weighting processing module is used for weighting the obtained phase difference of each pixel point in the first monitoring area to obtain the weighted phase difference of each pixel point in the first monitoring area in the current frame vibration image;

the third calculation module is used for summing the weighted phase differences of all pixel points in the first monitoring area in the current frame vibration image to obtain the object vibration quantity of the first monitoring area in the current frame vibration image;

and the generating module is used for generating a vibration signal of the target detection object image in the first monitoring area according to the object vibration quantity of the first monitoring area in each obtained frame of vibration image.

7. The system of claim 6, wherein the filter processing module comprises a fourier transform module, an image filter, and an inverse fourier transform module;

the Fourier transform module is used for carrying out Fourier transform on the brightness information of each pixel point in the first monitoring area to obtain a Fourier spectrum response function of the brightness information of each pixel point in the first monitoring area;

the image filter is used for filtering the Fourier spectrum response function to obtain a filtering response function of each pixel point in the first monitoring area, wherein the image filter H (u, v) meets a first relational expression, and the first relational expression is as follows:

wherein W represents the passband bandwidth of the image filter, D (u, v) represents the distance from (u, v) to the origin of the frequency plane, D₀Represents the cut-off frequency;

the inverse Fourier transform module is used for performing inverse Fourier transform on the filter response function to obtain the brightness information of each pixel point in the first monitoring area after being filtered

The filterLuminance information of wave after

Satisfying a second relational relationship:

wherein ,

the method comprises the steps of representing a Fourier spectrum response function, wherein x represents the pixel abscissa of a pixel point, y represents the pixel ordinate of the pixel point, M, N represents the size of a vibration image, j represents an imaginary unit, u represents a frequency variable in the x direction, and v represents a frequency variable in the y direction.

8. The system of claim 6, wherein the weighting module comprises an acquisition sub-module and a weighting sub-module;

the acquisition submodule is used for acquiring preset adjacent pixel point brightness information corresponding to each pixel point in a first monitoring area in the current frame vibration image;

the weighting processing submodule is used for weighting the phase difference of each pixel point in the first monitoring area in the current frame vibration image according to a third relation formula, so as to obtain the weighted phase difference of each pixel point in the first monitoring area in the current frame vibration image, and the third relation formula is as follows:

wherein ,

representing the weighted phase difference of each pixel point in the first monitoring area in the ith frame of vibration image,

indicating the brightness information of the preset adjacent pixel points corresponding to the pixel point with the pixel coordinate (x, y) in the ith frame of vibration image,

m represents the phase difference corresponding to the pixel point with the pixel coordinate (x, y) in the vibration image of the ith frame<k<n,m<l<And n, wherein m and n represent pixel coordinate values of preset adjacent pixel points corresponding to the pixel point with the pixel coordinate of (x, y).

9. The system of claim 6, further comprising a conversion module and a determination module:

the conversion module is used for converting the generated vibration signal of the target detection object image in each monitoring area into a vibration signal of an entity part corresponding to the target detection object image in each monitoring area;

and the determining module is used for determining the vibration frequency information of each entity part of the target detection object according to the vibration signal of the entity part corresponding to the target detection object image in each monitoring area.

10. The system of claim 9, wherein if the target detection object is a canopy, the converting module is configured to determine the vibration signal of the entity portion corresponding to the canopy image in the first monitoring area according to a fourth relationship:

R＝S×γ×1/cos(α)

wherein, R represents a vibration signal of an entity portion corresponding to the canopy roof image in the first monitoring area, S represents a vibration signal of the canopy roof image in the first monitoring area, γ represents a pixel equivalent ratio, and α represents an included angle between an entity vibration direction of the canopy roof and a vibration direction of the acquired canopy roof image;

the determining module is used for performing Fourier transform on the vibration signal of the entity part corresponding to the canopy ceiling image in the first monitoring area to obtain the vibration frequency information of the entity part corresponding to the canopy ceiling image in the first monitoring area.

Images