CN117237597A - Data processing terminal based on Beidou satellite data and AI graph fusion - Google Patents
Data processing terminal based on Beidou satellite data and AI graph fusion Download PDFInfo
- Publication number
- CN117237597A CN117237597A CN202311107490.0A CN202311107490A CN117237597A CN 117237597 A CN117237597 A CN 117237597A CN 202311107490 A CN202311107490 A CN 202311107490A CN 117237597 A CN117237597 A CN 117237597A
- Authority
- CN
- China
- Prior art keywords
- image
- module
- edge
- offset
- beidou
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 title claims abstract description 28
- 230000004927 fusion Effects 0.000 title claims abstract description 10
- XEEYBQQBJWHFJM-UHFFFAOYSA-N Iron Chemical compound [Fe] XEEYBQQBJWHFJM-UHFFFAOYSA-N 0.000 claims abstract description 166
- 229910052742 iron Inorganic materials 0.000 claims abstract description 83
- 239000013598 vector Substances 0.000 claims abstract description 43
- 230000003287 optical effect Effects 0.000 claims abstract description 31
- 238000003384 imaging method Methods 0.000 claims abstract description 23
- 238000004891 communication Methods 0.000 claims abstract description 21
- 238000000034 method Methods 0.000 claims description 35
- 238000006073 displacement reaction Methods 0.000 claims description 27
- 238000001514 detection method Methods 0.000 claims description 26
- 238000003708 edge detection Methods 0.000 claims description 26
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000007781 pre-processing Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 7
- 238000001914 filtration Methods 0.000 claims description 6
- 238000009499 grossing Methods 0.000 claims description 4
- 230000004044 response Effects 0.000 claims description 4
- 230000001629 suppression Effects 0.000 claims description 3
- DMBHHRLKUKUOEG-UHFFFAOYSA-N diphenylamine Chemical compound C=1C=CC=CC=1NC1=CC=CC=C1 DMBHHRLKUKUOEG-UHFFFAOYSA-N 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 229910000831 Steel Inorganic materials 0.000 description 1
- 238000007664 blowing Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000011895 specific detection Methods 0.000 description 1
- 239000010959 steel Substances 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Image Analysis (AREA)
Abstract
The application discloses a data processing terminal based on Beidou satellite data and AI graph fusion, which comprises: the system comprises a main control unit, an imaging module, a Beidou positioning module, a Beidou time service module, a communication module, a short message module, a storage module and a power supply module, wherein the terminal is arranged on a tower body of an iron tower, the imaging module continuously acquires images of adjacent iron towers at set time intervals, marks the images with time stamps generated by the Beidou time service module, sends the images with the time stamps to the main control unit, and other modules in the power supply module terminal supply power; the main control unit preprocesses the received images, calculates offset vectors of angular points in two adjacent images by using a sparse optical flow algorithm, and judges whether the iron tower is inclined or not by using the offset vectors. According to the application, beidou time service and Beidou positioning are combined, so that the inclination of the iron tower can be accurately and rapidly identified, and the fault point can be accurately reported.
Description
Technical Field
The application belongs to the technical field of image processing, and particularly relates to a data processing terminal based on Beidou satellite data and AI graph fusion.
Background
The iron tower is a high tower built by steel materials, and the traditional iron tower can be used in the technical fields of railways, electric power, communication and the like. The iron tower is generally built in deep mountain forests and remote areas, the geology of the position of the iron tower is likely to change due to earthquake, address disasters and the like, and the iron tower is easy to incline, so that the iron tower inclines along with a foundation, when the inclination angle reaches a certain degree, the root opening of the iron tower and the height difference of the tower legs are changed, the tower body structure generates larger additional stress, the local damage or the whole collapse of the tower body can be caused when serious, and under the condition of external force such as wind blowing, icing and the like, the collapse is easy to occur, and the safety of the iron tower and the stable operation of a line are directly threatened. Traditional detection to iron tower slope situation mainly relies on artifical periodic line inspection to accomplish, has the higher problem of detection degree of difficulty.
Optical flow (optical flow) is the instantaneous velocity of the pixel motion of a spatially moving object on an observation imaging plane. The optical flow method is a method for finding out the correspondence existing between the previous frame and the current frame by utilizing the change of pixels in an image sequence in a time domain and the correlation between adjacent frames, thereby calculating the motion information of an object between the adjacent frames. The Chinese patent with publication number of CN105023278B proposes a moving object tracking method and system based on an optical flow method, wherein the method comprises providing a video image and preprocessing the image to generate a preprocessed image; performing edge detection on the preprocessed image, extracting target information from the preprocessed image by using an optical flow method, and generating a complete moving target by fusing the information of the edge detection and the extracted target information; estimating and analyzing the moving target by using an optical flow method, and removing mismatching points generated by illumination by adopting a forward and backward error algorithm based on a characteristic point track; and creating a template image and carrying out template image matching to track the moving target. The optical flow method detects the offset amplitude and the offset direction of an object in an image, and eliminates the interference of normal swing according to the offset direction, however, the conventional optical flow method needs to detect all pixels in a gray image, and has the problem of low detection efficiency.
Disclosure of Invention
The application provides a data processing terminal based on the fusion of Beidou satellite data and an AI graph, which fuses Beidou satellite data and image processing data and improves an optical flow method when iron tower tilt detection is carried out, and aims to solve the problems of low detection efficiency and insufficient accuracy when iron tower tilt detection is carried out by using a conventional optical flow method.
The data processing terminal provided by the application comprises: the system comprises a main control unit, an imaging module, a Beidou positioning module, a Beidou time service module, a communication module, a short message module, a storage module and a power module, wherein the terminal is arranged on a tower body of an iron tower, the imaging module continuously acquires images of adjacent iron towers at set time intervals, marks the images with time stamps generated by the Beidou time service module, sends the images with the time stamps to the main control unit, and other modules in the power module terminal supply power.
The main control unit is configured to execute the following iron tower inclination detection method:
s1: the main control unit receives the image sent by the imaging module, pre-processes the image to generate an edge image, and stores the edge image in a storage module according to time sequence, wherein the pre-processing comprises graying, two times of downsampling and edge detection which are sequentially arranged, and the edge detection detects the image output by the graying, the first time of downsampling and the second time of downsampling respectively and outputs the edge image with three scales.
S2: the main control unit compares every two continuous edge images with three scales in the storage module, the comparison method adopts a sparse optical flow algorithm, firstly, the angular point in the first edge image is calculated, and the displacement amplitude and direction of the angular point between the two continuous edge images are calculated.
S3: the main control unit compares a positioning offset value of a terminal on a shot iron tower with a displacement amplitude at a corresponding corner point of the terminal, firstly, converts the offset amplitude obtained by the Beidou positioning module into the offset amplitude in the direction of the edge image according to an included angle between the positioning offset direction obtained by the Beidou positioning module and the offset direction of the edge image, then, represents the converted offset amplitude as a pixel offset value to obtain a positioning offset value, and takes the edge image with the deviation of the positioning offset value and the displacement amplitude at the corresponding corner point not exceeding 3 pixels as an offset image.
S4: the main control unit judges whether the iron tower is inclined or not according to the offset image, calculates corresponding offset distances according to offset vectors of all angular points of the offset image, calculates standard deviation of the offset distances, judges that the edge image with the standard deviation not smaller than 2 is inclined, and judges that the edge image with the standard deviation smaller than 2 is integrally offset. If the iron tower is judged to incline, whether the iron tower swings normally or not is determined according to the displacement direction, and if the iron tower swings abnormally, warning information is sent to the management platform through the communication module and the short message module respectively, and the warning information further comprises position information generated by the Beidou positioning module.
Preferably, the edge detection adopts a Canny algorithm with an adaptive threshold, and an appropriate threshold is automatically selected according to the statistical characteristics of the image, and the detection steps comprise:
and Gaussian filtering is used for smoothing the input gray level image, filtering noise and obtaining a filtered image.
And calculating the gradient amplitude and the gradient direction of each pixel point in the filtered image by using a Sobel operator.
Non-maximum suppression is applied to all gradient magnitudes for eliminating spurious responses from edge detection.
The strong edge, the non-edge and the weak edge are determined by applying self-adaptive high-low double-threshold detection.
Edge connection connects weak edge points with adjacent strong edge points.
Preferably, the method for acquiring the high and low double thresholds of the adaptive threshold comprises the following steps: firstly, calculating a gray level histogram of an input image, calculating a gray level value median according to the gray level histogram, and then calculating a high threshold and a low threshold according to the gray level value median and a preset sigma value, wherein the calculation formula is as follows:
min＝(1-σ)×median
max＝(1+σ)×median
wherein min is a low threshold for the Canny algorithm, max is a high threshold for the Canny algorithm, sigma is a preset sigma value, and mean is a median of gray values calculated according to the gray histogram. If a high threshold max >255 is calculated, max is set to 255.
Preferably, the method for detecting the corner points comprises the following steps:
s2-1: and selecting a white pixel point P with a gray value of 255 from the edge image.
S2-2: and drawing a discretization circle with a radius of 3 pixels by taking the white pixel as a center, wherein 16 boundary pixels are arranged on the boundary of the discretization circle.
S2-3: if there are 10 consecutive pixels with gray values of 0 in the 16 boundary pixels of the discretization circle, the point P is determined as a corner point.
S2-4: and repeating the steps S2-1 to S2-3, traversing all pixel points in the edge image, and obtaining all corner points in the edge image.
Preferably, the sparse optical flow algorithm specifically comprises:
for a corner in the first edge image, defining a square first neighborhood window with the corner as the center in the first edge image.
Constructing an initial offset vector (u=0, v=0), acquiring a second neighborhood window with the same size as the first neighborhood window in the second edge image by using the initial offset vector, and if the second neighborhood window is different from the first neighborhood window, changing the offset vector to acquire the second neighborhood window again, and iteratively adjusting the offset vector until the second neighborhood window acquired in the second edge image is the same as the first neighborhood window, wherein the offset vector used in the iteration is used as the optimal offset vector of the corner point.
Traversing all the corner points of the first edge image, and obtaining the optimal offset vector of all the corner points.
The best displacement vector of the corner point is visualized as an arrow form.
Preferably, the size of the neighborhood window is set to 7×7 pixels.
Preferably, the communication module communicates with the management platform through a mobile network, a wireless network bridge or an electric power integrated data network, and sends the alarm information or the image to the management platform.
Preferably, the communication module further receives a control instruction of the management platform, the control instruction is forwarded to the main control unit, and the main control unit controls the corresponding module to work according to the received control instruction.
Preferably, the power supply module is a solar power supply system, and comprises a solar panel and a storage battery connected with the solar panel.
Compared with the prior art, the application has the following technical effects:
1. according to the data processing terminal provided by the application, the main control unit is adopted to calculate the displacement of the iron tower in the input image in real time, alarm information is sent to the inclination of the iron tower according to the calculation result, the accurate time acquired by the Beidou time service module is combined, the calculated inclination amplitude and the Beidou positioning offset value obtained by image processing calculation are fused to comprehensively judge whether the iron tower is inclined, the error of judging the inclination of the iron tower by a single means is effectively reduced, and the accuracy of iron tower inclination detection is improved; the improved optical flow method is adopted to calculate the displacement of the iron tower, so that the efficiency of image processing can be effectively improved.
2. The iron tower inclination detection method provided by the application is used for preprocessing an input image, and comprises the steps of graying, twice downsampling and edge detection which are sequentially arranged, and an edge detection algorithm can be used for effectively detecting the edge in the image, so that the edge image is used as the input of an optical flow method to provide more accurate edge information, the optical flow method is facilitated to calculate the motion vector of the pixel more accurately, and a more reliable motion calculation result is obtained; the conventional optical flow method performs pixel-level calculation on the whole image, and the use of the edge image can limit the optical flow calculation to the edge area of the image, so that the calculated amount can be reduced, the running speed can be increased, and the movement condition of the edge area can be focused more intensively; thus, using edge images as input to the optical flow method may provide more accurate, faster, and finer motion calculations.
3. The method for detecting the inclination of the iron tower provided by the application adopts the self-adaptive high and low threshold values when the edge detection is carried out, does not need to manually select the threshold values and adjust parameters for a plurality of times to determine the division of the edge pixels and the non-edge pixels, can automatically calculate different threshold values for different images and application scenes, automatically select the proper threshold values according to the statistical characteristics or local pixel information of the images, has good edge detection effect on the images acquired in different environments, can adapt to various weather in the working environment of the iron tower, and improves the robustness of edge detection on the iron tower images.
4. According to the iron tower inclination detection method provided by the application, the displacement amplitude and the displacement direction of a plurality of points on the iron tower in two continuous images are calculated respectively by using a sparse optical flow algorithm, the displacement amplitude is used for judging whether the iron tower is displaced, the displacement direction can be used for eliminating normal swing of the iron tower caused by environmental factors, false alarm is reduced, and the accuracy of iron tower inclination detection is improved.
5. The iron tower inclination detection method provided by the application uses a sparse optical flow algorithm, only the displacement amplitude and direction of a plurality of angular points in the image are calculated, and compared with a dense optical flow algorithm, the iron tower inclination detection method provided by the application can be used for detecting the displacement amplitude and direction of all pixels in the image, so that the calculation cost can be greatly saved, and the calculation efficiency can be improved.
6. The data processing terminals communicate by adopting the mode of the mobile network, the wireless bridge or the electric power integrated data network, and a proper communication mode can be selected according to the actual environment of the iron tower, so that the flexibility of deployment is effectively improved.
7. The data processing terminal provided by the application comprises the Beidou short message module, can be used for sending the alarm information, provides a redundant communication link under the condition that the communication module cannot work normally, ensures the timely sending of the alarm information, and improves the robustness of the iron tower inclination detection system.
Drawings
FIG. 1 is a schematic diagram of a data processing terminal according to an embodiment of the present application;
FIG. 2 is a schematic diagram of an imaging terminal deployment and imaging module imaging according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a processing flow of a main control unit in a data processing terminal according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a pretreatment flow in accordance with an embodiment of the present application;
FIG. 5 is a gray histogram of a grayed-out image of an embodiment of the present application;
fig. 6 is a schematic diagram of corner detection according to an embodiment of the present application;
FIG. 7 is a diagram of discretized circles during corner detection in accordance with an embodiment of the present application;
FIG. 8 is a schematic view of corner points of an edge image according to an embodiment of the present application;
FIG. 9 is a schematic diagram of a neighborhood window according to an embodiment of the present application;
FIG. 10 is a schematic view of a partial corner shift vector according to an embodiment of the present application;
fig. 11 is a schematic view of partial corner shift visualization according to an embodiment of the present application.
Reference numerals: 1. a main control unit; 2. an imaging module; 3. the Beidou positioning module; 4. the Beidou time service module; 5. a communication module; 6. a short message module; 7. a storage module; 8. a power module; 91. a first iron tower; 92. a second iron tower; 93. and a third iron tower.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings in conjunction with specific embodiments of the present application.
As shown in fig. 1, a data processing terminal based on the fusion of beidou satellite data and AI graphics includes: the system comprises a main control unit 1, an imaging module 2, a Beidou positioning module 3, a Beidou time service module 4, a communication module 5, a short message module 6, a storage module 7 and a power supply module 8.
Deployment of towers and terminals as shown in fig. 2, the second tower 92 has two adjacent towers: a first pylon 91 and a third pylon 93. The terminal is arranged on the tower body of the iron tower, the imaging module 2 continuously acquires images of adjacent iron towers at set time intervals, the time interval of acquiring the images by the imaging module 2 can be set to be any value between 1 and 120 minutes, and the imaging module can be adjusted by the main control unit 1 according to the instruction of the management platform when necessary. The imaging module 2 marks the acquired image with a time stamp, the time stamp is generated by the Beidou time service module 4, and the imaging module 2 sends the image with the time stamp to the main control unit 1 after marking the time stamp. The terminal disposed on the second iron tower 92 can acquire and process the images of the first iron tower 91 and the third iron tower 93, and correspondingly, the terminal disposed on the first iron tower 91 can acquire and process the images of the second iron tower 92 and the other iron tower adjacent to the first iron tower 91, and the terminal disposed on the third iron tower 93 can acquire and process the images of the second iron tower 92 and the other iron tower adjacent to the third iron tower 93.
The power supply module 8 is a solar power supply system and comprises a solar panel and a storage battery connected with the solar panel. The communication module 5 communicates with the management platform through a mobile network, a wireless network bridge or an electric power integrated data network, and the adopted communication mode can be selected according to the actual environment of the iron tower, so that the flexibility of deployment is effectively improved, and alarm information or images are sent to the management platform. The communication module 5 also receives a control instruction of the management platform, forwards the control instruction to the main control unit 1, and the main control unit 1 controls the corresponding module to work according to the received control instruction.
As shown in fig. 3, the main control unit 1 is configured to perform the following iron tower inclination detection method:
s1: the main control unit 1 receives the image sent by the imaging module 2, pre-processes the image to generate an edge image, and stores the edge image in a storage module 7 according to time sequence, wherein the pre-processing comprises graying, two times of downsampling and edge detection which are sequentially arranged, the edge detection detects the image output by the graying, the first time of downsampling and the second time of downsampling, and outputs the edge image with three scales.
S2: the main control unit 1 compares every two continuous edge images with three scales in the storage module 7, and compares the three-scale edge images generated by each image by using a sparse optical flow algorithm to obtain three comparison results, wherein a plurality of comparison results are adopted only when at least two comparison results are the same. The comparison method adopts a sparse optical flow algorithm, firstly calculates the angular point in a first edge image, and calculates the displacement amplitude and direction of the angular point between two continuous edge images. In computer vision, corner points (markers) are also called points of interest (points), key points (key points) or feature points (feature points), and are widely used for solving problems in the fields of object recognition, image matching, vision tracking, 3D reconstruction, and the like. A corner point is generally defined as the intersection of two edges, more strictly speaking, the local neighborhood of the corner point should have boundaries of different directions for two different regions. In practical applications, most corner detection methods detect image points with specific features, not just "corner points". These feature points have specific coordinates in the image and have certain mathematical features such as local maximum or minimum gray scale, certain gradient features, etc.
S3: the main control unit 1 compares a positioning offset value of a terminal on a shot iron tower with a displacement amplitude at a corresponding corner point of the terminal, firstly, according to an included angle between a positioning offset direction acquired by the Beidou positioning module 3 and an offset direction of an edge image, the offset amplitude acquired by the Beidou positioning module 3 is converted into an offset amplitude in the direction of the edge image, then the converted offset amplitude is expressed as an offset value of a pixel unit, a positioning offset value is obtained, and the edge image with the deviation of the positioning offset value and the displacement amplitude at the corresponding corner point not exceeding 3 pixels is used as an offset image.
S4: the main control unit 1 judges whether the iron tower is inclined or not according to the offset image, calculates the corresponding offset distance according to the offset vectors of all the corner points of the offset image, calculates the standard deviation of the offset distance, judges the edge image with the standard deviation not less than 2 as inclined, and judges the edge image with the standard deviation less than 2 as integral offset. If the iron tower is judged to incline, whether the iron tower swings normally or not is determined according to the displacement direction, and if the iron tower swings abnormally, warning information is sent to the management platform through the communication module 5 and the short message module 6 respectively, and the warning information further comprises position information generated by the Beidou positioning module 3.
In step S1, as shown in fig. 4, the preprocessing includes graying, two downsampling, and edge detection, which are sequentially arranged. After the iron tower image acquired by the imaging module 2 is transmitted to the main control unit 1, the main control unit 1 firstly grey-scales the input image and converts the color input image of the RGB channel into a grey-scale image; the gray level image is subjected to one-time downsampling operation to generate a first downsampled image; the first downsampled image is subjected to a downsampling operation to generate a second downsampled image. And respectively carrying out edge detection on the gray level image, the first downsampled image and the second downsampled image to obtain three edge images with different scales. By downsampling the original gray level image to obtain a first downsampled image and a second downsampled image, image information under different scales can be obtained. The images with different scales contain detail and structure information with different scales, and when edge detection is carried out, a more comprehensive edge detection result can be provided by using multi-scale information, and edge features with different scales are captured. And the image downsampling can play a role of smoothing the image, reduce high-frequency noise in the image, reduce the influence of noise on edge detection, and improve the accuracy and the robustness of the edge detection.
The conventional edge detection needs to manually set two thresholds, namely a high threshold and a low threshold, detect the edge in the image by using the set two thresholds through a Canny algorithm, and finely adjust the two thresholds according to the edge detection effect until the optimal detection effect is obtained. In the embodiment, a Canny algorithm of a self-adaptive threshold is adopted, and a proper high-low threshold is automatically selected according to the statistical characteristics of an image, wherein the specific detection method comprises the following steps of S1-1 to S1-5:
s1-1: gaussian filtering is used for smoothing an input gray image, filtering noise and obtaining a filtered image;
and constructing a Gaussian filter with standard deviation of 0.2 by using a Gaussian kernel with the size of 3 multiplied by 3, carrying out convolution operation on the constructed Gaussian filter and an input image, and carrying out weighted average on neighborhood pixels around each pixel and the Gaussian kernel by the convolution operation so as to obtain a blurred pixel value. Gaussian blur can effectively smooth noise in an image, especially high frequency noise. By reducing the details and irregularities of the noise, the image becomes clearer. Gaussian blur can blur details of an image, producing a smooth effect that can be used for image processing tasks and visual effects to effectively reduce jagged edges of the image or reduce the intensity of textures.
S1-2: calculating the gradient amplitude and the gradient direction of each pixel point in the filtered image by using a Sobel operator;
the Sobel operator is a convolution kernel with the size of 3 multiplied by 3, and each pixel in the convolution check image is used for carrying out convolution operation to obtain a gradient G in the horizontal direction x And gradient G in the vertical direction y Calculating the gradient amplitude of each pixel point according to the gradients in the horizontal and vertical directionsAnd a gradient direction atan2 (G y ,G x ) The gradient direction is specifically expressed as:
s1-3: applying non-maximum suppression to all gradient amplitude values, suppressing the response of a non-maximum region, and eliminating stray response brought by edge detection;
s1-4: and determining a strong edge, a non-edge and a weak edge by using self-adaptive high-low dual-threshold detection, marking the pixel point as a strong edge point if the gradient amplitude is larger than a high threshold, eliminating the pixel point as an edge point if the gradient amplitude is smaller than a low threshold, and marking the pixel point as a weak edge point if the gradient amplitude is between the low threshold and the high threshold.
S1-5: edge connection connects weak edge points with adjacent strong edge points.
The method for acquiring the high and low double thresholds of the self-adaptive threshold in the step S1-4 comprises the following steps: first, a gray histogram of an input image is calculated, fig. 5 is a gray histogram calculated from a gray image used in the present embodiment, and the vertical axis represents the number of pixels and the horizontal axis represents a gray value. Calculating a gray value median according to the gray histogram, and then calculating a high threshold and a low threshold according to the gray value median and a preset sigma value, wherein the calculation formula is as follows:
min＝(1-σ)×median
max＝(1+σ)×median
wherein min is a low threshold for the Canny algorithm, max is a high threshold for the Canny algorithm, sigma is a preset sigma value, and mean is a median of gray values calculated according to the gray histogram. If a high threshold max >255 is calculated, max is set to 255. In this embodiment, the value of σ is set to 0.3, when the median of the gray values is 152 by the gray histogram shown in fig. 5, the value of the low threshold value min is (1-0.3) ×152=106, the value of the high threshold value max is (1+0.3) ×152=197, all pixels of the gray image are traversed, the pixel point having the gray value not greater than 106 is marked as a weak edge point, the pixel point having the gray value not less than 197 is marked as a strong edge point, and the pixel point having the gray value in the range from 106 to 197 is marked as a non-edge point.
After the processing in step S1, in the edge image, the iron tower area is significantly different from the background, and because the optical flow algorithm used in this embodiment is a sparse optical flow algorithm, the offset vectors of all pixels of the edge image are not calculated, and only the offset vector of the corner is calculated, so before the sparse optical flow algorithm is applied, the corner of the edge image needs to be calculated, and in step S2, the method for detecting the corner is as follows:
s2-1: as shown in fig. 6, a white pixel point P with a gray value of 255 is selected from the edge image.
S2-2: as shown in fig. 7, a discretized circle with a radius of 3 pixels is drawn around the white pixel, and 16 boundary pixels are arranged on the boundary of the discretized circle, wherein the number of the boundary pixels is respectively 3 pixels right above, right below, right left and right, and the total number of the boundary pixels is 12, and the total number of the boundary pixels is respectively marked as 1-16 in the figure in the directions of 45 °, 135 °, 225 ° and 315 ° of the point P, and the total number of the boundary pixels is 4.
S2-3: if there are 10 consecutive pixels with gray values of 0 (i.e., black) for the 16 boundary pixels of the discretized circle, the point P is determined as a corner point. In fig. 7, the pixels 16, 1, 2 to 11 are 12 consecutive pixels with gray values of 0, and satisfy the corner judgment conditions: there are 10 consecutive gray values of 0, so the P point is determined as the corner point.
S2-4: repeating steps S2-1 to S2-3, traversing all pixel points in the edge image, and obtaining all corner points in the edge image as shown in figure 8.
For the edge images of the other two scales, corner points are also respectively selected.
The sparse optical flow algorithm compares two adjacent edge images in the storage module 7, in this embodiment, the first edge image and the second edge image are obtained by preprocessing two continuously shot images, the shooting time of the first edge image is longer than that of the second edge image, and the sparse optical flow algorithm specifically includes:
for a corner in the first edge image, a square first neighborhood window is defined in the first edge image by taking the corner as the center, as shown in fig. 9, a P point is taken as a corner in the first edge image, and a neighborhood window with the size of 7×7 pixels is constructed in the first edge image by taking the P point as the center.
Constructing an initial offset vector (u=0, v=0), acquiring a second neighborhood window with the same size as the first neighborhood window in the second edge image by using the initial offset vector, and if the second neighborhood window is different from the first neighborhood window, changing the offset vector to acquire the second neighborhood window again, and iteratively adjusting the offset vector until the second neighborhood window acquired in the second edge image is the same as the first neighborhood window, wherein the offset vector used in the iteration is used as the optimal offset vector of the corner point. The iterative method comprises the following steps:
taking (u, v) = (0, 0) as an initial value, firstly traversing the values of u, v within the integer range that the u is not less than-1 and not more than 1 and the v is not less than-1 and not more than 1; then expanding u and v to be equal to or more than-2 and equal to or less than-2 and less than or equal to 2, and continuing traversing the values of u and v; and so on, the range of u, v is gradually enlarged until the offset vector exceeds the pixel range of the second edge image. In each iteration process, a second neighborhood window with the size of 7 multiplied by 7 is obtained from the second edge image by using the offset vector (u, v), the second neighborhood window is compared with the first neighborhood window pixel by pixel, if the second neighborhood window is the same as the first neighborhood window, the iteration is stopped, and the offset vector (u, v) used in the iteration is used as the offset vector of the angular point.
Traversing all the corner points of the first edge image to obtain optimal offset vectors of all the corner points, as shown in fig. 10, calculating offset vectors of partial corner points, wherein the offset vectors of partial corner points are respectively A (9-2), B (9-2), C (7-1), D (7-1), E (4, 0), F (3, 0), G (1, 0) and H (0, 0). Where a (9, -2) denotes that the a corner in the second edge image is shifted to the right by 9 pixels and down by 2 pixels with respect to the first edge image.
As shown in fig. 11, the best displacement vector of the corner point is visualized in the form of an arrow, and the visualized picture can be saved.
In step S3, the positioning offset value is an offset value obtained according to the offset amplitude and the offset direction obtained by the beidou positioning module, and the offset amplitude obtained by the beidou positioning module is converted into an offset amplitude in the direction of the edge image, namely the positioning offset value, according to the included angle between the offset direction obtained by the beidou positioning module and the offset direction of the edge image.
In step S4, when the main control unit 1 detects that the iron tower is displaced according to the offset image, it is first determined whether the overall offset of the image is the offset vector of all the corner points. The method for detecting the integral offset specifically comprises the following steps: and calculating corresponding offset distances according to offset vectors of all angular points of the offset image, calculating standard deviation of the offset distances, judging the offset image with the standard deviation not smaller than 2 as inclined, and judging the offset image with the standard deviation smaller than 2 as integral offset instead of inclined. Taking example corner points A (9-2), B (9-2), C (7-1), D (7-1), E (4, 0), F (3, 0), G (1, 0) and H (0, 0) as points, the offset distance is calculated first:
…
re-calculating the average valueAnd calculating the standard deviation of the offset distances of the eight example corner points according to all the offset distances and the average value to be 3.3568126040810657, and judging the offset image as inclined according to a judging rule. If the calculated standard deviation is smaller than 2, the iron tower is judged to be wholly offset, and no treatment is carried out.
When the main control unit 1 detects that the iron tower is inclined, a continuous photographing instruction can be sent to the imaging module 2, continuous imaging is carried out in 2-4 swing periods of the iron tower, continuous images in the 2-4 swing periods are compared, if the comparison result shows that the iron tower is inclined in a left-right regular manner, the iron tower can be judged to swing normally, and if the iron tower is not inclined in the left-right regular manner, warning information, offset data and visual pictures are sent to the management platform through the communication module 5 and the short message module 6. The judgment of the normal swing is specifically as follows: dividing all continuous images in 2-4 swing periods into two groups according to the principle of symmetry of the offset direction by taking a vertical line as a symmetry axis, calculating offset distances of corner points in all images, and acquiring a first offset distance matrix of all the corner points of any image in one group of images; if an image can be obtained from another group of images, the offset distance matrix of the corner points of the image is the same as the first offset distance matrix, and the iron tower is considered to swing normally.
In other embodiments of the present application, an amplitude threshold of the offset vector may be set, and if the amplitude of the offset vector is detected to exceed the set threshold, alarm information, offset data and a visual picture are sent to the management platform through the communication module 5 and the short message module 6.
The foregoing is merely a preferred embodiment of the present application, and it should be noted that modifications and improvements could be made by those skilled in the art without departing from the inventive concept, which falls within the scope of the present application.
Claims (9)
1. The utility model provides a data processing terminal based on big dipper satellite data and AI figure fuses which characterized in that includes: the system comprises a main control unit (1), an imaging module (2), a Beidou positioning module (3), a Beidou time service module (4), a communication module (5), a short message module (6), a storage module (7) and a power module (8), wherein the terminal is arranged on a tower body of an iron tower, the imaging module (2) continuously acquires images of adjacent iron towers at set time intervals, marks the images with time stamps generated by the Beidou time service module (4), sends the images with the time stamps to the main control unit (1), and other modules in the terminal of the power module (8) supply power;
the main control unit (1) is configured to perform the following iron tower inclination detection method:
s1: the main control unit (1) receives the image sent by the imaging module (2), pre-processes the image to generate an edge image, and stores the edge image in a storage module (7) according to time sequence, wherein the pre-processing comprises graying, two downsampling and edge detection which are sequentially arranged, and the edge detection detects the image output by the graying, the first downsampling and the second downsampling respectively and outputs the edge image with three scales;
s2: the main control unit (1) compares every two continuous edge images with three scales in the storage module (7), the comparison method adopts a sparse optical flow algorithm, firstly, the angular point in a first edge image is calculated, and the displacement amplitude and direction of the angular point between the two continuous edge images are calculated;
s3: the main control unit (1) compares a positioning offset value of a terminal on a shot iron tower with a displacement amplitude at a corresponding corner point of the terminal, firstly, converts the displacement amplitude obtained by the Beidou positioning module (3) into the displacement amplitude in the direction of an edge image according to an included angle between the positioning offset direction obtained by the Beidou positioning module (3) and the displacement direction of the edge image, then, represents the converted displacement amplitude as a pixel offset value to obtain a positioning offset value, and takes the edge image with the deviation of the positioning offset value and the displacement amplitude at the corresponding corner point not exceeding 3 pixels as an offset image;
s4: the main control unit (1) judges whether the iron tower is inclined or not according to the offset image, calculates corresponding offset distances according to offset vectors of all angular points of the offset image, calculates standard deviation of the offset distances, judges that the edge image with the standard deviation not less than 2 is inclined, and judges that the edge image with the standard deviation less than 2 is overall offset; if the iron tower is judged to incline, whether the iron tower swings normally or not is determined according to the displacement direction, and if the iron tower swings abnormally, warning information is sent to the management platform through the communication module (5) and the short message module (6), and the warning information further comprises position information generated by the Beidou positioning module (3).
2. The data processing terminal based on the fusion of Beidou satellite data and AI graphics according to claim 1, wherein the edge detection adopts a Canny algorithm of a self-adaptive threshold value, and an appropriate threshold value is automatically selected according to the statistical characteristics of an image, and the detection step comprises:
gaussian filtering is used for smoothing an input gray image, filtering noise and obtaining a filtered image;
calculating the gradient amplitude and the gradient direction of each pixel point in the filtered image by using a Sobel operator;
applying non-maximum suppression to all gradient amplitudes for eliminating spurious responses from edge detection;
determining strong edges, non-edges and weak edges by using self-adaptive high-low double-threshold detection;
edge connection connects weak edge points with adjacent strong edge points.
3. The data processing terminal based on the combination of Beidou satellite data and AI graphics according to claim 2, wherein the method for acquiring the high and low double thresholds of the self-adaptive threshold is as follows: firstly, calculating a gray level histogram of an input image, calculating a gray level value median according to the gray level histogram, and then calculating a high threshold and a low threshold according to the gray level value median and a preset sigma value, wherein the calculation formula is as follows:
min＝(1-σ)×mediam
max＝(1+σ)×mediam
wherein, min is a low threshold value for a Canny algorithm, max is a high threshold value for the Canny algorithm, sigma is a preset sigma value, and mean is a median of gray values calculated according to a gray histogram; if a high threshold max >255 is calculated, max is set to 255.
4. The data processing terminal based on the fusion of Beidou satellite data and AI graphics according to claim 1, wherein the corner detection method is as follows:
s2-1: selecting a white pixel point P with a gray value of 255 from the edge image;
s2-2: drawing a discretization circle with a radius of 3 pixels by taking the white pixel as a center, wherein 16 boundary pixels are arranged on the boundary of the discretization circle;
s2-3: if 16 boundary pixels of the discretization circle have continuous 10 pixels with gray values of 0, judging the point P as a corner point;
s2-4: and repeating the steps S2-1 to S2-3, traversing all pixel points in the edge image, and obtaining all corner points in the edge image.
5. The data processing terminal based on the fusion of Beidou satellite data and AI graphics according to claim 1, wherein the sparse optical flow algorithm is specifically:
defining a square first neighborhood window by taking the corner point as the center in the first edge image for the corner point in the first edge image;
constructing an initial offset vector (u=0, v=0), acquiring a second neighborhood window with the same size as the first neighborhood window in a second edge image by using the initial offset vector, and if the second neighborhood window is different from the first neighborhood window, changing the offset vector to acquire the second neighborhood window again, and iteratively adjusting the offset vector until the second neighborhood window acquired in the second edge image is the same as the first neighborhood window, wherein the offset vector used in the iteration is used as the optimal offset vector of the corner;
traversing all angular points of the first edge image to obtain the optimal offset vector of all the angular points;
the best displacement vector of the corner point is visualized as an arrow form.
6. The data processing terminal based on Beidou satellite data and AI graphic fusion of claim 5, wherein the size of the neighborhood window is set to 7 x 7 pixels.
7. The data processing terminal based on the combination of Beidou satellite data and AI graphics according to claim 1, wherein the communication module (5) communicates with the management platform through a mobile network, a wireless network bridge or an electric power integrated data network, and sends the alarm information or the image to the management platform.
8. The data processing terminal based on the combination of Beidou satellite data and AI graphics according to claim 1 or 7, wherein the communication module (5) further receives a control instruction of the management platform, forwards the control instruction to the main control unit (1), and the main control unit (1) controls the corresponding module to work according to the received control instruction.
9. The data processing terminal based on the combination of Beidou satellite data and AI graphics according to claim 1, wherein the power module (8) is a solar power supply system and comprises a solar panel and a storage battery connected with the solar panel.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311107490.0A CN117237597A (en) | 2023-08-30 | 2023-08-30 | Data processing terminal based on Beidou satellite data and AI graph fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311107490.0A CN117237597A (en) | 2023-08-30 | 2023-08-30 | Data processing terminal based on Beidou satellite data and AI graph fusion |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117237597A true CN117237597A (en) | 2023-12-15 |
Family
ID=89093889
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311107490.0A Pending CN117237597A (en) | 2023-08-30 | 2023-08-30 | Data processing terminal based on Beidou satellite data and AI graph fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117237597A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117854256A (en) * | 2024-03-05 | 2024-04-09 | 成都理工大学 | Geological disaster monitoring method based on unmanned aerial vehicle video stream analysis |
| CN118097569A (en) * | 2024-04-26 | 2024-05-28 | 深圳市汇恒通科技有限公司 | Security monitoring system based on image fusion |
-
2023
- 2023-08-30 CN CN202311107490.0A patent/CN117237597A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117854256A (en) * | 2024-03-05 | 2024-04-09 | 成都理工大学 | Geological disaster monitoring method based on unmanned aerial vehicle video stream analysis |
| CN117854256B (en) * | 2024-03-05 | 2024-06-11 | 成都理工大学 | Geological disaster monitoring method based on unmanned aerial vehicle video stream analysis |
| CN118097569A (en) * | 2024-04-26 | 2024-05-28 | 深圳市汇恒通科技有限公司 | Security monitoring system based on image fusion |
| CN118097569B (en) * | 2024-04-26 | 2024-07-12 | 深圳市汇恒通科技有限公司 | Security monitoring system based on image fusion |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107253485B (en) | Foreign matter invades detection method and foreign matter invades detection device | |
| US9465997B2 (en) | System and method for detection and tracking of moving objects | |
| CN117237597A (en) | Data processing terminal based on Beidou satellite data and AI graph fusion | |
| CN112800860B (en) | High-speed object scattering detection method and system with coordination of event camera and visual camera | |
| JP2010539740A (en) | Runway monitoring system and method | |
| CN101383004A (en) | Passenger target detecting method combining infrared and visible light images | |
| CN103984961A (en) | Image detection method for detecting foreign matter at bottom of vehicle | |
| AU2009295350A1 (en) | Detection of vehicles in an image | |
| CN111027541A (en) | Flame detection method and system based on visible light and thermal imaging and storage medium | |
| CN114299060A (en) | Deep learning dam crack detection method suitable for complex underwater environment | |
| WO2024108971A1 (en) | Agv system for vehicle chassis corrosion evaluation | |
| CN114170552A (en) | Natural gas leakage real-time early warning method and system based on infrared thermal imaging | |
| JP2003051017A (en) | White line detector | |
| CN111079675A (en) | Driving behavior analysis method based on target detection and target tracking | |
| CN116879870A (en) | Dynamic obstacle removing method suitable for low-wire-harness 3D laser radar | |
| CN117315547A (en) | Visual SLAM method for solving large duty ratio of dynamic object | |
| CN112924037A (en) | Infrared body temperature detection system and detection method based on image registration | |
| JP7078295B2 (en) | Deformity detection device, deformation detection method, and program | |
| CN111582076A (en) | Picture freezing detection method based on pixel motion intelligent perception | |
| CN116452976A (en) | Underground coal mine safety detection method | |
| US8433139B2 (en) | Image processing apparatus, image processing method and program for segmentation based on a degree of dispersion of pixels with a same characteristic quality | |
| Tupper et al. | Pedestrian proximity detection using RGB-D data | |
| Li et al. | Advanced multiple linear regression based dark channel prior applied on dehazing image and generating synthetic haze | |
| Denman et al. | Multi-view intelligent vehicle surveillance system | |
| JP2020106484A (en) | Adhering matter detection device and adhering matter detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |