CN114693807B - A mapping data reconstruction method and system for transmission line images and point clouds - Google Patents

Abstract

The invention discloses a method and system for reconstructing mapping data of transmission line images and point clouds, which include: collecting current images through a shooting device; acquiring pre-stored mapping data, where the mapping data includes pixels of an established reference image The mapping relationship between the coordinates and the point cloud space coordinates of the point cloud data; comparing the current image with the reference image to determine whether the current image is abnormal; when the current image is abnormal, calibrating the shooting device to obtain parameters information; reconstruct the mapping data according to the parameter information; this method can improve the accuracy of the ranging task using the current image and correct the ranging error.

Description

A mapping data reconstruction method and system for transmission line images and point clouds

Technical field

The present invention relates to the field of data processing technology, and in particular to a mapping data reconstruction method and system for transmission line images and point clouds.

Background technique

LiDAR scanning technology is an emerging three-dimensional data acquisition technology. LiDAR scanners mounted on different platforms such as tripods, cars, airplanes, and satellites can quickly obtain massive point cloud data. Point cloud data contains rich information such as the longitude and latitude coordinates, intensity, multiple echoes and color of each point, and has relevant applications in fields such as surveying and mapping, forestry, agriculture and digital cities.

The two-dimensional visible light images collected by drones need to undergo a reconstruction process to restore the three-dimensional power scene, and there will be a certain deviation from the actual situation. The technical advantage of lidar point cloud mapping is that it can accurately restore the spatial information of power scenes and perform distance measurements. However, the disadvantage of lidar point cloud mapping is that it cannot restore the color information in power scenes and has poor visualization effects. Using lidar Point cloud data is difficult to accurately classify electrical objects. Therefore, in practical applications, the pixels in the two-dimensional image are often corresponding to the point cloud spatial coordinates in the laser point cloud data. For the image and point cloud data of the same transmission line site, the target pixel coordinates in the image can be matched with the target point. The point cloud data (spatial coordinates of the target point) establishes a mapping relationship. Based on this mapping relationship, ranging tasks can be performed. For example, the actual distance in the real world between targets corresponding to two target pixels in the image can be determined.

However, after the shooting device becomes abnormal due to aging, displacement, etc., the shooting device will output abnormal images, and the mapping relationship established between the original point cloud data and the image will not be applicable to the abnormal image at this time. Using the abnormal image When performing ranging tasks, large ranging errors will occur, and ranging accuracy cannot be guaranteed. Patent document CN102982548A provides a multi-view stereo video acquisition system and its camera parameter calibration method: obtain the internal and external parameters of each camera in the system; collect multi-viewpoint images of ordinary scenes through each camera at the same time, and perform feature point detection on the multi-viewpoint images and matching to obtain the matching points between each viewpoint image; use camera parameter reconstruction to obtain the three-dimensional space point cloud coordinates of the matching points between each viewpoint image; use sparse bundle adjustment and optimization to obtain the weight based on the three-dimensional space point cloud coordinates and the internal and external parameters of the camera projection error, and optimize the reprojection error and the internal and external parameters of the camera; determine whether to perform secondary optimization based on the optimized reprojection error; and determine whether to perform parameter recalibration based on the secondary optimization results. However, this method cannot solve the problem of inaccurate acquisition of point cloud coordinates in abnormal images due to aging, displacement and other reasons of the shooting device, which affects the ranging task.

Contents of the invention

The present invention provides a mapping data reconstruction method and system for transmission line images and point clouds, which can adjust existing mapping data, re-establish the mapping relationship for the current image, and improve the measurement results using the current image. distance task accuracy, correcting the ranging error.

A method for reconstructing mapping data of transmission line images and point clouds, including:

Collect the current image through the shooting device;

Obtain pre-stored mapping data, which includes the established mapping relationship between the pixel coordinates of the reference image and the point cloud spatial coordinates of the point cloud data;

Compare the current image with the reference image to determine whether the current image is abnormal;

When the current image is abnormal, calibrate the shooting device to obtain parameter information;

The mapping data is reconstructed according to the parameter information.

Further, comparing the current image with the reference image to determine whether the current image is abnormal includes:

Select the feature points in the current image and compare the pixel coordinates of the feature points in the current image with the pixel coordinates of the corresponding feature points in the reference image. If the pixel coordinates of the feature points in the current image are consistent with the pixels of the corresponding feature points in the reference image If the coordinates are equal, the current image is normal. If the pixel coordinates of the feature points in the current image are not equal to the pixel coordinates of the corresponding feature points in the reference image, the current image is determined to be an abnormal image.

Further, determining whether the current image is abnormal also includes:

Calculate multiple sets of offsets between the pixel coordinates of multiple feature points in the current image and the pixel coordinates of the corresponding multiple feature points in the reference image. If the offsets in the multiple sets are equal, it is determined that the current image is offset. If the offsets in the multiple sets are equal, If the shift amounts are not equal, it is determined that the current image is distorted.

Further, the parameter information includes internal parameters, external parameters and distortion parameters of the shooting device.

Further, reconstructing the mapping data according to the parameter information includes:

For the current image that has been shifted, the mapping data is reconstructed based on the external parameters, internal parameters, pixel coordinates of multiple feature points in the current image and the spatial coordinates of the corresponding point clouds in the point cloud data.

Further, reconstructing the mapping data according to the parameter information includes:

For the current distorted image, the mapping data is reconstructed according to the internal parameters, external parameters and distortion parameters.

Further, reconstructing the mapping data according to the internal parameters, external parameters and distortion parameters includes:

According to the internal parameters and external parameters, the spatial coordinates of the characteristic point cloud in the calculated point cloud data correspond to the normal pixel coordinates in the normal image;

According to the distortion parameter, calculate the normal pixel coordinates corresponding to the target pixel coordinates in the distorted current image;

The mapping data is reconstructed according to the internal parameters, external parameters, target pixel coordinates and spatial coordinates of the feature point cloud.

Further, the distortion parameters include a radial deformation coefficient and a tangential deformation coefficient, and the target pixel coordinates are calculated by the following formula:

x”=x'×(1+k ₁ ×r ² +k ₂ ×r ⁴ )+2×p ₁ ×x'×y'+p ₂ ×(r ² +2×x ^'2 );

y”=y'×(1+k ₁ ×r ² +k ₂ ×r ⁴ )+2×p ₂ ×x'×y'+p ₁ ×(r ² +2×y ^'2 );

Among them, r ² =x ^'2 + y ^'2 , r represents the distortion factor of the physical coordinates of the image, k1 and k2 are the radial deformation coefficients, p1 and p2 are the tangential deformation coefficients, u _d and v _d are the current distortion coefficients The pixel coordinates in the image, u, v are the normal pixel coordinates in the normal image, x', y' are the intermediate quantities from the camera coordinate system to the image coordinate system, x" and y" are the coordinates of the distortion position, c _x , c _y is the offset of the optical axis from the coordinate center of the projection plane, f _x and f _y are the focal lengths of the shooting device.

Further, after calculating that the normal pixel coordinates correspond to the target pixel coordinates in the distorted current image, the method further includes:

Rounds non-integer target pixel coordinates.

A mapping data reconstruction system for transmission line images and point clouds, including a shooting device and a server. The server includes a processor and a storage device. The storage device stores a plurality of instructions. The processor is used to read the Multiple instructions and execute the above method.

The mapping data reconstruction method and system of transmission line images and point clouds provided by the present invention at least include the following beneficial effects:

Effective abnormality identification can be performed on the current image collected by the shooting device, so that the mapping data can be reconstructed according to the abnormal situation (offset or distortion), and the relationship between the current image captured by the shooting device and the point cloud data can be re-established. Mapping relationship, improves the accuracy of ranging tasks using the current image, and corrects ranging errors.

Description of drawings

Figure 1 is a flow chart of an embodiment of the reconstruction method of mapping data of transmission line images and point clouds provided by the present invention.

Figure 2 is a schematic diagram of a distorted image in an application scenario of the reconstruction method of mapping data of transmission line images and point clouds provided by the present invention.

Figure 3 is a schematic structural diagram of an embodiment of a reconstruction device for mapping data of transmission line images and point clouds provided by the present invention.

Figure 4 is a flow chart of an embodiment of the reconstruction system for mapping data of transmission line images and point clouds provided by the present invention.

Detailed ways

In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation modes.

In order to facilitate the understanding of this application, some concepts involved in this application are first described.

Lidar (Laser Imaging Detection and Ranging, referred to as LIDAR): By emitting outgoing light (such as a laser beam) with a wavelength of about 900nm, the outgoing light will be reflected by the obstacle when it encounters an obstacle. Calculate the distance between the obstacle and the lidar using the time difference between them. In addition, the processing unit can also estimate the reflectivity of the target based on the cross-sectional condition of the reflected light signal obtained after receiving the reflected light. Due to its small size and high degree of integration, airborne lidar has more and more application scenarios.

Point cloud data refers to a set of vectors in a three-dimensional coordinate system. Scanning data is recorded in the form of points. Each point contains three-dimensional coordinates, and some may contain color information (such as red, green, and blue) or reflection intensity information (Intensity).

In related technologies, airborne lidar and other methods can be used for map surveying, and the map contains location information of multiple objects. However, point cloud data is not as intuitive as image data. If image data can be directly used to measure target objects, it can effectively improve the convenience of transmission line detection.

In order to directly use transmission line images for accurate measurement, each pixel in the image needs to have spatial coordinate information. By fusing two types of data (image data and point cloud data) for the same target object, the fused transmission line image can be used to measure the clearance distance of the transmission line, etc., with an accuracy reaching sub-meter level.

Referring to Figure 1, in some embodiments, a method for reconstructing mapping data of transmission line images and point clouds is provided, including:

S1. Collect the current image through the shooting device;

S2. Obtain pre-stored mapping data, which includes the established mapping relationship between the pixel coordinates of the reference image and the point cloud spatial coordinates of the point cloud data;

S3. Compare the current image with the reference image to determine whether the current image is abnormal;

S4. When the current image is abnormal, calibrate the shooting device to obtain parameter information;

S5. Reconstruct the mapping data according to the parameter information.

Specifically, in step S1, the photographing device can be installed on a power tower on the transmission line, and can capture images corresponding to the transmission line.

Further, in step S2, the shooting device captures an image in a normal state to establish a mapping relationship with the point cloud data. The image is a reference image. The point cloud data is obtained through lidar and the pixel coordinates and points of the reference image are established. The mapping relationship of the point cloud spatial coordinates of the cloud data is formed into mapping data for storage.

In this step, the mapping relationship between the reference image and the point cloud data includes the mapping relationship between the target pixel coordinates in the reference image and the spatial coordinates of the target point in the point cloud data. The target points may be characteristic points (such as corner points, end points, vertices, center points, etc.) of equipment (such as cross arms, insulators, etc.) on the transmission line site. For example, the mapping data may include: a mapping relationship between the pixel coordinates of the center point of the insulator in the reference image and the spatial coordinates of the center point of the insulator in the point cloud data (such as world coordinates, including longitude, latitude and altitude), For example, the pixel coordinates (u ₁ , v ₁ ) of the center point of the insulator in the reference image correspond to the spatial coordinates (X ₁ , Y ₁ , Z ₁ ) of the center point of the insulator in the point cloud data. The mapping relationship can be a functional relationship. After obtaining the target pixel coordinates in the reference image, the spatial coordinates of the target point in the point cloud data can be obtained correspondingly according to the mapping relationship. After obtaining the spatial coordinates of the target point in the point cloud data, the target pixel coordinates in the reference image can also be obtained correspondingly according to the mapping relationship.

It can be understood that after obtaining the mapping data, the mapping relationship between the target pixel coordinates in the reference image and the spatial coordinates of the target point in the point cloud data can be learned. That is to say, select a target pixel point in the reference image, and use the mapping data to obtain the spatial coordinates corresponding to the target pixel point, and then determine the actual distance in the real world between the two target pixel points in the reference image. In this way, ranging tasks can be performed based on the reference image, which can be applied to on-site monitoring of transmission lines. When the photographing device is used as a surveillance camera installed on a power transmission line site, the images captured by the photographing device are relatively stable unless there is any abnormality in the photographing device due to aging, displacement, or other reasons. That is to say, when there is no abnormality in the photographing device due to aging, displacement, etc., the current image captured by the photographing device is equivalent to the reference image. Therefore, according to the established mapping relationship between the reference image captured by the capturing device and the point cloud data, the current image captured by the capturing device can be used to perform the ranging task. However, after the shooting device becomes abnormal due to aging, displacement, etc., the current image captured by the shooting device will be abnormal. The current image captured by the shooting device is not equal to the reference image. If the abnormal current image is used for ranging Tasks, large ranging errors will occur, and ranging accuracy cannot be guaranteed.

Therefore, it is necessary to determine whether an abnormality has occurred in the photographing device based on the current image collected by the photographing device.

Further, in step S3, the current image is compared with the reference image to determine whether the current image is abnormal, including:

Select the feature points in the current image and compare the pixel coordinates of the feature points in the current image with the pixel coordinates of the corresponding feature points in the reference image. If the pixel coordinates of the feature points in the current image are consistent with the pixels of the corresponding feature points in the reference image If the coordinates are equal, the current image is normal. If the pixel coordinates of the feature points in the current image are not equal to the pixel coordinates of the corresponding feature points in the reference image, the current image is determined to be an abnormal image.

Further, determining whether the current image is abnormal also includes:

Calculate multiple sets of offsets between the pixel coordinates of multiple feature points in the current image and the pixel coordinates of the corresponding multiple feature points in the reference image. If the offsets in the multiple sets are equal, it is determined that the current image is offset. If the offsets in the multiple sets are equal, If the shift amounts are not equal, it is determined that the current image is distorted.

The distortion may include lens distortion, such as barrel distortion and pincushion distortion. Lens distortion is the general term for the inherent perspective distortion of optical lenses, which is the distortion caused by perspective. Referring to Figure 2, barrel distortion and pincushion distortion are shown.

The wide-angle lens brings barrel distortion while achieving a wide field of view and special shooting effects. Although barrel distortion does not affect imaging clarity, it does affect the positional accuracy of imaging, which can bring errors or even misjudgments to image analysis and image measurement. The barrel distortion that a wide-angle lens brings to the visual system is non-linear. The deformation is relatively small at the center of the image, and the farther away from the center of the image, the greater the distortion. Pincushion distortion is the phenomenon of the image "shrinking" to the middle caused by the lens. Pincushion distortion is most easily noticeable when using a telephoto lens or when using the telephoto end of a zoom lens. In transmission line monitoring scenarios, due to the long distance between power towers, it is often necessary to use telephoto lenses or wide-angle lenses, resulting in obvious image distortion caused by lens distortion.

Further, in step S4, when the current image is abnormal, the shooting device is calibrated to obtain parameter information. In this step, the photographing device can be calibrated using a checkerboard calibration method to obtain parameter information of the photographing device.

Specifically, the checkerboard calibration method is used to calibrate the shooting device, which includes: preparing a checkerboard with a known size, shooting it from different angles with the shooting device, and obtaining a set of images, and comparing the feature points in the image. For example, if the corner points of the calibration board are detected, the pixel coordinates of the corners of the calibration board are obtained. Based on the known checkerboard size and the origin of the world coordinate system, the physical coordinates of the corners of the calibration board are calculated. The pixel coordinates of each corner point, The physical coordinates of each corner point in the world coordinate system are used to calibrate the camera and obtain the internal and external parameter matrix and distortion parameters of the camera.

Parameter information includes internal parameters, external parameters and distortion parameters of the shooting device.

Further, in step S5, for the current image that has been shifted but not distorted, the mapping data is reconstructed according to the parameter information, including:

For the current image that has been shifted, the mapping data is reconstructed based on the external parameters, internal parameters, pixel coordinates of multiple feature points in the current image and the spatial coordinates of the corresponding point clouds in the point cloud data.

Specifically, the internal parameters include the physical size dx of each pixel on the horizontal axis x of the image, the physical size dy of each pixel on the vertical axis y of the image, the distortion factor r of the physical coordinates of the image, the focal length f, the central pixel coordinates of the image and The number of horizontal pixels u ₀ that differs between the pixel coordinates of the origin of the image, the number of horizontal and vertical pixels v ₀ that differs between the center pixel coordinates of the image and the pixel coordinates of the origin of the image, that is to say, (u ₀ , v ₀ ) represents the optical axis of the camera Pixel coordinates of intersection with the image plane.

The external parameters include the rotation matrix R and the translation vector T that transform the spatial coordinate system into the camera coordinate system.

Further, when the current image is not distorted, for the same target, the spatial coordinates of the target in the point cloud data, the imaging midpoint of the shooting device, and the pixel coordinates of the target in the current image are: Collinear, based on this collinear relationship, construct an expression as shown in equation (1):

In formula (1), (x, y) is the image plane coordinate of the target point. (u ₀ , v ₀ ) are the pixel coordinates of the intersection between the optical axis of the shooting device and the image plane, and f represents the focal length. (X _s , Y _S , Z _S ) are the coordinates of the center of the shooting device in the point cloud coordinate system, and (X _A , Y _A , Z _A ) represent the spatial coordinates of the target point in the point cloud data. a _i , b _i , c _i (i=1,2,3) are the rotation matrices of the image.

Among them, the expression of the rotation matrix R is as shown in Equation (2):

Since the image plane coordinates (x, y) of the target point and the pixel coordinates (u, v) of the target point can be transformed accordingly. By calibrating the shooting device, the intrinsic and extrinsic parameters are known, that is, (u ₀ , v ₀ ), f, (X _s , Y _S , Z _S ) are known. In this way, according to the spatial coordinates (X _A , Y _A , Z _A ) of the target point in the point cloud data, the target pixel coordinates (u, v) in the current image can be determined. In this way, the mapping relationship between the target pixel coordinates in the current image captured by the shooting device and the spatial coordinates of the target point in the point cloud data is constructed.

In another embodiment, according to the intrinsic and extrinsic parameters in the parameter information, the mapping relationship between the current image captured by the shooting device and the point cloud data can be constructed according to the spatial transformation relationship.

The conversion relationship between pixel coordinates and spatial coordinates can be expressed as shown in equation (3):

In formula (3), d _x and d _y represent the physical size of each pixel on the horizontal axis x and vertical axis y of the image respectively, and (u ₀ , v ₀ ) is the pixel coordinate of the intersection of the optical axis of the shooting device and the image plane. , f represents the focal length of the shooting device, and the above parameters are all internal parameters. (u, v) are the pixel coordinates of the target point, (X _w , Y _W , Z _W ) are the spatial coordinates of the target point in the point cloud data. R represents the rotation matrix and T represents the translation vector.

In some embodiments, the rotation matrix R can also be expressed as shown in equation (4):

For example, Indicates the angle at which the camera coordinate axis rotates around the y-axis, x-axis and z-axis of the point cloud coordinate system respectively, that is, the attitude angle, as shown in Equation (5).

T＝[t _x , t _y , t _z ]; (5)

Among them, t _x , t _y , and t _z respectively represent the position coordinate values of the camera center in the point cloud coordinate system.

Since the intrinsic and extrinsic parameters of the shooting device are known, according to the spatial coordinates (X _w , Y _W , Z _W ) of the target point in the point cloud data, the target pixel coordinates (u, v) in the current image can be determined. In this way, the mapping relationship between the target pixel coordinates in the current image captured by the shooting device and the spatial coordinates of the target point in the point cloud data is constructed.

Further, in step S5, if the current image has been distorted, reconstruct the mapping data according to the parameter information, including:

For the current distorted image, the mapping data is reconstructed according to the internal parameters, external parameters and distortion parameters.

Specifically, reconstructing the mapping data according to the internal parameters, external parameters and distortion parameters includes:

According to the internal and external parameters, the spatial coordinates of the characteristic point cloud in the calculated point cloud data correspond to the normal pixel coordinates in the normal image. Specifically, in this step, according to the internal and external parameters in the parameter information, the above expression is used Formula or Formula (3) can calculate the spatial coordinates of the target point in the point cloud data corresponding to the normal pixel coordinates in the normal image;

According to the distortion parameter, calculate the normal pixel coordinates corresponding to the target pixel coordinates in the distorted current image;

The mapping data is reconstructed according to the internal parameters, external parameters, target pixel coordinates and spatial coordinates of the feature point cloud. Specifically, the mapping data can be reconstructed according to the above expressions (1)-(5).

The mathematical model of distortion is explained below.

The camera's internal parameter matrix A(dx,dy,r,u,v,f), external parameter matrix [R|T], distortion coefficient [k1,k2,k3,p1,p2], the physical size dx and dy of a pixel , focal length f, distortion factor r of the physical coordinates of the image.

The mathematical model of radial distortion is shown in Equation (6):

Among them, r ² = ^x'2 + ^y'2 , the radial distortion at the edge of the image is large.

The mathematical model of tangential distortion is shown in Equation (7):

Among them, the above five vectors k1, k2, k3, P1, and P2 are all distortion parameters, u and v are the pixel coordinates in the distorted image, and u' and v' are the corrected pixel coordinates.

It can be understood that x _c , y _c , z _c are the coordinates of the pixel point in the camera coordinate system, x', y' are the intermediate quantities from the camera coordinate system to the image coordinate system, and the normal position of a pixel in the pixel coordinate system of the image The coordinates can be expressed as equations (8) to (10):

x'＝x _c /z _c ; (8)

y'＝y _c /z _c ; (9)

x” and y” are the distortion position coordinates, which can be expressed as equations (11) to (13):

x”＝x'×(1+k ₁ ×r ² +k ₂ ×r ⁴ )+2×p ₁ ×x'×y'+p ₂ ×(r ² +2×x ^'2 ); (11)

y”=y'×(1+k ₁ ×r ² +k ₂ ×r ⁴ )+2×p ₂ ×x'×y'+p ₁ ×(r ² +2×y ^'2 ); (12)

Among them, r ² =x ^'2 + y ^'2 , r represents the distortion factor of the physical coordinates of the image, k1 and k2 are the radial deformation coefficients, p1 and p2 are the tangential deformation coefficients, u _d and v _d are the current distortion coefficients The pixel coordinates in the image, u, v are the normal pixel coordinates in the normal image, x', y' are the intermediate quantities from the camera coordinate system to the image coordinate system, x" and y" are the coordinates of the distortion position, c _x , c _y is the offset of the optical axis from the coordinate center of the projection plane, f _x and f _y are the focal lengths of the shooting device.

In order to determine an image without distortion based on a known distorted image, its mapping relationship can be derived through a distortion model.

The relationship between the normal image imgR and the distorted image imgD is as shown in Equations (14) to Equations (16):

Among them, r ² =x ^'2 + y ^'2 , r represents the distortion factor of the physical coordinates of the image, k1 and k2 are the radial deformation coefficients, p1 and p2 are the tangential deformation coefficients, u _d and v _d are the current distortion coefficients The pixel coordinates in the image, u, v are the normal pixel coordinates in the normal image, x', y' are the intermediate quantities from the camera coordinate system to the image coordinate system, x" and y" are the coordinates of the distortion position, c _x , c _y is the offset of the optical axis from the coordinate center of the projection plane, f _x and f _y are the focal lengths of the shooting device, and generally they are equal.

Since it has been determined that the current image captured by the shooting device is distorted, the current image is a distorted abnormal image. In this way, after the normal pixel coordinates (u, v) in the normal image are calculated in the above steps, the normal pixel coordinates (u, v) corresponding to the current image can be calculated according to the above equations (14) to (16). The target pixel coordinates (u _d , v _d ) are thus constructed to construct a mapping relationship between the target pixel coordinates in the current image captured by the shooting device and the spatial coordinates of the target point in the point cloud data.

It can be understood that the normal pixel coordinates (u, v) of a normal image are integers, for example, the normal pixel coordinates (1,1). However, the calculated target pixel coordinates in the current image corresponding to the normal pixel coordinates may be non-integers. For example, the target pixel coordinates (1.1, 1.4) are calculated.

In some embodiments, when the calculated target pixel coordinate is a non-integer pixel coordinate, an integer pixel coordinate adjacent to the non-integer pixel coordinate in the current image is used as the target pixel coordinate. For example, you can approximate values by rounding. For example, when the calculated target pixel coordinate is (1.1, 1.2), the integer pixel coordinate (1, 1) in the current image can be used as the target pixel coordinate. In this way, the mapping relationship between the target pixel coordinates in the current image and the spatial coordinates of the target point in the point cloud data can be constructed.

In another implementation, when the calculated target pixel coordinate is a non-integer pixel coordinate, the current image is enlarged proportionally, and an integer pixel coordinate corresponding to the non-integer pixel coordinate is selected as the target in the enlarged current image. Pixel coordinates. For example, when the calculated target pixel coordinate is (1.1, 1.2), the current image can be enlarged 10 times, and the integer pixel coordinate (11, 12) in the enlarged current image is selected as the target pixel coordinate. In this way, the mapping relationship between the target pixel coordinates in the enlarged current image and the spatial coordinates of the target point in the point cloud data can be constructed.

It can be seen from this embodiment that the method provided by the embodiment of the present application can use the newly constructed mapping relationship between the current image and the point cloud data to adjust the relationship between the established reference image and the point cloud data in the original mapping data. Mapping relationship to obtain adjusted mapping data. Based on the mapping relationship between the current image and the point cloud data in the adjusted mapping data, the current image captured by the shooting device can be used to perform the ranging task, and the ranging accuracy is high and the error is small.

The method provided by the above embodiments at least includes the following beneficial effects:

Effective abnormality identification can be performed on the current image collected by the shooting device, so that the mapping data can be reconstructed according to the abnormal situation (offset or distortion), and the relationship between the current image captured by the shooting device and the point cloud data can be re-established. Mapping relationship, improves the accuracy of ranging tasks using the current image, and corrects ranging errors.

Further, referring to Figure 3, in some embodiments, a device for reconstructing mapping data of transmission line images and point clouds is also provided, including:

The collection module 201 is used to collect the current image through the shooting device;

The data acquisition module 202 is used to acquire pre-stored mapping data, which includes the established mapping relationship between the pixel coordinates of the reference image and the point cloud spatial coordinates of the point cloud data;

Determination module 203 is used to compare the current image with the reference image and determine whether the current image is abnormal;

Calibration module 204: when the current image is abnormal, calibrate the shooting device to obtain parameter information;

The reconstruction module 205 is configured to reconstruct the mapping data according to the parameter information.

Specifically, the judgment module 203 is also used to:

Select the feature points in the current image and compare the pixel coordinates of the feature points in the current image with the pixel coordinates of the corresponding feature points in the reference image. If the pixel coordinates of the feature points in the current image are consistent with the pixels of the corresponding feature points in the reference image If the coordinates are equal, the current image is normal. If the pixel coordinates of the feature points in the current image are not equal to the pixel coordinates of the corresponding feature points in the reference image, the current image is determined to be an abnormal image.

The judgment module 203 is also used for::

Calculate multiple sets of offsets between the pixel coordinates of multiple feature points in the current image and the pixel coordinates of the corresponding multiple feature points in the reference image. If the offsets in the multiple sets are equal, it is determined that the current image is offset. If the offsets in the multiple sets are equal, If the shift amounts are not equal, it is determined that the current image is distorted.

The parameter information includes internal parameters, external parameters and distortion parameters of the shooting device.

Further, the reconstruction module 205 is also used to:

For the current image that has been shifted, the mapping data is reconstructed based on the external parameters, internal parameters, pixel coordinates of multiple feature points in the current image and the spatial coordinates of the corresponding point clouds in the point cloud data.

Further, the reconstruction module 205 is also used to:

For the current distorted image, the mapping data is reconstructed according to the internal parameters, external parameters and distortion parameters.

Further, the reconstruction module 205 is also used to:

According to the internal parameters and external parameters, the spatial coordinates of the characteristic point cloud in the calculated point cloud data correspond to the normal pixel coordinates in the normal image;

According to the distortion parameter, calculate the normal pixel coordinates corresponding to the target pixel coordinates in the distorted current image;

The mapping data is reconstructed according to the internal parameters, external parameters, target pixel coordinates and spatial coordinates of the feature point cloud.

Please refer to the above embodiment for the specific reconstruction method, which will not be described again here.

Referring to Figure 4, in some embodiments, a mapping data reconstruction system for transmission line images and point clouds is also provided, including a shooting device 301 and a server 302. The server 302 includes a processor 3021 and a storage device 3022. The storage device 3022 stores There are multiple instructions, and the processor 3021 is used to read the multiple instructions and execute the above method.

The processor can be a central processing unit (Central Processing Unit, CPU), or other general-purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable Gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

Storage may include various types of storage units, such as system memory, read-only memory (ROM), and persistent storage.

Although the preferred embodiments of the present invention have been described, those skilled in the art will be able to make additional changes and modifications to these embodiments once the basic inventive concepts are apparent. Therefore, it is intended that the appended claims be construed to include the preferred embodiments and all changes and modifications that fall within the scope of the invention. Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the invention. In this way, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims (10)

1. A method for reconstructing mapping data of transmission line images and point clouds, which is characterized by including: Collect the current image through the shooting device; Obtain pre-stored mapping data, which includes the established mapping relationship between the pixel coordinates of the reference image and the point cloud spatial coordinates of the point cloud data; Compare the current image with the reference image to determine whether the current image is abnormal; When the current image is abnormal, calibrate the shooting device to obtain parameter information; reconstruct the mapping data according to the parameter information; Compare the current image with the reference image to determine whether the current image is abnormal, including: Select the feature points in the current image and compare the pixel coordinates of the feature points in the current image with the pixel coordinates of the corresponding feature points in the reference image. If the pixel coordinates of the feature points in the current image are consistent with the pixels of the corresponding feature points in the reference image If the coordinates are equal, the current image is normal. If the pixel coordinates of the feature points in the current image are not equal to the pixel coordinates of the corresponding feature points in the reference image, the current image is determined to be an abnormal image; Determining whether the current image is abnormal also includes: Calculate multiple sets of offsets between the pixel coordinates of multiple feature points in the current image and the pixel coordinates of the corresponding multiple feature points in the reference image. If the offsets in the multiple sets are equal, it is determined that the current image is offset. If the offsets in the multiple sets are equal, If the shift amounts are not equal, it is determined that the current image is distorted; The parameter information includes internal parameters, external parameters and distortion parameters of the shooting device; Reconstructing the mapping data according to the parameter information includes: For the current distorted image, reconstruct the mapping data according to the internal parameters, external parameters and distortion parameters; According to the internal parameters, external parameters and distortion parameters, the mapping data is reconstructed, including: According to the internal parameters and external parameters, the spatial coordinates of the characteristic point cloud in the calculated point cloud data correspond to the normal pixel coordinates in the normal image; According to the distortion parameter, calculate the normal pixel coordinates corresponding to the target pixel coordinates in the distorted current image; Reconstruct the mapping data according to the internal parameters, external parameters, target pixel coordinates and spatial coordinates of the feature point cloud; The distortion parameters include radial deformation coefficients and tangential deformation coefficients, and the target pixel coordinates are calculated by the following formula: x″=x′×(1+k ₁ ×r ² +k ₂ ×r ⁴ )+2×p ₁ ×x′×y′+p ₂ ×(r ² +2×x′ ² ); y″=y′×(1+k ₁ ×r ² +k ₂ ×r ⁴ )+2×p ₂ ×x′×y′+p ₁ ×(r ² +2×y′ ² ); Among them, r ² =x′ ² +y′ ² , r represents the distortion factor of the physical coordinates of the image, k1 and k2 are the radial deformation coefficients, p1 and p2 are the tangential deformation coefficients, u _d and v _d are the current distortion coefficients The pixel coordinates in the image, u, v are the normal pixel coordinates in the normal image, x′, y′ are the intermediate quantities from the camera coordinate system to the image coordinate system, x″ and y″ are the coordinates of the distortion position, c _x , c _y is the offset of the optical axis from the coordinate center of the projection plane, f _x and f _y are the focal lengths of the shooting device. 2. The method according to claim 1, characterized in that, reconstructing the mapping data according to the parameter information includes: For the current image that has been shifted, the mapping data is reconstructed based on the external parameters, internal parameters, pixel coordinates of multiple feature points in the current image and the spatial coordinates of the corresponding point clouds in the point cloud data. 3. The method according to claim 1, characterized in that after calculating that the normal pixel coordinates correspond to the target pixel coordinates in the distorted current image, it further includes: Rounds non-integer target pixel coordinates. 4. The method according to claim 1, characterized in that the photographing device is installed on a power tower of a power transmission line. 5. The method according to claim 1, characterized in that the point cloud data is obtained by laser radar. 6. The method according to claim 1, characterized in that the photographing device is calibrated using a checkerboard calibration method to obtain parameter information. 7. The method according to claim 6, characterized in that using a checkerboard calibration method to calibrate the shooting device includes: Prepare a checkerboard with a known size. Use a shooting device to shoot it from different angles to obtain a set of images. Detect the corner points of the calibration plate in the image and obtain the pixel coordinates of the corner points of the calibration plate. According to With the known checkerboard size and the origin of the world coordinate system, we can calculate the physical coordinate values of the corner points of the calibration board, the pixel coordinates of each corner point of the calibration board, and the physical coordinates of each corner point of the calibration board in the world coordinate system. Calibration of the shooting device to obtain the internal and external parameter matrix and distortion parameters of the shooting device. 8. The method according to claim 1, characterized in that the internal parameters include the physical size of each pixel on the horizontal axis of the image, the physical size of each pixel on the vertical axis of the image, the distortion factor r of the physical coordinates of the image, and the focal length. f. The number of horizontal pixels that differ between the center pixel coordinates of the image and the pixel coordinates of the origin of the image, and the number of horizontal and vertical pixels that differ between the center pixel coordinates of the image and the pixel coordinates of the origin of the image. 9. The method according to claim 1, characterized in that the external parameters include a rotation matrix and a translation vector for converting the spatial coordinate system into the camera coordinate system. 10. A mapping data reconstruction system for transmission line images and point clouds, characterized in that it includes a shooting device and a server, the server includes a processor and a storage device, the storage device stores a plurality of instructions, and the processing The processor is used to read the plurality of instructions and execute the method according to any one of claims 1-9.