CN113807348A - A kind of high-voltage cable target identification and positioning method and device - Google Patents

Abstract

The invention relates to a high-voltage cable target identification and positioning method and device. The Yolo v4 deep neural network model is optimized and the ROI area of the target is extracted by using it; then, combined with the HSV color tracking technology, the minimum external connection of the cable target in the ROI area is extracted. Finally, combined with the parameter information of the depth camera, the three-dimensional coordinate information of multiple equal points on the cable target is obtained, so as to realize the extraction of the three-dimensional coordinates of the high-voltage cable target, and then provide more information for the action of the mechanical arm of the live working robot. Accurate target space information. The positioning and identification method provided by the present invention has faster identification speed in identification of high-voltage cables, improved identification accuracy, and higher reliability, and solves the problem of inconsistencies in identification and positioning of target identification in the prior art under different backgrounds. exact question.

Description

A kind of high-voltage cable target identification and positioning method and device

technical field

The invention relates to the technical field of power grid equipment detection, and in particular, to a method and device for identifying and locating a high-voltage cable target.

Background technique

In recent years, with the steady progress of the national smart grid construction and the increasing requirements for power supply stability, traditional distribution network maintenance methods have become more and more difficult to meet actual needs due to the shortcomings of high work intensity, high safety risks, and low operation efficiency. In order to follow the principle of "unstoppable operation" in the maintenance work of the distribution network, power supply enterprises in various regions are constantly strengthening the construction of the uninterrupted power supply of the distribution network, and the uninterrupted operation has become an important means of equipment maintenance.

Relying on robotic arm control technology, artificial intelligence algorithms, visual sensors, etc., the live working robot system has become a relatively common means of power maintenance, operation and maintenance, and has become an important operation method to replace manual operations. In the prior art, the development of semi-autonomous and fully autonomous live working robots has gradually become an inevitable trend in the overhaul and maintenance of power scenarios. However, there are still various problems in the practical application of live working robots, especially the accuracy of target positioning and recognition, which needs to be further improved.

SUMMARY OF THE INVENTION

Based on the above situation of the prior art, the purpose of the present invention is to provide a high-voltage cable target identification and positioning method and device, using the optimized Yolo v4 target detection technology and color tracking technology, combined with a depth camera, to achieve high-voltage cable target detection Extraction of 3D coordinates. Thus, the problem of inaccurate identification and positioning in the prior art of target identification under different backgrounds is solved.

In order to achieve the above object, according to one aspect of the present invention, a method for identifying and locating a target of a high-voltage cable is provided, comprising the steps of:

Collect high-voltage cable images and perform preprocessing;

Optimize the YOLO v4 deep neural network model, simplify the number of residual units in its backbone network, and reduce the number of channels;

Use the trained deep neural network model to identify the high-voltage cable target in the image and extract the ROI area of the target;

According to the intrinsic properties and depth information of the high-voltage cable target, extract the minimum circumscribed rectangle of the target in the ROI area;

Divide the cable target into N equal parts in length according to the minimum circumscribed rectangle, and obtain the three-dimensional coordinates of the divided points;

Positioning is performed using the three-dimensional coordinates.

Further, the collecting and preprocessing of the high-voltage cable target image includes:

Use a depth camera to collect high-voltage cable images;

Annotate the acquired images.

Further, using the trained deep neural network model to identify the high-voltage cable target in the image and extract the ROI area of the target, including:

Use the trained deep neural network model to perform target detection on the image;

Determine whether the image contains a high-voltage cable target, and if so, proceed to the next step; if not, return to the step of collecting the high-voltage cable image;

The ROI area of the cable target is masked on the collected image according to the position information output from the detection result.

Further, according to the intrinsic properties and depth information of the high-voltage cable target, extracting the minimum circumscribed rectangle of the target in the ROI area, including:

Perform HSV feature transformation on the ROI area, and count its distribution features;

According to the statistics of HSV distribution characteristics and the inherent properties and depth information of the high-voltage cable target, a threshold is selected for binarization;

The binarized image is filtered by morphological processing;

The minimum circumscribed rectangle of the cable target is obtained by using the moments of the image.

Further, it also includes steps:

After the minimum circumscribed rectangle of the cable target is obtained, it is judged whether the rectangle contains the target, if so, proceed to the next step; if not, return to the step of collecting high-voltage cables.

Further, the cable target is divided into N equal parts in length, and the three-dimensional coordinates of the divided points are obtained, including:

According to the position information of the minimum circumscribed rectangle and the rotation angle, the long side of the rectangle is divided into N equal parts; the rotation angle is obtained in the process of obtaining the minimum circumscribed rectangle of the cable target;

According to the camera depth information and parameter information, the three-dimensional coordinate value of the equally divided point is obtained.

According to a second aspect of the present invention, a method for guiding a robotic arm of a live working robot is provided. The method utilizes three-dimensional coordinates obtained by a high-voltage cable target recognition and positioning method to guide the robotic arm of the live working robot to perform operations. The target identification and positioning method includes the method described in the first aspect of the present invention.

According to a third aspect of the present invention, a high-voltage cable target identification and positioning device is provided, including an image acquisition module, a neural network model optimization module, an image identification module, a coordinate acquisition module, and a positioning module; wherein,

The image acquisition module is used to acquire high-voltage cable images and perform preprocessing;

The neural network model optimization module is used to optimize the YOLO v4 deep neural network model, simplify the number of residual units in its backbone network, and reduce the number of channels;

The image recognition module uses the trained deep neural network model to recognize the high-voltage cable target in the image and extract the ROI area of the target;

The coordinate acquisition module is used to extract the minimum circumscribed rectangle of the target in the ROI area according to the inherent properties and depth information of the high-voltage cable target; and divide the cable target into N equal parts in length according to the minimum circumscribed rectangle, Find the three-dimensional coordinates of the equal points;

The positioning module performs positioning using the three-dimensional coordinates.

Further, the coordinate acquisition module extracts the minimum circumscribed rectangle of the target in the ROI area according to the inherent attributes and depth information of the high-voltage cable target, including:

Perform HSV feature transformation on the ROI area, and count its distribution features;

According to the statistics of HSV distribution characteristics and the inherent properties and depth information of the high-voltage cable target, a threshold is selected for binarization;

The binarized image is filtered by morphological processing;

The minimum circumscribed rectangle of the cable target is obtained by using the moments of the image.

According to a fourth aspect of the present invention, there is provided a storage medium storing a computer program, the computer program implementing the method according to the first aspect of the present invention when executed by a processor.

In summary, the present invention provides a high-voltage cable target identification and positioning method and device, which optimizes the Yolo v4 deep neural network model and uses it to extract the ROI area of the target; then, combined with the HSV color tracking technology, extracts the ROI area The minimum circumscribed rectangle of the inner cable target; finally, combined with the parameter information of the depth camera, the three-dimensional coordinate information of multiple equal points on the cable target is obtained, so as to realize the extraction of the three-dimensional coordinates of the high-voltage cable target, and then for the live working robot The motion of the robotic arm provides more accurate target space information. The positioning and identification method provided by the present invention has faster identification speed in identification of high-voltage cables, improved identification accuracy, and higher reliability, and solves the problem of inconsistencies in identification and positioning in the prior art for target identification under different backgrounds. exact question.

Description of drawings

Fig. 1 is the flow chart of the high-voltage cable target identification and positioning method of the present invention;

Figure 2 is the distribution diagram of cable target HSV characteristics under different light and angle statistics;

FIG. 3 is a block diagram of the structure of the high-voltage cable target identification and positioning device of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the specific embodiments and the accompanying drawings. It should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. Also, in the following description, descriptions of well-known structures and techniques are omitted to avoid unnecessarily obscuring the concepts of the present invention.

The technical solutions of the present invention will be described in detail below with reference to the accompanying drawings. According to an embodiment of the present invention, a method for identifying and locating a target of a high-voltage cable is provided. The flowchart of the method is shown in FIG. 1 and includes the following steps:

S1. Collect high-voltage cable images and perform preprocessing. A depth camera can be used to collect an image of a high-voltage cable target, and the image includes information such as various positions, heights, and angles of the cable. Compared with traditional cameras, the depth camera adds a depth measurement function to the function, which makes it more convenient and accurate to perceive the surrounding environment and changes. Preprocess the image to remove the influence of noise. For example, you can use processing software such as Yolo_mark or labelImg to label the image and save it in txt format.

S2. Optimize the YOLO v4 deep neural network model. In this embodiment, the YOLO v4 deep neural network model is used for image recognition. The YOLO v4 deep neural network model is an algorithm that combines a large number of previous research techniques, combined with appropriate innovation, and achieves a balance between speed and accuracy. For specific application objects, this embodiment appropriately optimizes the YOLOv4 deep neural network model, so that it can recognize high-voltage cables faster, improve the recognition accuracy, and have higher reliability. The optimization process includes: streamlining and optimizing the feature extraction backbone network. The CSPDarknet53 backbone network of YOLO v4 has 5 downsampling layers and 5 CSPNet structures, and the CSPNet structure has 1, 2, 8, 8, and 4 residual units Res respectively. -unit. In this embodiment, the number of residual units in the CSPNet structure is simplified to 1, 2, 4, 4, and 2 residual units, and the number of channels is halved to further reduce the size of the model; the SPP structure network is improved by adding a Shortcut layer , form a structure similar to CSP unit, namely CSP-SPP module; use the network cutting method to cut in the channel dimension; use the k-Means clustering method, combined with the actual target of the high-voltage cable, generate the anchors size of the high-voltage cable, replace Anchors within the configuration file.

S3. Use the trained deep neural network model to identify the high-voltage cable target in the image and extract the ROI area of the target. The following steps can be taken:

Use the trained deep neural network model to perform target detection on the image;

Determine whether the image contains a high-voltage cable target, and if so, proceed to the next step; if not, return to the step of collecting the high-voltage cable image;

The ROI area of the cable target is masked on the collected image according to the position information output from the detection result.

S4. According to the intrinsic properties and depth information of the high-voltage cable target, extract the minimum circumscribed rectangle of the cable target. A detailed description will be given below.

HSV feature transformation is performed on the ROI region, and its distribution features are counted. In this embodiment, according to the imaging characteristics of cables with different lighting and different angles, the H, S, and V three-component values of more than 1300 cable pixel points are counted, and their distribution characteristics are counted. The target HSV feature distribution is shown in Figure 2, where the abscissa represents the magnitude of H, S, and V, and the ordinate represents its intensity. According to the statistics of the HSV distribution characteristics and the inherent properties of the high-voltage cable target, the threshold is selected for binarization processing, that is, in the HSV color space, according to the inherent properties and depth information of the high-voltage cable color information in different environments, select A suitable threshold is used for binarization. Specifically, the depth map corresponding to the ROI area can be binarized according to the preset range of the cable target distance and depth camera, and then the two binarized maps are compared with each other. operation to obtain a more accurate binarized image. The binarized image is filtered by morphological processing; the minimum circumscribed rectangle of the cable target is obtained by using the moment of the image. Since the cable image obtained after binarization is a black and white image, under normal circumstances, the white part in the figure represents the cable, and the black part represents the background; however, under abnormal conditions, some pseudo-targets with colors that are closer to the cable color will also be extracted. , performing the above processing on the image can obtain the target image with higher reliability. In this step, for example, the basic functions in the commonly used image processing software OpenCV can be used to obtain the minimum circumscribed rectangle, and parameters such as the aspect ratio, area size, and rotation angle of the minimum circumscribed rectangle can be obtained in this process.

After the minimum circumscribed rectangle of the cable target is obtained, it is judged whether the rectangle contains the target, if so, go to step S5; if not, return to step S1 to collect the high-voltage cable image. The aspect ratio, area size and rotation angle of the minimum circumscribed rectangle obtained in the previous step can be used to determine whether the obtained minimum circumscribed rectangle contains a high-voltage cable target, that is, the target recognition result is performed again. verify.

S5. Divide the cable target into N equal parts in length according to the minimum circumscribed rectangle, and obtain the three-dimensional coordinates of the divided points. According to the position information of the minimum circumscribed rectangle and the parameter rotation angle obtained in the previous step, divide N equally on the long side of the rectangle; obtain the three-dimensional coordinate value of the divided point according to the camera depth information and parameter information. The parameter information includes internal parameter information of the camera and information such as the focal length of the lens. The three-dimensional coordinate value can be obtained by using the commonly used projection formula.

S6. Use the three-dimensional coordinates to perform positioning.

According to a second embodiment of the present invention, a method for guiding a robotic arm of a live working robot is provided, which uses the three-dimensional coordinates obtained by a high-voltage cable target identification and positioning method to guide the robotic arm of the live working robot to perform operations. The high-voltage cable target identification and positioning method may adopt the positioning method described in the first embodiment of the present invention. In practical applications, after the host computer uses this positioning method to obtain the three-dimensional coordinate information of the equidistant point, it uses a communication protocol such as socket to upload the three-dimensional coordinate information to the control center, and the control center will carry out the live work robot according to the three-dimensional coordinate information. The robot arm sends control commands to control it to the specified position.

According to a third embodiment of the present invention, a high-voltage cable target identification and positioning device is provided. The block diagram of the device is shown in FIG. 3 , including an image acquisition module, a neural network model optimization module, an image recognition module, and a coordinate acquisition module. module, and positioning module.

The image acquisition module is used to collect high-voltage cable images and perform preprocessing;

The neural network model optimization module is used to optimize the YOLO v4 deep neural network model;

The image recognition module uses the trained deep neural network model to identify and extract the ROI area of the high-voltage cable target image;

The coordinate acquisition module is used to extract the minimum circumscribed rectangle of the cable target according to the inherent properties and depth information of the high-voltage cable target; and divide the length of the cable target into N equal parts according to the minimum circumscribed rectangle to obtain equal divisions the three-dimensional coordinates of the point;

The positioning module uses the three-dimensional coordinates for positioning.

The specific process of each module in the device implementing its function is the same as each step of the fault location method in the first embodiment provided by the present invention, which is not repeated here.

According to a fourth embodiment of the present invention, there is provided a storage medium storing a computer program which, when executed by a processor, implements the method described in the first or second embodiment of the present invention method described.

In summary, the present invention relates to a high-voltage cable target identification and positioning method and device, which optimizes the Yolo v4 deep neural network model and uses it to extract the ROI area of the target; and then, combined with the HSV color tracking technology, extracts the ROI area. The minimum circumscribed rectangle of the cable target; finally, combined with the parameter information of the depth camera, the three-dimensional coordinate information of multiple equal points on the cable target is obtained, so as to realize the extraction of the three-dimensional coordinates of the high-voltage cable target, and then for the live working robot. The motion of the robotic arm provides more accurate target space information. The positioning and identification method provided by the present invention has faster identification speed in identification of high-voltage cables, improved identification accuracy, and higher reliability, and solves the problem of inconsistencies in identification and positioning in the prior art for target identification under different backgrounds. exact question.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

It should be understood that the above-mentioned specific embodiments of the present invention are only used to illustrate or explain the principle of the present invention, but not to limit the present invention. Therefore, any modifications, equivalent replacements, improvements, etc. made without departing from the spirit and scope of the present invention should be included within the protection scope of the present invention. Furthermore, the appended claims of this invention are intended to cover all changes and modifications that fall within the scope and boundaries of the appended claims, or the equivalents of such scope and boundaries.

Claims (10)

1. A high-voltage cable target identification and positioning method is characterized by comprising the following steps:

acquiring a high-voltage cable image and preprocessing the high-voltage cable image;

optimizing a YOLO v4 deep neural network model, simplifying the number of residual error units in a backbone network, and reducing the number of channels;

recognizing a high-voltage cable target in the image by adopting the trained deep neural network model and extracting an ROI (region of interest) of the target;

extracting a minimum circumscribed rectangle of the target in the ROI area according to the inherent attribute and the depth information of the high-voltage cable target;

according to the minimum external rectangle, carrying out N equal division on the length of the cable target to obtain three-dimensional coordinates of equal division points;

and positioning by using the three-dimensional coordinates.

2. The method of claim 1, wherein the acquiring and preprocessing high voltage cable target images comprises:

acquiring a high-voltage cable image by using a depth camera;

and labeling the acquired image.

3. The method of claim 2, wherein the identifying the high voltage cable target in the image and extracting the ROI area of the target by using the trained deep neural network model comprises:

carrying out target detection on the image by adopting the trained deep neural network model;

judging whether the image contains a high-voltage cable target or not, and if so, carrying out the next step; if not, returning to the step of acquiring the high-voltage cable image;

and performing mask processing on the ROI area of the cable target on the acquired image according to the position information output by the detection result.

4. The method of claim 3, wherein the extracting the minimum bounding rectangle of the high voltage cable target in the ROI area according to the intrinsic properties and the depth information of the target comprises:

performing HSV feature transformation on the ROI and counting distribution features of the ROI;

selecting a threshold value to carry out binarization processing according to the counted HSV distribution characteristics and the inherent attribute and depth information of the high-voltage cable target;

filtering the image after the binarization processing by using morphological processing;

and solving the minimum circumscribed rectangle of the cable target by using the moment of the image.

5. The method of claim 4, further comprising the step of:

after the minimum external rectangle of the cable target is obtained, judging whether the rectangle contains the target or not, and if so, carrying out the next step; and if not, returning to the step of acquiring the high-voltage cable image.

6. The method of claim 5, wherein the equally dividing the cable target by N in length to obtain three-dimensional coordinates of an equally divided point comprises:

dividing N equally on the long side of the rectangle according to the position information and the rotation angle of the minimum circumscribed rectangle; the rotation angle is obtained in the process of obtaining the minimum circumscribed rectangle of the cable target;

and obtaining the three-dimensional coordinate value of the bisector according to the camera depth information and the parameter information.

7. A method for guiding a robot arm of a live working robot by using three-dimensional coordinates obtained by a high-voltage cable target identification and positioning method, wherein the high-voltage cable target identification and positioning method comprises the method according to any one of claims 1 to 6.

8. A high-voltage cable target identification and positioning device is characterized by comprising an image acquisition module, a neural network model optimization module, an image identification module, a coordinate acquisition module and a positioning module; wherein,

the image acquisition module is used for acquiring a high-voltage cable image and carrying out preprocessing;

the neural network model optimization module is used for optimizing a YOLO v4 deep neural network model, simplifying the number of residual error units in a backbone network and reducing the number of channels;

the image recognition module recognizes a high-voltage cable target in the image by adopting the trained deep neural network model and extracts an ROI (region of interest) of the target;

the coordinate acquisition module is used for extracting the minimum external rectangle of the target in the ROI area according to the inherent attribute and the depth information of the high-voltage cable target; according to the minimum external rectangle, the length of the cable target is divided into N equal parts, and the three-dimensional coordinates of equal division points are obtained;

and the positioning module is used for positioning by utilizing the three-dimensional coordinates.

9. The apparatus of claim 8, wherein the coordinate obtaining module extracts a minimum bounding rectangle of the high voltage cable target in the ROI area according to the intrinsic property and the depth information of the target, comprising:

performing HSV feature transformation on the ROI and counting distribution features of the ROI;

selecting a threshold value to carry out binarization processing according to the counted HSV distribution characteristics and the inherent attribute and depth information of the high-voltage cable target;

filtering the image after the binarization processing by using morphological processing;

and solving the minimum circumscribed rectangle of the cable target by using the moment of the image.

10. A storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the method according to any one of claims 1-6.

Images