WO2023040176A1 - Power supply port positioning method and system for insulation test of electrical product - Google Patents

Power supply port positioning method and system for insulation test of electrical product Download PDF

Info

Publication number
WO2023040176A1
WO2023040176A1 PCT/CN2022/076358 CN2022076358W WO2023040176A1 WO 2023040176 A1 WO2023040176 A1 WO 2023040176A1 CN 2022076358 W CN2022076358 W CN 2022076358W WO 2023040176 A1 WO2023040176 A1 WO 2023040176A1
Authority
WO
WIPO (PCT)
Prior art keywords
power port
camera
close
power
center point
Prior art date
Application number
PCT/CN2022/076358
Other languages
French (fr)
Chinese (zh)
Inventor
王惯玉
Original Assignee
苏州超集信息科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州超集信息科技有限公司 filed Critical 苏州超集信息科技有限公司
Publication of WO2023040176A1 publication Critical patent/WO2023040176A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection

Definitions

  • the invention relates to the technical field of image positioning, in particular to a method and system for positioning a power port of an electrical product insulation test.
  • the high-voltage and high-current insulation test of electrical products widely exists in the production of electrical products. Most of the methods are to fix the position of the test device and the device under test and then use automated means. However, there are few solutions for testing electrical products in which the position of the device under test is not fixed.
  • the patent No. is CN112834861A, and the patent titled "a dual-output withstand voltage test assembly line" adopts this scheme of fixing the test device and the device under test.
  • the position of the power port of the electrical product is not fixed, and the power port may randomly appear in various positions of the electrical device. In addition, the direction of the power port of the device under test cannot be determined, making it difficult to meet production test requirements.
  • the purpose of the present invention is to provide a method and system for locating the power port of an electrical product insulation test, which solves the problems of random occurrence of the power port position of the electrical product and uncertain direction of the power port, and accurately locates the power port to meet the test requirements.
  • the present invention provides a method for locating the power port of an electrical product insulation test, which includes the following steps:
  • S2 Calibrate the distant view camera to obtain distortion parameters, and de-distort the overall image of the electrical appliance through the distortion parameters;
  • S3 Perform perspective transformation on the overall image of the electrical appliance after de-distortion to extract the image of the power port area of interest;
  • S4 Use the deep learning network model to identify and locate the position of the power port from the pictures of the power port area of interest;
  • S5 Drive the close-range camera to reach the position according to the position of the power port obtained by the distant view camera;
  • the close-range camera obtains the picture of the power port area, and uses the deep learning network model to identify and locate the position of the power port and the corresponding plug position from the picture of the power port area;
  • S7 Calculate the distance between the center point and the center point of the power port area picture according to the center point of the identified and positioned power port in S6, and take the power port corresponding to the minimum distance as the target power port;
  • the close-up camera is arranged in parallel with the power plugging device, and the center point of the close-up camera and the center point of the power plugging device are arranged on the same horizontal line.
  • step S2 specifically includes the following steps:
  • S24 Perform de-distortion processing on the overall image of the electrical appliance by using the obtained distortion parameters, and obtain the overall image of the electrical appliance after de-distortion processing.
  • step S3 specifically includes the following steps:
  • both step S4 and step S6 use a lightweight feature extraction network and feature map fusion operation to identify and locate the power outlet in the picture of the power outlet area of interest.
  • step S7 after the position of the power port is detected, the center point coordinates of each detected power port are extracted and the Euclidean distance between the center point coordinates and the picture center point coordinates is calculated, According to the size of the Euclidean distance, take the power port corresponding to the smallest Euclidean distance as the target power port for high-voltage testing.
  • the close-range camera is used repeatedly to identify and position the target power port and the close-range camera is driven to move according to the recognition and positioning results.
  • the close-range camera will not move when it reaches the designated position and will record the position, and drive the power plug-in device according to the recorded position.
  • the plug identification result is extracted, and in step S8, the Euclidean distance of the middle point of the plug is calculated, and the middle point of any one of the two plugs with the largest Euclidean distance is taken.
  • Point is the origin, judge the opening direction of the power port, and control the direction of the mobile power plugging device to correspond to the target power port according to the opening direction.
  • the long-range camera and the close-range camera are equipped with an algorithm end, and the algorithm end communicates with the software end to interact by sending messages in JSON format through MQTT, and the interaction method specifically includes the following steps:
  • the vision camera When the vision camera receives the signal that the appliance is in place, it turns on the camera and captures the overall image of the appliance;
  • the algorithm side of the long-range camera uses the deep learning network model to identify the power port to obtain the center coordinates of the power port. After conversion, it is sent to the software side through MQTT, and the software side drives the close-range camera according to the coordinates;
  • the close-up camera captures pictures and uses its algorithm-side deep learning model to identify and locate the power port and plug. After the recognition and positioning results are converted, they are sent to the software side through MQTT.
  • the terminal controls the movement of the close-range camera to reach the specified position according to the conversion result;
  • the close-up camera arrives at the designated position, turn on the close-up camera again and use the deep learning model on the algorithm side to identify the power port, and convert the result of the recognition and positioning to the software side through MQTT, and the software side controls the mobile close-up according to the result Camera, repeat this process until the center point of the picture of the close-range camera coincides with the center point of the target power port, and the software side drives the power plug-in device to the center of the close-range camera for electrical insulation testing.
  • a power port positioning system for electrical product insulation testing including: a long-range camera, a close-up camera and a software terminal;
  • the perspective camera is used to obtain the overall image of the electrical appliance, and obtains distortion parameters through calibration to de-distort the overall image of the electrical appliance, and is also used to perform perspective transformation on the de-distorted overall image of the electrical appliance to extract a picture of the power port area of interest.
  • the deep learning network model identifies and locates the position of the power port from the pictures of the power port area of interest and transmits it to the software side;
  • the close-range camera is used to obtain a picture of the power port area, and uses a deep learning network model to identify and locate the position of the power port and the corresponding plug position of the power port from the picture of the power port area; it is also used to locate the center point of the power port according to the identification Calculate the distance between the center point and the center point of the picture, take the power port corresponding to the minimum distance as the target power port, and calculate the distance between the center points of each plug according to the target power port and its corresponding plug identification results to determine the direction of the power port;
  • the software end is used to drive the center point of the image of the close-range camera to the position coincident with the center point of the power port according to the position of the power port obtained by the long-range camera; it is used to drive the close-range camera to a designated position according to the identification and positioning results of the close-range camera; it is also used to According to the direction of the power port and the target power port, drive the power plug-in device to the position of the close-up camera for product insulation testing.
  • the present invention uses a combination of a close-range camera and a long-range camera for deep learning recognition and positioning, improves recognition accuracy, and solves the problems of random occurrence of power port positions of electrical products, uncertain direction of power port, and difficulty in extracting relevant features;
  • the invention performs calibration and related image processing on the perspective camera, avoids the deviation of the positioning result of the perspective camera, and further improves the positioning accuracy; through the positioning of the power port of the present invention, the high-voltage test of electrical products can be carried out smoothly, the effect is good, and the detection efficiency is improved.
  • Fig. 1 is a schematic flow sheet of the method of the present invention
  • Fig. 2 is a schematic diagram of the style of the black and white calibration plate of the present invention.
  • Fig. 3 is a schematic diagram of the deep learning network structure of the vision camera of the present invention.
  • Fig. 4 is a schematic diagram of the deep learning network structure of the close-range camera of the present invention.
  • Fig. 5 is a schematic diagram of the algorithm interaction process between the algorithm end and the software end of the present invention.
  • an embodiment of the present invention provides a method for locating the power port of an electrical product insulation test, including the following steps:
  • S2 Calibrate the perspective camera to obtain distortion parameters, and de-distort the overall image of the appliance through the distortion parameters;
  • S3 Perform perspective transformation on the overall image of the electrical appliance after de-distortion to extract the image of the power port area of interest;
  • S4 Use the deep learning network model to identify and locate the position of the power port from the pictures of the power port area of interest;
  • S5 Drive the close-range camera to reach the position according to the position of the power port obtained by the distant view camera;
  • the close-range camera obtains the picture of the power port area, and uses the deep learning network model to identify and locate the position of the power port and the corresponding pin position from the picture of the power port area;
  • S7 Calculate the distance between the center point and the center point of the power port area picture according to the center point of the identified and positioned power port in S6, and take the power port corresponding to the minimum distance as the target power port;
  • the close-up camera needs a certain field of view to capture the information of the power port of interest and the camera should be parallel to the power plug-in device. Combined with the actual situation, select a certain type of camera and install it near the test set.
  • the horizontal and vertical distances between the center point of the close-range camera and the center point of the power plug-in device at this position are about 50mm and 11mm, respectively.
  • the calibration board is in the form of a black and white square grid, and the calibration board is divided into 9*7 black and white squares, and the length of each grid is 0.026m.
  • the specific form is shown in Figure 2 below;
  • the imaging process of the camera can be simplified to a pinhole imaging model. It involves four coordinate systems, namely the conversion relationship between coordinates of the world coordinate system, camera coordinate system, image coordinate system and pixel coordinate system. There is a rotation between the coordinate systems and a translation transformation between the origins between the world coordinate system and the camera coordinate system. The relationship between the camera coordinate system and the points in the image coordinate system can be obtained according to the principle of similar triangles.
  • the actual distance between the corner points on the checkerboard in the world coordinate system can be easily obtained.
  • multiple sets of pixel points and corresponding points in the world coordinate system can be obtained, and then H can be calculated.
  • the internal reference matrix K can be obtained according to the property that the rotation matrix is an orthogonal matrix, that is, its column vectors are unit vectors and are mutually orthogonal.
  • the lens In the pinhole imaging model, the lens is ideally equivalent to a pinhole, but in practice, in order to make the generated image brighter, the lens needs to have a certain diameter to achieve light collection. Due to the shape of the lens, the magnification of the image in the radial direction is inconsistent, causing the image to be distorted. This phenomenon is called radial distortion of the camera. Radial distortion is caused by the shape of the lens itself, and the effect of distortion is center-symmetric. In addition, when installing the camera artificially, it is impossible to completely avoid errors caused by human factors such as operation, and the relative positions of the lens plane and the imaging plane must not be strictly parallel, which will inevitably lead to Distortion occurs in the direction. Although the camera is also affected by other types of distortion, radial and tangential distortions account for the vast majority of all distortion types, and radial distortion is the main component.
  • the parameters obtained from the calibration can be used to reproject the points in the three-dimensional space to obtain the undistorted points, and then the polynomial is used to fit the mathematical model between the undistorted points and the distorted points.
  • Radial distortion is described by formula (3-2), which is a function of the distance r between the pixel point in the image and the origin of the image coordinate system.
  • Tangential distortion is described by formula (3-3). (u, v) in formulas (3-2) and (3-3) represent ideal coordinates without distortion, and (u′, v′) represent corresponding distorted coordinates.
  • the internal parameters are first initialized to a closed solution that does not consider camera distortion, and the ideal pixel coordinates are obtained by reprojecting the points in the world coordinate system through the initial camera parameters, and then based on the principle of minimizing the reprojection error.
  • Non-linear optimization The coefficients of the distortion model are fitted by the reprojected ideal image coordinates and the corner coordinates on the distorted image. The more complex the distortion model is, the better. Complex distortion models have high-order fitting relationships, which sometimes backfire and cause large calculation errors. If radial and tangential distortions are considered at the same time, it can be found that radial distortion accounts for the vast majority of the distortion model, while the influence of tangential distortion can be ignored.
  • the image is de-distorted to obtain pictures before and after de-distortion processing.
  • Perspective transformation is a method of projecting a picture to a new viewing plane. This method requires four known points on the picture. By calculating these four points to get the region of interest. Assuming that the pixel value of the picture before perspective transformation is (U, V), and the pixel value of the picture after perspective transformation is (x′, y′), according to the principle of perspective transformation:
  • the method of identifying and locating the power outlet in the area of interest based on deep learning After obtaining the picture using perspective transformation, identify and locate the power outlet in the picture, and the identification and positioning method uses the deep learning method.
  • a simplified small network model is used to identify and locate power ports.
  • the structure of the network is shown in Figure 3, using a relatively lightweight feature extraction network and using the operation of feature map fusion. Considering that the task completed by this network is relatively simple, only a small number of detection heads are used.
  • this application sets the ratio of training set to test set to 1:1 to increase the difficulty of model learning.
  • this application also uses a data enhancement solution, including image saturation adjustment, exposure adjustment, and hue adjustment. In order to get the result closer to the actual situation.
  • the setting of model training parameters is crucial to obtain better results.
  • the relevant parameters of the model training in this application are: the image size of the input model is set to 608*608, the optimization scheme of the model parameters uses SGD with momentum and uses the preheating operation and the pre-training model to speed up the model training; the iou loss of the model uses ciou Loss, nms of detection boxes use simple nms. The loss gradually decreases with the increase of the number of training rounds, and the map value on the test set gradually increases with the increase of the number of training rounds, and finally stabilizes. Use the detection performance of the model on the training set and test set.
  • the detected central point coordinates of the power port are based on the image coordinate system.
  • the origin of the image coordinate system needs to be placed at the lower left corner of the image after the perspective change.
  • the conversion principle between the camera coordinate system and the pixel coordinate system According to the conversion principle between the camera coordinate system and the pixel coordinate system, combined with the installation method of the close-up camera described above, this application uses this principle to make the center point of the image and the power port The camera is considered to have arrived at the specified position when the center points of are coincident. At this time, adjust the positions of the camera and the plugging device so that the plugging device reaches the position of the camera.
  • Power port identification and positioning method based on deep learning method When the left camera reaches the coordinates converted by the vision camera detection, identify and locate the power port and plug in the picture, and the identification and positioning method uses depth study method. Considering the limited computing resources of edge computing devices, a small network model is used to identify and locate power ports and plugs. The structure of the network is shown in Figure 4, which only uses a relatively lightweight feature extraction network and uses the operation of feature map fusion.
  • this application sets the ratio of training set to test set to 1:1 to increase the difficulty of model learning.
  • this application also uses a data enhancement solution, including image saturation adjustment, exposure adjustment, and hue adjustment. In order to get the result closer to the actual situation.
  • the setting of model training parameters is crucial to obtain better results.
  • the relevant parameters of the model training in this application are: the image size of the input model is set to 416*416, the optimization scheme of the model parameters uses SGD with momentum and uses the preheating operation and the pre-training model to speed up the model training; the iou loss of the model uses ciou Loss, nms of detection boxes use simple nms. The loss gradually decreases with the increase of the number of training rounds, and the map value on the test set gradually increases with the increase of the number of training rounds, and finally stabilizes.
  • the model has a good detection effect of the power port and has a good recognition effect on the pin of the power port closest to the center point of the image. In the process of electrical high-voltage testing, only one of the many power ports needs to be tested. Therefore, this model is good enough for production needs.
  • the center point coordinates of each detected power port are extracted to calculate the Euclidean distance between the center point coordinates and the picture center point coordinates.
  • the smallest Euclidean distance corresponding The power port is the target power port of this application to perform a high voltage test.
  • the image size captured by the close-range camera is 640:480.
  • the distance that the close-range camera needs to move is based on the arrival position of the long-range camera.
  • the calculation method is to multiply the ratio of the width and height of the detection frame to the width and height of the picture by the actual width and height of the power frame.
  • the actual width and height of the power box are 23mm and 30mm respectively.
  • the direction of the power port will change, sometimes to the left, sometimes to the right. At this time, it is necessary to judge the direction of the power port according to the identification result of the pin.
  • extract the plug recognition result calculate the Euclidean distance between the midpoints of each plug, take the midpoint of any of the two plugs with the largest Euclidean distance as the origin, and the right-hand side as the horizontal
  • the direction is the positive direction
  • the left-hand side is the negative direction of the horizontal direction.
  • the software side can control the movement of the moving axis to change the direction of the plug-in device through information interaction.
  • the communication method between the algorithm end and the software end the above-mentioned long-range camera and close-range camera are equipped with an algorithm end, and the algorithm end needs to interact with the software end, so as to realize various calculation uploads and control movements.
  • the algorithm end communicates with the software end Interact by sending messages in JSON format via MQTT.
  • the vision camera receives the signal that the electrical appliance is in place, it turns on the camera and captures a picture.
  • the model recognizes the power port to get the center coordinates of the power port. After conversion, it is sent to the software end through MQTT, and the software end acts according to the coordinates.
  • the actual effect test of the algorithm After completing the design of the above algorithm and the joint debugging with the software side, use the actual electrical product for testing. The test found that the algorithm has a good effect, except that the position of the electrical product changes due to the collision of the plug-in device and the power port of the electrical product cannot be located. The plug-in device can smoothly carry out the high-voltage test of the electrical product.
  • this embodiment provides a system for locating a power port for an electrical product insulation test.
  • the principle for solving the problem is similar to a method for locating a power port for an electrical product insulation test.
  • a power port positioning system for insulation testing of electrical products including: a long-range camera, a close-up camera and a software terminal;
  • the perspective camera is used to obtain the overall image of the electrical appliance, and obtains distortion parameters through calibration to de-distort the overall image of the electrical appliance, and is also used to perform perspective transformation on the de-distorted overall image of the electrical appliance to extract a picture of the power port area of interest.
  • the deep learning network model identifies and locates the position of the power port from the pictures of the power port area of interest and transmits it to the software side;
  • the close-range camera is used to obtain a picture of the power port area, and uses a deep learning network model to identify and locate the position of the power port and the corresponding plug position of the power port from the picture of the power port area; it is also used to locate the center point of the power port according to the identification Calculate the distance between the center point and the center point of the picture, take the power port corresponding to the minimum distance as the target power port, and calculate the distance between the center points of each plug according to the target power port and its corresponding plug identification results to determine the direction of the power port;
  • the software end is used to drive the center point of the image of the close-range camera to the position coincident with the center point of the power port according to the position of the power port obtained by the long-range camera; it is used to drive the close-range camera to a designated position according to the identification and positioning results of the close-range camera; it is also used to According to the direction of the power port and the target power port, drive the power plug-in device to the position of the close-up camera for product insulation testing.
  • the invention uses the combination of far and near cameras to locate and interactively control the positioning of the power port, solving the problems of random occurrence of the power port position of electrical products, uncertain direction of the power port, and difficulty in extracting relevant features of the identification and positioning technology based on manually selected feature extraction and difficult to achieve production Ask real-world questions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Geometry (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Analysis (AREA)

Abstract

Disclosed in the present invention are a power supply port positioning method and system for an insulation test of an electrical product, the method comprising the following steps: mounting a long-range camera and a close-range camera, wherein the long-range camera acquires an overall image of an electrical appliance; de-distorting the overall image of the electrical appliance by means of a distortion parameter; performing a perspective transformation to extract a picture of a power supply port area of interest; using a deep learning network model to identify and position the location of a power supply port; driving the close-range camera to reach the location according to the location of the power supply port obtained by the long-range camera; the close-range camera using the deep learning network model to identify and position the location of the power supply port and the location of a plug corresponding to the power supply port; taking the power supply port corresponding to the minimum distance as a target power supply port; determining the direction of the power supply port; and driving a power supply plugging apparatus to reach the location of the close-range camera at this time to perform an insulation test on a product. The present invention solves the problems of the random appearance of the location of a power supply port and the uncertainty of the direction of the power supply port of an electrical product, thereby achieving accurate positioning of the power supply port, and meeting test requirements.

Description

一种电器产品绝缘测试电源口定位方法及系统A method and system for locating a power port of an electrical product insulation test 技术领域technical field
本发明涉及图像定位技术领域，具体涉及一种电器产品绝缘测试电源口定位方法及系统。The invention relates to the technical field of image positioning, in particular to a method and system for positioning a power port of an electrical product insulation test.
背景技术Background technique
电器产品的高压高流绝缘测试广泛存在于电器产品的生产中，绝大多数的方法是固定测试装置与被测试装置的位置后再通过自动化的手段进行。而对于被测试装置位置不固定的电器产品进行测试的方案鲜有可见。专利号为CN112834861A，名为“一种双输出耐压测试流水线”的专利则采用这种固定测试装置与被测试装置的方案。而在实际生产过程中，电器产品的电源口的位置并不是固定的，电源口可能随机的出现在电器的各个位置。除此之外，被测试装置的电源口的方向也无法确定，难以达到生产测试需求。The high-voltage and high-current insulation test of electrical products widely exists in the production of electrical products. Most of the methods are to fix the position of the test device and the device under test and then use automated means. However, there are few solutions for testing electrical products in which the position of the device under test is not fixed. The patent No. is CN112834861A, and the patent titled "a dual-output withstand voltage test assembly line" adopts this scheme of fixing the test device and the device under test. However, in the actual production process, the position of the power port of the electrical product is not fixed, and the power port may randomly appear in various positions of the electrical device. In addition, the direction of the power port of the device under test cannot be determined, making it difficult to meet production test requirements.
发明内容Contents of the invention
本发明的目的是提供一种电器产品绝缘测试电源口定位方法及系统，解决电器产品电源口位置随机出现、电源口方向不确定问题，对电源口进行精确定位，满足测试需求。The purpose of the present invention is to provide a method and system for locating the power port of an electrical product insulation test, which solves the problems of random occurrence of the power port position of the electrical product and uncertain direction of the power port, and accurately locates the power port to meet the test requirements.
为了解决上述技术问题，本发明提供了一种电器产品绝缘测试电源口定位方法，包括以下步骤：In order to solve the above technical problems, the present invention provides a method for locating the power port of an electrical product insulation test, which includes the following steps:
S1：安装远景相机和近景相机，所述远景相机获取电器整体图像；S1: installing a long-range camera and a close-range camera, the long-range camera obtains an overall image of the electrical appliance;
S2：对远景相机进行标定获取畸变参数，通过畸变参数对电器整体图像进 行去畸变；S2: Calibrate the distant view camera to obtain distortion parameters, and de-distort the overall image of the electrical appliance through the distortion parameters;
S3：对去畸变后的电器整体图像进行透视变换提取出感兴趣电源口区域图片；S3: Perform perspective transformation on the overall image of the electrical appliance after de-distortion to extract the image of the power port area of interest;
S4：使用深度学习网络模型从感兴趣电源口区域图片中识别定位电源口位置；S4: Use the deep learning network model to identify and locate the position of the power port from the pictures of the power port area of interest;
S5：根据远景相机获得的电源口位置驱动近景相机到达该位置；S5: Drive the close-range camera to reach the position according to the position of the power port obtained by the distant view camera;
S6：近景相机获取电源口区域图片，使用深度学习网络模型从电源口区域图片中识别定位电源口位置以及电源口对应的插销位置；S6: The close-range camera obtains the picture of the power port area, and uses the deep learning network model to identify and locate the position of the power port and the corresponding plug position from the picture of the power port area;
S7：根据S6中识别定位到的电源口的中心点计算该中心点与电源口区域图片中心点距离，取最小距离对应的电源口为目标电源口；S7: Calculate the distance between the center point and the center point of the power port area picture according to the center point of the identified and positioned power port in S6, and take the power port corresponding to the minimum distance as the target power port;
S8：根据目标电源口及其对应的插销识别结果计算各插销中心点距离以判断电源口方向；S8: Calculate the center point distance of each plug according to the identification result of the target power port and the corresponding plug to determine the direction of the power port;
S9：根据电源口方向和目标电源口，驱动电源插拔装置达到此时近景相机位置进行产品的绝缘测试。S9: According to the direction of the power port and the target power port, drive the power plug-in device to the position of the close-up camera at this time to conduct the insulation test of the product.
作为本发明的进一步改进，所述近景相机与电源插拔装置平行设置且近景相机的中心点与电源插拔装置的中心点设置在同一水平线上。As a further improvement of the present invention, the close-up camera is arranged in parallel with the power plugging device, and the center point of the close-up camera and the center point of the power plugging device are arranged on the same horizontal line.
作为本发明的进一步改进，所述步骤S2具体包括以下步骤：As a further improvement of the present invention, the step S2 specifically includes the following steps:
S21:制作黑白方格板作为标定板，并固定远景相机；S21: Make a black and white grid plate as a calibration plate, and fix the distant view camera;
S22：将标定板放置于远景相机前各个位置并拍摄含有标定板的图像；S22: placing the calibration plate at various positions in front of the vision camera and taking images containing the calibration plate;
S23：采用张正有标定法对远景相机进行标定得到标定参数，通过标定得到的标定参数对三维空间的点进行重投影得到未畸变的点，对未畸变的点和畸变 点之间的数学模型进行拟合得到畸变参数；S23: Use Zhang Zhengyou calibration method to calibrate the distant view camera to obtain the calibration parameters, reproject the points in the three-dimensional space to obtain the undistorted points through the calibration parameters obtained through the calibration, and calculate the mathematical model between the undistorted points and the distorted points Fitting is performed to obtain the distortion parameters;
S24：使用得到的畸变参数对电器整体图像进行去畸变处理，得到去畸变处理后的电器整体图像。S24: Perform de-distortion processing on the overall image of the electrical appliance by using the obtained distortion parameters, and obtain the overall image of the electrical appliance after de-distortion processing.
作为本发明的进一步改进，所述步骤S3具体包括以下步骤：As a further improvement of the present invention, the step S3 specifically includes the following steps:
S31：设定进行透视变换前的图片像素值为(U，V)，进行透视变换后的图像素值为(x′，y′)S31: Set the image pixel values before perspective transformation to (U, V), and the image pixel values after perspective transformation to (x′, y′)
S32：根据透视变换原理则：S32: According to the principle of perspective transformation:
Figure PCTCN2022076358-appb-000001
Figure PCTCN2022076358-appb-000001
Figure PCTCN2022076358-appb-000002
Figure PCTCN2022076358-appb-000002
Figure PCTCN2022076358-appb-000003
Figure PCTCN2022076358-appb-000003
其中，
Figure PCTCN2022076358-appb-000004
为目标矩阵； in,
Figure PCTCN2022076358-appb-000004
is the target matrix;
S33：选取电器产品面板所在的四个角点为透视变换输入值，进行步骤S31-S32变换，得到透视变换后的感兴趣电源口区域图片。S33: Select the four corner points where the panel of the electrical product is located as the input value of the perspective transformation, perform the transformation in steps S31-S32, and obtain the picture of the power port area of interest after the perspective transformation.
作为本发明的进一步改进，所述步骤S4和步骤S6中均使用轻量级的特征提取网络和特征图融合操作，对感兴趣电源口区域图片中的电源口进行识别与定位。As a further improvement of the present invention, both step S4 and step S6 use a lightweight feature extraction network and feature map fusion operation to identify and locate the power outlet in the picture of the power outlet area of interest.
作为本发明的进一步改进，所述步骤S7中，当检测得到电源口的位置后， 提取出每个检测到的电源口的中心点坐标并计算该中心点坐标与图片中心点坐标的欧式距离，依据欧式距离的大小，取最小的欧式距离对应的电源口为目标电源口进行高压测试。As a further improvement of the present invention, in the step S7, after the position of the power port is detected, the center point coordinates of each detected power port are extracted and the Euclidean distance between the center point coordinates and the picture center point coordinates is calculated, According to the size of the Euclidean distance, take the power port corresponding to the smallest Euclidean distance as the target power port for high-voltage testing.
作为本发明的进一步改进，根据S6中的电源口识别定位结果与S7中的目标电源口结果，反复利用近景相机识别定位目标电源口并根据识别定位结果驱动近景相机移动，当目标电源口的中心点与近景相机的中心点重合时，近景相机到达指定位置不再移动并记录好该位置，根据记录位置驱动电源插拔装置。As a further improvement of the present invention, according to the power port identification and positioning results in S6 and the target power port results in S7, the close-range camera is used repeatedly to identify and position the target power port and the close-range camera is driven to move according to the recognition and positioning results. When the center of the target power port When the point coincides with the center point of the close-range camera, the close-range camera will not move when it reaches the designated position and will record the position, and drive the power plug-in device according to the recorded position.
作为本发明的进一步改进，所述步骤S7中确定目标电源口后，提取插销识别结果，在步骤S8中，计算插销中点的欧式距离，取欧式距离最大的两个插销中任一插销的中点为原点，判断电源口的开口方向，并根据开口方向控制移动电源插拔装置的方向对应目标电源口。As a further improvement of the present invention, after the target power outlet is determined in the step S7, the plug identification result is extracted, and in step S8, the Euclidean distance of the middle point of the plug is calculated, and the middle point of any one of the two plugs with the largest Euclidean distance is taken. Point is the origin, judge the opening direction of the power port, and control the direction of the mobile power plugging device to correspond to the target power port according to the opening direction.
作为本发明的进一步改进，所述远景相机和近景相机均安装有算法端，所述算法端与软件端通信通过MQTT发送JSON格式的消息来进行交互，所述交互方法具体包括以下步骤：As a further improvement of the present invention, the long-range camera and the close-range camera are equipped with an algorithm end, and the algorithm end communicates with the software end to interact by sending messages in JSON format through MQTT, and the interaction method specifically includes the following steps:
当远景相机收到电器就位的信号后，打开相机而抓拍电器整体图像；When the vision camera receives the signal that the appliance is in place, it turns on the camera and captures the overall image of the appliance;
远景相机的算法端使用深度学习网络模型识别电源口得到电源口的中心坐标，转换过后通过MQTT发送到软件端，软件端依据该坐标驱动近景相机；The algorithm side of the long-range camera uses the deep learning network model to identify the power port to obtain the center coordinates of the power port. After conversion, it is sent to the software side through MQTT, and the software side drives the close-range camera according to the coordinates;
当近景相机的移动轴到达指定坐标位置，打开近景相机，近景相机抓拍图片并使用其算法端深度学习模型识别定位电源口和插销，将识别定位后的结果换算过后通过MQTT传到软件端，软件端依照换算结果控制移动近景相机到达结果指定位置；When the moving axis of the close-up camera reaches the specified coordinate position, turn on the close-up camera. The close-up camera captures pictures and uses its algorithm-side deep learning model to identify and locate the power port and plug. After the recognition and positioning results are converted, they are sent to the software side through MQTT. The terminal controls the movement of the close-range camera to reach the specified position according to the conversion result;
其中，当近景相机到达指定位置后，再次打开近景相机并使用其算法端的深度学习模型识别电源口，并将识别定位后的结果换算后通过MQTT传到软件 端，软件端依照该结果控制移动近景相机，重复该过程直到近景相机的图片中心点与目标电源口中心点重合时，软件端驱动电源插拔装置到达近景相机中心位置进行电器的绝缘测试。Among them, when the close-up camera arrives at the designated position, turn on the close-up camera again and use the deep learning model on the algorithm side to identify the power port, and convert the result of the recognition and positioning to the software side through MQTT, and the software side controls the mobile close-up according to the result Camera, repeat this process until the center point of the picture of the close-range camera coincides with the center point of the target power port, and the software side drives the power plug-in device to the center of the close-range camera for electrical insulation testing.
一种电器产品绝缘测试电源口定位系统，包括：远景相机、近景相机和软件端；A power port positioning system for electrical product insulation testing, including: a long-range camera, a close-up camera and a software terminal;
所述远景相机，用于获取电器整体图像，并通过标定获取畸变参数对电器整体图像进行去畸变，还用于对去畸变后的电器整体图像进行透视变换提取出感兴趣电源口区域图片，使用深度学习网络模型从感兴趣电源口区域图片中识别定位电源口位置传输到软件端；The perspective camera is used to obtain the overall image of the electrical appliance, and obtains distortion parameters through calibration to de-distort the overall image of the electrical appliance, and is also used to perform perspective transformation on the de-distorted overall image of the electrical appliance to extract a picture of the power port area of interest. The deep learning network model identifies and locates the position of the power port from the pictures of the power port area of interest and transmits it to the software side;
所述近景相机，用于获取电源口区域图片，使用深度学习网络模型从电源口区域图片中识别定位电源口位置及电源口对应的插销位置；还用于根据识别定位到的电源口的中心点计算该中心点与图片中心点距离，取最小距离对应的电源口为目标电源口，根据目标电源口及其对应的插销识别结果计算各插销中心点距离以判断电源口方向；The close-range camera is used to obtain a picture of the power port area, and uses a deep learning network model to identify and locate the position of the power port and the corresponding plug position of the power port from the picture of the power port area; it is also used to locate the center point of the power port according to the identification Calculate the distance between the center point and the center point of the picture, take the power port corresponding to the minimum distance as the target power port, and calculate the distance between the center points of each plug according to the target power port and its corresponding plug identification results to determine the direction of the power port;
所述软件端，用于根据远景相机获得的电源口位置驱动近景相机的图像中心点到达与电源口中心点重合位置；用于根据近景相机的识别定位结果驱动近景相机到达指定位置；还用于根据电源口方向和目标电源口，驱动电源插拔装置达到近景相机位置进行产品的绝缘测试。The software end is used to drive the center point of the image of the close-range camera to the position coincident with the center point of the power port according to the position of the power port obtained by the long-range camera; it is used to drive the close-range camera to a designated position according to the identification and positioning results of the close-range camera; it is also used to According to the direction of the power port and the target power port, drive the power plug-in device to the position of the close-up camera for product insulation testing.
本发明的有益效果：本发明采用近景相机和远景相机相结合进行深度学习识别定位，提高识别准确性，解决电器产品电源口位置随机出现及电源口方向不确定、相关特征提取困难的问题；本发明对远景相机进行标定及相关图像处理，避免远景相机定位的结果偏差，进一步提高定位准确性；通过本发明电源口的定位可顺利进行电器产品高压测试，效果良好，提高检测效率。Beneficial effects of the present invention: the present invention uses a combination of a close-range camera and a long-range camera for deep learning recognition and positioning, improves recognition accuracy, and solves the problems of random occurrence of power port positions of electrical products, uncertain direction of power port, and difficulty in extracting relevant features; The invention performs calibration and related image processing on the perspective camera, avoids the deviation of the positioning result of the perspective camera, and further improves the positioning accuracy; through the positioning of the power port of the present invention, the high-voltage test of electrical products can be carried out smoothly, the effect is good, and the detection efficiency is improved.
附图说明Description of drawings
图1是本发明方法流程示意图；Fig. 1 is a schematic flow sheet of the method of the present invention;
图2是本发明黑白标定板样式示意图；Fig. 2 is a schematic diagram of the style of the black and white calibration plate of the present invention;
图3是本发明远景相机深度学习网络结构示意图；Fig. 3 is a schematic diagram of the deep learning network structure of the vision camera of the present invention;
图4是本发明近景相机深度学习网络结构示意图；Fig. 4 is a schematic diagram of the deep learning network structure of the close-range camera of the present invention;
图5是本发明算法端与软件端算法交互流程示意图。Fig. 5 is a schematic diagram of the algorithm interaction process between the algorithm end and the software end of the present invention.
具体实施方式Detailed ways
下面结合附图和具体实施例对本发明作进一步说明，以使本领域的技术人员可以更好地理解本发明并能予以实施，但所举实施例不作为对本发明的限定。The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, so that those skilled in the art can better understand the present invention and implement it, but the examples given are not intended to limit the present invention.
实施例一Embodiment one
参考图1，本发明实施例提供了一种电器产品绝缘测试电源口定位方法，包括以下步骤：Referring to Fig. 1, an embodiment of the present invention provides a method for locating the power port of an electrical product insulation test, including the following steps:
S1：安装远景相机和近景相机，所述远景相机获取电器整体图像；S1: installing a long-range camera and a close-range camera, the long-range camera obtains an overall image of the electrical appliance;
S2：对远景相机进行标定获取畸变参数，通过畸变参数对电器整体图像进行去畸变；S2: Calibrate the perspective camera to obtain distortion parameters, and de-distort the overall image of the appliance through the distortion parameters;
S3：对去畸变后的电器整体图像进行透视变换提取出感兴趣电源口区域图片；S3: Perform perspective transformation on the overall image of the electrical appliance after de-distortion to extract the image of the power port area of interest;
S4：使用深度学习网络模型从感兴趣电源口区域图片中识别定位电源口位置；S4: Use the deep learning network model to identify and locate the position of the power port from the pictures of the power port area of interest;
S5：根据远景相机获得的电源口位置驱动近景相机到达该位置；S5: Drive the close-range camera to reach the position according to the position of the power port obtained by the distant view camera;
S6：近景相机获取电源口区域图片，使用深度学习网络模型从电源口区域 图片中识别定位电源口位置以及电源口对应的插销位置；S6: The close-range camera obtains the picture of the power port area, and uses the deep learning network model to identify and locate the position of the power port and the corresponding pin position from the picture of the power port area;
S7：根据S6中识别定位到的电源口的中心点计算该中心点与电源口区域图片中心点距离，取最小距离对应的电源口为目标电源口；S7: Calculate the distance between the center point and the center point of the power port area picture according to the center point of the identified and positioned power port in S6, and take the power port corresponding to the minimum distance as the target power port;
S8：根据目标电源口及其对应的插销识别结果计算各插销中心点距离以判断电源口方向；S8: Calculate the center point distance of each plug according to the identification result of the target power port and the corresponding plug to determine the direction of the power port;
S9：根据电源口方向和目标电源口，驱动电源插拔装置达到此时近景相机位置进行产品的绝缘测试。S9: According to the direction of the power port and the target power port, drive the power plug-in device to the position of the close-up camera at this time to conduct the insulation test of the product.
具体的，1、相机选型和安装：Specifically, 1. Camera selection and installation:
1)远景相机选型与安装：远处的相机需要较大的视野以保证拍摄的图像能够看到电器的整体，结合相机的各个参数以及实际情况，本实施例选择海康相机且安装于测试机较远一端；1) Selection and installation of long-range cameras: distant cameras need a larger field of view to ensure that the captured images can see the whole appliance. Combined with the various parameters of the camera and the actual situation, this embodiment selects a Hikvision camera and installs it in the test the far end of the machine;
2)近景相机选型与安装：近处的相机需要一定的视野以满足抓拍到感兴趣的电源口信息且相机应与电源插拔装置平行。结合实际情况，选择某型号相机且安装于测试集较近位置，该位置的的近景相机中心点与电源插拔装置的中心点的横向与纵向距离大约分别为50mm与11mm。2) Selection and installation of the close-up camera: The close-up camera needs a certain field of view to capture the information of the power port of interest and the camera should be parallel to the power plug-in device. Combined with the actual situation, select a certain type of camera and install it near the test set. The horizontal and vertical distances between the center point of the close-range camera and the center point of the power plug-in device at this position are about 50mm and 11mm, respectively.
2、相机标定与去畸变：图像的畸变会导致图像定位结果的偏差，为了尽可能减少图像畸变对定位结果的影响，使用标定的方法对相机进行标定进而获取图像的内参及畸变参数从而对图像进行去畸变的操作。值得注意的是，在本申请中，由于近景相机离被拍摄对象较近且相机本身的畸变量可以忽略不计，则只考虑远景相机的标定及去畸变。远景相机的标定：2. Camera calibration and de-distortion: The distortion of the image will lead to the deviation of the image positioning results. In order to minimize the impact of image distortion on the positioning results, the calibration method is used to calibrate the camera and then obtain the internal parameters of the image and the distortion parameters. Perform the dewarping operation. It is worth noting that in this application, only the calibration and de-distortion of the long-range camera are considered because the close-range camera is relatively close to the subject and the distortion of the camera itself is negligible. Calibration of the perspective camera:
2.1标定板的制作：标定板采用黑白方格板的形式，将标定板划分为9*7的黑白方格，每一格的长度为0.026m，具体形式如下图2所示；2.1 Production of the calibration board: the calibration board is in the form of a black and white square grid, and the calibration board is divided into 9*7 black and white squares, and the length of each grid is 0.026m. The specific form is shown in Figure 2 below;
2.2标定图片的采集：在准备好标定板后，调整远景相机相关参数，使得这些参数能够满足我们的需求。调整好这些参数过后，固定远景相机的参数，将 标定板放置于相机前的不同位置，拍摄含有标定板的图像。一般来讲，拍摄的图像越多，标定的精度越高。这里在32个不同的位置处抓拍含有标定板的图片；2.2 Acquisition of calibration pictures: After preparing the calibration board, adjust the relevant parameters of the perspective camera so that these parameters can meet our needs. After adjusting these parameters, fix the parameters of the remote camera, place the calibration plate at different positions in front of the camera, and take images containing the calibration plate. Generally speaking, the more images captured, the higher the calibration accuracy. Here, the pictures containing the calibration board are captured at 32 different positions;
2.3使用张正友标定法进行相机标定：在不考虑畸变的情况下，相机的成像过程可以简化为小孔成像模型。其中涉及到四个坐标系，即世界坐标系、相机坐标系、图像坐标系与像素坐标系的坐标间的转换关系。世界坐标系和相机坐标系之间存在坐标系之间的旋转和原点之间的平移变换，相机坐标系与图像坐标系中的点之间的关系可以根据相似三角形的原理得出。张正友相机标定法中，对于拍摄到的棋盘格图像，假设棋盘格所处平面的Zw坐标为0，则棋盘格上任意一个角点的坐标可以表示为(X w，Y w，0)，此时即可将世界坐标系与像素坐标系间的关系简化为式(3-1)。 2.3 Camera calibration using Zhang Zhengyou's calibration method: Without considering the distortion, the imaging process of the camera can be simplified to a pinhole imaging model. It involves four coordinate systems, namely the conversion relationship between coordinates of the world coordinate system, camera coordinate system, image coordinate system and pixel coordinate system. There is a rotation between the coordinate systems and a translation transformation between the origins between the world coordinate system and the camera coordinate system. The relationship between the camera coordinate system and the points in the image coordinate system can be obtained according to the principle of similar triangles. In Zhang Zhengyou’s camera calibration method, for the captured checkerboard image, assuming that the Zw coordinate of the plane where the checkerboard is located is 0, then the coordinates of any corner point on the checkerboard can be expressed as (X w , Y w , 0), where The relationship between the world coordinate system and the pixel coordinate system can be simplified into formula (3-1).
Figure PCTCN2022076358-appb-000005
Figure PCTCN2022076358-appb-000005
世界坐标系中棋盘格上各角点之间的实际距离能够很容易得到，这时候可以获取多组像素点和对应的世界坐标系的点，进而可以求取H。之后可以根据旋转矩阵是正交矩阵的性质，即其列向量为单位向量且相互正交，来求取内参矩阵K。通过计算得到相机的内参矩阵的值有[f x f y]＝[2953.458072951.58045]，[c x c y]＝[1531.484111031.75475]且标定的反投影误差为0.17个像素，标定精度满足要求。 The actual distance between the corner points on the checkerboard in the world coordinate system can be easily obtained. At this time, multiple sets of pixel points and corresponding points in the world coordinate system can be obtained, and then H can be calculated. Then, the internal reference matrix K can be obtained according to the property that the rotation matrix is an orthogonal matrix, that is, its column vectors are unit vectors and are mutually orthogonal. The values of the internal reference matrix of the camera obtained through calculation are [f x f y ]=[2953.458072951.58045], [c x c y ]=[1531.484111031.75475] and the calibrated back-projection error is 0.17 pixels, and the calibration accuracy meets the requirements .
小孔成像模型中将透镜理想化的等效为一个小孔，但是实际中为了使得产生的图像更加明亮，透镜需要有一定的直径来实现采光。由于透镜的形状导致其径向方向的图像放大倍数不一致，使得成像发生扭曲，这种现象称为相机的径向畸变。径向畸变是因为透镜自身的形状而导致的，而且畸变影响效果呈中心对称。除此之外，人为安装相机时，不可能完全的避免因操作等人为因素导致的误差，透镜平面和成像平面的相对位置也必然不可能实现严格意义上的平行，这必将会导致在切向方向上出现畸变。虽然相机还会受到其他种类的畸变 的影响，但在所有畸变种类中径向畸变和切向畸变占据了绝大部分，而且径向畸变占主要成分。In the pinhole imaging model, the lens is ideally equivalent to a pinhole, but in practice, in order to make the generated image brighter, the lens needs to have a certain diameter to achieve light collection. Due to the shape of the lens, the magnification of the image in the radial direction is inconsistent, causing the image to be distorted. This phenomenon is called radial distortion of the camera. Radial distortion is caused by the shape of the lens itself, and the effect of distortion is center-symmetric. In addition, when installing the camera artificially, it is impossible to completely avoid errors caused by human factors such as operation, and the relative positions of the lens plane and the imaging plane must not be strictly parallel, which will inevitably lead to Distortion occurs in the direction. Although the camera is also affected by other types of distortion, radial and tangential distortions account for the vast majority of all distortion types, and radial distortion is the main component.
对相机进行标定后，可以使用标定得到的参数对三维空间的点进行重投影操作得到未畸变的点，再使用多项式对未畸变点和畸变点之间的数学模型进行拟合。径向畸变由公式(3-2)来描述，其是图像中像素点距图像坐标系原点的距离r的函数。切向畸变由公式(3-3)来描述。公式(3-2)、(3-3)中的(u，v)表示理想的没有发生畸变的坐标，(u′，v′)表示对应的畸变坐标。After the camera is calibrated, the parameters obtained from the calibration can be used to reproject the points in the three-dimensional space to obtain the undistorted points, and then the polynomial is used to fit the mathematical model between the undistorted points and the distorted points. Radial distortion is described by formula (3-2), which is a function of the distance r between the pixel point in the image and the origin of the image coordinate system. Tangential distortion is described by formula (3-3). (u, v) in formulas (3-2) and (3-3) represent ideal coordinates without distortion, and (u′, v′) represent corresponding distorted coordinates.
Figure PCTCN2022076358-appb-000006
Figure PCTCN2022076358-appb-000006
Figure PCTCN2022076358-appb-000007
Figure PCTCN2022076358-appb-000007
标定过程首先将内部参数初始化为不考虑相机畸变的封闭解，通过初始的相机参数对世界坐标系中的点进行重投影操作得到理想的像素坐标，之后基于最小化重投影误差的原理进行非线性优化。通过重投影的理想图像坐标和畸变图像上的角点坐标拟合畸变模型的系数，畸变模型并非越复杂越好。复杂的畸变模型存在高阶的拟合关系，有时会适得其反，引起很大的计算误差。如果同时考虑径向和切向两种畸变，可以发现径向畸变占畸变模型的绝大部分，而切向畸变的影响可以忽略不计。当同时考虑切向畸变和径向畸变来拟合相机畸变模型时，会发现切向畸变系数的不确定性很大，所以一般只考虑径向畸变对图像产生的影响。此时得到的相机的畸变参数为：[-0.3626 0.3051 -0.0004 0.0000 0.0000]。In the calibration process, the internal parameters are first initialized to a closed solution that does not consider camera distortion, and the ideal pixel coordinates are obtained by reprojecting the points in the world coordinate system through the initial camera parameters, and then based on the principle of minimizing the reprojection error. Non-linear optimization. The coefficients of the distortion model are fitted by the reprojected ideal image coordinates and the corner coordinates on the distorted image. The more complex the distortion model is, the better. Complex distortion models have high-order fitting relationships, which sometimes backfire and cause large calculation errors. If radial and tangential distortions are considered at the same time, it can be found that radial distortion accounts for the vast majority of the distortion model, while the influence of tangential distortion can be ignored. When tangential distortion and radial distortion are considered to fit the camera distortion model at the same time, it will be found that the uncertainty of the tangential distortion coefficient is very large, so generally only the influence of radial distortion on the image is considered. The distortion parameters of the camera obtained at this time are: [-0.3626 0.3051 -0.0004 0.0000 0.0000].
使用张正友标定法得到相机的畸变参数后，对图像进行去畸变处理，得到去畸变处理前后的图片。After using the Zhang Zhengyou calibration method to obtain the distortion parameters of the camera, the image is de-distorted to obtain pictures before and after de-distortion processing.
3、远处及近景相机电源口识别与定位3. Identify and locate the power port of the distant and close-up camera
3.1远景相机粗定位：3.1 Coarse positioning of the long-range camera:
3.1.1基于透视变换的感兴趣区域提取方法：透视变换是一种将图片投影到一个新的视平面的方法，该方法需要已知图片上的四个点，通过对这四个点进行计算而得到感兴趣的区域。假设进行透视变换前的图片像素值为(U，V)，进行透视变换后的图片像素值为(x′，y′)，根据透视变换的原理则有：3.1.1 Method of extracting regions of interest based on perspective transformation: Perspective transformation is a method of projecting a picture to a new viewing plane. This method requires four known points on the picture. By calculating these four points to get the region of interest. Assuming that the pixel value of the picture before perspective transformation is (U, V), and the pixel value of the picture after perspective transformation is (x′, y′), according to the principle of perspective transformation:
Figure PCTCN2022076358-appb-000008
Figure PCTCN2022076358-appb-000008
Figure PCTCN2022076358-appb-000009
Figure PCTCN2022076358-appb-000009
Figure PCTCN2022076358-appb-000010
Figure PCTCN2022076358-appb-000010
其中，
Figure PCTCN2022076358-appb-000011
为目标矩阵，依据以上的叙述，依次选取电器产品面板所在的四个角点为透视变换输入值，透视变换后的感兴趣区域图片。 in,
Figure PCTCN2022076358-appb-000011
As the target matrix, according to the above description, select the four corner points where the panel of the electrical product is located as the input value of the perspective transformation, and the image of the region of interest after the perspective transformation.
3.1.2基于深度学习的感兴趣区域电源口识别与定位方法：在得到使用透视变换的图片后，对图片中的电源口进行识别与定位，识别与定位的方法使用深度学习方法。考虑到边缘计算设备的计算资源有限，使用精简的小网络模型进行电源口的识别与定位。该网络的结构如图3所示，使用较为轻量级的特征提取网络并且使用特征图融合的操作。考虑到该网络完成的任务相应比较简单，则只使用少量的检测头。3.1.2 The method of identifying and locating the power outlet in the area of interest based on deep learning: After obtaining the picture using perspective transformation, identify and locate the power outlet in the picture, and the identification and positioning method uses the deep learning method. Considering the limited computing resources of edge computing devices, a simplified small network model is used to identify and locate power ports. The structure of the network is shown in Figure 3, using a relatively lightweight feature extraction network and using the operation of feature map fusion. Considering that the task completed by this network is relatively simple, only a small number of detection heads are used.
考虑到数据采集困难且采集到的数据量偏少，本申请将训练集与测试集的比例设置为1：1，以此来增加模型学习的难度。除此之外，本申请也使用了数据增强的方案，包括图片饱和度调整、曝光度调整与色度调整。以使得到的结果更接近于实际情况。Considering the difficulty of data collection and the small amount of collected data, this application sets the ratio of training set to test set to 1:1 to increase the difficulty of model learning. In addition, this application also uses a data enhancement solution, including image saturation adjustment, exposure adjustment, and hue adjustment. In order to get the result closer to the actual situation.
模型训练参数的设置对于获取较好的结果至关重要。本申请模型训练的相 关参数有：输入模型的图片大小设置为608*608，模型参数的优化方案使用带动量的SGD且使用预热操作以及预训练模型加快模型训练速度；模型的iou损失使用ciou损失，检测框的nms使用简单nms。损失随着训练轮数的增加而逐渐降低，测试集上的map值随着训练轮数的增大而逐渐增大，最终稳定。使用该模型在训练集与测试集上的检测效果。The setting of model training parameters is crucial to obtain better results. The relevant parameters of the model training in this application are: the image size of the input model is set to 608*608, the optimization scheme of the model parameters uses SGD with momentum and uses the preheating operation and the pre-training model to speed up the model training; the iou loss of the model uses ciou Loss, nms of detection boxes use simple nms. The loss gradually decreases with the increase of the number of training rounds, and the map value on the test set gradually increases with the increase of the number of training rounds, and finally stabilizes. Use the detection performance of the model on the training set and test set.
值得注意的是，检测得到的电源口的中心点坐标是基于图片坐标系的，在本申请中需要将图片坐标系的原点置于透视变化过后图片的左下角。除此之外，还需要将基于原始图片左下角为原点的坐标再次转换为以移动轴的原点的坐标，以便于与软件端进行信息的交互。It is worth noting that the detected central point coordinates of the power port are based on the image coordinate system. In this application, the origin of the image coordinate system needs to be placed at the lower left corner of the image after the perspective change. In addition, it is also necessary to convert the coordinates based on the origin of the lower left corner of the original image to the coordinates of the origin of the moving axis, so as to facilitate information interaction with the software.
3.2近景相机精确定位方案3.2 Close-range camera precise positioning scheme
由于电器产品每次停留的位置并不固定，基于远景相机的电源口识别定位只能大致确定电源口的位置，并不能每次都能精准给出电源口的中心点坐标。除此之外，电器产品的电源口的方向也不一致，远景相机并不能清晰抓拍到电源口中的插销。因此，本申请使用近景相机进行电源口精准的识别与定位。Since the position of the electrical product is not fixed each time, the power port identification and positioning based on the vision camera can only roughly determine the position of the power port, and cannot accurately give the center point coordinates of the power port every time. In addition, the direction of the power port of electrical products is also inconsistent, and the long-range camera cannot clearly capture the plug in the power port. Therefore, this application uses a close-range camera to accurately identify and locate the power port.
3.2.1相机坐标系与像素坐标系转换原理：根据相机坐标系与像素坐标系的转换原理，结合上文所叙述的近景相机安装方法，本申请利用该原理，使得图像的中心点与电源口的中心点重合时，认为相机到达了指定的位置。此时，调整相机与插拔装置的位置，使得插拔装置到达相机位置。3.2.1 The conversion principle between the camera coordinate system and the pixel coordinate system: According to the conversion principle between the camera coordinate system and the pixel coordinate system, combined with the installation method of the close-up camera described above, this application uses this principle to make the center point of the image and the power port The camera is considered to have arrived at the specified position when the center points of are coincident. At this time, adjust the positions of the camera and the plugging device so that the plugging device reaches the position of the camera.
3.2.2基于深度学习方法的电源口识别与定位方法：当左侧相机到达远景相机检测而转换到的坐标时，对图片中的电源口以及插销进行识别与定位，识别与定位的方法使用深度学习方法。考虑到边缘计算设备的计算资源有限，使用小网络模型进行电源口及插销的识别与定位。该网络的结构如图4所示，仅使用较为轻量级的特征提取网络并且使用特征图融合的操作。3.2.2 Power port identification and positioning method based on deep learning method: When the left camera reaches the coordinates converted by the vision camera detection, identify and locate the power port and plug in the picture, and the identification and positioning method uses depth study method. Considering the limited computing resources of edge computing devices, a small network model is used to identify and locate power ports and plugs. The structure of the network is shown in Figure 4, which only uses a relatively lightweight feature extraction network and uses the operation of feature map fusion.
考虑到数据采集困难且采集到的数据量偏少，本申请将训练集与测试集的比例设置为1：1，以此来增加模型学习的难度。除此之外，本申请也使用了数 据增强的方案，包括图片饱和度调整、曝光度调整与色度调整。以使得到的结果更接近于实际情况。Considering the difficulty of data collection and the small amount of collected data, this application sets the ratio of training set to test set to 1:1 to increase the difficulty of model learning. In addition, this application also uses a data enhancement solution, including image saturation adjustment, exposure adjustment, and hue adjustment. In order to get the result closer to the actual situation.
模型训练参数的设置对于获取较好的结果至关重要。本申请模型训练的相关参数有：输入模型的图片大小设置为416*416，模型参数的优化方案使用带动量的SGD且使用预热操作以及预训练模型加快模型训练速度；模型的iou损失使用ciou损失，检测框的nms使用简单nms。损失随着训练轮数的增加而逐渐降低，测试集上的map值随着训练轮数的增大而逐渐增大，最终稳定。The setting of model training parameters is crucial to obtain better results. The relevant parameters of the model training in this application are: the image size of the input model is set to 416*416, the optimization scheme of the model parameters uses SGD with momentum and uses the preheating operation and the pre-training model to speed up the model training; the iou loss of the model uses ciou Loss, nms of detection boxes use simple nms. The loss gradually decreases with the increase of the number of training rounds, and the map value on the test set gradually increases with the increase of the number of training rounds, and finally stabilizes.
值得注意的是，使用该模型在训练集与测试集上的检测效果可以看出，该模型具有良好的电源口检测效果而对于，对于离图像中心点最近的电源口的插销的识别效果良好。在电器高压测试的过程中，只需要对众多电源口中的一个进行测试即可。因此，该模型足以满足生产需求。It is worth noting that, using the detection effect of the model on the training set and the test set, it can be seen that the model has a good detection effect of the power port and has a good recognition effect on the pin of the power port closest to the center point of the image. In the process of electrical high-voltage testing, only one of the many power ports needs to be tested. Therefore, this model is good enough for production needs.
当检测得到电源口的位置后，提取出每个检测到的电源口的中心点坐标而计算该中心点坐标与图片中心点坐标的欧式距离，依据欧式距离的大小，取最小的欧式距离对应的电源口为本申请的目标电源口进行高压测试。本申请中，近景相机采集得到的图片尺寸为640：480。除此之外，近景相机需要移动的距离是在远景相机到达位置的基础上进行移动的，计算方法为检测框的宽高分别与图片宽高的比例乘以电源框的实际的宽与高，而电源框实际的宽高值分别为23mm与30mm。After the position of the power port is detected, the center point coordinates of each detected power port are extracted to calculate the Euclidean distance between the center point coordinates and the picture center point coordinates. According to the size of the Euclidean distance, the smallest Euclidean distance corresponding The power port is the target power port of this application to perform a high voltage test. In this application, the image size captured by the close-range camera is 640:480. In addition, the distance that the close-range camera needs to move is based on the arrival position of the long-range camera. The calculation method is to multiply the ratio of the width and height of the detection frame to the width and height of the picture by the actual width and height of the power frame. The actual width and height of the power box are 23mm and 30mm respectively.
在实际的场景中，电源口的方向会有变动，有的时候朝左，有的时候朝右。此时需要根据插销的识别结果判断电源口的方向。确定了离图片中心点最近的电源口后，提取插销识别的结果，计算各插销中点的欧式距离，取欧式距离最大的两个插销中任一插销的中点为原点，向右手边为水平方向正方向，向左手边为水平方向的负方向。当除欧式距离最大的两个插销外的第三个插销的的中心点的横坐标为负值时，则表明电源口的开口向左；当除欧式距离最大的两个插销外的第三个插销的的中心点的横坐标为负值时，则表明电源口的开口向右。此时可以通过信息交互的方式使得软件端控制移动轴动作而改变插拔装置的方 向。In the actual scene, the direction of the power port will change, sometimes to the left, sometimes to the right. At this time, it is necessary to judge the direction of the power port according to the identification result of the pin. After determining the power port closest to the center point of the picture, extract the plug recognition result, calculate the Euclidean distance between the midpoints of each plug, take the midpoint of any of the two plugs with the largest Euclidean distance as the origin, and the right-hand side as the horizontal The direction is the positive direction, and the left-hand side is the negative direction of the horizontal direction. When the abscissa of the center point of the third pin except the two pins with the largest Euclidean distance is a negative value, it indicates that the opening of the power port is left; when the third pin except the two pins with the largest Euclidean distance When the abscissa of the center point of the plug is a negative value, it indicates that the opening of the power port is to the right. At this time, the software side can control the movement of the moving axis to change the direction of the plug-in device through information interaction.
4、算法端与软件端通信方式：上述远景相机和近景相机均安装有算法端，算法端需要与软件端实现交互，从而实现各种计算上传和控制移动，具体的，算法端与软件端通信通过MQTT发送JSON格式的消息来进行交互。远景相机在收到电器就位的信号后，打开相机而抓拍图片，同时模型识别电源口得到电源口的中心坐标，转换过后通过MQTT发送到软件端，软件端依据该坐标动作。当移动轴到达指定位置，打开近景相机，近景相机抓拍图片从而使用深度学习模型识别定位，将识别定位后的结果换算过后通过MQTT传到软件端，软件端依照换算结果移动并最终动作，具体动作如图5所示。4. The communication method between the algorithm end and the software end: the above-mentioned long-range camera and close-range camera are equipped with an algorithm end, and the algorithm end needs to interact with the software end, so as to realize various calculation uploads and control movements. Specifically, the algorithm end communicates with the software end Interact by sending messages in JSON format via MQTT. After the vision camera receives the signal that the electrical appliance is in place, it turns on the camera and captures a picture. At the same time, the model recognizes the power port to get the center coordinates of the power port. After conversion, it is sent to the software end through MQTT, and the software end acts according to the coordinates. When the moving axis reaches the designated position, turn on the close-up camera, and the close-up camera captures pictures to identify and locate using the deep learning model. The result after the recognition and positioning is converted and sent to the software side through MQTT. The software side moves according to the conversion result and finally takes action. Specific actions As shown in Figure 5.
算法的实际效果测试：在完成上述算法的设计及与软件端的联合调试后，使用实际的电器产品进行测试。测试发现，除插拔装置碰撞导致电器产品位置改变而无法找准电器产品电源口外，算法具有良好的效果，插拔装置可以顺利进行电器产品的高压测试。The actual effect test of the algorithm: After completing the design of the above algorithm and the joint debugging with the software side, use the actual electrical product for testing. The test found that the algorithm has a good effect, except that the position of the electrical product changes due to the collision of the plug-in device and the power port of the electrical product cannot be located. The plug-in device can smoothly carry out the high-voltage test of the electrical product.
实施例二Embodiment two
基于同一发明构思，本实施例提供了一种电器产品绝缘测试电源口定位系统，其解决问题的原理与一种电器产品绝缘测试电源口定位方法类似，重复之处不再赘述。Based on the same inventive concept, this embodiment provides a system for locating a power port for an electrical product insulation test. The principle for solving the problem is similar to a method for locating a power port for an electrical product insulation test.
一种电器产品绝缘测试电源口定位系统，包括：包括：远景相机、近景相机和软件端；A power port positioning system for insulation testing of electrical products, including: a long-range camera, a close-up camera and a software terminal;
所述远景相机，用于获取电器整体图像，并通过标定获取畸变参数对电器整体图像进行去畸变，还用于对去畸变后的电器整体图像进行透视变换提取出感兴趣电源口区域图片，使用深度学习网络模型从感兴趣电源口区域图片中识别定位电源口位置传输到软件端；The perspective camera is used to obtain the overall image of the electrical appliance, and obtains distortion parameters through calibration to de-distort the overall image of the electrical appliance, and is also used to perform perspective transformation on the de-distorted overall image of the electrical appliance to extract a picture of the power port area of interest. The deep learning network model identifies and locates the position of the power port from the pictures of the power port area of interest and transmits it to the software side;
所述近景相机，用于获取电源口区域图片，使用深度学习网络模型从电源口区域图片中识别定位电源口位置及电源口对应的插销位置；还用于根据识别 定位到的电源口的中心点计算该中心点与图片中心点距离，取最小距离对应的电源口为目标电源口，根据目标电源口及其对应的插销识别结果计算各插销中心点距离以判断电源口方向；The close-range camera is used to obtain a picture of the power port area, and uses a deep learning network model to identify and locate the position of the power port and the corresponding plug position of the power port from the picture of the power port area; it is also used to locate the center point of the power port according to the identification Calculate the distance between the center point and the center point of the picture, take the power port corresponding to the minimum distance as the target power port, and calculate the distance between the center points of each plug according to the target power port and its corresponding plug identification results to determine the direction of the power port;
所述软件端，用于根据远景相机获得的电源口位置驱动近景相机的图像中心点到达与电源口中心点重合位置；用于根据近景相机的识别定位结果驱动近景相机到达指定位置；还用于根据电源口方向和目标电源口，驱动电源插拔装置达到近景相机位置进行产品的绝缘测试。The software end is used to drive the center point of the image of the close-range camera to the position coincident with the center point of the power port according to the position of the power port obtained by the long-range camera; it is used to drive the close-range camera to a designated position according to the identification and positioning results of the close-range camera; it is also used to According to the direction of the power port and the target power port, drive the power plug-in device to the position of the close-up camera for product insulation testing.
本发明通过远近相机结合的方式进行电源口的定位及交互控制定位，解决电器产品电源口位置随机出现、电源口方向不确定及基于手工选取特征提取的识别定位技术相关特征提取困难且难以达到生产要求的现实情况的问题。The invention uses the combination of far and near cameras to locate and interactively control the positioning of the power port, solving the problems of random occurrence of the power port position of electrical products, uncertain direction of the power port, and difficulty in extracting relevant features of the identification and positioning technology based on manually selected feature extraction and difficult to achieve production Ask real-world questions.
以上所述实施例仅是为充分说明本发明而所举的较佳的实施例，本发明的保护范围不限于此。本技术领域的技术人员在本发明基础上所作的等同替代或变换，均在本发明的保护范围之内。本发明的保护范围以权利要求书为准。The above-mentioned embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or transformations made by those skilled in the art on the basis of the present invention are all within the protection scope of the present invention. The protection scope of the present invention shall be determined by the claims.

Claims (10)

  1. 一种电器产品绝缘测试电源口定位方法，其特征在于：包括以下步骤：A method for locating a power port for insulation testing of electrical products, characterized in that it includes the following steps:
    S1：安装远景相机和近景相机，所述远景相机获取电器整体图像；S1: installing a long-range camera and a close-range camera, the long-range camera obtains an overall image of the electrical appliance;
    S2：对远景相机进行标定获取畸变参数，通过畸变参数对电器整体图像进行去畸变；S2: Calibrate the perspective camera to obtain distortion parameters, and de-distort the overall image of the appliance through the distortion parameters;
    S3：对去畸变后的电器整体图像进行透视变换提取出感兴趣电源口区域图片；S3: Perform perspective transformation on the overall image of the electrical appliance after de-distortion to extract the image of the power port area of interest;
    S4：使用深度学习网络模型从感兴趣电源口区域图片中识别定位电源口位置；S4: Use the deep learning network model to identify and locate the position of the power port from the pictures of the power port area of interest;
    S5：根据远景相机获得的电源口位置驱动近景相机到达该位置；S5: Drive the close-range camera to reach the position according to the position of the power port obtained by the distant view camera;
    S6：近景相机获取电源口区域图片，使用深度学习网络模型从电源口区域图片中识别定位电源口位置以及电源口对应的插销位置；S6: The close-range camera obtains the picture of the power port area, and uses the deep learning network model to identify and locate the position of the power port and the corresponding plug position from the picture of the power port area;
    S7：根据S6中识别定位到的电源口的中心点计算该中心点与电源口区域图片中心点距离，取最小距离对应的电源口为目标电源口；S7: Calculate the distance between the center point and the center point of the power port area picture according to the center point of the identified and positioned power port in S6, and take the power port corresponding to the minimum distance as the target power port;
    S8：根据目标电源口及其对应的插销识别结果计算各插销中心点距离以判断电源口方向；S8: Calculate the center point distance of each plug according to the identification result of the target power port and the corresponding plug to determine the direction of the power port;
    S9：根据电源口方向和目标电源口，驱动电源插拔装置达到此时近景相机位置进行产品的绝缘测试。S9: According to the direction of the power port and the target power port, drive the power plug-in device to the position of the close-up camera at this time to conduct the insulation test of the product.
  2. 如权利要求1所述的一种电器产品绝缘测试电源口定位方法，其特征在于：所述近景相机与电源插拔装置平行设置且近景相机的中心点与电源插拔装置的中心点设置在同一水平线上。A method for locating a power supply port for insulation testing of electrical products according to claim 1, wherein the close-up camera is arranged in parallel with the power plugging device, and the center point of the close-up camera is set at the same center point as the power plugging device. on the horizon.
  3. 如权利要求1所述的一种电器产品绝缘测试电源口定位方法及系统定位方法，其特征在于：所述步骤S2具体包括以下步骤：A method for locating a power port of an electrical product insulation test and a method for locating a system according to claim 1, wherein the step S2 specifically includes the following steps:
    S21:制作黑白方格板作为标定板，并固定远景相机；S21: Make a black and white grid plate as a calibration plate, and fix the distant view camera;
    S22：将标定板放置于远景相机前各个位置并拍摄含有标定板的图像；S22: placing the calibration plate at various positions in front of the vision camera and taking images containing the calibration plate;
    S23：采用张正有标定法对远景相机进行标定得到标定参数，通过标定得到的标定参数对三维空间的点进行重投影得到未畸变的点，对未畸变的点和畸变点之间的数学模型进行拟合得到畸变参数；S23: Use Zhang Zhengyou calibration method to calibrate the distant view camera to obtain the calibration parameters, reproject the points in the three-dimensional space to obtain the undistorted points through the calibration parameters obtained through the calibration, and calculate the mathematical model between the undistorted points and the distorted points Fitting is performed to obtain the distortion parameters;
    S24：使用得到的畸变参数对电器整体图像进行去畸变处理，得到去畸变处理后的电器整体图像。S24: Perform de-distortion processing on the overall image of the electrical appliance by using the obtained distortion parameters, and obtain the overall image of the electrical appliance after de-distortion processing.
  4. 如权利要求1所述的一种电器产品绝缘测试电源口定位方法，其特征在于：所述步骤S3具体包括以下步骤：A method for locating the power supply port of an electrical product insulation test according to claim 1, wherein the step S3 specifically includes the following steps:
    S31：设定进行透视变换前的图片像素值为(U，V)，进行透视变换后的图像素值为(x’，y’)S31: Set the image pixel value before perspective transformation to (U, V), and the image pixel value after perspective transformation to (x', y')
    S32：根据透视变换原理则：S32: According to the principle of perspective transformation:
    Figure PCTCN2022076358-appb-100001
    Figure PCTCN2022076358-appb-100001
    Figure PCTCN2022076358-appb-100002
    Figure PCTCN2022076358-appb-100002
    Figure PCTCN2022076358-appb-100003
    Figure PCTCN2022076358-appb-100003
    其中，
    Figure PCTCN2022076358-appb-100004
    为目标矩阵；     in,
    Figure PCTCN2022076358-appb-100004
    is the target matrix;
    S33：选取电器产品面板所在的四个角点为透视变换输入值，进行步骤S31-S32变换，得到透视变换后的感兴趣电源口区域图片。S33: Select the four corner points where the panel of the electrical product is located as the input value of the perspective transformation, perform the transformation in steps S31-S32, and obtain the picture of the power port area of interest after the perspective transformation.
  5. 如权利要求2所述的一种电器产品绝缘测试电源口定位方法，其特征在于：所述步骤S4和步骤S6中均使用轻量级的特征提取网络和特征图融合操作，对感兴趣电源口区域图片中的电源口进行识别与定位。A method for locating power outlets for insulation testing of electrical products as claimed in claim 2, characterized in that: both steps S4 and S6 use a lightweight feature extraction network and feature map fusion operation to locate the power outlet of interest Identify and locate the power outlets in the area picture.
  6. 如权利要求1所述的一种电器产品绝缘测试电源口定位方法，其特征在于，所述步骤S7中，当检测得到电源口的位置后，提取出每个检测到的电源口的中心点坐标并计算该中心点坐标与图片中心点坐标的欧式距离，依据欧式距离的大小，取最小的欧式距离对应的电源口为目标电源口进行高压测试。A method for locating a power supply port for insulation testing of electrical products as claimed in claim 1, wherein in said step S7, after the position of the power supply port is detected, the center point coordinates of each detected power supply port are extracted And calculate the Euclidean distance between the coordinates of the center point and the coordinates of the center point of the picture, and according to the size of the Euclidean distance, take the power port corresponding to the smallest Euclidean distance as the target power port for high-voltage testing.
  7. 如权利要求6所述的一种电器产品绝缘测试电源口定位方法，其特征在于：根据S6中的电源口识别定位结果与S7中的目标电源口结果，反复利用近景相机识别定位目标电源口并根据识别定位结果驱动近景相机移动，当目标电源口的中心点与近景相机的中心点重合时，近景相机到达指定位置不再移动并记录好该位置，根据记录位置驱动电源插拔装置。A method for locating a power port for insulation testing of electrical products as claimed in claim 6, characterized in that: according to the power port identification and positioning result in S6 and the target power port result in S7, the close-range camera is used repeatedly to identify and locate the target power port and Drive the close-up camera to move according to the recognition and positioning results. When the center point of the target power port coincides with the center point of the close-up camera, the close-up camera will not move when it reaches the designated position and record the position, and drive the power plug-in device according to the recorded position.
  8. 如权利要求6所述的一种电器产品绝缘测试电源口定位方法，其特征在于：所述步骤S7中确定目标电源口后，提取插销识别结果，在步骤S8中，计算插销中点的欧式距离，取欧式距离最大的两个插销中任一插销的中点为原点，判断电源口的开口方向，并根据开口方向控制移动电源插拔装置的方向对应目标电源口。A method for locating power outlets for insulation testing of electrical products as claimed in claim 6, characterized in that: after the target power outlet is determined in step S7, the identification result of the pin is extracted, and in step S8, the Euclidean distance of the middle point of the pin is calculated , take the midpoint of any one of the two pins with the largest European distance as the origin, judge the opening direction of the power port, and control the direction of the mobile power plug-in device to correspond to the target power port according to the opening direction.
  9. 如权利要求1-8中任一项所述的一种电器产品绝缘测试电源口定位方法，其特征在于：所述远景相机和近景相机均安装有算法端，所述算法端与软件端通信通过MQTT发送JSON格式的消息来进行交互，所述交互方法具体包括以 下步骤：A method for locating the power supply port of an electrical product insulation test as claimed in any one of claims 1-8, wherein: the long-range camera and the close-range camera are equipped with an algorithm terminal, and the algorithm terminal communicates with the software terminal through MQTT sends messages in JSON format for interaction, and the interaction method specifically includes the following steps:
    当远景相机收到电器就位的信号后，打开相机而抓拍电器整体图像；When the vision camera receives the signal that the appliance is in place, it turns on the camera and captures the overall image of the appliance;
    远景相机的算法端使用深度学习网络模型识别电源口得到电源口的中心坐标，转换过后通过MQTT发送到软件端，软件端依据该坐标驱动近景相机；The algorithm side of the long-range camera uses the deep learning network model to identify the power port to obtain the center coordinates of the power port. After conversion, it is sent to the software side through MQTT, and the software side drives the close-range camera according to the coordinates;
    当近景相机的移动轴到达指定坐标位置，打开近景相机，近景相机抓拍图片并使用其算法端深度学习模型识别定位电源口和插销，将识别定位后的结果换算过后通过MQTT传到软件端，软件端依照换算结果控制移动近景相机到达结果指定位置；When the moving axis of the close-up camera reaches the specified coordinate position, turn on the close-up camera. The close-up camera captures pictures and uses its algorithm-side deep learning model to identify and locate the power port and plug. After the recognition and positioning results are converted, they are sent to the software side through MQTT. The terminal controls the movement of the close-range camera to reach the specified position according to the conversion result;
    其中，当近景相机到达指定位置后，再次打开近景相机并使用其算法端的深度学习模型识别电源口，并将识别定位后的结果换算后通过MQTT传到软件端，软件端依照该结果控制移动近景相机，重复该过程直到近景相机的图片中心点与目标电源口中心点重合时，软件端驱动电源插拔装置到达近景相机中心位置进行电器的绝缘测试。Among them, when the close-up camera arrives at the designated position, turn on the close-up camera again and use the deep learning model on the algorithm side to identify the power port, and convert the result of the recognition and positioning to the software side through MQTT, and the software side controls the mobile close-up according to the result Camera, repeat this process until the center point of the picture of the close-range camera coincides with the center point of the target power port, and the software side drives the power plug-in device to the center of the close-range camera for electrical insulation testing.
  10. 一种电器产品绝缘测试电源口定位系统，其特征在于：包括：远景相机、近景相机和软件端；A power port positioning system for insulation testing of electrical products, characterized in that it includes: a long-range camera, a close-up camera and a software terminal;
    所述远景相机，用于获取电器整体图像，并通过标定获取畸变参数对电器整体图像进行去畸变，还用于对去畸变后的电器整体图像进行透视变换提取出感兴趣电源口区域图片，使用深度学习网络模型从感兴趣电源口区域图片中识别定位电源口位置传输到软件端；The perspective camera is used to obtain the overall image of the electrical appliance, and obtains distortion parameters through calibration to de-distort the overall image of the electrical appliance, and is also used to perform perspective transformation on the de-distorted overall image of the electrical appliance to extract a picture of the power port area of interest. The deep learning network model identifies and locates the position of the power port from the pictures of the power port area of interest and transmits it to the software side;
    所述近景相机，用于获取电源口区域图片，使用深度学习网络模型从电源口区域图片中识别定位电源口位置及电源口对应的插销位置；还用于根据识别定位到的电源口的中心点计算该中心点与图片中心点距离，取最小距离对应的电源口为目标电源口，根据目标电源口及其对应的插销识别结果计算各插销中 心点距离以判断电源口方向；The close-range camera is used to obtain a picture of the power port area, and uses a deep learning network model to identify and locate the position of the power port and the corresponding plug position of the power port from the picture of the power port area; it is also used to locate the center point of the power port according to the identification Calculate the distance between the center point and the center point of the picture, take the power port corresponding to the minimum distance as the target power port, and calculate the distance between the center points of each plug according to the target power port and its corresponding plug identification results to determine the direction of the power port;
    所述软件端，用于根据远景相机获得的电源口位置驱动近景相机的图像中心点到达与电源口中心点重合位置；用于根据近景相机的识别定位结果驱动近景相机到达指定位置；还用于根据电源口方向和目标电源口，驱动电源插拔装置达到近景相机位置进行产品的绝缘测试。The software end is used to drive the center point of the image of the close-range camera to the position coincident with the center point of the power port according to the position of the power port obtained by the long-range camera; it is used to drive the close-range camera to a designated position according to the recognition and positioning results of the close-range camera; it is also used to According to the direction of the power port and the target power port, drive the power plug-in device to the position of the close-up camera for product insulation testing.
PCT/CN2022/076358 2021-09-15 2022-02-15 Power supply port positioning method and system for insulation test of electrical product WO2023040176A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111082500.0 2021-09-15
CN202111082500.0A CN114119459A (en) 2021-09-15 2021-09-15 Method and system for positioning power supply port for insulation test of electrical product

Publications (1)

Publication Number Publication Date
WO2023040176A1 true WO2023040176A1 (en) 2023-03-23

Family

ID=80441105

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/076358 WO2023040176A1 (en) 2021-09-15 2022-02-15 Power supply port positioning method and system for insulation test of electrical product

Country Status (2)

Country Link
CN (1) CN114119459A (en)
WO (1) WO2023040176A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117437187A (en) * 2023-10-13 2024-01-23 深圳市锐巽自动化设备有限公司 Detonator detection method, detonator detection system and storage medium

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117455921B (en) * 2023-12-26 2024-04-02 菲特（天津）检测技术有限公司 Large-depth-of-field imaging detection method and system based on small-field lens

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160325836A1 (en) * 2015-05-07 2016-11-10 The Boeing Company Airplane Cabin Panoramic View System
CN112101118A (en) * 2020-08-18 2020-12-18 东南大学 Far-view and near-view image combined photovoltaic module positioning and matching method
EP3842820A1 (en) * 2019-12-24 2021-06-30 Gunitech Corp. Power positioning method and power positioning device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160325836A1 (en) * 2015-05-07 2016-11-10 The Boeing Company Airplane Cabin Panoramic View System
EP3842820A1 (en) * 2019-12-24 2021-06-30 Gunitech Corp. Power positioning method and power positioning device
CN112101118A (en) * 2020-08-18 2020-12-18 东南大学 Far-view and near-view image combined photovoltaic module positioning and matching method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"Master's Thesis", 1 June 2019, HARBIN INSTITUTE OF TECHNOLOGY, CN, article CHENG, XIAOJING: "Research on RRU Power Port Detection Technology Based on Deep Neural Network", pages: 1 - 74, XP009544509, DOI: 10.27061/d.cnki.ghgdu.2019.001501 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117437187A (en) * 2023-10-13 2024-01-23 深圳市锐巽自动化设备有限公司 Detonator detection method, detonator detection system and storage medium

Also Published As

Publication number Publication date
CN114119459A (en) 2022-03-01

Similar Documents

Publication Publication Date Title
WO2023040176A1 (en) Power supply port positioning method and system for insulation test of electrical product
CN108257183B (en) Camera lens optical axis calibration method and device
WO2017092631A1 (en) Image distortion correction method for fisheye image, and calibration method for fisheye camera
CN109685913B (en) Augmented reality implementation method based on computer vision positioning
CN102810205B (en) The scaling method of a kind of shooting or photographic means
CN110809786B (en) Calibration device, calibration chart, chart pattern generation device, and calibration method
CN106887023A (en) For scaling board and its scaling method and calibration system that binocular camera is demarcated
CN103278138B (en) Method for measuring three-dimensional position and posture of thin component with complex structure
CN108288294A (en) A kind of outer ginseng scaling method of a 3D phases group of planes
CN104361580B (en) Projected image real-time correction method based on planar screen
TW201715880A (en) An image calibrating, composing and depth rebuilding method of a panoramic fish-eye camera and a system thereof
CN103337094A (en) Method for realizing three-dimensional reconstruction of movement by using binocular camera
CN109961485A (en) A method of target positioning is carried out based on monocular vision
CN108053373A (en) One kind is based on deep learning model fisheye image correcting method
JP4948552B2 (en) Camera calibration apparatus, camera calibration method, camera calibration program, and recording medium recording the program
CN111461963B (en) Fisheye image stitching method and device
CN110838164A (en) Monocular image three-dimensional reconstruction method, system and device based on object point depth
CN110349257B (en) Phase pseudo mapping-based binocular measurement missing point cloud interpolation method
CN109151418A (en) A kind of system and method automatically correcting projected image geometric distortion
CN115797461A (en) Flame space positioning system calibration and correction method based on binocular vision
CN112753047B (en) Method and system for in-loop calibration and target point setting of hardware of camera and related equipment
CN107038714A (en) Many types of visual sensing synergistic target tracking method
CN108090877A (en) A kind of RGB-D camera depth image repair methods based on image sequence
CN105739106A (en) Somatosensory multi-view point large-size light field real three-dimensional display device and method
CN113763480B (en) Combined calibration method for multi-lens panoramic camera

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22868556

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22868556

Country of ref document: EP

Kind code of ref document: A1