CN112784853A - Terminal connection state detection method and device - Google Patents
Terminal connection state detection method and device Download PDFInfo
- Publication number
- CN112784853A CN112784853A CN202011552665.5A CN202011552665A CN112784853A CN 112784853 A CN112784853 A CN 112784853A CN 202011552665 A CN202011552665 A CN 202011552665A CN 112784853 A CN112784853 A CN 112784853A
- Authority
- CN
- China
- Prior art keywords
- terminal connection
- plug connector
- plug
- image
- connectors
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 65
- 238000000034 method Methods 0.000 claims abstract description 37
- 239000003086 colorant Substances 0.000 claims abstract description 23
- 238000004422 calculation algorithm Methods 0.000 claims description 26
- 238000012545 processing Methods 0.000 claims description 17
- 238000007781 pre-processing Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 14
- 238000013139 quantization Methods 0.000 claims description 12
- 238000003860 storage Methods 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 10
- 102100030148 Integrator complex subunit 8 Human genes 0.000 claims description 6
- 101710092891 Integrator complex subunit 8 Proteins 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 5
- 238000005498 polishing Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 description 8
- 239000011159 matrix material Substances 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 4
- 238000011179 visual inspection Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 210000001503 joint Anatomy 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000011031 large-scale manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
Abstract
The application relates to the technical field of image recognition, and provides a terminal connection state detection method and a terminal connection state detection device, wherein the method comprises the following steps: determining position coordinates of the plurality of first plug connectors and the plurality of second plug connectors based on the terminal connection image; matching each first plug connector with the second plug connector closest to the first plug connector to obtain a matching result based on the position coordinates of the first plug connectors and the second plug connectors; summing the distances of the first plug connector and the second plug connector of the plurality of matching results to obtain a target distance, and determining the terminal embedding state based on the target distance and a distance threshold; and if the terminal embedding state is embedded, determining the wiring colors of the first plug connector and the second plug connector corresponding to each matching result based on the terminal connection image, and if the wiring colors of the first plug connector and the second plug connector of each matching result are matched, determining that the terminal connection state is normal. The terminal connection state detection method and device can reduce labor cost and improve detection accuracy and efficiency.
Description
Technical Field
The present application relates to the field of image recognition technologies, and in particular, to a method and an apparatus for detecting a terminal connection state.
Background
With the increasing development of electric devices, the transmission of electrical signals needs to be realized through a terminal, which is also called a connection terminal or a terminal, the terminal is generally composed of a joint and an interface, and the connection state of the terminal directly determines the stability and accuracy of the transmission of the electrical signals, so the connection state of the terminal needs to be detected.
At present, the mode of detecting the connection state of the terminal is mainly manual visual inspection, the connection state of the terminal of the electric equipment is manually visual inspection on a large-scale production line, the consumed labor cost is high, mistakes are easy to occur, and the detection efficiency is low.
Disclosure of Invention
The application provides a terminal connection state detection method and device, which are used for reducing labor cost and improving detection accuracy and efficiency.
The application provides a terminal connection state detection method, which comprises the following steps: determining position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors based on a terminal connection image acquired by a camera; matching each first plug connector with the second plug connector closest to the first plug connector based on the position coordinates corresponding to the first plug connectors and the second plug connectors to obtain a matching result; summing the distances between the first plug connector and the second plug connector corresponding to the matching results to obtain a target distance, and determining the terminal embedding state based on the target distance and a distance threshold; and if the terminal embedding state is embedded, determining the wiring colors of the first plug connector and the second plug connector corresponding to each matching result based on the terminal connection image, and if the wiring colors of the first plug connector and the second plug connector corresponding to each matching result are matched, determining that the terminal connection state is normal.
According to the terminal connection state detection method provided by the application, the method for determining the position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors based on the terminal connection image acquired by the camera comprises the following steps: inputting the terminal connection image into a position identification model, and outputting position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors; the position identification model is obtained by training by taking terminal connection image sample data as a sample and position coordinate sample data corresponding to a plurality of first plug connectors and a plurality of second plug connectors corresponding to the terminal connection image sample data as sample labels.
According to the terminal connection state detection method provided by the application, the position identification model adopts a product quantization algorithm added with Quant-Noise and/or an Int8 quantization algorithm during training.
According to the terminal connection state detection method provided by the application, the method for determining the position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors based on the terminal connection image acquired by the camera comprises the following steps: acquiring the terminal connection image through a camera; intercepting a target area image from the terminal connection image; preprocessing the target area image to obtain a reference image; and determining the position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors based on the reference image.
According to the terminal connection state detection method provided by the application, the preprocessing the target area image to obtain a reference image comprises the following steps: and performing matrixing processing and/or color processing on the target area image to obtain a reference image.
According to the terminal connection state detection method provided by the application, the terminal connection image acquired by the camera is as follows: the method comprises the steps of collecting an original image by continuous curved surface polishing of a camera in a diffuse reflection environment, and filtering the original image based on a neighborhood clustering algorithm to obtain the target.
According to the terminal connection state detection method provided by the application, based on the position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors, each first plug connector is matched with the second plug connector closest to the first plug connector to obtain a matching result, the method includes the following steps: and matching each first plug connector with the second plug connector which is closest to the first plug connector based on the Hungarian algorithm and the position coordinates corresponding to the first plug connectors and the second plug connectors to obtain a matching result.
The present application also provides a terminal connection state detection device, the terminal connection state detection device includes: the first determining module is used for determining position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors based on the terminal connection image acquired by the camera; the matching module is used for matching each first plug connector with the second plug connector closest to the first plug connector based on the position coordinates corresponding to the first plug connectors and the second plug connectors to obtain a matching result; the second determining module is used for summing the distances between the first plug connector and the second plug connector corresponding to the matching results to obtain a target distance, and determining the terminal embedding state based on the target distance and a distance threshold; and the third determining module is used for determining the wiring colors of the first plug connector and the second plug connector corresponding to each matching result based on the terminal connection image if the terminal embedding state is embedded, and confirming that the terminal connection state is normal if the wiring colors of the first plug connector and the second plug connector corresponding to each matching result are matched.
According to the terminal connection state detection device provided by the application, the first determination module is further used for inputting the terminal connection image into the position recognition model and outputting position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors; the position identification model is obtained by training sample data of terminal connection image samples, and position coordinate sample data corresponding to a plurality of first plug connectors and a plurality of second plug connectors corresponding to the terminal connection image sample data are sample labels.
According to a terminal connection state detection apparatus provided by the present application, the first determination module includes: the acquisition submodule is used for acquiring the terminal connection image through a camera; the intercepting submodule is used for intercepting a target area image from the terminal connection image; the preprocessing submodule is used for preprocessing the target area image to obtain a reference image; and the position coordinate determination submodule is used for determining the position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors based on the reference image.
The present application further provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-mentioned terminal connection state detection methods when executing the computer program.
The present application also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the terminal connection state detection method as any one of the above.
According to the terminal connection state detection method and device provided by the embodiment of the application, the terminal embedding state is judged according to the position coordinates of the first plug connector and the second plug connector, and when the terminal embedding state is embedded, whether the corresponding relation between the second plug connector and the first plug connector is correct or not is judged according to the colors of the first plug connector and the second plug connector, so that the terminal connection state is detected, the labor cost can be reduced, and the accuracy and the efficiency of detection are improved.
Drawings
In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a terminal connection state detection method provided in the present application;
FIG. 2 is a schematic flow chart diagram illustrating an embodiment of step 110 in a terminal connection status detection method provided herein;
fig. 3 is a schematic structural diagram of a terminal connection state detection device provided in the present application;
fig. 4 is a schematic structural diagram of a first determination module of the terminal connection state detection apparatus provided in the present application;
fig. 5 is a schematic structural diagram of an electronic device provided in the present application.
Detailed Description
To make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings in the present application, and it is obvious that the described embodiments are some, but not all embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terminal connection state detection method and apparatus of the present application are described below with reference to fig. 1 to 5.
As shown in fig. 1, the present application provides a terminal connection state detection method including the following steps 110 to 140.
In step 110, position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors are determined based on the terminal connection image acquired by the camera.
It can be understood that, in the connection process of the terminal of the electric equipment, a plurality of first plug connectors and a plurality of second plug connectors of the terminal are often required to be correspondingly connected, that is, the first plug connector is embedded into the corresponding second plug connector, but because there may be an error during assembly, the first plug connector is not necessarily embedded with the corresponding second plug connector, and the connection state of the first plug connector and the second plug connector needs to be detected.
When the first plug connector is a joint, the second plug connector is an interface; when the first connector is an interface, the second connector is a joint.
The terminal connection state detection method provided by the embodiment of the application can be realized based on a hardware system, the hardware system comprises a camera, a light source, a signal connection card and an analysis host, the analysis host can comprise electronic elements such as a memory, a display card, a processor, a hard disk and a mainboard, a linux or Windows10 system can be operated on the analysis host, and the camera can be an industrial camera.
The camera shoots the joint of the terminal to obtain a terminal connection image, and a coordinate system can be established according to the relative position relation of the first plug connector and the second plug connector in the terminal connection image to obtain the position coordinates of the plurality of first plug connectors and the position coordinates of the plurality of second plug connectors.
And step 120, matching each first plug connector with the second plug connector closest to the first plug connector based on the position coordinates corresponding to the first plug connectors and the second plug connectors to obtain a matching result.
It can be understood that, when the terminal connection state is normal, the first plug connector and the second plug connector are tightly embedded, each first plug connector corresponds to one second plug connector to be embedded, the distance between each first plug connector and each second plug connector is calculated according to the position coordinates of the first plug connector and the second plug connector, the second plug connector with the closest distance between each first plug connector and the second plug connector is found, each first plug connector is matched with the second plug connector with the closest distance, the matching result is obtained, that is, the first plug connectors and the second plug connectors are correspondingly matched, a plurality of matching results are obtained, and each matching result comprises the first plug connector and the second plug connector.
It should be noted that when the connectors are matched with the interfaces, each connector can be found to be matched with the interface closest to the connector, so as to obtain a matching result, and each interface can also be found to be matched with the connector closest to the connector, so as to obtain a matching result.
And step 130, summing the distances between the first plug connector and the second plug connector corresponding to the matching results to obtain a target distance, and determining the terminal embedding state based on the target distance and the distance threshold.
It can be understood that each matching result corresponds to one first plug connector and one second plug connector, the distance between the corresponding first plug connector and the corresponding second plug connector in each matching result can be obtained, the distances between the corresponding first plug connectors and the corresponding second plug connectors of all the matching results are added, the sum result is used as a target distance, a distance threshold value is preset, and the terminal embedding state is determined based on the comparison between the target distance and the distance threshold value, and can be embedded or not embedded.
If the terminal is in the embedded state, all the first plug connectors and the second plug connectors of the terminal are in one-to-one butt joint and are embedded tightly, and if the terminal is not in the embedded state, all the first plug connectors and the second plug connectors of the terminal are not embedded tightly, so that the terminal is loosened.
For example, if the target distance is greater than the distance threshold, it is determined that the first connector and the second connector are in the unmated state, and if the target distance is less than or equal to the distance threshold, it is determined that the first connector and the second connector are in the mated state, and the distance threshold may be 0mm to 2mm, for example, 1 mm.
And 140, if the terminal embedding state is embedded, determining the wiring colors of the first plug connector and the second plug connector corresponding to each matching result based on the terminal connection image, and if the wiring colors of the first plug connector and the second plug connector corresponding to each matching result are matched, determining that the terminal connection state is normal.
It can be understood that, when it is determined that the terminal is in the embedded state, that is, all the first plug connectors and the second plug connectors of the terminal are in one-to-one butt joint and are embedded tightly, it is necessary to determine whether the corresponding relationship between the first plug connector and the second plug connector in each matching result is correct or not, and whether an erroneous insertion exists or not.
Here, the connection color of the first plug connector and the connection color of the second plug connector in each matching result are obtained according to the terminal connection image, whether the connection color of the first plug connector and the connection color of the second plug connector are matched is judged, for example, when the connection color of the first plug connector and the connection color of the second plug connector are the same, it is determined that the corresponding relationship between the first plug connector and the second plug connector in the matching results is correct.
And if the wiring colors of the first plug connector and the second plug connector corresponding to all the matching results are matched, the corresponding relation between the first plug connector and the second plug connector in all the matching results is correct, and then the terminal connection state is normal.
When the terminal embedding state is judged, the wiring color of the terminal does not need to be considered, and the operation is simple, so that whether the first plug connector and the second plug connector are embedded tightly or not is judged, whether the correspondence between the first plug connector and the second plug connector is correct or not is judged according to the wiring color of the terminal under the condition that the terminal embedding state is embedded, the processing sequence which is easy before and difficult after is adopted, and the operation speed can be improved.
According to the terminal connection state detection method provided by the embodiment of the application, the terminal embedding state is judged according to the position coordinates of the first plug connector and the second plug connector, and when the terminal embedding state is embedded, whether the corresponding relation between the second plug connector and the first plug connector is correct or not is judged according to the colors of the first plug connector and the second plug connector, so that the terminal connection state is detected, the labor cost can be reduced, and the accuracy and the efficiency of detection are improved.
In some embodiments, the determining, in step 110, the position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors based on the terminal connection image captured by the camera includes: and inputting the terminal connection image into the position identification model, and outputting position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors.
It is understood that the position recognition model may be a deep learning neural network, for example, a convolutional neural network, and the input of the position recognition model is the terminal connection image and the output is the position coordinates corresponding to the plurality of first connectors and the plurality of second connectors.
The position identification model is obtained by training a plurality of first plug connectors and a plurality of second plug connectors corresponding to the terminal connection image sample data as sample labels by using the terminal connection image sample data as a sample.
It can be understood that the position recognition model can be trained by connecting image sample data and position coordinate sample data corresponding to the plurality of first plug connectors and the plurality of second plug connectors marked in advance through a large number of terminals, and the training accuracy of the position recognition model can be ensured in a supervised learning mode.
The deep learning neural network used by the position recognition model can pick out the features in the input terminal connection image sample data, each feature is used for obtaining an output result, each output result is compared with a sample label, the features meeting the requirements after comparison can be reserved, the features meeting the requirements after comparison are ignored through the Loss parameter, the core features needing to be memorized can be learned finally through continuous iterative training of a large number of input terminal connection images, different core features are classified, and finally the newly input terminal connection images can be distinguished according to the core features.
Before the position recognition model is trained, the filter of the convolution layer of the deep learning neural network is completely random, the filter can not activate any feature, namely, can not detect any feature, in the training process, the blank filter is modified with weight to enable the blank filter to detect a specific scene, which is a supervised learning mode, and based on the supervised learning mode, the deep learning neural network can learn the core features by itself to judge the newly input terminal connection image according to the core features.
As shown in table 1, when the position recognition model is trained, the detection accuracy of the trained position recognition model increases as the data amount of the input terminal connection image sample data increases.
TABLE 1 correlation graph of detection accuracy of position recognition model and terminal connection image sample data
| Number of terminal connection image sample data | Detection accuracy of position recognition model |
| 200 | 98.0％ |
| 300 | 99.0％ |
| 1000 | 99.9％ |
Through experimental verification, this application embodiment is when actual detection terminal connection state, if basic data volume is big enough and background data is enough complicated, just can ignore different light sources and wait to detect the external factors irrelevant with core characteristic such as putting angle and the shadow condition of terminal, can detect the terminal of putting with arbitrary angle, compares artifical visual inspection commonality and accuracy and all has great promotion when detecting in complex environment and the environment that changes often.
In some embodiments, the location identification model is trained using a product quantization algorithm with Quant-Noise added and/or an Int8 quantization algorithm.
It can be understood that, when the position identification model is trained, a Product Quantizer (PQ) algorithm with a Quant-Noise added is adopted, and the Product Quantizer and the Int8 quantization algorithm can be combined to quantize the network weight and Activation, so that an extreme compression effect can be obtained while ensuring the accuracy of the position identification model.
Taking the real matrix W as an example, when the position recognition model is trained, the Quant-Noise operates as follows: a real matrix (fixed real matrix) W is preset, W ∈ R ^ n × p, and it is assumed that the real matrix W is split into m × q blocks (b _ kl): first, a block (b _ kl) related to a target quantization method is calculated; then in each forward propagation process, randomly selecting a subset of these blocks (b _ kl) and adding some distortion; in the back propagation process, the weights of the STE approximation distortion are used, and then the gradient values of all weights are calculated.
In other words, a given index tuple
Where k is 1. ltoreq. m, l is 1. ltoreq. q, while a distortion or noise function phi is applied to the blocks, an operator psi ("| J) is defined, applied to each block (b _ kl) to down-convert the formula:
the product quantization algorithm added with Quant-Noise and/or the Int8 quantization algorithm can shorten the judgment time of the position identification model by more than half, namely, on the image judgment of 416 × 416, the judgment time can be shortened from 0.2s to 50ms, so that the application scene of the terminal connection state detection is satisfied.
As shown in fig. 2, in some embodiments, the determining, in step 110, the position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors based on the terminal connection image captured by the camera includes: step 111-step 114 as follows.
And step 111, collecting a terminal connection image through a camera.
It can be understood that the lens of the camera is aligned with the joint of the first plug piece and the second plug piece of the terminal, and the terminal connection image can be collected through the camera.
And step 112, intercepting a target area image from the terminal connection image.
It can be understood that the target area image may be cut from the terminal connection image by a template matching method, the target area image includes all the first connectors and the corresponding second connectors of the terminal, and the target area image removes background content that is not related to the detection of the terminal connection state compared to the terminal connection image.
And 113, preprocessing the target area image to obtain a reference image.
It is understood that the target area image is preprocessed, and the preprocessing may include: and obtaining a reference image by at least one of denoising processing, matrixing processing, color processing, smoothing processing and image enhancement processing, and improving the identification degree of key features in the target region image and reducing the identification degree of irrelevant features by preprocessing.
And step 114, determining the corresponding position coordinates of the plurality of first plug connectors and the plurality of second plug connectors based on the reference image.
It is understood that, on the basis of the reference image, the method in the above embodiment may be adopted to obtain the position coordinates corresponding to the plurality of first connectors and the plurality of second connectors.
In some embodiments, the preprocessing the target area image in step 113 to obtain the reference image includes: and performing matrixing processing and/or color processing on the target area image to obtain a reference image.
It can be understood that the matrixing process is to analyze and process the target area image by adopting a matrix theory and a matrix algorithm, express the target area image in a matrix form, and can express the target area image by using a two-dimensional array, wherein a row of the two-dimensional array corresponds to the height of the target area image, a column of the two-dimensional array corresponds to the width of the target area image, an element of the two-dimensional array corresponds to a pixel of the target area image, a value of the element of the two-dimensional array is the gray level of the pixel, and the target area image is expressed by using the two-dimensional array, so that the method conforms to the row and column characteristics of the two-dimensional image, and can improve the efficiency of image recognition.
The color processing is to perform graying processing on the target area image, the target area image can be a color image, the color of each pixel in the color image is determined by R, G, B three components, and one component can be determined from 255 values, so that about 1600 ten thousand color change ranges exist in one pixel point, the color image can be changed into a grayscale image by performing the color processing on the target area image, the subsequent calculation amount can be reduced, the grayscale image and the color image both reflect the distribution and the characteristics of the whole and local chromaticity and brightness levels of the whole image, the accuracy of feature extraction cannot be influenced in an application scene of terminal embedding state detection, and the detection speed can be improved.
In some embodiments, the terminal connection image acquired by the camera in step 110 is: the method comprises the steps of collecting an original image by continuous curved surface polishing of a camera in a diffuse reflection environment, and filtering the original image based on a neighborhood clustering algorithm to obtain the target.
It can be understood that, in the process of target detection by using the non-contact visual detection technology, the interference of factors such as ambient light and strong reflected light on the surface of a detected object can be inevitable, so that noise points appear on the measured data, which seriously affects subsequent image processing processes such as point cloud segmentation, feature extraction, curved surface reconstruction and error analysis, and the noise points can cause burrs on the light ray edge, and small local bright spots can be generated outside the main point cloud, which can cause the light ray stripe pixel identification error, so that the point cloud noise of the image must be filtered.
The camera is placed in a diffuse reflection environment, continuous curved surface polishing is carried out, and noise points in an original image collected by the camera can be reduced.
The natural light pixel points can be classified by utilizing a neighborhood clustering algorithm to remove noise points in the original image, so that filtering is realized, and the accuracy of subsequent image identification can be improved.
In some embodiments, the step 120 of matching each first plug connector with the second plug connector closest to the first plug connector based on the position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors to obtain a matching result includes: and matching each first plug connector with the second plug connector closest to the first plug connector based on the Hungarian algorithm and the position coordinates corresponding to the first plug connectors and the second plug connectors to obtain a matching result.
The Hungarian algorithm is a combined optimization algorithm for solving the task allocation problem in the polynomial time, and as a mature matching algorithm, the Hungarian algorithm can realize one-to-one quick matching of the plurality of first plug connectors and the plurality of second plug connectors, and can quickly match each first plug connector with the second plug connector closest to the first plug connector to obtain a matching result.
The following describes the terminal connection state detection device provided in the present application, and the terminal connection state detection device described below and the terminal connection state detection method described above may be referred to in correspondence with each other.
As shown in fig. 3, the present application also provides a terminal connection state detection device including: a first determination module 310, a matching module 320, a second determination module 330, and a third determination module 340.
The first determining module 310 is configured to determine, based on the terminal connection image acquired by the camera, position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors.
The matching module 320 is configured to match each first plug connector with the second plug connector closest to the first plug connector based on the position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors, so as to obtain a matching result.
The second determining module 330 is configured to sum distances between the first plug connector and the second plug connector corresponding to the multiple matching results to obtain a target distance, and determine the terminal tabling state based on the target distance and a distance threshold.
A third determining module 340, configured to determine, based on the terminal connection image, the connection colors of the first plug connector and the second plug connector corresponding to each matching result if the terminal tabling state is tabling, and determine that the terminal connection state is normal if the connection colors of the first plug connector and the second plug connector corresponding to each matching result are both matched.
In some embodiments, the first determining module is further configured to input the terminal connection image to the position recognition model, and output position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors; the position identification model is obtained by training a plurality of first plug connectors and a plurality of second plug connectors corresponding to the terminal connection image sample data as sample labels by using the terminal connection image sample data as a sample.
In some embodiments, the location identification model is trained using a product quantization algorithm with Quant-Noise added and/or an Int8 quantization algorithm.
As shown in fig. 4, in some embodiments, the first determining module comprises: an acquisition sub-module 410, an interception sub-module 420, a pre-processing sub-module 430, and a location coordinate determination sub-module 440.
And the acquisition submodule 410 is used for acquiring the terminal connection image through the camera.
And a clipping sub-module 420 for clipping the target area image from the terminal connection image.
And the preprocessing submodule 430 is configured to preprocess the target area image to obtain a reference image.
And the position coordinate determining submodule 440 is configured to determine position coordinates corresponding to the plurality of first connectors and the plurality of second connectors based on the reference image.
In some embodiments, the preprocessing sub-module 430 is further configured to perform matrixing and/or color processing on the target region image to obtain a reference image.
In some embodiments, the terminal connection image captured by the camera is: the method comprises the steps of collecting an original image by continuous curved surface lighting of a camera in a diffuse reflection environment, and filtering the original image based on a neighborhood clustering algorithm to obtain the target.
In some embodiments, the matching module 320 is further configured to match each first plug connector with the second plug connector closest to the first plug connector based on a hungarian algorithm and the position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors, so as to obtain a matching result.
The terminal connection state detection device provided by the embodiment of the application is used for executing the terminal connection state detection method, and the implementation manner of the terminal connection state detection device is consistent with that of the terminal connection state detection method provided by the application, and the same beneficial effects can be achieved, and the description is omitted here.
Fig. 5 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 5: the processor (processor)510, the communication second Interface (communication Interface)520, the memory (memory)530 and the communication bus 540, wherein the processor 510, the communication second Interface 520 and the memory 530 complete mutual communication through the communication bus 540. Processor 510 may invoke logic instructions in memory 530 to perform a terminal connection state detection method comprising: determining position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors based on the terminal connection image acquired by the camera; matching each first plug connector with the second plug connector closest to the first plug connector based on the position coordinates corresponding to the first plug connectors and the second plug connectors to obtain a matching result; summing the distances between the first plug connector and the second plug connector corresponding to the matching results to obtain a target distance, and determining the terminal embedding state based on the target distance and a distance threshold; and if the terminal embedding state is embedded, determining the wiring colors of the first plug connector and the second plug connector corresponding to each matching result based on the terminal connection image, and if the wiring colors of the first plug connector and the second plug connector corresponding to each matching result are matched, determining that the terminal connection state is normal.
Furthermore, the logic instructions in the memory 530 may be implemented in the form of software functional units and stored in a computer readable storage medium when sold or used as a stand-alone product. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The processor 510 in the electronic device provided in the embodiment of the present application may call a logic instruction in the memory 530 to implement the terminal connection state detection method, and an implementation manner of the terminal connection state detection method is consistent with that of the terminal connection state detection method provided in the present application, and the same beneficial effects may be achieved, which is not described herein again.
On the other hand, the present application also provides a computer program product, which is described below, and the computer program product described below and the terminal connection state detection method described above can be referred to correspondingly.
The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions that, when executed by a computer, enable the computer to perform the terminal connection state detection method provided by the above-described methods, the method including: determining position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors based on a terminal connection image acquired by a camera; matching each first plug connector with the second plug connector closest to the first plug connector based on the position coordinates corresponding to the first plug connectors and the second plug connectors to obtain a matching result; summing the distances between the first plug connector and the second plug connector corresponding to the matching results to obtain a target distance, and determining the terminal embedding state based on the target distance and a distance threshold; and if the terminal embedding state is embedded, determining the wiring colors of the first plug connector and the second plug connector corresponding to each matching result based on the terminal connection image, and if the wiring colors of the first plug connector and the second plug connector corresponding to each matching result are matched, determining that the terminal connection state is normal.
When the computer program product provided by the embodiment of the present application is executed, the method for detecting a terminal connection state is implemented, and an implementation manner of the method is consistent with that of the method for detecting a terminal connection state provided by the present application, and the same beneficial effects can be achieved, and details are not repeated here.
In yet another aspect, the present application further provides a non-transitory computer-readable storage medium, which is described below, and the non-transitory computer-readable storage medium described below and the terminal connection state detection method described above may be referred to in correspondence with each other.
The present application also provides a non-transitory computer-readable storage medium having stored thereon a computer program that, when executed by a processor, is implemented to perform the terminal connection state detection methods provided above, the method including: determining position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors based on a terminal connection image acquired by a camera; matching each first plug connector with the second plug connector closest to the first plug connector based on the position coordinates corresponding to the first plug connectors and the second plug connectors to obtain a matching result; summing the distances between the first plug connector and the second plug connector corresponding to the matching results to obtain a target distance, and determining the terminal embedding state based on the target distance and a distance threshold; and if the terminal embedding state is embedded, determining the wiring colors of the first plug connector and the second plug connector corresponding to each matching result based on the terminal connection image, and if the wiring colors of the first plug connector and the second plug connector corresponding to each matching result are matched, determining that the terminal connection state is normal.
When the computer program stored on the non-transitory computer readable storage medium provided in the embodiment of the present application is executed, the method for detecting a terminal connection state is implemented, and an implementation manner of the method is consistent with that of the method for detecting a terminal connection state provided in the present application, and the same beneficial effects can be achieved, and details are not repeated here.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate components may or may not be physically separate, and components displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the embodiment. One of ordinary skill in the art can understand and implement the present invention without any inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may be modified or some technical features may be equivalently replaced; such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (12)
1. A terminal connection state detection method is characterized by comprising the following steps:
determining position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors based on a terminal connection image acquired by a camera;
matching each first plug connector with the second plug connector closest to the first plug connector based on the position coordinates corresponding to the first plug connectors and the second plug connectors to obtain a matching result;
summing the distances between the first plug connector and the second plug connector corresponding to the matching results to obtain a target distance, and determining a terminal embedding state based on the target distance and a distance threshold;
and if the terminal embedding state is embedded, determining the wiring colors of the first plug connector and the second plug connector corresponding to each matching result based on the terminal connection image, and if the wiring colors of the first plug connector and the second plug connector corresponding to each matching result are matched, determining that the terminal connection state is normal.
2. The method of claim 1, wherein determining the position coordinates corresponding to the plurality of first connectors and the plurality of second connectors based on the terminal connection image captured by the camera comprises:
inputting the terminal connection image into a position identification model, and outputting position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors;
the position identification model is obtained by training sample data of a terminal connection image as a sample, and position coordinate sample data corresponding to a plurality of first plug connectors and a plurality of second plug connectors corresponding to the terminal connection image sample data as sample labels.
3. The method for detecting a terminal connection state according to claim 2, wherein the position recognition model is trained by using a product quantization algorithm with Quant-Noise added and/or an Int8 quantization algorithm.
4. The method according to any one of claims 1 to 3, wherein determining the position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors based on the terminal connection image acquired by the camera includes:
acquiring the terminal connection image through a camera;
intercepting a target area image from the terminal connection image;
preprocessing the target area image to obtain a reference image;
and determining the position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors based on the reference image.
5. The method according to claim 4, wherein the preprocessing the target area image to obtain a reference image comprises:
and performing matrixing processing and/or color processing on the target area image to obtain a reference image.
6. A terminal connection state detection method according to any one of claims 1 to 3, wherein the terminal connection image acquired by the camera is: the method comprises the steps of collecting an original image by continuous curved surface polishing of a camera in a diffuse reflection environment, and filtering the original image based on a neighborhood clustering algorithm to obtain the target.
7. The method according to any one of claims 1 to 3, wherein the matching each of the first connectors with the second connector closest to the first connector based on the position coordinates corresponding to the first connectors and the second connectors to obtain a matching result includes:
and matching each first plug connector with the second plug connector which is closest to the first plug connector based on the Hungarian algorithm and the position coordinates corresponding to the first plug connectors and the second plug connectors to obtain a matching result.
8. A terminal connection state detection device characterized by comprising:
the first determining module is used for determining position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors based on the terminal connection image acquired by the camera;
the matching module is used for matching each first plug connector with the second plug connector closest to the first plug connector based on the position coordinates corresponding to the first plug connectors and the second plug connectors to obtain a matching result;
the second determining module is used for summing the distances between the first plug connector and the second plug connector corresponding to the matching results to obtain a target distance, and determining the terminal embedding state based on the target distance and a distance threshold;
and the third determining module is used for determining the wiring colors of the first plug connector and the second plug connector corresponding to each matching result based on the terminal connection image if the terminal embedding state is embedded, and confirming that the terminal connection state is normal if the wiring colors of the first plug connector and the second plug connector corresponding to each matching result are matched.
9. The apparatus of claim 8, wherein the first determining module is further configured to input the terminal connection image into a position recognition model, and output position coordinates corresponding to the plurality of first connectors and the plurality of second connectors;
the position identification model is obtained by training sample data of a terminal connection image as a sample, and position coordinate sample data corresponding to a plurality of first plug connectors and a plurality of second plug connectors corresponding to the terminal connection image sample data as sample labels.
10. The terminal connection state detection device according to claim 8, wherein the first determination module includes:
the acquisition submodule is used for acquiring the terminal connection image through a camera;
the intercepting submodule is used for intercepting a target area image from the terminal connection image;
the preprocessing submodule is used for preprocessing the target area image to obtain a reference image;
and the position coordinate determination submodule is used for determining the position coordinates corresponding to the plurality of first plug connectors and the plurality of second plug connectors based on the reference image.
11. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the terminal connection state detection method according to any one of claims 1 to 7 are implemented when the processor executes the program.
12. A non-transitory computer-readable storage medium on which a computer program is stored, the computer program realizing the steps of the terminal connection state detection method according to any one of claims 1 to 7 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011552665.5A CN112784853B (en) | 2020-12-24 | 2020-12-24 | Terminal connection state detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011552665.5A CN112784853B (en) | 2020-12-24 | 2020-12-24 | Terminal connection state detection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112784853A true CN112784853A (en) | 2021-05-11 |
| CN112784853B CN112784853B (en) | 2022-05-03 |
Family
ID=75752297
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011552665.5A Active CN112784853B (en) | 2020-12-24 | 2020-12-24 | Terminal connection state detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112784853B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113706506A (en) * | 2021-08-27 | 2021-11-26 | 创新奇智(重庆)科技有限公司 | Method and device for detecting assembly state, electronic equipment and storage medium |
| CN114979184A (en) * | 2022-04-25 | 2022-08-30 | 佛山智优人科技有限公司 | Method and system for online automatic inspection of electrical manual wiring quality |
| CN117213830A (en) * | 2023-11-08 | 2023-12-12 | 深圳市凯比特微电子有限公司 | Connector state monitoring system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105264724A (en) * | 2013-05-10 | 2016-01-20 | 矢崎总业株式会社 | Terminal insertion device and terminal insertion method |
| CN207459252U (en) * | 2017-11-15 | 2018-06-05 | 东莞市正立机电有限公司 | A kind of signal terminal connection plug |
| CN210404220U (en) * | 2019-11-06 | 2020-04-24 | 苏州思科赛德电子科技有限公司 | Guide mechanism for terminal insertion |
| CN112924901A (en) * | 2021-01-28 | 2021-06-08 | 浩亭(珠海)制造有限公司 | Terminal wiring induction system and working method thereof |
| CN113540860A (en) * | 2021-07-06 | 2021-10-22 | 国网江苏省电力有限公司淮安供电分公司 | Wiring terminal capable of automatically switching on and off and displaying switching-on and switching-off states |
-
2020
- 2020-12-24 CN CN202011552665.5A patent/CN112784853B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105264724A (en) * | 2013-05-10 | 2016-01-20 | 矢崎总业株式会社 | Terminal insertion device and terminal insertion method |
| CN207459252U (en) * | 2017-11-15 | 2018-06-05 | 东莞市正立机电有限公司 | A kind of signal terminal connection plug |
| CN210404220U (en) * | 2019-11-06 | 2020-04-24 | 苏州思科赛德电子科技有限公司 | Guide mechanism for terminal insertion |
| CN112924901A (en) * | 2021-01-28 | 2021-06-08 | 浩亭(珠海)制造有限公司 | Terminal wiring induction system and working method thereof |
| CN113540860A (en) * | 2021-07-06 | 2021-10-22 | 国网江苏省电力有限公司淮安供电分公司 | Wiring terminal capable of automatically switching on and off and displaying switching-on and switching-off states |
Non-Patent Citations (2)
| Title |
|---|
| GE QIANG 等: "Detection method of PCB component based on automatic optical stitching algorithm", 《CIRCUIT WORLD》 * |
| 王雨青等: "线束端子角度姿态的视觉检测方法", 《国外电子测量技术》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113706506A (en) * | 2021-08-27 | 2021-11-26 | 创新奇智(重庆)科技有限公司 | Method and device for detecting assembly state, electronic equipment and storage medium |
| CN114979184A (en) * | 2022-04-25 | 2022-08-30 | 佛山智优人科技有限公司 | Method and system for online automatic inspection of electrical manual wiring quality |
| CN117213830A (en) * | 2023-11-08 | 2023-12-12 | 深圳市凯比特微电子有限公司 | Connector state monitoring system |
| CN117213830B (en) * | 2023-11-08 | 2024-03-26 | 深圳市凯比特微电子有限公司 | Connector state monitoring system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112784853B (en) | 2022-05-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112784853B (en) | Terminal connection state detection method and device | |
| CN110148130B (en) | Method and device for detecting part defects | |
| CN111833306B (en) | Defect detection method and model training method for defect detection | |
| CN109116129B (en) | Terminal detection method, detection device, system and storage medium | |
| CN110008917B (en) | Fault detection method of fixed value single data of relay protection device based on table understanding | |
| CN111259908A (en) | Machine vision-based steel coil number identification method, system, equipment and storage medium | |
| CN112989995B (en) | Text detection method and device and electronic equipment | |
| CN114067051A (en) | Three-dimensional reconstruction processing method, device, electronic device and storage medium | |
| CN111225299A (en) | ONU (optical network Unit) fault identification and repair method and device | |
| CN115984662A (en) | Multi-mode data pre-training and recognition method, device, equipment and medium | |
| CN114298985B (en) | Defect detection method, device, equipment and storage medium | |
| CN111553176A (en) | Wireless transmission checking method and system suitable for wiring of transformer substation cubicle | |
| CN113705361A (en) | Method and device for detecting model in living body and electronic equipment | |
| CN114626118A (en) | Building indoor model generation method and device | |
| CN109919164B (en) | User interface object identification method and device | |
| CN117095275A (en) | Asset inventory method, system, device and storage medium for data center | |
| CN113888635B (en) | Visual positioning method and related device | |
| CN108629219B (en) | Method and device for identifying one-dimensional code | |
| CN115619796A (en) | Method and device for obtaining photovoltaic module template and nonvolatile storage medium | |
| CN115619725A (en) | Electronic component detection method and device, electronic equipment and automatic quality inspection equipment | |
| CN114782822A (en) | Method and device for detecting state of power equipment, electronic equipment and storage medium | |
| CN115374517A (en) | Testing method and device for wiring software, electronic equipment and storage medium | |
| CN115019057A (en) | Image feature extraction model determining method and device and image identification method and device | |
| CN108447107B (en) | Method and apparatus for generating video | |
| CN112967224A (en) | Electronic circuit board detection system, method and medium based on artificial intelligence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230308 Address after: 201306 building C, No. 888, Huanhu West 2nd Road, Lingang New District, China (Shanghai) pilot Free Trade Zone, Pudong New Area, Shanghai Patentee after: Shenlan Intelligent Technology (Shanghai) Co.,Ltd. Address before: 213000 No.103, building 4, Chuangyan port, Changzhou science and Education City, No.18, middle Changwu Road, Wujin District, Changzhou City, Jiangsu Province Patentee before: SHENLAN ARTIFICIAL INTELLIGENCE CHIP RESEARCH INSTITUTE (JIANGSU) Co.,Ltd. |
|
| TR01 | Transfer of patent right |