CN111141211B - Deep learning-based capacitance positioning detection method, server and detection system - Google Patents
Deep learning-based capacitance positioning detection method, server and detection system Download PDFInfo
- Publication number
- CN111141211B CN111141211B CN201911353811.9A CN201911353811A CN111141211B CN 111141211 B CN111141211 B CN 111141211B CN 201911353811 A CN201911353811 A CN 201911353811A CN 111141211 B CN111141211 B CN 111141211B
- Authority
- CN
- China
- Prior art keywords
- capacitance
- capacitor
- point
- deep learning
- key
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a capacitance positioning detection method based on deep learning, which is applied to a detection system and comprises the following steps: inputting the capacitance image into a deep learning network model to identify a plurality of capacitance key points, wherein the capacitance key points comprise a capacitance main body angular point, a capacitance left pin point and a capacitance right pin point; dividing the capacitance key points into a plurality of key point sets, and acquiring a capacitance area corresponding to each key point set; and carrying out capacitance positioning detection according to the capacitance area. The invention also discloses a server and a detection system. The invention provides an innovative method by identifying the key points of the capacitor by using a deep learning algorithm and dividing the key points of the capacitor into corresponding capacitor areas, and can obtain accurate estimation on the position information of the capacitor, thereby realizing accurate positioning on the capacitor based on the position information of the capacitor.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a capacitance positioning detection method, a server and a detection system based on deep learning.
Background
With the recent development and popularity of the manipulator arm, it has gradually begun to replace the traditional human labor force, and has been increasingly widely used in the industrial field in an accurate, safe, and low-cost manner. The system carries out key area detection and positioning on the capacitor through a capacitor key point positioning algorithm operated on embedded computing equipment, and then can grab the capacitor.
Whether the capacitor can be accurately captured or not is achieved, and the key area positioning detection of the capacitor is very key work. In the traditional mode, more manual assistance is used for grabbing the capacitor or a depth camera is used as input information to position key points of the capacitor, so that the cost is high, and large-scale deployment and production efficiency improvement are not facilitated.
The above is only for the purpose of assisting understanding of the technical aspects of the present invention, and does not represent an admission that the above is prior art.
Disclosure of Invention
The invention mainly aims to provide a capacitance positioning detection method, a server and a detection system based on deep learning, and aims to solve the technical problem of low capacitance capturing efficiency in the prior art.
In order to achieve the above object, the present invention provides a capacitance positioning detection method based on deep learning, which includes the following steps:
inputting the capacitance image into a deep learning network model to identify a plurality of capacitance key points, wherein the capacitance key points comprise a capacitance main body angular point, a capacitance left pin point and a capacitance right pin point;
dividing the capacitance key points into a plurality of key point sets, and acquiring a capacitance area corresponding to each key point set;
and carrying out capacitance positioning detection according to the capacitance area.
Optionally, the step of dividing the capacitance keypoints into a plurality of keypoint sets includes:
calculating Euclidean distances among the key points of the capacitors;
and dividing the capacitance key points into a plurality of key point sets according to the Euclidean distance.
Optionally, the capacitor body corner points include four body corner points, the step of calculating euclidean distances between the capacitor key points, and dividing the capacitor key points into a plurality of key point sets according to the euclidean distances includes:
mapping the four main body angular points, the capacitor left pin point and the capacitor right pin point to six types of key points;
dividing the first class of key points into a plurality of key point sets;
for each key point to be allocated in the second category key points to the sixth category key points, sequentially allocating the second category key points to the sixth category key points to the key point set according to the following steps:
and traversing and calculating the Euclidean distance between the key point to be distributed and the key point set, and adding the key point to be distributed into a target key point set, wherein the Euclidean distance between the key point to be distributed and the target key point set falls into a preset distance range.
Optionally, the step of inputting the capacitance image into the deep learning network model to identify a plurality of capacitance key points further comprises:
inputting the capacitance training image data into the deep learning network model to obtain the predicted Gaussian probability of the capacitance key point;
acquiring a difference value between the prediction Gaussian probability and the Gaussian probability of the labeled capacitance key point, wherein the Gaussian probability of the labeled capacitance key point is obtained by processing according to the capacitance training image data;
and adjusting parameters of the deep learning network model according to the difference value.
Optionally, the step of inputting the capacitance training image data into the deep learning network model to obtain the predicted gaussian probability of the capacitance key point includes:
taking the position of the key point of the capacitor as the peak position of Gaussian distribution;
and obtaining the predicted Gaussian probability of the capacitance key point according to the peak position.
Optionally, the step of adjusting the parameter of the deep learning network according to the difference further includes:
identifying a central point of a capacitor main body in the capacitor training image, and obtaining a prediction Gaussian probability of the central point of the capacitor main body;
acquiring a difference value between the predicted Gaussian probability of the center point of the capacitor main body and the labeled Gaussian probability of the center point of the capacitor main body, wherein the labeled Gaussian probability of the center point of the capacitor main body is obtained by processing according to the capacitor training image data;
and adjusting parameters of the deep learning network model according to the difference value.
Further, to achieve the above object, the present invention also provides a server comprising: the system comprises a memory, a processor and a deep learning based capacitive positioning detection processing program stored on the memory and capable of running on the processor, wherein the deep learning based capacitive positioning detection processing program realizes the steps of the deep learning based capacitive positioning detection method when being executed by the processor.
In addition, in order to achieve the above object, the present invention further provides a positioning detection system, where the positioning detection system includes an embedded device, a camera connected to the embedded device, and a manipulator, where the camera is used to take a capacitor picture, the embedded device is used to identify a capacitor position in the capacitor picture, and the manipulator captures a capacitor according to the capacitor position.
In addition, in order to achieve the above object, the present invention further provides a capacitive positioning detection method, which is applied to the positioning detection system, and the capacitive positioning detection method includes the following steps:
controlling a camera to shoot a capacitance picture in a preset area and sending the capacitance picture;
the embedded equipment receives the capacitance picture and identifies the position information of the capacitor in the capacitance picture;
the embedded equipment sends the position information of the capacitor to a manipulator;
and the manipulator captures the capacitor according to the position information of the capacitor.
In addition, to achieve the above object, the present invention further provides a medium having stored thereon a deep learning based capacitive positioning detection processing program, which when executed by a processor implements the steps of the deep learning based capacitive positioning detection method as described above.
According to the capacitance positioning detection method based on deep learning, the server and the readable computer storage medium, provided by the embodiment of the invention, the key points of the capacitance are identified by using a deep learning algorithm and are divided into corresponding capacitance areas, so that an innovative method is provided, accurate estimation on capacitance position information can be obtained, and accurate positioning on the capacitance is realized based on the capacitance position information.
Drawings
Fig. 1 is a schematic diagram of a server structure according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart illustrating a deep learning-based capacitive positioning detection method according to a first embodiment of the present invention;
FIG. 3 is a schematic diagram of a capacitance key point in a first embodiment of the deep learning-based capacitance positioning detection method of the present invention;
FIG. 4 is a flow chart illustrating the step S20 of the deep learning-based capacitive localization detection method according to the first embodiment of the present invention;
FIG. 5 is a schematic diagram of a reference distance in a first embodiment of a deep learning-based capacitive positioning detection method according to the present invention;
FIG. 6 is a schematic diagram illustrating a comparison between Euclidean distance and a reference distance in a first embodiment of a deep learning-based capacitive positioning detection method according to the present invention;
FIG. 7 is a flowchart illustrating a deep learning-based capacitive positioning detection method according to a second embodiment of the present invention;
FIG. 8 is a Gaussian probability distribution diagram in a second embodiment of the deep learning-based capacitance localization detection method of the present invention;
FIG. 9 is a schematic flowchart of a third embodiment of a deep learning-based capacitive positioning detection method according to the present invention;
FIG. 10 is a schematic view of the detection system of the present invention;
FIG. 11 is a flowchart illustrating a capacitive positioning detection method according to the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The main solution of the embodiment of the invention is as follows: inputting the capacitance image into a deep learning network model to identify a plurality of capacitance key points, wherein the capacitance key points comprise a capacitance main body angular point, a capacitance left pin point and a capacitance right pin point; dividing the capacitance key points into a plurality of key point sets, and acquiring a capacitance area corresponding to each key point set; and carrying out capacitance positioning detection according to the capacitance area.
The invention provides an innovative method by identifying the key points of the capacitor by using a deep learning algorithm and dividing the key points of the capacitor into corresponding capacitor areas, and can obtain accurate estimation on the position information of the capacitor, thereby realizing accurate positioning on the capacitor based on the position information of the capacitor.
As shown in fig. 1, fig. 1 is a schematic structural diagram of a server according to an embodiment of the present invention.
As shown in fig. 1, the server may include: a processor 1001, such as a CPU, a communication bus 1002, and a memory 1003. Wherein a communication bus 1002 is used to enable connective communication between these components. The memory 1003 may be a high-speed RAM memory or a non-volatile memory (e.g., a disk memory). The memory 1003 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the configuration of the server shown in FIG. 1 is not intended to be limiting, and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, the memory 1003, which is a kind of computer storage medium, may include therein an operating system and a capacitance key point location detection processing program based on deep learning.
In the apparatus shown in fig. 1, the processor 1001 may be configured to call a deep learning based capacitive keypoint location detection processing program stored in the memory 1003, and perform the following operations:
inputting the capacitance image into a deep learning network model to identify a plurality of capacitance key points, wherein the capacitance key points comprise a capacitance main body angular point, a capacitance left pin point and a capacitance right pin point;
dividing the capacitance key points into a plurality of key point sets, and acquiring a capacitance area corresponding to each key point set;
and carrying out capacitance positioning detection according to the capacitance area.
Further, the processor 1001 may call the deep learning based capacitance key point location detection processing program stored in the memory 1003, and further perform the following operations:
calculating Euclidean distances among the key points of the capacitors;
and dividing the capacitance key points into a plurality of key point sets according to the Euclidean distance.
Further, the processor 1001 may call the deep learning based capacitance key point location detection processing program stored in the memory 1003, and further perform the following operations:
mapping the four main body angular points, the capacitor left pin point and the capacitor right pin point to six types of key points;
dividing the first class of key points into a plurality of key point sets;
for each key point to be allocated in the second category key points to the sixth category key points, sequentially allocating the second category key points to the sixth category key points to the key point set according to the following steps:
and traversing and calculating the Euclidean distance between the key point to be distributed and the key point set, and adding the key point to be distributed into a target key point set, wherein the Euclidean distance between the key point to be distributed and the target key point set falls into a preset distance range.
Further, the processor 1001 may call the deep learning based capacitance key point location detection processing program stored in the memory 1003, and further perform the following operations:
inputting the capacitance training image data into the deep learning network model to obtain the predicted Gaussian probability of the capacitance key point;
acquiring a difference value between the prediction Gaussian probability and the Gaussian probability of the labeled capacitance key point, wherein the Gaussian probability of the labeled capacitance key point is obtained by processing according to the capacitance training image data;
and adjusting parameters of the deep learning network model according to the difference value.
Further, the processor 1001 may call the deep learning based capacitance key point location detection processing program stored in the memory 1003, and further perform the following operations:
taking the position of the key point of the capacitor as the peak position of Gaussian distribution;
and obtaining the predicted Gaussian probability of the capacitance key point according to the peak position.
Further, the processor 1001 may call the deep learning based capacitance key point location detection processing program stored in the memory 1003, and further perform the following operations:
identifying a central point of a capacitor main body in the capacitor training image, and obtaining a prediction Gaussian probability of the central point of the capacitor main body;
acquiring a difference value between the predicted Gaussian probability of the center point of the capacitor main body and the labeled Gaussian probability of the center point of the capacitor main body, wherein the labeled Gaussian probability of the center point of the capacitor main body is obtained by processing according to the capacitor training image data;
and adjusting parameters of the deep learning network model according to the difference value.
Referring to fig. 2, a first embodiment of the present invention provides a deep learning-based capacitive localization detection method, where the method includes:
step S10, inputting the capacitance image into a deep learning network model to identify a plurality of capacitance key points, wherein the capacitance key points comprise a capacitance main body corner point, a capacitance left pin point and a capacitance right pin point;
as shown in fig. 3, a total of 6 key points are defined for each capacitance: 4 main body corner points (1-4 in fig. 3), 1 left pin (5 in fig. 3) and 1 right pin (6 in fig. 3).
Step S20, dividing the capacitance key points into a plurality of key point sets, and acquiring capacitance areas corresponding to the key point sets;
because a plurality of capacitors are often included in one capacitor picture, a plurality of types of capacitor key points can be identified from the capacitor picture, namely, the capacitor picture includes a plurality of main body corner points, a plurality of left pin points and a plurality of right pin points. In order to finally obtain the positioning information of each capacitor in the capacitor picture, the multi-type capacitor key points need to be subjected to key point set division, that is, which four main body corner points, which left pin and which right pin belong to one capacitor are identified, and the four main body corner points, the left pin and the right pin which belong to the same capacitor are divided into one key point set.
In the embodiment, the identified capacitance key points are divided into a plurality of key point sets according to the euclidean distances by calculating the euclidean distances between the capacitance key points. As shown in fig. 4, the specific steps include:
step S201, mapping the four main body angular points, the capacitor left pin point and the capacitor right pin point to six types of key points;
the capacitor is generally a rectangular body, and the main angular points of the capacitor are four angular points of the rectangular body. In this step, the mapping sequence is not limited, that is, the four corner points may be sequentially mapped to a first class key point, a second class key point, a third class key point to a fourth class key point, the left pin point is mapped to a fifth class key point, and the right pin point is mapped to a sixth class key point, or the left pin point and the right pin point may be first mapped to the first class key point and the second class key point, and then the four corner points are sequentially mapped to the third class key point to the sixth class key point. It should be noted that the order of mapping determines the order of calculating euclidean distances between keypoints in the candidate step.
Step S202, dividing the first-class key points into a plurality of key point sets;
as an initial step of the key point set division, each key point in the first type of key points is firstly used as a key point set, that is, each key point set only includes one first type of key point at this time.
Step S203, for each to-be-allocated keypoint of the second to sixth types of keypoints, sequentially allocating the second to sixth types of keypoints to the keypoint set according to the following steps:
and for each key point to be distributed, traversing and calculating the Euclidean distance between the key point to be distributed and the key point set, and adding the key point to be distributed into a target key point set, wherein the Euclidean distance between the key point to be distributed and the target key point set falls into a preset distance range.
After an initial key point set is determined according to the first class of key points, the second class of key points, the third class of key points, the fourth class of key points, the fifth class of key points and the sixth class of key points need to be added to the corresponding key point sets in sequence, that is, each key point set finally contains six different types of key points, and the six key points belong to the same capacitor.
The adding process of the key point set is described by taking the case that four angular points (an upper left angular point, a lower left angular point, an upper right angular point and a lower right angular point) are sequentially mapped as first-class to fourth-class key points, and a left pin point and a right pin point are sequentially mapped as a fifth-class key point and a sixth-class key point:
1. assuming that the number of each type of key points is N, dividing the first type of key points, namely the upper left corner points, into N key point sets, and traversing and calculating the Euclidean distance between each second type of key points, namely the lower left corner points and each upper left corner point, so as to obtain N x N Euclidean distances in total. That is, for each key point set including only one top left corner point, N euclidean distances d12 between the key point set and the bottom left corner point may be calculated, d12 is compared with the preset distance range Len12, and the bottom left corner point associated with d12 within the preset distance range Len12 is added to the key point set.
The preset distance range Len12 is calculated by the following method: since the volume of the capacitor is known in advance, a distance reference value between six key points belonging to one capacitor can be obtained, for example, as shown in fig. 5, the distance reference value between the upper left corner point and the lower left corner point is r12, the distance reference value between the upper left corner point and the upper right corner point is r13, the distance reference value between the upper left corner point and the lower right corner point is r14, the distance reference value between the upper left corner point and the left pin point is r15, the distance reference value between the upper left corner point and the right pin point is r16, the distance reference value between the lower left corner point and the lower right corner point is r24, the distance reference value between the lower left corner point and the upper right corner point is r23, the distance reference value between the upper right corner point and the lower right corner point is r34, the distance reference value between the upper right corner point and the left pin point is r35, the distance reference value between the upper right corner point and the right pin point is r36, the distance reference value between the left pin point and the right pin point is r56, the distance between the left pin point and the upper left corner point is referred to as r 15. In consideration of measurement and estimation errors, the value of Len12 representing the distance range between the upper left corner point and the lower left corner point can be set to [ r12-a, r12+ a ], wherein a is an allowable error value, calculated based on theory and obtained according to simulation.
Fig. 6 is a simple example, and there are two capacitors in fig. 6, and two d12 can be calculated, and by comparing two d12 with the preset distance range r12, it is obvious that the right d12 in the figure is not within the preset distance range r12, and the left d12 is within the preset distance range.
2. At this time, there are two types of keypoints in each keypoint set, namely, an upper left corner point and a lower left corner point. In order to determine which capacitance each third-class key point belongs to, that is, which key point set belongs to, it is necessary to calculate the euclidean distances between each third-class key point and all key points in each key point set, that is, for each third-class key point, that is, the upper-right corner point, the euclidean distances d13 between the N upper-right corner points and the upper-left corner point and the euclidean distances d23 between the N upper-right corner points and the lower-left corner points are obtained. Similarly, based on r13 and r23, which can be obtained from the volumes of the capacitors known in advance, Len13, which represents the distance range between the upper right corner point and the upper left corner point, is [ r13-A, r13+ A ], and Len23, which represents the distance range between the upper right corner point and the lower left corner point, is [ r23-A, r23+ A ]. For each top-right corner point, its calculated N groups d13, d23 are compared to Len13, Len23, respectively, and the top-right corner points with d13 in the Len13 range and d23 in the Len23 range are added to the corresponding set of keypoints.
3. At this time, there are three types of key points in each key point set, namely, an upper left corner point, a lower left corner point, and an upper right corner point. And for the remaining fourth-class key points (lower right corner points), fifth-class key points (left pin points) and sixth-class key points (right pin points), adding one of the remaining fourth-class key points (lower right corner points), the remaining fifth-class key points (left pin points) and the remaining sixth-class key points (right pin points) to the corresponding key point set by adopting a method similar to the method in the step 2. When adding the lower right corner point, the distance range for comparison includes: len14 representing the distance range between the lower right corner point and the upper left corner point is [ r14-A, r14+ A ], Len24 representing the distance range between the lower right corner point and the lower left corner point is [ r24-A, r24+ A ], and Len34 representing the distance range between the lower right corner point and the upper right corner point is [ r34-A, r34+ A ]; when adding the left pin point, the distance ranges for comparison include: len15 representing the distance range between the left pin point and the upper left corner point is [ r15-A, r15+ A ], Len35 representing the distance range between the left pin point and the upper right corner point is [ r35-A, r35+ A ]; when adding the right pin point, the distance ranges for comparison include: len16 representing the distance range between the right pin point and the upper left corner point is [ r16-A, r16+ A ], Len36 representing the distance range between the right pin point and the upper right corner point is [ r36-A, r36+ A ], and Len56 representing the distance range between the right pin point and the left pin point is [ r56-A, r56+ A ].
It should be noted that the above steps are only an example of adding the set of key points, and the number and specific numerical values of the distance ranges used for comparison in the adding process are not limited, that is, changes to the comparison order and adjustments to the distance ranges are all equivalent schemes and are all within the protection scope of the present patent.
Each set of capacitance keypoints obtained in this step corresponds to a capacitance region, and each capacitance region corresponds to a capacitance.
And step S30, performing capacitance positioning detection according to the capacitance area.
When the deep learning network model identifies the capacitance key points from the capacitance picture, the coordinate information of the pixels where the capacitance key points are located is obtained, each capacitance key point is further divided into corresponding key point sets one by one, namely a group of capacitance key points belonging to the same capacitance are found, at the moment, the coordinate information of the pixels in the area where the corresponding capacitance is located can be obtained according to the coordinate information of the pixels where each group of capacitance key points are located, the coordinate information is detected to be complete capacitance coordinate information, and then the position of the capacitance can be accurately positioned according to the complete capacitance coordinate information.
In the embodiment, the capacitance key points are identified by using a deep learning algorithm and are divided into corresponding capacitance areas, so that an innovative method is provided, accurate estimation of capacitance position information can be obtained, and accurate positioning of the capacitance is realized based on the capacitance position information.
Further, referring to fig. 7, a second embodiment of the present invention provides a method for detecting capacitive location based on deep learning based on the first embodiment, where the present embodiment further includes, before step S10:
step S40, inputting capacitance training image data into the deep learning network model to obtain the prediction Gaussian probability of the capacitance key point;
the capacitance training image data required by the deep learning network model construction for capacitance positioning detection is collected under natural light, and the number of the collected capacitors in each picture can reach hundreds. In order to detect more accurately, ten thousand pictures are recommended to be collected, so that the accuracy of the model can be improved, and the generalization capability of the model is improved. In the capacitive picture, each capacitor takes 4 different poses, namely: pin up, pin down, and pin down.
Because the shape of the whole capacitor can be seen for the capacitor in the lying posture, the capacitor is more intuitive and simple. Therefore, the method mainly aims at recognizing and detecting the capacitance in the lying posture, and obtains an accurate capacitance key point positioning detection model by constructing a deep learning network model.
Based on the collected capacitance training image, firstly, the data collected in the pictures (1) and (c) are loaded by using a data labeling tool, and secondly, 6 key points (namely 4 main body angular points, 1 left pin and 1 right pin) on the pictures are artificially labeled by using a mouse. The 6 key points will automatically generate the corresponding coordinate value (x, y), and the other key point, namely the center point of the main body, will automatically calculate and generate the coordinate value through the positions of the other key points.
In this step, the capacitance training image data is input into the deep learning network model, and the positions of the capacitances in the capacitance training image corresponding to the multiple types of key points can be predicted, wherein the capacitance key points include four main body corner points of the capacitances, a left pin point of the capacitances and a right pin point of the capacitances.
For each type of capacitance key point, the position of the capacitance key point can be used as the peak position of Gaussian distribution, and the Gaussian probability of the capacitance key point is obtained according to the peak position. Specifically, the coordinate value of the position of the capacitance key point corresponds to the maximum probability value of the gaussian distribution, the position coordinate of the capacitance key point is used as the center of a circle to extend outwards, and the probability value of the gaussian distribution corresponding to the coordinate value of the position in the extending direction continuously decreases until approaching to two edge probability values of the gaussian distribution. As shown in fig. 8, the darker the peak position representing the gaussian distribution, the lighter the color approaches to both edges of the gaussian distribution.
Step S50, obtaining a difference value between the prediction Gaussian probability and the Gaussian probability of the labeled capacitance key point, wherein the labeled capacitance key point Gaussian probability is obtained by processing according to the capacitance training image data;
in this step, a cost function is constructed according to the predicted gaussian probability and the gaussian probability of the labeled capacitance key point, as shown in the following formula:
wherein, FtFor the cost function of the training stage t, p represents the key point of the capacitance of the p-th class, Z represents the pixel coordinate on the capacitance image, Z represents the whole image plane,
representing the probability of the p-th class capacitance key point in the t training stage at the image coordinate z, which is a Gaussian probability value predicted by the deep learning model,
and representing a labeled Gaussian probability value of the p-th type capacitance key point in the t training stage at the image coordinate z, wherein the labeled Gaussian probability value is obtained by processing according to capacitance training image data, specifically, the position of the capacitance key point can be used as the peak position of Gaussian distribution, and the predicted Gaussian probability of the capacitance key point is obtained according to the peak position.
It should be noted that there may be multiple training stages, that is, the value of t may be greater than or equal to 1.
And step S60, adjusting the parameters of the deep learning network model according to the difference.
In this embodiment, the position of the key point of the capacitor is used as the peak position of gaussian distribution, and a cost function is constructed according to the peak position, so that the optimal parameter of the deep learning network model can be obtained in the training process, and the deep learning network model is used for accurately and efficiently positioning the capacitor from the capacitor image.
Further, referring to fig. 9, a third embodiment of the present invention provides a method for detecting capacitive location based on deep learning based on the second embodiment, where the present embodiment further includes, after step S60:
step S70, recognizing a capacitance main body central point in the capacitance training image, and obtaining a prediction Gaussian probability of the capacitance main body central point;
in this step, the capacitance training image is input into the trained deep learning network model, and in addition to six types of capacitance key points including four main body corner points, a left pin point and a right pin point, a main body center point of the capacitance is additionally identified. Similarly, the position of the center point of the capacitor body is taken as the peak position of the gaussian distribution, and the gaussian probability of the center point of the capacitor body, that is, the predicted gaussian probability of the center point of the capacitor body, is obtained according to the peak position.
Step S80, obtaining a difference value between the predicted Gaussian probability of the center point of the capacitor main body and the Gaussian probability of the center point of the labeled capacitor main body, wherein the Gaussian probability of the center point of the labeled capacitor main body is obtained by processing according to the capacitor training image data;
after the predicted gaussian probability of the center point of the capacitor body is obtained, a difference between the predicted gaussian probability and the gaussian probability of the labeled center point of the capacitor can be calculated.
And step S90, adjusting the parameters of the deep learning network model according to the difference.
And after the difference value between the prediction Gaussian probability obtained in the last step and the Gaussian probability of the marked capacitance center point is obtained, a cost function of the deep learning network model can be constructed according to the difference value, and then the parameters of the deep learning network model are adjusted by using a back propagation algorithm.
In this embodiment, by adding prediction on the central point of the capacitor main body in the training process, the accuracy of recognizing the key point of the capacitor by the deep learning network model can be further improved.
The invention also provides a server, which comprises a memory, a processor and a deep learning-based capacitance positioning detection processing program which is stored on the memory and can run on the processor, wherein the deep learning-based capacitance positioning detection processing program realizes the steps of the deep learning-based capacitance positioning detection method when being executed by the processor.
As shown in fig. 10, the present invention further provides a detection system, where the detection system includes an embedded device 20, a camera 10 connected to the embedded device, and a manipulator 30, where the camera 10 is configured to take a capacitor picture, the embedded device 20 is configured to identify a capacitor position in the capacitor picture, and the manipulator 30 captures a capacitor according to the capacitor position. The embedded equipment, the camera and the manipulator are connected in a wired or wireless mode for network communication.
When the embedded equipment adopted by the detection system has sufficient computing resources, the deep learning network model can be directly trained in the embedded equipment. When the computing resources of the embedded device adopted by the detection system are limited, for example, the adopted embedded device is Nvidia Jetson TK1, Jetson TK1 is an embedded Linux development platform of Nvidia, a Tegra K1 SOC (single chip CPU + GPU + ISP) is adopted, the memory (2GB) and the computing resources of the device are very limited, the deep learning network model in the detection system constructed by the present invention is trained on the server, and the capacitance positioning detection processing software is deployed on the embedded device Jetson TK 1.
Further, when the memory resource of the embedded device is limited and the whole detection model cannot be loaded, if the trained detection model is to be transplanted on the embedded device, the limited memory resource and the computing resource make the transplanting work face a huge challenge (such as the inability to process the original high-resolution image). To address the above challenges, measures may be taken to reduce the memory requirements of the embedded device for testing and applications. For example, SVD (singular value decomposition) is adopted to compress parameters of a full connection layer, and the method can also reduce the memory requirement of the detection model on the embedded equipment; or downsize the input image for model tuning and testing.
It should be noted that, in order to train better parameters, a multi-stage training mode may be adopted during deep learning network model training, and at this time, consumption of video memory and computing resources is large, but since the training task is executed on the server, the requirement of resources can be completely met.
The present invention further provides a capacitance positioning detection method, which is applied to the detection system described above, as shown in fig. 11, the capacitance positioning detection method includes the following steps:
step S100, controlling a camera to shoot a capacitance picture in a preset area and sending the capacitance picture;
in this step, the preset region refers to a region where the capacitor to be positioned and grasped is located. For example, the application scenario of the patent may be that the device is used in a production line, for example, a manipulator picks up a capacitor and then inserts the capacitor onto a circuit board, thereby reducing labor cost. At this time, the preset area is an area for placing the capacitor on the production line.
Step S110, the embedded equipment receives the capacitance picture and identifies the position information of the capacitor in the capacitance picture;
and identifying a capacitance key point from the capacitance picture by capacitance positioning detection processing software on the embedded device, dividing a capacitance key point set and mapping a capacitance area according to the capacitance key point, and finally obtaining two-dimensional coordinate position information of each capacitor in the capacitance picture.
Step S120, the embedded equipment sends the position information of the capacitor to a manipulator;
in this step, the embedded device converts the two-dimensional coordinate position information of the key point of the capacitor into a three-dimensional coordinate used by the mechanical arm, so as to realize the grabbing of the capacitor by the mechanical arm.
And step S130, the manipulator captures the capacitor according to the position information of the capacitor.
By arranging the capacitance positioning detection method provided by the invention in a detection system, when the method is applied to a manipulator on a production line to grab a capacitor, the production efficiency of the production line can be greatly improved.
In addition, an embodiment of the present invention further provides a computer-readable storage medium, where a deep learning-based capacitive positioning detection processing program is stored on the computer-readable storage medium, and when the deep learning-based capacitive positioning detection processing program is executed by a processor, the steps of the deep learning-based capacitive positioning detection method are implemented.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (8)
1. A capacitance positioning detection method based on deep learning is applied to a detection system and is characterized by comprising the following steps:
inputting the capacitance image into a deep learning network model to identify a plurality of capacitance key points, wherein the capacitance key points comprise a capacitance main body angular point, a capacitance left pin point and a capacitance right pin point;
dividing the capacitance key points into a plurality of key point sets, and acquiring a capacitance area corresponding to each key point set;
carrying out capacitance positioning detection according to the capacitance area;
the step of dividing the capacitance keypoints into a plurality of keypoint sets comprises:
calculating Euclidean distances among the key points of the capacitors;
dividing the capacitance key points into a plurality of key point sets according to the Euclidean distance;
the capacitor main body angular points comprise four main body angular points, the step of calculating Euclidean distances between the capacitor key points and dividing the capacitor key points into a plurality of key point sets according to the Euclidean distances comprises the following steps:
mapping the four main body angular points, the capacitor left pin point and the capacitor right pin point to six types of key points;
dividing the first class of key points into a plurality of key point sets;
for each key point to be allocated in the second category key points to the sixth category key points, sequentially allocating the second category key points to the sixth category key points to the key point set according to the following steps:
and traversing and calculating the Euclidean distance between the key point to be distributed and the key point set, and adding the key point to be distributed into a target key point set, wherein the Euclidean distance between the key point to be distributed and the target key point set falls into a preset distance range.
2. The deep learning based capacitive localization detection method of claim 1, wherein the step of inputting a capacitive image into the deep learning network model to identify a plurality of capacitive keypoints is preceded by the step of:
inputting the capacitance training image data into the deep learning network model to obtain the predicted Gaussian probability of the capacitance key point;
acquiring a difference value between the prediction Gaussian probability and the Gaussian probability of the labeled capacitance key point, wherein the Gaussian probability of the labeled capacitance key point is obtained by processing according to the capacitance training image data;
and adjusting parameters of the deep learning network model according to the difference value.
3. The deep learning based capacitive localization detection method of claim 2, wherein the step of inputting capacitive training image data into the deep learning network model to obtain the predicted gaussian probability of capacitive keypoints comprises:
taking the position of the key point of the capacitor as the peak position of Gaussian distribution;
and obtaining the predicted Gaussian probability of the capacitance key point according to the peak position.
4. The deep learning based capacitive location sensing method of claim 2, wherein the step of adjusting the parameters of the deep learning network according to the difference further comprises:
identifying a central point of a capacitor main body in the capacitor training image, and obtaining a prediction Gaussian probability of the central point of the capacitor main body;
acquiring a difference value between the predicted Gaussian probability of the center point of the capacitor main body and the labeled Gaussian probability of the center point of the capacitor main body, wherein the labeled Gaussian probability of the center point of the capacitor main body is obtained by processing according to the capacitor training image data;
and adjusting parameters of the deep learning network model according to the difference value.
5. A server comprising a memory, a processor and a deep learning based capacitive location detection processing program stored on the memory and executable on the processor, the deep learning based capacitive location detection processing program when executed by the processor implementing the steps of the deep learning based capacitive location detection method according to any one of claims 1 to 4.
6. A detection system for implementing the steps of the deep learning based capacitance positioning detection method according to any one of claims 1 to 4, wherein the detection system comprises an embedded device, a camera connected with the embedded device, and a manipulator, wherein the camera is used for taking a capacitance picture, the embedded device is used for identifying a capacitance position in the capacitance picture, and the manipulator captures a capacitance according to the capacitance position.
7. A method for detecting capacitive positioning, applied to the detection system as claimed in claim 6, the method for detecting capacitive positioning comprising the steps of:
controlling a camera to shoot a capacitance picture in a preset area and sending the capacitance picture;
the embedded equipment receives the capacitance picture and identifies the position information of the capacitor in the capacitance picture;
the embedded equipment sends the position information of the capacitor to a manipulator;
and the manipulator captures the capacitor according to the position information of the capacitor.
8. A medium, characterized in that the medium has stored thereon a processing program for deep learning based capacitive localization detection, which when executed by a processor implements the steps of the deep learning based capacitive localization detection method according to any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911353811.9A CN111141211B (en) | 2019-12-24 | 2019-12-24 | Deep learning-based capacitance positioning detection method, server and detection system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911353811.9A CN111141211B (en) | 2019-12-24 | 2019-12-24 | Deep learning-based capacitance positioning detection method, server and detection system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111141211A CN111141211A (en) | 2020-05-12 |
| CN111141211B true CN111141211B (en) | 2021-09-10 |
Family
ID=70519863
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911353811.9A Active CN111141211B (en) | 2019-12-24 | 2019-12-24 | Deep learning-based capacitance positioning detection method, server and detection system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111141211B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112200776A (en) * | 2020-09-25 | 2021-01-08 | 杭州加速科技有限公司 | Chip packaging defect detection method and detection device |
| CN112560964A (en) * | 2020-12-18 | 2021-03-26 | 深圳赛安特技术服务有限公司 | Method and system for training Chinese herbal medicine pest and disease identification model based on semi-supervised learning |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106846816A (en) * | 2017-04-12 | 2017-06-13 | 山东理工大学 | A kind of discretization traffic state judging method based on deep learning |
| CN108268504A (en) * | 2016-12-31 | 2018-07-10 | 中国移动通信集团陕西有限公司 | A kind of method and apparatus for the region of search for dividing point of interest |
| CN109410275A (en) * | 2018-10-15 | 2019-03-01 | 深圳步智造科技有限公司 | Position predicting method, device, equipment and readable storage medium storing program for executing |
| CN110298292A (en) * | 2019-06-25 | 2019-10-01 | 东北大学 | Detection method is grabbed when the high-precision real of rule-based object polygon Corner Detection |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP7118622B2 (en) * | 2017-11-16 | 2022-08-16 | 株式会社Preferred Networks | OBJECT DETECTION DEVICE, OBJECT DETECTION METHOD AND PROGRAM |
-
2019
- 2019-12-24 CN CN201911353811.9A patent/CN111141211B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108268504A (en) * | 2016-12-31 | 2018-07-10 | 中国移动通信集团陕西有限公司 | A kind of method and apparatus for the region of search for dividing point of interest |
| CN106846816A (en) * | 2017-04-12 | 2017-06-13 | 山东理工大学 | A kind of discretization traffic state judging method based on deep learning |
| CN109410275A (en) * | 2018-10-15 | 2019-03-01 | 深圳步智造科技有限公司 | Position predicting method, device, equipment and readable storage medium storing program for executing |
| CN110298292A (en) * | 2019-06-25 | 2019-10-01 | 东北大学 | Detection method is grabbed when the high-precision real of rule-based object polygon Corner Detection |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111141211A (en) | 2020-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021057742A1 (en) | Positioning method and apparatus, device, and storage medium | |
| CN110738143B (en) | Positioning method and device, equipment and storage medium | |
| WO2021227694A1 (en) | Image processing method and apparatus, electronic device, and storage medium | |
| CN110310333B (en) | Positioning method, electronic device and readable storage medium | |
| JP6815707B2 (en) | Face posture detection method, device and storage medium | |
| CN112528831B (en) | Multi-target attitude estimation method, multi-target attitude estimation device and terminal equipment | |
| CN111141211B (en) | Deep learning-based capacitance positioning detection method, server and detection system | |
| US8531519B1 (en) | Automatic multi-device localization and collaboration using cameras | |
| CN112198878B (en) | Instant map construction method and device, robot and storage medium | |
| CN111753826B (en) | Vehicle and license plate association method, device and electronic system | |
| CN111401363A (en) | Frame number image generation method and device, computer equipment and storage medium | |
| CN112465908B (en) | Object positioning method, device, terminal equipment and storage medium | |
| CN111767912B (en) | Marking point identification method, device, equipment and storage medium | |
| CN115081643B (en) | Confrontation sample generation method, related device and storage medium | |
| JP6393495B2 (en) | Image processing apparatus and object recognition method | |
| CN115471416A (en) | Object recognition method, storage medium, and apparatus | |
| WO2018135326A1 (en) | Image processing device, image processing system, image processing program, and image processing method | |
| JP2011028419A (en) | Device, method and program for detecting object | |
| EP3767589A1 (en) | Information processing device, information processing method, and program | |
| US9600081B2 (en) | Information processing method and electronic device for improving gesture identification | |
| US20240119622A1 (en) | Object recognition method of three-dimensional space and computing apparatus | |
| US20220139073A1 (en) | Automatic topology mapping processing method and system based on omnidirectional image information | |
| US9239643B2 (en) | Method and system for touch shape recognition, related screen apparatus, and computer program product | |
| WO2021057582A1 (en) | Image matching, 3d imaging and pose recognition method, device, and system | |
| CN114419688A (en) | Real-time face recognition method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230801 Address after: 401147 No. 6, Middle Section of Mount Huangshan Avenue, Liangjiang New District, Yubei District, Chongqing Patentee after: Chongqing youchenxu Intelligent Technology Co.,Ltd. Address before: 518000 no.2188 Longxiang Avenue, Longcheng street, Longgang District, Shenzhen City, Guangdong Province Patentee before: SHENZHEN INSTITUTE OF INFORMATION TECHNOLOGY |