CN115187896A - Narrow-mouth bottle inclination angle detection method for intelligent experiment evaluation - Google Patents
Narrow-mouth bottle inclination angle detection method for intelligent experiment evaluation Download PDFInfo
- Publication number
- CN115187896A CN115187896A CN202210729388.3A CN202210729388A CN115187896A CN 115187896 A CN115187896 A CN 115187896A CN 202210729388 A CN202210729388 A CN 202210729388A CN 115187896 A CN115187896 A CN 115187896A
- Authority
- CN
- China
- Prior art keywords
- loss
- key point
- reagent bottle
- neg
- pos
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/766—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using regression, e.g. by projecting features on hyperplanes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for detecting the inclination angle of a narrow-mouth bottle for intelligent experimental evaluation, which comprises the following steps: taking out a picture containing the reagent bottle when labeling/reasoning is carried out on the related experiment evaluation video; performing key point regression model training, and performing iterative optimization of the key point regression model according to a loss function; sending the taken picture containing the reagent bottle into a key point regression model to obtain coordinates (x 1, y 1) and (x 2, y 2) of two key points at the position of the reagent bottle mouth; according to the formula
And calculating the inclination angle of the reagent bottle. The method detects the reagent bottle firstly, and then judges the inclination angle by positioning two key points of the reagent bottle mouth, has the advantages of being beneficial to marking, high in detection precision and the like, can better perform more accurate examination, teaching and evaluation aiming at related chemical experiments, and can greatly reduce the computational cost by combining a light-weight backbone network.
Description
Technical Field
The invention relates to the field of intelligent experiment evaluation, in particular to a method for detecting the inclination angle of a narrow-mouth bottle for intelligent experiment evaluation.
Background
With the development of deep learning, computer vision technology is continuously developed, particularly, object detection technology represented by target detection and example segmentation is particularly remarkable, various industries have good landing schemes, common intelligent evaluation schemes are mainly based on target detection, and whether some experimental operations are correct or not is judged through relative positions and changes of detection frames.
For the inclined angle of the related reagent bottle during use, the following methods are mainly used in the prior art:
1. the method is only capable of roughly estimating the inclination angle of the reagent bottle and cannot meet the requirements of experimental investigation.
2. The detection method based on the semantic segmentation is characterized in that the inclination angle is detected according to the detected overall contour of the object, the accuracy is higher than that of the target detection method based on the rough judgment of the length-width ratio of the detection frame, the judgment of the inclination angle is relatively accurate based on the semantic segmentation method, but the contour information is easily lost because the related detection equipment is a transparent object, the overall contour detection has deviation, the detection of the final inclination angle is greatly influenced, and meanwhile, the marking of the semantic segmentation also costs huge manpower and material resources.
Disclosure of Invention
The invention aims to provide a method for detecting the inclination angle of a narrow-mouth bottle for intelligent experimental evaluation, so that the experimental evaluation can be accurately carried out, and meanwhile, the computational power of the intelligent evaluation is reduced.
In order to solve the technical problem, the invention provides a method for detecting the inclination angle of a narrow-mouth bottle for intelligent experimental evaluation, which comprises the following steps:
taking out a picture containing the reagent bottle when the related experiment evaluation video is marked/inferred;
performing key point regression model training, and performing iterative optimization of the key point regression model according to a loss function;
sending the taken picture containing the reagent bottle into a key point regression model to obtain coordinates (x 1, y 1) and (x 2, y 2) of two key points at the position of the reagent bottle mouth;
according to the formula
Further, the key point model comprises a feature extractor, two up-sampling modules, a first convolution neural network and a convolution layer which are connected in sequence, wherein the first convolution neural network outputs a feature tensor result1, and the convolution layer outputs a feature tensor result2.
Further, the specific steps of the key point model training are as follows:
inputting a picture containing a reagent bottle into the key point model, and acquiring feature tensors result1 and result2 with the number of channels being the number of required key points;
after activating function processing is carried out on result1, the feature map of the first channel is used as a heat map1 of a first key point, and the feature map of the second channel is used as a heat map2 of a second key point;
generating a heat map1 'and a heat map2' as label labels using a two-dimensional Gaussian kernel function;
and taking out positive and negative samples according to the positive and negative sample indexes to respectively perform loss, and performing iterative optimization on the model according to a final loss function.
Further, taking out the positive and negative samples according to the positive and negative sample indexes to respectively make loss specifically:
taking a region with a value larger than 0 in the heat map heatmap1 'and the heat map heatmap2' as a positive sample, and obtaining indexes one _ pos _ mask and two _ pos _ mask of the positive sample in a feature vector matrix;
taking an area which is equal to 0 in the heat map heatmap1 'and the heat map2' as a negative sample, and obtaining indexes of one _ neg _ mask and two _ neg _ mask in a feature vector matrix;
according to the positive and negative sample indexes, positive and negative samples in the heatmap1 and the heatmap1 'can be taken out to be lost respectively to obtain one _ pos _ loss and one _ neg _ loss, and similarly, the heatmap2 and the heatmap2' are used to obtain two _ pos _ loss and two _ neg _ loss;
similarly, result2 is processed in the manner of result 1.
Further, the loss function of the loss of the positive and negative samples is MSELoss.
Further, the final loss function is:
Loss＝one_pos_loss+two_pos_loss+0.1*one_neg_loss+0.1*two_neg_loss；
wherein one _ pos _ loss is a positive sample loss of result1, one _ neg _ loss is a negative sample loss of result1, two _ pos _ loss is a positive sample loss of result2, and two _ neg _ loss is a negative sample loss of result2.
Compared with the prior art, the invention at least has the following beneficial effects:
the invention provides a key point detection scheme based on heatmap regression, which is characterized in that a reagent bottle is detected immediately after a target detection network, and then the inclination angle is judged by positioning two key points of the bottle mouth of the reagent bottle.
Drawings
FIG. 1 is a flowchart of an embodiment of a method for detecting the inclination angle of a narrow-mouth bottle for intelligent experimental evaluation according to the present invention;
FIG. 2 is a schematic diagram of the overall structure of a key point training model according to an embodiment of the method for detecting the inclination angle of a narrow-mouth bottle for intelligent experimental evaluation of the invention;
fig. 3 is a schematic diagram of a key point detection network structure of an embodiment of the method for detecting the inclination angle of the narrow-mouth bottle for the intelligent experimental evaluation of the invention.
Detailed Description
The method for detecting the tilt angle of a narrow-mouth bottle for intellectual experimental evaluation according to the present invention will be described in more detail with reference to the accompanying schematic drawings, in which preferred embodiments of the present invention are shown, it being understood that those skilled in the art can modify the present invention described herein while still achieving the advantageous effects of the present invention. Accordingly, the following description should be construed as broadly as possible to those skilled in the art and not as limiting the invention.
The invention is described in more detail in the following paragraphs by way of example with reference to the accompanying drawings. Advantages and features of the present invention will become apparent from the following description and from the claims. It is to be noted that the drawings are in a very simplified form and are not to precise scale, which is provided for the purpose of facilitating and clearly illustrating embodiments of the present invention.
As shown in fig. 1, an embodiment of the present invention provides a method for detecting an inclination angle of a narrow-mouth bottle for intelligent experimental evaluation, including the following steps:
taking out a picture containing the reagent bottle when the related experiment evaluation video is marked/inferred;
performing key point regression model training, and performing iterative optimization of the key point regression model according to a loss function;
sending the taken picture containing the reagent bottle into a key point regression model to obtain coordinates (x 1, y 1) and (x 2, y 2) of two key points at the reagent bottle mouth;
according to the formula
The following is a list of preferred embodiments of the method for detecting the inclined angle of a narrow-mouth bottle for intelligent experimental evaluation, so as to clearly illustrate the content of the present invention, it should be understood that the content of the present invention is not limited to the following embodiments, and other modifications by conventional technical means of those skilled in the art are within the scope of the idea of the present invention.
(1) And taking out pictures containing the reagent bottles when the related experiment evaluation videos are marked/inferred.
Specifically, the common intelligent evaluation scheme is based on target detection, whether some experimental operations are correct or not is judged through the relative position and change of a detection frame, and depending on the judgment, the contained reagent bottle pictures are taken out when related experimental videos are marked/inferred.
(2) And training the key point regression model, and performing iterative optimization on the key point regression model according to the loss function.
Specifically, the labeling of the key points of the reagent bottle is mainly two key points at the bottle mouth, the left side is point 1, the right side is point 2, the specific labeling conditions are as shown in fig. 1 below, the label content is the coordinate value of the relevant key point, for the label content, taking the width and the height as W and H as examples, the key point coordinate is (x 1, y 1), and the label content is (x 1/W, y 1/H).
In an embodiment, (a) the overall structure of the key point training model is as shown in fig. 2, a lightweight backbone network is used as a feature extractor, two DUC modules are connected behind the lightweight backbone network to perform upsampling on a feature map (the number of the upsampling modules and the times of the upsampling can be flexibly adjusted), a convolutional neural network is used to obtain a feature tensor result1 with the channel number being the required number of key points, the tensor is concat with a tensor generated by a second DUC module to realize feature fusion, and the fused features are subjected to convolutional layer to obtain a feature tensor result2 with the channel number being the required number of key points.
(b) In the training process, as shown in fig. two, result1 and result2 are obtained, where we take the input image size 480 × 480 as an example, output tensors of which result1 and result2 are both 1 × 2 × 60 (the above-mentioned keypoint regression model, if n keypoints need to be regressed, the tensor shape is generated here as 1 × n × 60), perform sigmoid processing on result1, the feature map of the first channel is used as heatmap1 of the first keypoint, the shape is 1 × 60, the feature map of the second channel is used as heatmap2 of the second keypoint, generate heatmap1' and heatmap2' as labels by using two-dimensional gaussian kernel functions according to the coordinate information in the label, taking a region with a value greater than 0 in the heatmap1' as a positive sample to obtain an index one _ pos _ mask in the feature vector matrix, taking a region with a value equal to 0 as a negative sample to obtain an index one _ neg _ mask in the feature vector matrix, similarly obtaining an index two _ pos _ mask and an index two _ neg _ mask in the heatmap2', according to the indexes of the positive and negative samples, taking the positive and negative samples in the heatmap1' and the heatmap1' out to respectively perform loss (the loss function is MSELoss) to obtain a one _ pos _ loss and a one _ neg _ loss, and similarly obtaining the two _ pos _ loss and the tto _ neg _ loss from the heatmap2' and the heatmap2
(c) Referring to the related operation of the result1 in the step 2, the same operation is performed on the result2, so that the four losses obtained by the result1 are respectively one _ pos _ loss1, one _ neg _ loss1, two _ pos _ loss1 and two _ neg _ loss1, and the four losses obtained by the result2 are respectively one _ pos _ loss2, one _ neg _ loss2, two _ pos _ loss1 and two _ neg _ loss2, and the losses are taken as follows:
one_pos_loss＝one_pos_loss1+2*one_pos_loss2
one_neg_loss＝one_neg_loss1+2*one_neg_loss2
two_pos_loss＝two_pos_loss1+2*two_pos_loss2
two_neg_loss＝two_neg_loss1+2*two_neg_loss2
the final loss function is taken as follows:
after Loss = one _ pos _ Loss + two _ pos _ Loss +0.1 one \/neg _loss +0.1 two \/neg _/Loss, iterative optimization of the model is performed using the above Loss function.
(3) Sending the taken picture containing the reagent bottle into a key point regression model, obtaining coordinates (x 1, y 1) and (x 2, y 2) of two key points at the position of the reagent bottle mouth, and obtaining coordinates according to a formula
Specifically, depending on the original target detection network, when a reagent bottle is detected, the reagent bottle is cut out according to a detection frame and sent to the key point detection network (the network directly takes result2 as a result during reasoning as shown in fig. 3), so that two point coordinates (x 1, y 1) and (x 2, y 2) at the position of the reagent bottle mouth are obtained, and the required inclination angle can be obtained according to a formula arctan ((y 2-y 1)/(x 2-x 1)).
In conclusion, the key point detection scheme based on heatmap regression provided by the invention detects the reagent bottle immediately after the target detection network, and then judges the inclination angle by positioning two key points at the bottle mouth of the reagent bottle, so that the method has the advantages of being beneficial to marking, high in detection precision and the like, can better perform more accurate examination, teaching and evaluation aiming at related chemical experiments, and can greatly reduce the computational cost by combining with a lightweight backbone network.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (6)
1. A method for detecting the inclination angle of a narrow-mouth bottle for intelligent experimental evaluation is characterized by comprising the following steps:
taking out a picture containing the reagent bottle when the related experiment evaluation video is marked/inferred;
performing key point regression model training, and performing iterative optimization of the key point regression model according to a loss function;
sending the taken picture containing the reagent bottle into a key point regression model to obtain coordinates (x 1, y 1) and (x 2, y 2) of two key points at the position of the reagent bottle mouth;
according to the formula
And calculating the inclination angle of the reagent bottle.
2. The method for detecting the inclined angle of the narrow-necked bottle for the intelligent experimental evaluation as claimed in claim 1, wherein the key point model comprises a feature extractor, two up-sampling modules, a first convolutional neural network and a convolutional layer which are connected in sequence, the first convolutional neural network outputs a feature tensor result1, and the convolutional layer outputs a feature tensor result2.
3. The method for detecting the inclined angle of the narrow-mouth bottle for the intelligent experimental evaluation as claimed in claim 1, wherein the specific steps of the key point model training are as follows:
inputting a picture containing a reagent bottle into the key point model, and acquiring feature tensors result1 and result2 with the number of channels being the number of required key points;
after activating function processing is carried out on result1, the feature map of the first channel is used as a heat map1 of a first key point, and the feature map of the second channel is used as a heat map2 of a second key point;
generating a heat map1 'and a heat map2' as label labels using a two-dimensional Gaussian kernel function;
and taking out positive and negative samples according to the positive and negative sample indexes to respectively perform loss, and performing iterative optimization on the model according to a final loss function.
4. The method for detecting the inclined angle of the narrow-necked bottle for the intelligent experimental evaluation according to claim 3, wherein the steps of taking out the positive and negative samples according to the positive and negative sample indexes and respectively losing are as follows:
taking an area larger than 0 in the heat map heatmap1 'and the heat map2' as a positive sample to obtain indexes one _ pos _ mask and two _ pos _ mask in a feature vector matrix;
taking the area equal to 0 in the heat map heatmap1 'and the heat map heatmap2' as a negative sample, and obtaining the indexes of one _ neg _ mask and two _ neg _ mask in the feature vector matrix;
according to the positive and negative sample indexes, positive and negative samples in the heatmap1 and the heatmap1 'can be taken out to be lost respectively to obtain one _ pos _ loss and one _ neg _ loss, and similarly, the heatmap2 and the heatmap2' are used to obtain two _ pos _ loss and two _ neg _ loss;
similarly, result2 is processed in the manner of result 1.
5. The method of claim 4, wherein the loss function of the positive and negative samples for loss is MSELoss.
6. The method of claim 1, wherein the final loss function is:
Loss＝one_pos_loss+two_pos_loss+0.1*one_neg_loss+0.1*two_neg_loss；
wherein, one _ pos _ loss is a positive sample loss of result1, one _ neg _ loss is a negative sample loss of result1, two _ pos _ loss is a positive sample loss of result2, and two _ neg _ loss is a negative sample loss of result2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210729388.3A CN115187896A (en) | 2022-06-24 | 2022-06-24 | Narrow-mouth bottle inclination angle detection method for intelligent experiment evaluation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210729388.3A CN115187896A (en) | 2022-06-24 | 2022-06-24 | Narrow-mouth bottle inclination angle detection method for intelligent experiment evaluation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115187896A true CN115187896A (en) | 2022-10-14 |
Family
ID=83515171
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210729388.3A Pending CN115187896A (en) | 2022-06-24 | 2022-06-24 | Narrow-mouth bottle inclination angle detection method for intelligent experiment evaluation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115187896A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116740704A (en) * | 2023-06-16 | 2023-09-12 | 安徽农业大学 | Wheat leaf phenotype parameter change rate monitoring method and device based on deep learning |
-
2022
- 2022-06-24 CN CN202210729388.3A patent/CN115187896A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116740704A (en) * | 2023-06-16 | 2023-09-12 | 安徽农业大学 | Wheat leaf phenotype parameter change rate monitoring method and device based on deep learning |
| CN116740704B (en) * | 2023-06-16 | 2024-02-27 | 安徽农业大学 | Wheat leaf phenotype parameter change rate monitoring method and device based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110837835A (en) | End-to-end scene text identification method based on boundary point detection | |
| CN110728200A (en) | Real-time pedestrian detection method and system based on deep learning | |
| CN111563452B (en) | Multi-human-body gesture detection and state discrimination method based on instance segmentation | |
| CN112836650B (en) | Semantic analysis method and system for quality inspection report scanning image table | |
| CN111061904B (en) | Local picture rapid detection method based on image content identification | |
| CN110796143A (en) | Scene text recognition method based on man-machine cooperation | |
| CN112541491B (en) | End-to-end text detection and recognition method based on image character region perception | |
| CN110163208A (en) | A kind of scene character detecting method and system based on deep learning | |
| CN111476210A (en) | Image-based text recognition method, system, device and storage medium | |
| CN115272204A (en) | Bearing surface scratch detection method based on machine vision | |
| CN113240716B (en) | Twin network target tracking method and system with multi-feature fusion | |
| CN114119480A (en) | Crack defect detection system based on deep learning | |
| CN112926486A (en) | Improved RFBnet target detection algorithm for ship small target | |
| Farag | A lightweight vehicle detection and tracking technique for advanced driving assistance systems | |
| Liao et al. | Solder joint defect inspection method based on ConvNeXt-YOLOX | |
| CN115187896A (en) | Narrow-mouth bottle inclination angle detection method for intelligent experiment evaluation | |
| CN114743201A (en) | Multimeter reading identification method and system based on rotating target detection | |
| CN114330247A (en) | Automatic insurance clause analysis method based on image recognition | |
| CN109726754A (en) | A kind of LCD screen defect identification method and device | |
| CN117173547A (en) | Underwater target detection method based on improved YOLOv6 algorithm | |
| CN116824333A (en) | Nasopharyngeal carcinoma detecting system based on deep learning model | |
| CN116129176A (en) | Few-sample target detection method based on strong-correlation dynamic learning | |
| Song et al. | Bi-directional seed attention network for interactive image segmentation | |
| CN114359493A (en) | Method and system for generating three-dimensional semantic map for unmanned ship | |
| CN113469984A (en) | Display panel appearance detection method based on YOLO structure |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |