CN117115861B - Glove detection method and device, electronic equipment and storage medium - Google Patents
Glove detection method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117115861B CN117115861B CN202311353217.6A CN202311353217A CN117115861B CN 117115861 B CN117115861 B CN 117115861B CN 202311353217 A CN202311353217 A CN 202311353217A CN 117115861 B CN117115861 B CN 117115861B
- Authority
- CN
- China
- Prior art keywords
- position frame
- frame information
- glove
- information
- key point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 129
- 238000003860 storage Methods 0.000 title claims abstract description 11
- 230000009471 action Effects 0.000 claims abstract description 60
- 238000000034 method Methods 0.000 claims abstract description 18
- 238000010586 diagram Methods 0.000 claims description 48
- 238000002372 labelling Methods 0.000 claims description 37
- 210000000988 bone and bone Anatomy 0.000 claims description 22
- 238000000605 extraction Methods 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 abstract description 6
- 230000000875 corresponding effect Effects 0.000 description 40
- 238000012544 monitoring process Methods 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 125000004122 cyclic group Chemical group 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 238000011049 filling Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000010998 test method Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000000137 annealing Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000000391 smoking effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/107—Static hand or arm
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Computational Linguistics (AREA)
- Image Analysis (AREA)
Abstract
The application provides a glove detection method, a glove detection device, electronic equipment and a storage medium. The method comprises the following steps: acquiring character coordinate position frame information and skeleton key point information in an image to be identified; positioning hand position frame information in the image to be identified according to the character coordinate position frame information and the skeleton key point information; and performing glove identification on the hand position frame through the glove detection model to obtain a first glove detection result. Because skeleton key point information is acquired through the character coordinate position frame information in the glove detection method, the hand position frame information can be accurately identified through the character coordinate position frame information and the skeleton key point information in the complex environments such as at night, and the problem that the hand position frame information needs to be identified in the character coordinate position frame information in the existing glove identification method, when the image environment is complex, the hand position frame is inaccurate due to the fact that the hand type action and the visual angle are changed, and the problem that the wearing judgment of the gloves of staff in an oil and gas station is wrong is solved.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a glove detection method, a device, an electronic apparatus, and a storage medium.
Background
The antistatic glove is a labor protection article which is needed to be worn by staff in an oil and gas station when the oil unloading work is carried out, and serious potential safety hazards exist when the work is carried out without wearing the antistatic glove. The manual supervision method requires monitoring personnel to monitor illegal operation behaviors of staff in the oil and gas station in real time, and is low in monitoring efficiency and poor in monitoring effect. In the prior art, a trained deep learning model is often adopted to detect images acquired by a camera in real time, and when staff in the images are detected to be not wearing gloves, the staff can give an alarm in time. However, in the existing glove detection method, hand features are often identified in a person position detection frame, so that whether a person wears gloves or not is judged, when the environment where an image is located is complex, the detection accuracy is low when the action of the person is complex.
Disclosure of Invention
The application provides a glove detection method, a glove detection device, electronic equipment and a storage medium. According to the glove detection method, the hand position frame information is determined according to the person coordinate position frame information and the skeleton key point information, and whether the person in the image to be identified wears the glove or not is judged according to the hand position frame information. Because skeleton key point information is acquired through the character coordinate position frame information in the glove detection method, the hand position frame information can be accurately identified through the character coordinate position frame information and the skeleton key point information in the complex environments such as at night, and the problem that the hand position frame information needs to be identified in the character coordinate position frame information in the existing glove identification method, when the image environment is complex, the hand position frame is inaccurate due to the fact that the hand type action and the visual angle are changed, and the problem that the wearing judgment of the gloves of staff in an oil and gas station is wrong is solved.
In a first aspect, the present application provides a glove detection method comprising:
acquiring character coordinate position frame information and corresponding skeleton key point information in an image to be identified;
positioning hand position frame information in the image to be identified according to the character coordinate position frame information and the skeleton key point information;
and performing glove identification on the hand position frame through a pre-trained glove detection model to obtain a first glove detection result.
Optionally, the glove detection method provided in the present application further includes:
extracting the character coordinate position frame information from the image to be identified through a preset character detection model;
and extracting skeleton key point information corresponding to the character coordinate position frame information from the image to be identified through a preset skeleton detection model.
Optionally, the glove detection method provided in the present application further includes:
according to the skeletal key point information, executing a hand region labeling action in the character coordinate position frame information, and obtaining a hand region labeling result;
and acquiring the hand position frame information from the character coordinate position frame information according to the hand region labeling result.
Optionally, the glove detection method provided in the present application further includes:
performing character action judgment according to the skeleton key point information to obtain character action judgment results;
screening a second action algorithm corresponding to the bone key point information from a preset first action algorithm according to the character action judging result;
and executing a hand region labeling action in the character coordinate position frame information through the second action algorithm according to the skeleton key point information, and obtaining a hand region labeling result.
Optionally, the glove detection method provided in the present application further includes:
acquiring single person coordinate position frame information from a plurality of pieces of person coordinate position frame information;
acquiring single-person skeleton key point information according to the single-person coordinate position frame information and the skeleton key point information;
positioning single hand position frame information corresponding to the single coordinate position frame information according to the single coordinate position frame information and the single skeleton key point information;
and performing glove identification on the single hand position frame through the glove detection model to obtain a second glove detection result.
Optionally, the glove detection method provided in the present application further includes:
constructing a single thermodynamic diagram according to the single hand position frame information and the single skeleton key point information;
performing channel splicing action on the single hand position frame information and the single thermodynamic diagram to obtain a first splicing result;
inputting the first splicing result into the glove detection model to obtain the second glove detection result.
Optionally, the single hand position frame information includes second single hand position frame information and first single hand position frame information that are adjacent according to the time frame ordering, first single hand position frame information with first concatenation result is corresponding, and the glove detection method that this application provided still includes:
splicing the first splicing result with the second single hand position frame information to obtain a second splicing result;
inputting the second splicing result into the glove detection model to obtain a third glove detection result.
In a second aspect, the present application also provides a glove detection apparatus comprising:
the character labeling module is used for acquiring character coordinate position frame information and corresponding skeleton key point information in the image to be identified;
the hand marking module is used for positioning hand position frame information in the image to be identified according to the character coordinate position frame information and the skeleton key point information;
the first glove detection module is used for carrying out glove identification on the hand position frame through a pre-trained glove detection model, and obtaining a first glove detection result.
In a third aspect, the present application also provides an electronic device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the glove detection method as described in the first aspect.
In a fourth aspect, embodiments of the present application provide a storage medium having stored thereon a program or instructions which when executed by a processor implement the steps of the glove detection method according to the first aspect.
According to the glove detection method, the hand position frame information is determined according to the person coordinate position frame information and the skeleton key point information, and whether the person in the image to be identified wears the glove or not is judged according to the hand position frame information. Because skeleton key point information is acquired through the character coordinate position frame information in the glove detection method, the hand position frame information can be accurately identified through the character coordinate position frame information and the skeleton key point information in the complex environments such as at night, and the problem that the hand position frame information needs to be identified in the character coordinate position frame information in the existing glove identification method, when the image environment is complex, the hand position frame is inaccurate due to the fact that the hand type action and the visual angle are changed, and the problem that the wearing judgment of the gloves of staff in an oil and gas station is wrong is solved.
The foregoing description is merely an overview of the technical solutions provided in the present application, and may be implemented according to the content of the specification in order to make the technical means of the present application more clearly understood, and in order to make the above-mentioned and other objects, features and advantages of the present application more clearly understood, the following detailed description of the present application is given.
Drawings
One or more embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements, and in which the figures of the drawings are not to be taken in a limiting sense, unless otherwise indicated.
FIG. 1 is a schematic diagram of a glove detection method according to an embodiment of the present application;
FIG. 2 is a second schematic diagram of a glove detection method according to an embodiment of the present disclosure;
FIG. 3 is a third schematic diagram of a glove detection method according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of a glove detection method according to an embodiment of the present disclosure;
FIG. 5 is a fifth schematic diagram of a glove detection method according to an embodiment of the present disclosure;
FIG. 6 is a schematic diagram of a glove detection method according to an embodiment of the present disclosure;
FIG. 7 is a schematic diagram of a glove test method according to an embodiment of the present disclosure;
FIG. 8 is an example glove test procedure provided herein;
FIG. 9 is a schematic diagram of a glove detection apparatus provided in an embodiment of the present application;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The terms first, second and the like in the description and in the claims, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged, as appropriate, such that embodiments of the present application may be implemented in sequences other than those illustrated or described herein, and that the objects identified by "first," "second," etc. are generally of a type and not limited to the number of objects, e.g., the first object may be one or more. Furthermore, in the description and claims, "and/or" means at least one of the connected objects, and the character "/", generally means that the associated object is an "or" relationship.
The glove detecting method, the device, the electronic equipment and the storage medium provided by the application are described in detail below through specific embodiments and application scenes thereof with reference to the accompanying drawings.
A first embodiment of the present application relates to a glove detection method, as shown in fig. 1, including:
step 101, acquiring character coordinate position frame information and corresponding skeleton key point information in an image to be identified;
102, positioning hand position frame information in the image to be identified according to the character coordinate position frame information and the skeleton key point information;
and 103, performing glove identification on the hand position frame through a pre-trained glove detection model to obtain a first glove detection result.
Specifically, according to the glove detection method, images to be identified are required to be continuously acquired by means of a plurality of cameras, so that the real-time monitoring of the glove wearing behaviors of staff in an oil and gas station is realized. For example, the protection can be distributed in the working area by a plurality of high-definition cameras, the layout interval of the plurality of cameras is within ten meters, and more than one meter, and in order to ensure the integrity and the definition of the image, for example, a ipx-level waterproof 200-ten-thousand-pixel high-definition camera, i.e. 1920×1080 high-definition camera, can be adopted. The glove detection method provided by the application realizes glove identification by using the deep learning model, and needs to be carried on a computing platform for realizing. For example, the computing platform may be a computer including 1 GPU 1080Ti and above, the memory of the computer is not less than 8G, and the main frequency of the processor of the computer is not less than 2.3GHz.
After the preparation of the early work is finished, the glove detection method provided by the application needs to acquire images of the oil and gas station staff in the oil unloading process from the camera, extract frames to obtain images to be identified, obtain the coordinate position information of the overall outline of the person in the images to be identified, and frame-select to obtain the coordinate position frame information of the person, and obtain the corresponding skeleton key point information in the coordinate position frame information of the person.
And then, positioning the character coordinate position frame information according to the character coordinate position frame information and the corresponding skeleton key point information to obtain the hand position frame information. And finally, judging the hand position frame information through a glove detection model to obtain a first glove detection result, and sending out an alarm signal when the first glove detection result is that the hand position frame information in the image to be identified is not wearing gloves, so as to prevent illegal operation of staff in the oil and gas station.
According to the glove detection method, the hand position frame information is determined according to the person coordinate position frame information and the skeleton key point information, and whether the person in the image to be identified wears the glove or not is judged according to the hand position frame information. Because skeleton key point information is acquired through the character coordinate position frame information in the glove detection method, the hand position frame information can be accurately identified through the character coordinate position frame information and the skeleton key point information in the complex environments such as at night, and the problem that the hand position frame information needs to be identified in the character coordinate position frame information in the existing glove identification method, when the image environment is complex, the hand position frame is inaccurate due to the fact that the hand type action and the visual angle are changed, and the problem that the wearing judgment of the gloves of staff in an oil and gas station is wrong is solved.
On the basis of the above embodiment, as shown in fig. 2, in the glove detection method provided in the present application, step 101 includes:
step 111, extracting the character coordinate position frame information from the image to be identified through a preset character detection model;
and 112, extracting skeleton key point information corresponding to the character coordinate position frame information from the image to be identified through a preset skeleton detection model.
Specifically, in order to reduce the pressure of data annotation, the character coordinate position information may be obtained by using, but not limited to, an open-source character detection model, and the bone key point information may be obtained by using, but not limited to, an open-source bone detection model.
Specifically, after an image to be identified is obtained from a camera, the image to be identified is input into an open-source character detection model to obtain position coordinate frames of a plurality of tasks in the image to be identified, namely character coordinate position frame information. And then carrying out visual confirmation on the character coordinate position frame information, and modifying the missed detection object and the false detection object. And labeling hand data and glove data according to the coordinate position frame information of a plurality of characters in the image to be identified, acquiring corresponding skeleton key points through an open-source skeleton detection model, and confirming and modifying the data of the skeleton key points.
On the basis of the embodiment, the character coordinate position frame information and the bone key point information are acquired through the preset character detection model and the bone detection model, so that the increase of training cost caused by model training is avoided, and the pressure of data marking is reduced by the existing mature character detection model and bone detection model.
Based on the above embodiment, as shown in fig. 3, in the glove detection method provided in the present application, step 102 includes:
step 121, executing a hand region labeling action in the character coordinate position frame information according to the skeleton key point information, and obtaining a hand region labeling result;
and step 122, acquiring the hand position frame information from the character coordinate position frame information according to the hand region labeling result.
Specifically, after bone information corresponding to the character coordinate position frame information is obtained according to the bone key point information, a hand region in the character coordinate position frame information is corresponding to the bone information, and a hand labeling action and a glove labeling action are performed on the hand region, so that a hand region labeling result is obtained. And then determining the hand position frame information according to the hand region labeling result.
On the basis of the embodiment, the problem that the effect is poor and the precision is low in the player part area of the character coordinate position frame information frame directly because the image to be recognized is in a complex environment such as night is avoided by determining the hand area in the character coordinate position frame information according to the bone key point information.
Based on the above embodiment, as shown in fig. 4, in the glove detection method provided in the present application, step 121 includes:
step 123, performing character action judgment according to the skeleton key point information to obtain character action judgment results;
step 124, screening a second action algorithm corresponding to the bone key point information from preset first action algorithms according to the character action judging result;
and step 125, executing a hand region labeling action in the character coordinate position frame information through the second action algorithm according to the skeleton key point information, and obtaining a hand region labeling result.
Specifically, the glove detection method provided by the application can judge the action of the person according to the skeleton key point information, for example, under various action states such as smoking, calling, leaving a post, falling, and the like, hands are unfolded or held, after the difference exists in hand visual angles, a second action algorithm conforming to the action is screened from the preset first action algorithm according to the corresponding conditions, and the hand region marking is carried out through the second action algorithm according to the skeleton key point information.
On the basis of the embodiment, through a plurality of preset different action algorithms, when the glove is detected, the action of the corresponding person can be judged according to the skeleton key point information, and then the hand marking is carried out through the corresponding action algorithm, so that support is provided for a plurality of different scenes under the condition of avoiding the resource consumption of the model, and the application occasion is expanded.
In addition to the above embodiment, as shown in fig. 5, the glove detection method provided in the present application further includes, after step 101:
104, acquiring single person coordinate position frame information from a plurality of pieces of person coordinate position frame information;
step 105, acquiring single-person skeleton key point information according to the single-person coordinate position frame information and the skeleton key point information;
step 106, positioning single hand position frame information corresponding to the single coordinate position frame information according to the single coordinate position frame information and the single skeleton key point information;
and 107, performing glove identification on the single hand position frame through the glove detection model to obtain a second glove detection result.
Specifically, a plurality of characters in the image to be identified provided by the application are needed to detect whether the characters wear gloves or not. After the plurality of person coordinate position frame information is acquired, the single person coordinate position frame information corresponding to each person is acquired from the plurality of person coordinate position frame information, the single hand position frame information corresponding to the single person coordinate position frame information is acquired according to the single person coordinate position frame information and the corresponding single skeleton key point information, so that single glove wearing recognition is realized, and after glove wearing recognition is carried out on the person coordinate position frame information corresponding to the plurality of persons in the image to be recognized, a recognition result corresponding to the image to be recognized is acquired.
The glove detection method provided by the application can be used for detecting whether the person in the image to be identified wears the glove or not by adopting, but not limited to, a two-stage network model. The network model expression in the first stage obtains a plurality of character coordinate position frame information and corresponding skeleton key point information from the image to be identified, obtains single person coordinate position frame information and single person coordinate position frame information according to the character coordinate position frame information and the corresponding skeleton key point information, and then inputs the data into the second stage model to detect the wearing of the glove, thereby ensuring the detection accuracy of the hand and the glove and improving the detection efficiency.
In order to improve the learning efficiency of the network model used in the method, the optimal learning rate parameters can be selected by adopting methods such as grid parameter adjustment and the like, and strategies such as cosine annealing and the like can be added to dynamically adjust the learning rate so as to prevent the learning rate from falling into a local optimal solution. In order to shorten training time and speed up loss reduction, a coco large-scale data set can be used as a pre-training model, and focal_loss is introduced to increase the loss of a difficult sample, so that accuracy is improved. In addition, the accuracy of the algorithm model can be evaluated through precision, recall, mAP.
Based on the above embodiment, as shown in fig. 6, in the glove detection method provided in the present application, step 107 includes:
step 171, constructing a single thermodynamic diagram according to the single hand position frame information and the single skeleton key point information;
step 172, performing channel splicing action on the single hand position frame information and the single thermodynamic diagram to obtain a first splicing result;
step 173, inputting the first splicing result into the glove detection model to obtain the second glove detection result.
Specifically, after the single hand position frame information and the single skeleton key point information are obtained, a single skeleton thermodynamic diagram may be constructed, for example, gao Siyuan thermodynamic diagrams with the center point of the single hand position frame information as the midpoint and the size of the labeling frame of the hand and glove as the radius R are made. And then the single hand position frame information and the corresponding single thermodynamic diagram are spliced on the channel to obtain a first splicing result. And finally, inputting the first splicing result into a glove detection model to obtain a second glove detection result.
Wherein the first stage model is capable of detecting the hand position frame information of the owner and the skeletal thermodynamic diagram of the owner from the image to be identified. And the second stage model is used for detecting whether the single person wears gloves or not according to the hand position information of the single person and the skeletal thermodynamic diagram of the single person.
On the basis of the above embodiment, as shown in fig. 7, the single hand position frame information includes second single hand position frame information and first single hand position frame information that are adjacent according to a time frame sequence, where the first single hand position frame information corresponds to the first splicing result, and after step 172, the glove detection method provided in the present application further includes:
step 174, splicing the first splicing result with the second single hand position frame information to obtain a second splicing result;
step 175, inputting the second splicing result into the glove detection model to obtain a third glove detection result.
Specifically, in the glove detection method provided by the application, the second splicing result of the input model comprises a first splicing result and second single hand position frame information, namely, the data of the input model simultaneously comprises the first splicing result and the second single hand position frame information of the frame before the first single hand position frame information corresponding to the first splicing result. For example, BGR averages of a full-image dataset and a person matting dataset of an image to be identified are calculated, respectively. And then, performing cyclic matting on a character data set of the image to be identified, generating an all-0 picture with the size of the image, filling by using the average value of the character matting data, and manufacturing a Gao Siyuan thermodynamic diagram with the size of a labeling frame of a hand or a glove as a radius R by taking the center point of single hand position frame information as a midpoint on the generated image. And then the single hand position frame information and the corresponding single thermodynamic diagram are spliced on the channel to obtain a first splicing result. And then, performing cyclic matting on the full-image data set of the image to be identified, generating an all-0 image with the image size, filling the full-image data set with the average value, restoring the thermodynamic diagram of the corresponding single person image to the corresponding position of the full-image, and recording the corresponding frame number of the thermodynamic diagram. And splicing the thermodynamic diagrams corresponding to the first splicing result and the second single hand position frame information to obtain a second splicing result. And when the first splicing result is the first frame image, directly splicing the generated average image with the first splicing result to obtain a second splicing result.
Wherein, the first stage model and the second stage model are modified correspondingly. For example, the input channels of the first stage model are changed from three channels to four channels, so as to simultaneously input the first splicing result and the thermodynamic diagram corresponding to the second single hand position frame information. The first stage model is added with a thermodynamic diagram of the same-layer neural network size, the multi-head multi-size thermodynamic diagram is up-sampled to the original diagram size, then the original diagram size is spliced, and then the channel fusion is carried out on the original diagram size through a convolution block to obtain a skeleton thermodynamic diagram with the channel number of 1, so that the channel splicing effect is realized. And the input channel of the second stage model is changed from a three-channel to a four-channel and a corresponding thermodynamic diagram is added, so that the detection of a second splicing result is realized.
On the basis of the embodiment, the prior information of the time sequence is added, so that the figure and skeleton thermodynamic diagram of the output of the model are detected more accurately, and the risk of false detection is reduced.
On the basis of the embodiment, as shown in fig. 8, the application further provides an oil static prevention glove detection example.
After the camera collects multiple frames of images to be identified which are arranged according to time sequence, the images to be identified are input into the first-stage network model frame by frame. The first-stage network model respectively acquires character detection information and a full-image skeleton thermodynamic diagram through an open-source character detection model and a skeleton detection model. Then, the first-stage network model gathers and acquires a plurality of single characteristics from the person detection information, and obtains a single corresponding area thermodynamic diagram from the full-image skeleton thermodynamic diagram according to the area coordinate information of the single characteristics. In order to ensure the accuracy of the full-image bone thermodynamic diagram, the full-image bone thermodynamic diagram of each frame of image to be identified is considered in combination with the full-image bone thermodynamic diagram of the previous frame of image to be identified, so that the accuracy degree of the bone thermodynamic diagram is improved. When the image to be identified is the first frame in the multi-frame, the full-image skeleton thermodynamic diagram and the average full-image skeleton thermodynamic diagram of the image to be identified of the first frame are combined to input a second-stage network model. And when the second-stage network model is input, channel splicing is carried out on the single-person characteristics and the regional thermodynamic diagrams corresponding to the single-person characteristics, and the spliced result is input into the second-stage network model to obtain glove detection information, namely glove detection results.
A second embodiment of the present application relates to a glove detecting device, as shown in fig. 9, comprising:
the person labeling module 201 is configured to obtain person coordinate position frame information and corresponding skeleton key point information in an image to be identified;
the hand labeling module 202 is configured to position the hand position frame information in the image to be identified according to the character coordinate position frame information and the skeletal key point information;
the first glove detecting module 203 is configured to perform glove recognition on the hand position frame through a pre-trained glove detecting model, and obtain a first glove detecting result.
In addition to the above embodiment, the person annotation module 201 includes:
a person position extraction unit 211 for extracting the person coordinate position frame information from the image to be recognized by a person detection model set in advance;
and a bone key point extracting unit 212, configured to extract bone key point information corresponding to the character coordinate position frame information from the image to be identified through a preset bone detection model.
Based on the above embodiment, the hand labeling module 202 includes:
a hand region labeling unit 221, configured to perform a hand region labeling action in the character coordinate position frame information according to the skeletal key point information, and obtain a hand region labeling result;
and a hand position frame selection unit 222, configured to obtain the hand position frame information from the character coordinate position frame information according to the hand region labeling result.
In addition to the above embodiment, the hand region labeling unit 221 includes:
an action judging subunit 223, configured to perform person action judgment according to the skeletal key point information, and obtain a person action judgment result;
an algorithm obtaining subunit 224, configured to screen, according to the person action determination result, a second action algorithm corresponding to the skeletal key point information from preset first action algorithms;
and the hand labeling subunit 225 is configured to perform a hand region labeling action in the character coordinate position frame information according to the skeletal key point information through the second action algorithm, and obtain a hand region labeling result.
On the basis of the above embodiment, the glove detecting device provided in the present application further includes:
a single person position extraction module 204, configured to obtain single person coordinate position frame information from a plurality of pieces of person coordinate position frame information;
a single skeleton key point extraction module 205, configured to obtain single skeleton key point information according to the single coordinate position frame information and the skeleton key point information;
a single hand position labeling module 206, configured to position single hand position frame information corresponding to the single coordinate position frame information according to the single coordinate position frame information and the single skeletal key point information;
and the second glove detection module 207 is configured to perform glove recognition on the single hand position frame through the glove detection model, and obtain a second glove detection result.
On the basis of the above embodiment, the second sleeve detection module 207 includes:
a thermodynamic diagram construction unit 271 for constructing a single thermodynamic diagram according to the single hand position frame information and the single skeletal key point information;
a first splicing unit 272, configured to perform a channel splicing operation on the single hand position frame information and the single thermodynamic diagram to obtain a first splicing result;
and a second detecting unit 273, configured to input the first splicing result into the glove detecting model, and obtain the second glove detecting result.
In addition to the above embodiment, the second sleeve detection module 207 further includes:
a second stitching unit 274, configured to stitch the first stitching result with the second single hand position frame information to obtain a second stitching result;
and the third detection unit 275 is configured to input the second splicing result into the glove detection model, and obtain a third glove detection result.
A third embodiment of the present application relates to a mobile terminal, as shown in fig. 10, including:
at least one processor 161; the method comprises the steps of,
a memory 162 communicatively coupled to the at least one processor 161; wherein,
the memory 162 stores instructions executable by the at least one processor 161 to enable the at least one processor 161 to implement the glove detection method of the first or second embodiments of the present application.
Where the memory and the processor are connected by a bus, the bus may comprise any number of interconnected buses and bridges, the buses connecting the various circuits of the one or more processors and the memory together. The bus may also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art, and therefore, will not be described any further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or may be a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor is transmitted over the wireless medium via the antenna, which further receives the data and transmits the data to the processor.
The processor is responsible for managing the bus and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And memory may be used to store data used by the processor in performing operations.
A fourth embodiment of the present application relates to a computer storage medium storing a computer program. The computer program, when executed by a processor, implements the glove detection method according to the first or second embodiment of the present application.
That is, it will be understood by those skilled in the art that all or part of the steps in implementing the methods of the embodiments described above may be implemented by a program stored in a storage medium, where the program includes several instructions for causing a device (which may be a single-chip microcomputer, a chip or the like) or a processor (processor) to perform all or part of the steps in the methods of the embodiments described herein. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a Read-only memory (ROM), a random access memory (RAM, randomAccessMemory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It is to be understood that the present application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.
Claims (8)
1. A glove detection method, the method comprising:
acquiring character coordinate position frame information and corresponding skeleton key point information in an image to be identified;
performing character action judgment according to the skeleton key point information to obtain character action judgment results;
screening a second action algorithm corresponding to the bone key point information from a preset first action algorithm according to the character action judging result;
according to the skeletal key point information, executing a hand region labeling action in the character coordinate position frame information through the second action algorithm, and obtaining a hand region labeling result;
acquiring hand position frame information from the character coordinate position frame information according to the hand region labeling result;
and performing glove identification on the hand position frame through a pre-trained glove detection model to obtain a first glove detection result.
2. The method of claim 1, wherein the acquiring the person coordinate position frame information and the corresponding bone keypoint information in the image to be identified comprises:
extracting the character coordinate position frame information from the image to be identified through a preset character detection model;
and extracting skeleton key point information corresponding to the character coordinate position frame information from the image to be identified through a preset skeleton detection model.
3. The method according to claim 1, wherein the number of pieces of person coordinate position frame information is plural, and the step of obtaining the person coordinate position frame information and the corresponding bone key point information in the image to be identified further includes:
acquiring single person coordinate position frame information from a plurality of pieces of person coordinate position frame information;
acquiring single-person skeleton key point information according to the single-person coordinate position frame information and the skeleton key point information;
positioning single hand position frame information corresponding to the single coordinate position frame information according to the single coordinate position frame information and the single skeleton key point information;
and performing glove identification on the single hand position frame through the glove detection model to obtain a second glove detection result.
4. The method of claim 3, wherein said performing glove recognition on said single hand position box with said glove detection model, obtaining a second glove detection result comprises:
constructing a single thermodynamic diagram according to the single hand position frame information and the single skeleton key point information;
performing channel splicing action on the single hand position frame information and the single thermodynamic diagram to obtain a first splicing result;
inputting the first splicing result into the glove detection model to obtain the second glove detection result.
5. The method of claim 4, wherein the single hand position frame information comprises second single hand position frame information and first single hand position frame information adjacent in time frame order, the first single hand position frame information corresponding to the first splice result, the performing a channel splice action on the single hand position frame information and the single thermodynamic diagram resulting in a first splice result, further comprising:
splicing the first splicing result with the second single hand position frame information to obtain a second splicing result;
inputting the second splicing result into the glove detection model to obtain a third glove detection result.
6. A glove detection apparatus, comprising:
the character labeling module is used for acquiring character coordinate position frame information and corresponding skeleton key point information in the image to be identified;
the action judging module is used for judging the action of the person according to the skeletal key point information and obtaining a judgment result of the action of the person;
the action algorithm acquisition module is used for screening a second action algorithm corresponding to the bone key point information from a preset first action algorithm according to the character action judging result;
the hand region standard module is used for executing hand region labeling actions in the character coordinate position frame information through the second action algorithm according to the skeleton key point information to obtain a hand region labeling result;
the hand position frame extraction module is used for acquiring hand position frame information from the character coordinate position frame information according to the hand region labeling result;
the first glove detection module is used for carrying out glove identification on the hand position frame through a pre-trained glove detection model, and obtaining a first glove detection result.
7. An electronic device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the glove detection method of any of claims 1-5.
8. A storage medium storing a computer program which, when executed by a processor, implements the glove detection method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311353217.6A CN117115861B (en) | 2023-10-19 | 2023-10-19 | Glove detection method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311353217.6A CN117115861B (en) | 2023-10-19 | 2023-10-19 | Glove detection method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117115861A CN117115861A (en) | 2023-11-24 |
| CN117115861B true CN117115861B (en) | 2024-01-26 |
Family
ID=88796827
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311353217.6A Active CN117115861B (en) | 2023-10-19 | 2023-10-19 | Glove detection method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117115861B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110751056A (en) * | 2019-09-27 | 2020-02-04 | 湖北工业大学 | Pedestrian motion prediction method based on improved top-down method multi-person posture detection |
| CN111222379A (en) * | 2018-11-27 | 2020-06-02 | 株式会社日立制作所 | Hand detection method and device |
| CN111507317A (en) * | 2020-06-30 | 2020-08-07 | 之江实验室 | Vision-based rotary equipment operation glove wearing detection method and system |
| CN111666917A (en) * | 2020-06-19 | 2020-09-15 | 北京市商汤科技开发有限公司 | Attitude detection and video processing method and device, electronic equipment and storage medium |
| CN111783741A (en) * | 2020-07-30 | 2020-10-16 | 国网江苏省电力有限公司南通供电分公司 | Key element positioning insulating glove use identification method, special identification device and electronic equipment |
| CN112004061A (en) * | 2020-09-03 | 2020-11-27 | 四川弘和通讯有限公司 | Oil discharge flow normative intelligent monitoring method based on computer vision |
| CN113761947A (en) * | 2020-06-04 | 2021-12-07 | 北京易智时代数字科技有限公司 | Virtual simulation multi-person interactive system |
| WO2023007601A1 (en) * | 2021-07-28 | 2023-02-02 | 日本電気株式会社 | Operation detection system, operation detection method, and non-transitory computer-readable medium |
| CN115830635A (en) * | 2022-12-09 | 2023-03-21 | 南通大学 | PVC glove identification method based on key point detection and target identification |
| KR20230040849A (en) * | 2021-09-16 | 2023-03-23 | 국민대학교산학협력단 | Method and apparatus for classifying action based on hand tracking |
| CN116189311A (en) * | 2023-04-27 | 2023-05-30 | 成都愚创科技有限公司 | Protective clothing wears standardized flow monitoring system |
| CN116563949A (en) * | 2023-07-05 | 2023-08-08 | 四川弘和数智集团有限公司 | Behavior recognition method, device, equipment and medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170177087A1 (en) * | 2015-12-18 | 2017-06-22 | Intel Corporation | Hand skeleton comparison and selection for hand and gesture recognition with a computing interface |
-
2023
- 2023-10-19 CN CN202311353217.6A patent/CN117115861B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111222379A (en) * | 2018-11-27 | 2020-06-02 | 株式会社日立制作所 | Hand detection method and device |
| CN110751056A (en) * | 2019-09-27 | 2020-02-04 | 湖北工业大学 | Pedestrian motion prediction method based on improved top-down method multi-person posture detection |
| CN113761947A (en) * | 2020-06-04 | 2021-12-07 | 北京易智时代数字科技有限公司 | Virtual simulation multi-person interactive system |
| CN111666917A (en) * | 2020-06-19 | 2020-09-15 | 北京市商汤科技开发有限公司 | Attitude detection and video processing method and device, electronic equipment and storage medium |
| CN111507317A (en) * | 2020-06-30 | 2020-08-07 | 之江实验室 | Vision-based rotary equipment operation glove wearing detection method and system |
| CN111783741A (en) * | 2020-07-30 | 2020-10-16 | 国网江苏省电力有限公司南通供电分公司 | Key element positioning insulating glove use identification method, special identification device and electronic equipment |
| CN112004061A (en) * | 2020-09-03 | 2020-11-27 | 四川弘和通讯有限公司 | Oil discharge flow normative intelligent monitoring method based on computer vision |
| WO2023007601A1 (en) * | 2021-07-28 | 2023-02-02 | 日本電気株式会社 | Operation detection system, operation detection method, and non-transitory computer-readable medium |
| KR20230040849A (en) * | 2021-09-16 | 2023-03-23 | 국민대학교산학협력단 | Method and apparatus for classifying action based on hand tracking |
| CN115830635A (en) * | 2022-12-09 | 2023-03-21 | 南通大学 | PVC glove identification method based on key point detection and target identification |
| CN116189311A (en) * | 2023-04-27 | 2023-05-30 | 成都愚创科技有限公司 | Protective clothing wears standardized flow monitoring system |
| CN116563949A (en) * | 2023-07-05 | 2023-08-08 | 四川弘和数智集团有限公司 | Behavior recognition method, device, equipment and medium |
Non-Patent Citations (4)
| Title |
|---|
| A Novel Data Glove for Hand-object Pressure Capture;Xi Zhao等;《2021 36th Youth Academic Annual Conference of Chinese Association of Automation (YAC)》;459-464 * |
| 动态手势识别和跟踪算法研究;刘玉鹏;《中国优秀硕士学位论文全文数据库信息科技辑》(第09期);I138-1076 * |
| 基于卷积神经网络的手语识别算法研究及部署;赵金龙;《中国优秀硕士学位论文全文数据库信息科技辑》(第03期);I138-1602 * |
| 基于骨架信息的人体动作识别与实时交互技术;张继凯 等;《内蒙古科技大学学报》(第03期);66-72 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117115861A (en) | 2023-11-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109902659B (en) | Method and apparatus for processing human body image | |
| CN109690570B (en) | Computer room management method and dynamic loop system | |
| CN109085174A (en) | Display screen peripheral circuit detection method, device, electronic equipment and storage medium | |
| CN110458895A (en) | Conversion method, device, equipment and the storage medium of image coordinate system | |
| CN111062303A (en) | Image processing method, system and computer storage medium | |
| CN112101123B (en) | Attention detection method and device | |
| CN109410192A (en) | A kind of the fabric defect detection method and its device of multi-texturing level based adjustment | |
| CN111626243A (en) | Identity recognition method and device for face covered by mask and storage medium | |
| CN113435236A (en) | Home old man posture detection method, system, storage medium, equipment and application | |
| CN115620241B (en) | Image processing-based field safety measure identification method and device | |
| CN113052127A (en) | Behavior detection method, behavior detection system, computer equipment and machine readable medium | |
| CN117726991B (en) | High-altitude hanging basket safety belt detection method and terminal | |
| CN114937293A (en) | Agricultural service management method and system based on GIS | |
| CN117115861B (en) | Glove detection method and device, electronic equipment and storage medium | |
| CN112925941A (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN117158955A (en) | User safety intelligent monitoring method based on wearable monitoring equipment | |
| CN112215567A (en) | Production flow compliance checking method and system, storage medium and terminal | |
| CN114067256B (en) | Wi-Fi signal-based human body key point detection method and system | |
| CN115438945A (en) | Risk identification method, device, equipment and medium based on power equipment inspection | |
| CN106681868A (en) | Image data testing method and system | |
| CN111932581B (en) | Safety rope detection method, device, electronic equipment and readable storage medium | |
| CN115019394A (en) | Process tracking method, device, equipment and storage medium based on equipment maintenance | |
| CN114519804A (en) | Human body skeleton labeling method and device and electronic equipment | |
| CN113469150A (en) | Method and system for identifying risk behaviors | |
| CN112434560A (en) | Safety equipment real-time detection method and device based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |