CN112733730B - Oil extraction operation field smoke suction personnel identification processing method and system - Google Patents
Oil extraction operation field smoke suction personnel identification processing method and system Download PDFInfo
- Publication number
- CN112733730B CN112733730B CN202110038437.4A CN202110038437A CN112733730B CN 112733730 B CN112733730 B CN 112733730B CN 202110038437 A CN202110038437 A CN 202110038437A CN 112733730 B CN112733730 B CN 112733730B
- Authority
- CN
- China
- Prior art keywords
- data
- type
- information
- prediction
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
- G06V10/464—Salient features, e.g. scale invariant feature transforms [SIFT] using a plurality of salient features, e.g. bag-of-words [BoW] representations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Abstract
The invention provides a method and a system for identifying and processing smokers in an oil production operation field, which belong to the technical field of machine vision, and are used for calling a camera to take pictures of the oil production operation field to obtain an environmental picture; determining whether the environment photo has cigarette ends by using a recognition model; the recognition model is obtained by using multiple groups of data through machine learning training; the multiple groups of data comprise a first type of data and a second type of data; each set of data in the first type of data includes: a photograph containing butts and a label identifying the contained butts in the photograph; each group of the second type data comprises: a photograph not containing butts and a label identifying that no butts are contained in the photograph; and under the condition that the cigarette ends exist in the environment picture, judging that the smoking behavior of personnel on the site of oil extraction operation exists, and sending out an early warning signal. The invention extracts the multi-scale features of the image, obtains the multi-layer feature mapping and obtains the feature pyramid, solves the problem of scale change and can quickly and accurately identify the cigarette ends in the scene photo.
Description
Technical Field
The invention relates to the technical field of machine vision, in particular to an oil extraction operation field smoking person identification processing method and system based on anchor-free and multi-scale feature self-adaptive fusion.
Background
Because the field environment of oil extraction operation is complex, the nonstandard operation of oil extraction constructors has great potential safety hazard, and particularly, the smoke problem of the constructors can cause great accidents. The method can accurately detect the smoking behavior of personnel on the oil extraction site in real time and initiate an alarm at the first time, and has important significance for ensuring the production safety on the oil extraction site. The cigarette end in the field video image is quickly and accurately detected, so that the cigarette end is important for identifying whether field personnel smoke or not, and the cigarette end belongs to the small target problem and is difficult to detect.
In the field of computer vision, a plurality of targets usually exist in one picture, the sizes and postures of the targets are different, and the multi-scale problem, especially the small target problem, hinders the improvement of the detection precision all the time. Compared with other targets, small targets have the problems of low resolution, fuzziness and less carried information, so that the detection network has weak capability of extracting the features of the targets, and is difficult to acquire enough features to accurately position and label the targets.
The target detectors currently employed are at the expense of accuracy or speed. In recent decades, due to the advent of Convolutional Neural Networks (CNNs), anchor-based two-stage and one-stage target detection algorithms have been greatly improved in terms of accuracy and speed, respectively. However, two-stage detectors achieve better accuracy but are slow, while one-stage detectors have high efficiency but lower accuracy. Therefore, it is often beneficial to fuse different detection frameworks or methods to exploit their advantages and overcome their disadvantages.
In order to solve the problems of large parameter and limited scale of the CNN, the Full Convolution Network (FCN) realizes parameter sharing, can classify the image at pixel level, can accept the input image with any size, and uses the deconvolution layer to up-sample the feature map of the last convolution layer to restore it to the same size of the input image, so as to generate a prediction for each pixel while preserving the spatial information in the original input image, but the Full Convolution Network (FCN) does not fully consider the relationship between pixels.
Disclosure of Invention
The invention aims to provide an oil extraction operation field smokers identification processing method and system based on anchor-free and multi-scale feature self-adaptive fusion, so as to solve at least one technical problem in the background technology.
In order to achieve the purpose, the invention adopts the following technical scheme:
in one aspect, the invention provides a method for smoke detection, identification and processing of personnel in an oil extraction operation field, which comprises the following steps:
a camera is called to take a picture of an oil extraction operation site to obtain an environmental picture;
analyzing the environment photo by using a recognition model to determine whether the environment photo contains cigarette ends; wherein the recognition model is obtained by machine learning training by using multiple groups of data; the multiple groups of data comprise a first type of data and a second type of data; each set of data in the first type of data comprises: a photograph containing butts and a label identifying the contained butts in the photograph; each set of data in the second class of data includes: a photograph containing no butts and a label identifying that no butts are contained in the photograph;
and under the condition that the cigarette ends exist in the environment picture, judging that the personnel on the site of oil extraction operation have smoking behaviors, and sending out an early warning signal.
Preferably, training the recognition model through machine learning using the plurality of sets of data comprises:
respectively acquiring first prediction information of each pixel of a picture in the first type of data and second prediction information of each pixel of the picture in the second type of data;
respectively calculating a first error of the first prediction information and the label information of the first type of data and a second error of the second prediction information and the label information of the second type of data by using a loss function;
and optimizing the parameters of the pre-constructed detection identification network by adopting a back propagation method until the first error and the second error respectively reach respective expected values to obtain the identification model.
Preferably, the acquiring the first prediction information and the second prediction information includes:
respectively extracting a plurality of first characteristic information of the photos in the first type of data and a plurality of second characteristic information of the photos in the second type of data;
respectively combining the plurality of first characteristic information and the plurality of second characteristic information to obtain first fusion characteristic information and second fusion characteristic information;
and respectively acquiring first prediction information of each pixel of the picture in the first type of data and second prediction information of each pixel of the picture in the second type of data according to the first fusion characteristic information and the second fusion characteristic information.
Preferably, the first prediction information and the second prediction information each include category information, center offset information, and regression information of each pixel in the corresponding photograph.
Preferably, the first error and the second error each include a sum of classification errors, a sum of regression errors, and a sum of center point offsets of each pixel in the corresponding photograph.
Preferably, the loss function is:
wherein p is x,y 、t x,y A two-dimensional vector of classification labels and a four-dimensional vector of bounding box coordinates representing the position (x, y) of each pixel, respectively; n is a radical of pos The number of pixels representing the entire image;
representing the sum of the classification losses for each pixel; l is cls 、tc x,y Class labels representing the focal loss and the truth box for each location (x, y), respectively; p { tc x,y >0 represents the probability of each pixel being a positive sample, if tc x,y >0, then p =1, otherwise p =0; l is a radical of an alcohol reg 、tr x,y Training regression representing IOU loss and each pixel location (x, y), respectivelyA target; l is a radical of an alcohol BCE A two-class cross entropy loss function representing center shift.
In a second aspect, the present invention provides a system for smoke detection, identification and processing of personnel in an oil production operation field, the system comprising:
the acquisition module is used for calling the camera to take a picture of an oil extraction operation site to acquire an environmental picture;
the recognition module is used for analyzing the environment picture by using a recognition model and determining whether cigarette ends exist in the environment picture; wherein the recognition model is obtained by machine learning training by using multiple groups of data; the multiple groups of data comprise first type data and second type data; each set of data in the first type of data comprises: a photo containing cigarette ends and a label for identifying the cigarette ends contained in the photo; each set of data in the second class of data includes: a photograph containing no butts and a label identifying that no butts are contained in the photograph;
and the judgment module is used for judging that the personnel on the site of oil extraction work have smoking behavior under the condition that the cigarette ends exist in the environment picture and sending out an early warning signal.
Preferably, the identification module comprises a prediction unit, a calculation unit and an optimization unit;
the prediction unit is used for respectively acquiring first prediction information of each pixel of a photo in the first type of data and second prediction information of each pixel of the photo in the second type of data by utilizing a pre-constructed detection and identification network;
the calculation unit is used for calculating a first error between the first prediction information and the label information of the first type of data and a second error between the second prediction information and the label information of the second type of data by using a loss function;
and the optimization unit is used for optimizing the parameters of the pre-constructed detection identification network by adopting a back propagation method until the first error and the second error respectively reach respective expected values, so as to obtain the identification model.
Preferably, the pre-constructed detection recognition network comprises a backbone network, a neck part, a feature mapping part and a head part;
the backbone network adopts a ResNet-50 network and is used for extracting and storing first characteristic information of photos in the first type of data and second characteristic information of photos in the second type of data;
the neck comprises a plurality of self-adaptive spatial pooling modules, wherein the self-adaptive spatial pooling modules comprise self-adaptive pooling and self-adaptive spatial fusion and are used for increasing the receptive fields of the first characteristic information and the second characteristic information to respectively obtain a first characteristic mapping and a second characteristic mapping;
the feature mapping part performs adaptive spatial fusion and residual connection on the first feature mapping and the second feature mapping through an adaptive pyramid to obtain a final feature pyramid;
the header is mainly composed of a plurality of prediction subnetworks for outputting the first prediction information and the second prediction information.
Preferably, the prediction sub-network comprises parallel classification branches and regression branches, and a center offset sub-branch is added to the classification branches to block the bounding box generated by a position far away from the target center point.
The invention has the beneficial effects that: the recognition model has simple structure, easy training, less calculation amount and convenient optimization, and can quickly and accurately recognize the cigarette ends in the scene photos; multi-scale features of the image are extracted, a plurality of self-adaptive spatial pooling modules are inserted, and the acceptable field of the features is increased to obtain multilayer feature mapping; the self-adaptive pyramid module is used for fusing multilayer feature mapping to obtain a feature pyramid, so that the problem of scale change is solved, and simultaneously, multi-scale context information is obtained to establish the relation between pixels, thereby being beneficial to accurate and rapid detection of small targets; the prediction information of each pixel of the image is obtained through each layer of features of the feature pyramid by a head, so that the use of anchors is eliminated, and the memory and the complex calculation are reduced; and fusing the plurality of pieces of prediction information, removing redundant bounding boxes through non-maximum suppression (NMS), and outputting a final detection result.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic block diagram of a system for detecting, identifying and processing smoking of personnel in an oil recovery operation site according to embodiment 1 of the present invention.
Fig. 2 is a flowchart of a smoking detection and identification processing method for personnel at an oil recovery operation site according to embodiment 3 of the present invention.
Fig. 3 is a flowchart of a smoking detection and identification processing method for personnel at an oil recovery operation site according to embodiment 4 of the present invention.
Fig. 4 is a schematic network structure diagram of an oil recovery operation field personnel smoking detection network according to embodiment 4 of the present invention.
Fig. 5 is a schematic structural diagram of an adaptive pooling module of a smoking detection network for personnel at an oil recovery operation site according to embodiment 4 of the present invention.
Fig. 6 is a schematic view of a sub-detection network structure of the oil recovery operation field personnel smoking detection network according to embodiment 4 of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below by way of the drawings are illustrative only and are not to be construed as limiting the invention.
It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
In the description of the specification, reference to the description of "one embodiment," "some embodiments," "an example," "a specific example," or "some examples" or the like means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
For the purpose of facilitating an understanding of the present invention, the present invention will be further explained by way of specific embodiments with reference to the accompanying drawings, which are not intended to limit the present invention.
It should be understood by those skilled in the art that the drawings are merely schematic representations of embodiments and that the elements shown in the drawings are not necessarily required to practice the invention.
Example 1
As shown in fig. 1, an embodiment 1 of the present invention provides a system for detecting, identifying and processing smoking of personnel in an oil production operation field, the system including:
the acquisition module is used for calling the camera to take a picture of an oil extraction operation site to acquire an environmental picture;
the recognition module is used for analyzing the environment photo by using a recognition model and determining whether the cigarette ends exist in the environment photo; wherein the recognition model is obtained by machine learning training by using multiple groups of data; the multiple groups of data comprise a first type of data and a second type of data; each set of data in the first type of data comprises: a photo containing cigarette ends and a label for identifying the cigarette ends contained in the photo; each set of data in the second class of data includes: a photograph not containing butts and a label identifying that no butts are contained in the photograph;
and the judgment module is used for judging that the personnel on the site of oil extraction work have smoking behavior under the condition that the cigarette ends exist in the environment picture and sending out an early warning signal.
In this embodiment 1, the identification module includes a prediction unit, a calculation unit, and an optimization unit.
The prediction unit is used for respectively acquiring first prediction information of each pixel of a photo in the first type of data and second prediction information of each pixel of the photo in the second type of data by utilizing a pre-constructed detection and identification network;
the calculation unit is used for calculating a first error between the first prediction information and the label information of the first type of data and a second error between the second prediction information and the label information of the second type of data by using a loss function;
and the optimization unit is used for optimizing the parameters of the pre-constructed detection identification network by adopting a back propagation method until the first error and the second error respectively reach respective expected values, so as to obtain the identification model.
In this example 1. The pre-constructed detection and identification network comprises a backbone network, a neck part, a feature mapping part and a head part;
the backbone network adopts a ResNet-50 network and is used for extracting and storing first characteristic information of photos in the first type of data and second characteristic information of photos in the second type of data;
the neck comprises a plurality of self-adaptive spatial pooling modules, wherein the self-adaptive spatial pooling modules comprise self-adaptive pooling and self-adaptive spatial fusion and are used for increasing the receptive fields of the first characteristic information and the second characteristic information to respectively obtain a first characteristic mapping and a second characteristic mapping;
the feature mapping part performs adaptive spatial fusion and residual connection on the first feature mapping and the second feature mapping through an adaptive pyramid to obtain a final feature pyramid;
the header is mainly composed of a plurality of prediction subnetworks for outputting the first prediction information and the second prediction information.
The prediction sub-network comprises parallel classification branches and regression branches, and a center offset sub-branch is added in the classification branches for blocking a bounding box generated by a position far away from a target center point.
In this embodiment 1, the system for detecting, identifying and processing smoking of personnel in the oil recovery operation field is used to implement a method for detecting, identifying and processing smoking of personnel in the oil recovery operation field, and the method includes the following steps:
a camera is called to take a picture of an oil extraction operation site to obtain an environmental picture;
analyzing the environment picture by using a recognition model to determine whether cigarette ends exist in the environment picture; wherein the recognition model is obtained by machine learning training by using multiple groups of data; the multiple groups of data comprise a first type of data and a second type of data; each set of data in the first type of data comprises: a photograph containing butts and a label identifying the contained butts in the photograph; each set of data in the second class of data includes: a photograph containing no butts and a label identifying that no butts are contained in the photograph;
and under the condition that the cigarette ends exist in the environment picture, judging that the personnel on the site of oil extraction operation have smoking behaviors, and sending out an early warning signal.
Training the recognition model through machine learning using multiple sets of data includes:
respectively acquiring first prediction information of each pixel of a picture in the first type of data and second prediction information of each pixel of the picture in the second type of data;
respectively calculating a first error of the first prediction information and the label information of the first type of data and a second error of the second prediction information and the label information of the second type of data by using a loss function;
and optimizing the parameters of the pre-constructed detection identification network by adopting a back propagation method until the first error and the second error respectively reach respective expected values to obtain the identification model.
The acquiring the first prediction information and the second prediction information includes:
respectively extracting a plurality of first characteristic information of the photos in the first type of data and a plurality of second characteristic information of the photos in the second type of data;
respectively combining the plurality of first characteristic information and the plurality of second characteristic information to obtain first fusion characteristic information and second fusion characteristic information;
and respectively acquiring first prediction information of each pixel of the picture in the first type of data and second prediction information of each pixel of the picture in the second type of data according to the first fusion characteristic information and the second fusion characteristic information.
The first prediction information and the second prediction information each include category information, center offset information, and regression information of each pixel in the corresponding photograph.
The first error and the second error each include a sum of classification errors, a sum of regression errors, and a sum of center point offsets for each pixel in the corresponding photograph.
The loss function is:
wherein p is x,y 、t x,y A two-dimensional vector of classification labels and a four-dimensional vector of bounding box coordinates representing the position (x, y) of each pixel, respectively; n is a radical of pos Representing the number of pixels of the entire image;
representing the sum of the classification losses for each pixel; l is cls 、tc x,y Class labels representing the focal loss and the truth box for each location (x, y), respectively; p { tc x,y >0 represents the probability of each pixel being a positive sample, if tc x,y >0, then p =1, otherwise p =0; l is reg 、tr x,y Training regression targets representing IOU loss and each pixel location (x, y), respectively; l is BCE A two-class cross-entropy loss function representing center offset.
Example 2
The embodiment 2 of the invention provides an oil extraction operation field personnel smoking detection and identification processing method based on anchor-free and multi-scale feature self-adaptive fusion, which comprises the following steps:
acquiring front view images with or without butts in a field environment by using a camera installed in an oil extraction operation field as a binary data set, preprocessing the front view images, and then labeling to obtain label information of the front view images; inputting the information into a pre-constructed smoking detection network of an initial oil extraction operation site, extracting a plurality of pieces of characteristic information, and combining the plurality of pieces of characteristic information to obtain fused characteristic information;
acquiring the prediction information of each pixel of the front view image according to the fusion characteristic information, and calculating the error between the prediction information and the label information by using a loss function;
and optimizing the parameters of the smoking detection network of the initial oil extraction operation site by adopting a back propagation method until the error reaches an expected value to obtain the trained smoking detection network of the personnel of the oil extraction operation site.
And acquiring a scene environment picture as a front view image to be detected by using a camera installed on an oil extraction operation scene, inputting the front view image to the trained smoking detection network of the personnel on the oil extraction operation scene, and outputting the detection information of the front view image to be detected.
And judging whether the personnel smokes on the site of the oil extraction operation according to the type in the detection information, and if the cigarette end is detected in the environment picture, judging that the personnel smokes and giving a warning.
The pre-processing the elevation image comprises: the method comprises the following steps of deblurring, illumination distortion, cutting, horizontal turning, rotation, random cutting of the image into 640-800 pixels and the like, and mainly aims to enable the front view image to be clearer, increase the number of data sets and improve the identification precision.
The prediction information of each pixel includes: the category information, the center offset information and the regression information of each pixel;
the category information of each pixel is an H multiplied by W multiplied by C three-dimensional matrix; c is the category number of the data set; the center offset information of each pixel is a three-dimensional matrix H multiplied by W multiplied by 1; the regression information of each pixel is an H multiplied by W multiplied by 4 three-dimensional matrix;
further preferably, the loss function is:
wherein p is x,y 、t x,y A two-dimensional vector of classification labels and a four-dimensional vector of bounding box coordinates representing the position (x, y) of each pixel, respectively; n is a radical of pos Representing the number of pixels of the entire image;
representing the sum of the classification losses for each pixel; l is cls 、tc x,y Class labels representing the focal loss and the truth box for each location (x, y), respectively; p { tc x,y >0 represents the probability of each pixel being a positive sample, if tc x,y >0, then p =1, otherwise p =0; l is reg 、tr x,y Training regression targets representing IOU loss and each pixel location (x, y), respectively; l is BCE A two-class cross-entropy loss function representing center offset.
In this embodiment 2, the oil recovery operation field personnel smoking detection network includes:
the backbone network is used for extracting multi-scale characteristic information of the front view image;
the neck part is used for respectively obtaining feature mapping of the multi-scale feature information through a self-adaptive spatial pooling module;
obtaining a characteristic pyramid by the characteristic mapping through a self-adaptive pyramid module;
the head part is used for obtaining a plurality of specific prediction information from each layer of characteristics of the characteristic pyramid;
and fusing the plurality of pieces of prediction information to obtain detection information of the front view image.
In this embodiment 2, the backbone network is ResNet-50, and the multi-scale feature information is feature information of 100 × 128, 50 × 64, and 25 × 32 pixels obtained by performing 8-fold, 16-fold, and 32-fold down-sampling on an original image of 800 × 1024 pixels in three-layer scale;
the neck comprises adaptive pooling and adaptive spatial fusion; the adaptive pyramid module comprises adaptive spatial fusion and residual connection; the header portion comprises a plurality of prediction subnetworks; the plurality of prediction subnetworks are composed of a plurality of identical classification branches and regression branches; the regression branch and the classification branch are parallel and share parameters; the detection information is the category and the detection bounding box of the target in the front view image.
Example 3
As shown in fig. 2, embodiment 3 of the present invention provides an oil recovery operation field personnel smoking detection and identification processing method based on anchor-free and multi-scale feature adaptive fusion, which specifically includes the following operation steps:
step S11: and deploying the smoking detection network of the personnel on the oil extraction operation site which is trained in advance to a server on the oil extraction operation site.
Step S12: the video stream generated by a camera of an oil extraction operation field is accessed to the server through the central processing unit, the obtained video of a constructor of the oil extraction operation field is input to a smoking detection network of an oil extraction operation field constructor who is trained in advance, and the detection information of the video is output.
Step S13: and judging whether the oil extraction operation site has a cigarette end or not according to the detection information of the video.
Step S14: and according to the judgment result, if the cigarette end exists in the oil extraction operation site, judging that the smoking behavior of the constructor in the oil extraction operation site exists, and acquiring the smoking position of the constructor in the oil extraction operation site according to the camera ID of the oil extraction operation site.
Step S15: and producing an early warning signal according to the smoking position of the oil extraction operation site constructor and sending the early warning signal to an oil extraction operation site monitoring operator.
In this embodiment 3, the oil extraction job site worker smoking detection method based on anchor-free and multi-scale feature adaptive fusion is applied to the oil extraction job site worker smoking detection system, and after the detection information of the video is acquired by using the oil extraction job site worker smoking detection network, the smoking behavior of the oil extraction job site worker is judged and warned according to the detection information.
Example 4
The embodiment 4 of the invention provides an oil extraction operation field personnel smoking detection identification processing method based on anchor-free and multi-scale feature self-adaptive fusion, which effectively improves the precision of small target detection and simultaneously keeps real-time detection.
As shown in fig. 3, in this embodiment 4, the smoking detection, identification and processing method for personnel in the oil recovery operation field specifically includes the following operation steps:
step S01: acquiring a field environment front view image containing cigarette ends and a field environment front view image not containing cigarette ends as a two-classification training set by using a camera installed on an oil extraction operation field, marking the front view images to obtain label information, and then preprocessing an initial image;
the pretreatment operation mainly comprises the following steps: deblurring, illumination distortion, cutting, horizontal turning, rotating, normalization processing and random cutting of the image into 640 pixels to 800 pixels and the like, mainly makes the front view image clearer, increases the number of data sets and improves the identification precision.
Step S02: inputting the front view image into a pre-constructed smoking detection network of initial oil extraction operation field personnel, extracting a plurality of front view characteristic information and outputting prediction information;
as shown in fig. 4, the pre-constructed smoking detection network for personnel in the initial oil extraction operation field provided in this embodiment 4 is an end-to-end one-stage anchor-free detection network, and includes a backbone network, a neck portion, a feature map, a feature pyramid, and a head portion.
The detection network is based on full convolution for predicting the class information and regression information for each pixel in the image.
The backbone network adopts a ResNet-50 network and is used for extracting and storing characteristic information of construction personnel smoking and non-smoking front view images acquired by a camera on an oil extraction operation site.
As shown in fig. 5, the neck mainly includes a plurality of adaptive spatial pooling modules, which include adaptive pooling and adaptive spatial fusion, for increasing the receptive field of the feature information to obtain the feature map.
The feature mapping is performed by fusing multiple scales through a self-adaptive pyramid module to obtain a final feature pyramid, and the self-adaptive pyramid module mainly comprises self-adaptive spatial fusion and residual connection.
The header is mainly composed of a plurality of prediction subnetworks, as shown in fig. 6. The prediction sub-network shown contains parallel classification branches and regression branches, in which a center-shifted sub-branch is added to block low quality bounding boxes that result from locations away from the target center point.
Step S03: calculating an error of the prediction information and the data set label information by a loss function.
The errors mainly comprise the sum of classification errors, the sum of regression errors and the sum of center point offsets of each pixel of the front view image.
Step S04: and optimizing parameters of the smoking detection network of the personnel on the site of the initial oil extraction operation by adopting a back propagation algorithm so that the error between a predicted value output by the network and a label value reaches a minimum expected value, finishing the training of the smoking detection network of the personnel on the site of the initial oil extraction operation, and obtaining the smoking detection network of the personnel on the site of the oil extraction operation.
Step S05: and inputting the front view image to be detected acquired by the camera on the oil extraction operation site into the smoking detection network of the personnel on the oil extraction operation site, and outputting the detection information of the front view image to be detected.
In the oil extraction operation site personnel smoking detection recognition processing method based on the anchor-free and multi-scale feature adaptive fusion provided by the embodiment 4, a set of anchor-free one-stage pixel-level detection network architecture is developed, the use of anchors is eliminated, the quantity of parameters and complex calculation related to the anchors are reduced, and an adaptive pooling module is innovatively introduced into the network architecture and is used for increasing the receptive field of features; and the self-adaptive pyramid module is used for fusing multi-scale features to acquire multi-scale context information, so that the detection precision of the small target is greatly improved, and the real-time detection is kept. Particularly, under the scene of a complex oil extraction operation site, the smoke of constructors can be well detected, and the arrangement and implementation are easy.
Example 5
The embodiment 5 of the present invention provides a computer device, which includes a memory and a processor, wherein the processor and the memory are in communication with each other, the memory stores a program instruction executable by the processor, and the processor calls the program instruction to execute a method for identifying and processing smoking detection of personnel in an oil production job site, and the method includes the following steps:
a camera is called to take a picture of an oil extraction operation field to obtain an environmental picture;
analyzing the environment picture by using a recognition model to determine whether cigarette ends exist in the environment picture; wherein the recognition model is obtained by machine learning training by using a plurality of groups of data; the multiple groups of data comprise first type data and second type data; each set of data in the first type of data comprises: a photo containing cigarette ends and a label for identifying the cigarette ends contained in the photo; each set of data in the second class of data includes: a photograph containing no butts and a label identifying that no butts are contained in the photograph;
and under the condition that the cigarette ends exist in the environment picture, judging that the personnel on the site of oil extraction operation have smoking behaviors, and sending out an early warning signal.
Example 6
An embodiment 6 of the present invention provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method for detecting and identifying smoking of a person in an oil recovery operation field is implemented, the method including the following steps:
a camera is called to take a picture of an oil extraction operation site to obtain an environmental picture;
analyzing the environment photo by using a recognition model to determine whether the environment photo contains cigarette ends; wherein the recognition model is obtained by machine learning training by using multiple groups of data; the multiple groups of data comprise first type data and second type data; each set of data in the first type of data comprises: a photograph containing butts and a label identifying the contained butts in the photograph; each set of data in the second class of data includes: a photograph containing no butts and a label identifying that no butts are contained in the photograph;
and under the condition that the cigarette ends exist in the environment picture, judging that the personnel on the site of oil extraction operation have smoking behaviors, and sending out an early warning signal.
In summary, in the oil recovery operation site personnel smoking detection identification processing method in the embodiment of the invention, the front view image to be detected acquired by the camera in the oil recovery operation site is input to the oil recovery operation site personnel smoking detection network, and the oil recovery operation site personnel smoking detection information of the front view image to be detected is output. And judging whether the oil extraction site constructor has smoking behavior according to the smoking detection information of the oil extraction site constructor, and generating an early warning signal according to the judgment result.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure, and various modifications and changes may be made to the present disclosure by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.
Although the present disclosure has been described with reference to the specific embodiments shown in the drawings, it is not intended to limit the scope of the present disclosure, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive faculty based on the technical solutions disclosed in the present disclosure.
Claims (6)
1. A smoke detection, identification and processing method for personnel on an oil extraction operation site is characterized by comprising the following steps:
a camera is called to take a picture of an oil extraction operation site to obtain an environmental picture;
analyzing the environment photo by using a recognition model to determine whether the environment photo contains cigarette ends; wherein the recognition model is obtained by machine learning training by using a plurality of groups of data; the multiple groups of data comprise a first type of data and a second type of data; each set of data in the first type of data comprises: a photograph containing butts and a label identifying the contained butts in the photograph; each group of the second type data comprises: a photograph containing no butts and a label identifying that no butts are contained in the photograph; training the recognition model through machine learning using multiple sets of data includes:
respectively acquiring first prediction information of each pixel of a photo in the first type of data and second prediction information of each pixel of the photo in the second type of data by utilizing a pre-constructed detection and identification network; the pre-constructed detection and identification network comprises a backbone network, a neck part, a feature mapping part and a head part; the backbone network adopts a ResNet-50 network and is used for extracting and storing first characteristic information of photos in the first type of data and second characteristic information of photos in the second type of data; the neck comprises a plurality of self-adaptive spatial pooling modules, wherein the self-adaptive spatial pooling modules comprise self-adaptive pooling and self-adaptive spatial fusion and are used for increasing the receptive fields of the first characteristic information and the second characteristic information to respectively obtain a first characteristic mapping and a second characteristic mapping; the feature mapping part performs adaptive spatial fusion and residual connection on the first feature mapping and the second feature mapping through an adaptive pyramid to obtain a final feature pyramid; the header is mainly composed of a plurality of prediction subnetworks for outputting the first prediction information and the second prediction information; the prediction sub-network comprises parallel classification branches and regression branches, and a center offset sub-branch is added in each classification branch and is used for preventing a boundary box generated by a position far away from a target center point;
respectively calculating a first error of the first prediction information and the label information of the first type of data and a second error of the second prediction information and the label information of the second type of data by using a loss function;
optimizing the parameters of the pre-constructed detection identification network by adopting a back propagation method until the first error and the second error respectively reach respective expected values to obtain the identification model;
and under the condition that the cigarette ends exist in the environment picture, judging that smoking behaviors exist in the personnel on the site of oil extraction operation, and sending out an early warning signal.
2. The method for detecting, identifying and processing smoking by personnel in oil recovery operation field according to claim 1, wherein the obtaining of the first prediction information and the second prediction information comprises:
respectively extracting a plurality of first characteristic information of the photos in the first type of data and a plurality of second characteristic information of the photos in the second type of data;
respectively combining the plurality of first characteristic information and the plurality of second characteristic information to obtain first fusion characteristic information and second fusion characteristic information;
and respectively acquiring first prediction information of each pixel of the picture in the first type of data and second prediction information of each pixel of the picture in the second type of data according to the first fusion characteristic information and the second fusion characteristic information.
3. The method of claim 2, wherein the first prediction information and the second prediction information each include category information, center offset information, and regression information for each pixel in the corresponding photo.
4. The method of claim 3, wherein the first error and the second error each comprise a sum of classification errors, a sum of regression errors, and a sum of center point offsets for each pixel in the corresponding photo.
5. The method for detecting, identifying and processing smoking by personnel in oil extraction operation field according to claim 1, wherein the loss function is:
wherein the content of the first and second substances,p x,y 、t x,y respectively representing the position of each pixel (x,y) The two-dimensional vector of the classification label and the four-dimensional vector of the bounding box coordinate;N pos representing the number of pixels of the entire image;
representing the sum of the classification losses for each pixel;L cls 、tc x,y respectively representing the focal loss and each position: (x,y) Class label of the truth box of (1);p{tc x,y >0 represents the probability of each pixel being a positive sample, iftc x,y >0, thenp=1, otherwisep=0；L reg 、tr x,y Respectively representing IOU penalty and each pixel location: (x,y) Training a regression target;L BCE a two-class cross-entropy loss function representing center offset.
6. An oil recovery operation field personnel smoking detection discernment processing system which characterized in that includes:
the acquisition module is used for calling a camera to take a picture of an oil extraction operation field to obtain an environmental picture;
the recognition module is used for analyzing the environment photo by using a recognition model and determining whether the cigarette ends exist in the environment photo; wherein the recognition model is obtained by machine learning training by using multiple groups of data; the multiple groups of data comprise a first type of data and a second type of data; each set of data in the first type of data comprises: a photograph containing butts and a label identifying the contained butts in the photograph; each set of data in the second class of data includes: a photograph containing no butts and a label identifying that no butts are contained in the photograph;
the judgment module is used for judging that the personnel on the site of oil extraction operation have smoking behavior under the condition that the environmental photo has a cigarette end, and sending out an early warning signal;
the identification module comprises a prediction unit, a calculation unit and an optimization unit;
the prediction unit is used for respectively acquiring first prediction information of each pixel of a photo in the first type of data and second prediction information of each pixel of the photo in the second type of data by utilizing a pre-constructed detection and identification network; the pre-constructed detection and identification network comprises a backbone network, a neck part, a feature mapping part and a head part; the backbone network adopts a ResNet-50 network and is used for extracting and storing first characteristic information of photos in the first type of data and second characteristic information of photos in the second type of data; the neck comprises a plurality of self-adaptive spatial pooling modules, wherein the self-adaptive spatial pooling modules comprise self-adaptive pooling and self-adaptive spatial fusion and are used for increasing the receptive fields of the first characteristic information and the second characteristic information to respectively obtain a first characteristic mapping and a second characteristic mapping; the feature mapping part performs adaptive spatial fusion and residual connection on the first feature mapping and the second feature mapping through an adaptive pyramid to obtain a final feature pyramid; the header is mainly composed of a plurality of prediction subnetworks for outputting the first prediction information and the second prediction information; the prediction sub-network comprises parallel classification branches and regression branches, and a center offset sub-branch is added in each classification branch and is used for preventing a boundary box generated by a position far away from a target center point;
the calculation unit is used for calculating a first error between the first prediction information and the label information of the first type of data and a second error between the second prediction information and the label information of the second type of data by using a loss function;
and the optimization unit is used for optimizing the parameters of the pre-constructed detection identification network by adopting a back propagation method until the first error and the second error respectively reach respective expected values, so as to obtain the identification model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110038437.4A CN112733730B (en) | 2021-01-12 | 2021-01-12 | Oil extraction operation field smoke suction personnel identification processing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110038437.4A CN112733730B (en) | 2021-01-12 | 2021-01-12 | Oil extraction operation field smoke suction personnel identification processing method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112733730A CN112733730A (en) | 2021-04-30 |
| CN112733730B true CN112733730B (en) | 2022-11-18 |
Family
ID=75591452
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110038437.4A Active CN112733730B (en) | 2021-01-12 | 2021-01-12 | Oil extraction operation field smoke suction personnel identification processing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112733730B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110969166A (en) * | 2019-12-04 | 2020-04-07 | 国网智能科技股份有限公司 | Small target identification method and system in inspection scene |
| CN111027372A (en) * | 2019-10-10 | 2020-04-17 | 山东工业职业学院 | Pedestrian target detection and identification method based on monocular vision and deep learning |
| CN111126278A (en) * | 2019-12-24 | 2020-05-08 | 北京邮电大学 | Target detection model optimization and acceleration method for few-category scene |
| CN111310850A (en) * | 2020-03-02 | 2020-06-19 | 杭州雄迈集成电路技术股份有限公司 | License plate detection model construction method and system and license plate detection method and system |
| CN112085739A (en) * | 2020-08-20 | 2020-12-15 | 深圳力维智联技术有限公司 | Semantic segmentation model training method, device and equipment based on weak supervision |
| CN112084882A (en) * | 2020-08-18 | 2020-12-15 | 深圳英飞拓科技股份有限公司 | Behavior detection method and device and computer readable storage medium |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108629282B (en) * | 2018-03-29 | 2021-12-24 | 福建海景科技开发有限公司 | Smoking detection method, storage medium and computer |
| CN109410218B (en) * | 2018-10-08 | 2020-08-11 | 百度在线网络技术（北京）有限公司 | Method and apparatus for generating vehicle damage information |
| CN111178424A (en) * | 2019-12-26 | 2020-05-19 | 中国石油大学(北京) | Petrochemical production site safety compliance real-time detection system and method |
| CN111209893A (en) * | 2020-01-20 | 2020-05-29 | 成都国翼电子技术有限公司 | Recognition method for policing supervision of smoking behavior |
| CN111274930B (en) * | 2020-04-02 | 2022-09-06 | 成都鼎安华智慧物联网股份有限公司 | Helmet wearing and smoking behavior identification method based on deep learning |
| CN111476252B (en) * | 2020-04-03 | 2022-07-29 | 南京邮电大学 | Computer vision application-oriented lightweight anchor-frame-free target detection method |
| CN111598843B (en) * | 2020-04-24 | 2022-11-11 | 国电南瑞科技股份有限公司 | Power transformer respirator target defect detection method based on deep learning |
| CN111814704B (en) * | 2020-07-14 | 2021-11-26 | 陕西师范大学 | Full convolution examination room target detection method based on cascade attention and point supervision mechanism |
| CN112017161A (en) * | 2020-08-06 | 2020-12-01 | 杭州深睿博联科技有限公司 | Pulmonary nodule detection method and device based on central point regression |
| CN111932518A (en) * | 2020-08-12 | 2020-11-13 | 杭州深睿博联科技有限公司 | Deep learning panoramic dental film focus detection and segmentation method and device |
| CN112070729B (en) * | 2020-08-26 | 2023-07-07 | 西安交通大学 | Anchor-free remote sensing image target detection method and system based on scene enhancement |
-
2021
- 2021-01-12 CN CN202110038437.4A patent/CN112733730B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111027372A (en) * | 2019-10-10 | 2020-04-17 | 山东工业职业学院 | Pedestrian target detection and identification method based on monocular vision and deep learning |
| CN110969166A (en) * | 2019-12-04 | 2020-04-07 | 国网智能科技股份有限公司 | Small target identification method and system in inspection scene |
| CN111126278A (en) * | 2019-12-24 | 2020-05-08 | 北京邮电大学 | Target detection model optimization and acceleration method for few-category scene |
| CN111310850A (en) * | 2020-03-02 | 2020-06-19 | 杭州雄迈集成电路技术股份有限公司 | License plate detection model construction method and system and license plate detection method and system |
| CN112084882A (en) * | 2020-08-18 | 2020-12-15 | 深圳英飞拓科技股份有限公司 | Behavior detection method and device and computer readable storage medium |
| CN112085739A (en) * | 2020-08-20 | 2020-12-15 | 深圳力维智联技术有限公司 | Semantic segmentation model training method, device and equipment based on weak supervision |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112733730A (en) | 2021-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112560999B (en) | Target detection model training method and device, electronic equipment and storage medium | |
| CN111681273B (en) | Image segmentation method and device, electronic equipment and readable storage medium | |
| CN111223129A (en) | Detection method, detection device, monitoring equipment and computer readable storage medium | |
| CN111445459A (en) | Image defect detection method and system based on depth twin network | |
| CN111898581A (en) | Animal detection method, device, electronic equipment and readable storage medium | |
| CN110599453A (en) | Panel defect detection method and device based on image fusion and equipment terminal | |
| CN115294117A (en) | Defect detection method and related device for LED lamp beads | |
| CN110942456B (en) | Tamper image detection method, device, equipment and storage medium | |
| CN113516146A (en) | Data classification method, computer and readable storage medium | |
| CN112434612A (en) | Smoking detection method and device, electronic equipment and computer readable storage medium | |
| CN114331949A (en) | Image data processing method, computer equipment and readable storage medium | |
| CN109977875A (en) | Gesture identification method and equipment based on deep learning | |
| CN115880765A (en) | Method and device for detecting abnormal behavior of regional intrusion and computer equipment | |
| CN114399734A (en) | Forest fire early warning method based on visual information | |
| US11532158B2 (en) | Methods and systems for customized image and video analysis | |
| CN112597995B (en) | License plate detection model training method, device, equipment and medium | |
| CN113570615A (en) | Image processing method based on deep learning, electronic equipment and storage medium | |
| CN112733730B (en) | Oil extraction operation field smoke suction personnel identification processing method and system | |
| CN114078127B (en) | Object defect detection and counting method, device, equipment and storage medium | |
| CN115358981A (en) | Glue defect determining method, device, equipment and storage medium | |
| CN117274957B (en) | Road traffic sign detection method and system based on deep learning | |
| CN111008582B (en) | Head photo analysis method, system and equipment | |
| CN116468974B (en) | Smoke detection method, device and storage medium based on image generation | |
| WO2024025134A1 (en) | A system and method for real time optical illusion photography | |
| CN115601641A (en) | Oil tank identification method and device based on remote sensing image, storage medium and terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |