CN111914656A - Personnel behavior detection method and device, electronic equipment and storage medium - Google Patents
Personnel behavior detection method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111914656A CN111914656A CN202010640306.9A CN202010640306A CN111914656A CN 111914656 A CN111914656 A CN 111914656A CN 202010640306 A CN202010640306 A CN 202010640306A CN 111914656 A CN111914656 A CN 111914656A
- Authority
- CN
- China
- Prior art keywords
- image
- person
- behavior
- detected
- detection model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 262
- 238000003860 storage Methods 0.000 title claims abstract description 16
- 238000000034 method Methods 0.000 claims abstract description 43
- 230000006399 behavior Effects 0.000 claims description 139
- 238000012549 training Methods 0.000 claims description 24
- 238000004891 communication Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 17
- 230000008569 process Effects 0.000 claims description 12
- 238000002372 labelling Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 abstract description 16
- 238000010586 diagram Methods 0.000 description 15
- 239000003245 coal Substances 0.000 description 6
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Abstract
The invention discloses a personnel behavior detection method, a personnel behavior detection device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring a first image of a person to be detected; and inputting the first image into a pre-trained behavior detection model, and determining the behavior of the person to be detected based on the behavior detection model. In the embodiment of the invention, the first image of the person to be detected is acquired, the first image is subjected to relevant processing through machine vision, the behavior of the person to be detected of the first image is determined, and manual intervention is not needed, so that the problems of missed detection caused by negligence or distraction of the person to be detected, low detection efficiency and poor reliability of a scheme for detecting by relying on manual work are solved.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method and an apparatus for detecting a person behavior, an electronic device, and a storage medium.
Background
With the rapid development of the coal mining industry in recent years, the coal mining industry is important to obtain benefits and safe production, and particularly in a haulage roadway, illegal behaviors such as vehicle taking and pedaling of underground operators are very easy to cause serious accidents of casualties. Therefore, behavior detection for downhole operators becomes particularly important.
At present, the personnel's action of coal mine tunnel transportation detects the manual inspection that most relies on in artifical safety supervision and the monitor of underground, and the problem that prior art exists relies on the manual work to carry out the detection of personnel's action, leads to missing the inspection easily because measurement personnel's negligence or distraction to the scheme detection efficiency who relies on the manual work to detect is low, and the reliability is also relatively poor.
Disclosure of Invention
The embodiment of the invention provides a personnel behavior detection method, a personnel behavior detection device, electronic equipment and a storage medium, which are used for solving the problems that in the prior art, personnel behavior detection is carried out manually, missed detection is easily caused by negligence or distraction of detection personnel, and a scheme for carrying out detection manually has low detection efficiency and poor reliability.
The embodiment of the invention provides a personnel behavior detection method, which comprises the following steps:
acquiring a first image of a person to be detected;
and inputting the first image into a pre-trained behavior detection model, and determining the behavior of the person to be detected based on the behavior detection model.
Further, the acquiring the first image of the person to be detected includes:
and acquiring a second image containing the person to be detected, inputting the second image into a pre-trained target detection model, and determining a first image of the person to be detected in the second image based on the target detection model.
Further, the determining the first image of the person to be detected in the second image based on the target detection model includes:
determining a detection frame of the mine car and a detection frame of the person in the second image based on the target detection model;
determining the intersection ratio of the detection frames of the mine car and the detection frames of the personnel, judging whether the intersection ratio is larger than a preset threshold value, and if so, determining the image surrounded by the detection frames of the personnel as a first image of the personnel to be detected.
Further, after determining the detection frame of the mine car and the detection frame of the person in the second image based on the target detection model, before determining the intersection ratio of the detection frame of the mine car and the detection frame of the person, the method further comprises:
and expanding the detection frame of the mine car by a preset multiple.
Further, the method further comprises:
and judging whether the behavior of the person to be detected belongs to the violation behavior, and if so, outputting alarm information.
Further, the training process of the behavior detection model comprises:
aiming at a third image of each person in a first sample set, inputting the third image and a first annotation image corresponding to the third image into the behavior detection model, and training the behavior detection model; and the first annotation image is marked with the behavior information of the personnel in the third image.
Further, the training process of the target detection model comprises:
for each fourth image in the second sample set, inputting the fourth image and a second annotation image corresponding to the fourth image into the target detection model, and training the target detection model; and the second labeling image is labeled with the position information of the mine car, the type information of the mine car, the position information of the personnel and the type information of the personnel in the fourth image.
In another aspect, an embodiment of the present invention provides a device for detecting a behavior of a person, where the device includes:
the acquisition module is used for acquiring a first image of a person to be detected;
and the detection module is used for inputting the first image into a behavior detection model which is trained in advance, and determining the behavior of the person to be detected based on the behavior detection model.
Further, the acquiring module is specifically configured to acquire a second image including the person to be detected, input the second image into a pre-trained target detection model, and determine, based on the target detection model, a first image of the person to be detected in the second image.
Further, the obtaining module is specifically configured to determine a detection frame of the mine car and a detection frame of the person in the second image based on the target detection model; determining the intersection ratio of the detection frames of the mine car and the detection frames of the personnel, judging whether the intersection ratio is larger than a preset threshold value, and if so, determining the image surrounded by the detection frames of the personnel as a first image of the personnel to be detected.
Further, the apparatus further comprises:
and the expanding module is used for expanding the detection frame of the mine car by preset times.
Further, the apparatus further comprises:
and the alarm module is used for judging whether the behavior of the person to be detected belongs to the violation behavior or not, and outputting alarm information if the behavior of the person to be detected belongs to the violation behavior.
Further, the apparatus further comprises:
the first training module is used for inputting a third image of each person in the first sample set and a first labeled image corresponding to the third image into the behavior detection model and training the behavior detection model; and the first annotation image is marked with the behavior information of the personnel in the third image.
Further, the apparatus further comprises:
the second training module is used for inputting the fourth image and a second labeled image corresponding to the fourth image into the target detection model aiming at each fourth image in the second sample set and training the target detection model; and the second labeling image is labeled with the position information of the mine car, the type information of the mine car, the position information of the personnel and the type information of the personnel in the fourth image.
In another aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete mutual communication through the communication bus;
a memory for storing a computer program;
a processor for implementing any of the above method steps when executing a program stored in the memory.
In yet another aspect, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method steps of any one of the above.
The embodiment of the invention provides a personnel behavior detection method, a personnel behavior detection device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring a first image of a person to be detected; and inputting the first image into a pre-trained behavior detection model, and determining the behavior of the person to be detected based on the behavior detection model. In the embodiment of the invention, the first image of the person to be detected is acquired, the first image is subjected to relevant processing through machine vision, the behavior of the person to be detected of the first image is determined, and manual intervention is not needed, so that the problems of missed detection caused by negligence or distraction of the person to be detected, low detection efficiency and poor reliability of a scheme for detecting by relying on manual work are solved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic diagram of a personnel behavior detection process provided in embodiment 1 of the present invention;
fig. 2 is a schematic diagram of a flow of detecting a person behavior according to embodiment 6 of the present invention;
fig. 3 is a schematic structural diagram of a personnel behavior detection device according to embodiment 7 of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to embodiment 8 of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the attached drawings, and it should be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1:
fig. 1 is a schematic diagram of a human behavior detection process provided in an embodiment of the present invention, where the process includes the following steps:
s101: a first image of a person to be detected is acquired.
S102: and inputting the first image into a pre-trained behavior detection model, and determining the behavior of the person to be detected based on the behavior detection model.
The personnel behavior detection method provided by the embodiment of the invention is applied to electronic equipment, and the electronic equipment can be equipment such as a PC (personal computer), a tablet personal computer and the like, and can also be image acquisition equipment. If the electronic equipment is the image acquisition equipment, the process of behavior detection of the person to be detected is directly carried out after the first image of the person to be detected is acquired by the image acquisition equipment. If the electronic equipment is a PC, a tablet computer and the like, after the image acquisition equipment acquires the first image of the person to be detected, the first image is sent to the electronic equipment, or the electronic equipment acquires the first image of the person to be detected acquired by the image acquisition equipment, and the electronic equipment performs a behavior detection process on the person to be detected.
Specifically, a behavior detection model trained in advance is stored in the electronic device, and the behavior detection model is used for performing correlation processing on an input image and outputting the behavior of a person in the image. After the electronic equipment acquires the first image of the person to be detected, the first image is input into a pre-trained behavior detection model, and the behavior detection model performs relevant processing on the first image to determine the behavior of the person to be detected.
In the embodiment of the invention, the first image of the person to be detected is acquired, the first image is subjected to relevant processing through machine vision, the behavior of the person to be detected of the first image is determined, and manual intervention is not needed, so that the problems of missed detection caused by negligence or distraction of the person to be detected, low detection efficiency and poor reliability of a scheme for detecting by relying on manual work are solved.
Example 2:
the image acquired by the image acquisition device is generally an environment image including a person to be detected, wherein the environment image includes other scene information besides the person to be detected. To avoid adverse effects on behavior detection caused by other scene information in the image, on the basis of the above embodiment, in an embodiment of the present invention, the acquiring the first image of the person to be detected includes:
and acquiring a second image containing the person to be detected, inputting the second image into a pre-trained target detection model, and determining a first image of the person to be detected in the second image based on the target detection model.
In the embodiment of the invention, an environment image including a person to be detected and acquired by an image acquisition device is called a second image, a pre-trained target detection model is stored in an electronic device, the target detection model is used for performing relevant processing on an input image and outputting the image of the person to be detected in the image, and the image of the person to be detected is called a first image in the embodiment of the invention.
Specifically, after acquiring a second image including a person to be detected, the electronic device inputs the second image into a pre-trained target detection model, and the target detection model performs correlation processing on the second image to determine a first image of the person to be detected in the second image. It should be noted that if the second image only includes one person to be detected, there is only one determined first image, and if the second image includes a plurality of persons to be detected, there are a plurality of determined first images.
In the embodiment of the invention, the electronic equipment acquires the second image including the person to be detected, inputs the second image into the pre-trained target detection model, and determines the first image of the person to be detected in the second image based on the target detection model. Therefore, the environmental interference around the person to be detected can be removed, and after the first image is input into the pre-trained behavior detection model, the detection result of the behavior detection model can be more accurate.
Example 3:
when the method for detecting the behavior of the person provided by the embodiment of the present invention is applied to the field of monitoring of a mine roadway pedalling vehicle, on the basis of the above embodiments, in the embodiment of the present invention, in order to reduce the amount of calculation of the behavior of the person, determining the first image of the person to be detected in the second image based on the target detection model includes:
determining a detection frame of the mine car and a detection frame of the person in the second image based on the target detection model;
determining the intersection ratio of the detection frames of the mine car and the detection frames of the personnel, judging whether the intersection ratio is larger than a preset threshold value, and if so, determining the image surrounded by the detection frames of the personnel as a first image of the personnel to be detected.
In the embodiment of the invention, the electronic equipment needs to detect whether the illegal behaviors such as vehicle pushing and pedaling exist for the coal mine tunnel workers, so that the behavior of the workers in contact with the mine car only needs to be detected, and the illegal behaviors such as vehicle pushing and pedaling cannot exist for the workers in no contact with the mine car, so that the detection is not needed.
In view of the above, in the embodiment of the present invention, the electronic device stores in advance an object detection model for detecting the mine car and the person in the second image, and the detection result is represented by the detection frame. After the electronic equipment inputs the second image into the target detection model, the target detection model carries out relevant processing on the second image, and outputs the detection frames of the mine car and the detection frames of the personnel in the second image, wherein each detection frame has a corresponding class identifier for distinguishing the detection frame of the personnel from the detection frame of the mine car. It should be noted that the number of the detection frames of the mine cars and the number of the detection frames of the persons are the same as the number of the mine cars and the persons existing in the second image.
And the electronic equipment determines the intersection ratio of the detection frame of the mine car and the detection frame of the person, judges whether the intersection ratio is greater than a preset threshold value or not, and if so, determines the image surrounded by the detection frames of the person as a first image of the person to be detected. If the number of the mine car detection frames and the number of the personnel detection frames are multiple, judging whether the mine car detection frames intersected with the personnel detection frames exist or not aiming at the detection frames of each personnel, if so, calculating the largest intersection and comparison ratio of the personnel detection frames and the mine car detection frames, then judging whether the intersection and comparison ratio is larger than a preset threshold value or not, and if so, determining the image surrounded by the personnel detection frames as a first image of the personnel to be detected.
In the embodiment of the invention, the detection frame of the mine car and the detection frame of the personnel in the second image are determined based on the target detection model; determining the intersection ratio of the detection frames of the mine car and the detection frames of the personnel, judging whether the intersection ratio is larger than a preset threshold value, and if so, determining the image surrounded by the detection frames of the personnel as a first image of the personnel to be detected. The first image determined by the embodiment of the invention is the first image of the personnel who are in contact with the mine car, and the personnel who are not in contact with the mine car are filtered, so that the calculation amount of personnel behavior detection is reduced, and the detection efficiency is improved.
Example 4:
in order to ensure that the image surrounded by the detection frames of the people contacting with the mine car is determined as the first image of the people to be detected so as to avoid missing detection, on the basis of the above embodiments, in an embodiment of the present invention, after determining the detection frames of the mine car and the people in the second image based on the target detection model, before determining the intersection ratio between the detection frames of the mine car and the detection frames of the people, the method further includes:
and expanding the detection frame of the mine car by a preset multiple.
In the embodiment of the invention, in order to avoid the behavior of the missed detection personnel, the electronic equipment enlarges the detection frame of the mine car by a preset multiple after determining the detection frame of the mine car and the detection frame of the personnel in the second image based on the target detection model. And then determining the intersection ratio of the detection frame of the mine car and the detection frame of the person, and performing subsequent follow-up intersection ratio to determine a first image of the person to be detected.
The preset multiple can be 1.1 times, 1.2 times and the like, any pixel point in the detection frame of the mine car can be used as the center for expanding when the detection frame of the mine car is expanded by the preset multiple, and preferably, the central pixel point in the detection frame of the mine car can be used as the center for expanding.
According to the embodiment of the invention, by means of the method for expanding the detection frame of the mine car, the image surrounded by the detection frame of the person contacting with the mine car can be ensured to be the first image of the person to be detected, so that the behavior of the person who is missed to be detected is avoided.
Example 5:
in order to remind the person of the violation in time, on the basis of the above embodiments, in an embodiment of the present invention, the method further includes:
and judging whether the behavior of the person to be detected belongs to the violation behavior, and if so, outputting alarm information.
The method comprises the steps that which behaviors belong to the illegal behaviors are pre-stored in the electronic equipment, after the behaviors of a person to be detected are determined, whether the behaviors of the person to be detected belong to the illegal behaviors is judged, and if yes, alarm information is output to prompt the person that the behaviors are illegal and timely corrected. The alarm information can be voice prompt information, light information and the like, and only needs to play a role of prompting personnel in a scene, and the embodiment of the invention does not limit the alarm information.
Example 6:
on the basis of the above embodiments, in an embodiment of the present invention, a training process of the behavior detection model includes:
aiming at a third image of each person in a first sample set, inputting the third image and a first annotation image corresponding to the third image into the behavior detection model, and training the behavior detection model; and the first annotation image is marked with the behavior information of the personnel in the third image.
The electronic equipment stores a first sample set, images in the first sample set are third images of people, each third image has a corresponding first annotation image, and the first annotation image is marked with behavior information of the people in the third images. And inputting the third image of each person and the corresponding first labeled image into the behavior detection model to finish the training of the behavior detection model.
The training process of the target detection model comprises the following steps:
for each fourth image in the second sample set, inputting the fourth image and a second annotation image corresponding to the fourth image into the target detection model, and training the target detection model; and the second labeling image is labeled with the position information of the mine car, the type information of the mine car, the position information of the personnel and the type information of the personnel in the fourth image.
And storing a second sample set in the electronic equipment, wherein images in the second sample set are fourth images, each fourth image has a corresponding second annotation image, the first annotation image is annotated with the position information of the mine car, the type information of the mine car, the position information of the personnel and the type information of the personnel in the fourth image, and each fourth image and the corresponding second annotation image are input into the target detection model to finish the training of the target detection model.
The following describes in detail the method for detecting a person behavior provided by the embodiment of the present invention.
The method specifically comprises the following steps:
1) the method comprises the steps of collecting monitoring videos of all cameras in a coal mine tunnel, wherein the cameras can be explosion-proof IP cameras, imaging spectrums of the cameras can be visible light or infrared light, and installation angles of the cameras can be conventional installation and cover a travelling rail and a sidewalk in the tunnel. And acquiring a second image containing the person to be detected based on the monitoring video.
2) A target detection model:
a. inputting a pre-trained target detection model into the video in the step 1) by using an image sequence, wherein the target detection model is a neural network model including but not limited to RCNN series, SSD, YOLO series and the like, and the output of the target detection model is a detection category and a detection frame coordinate (xl, yl, xr, yr). Wherein the detection categories are mine cars and people. The coordinates of the detection frame include coordinates (xl, yl) of the vertex at the upper left corner and coordinates (xr, yr) of the vertex at the lower right corner of the detection frame.
b. The detection frame size of the detection frame for the mine car of the detection type is enlarged by 1.1 times, 1.2 times or 1.5 times to obtain the enlarged detection frame (xl-ext, yl-ext, xr-ext, yr-ext) of the mine car.
c. And (3) calculating the overlapping proportion of the expanded mine car detection frame and the personnel detection frame, namely the intersection ratio IoU:
when IoU is smaller than a preset threshold value IoU-thresh, it can be judged that the personnel is not related to the mine car, and the detection frame of the corresponding personnel is excluded, wherein IoU-thresh can be 0.3, 0.5, 0.7 and the like according to the actual scene; when IoU is preset to be larger than a threshold value IoU-thresh, the person is judged to be near the mine car, the image surrounded by the detection frame of the person is determined to be a first image of the person to be detected, and the behavior of the person to be detected in the first image is determined based on a behavior detection model.
3) And (3) behavior detection model:
a. cutting out a first image by utilizing the personnel detection frame near the mine car detected and screened out in the step 2).
b. And inputting the first image into a behavior detection model which is trained in advance. The behavior detection model employs neural network models including, but not limited to, VGGNet, AlexNet, ResNet, DenseNet, and the like. The output categories of the behavior detection model are as follows: such as picking up a car, pedaling a car, walking, running, etc.
c. When the result output by the behavior detection model is illegal behaviors such as 'picking up' or 'pedaling' and the confidence coefficient is more than or equal to the preset value conf _ thresh, outputting picking up and pedaling alarm information, and if the confidence coefficient is less than the preset value conf _ thresh, outputting no alarm; when the output result of the behavior detection model is "walking" or "running", no alarm is output.
Fig. 2 is a schematic diagram of a flow of detecting a person behavior according to an embodiment of the present invention, which includes the following steps:
s201: and acquiring a monitoring video of each camera of the coal mine tunnel, and acquiring a second image containing the person to be detected based on the monitoring video.
S202: and determining the detection frame of the mine car and the detection frame of the personnel in the second image based on the target detection model.
S203: and expanding the detection frame of the mine car by a preset multiple. And determining the intersection ratio of the detection frame of the mine car and the detection frame of the personnel.
S204: and judging whether the intersection ratio is larger than a preset threshold value or not, and if so, determining an image surrounded by the detection frames of the personnel as a first image of the personnel to be detected.
S205: and inputting the first image into a pre-trained behavior detection model, and determining the behavior of the person to be detected based on the behavior detection model.
S206: and judging whether the behavior of the person to be detected belongs to the violation behavior, if so, outputting alarm information, and if not, not outputting the alarm information.
Example 7:
fig. 3 is a schematic structural diagram of a device for detecting a person behavior according to an embodiment of the present invention, where the device includes:
the acquiring module 31 is used for acquiring a first image of a person to be detected;
the detection module 32 is configured to input the first image into a behavior detection model which is trained in advance, and determine the behavior of the person to be detected based on the behavior detection model.
The obtaining module 31 is specifically configured to obtain a second image including the person to be detected, input the second image into a pre-trained target detection model, and determine, based on the target detection model, a first image of the person to be detected in the second image.
The obtaining module 31 is specifically configured to determine a detection frame of the mine car and a detection frame of the person in the second image based on the target detection model; determining the intersection ratio of the detection frames of the mine car and the detection frames of the personnel, judging whether the intersection ratio is larger than a preset threshold value, and if so, determining the image surrounded by the detection frames of the personnel as a first image of the personnel to be detected.
The device further comprises:
and the expanding module 33 is used for expanding the detection frame of the mine car by preset times.
The device further comprises:
and the alarm module 34 is configured to determine whether the behavior of the to-be-detected person belongs to an illegal behavior, and if so, output alarm information.
The device further comprises:
a first training module 35, configured to input, to a third image of each person in the first sample set, the third image and a first labeled image corresponding to the third image into the behavior detection model, and train the behavior detection model; and the first annotation image is marked with the behavior information of the personnel in the third image.
The device further comprises:
a second training module 36, configured to input, for each fourth image in the second sample set, the fourth image and a second labeled image corresponding to the fourth image into the target detection model, and train the target detection model; and the second labeling image is labeled with the position information of the mine car, the type information of the mine car, the position information of the personnel and the type information of the personnel in the fourth image.
Example 8:
on the basis of the foregoing embodiments, an embodiment of the present invention further provides an electronic device, as shown in fig. 4, including: the system comprises a processor 301, a communication interface 302, a memory 303 and a communication bus 304, wherein the processor 301, the communication interface 302 and the memory 303 complete mutual communication through the communication bus 304;
the memory 303 has stored therein a computer program which, when executed by the processor 301, causes the processor 301 to perform the steps of:
acquiring a first image of a person to be detected;
and inputting the first image into a pre-trained behavior detection model, and determining the behavior of the person to be detected based on the behavior detection model.
Based on the same inventive concept, the embodiment of the present invention further provides an electronic device, and as the principle of the electronic device for solving the problem is similar to the method for detecting the behavior of the person, the implementation of the electronic device may refer to the implementation of the method, and repeated details are not repeated.
The electronic device provided by the embodiment of the invention can be a desktop computer, a portable computer, a smart phone, a tablet computer, a Personal Digital Assistant (PDA), a network side device and the like.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface 302 is used for communication between the above-described electronic apparatus and other apparatuses.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a central processing unit, a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like.
When the processor executes the program stored in the memory in the embodiment of the invention, the first image of the person to be detected is acquired; and inputting the first image into a pre-trained behavior detection model, and determining the behavior of the person to be detected based on the behavior detection model. In the embodiment of the invention, the first image of the person to be detected is acquired, the first image is subjected to relevant processing through machine vision, the behavior of the person to be detected of the first image is determined, and manual intervention is not needed, so that the problems of missed detection caused by negligence or distraction of the person to be detected, low detection efficiency and poor reliability of a scheme for detecting by relying on manual work are solved.
Example 9:
on the basis of the foregoing embodiments, an embodiment of the present invention further provides a computer storage readable storage medium, in which a computer program executable by an electronic device is stored, and when the program is run on the electronic device, the electronic device is caused to execute the following steps:
acquiring a first image of a person to be detected;
and inputting the first image into a pre-trained behavior detection model, and determining the behavior of the person to be detected based on the behavior detection model.
Based on the same inventive concept, embodiments of the present invention further provide a computer-readable storage medium, and since a principle of solving a problem when a processor executes a computer program stored in the computer-readable storage medium is similar to a human behavior detection method, implementation of the computer program stored in the computer-readable storage medium by the processor may refer to implementation of the method, and repeated details are not repeated.
The computer readable storage medium may be any available medium or data storage device that can be accessed by a processor in an electronic device, including but not limited to magnetic memory such as floppy disks, hard disks, magnetic tape, magneto-optical disks (MOs), etc., optical memory such as CDs, DVDs, BDs, HVDs, etc., and semiconductor memory such as ROMs, EPROMs, EEPROMs, non-volatile memory (NAND FLASH), Solid State Disks (SSDs), etc.
The computer program is stored in a computer readable storage medium provided in the embodiment of the invention, and when being executed by a processor, the computer program realizes acquisition of a first image of a person to be detected; and inputting the first image into a pre-trained behavior detection model, and determining the behavior of the person to be detected based on the behavior detection model. In the embodiment of the invention, the first image of the person to be detected is acquired, the first image is subjected to relevant processing through machine vision, the behavior of the person to be detected of the first image is determined, and manual intervention is not needed, so that the problems of missed detection caused by negligence or distraction of the person to be detected, low detection efficiency and poor reliability of a scheme for detecting by relying on manual work are solved.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (16)
1. A method of human behavior detection, the method comprising:
acquiring a first image of a person to be detected;
and inputting the first image into a pre-trained behavior detection model, and determining the behavior of the person to be detected based on the behavior detection model.
2. The method of claim 1, wherein said acquiring a first image of a person to be detected comprises:
and acquiring a second image containing the person to be detected, inputting the second image into a pre-trained target detection model, and determining a first image of the person to be detected in the second image based on the target detection model.
3. The method of claim 2, wherein determining the first image of the person to be detected in the second image based on the object detection model comprises:
determining a detection frame of the mine car and a detection frame of the person in the second image based on the target detection model;
determining the intersection ratio of the detection frames of the mine car and the detection frames of the personnel, judging whether the intersection ratio is larger than a preset threshold value, and if so, determining the image surrounded by the detection frames of the personnel as a first image of the personnel to be detected.
4. The method of claim 3, wherein after determining the detection frame of the mine car and the detection frame of the person in the second image based on the object detection model, and before determining the intersection ratio of the detection frame of the mine car and the detection frame of the person, the method further comprises:
and expanding the detection frame of the mine car by a preset multiple.
5. The method of claim 1, wherein the method further comprises:
and judging whether the behavior of the person to be detected belongs to the violation behavior, and if so, outputting alarm information.
6. The method of claim 1, wherein the training process of the behavior detection model comprises:
aiming at a third image of each person in a first sample set, inputting the third image and a first annotation image corresponding to the third image into the behavior detection model, and training the behavior detection model; and the first annotation image is marked with the behavior information of the personnel in the third image.
7. The method of claim 2, wherein the training process of the target detection model comprises:
for each fourth image in the second sample set, inputting the fourth image and a second annotation image corresponding to the fourth image into the target detection model, and training the target detection model; and the second labeling image is labeled with the position information of the mine car, the type information of the mine car, the position information of the personnel and the type information of the personnel in the fourth image.
8. A person behavior detection apparatus, characterized in that the apparatus comprises:
the acquisition module is used for acquiring a first image of a person to be detected;
and the detection module is used for inputting the first image into a behavior detection model which is trained in advance, and determining the behavior of the person to be detected based on the behavior detection model.
9. The apparatus according to claim 8, wherein the acquiring module is specifically configured to acquire a second image including the person to be detected, input the second image into a pre-trained object detection model, and determine the first image of the person to be detected in the second image based on the object detection model.
10. The apparatus according to claim 9, wherein the acquisition module is configured to determine a detection frame of the mine car and a detection frame of the person in the second image based on the object detection model; determining the intersection ratio of the detection frames of the mine car and the detection frames of the personnel, judging whether the intersection ratio is larger than a preset threshold value, and if so, determining the image surrounded by the detection frames of the personnel as a first image of the personnel to be detected.
11. The apparatus of claim 10, wherein the apparatus further comprises:
and the expanding module is used for expanding the detection frame of the mine car by preset times.
12. The apparatus of claim 8, wherein the apparatus further comprises:
and the alarm module is used for judging whether the behavior of the person to be detected belongs to the violation behavior or not, and outputting alarm information if the behavior of the person to be detected belongs to the violation behavior.
13. The apparatus of claim 8, wherein the apparatus further comprises:
the first training module is used for inputting a third image of each person in the first sample set and a first labeled image corresponding to the third image into the behavior detection model and training the behavior detection model; and the first annotation image is marked with the behavior information of the personnel in the third image.
14. The apparatus of claim 9, wherein the apparatus further comprises:
the second training module is used for inputting the fourth image and a second labeled image corresponding to the fourth image into the target detection model aiming at each fourth image in the second sample set and training the target detection model; and the second labeling image is labeled with the position information of the mine car, the type information of the mine car, the position information of the personnel and the type information of the personnel in the fourth image.
15. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any one of claims 1 to 7 when executing a program stored in the memory.
16. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010640306.9A CN111914656A (en) | 2020-07-06 | 2020-07-06 | Personnel behavior detection method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010640306.9A CN111914656A (en) | 2020-07-06 | 2020-07-06 | Personnel behavior detection method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111914656A true CN111914656A (en) | 2020-11-10 |
Family
ID=73227414
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010640306.9A Pending CN111914656A (en) | 2020-07-06 | 2020-07-06 | Personnel behavior detection method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111914656A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112818844A (en) * | 2021-01-29 | 2021-05-18 | 成都商汤科技有限公司 | Security check abnormal event detection method and device, electronic equipment and storage medium |
| CN114266999A (en) * | 2021-12-29 | 2022-04-01 | 北京立思辰智汇科技有限公司 | Automatic detection system and method for airplane wheel gear time |
| CN114882597A (en) * | 2022-07-11 | 2022-08-09 | 浙江大华技术股份有限公司 | Target behavior identification method and device and electronic equipment |
| CN117037289A (en) * | 2023-10-10 | 2023-11-10 | 深圳联友科技有限公司 | Personnel operation behavior recognition method and system based on machine vision |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109086662A (en) * | 2018-06-19 | 2018-12-25 | 浙江大华技术股份有限公司 | A kind of anomaly detection method and device |
| CN110287856A (en) * | 2019-06-21 | 2019-09-27 | 上海闪马智能科技有限公司 | A kind of security personnel's behavior analysis system, method and device |
| CN110796039A (en) * | 2019-10-15 | 2020-02-14 | 北京达佳互联信息技术有限公司 | Face flaw detection method and device, electronic equipment and storage medium |
| CN110826370A (en) * | 2018-08-09 | 2020-02-21 | 广州汽车集团股份有限公司 | Method and device for identifying identity of person in vehicle, vehicle and storage medium |
| CN110969600A (en) * | 2019-11-12 | 2020-04-07 | 华北电力大学扬中智能电气研究中心 | Product defect detection method and device, electronic equipment and storage medium |
| CN111160187A (en) * | 2019-12-20 | 2020-05-15 | 浙江大华技术股份有限公司 | Method, device and system for detecting left-behind object |
-
2020
- 2020-07-06 CN CN202010640306.9A patent/CN111914656A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109086662A (en) * | 2018-06-19 | 2018-12-25 | 浙江大华技术股份有限公司 | A kind of anomaly detection method and device |
| CN110826370A (en) * | 2018-08-09 | 2020-02-21 | 广州汽车集团股份有限公司 | Method and device for identifying identity of person in vehicle, vehicle and storage medium |
| CN110287856A (en) * | 2019-06-21 | 2019-09-27 | 上海闪马智能科技有限公司 | A kind of security personnel's behavior analysis system, method and device |
| CN110796039A (en) * | 2019-10-15 | 2020-02-14 | 北京达佳互联信息技术有限公司 | Face flaw detection method and device, electronic equipment and storage medium |
| CN110969600A (en) * | 2019-11-12 | 2020-04-07 | 华北电力大学扬中智能电气研究中心 | Product defect detection method and device, electronic equipment and storage medium |
| CN111160187A (en) * | 2019-12-20 | 2020-05-15 | 浙江大华技术股份有限公司 | Method, device and system for detecting left-behind object |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112818844A (en) * | 2021-01-29 | 2021-05-18 | 成都商汤科技有限公司 | Security check abnormal event detection method and device, electronic equipment and storage medium |
| CN114266999A (en) * | 2021-12-29 | 2022-04-01 | 北京立思辰智汇科技有限公司 | Automatic detection system and method for airplane wheel gear time |
| CN114266999B (en) * | 2021-12-29 | 2024-02-02 | 北京航易智汇科技有限公司 | Automatic detection system and method for airplane wheel gear time |
| CN114882597A (en) * | 2022-07-11 | 2022-08-09 | 浙江大华技术股份有限公司 | Target behavior identification method and device and electronic equipment |
| CN117037289A (en) * | 2023-10-10 | 2023-11-10 | 深圳联友科技有限公司 | Personnel operation behavior recognition method and system based on machine vision |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109508688B (en) | Skeleton-based behavior detection method, terminal equipment and computer storage medium | |
| CN111914656A (en) | Personnel behavior detection method and device, electronic equipment and storage medium | |
| CN110807429B (en) | Construction safety detection method and system based on tiny-YOLOv3 | |
| CN109145680B (en) | Method, device and equipment for acquiring obstacle information and computer storage medium | |
| CN106952303B (en) | Vehicle distance detection method, device and system | |
| CN111310562B (en) | Vehicle driving risk management and control method based on artificial intelligence and related equipment thereof | |
| CN112016413B (en) | Method and device for detecting abnormal behaviors among objects | |
| CN109409238B (en) | Obstacle detection method and device and terminal equipment | |
| CN110706261A (en) | Vehicle violation detection method and device, computer equipment and storage medium | |
| US8675917B2 (en) | Abandoned object recognition using pedestrian detection | |
| CN103810696B (en) | Method for detecting image of target object and device thereof | |
| CN111368615B (en) | Illegal building early warning method and device and electronic equipment | |
| CN109711427A (en) | Object detection method and Related product | |
| CN112330964B (en) | Road condition information monitoring method and device | |
| CN111259868A (en) | Convolutional neural network-based method, system and medium for detecting vehicles in reverse driving | |
| CN112071084A (en) | Method and system for judging illegal parking by utilizing deep learning | |
| CN113343779B (en) | Environment abnormality detection method, device, computer equipment and storage medium | |
| CN107764318A (en) | Method for detecting abnormality and Related product | |
| US20190116310A1 (en) | Method of processing object in image and apparatus for same | |
| CN115965913A (en) | Security monitoring method, device and system and computer readable storage medium | |
| CN110298302B (en) | Human body target detection method and related equipment | |
| CN114663390A (en) | Intelligent anti-pinch method, device, equipment and storage medium for automatic door | |
| CN110865415A (en) | Security inspection method and device | |
| CN111160187A (en) | Method, device and system for detecting left-behind object | |
| CN114663871A (en) | Image recognition method, training method, device, system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |