CN107247946B - Behavior recognition method and device - Google Patents

Behavior recognition method and device Download PDF

Info

Publication number
CN107247946B
CN107247946B CN201710544459.1A CN201710544459A CN107247946B CN 107247946 B CN107247946 B CN 107247946B CN 201710544459 A CN201710544459 A CN 201710544459A CN 107247946 B CN107247946 B CN 107247946B
Authority
CN
China
Prior art keywords
pedestrian
behavior
result
recognition
area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710544459.1A
Other languages
Chinese (zh)
Other versions
CN107247946A (en
Inventor
陶铁牛
王帼筊
张丽媛
赵爱巧
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing anningwell emergency fire safety technology Co.,Ltd.
Original Assignee
Beijing Anywell Technology Development Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Anywell Technology Development Co ltd filed Critical Beijing Anywell Technology Development Co ltd
Priority to CN201710544459.1A priority Critical patent/CN107247946B/en
Publication of CN107247946A publication Critical patent/CN107247946A/en
Application granted granted Critical
Publication of CN107247946B publication Critical patent/CN107247946B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10048—Infrared image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a behavior identification method and device. The method is applied to a computing device comprising a recognition model for performing behavior recognition. The method comprises the following steps: carrying out image processing on the collected visible light image and infrared image to obtain a target area to be tracked; detecting whether a target area to be tracked comprises a pedestrian or not; when the pedestrians are included, the pedestrians are tracked, and edges of the pedestrians are detected in the tracking process to obtain a to-be-identified area of the pedestrians from the to-be-tracked target area; and inputting the area to be recognized of the pedestrian into the recognition model to obtain a behavior recognition result of the pedestrian. Thereby, the behavior of the pedestrian in the image is recognized.

Description

Behavior recognition method and device
Technical Field
The invention relates to the technical field of computer vision, in particular to a behavior identification method and device.
Background
With the increase of safety awareness of people and the increase of various emergencies (such as fire) faced by society, security monitoring is increasingly valued by society, institutions and individuals. The traditional safety monitoring system mainly realizes the monitoring of scenes in a manual monitoring mode and does not have the real-time and active environment monitoring capability.
For example, when a fire occurs, the reason of the fire can only be found through a video subsequently, and the fire includes a fire caused by artificial active fire release, electrical or equipment aging and the like. The traditional method cannot effectively predict the behavior characteristics of the personnel in the scene, cannot judge whether the monitored personnel has illegal behaviors or illegal behaviors, and cannot achieve the purposes of real-time monitoring and alarming. Therefore, how to automatically identify the behavior of a person from a surveillance video is a problem that those skilled in the art will continuously solve.
Disclosure of Invention
In order to overcome the above-mentioned deficiencies in the prior art, the present invention provides a behavior recognition method and apparatus, which can automatically recognize the behavior of a pedestrian in an image to prevent an emergency.
The invention provides a behavior recognition method, which is applied to computing equipment, wherein the computing equipment comprises a recognition model for behavior recognition, and the method comprises the following steps:
carrying out image processing on the collected visible light image and infrared image to obtain a target area to be tracked;
detecting whether the target area to be tracked comprises a pedestrian or not;
when the pedestrians are included, the pedestrians are tracked, and edges of the pedestrians are detected in the tracking process so as to obtain a to-be-identified area of the pedestrians from the to-be-tracked target area;
and inputting the area to be recognized of the pedestrian into a recognition model to obtain a behavior recognition result of the pedestrian.
The preferred embodiment of the present invention further provides a behavior recognition apparatus, applied to a computing device, where the computing device includes a recognition model for performing behavior recognition, and the apparatus includes:
the processing module is used for carrying out image processing on the collected visible light image and infrared image to obtain a target area to be tracked;
the detection module is used for detecting whether the target area to be tracked comprises a pedestrian or not;
the processing module is further used for tracking the pedestrians when the pedestrians comprise the target area, and detecting edges of the pedestrians in the tracking process to obtain the target area to be identified of the pedestrians from the target area to be tracked;
and the identification module is used for inputting the to-be-identified region of the pedestrian into an identification model to obtain a behavior identification result of the pedestrian.
Compared with the prior art, the invention has the following beneficial effects:
the preferred embodiment of the invention provides a behavior identification method and a behavior identification device. The method is applied to a computing device comprising a recognition model for performing behavior recognition. And after the visible light image and the infrared image are obtained, performing image processing on the obtained images to obtain a target area to be tracked. When the target area to be tracked comprises the pedestrian, the pedestrian is tracked, and the edge of the pedestrian is detected in the tracking process to obtain the area to be identified of the pedestrian. And inputting the to-be-recognized area of the pedestrian into a recognition model to obtain a behavior recognition result of the pedestrian. Therefore, the pedestrian in the image is automatically identified, and the occurrence of an emergency can be prevented.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
FIG. 1 is a block diagram of a computing device according to a preferred embodiment of the present invention.
Fig. 2 is a flowchart illustrating a behavior recognition method according to a preferred embodiment of the invention.
Fig. 3 is a flowchart illustrating sub-steps included in step S120 in fig. 2.
Fig. 4 is a flowchart illustrating sub-steps included in sub-step S123 in fig. 3.
Fig. 5 is a flowchart illustrating sub-steps included in step S150 in fig. 2.
Fig. 6 is a flowchart illustrating sub-steps included in sub-step S154 in fig. 5.
Fig. 7 is a second flowchart illustrating a behavior recognition method according to a preferred embodiment of the invention.
Fig. 8 is a third schematic flow chart of a behavior recognition method according to a preferred embodiment of the present invention.
Fig. 9 is a flowchart illustrating a part of sub-steps included in step S110 in fig. 8.
Fig. 10 is a schematic flow chart of another part of sub-steps included in step S110 in fig. 8.
Fig. 11 is a block diagram of a behavior recognition device according to a preferred embodiment of the present invention.
Icon: 100-a computing device; 110-a memory; 120-a memory controller; 130-a processor; 200-a behavior recognition device; 220-a processing module; 230-a detection module; 250-identification module.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present invention, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
Some embodiments of the invention are described in detail below with reference to the accompanying drawings. The embodiments described below and the features of the embodiments can be combined with each other without conflict.
Referring to fig. 1, fig. 1 is a block diagram of a computing device 100 according to a preferred embodiment of the invention. The computing device 100 in the embodiment of the present invention may be, but is not limited to, a computer, a server, etc. As shown in fig. 1, the computing device 100 includes: memory 110, memory controller 120, processor 130, and behavior recognition device 200.
The elements of the memory 110, the memory controller 120 and the processor 130 are electrically connected directly or indirectly to realize data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The memory 110 stores therein a behavior recognizing device 200, and the behavior recognizing device 200 includes at least one software functional module which can be stored in the memory 110 in the form of software or firmware (firmware). The processor 130 executes various functional applications and data processing by executing software programs and modules stored in the memory 110, such as the behavior recognition device 200 in the embodiment of the present invention, so as to implement the behavior recognition method in the embodiment of the present invention.
The Memory 110 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like. The memory 110 is used for storing a program, and the processor 130 executes the program after receiving the execution instruction. Access to the memory 110 by the processor 130 and possibly other components may be under the control of the memory controller 120.
The processor 130 may be an integrated circuit chip having signal processing capabilities. The Processor 130 may be a general-purpose Processor including a Central Processing Unit (CPU), a Network Processor (NP), and the like. But may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It will be appreciated that the configuration shown in FIG. 1 is merely illustrative, and that computing device 100 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1. The components shown in fig. 1 may be implemented in hardware, software, or a combination thereof.
Referring to fig. 2, fig. 2 is a flow chart of a behavior recognition method according to a preferred embodiment of the invention. The method is applied to the computing device 100. The following describes the specific flow of the behavior recognition method in detail.
And step S120, carrying out image processing on the collected visible light image and infrared image to obtain a target area to be tracked.
In this embodiment, the computing device 100 may be communicatively coupled to a visible light camera and an infrared camera, respectively. Thus, the computing device 100 obtains a visible light image and an infrared image.
Referring to fig. 3, fig. 3 is a flowchart illustrating sub-steps included in step S120 in fig. 2. The step S120 may include a substep S121, a substep S122, and a substep S123.
And a substep S121, collecting multiple frames of visible light background images to establish a first background image, and processing the current frame image and the first background image to obtain a first target motion suspicious region.
In this embodiment, the computing device 100 establishes a first background image by using a gaussian model for the collected multiple frames of visible light images, so as to obtain a suspicious region of the first target motion according to the current frame image and the first background image. Meanwhile, in the tracking process, the first background image may be automatically updated according to a received update instruction or timing (e.g., 0.5 s).
The gaussian model is a model formed based on a gaussian probability density function (normal distribution curve) by accurately quantizing objects by using the gaussian probability density function (normal distribution curve) and decomposing one object into a plurality of objects.
The following describes how to obtain the suspicious region of the first object motion according to the current frame image and the first background image by way of example.
For example, the difference between the current frame image and the first background image is calculated in a certain area at two sides (i.e., an entrance and an exit) of the video monitoring area. And comparing the calculated difference with a preset difference threshold value, and processing according to a comparison result. And when the calculated difference is greater than a preset difference threshold value, taking the interval greater than the preset difference threshold value as the suspicious region of the first target motion. And when the calculated difference is smaller than the preset difference threshold value, no subsequent processing is performed. The preset difference threshold is suspicious set according to an actual situation (for example, 10 brightness values).
Image difference means that corresponding pixel values of two images are subtracted to weaken similar parts of the images and highlight changed parts of the images. For example, the differential image may detect the contour of a moving object and extract the trajectory of a blinking catheter.
And a substep S122, collecting a plurality of frames of infrared background images to establish a second background image, and processing the current infrared frame image and the second background image to obtain a second target motion suspicious region.
In this embodiment, the suspicious region of the second target motion is obtained in the same manner as the suspicious region of the first target motion, which is not described herein again.
And a substep S123, obtaining a target region to be tracked through the first target motion suspicious region and the second target motion suspicious region.
Referring to fig. 4, fig. 4 is a flowchart illustrating sub-steps included in sub-step S123 in fig. 3. The substep S123 may include substep S1231, substep S1232, substep S1233, and substep S1234.
And a substep S1231, calculating the overlapping degree of the first object motion suspicious region and the second object motion suspicious region.
In this embodiment, since the infrared image is related to the temperature, it can be assisted by the suspicious moving area of the second object to determine whether there is a pedestrian in the suspicious moving area of the first object. Firstly, the overlapping degree of a first target motion area and the second target motion suspicious area is calculated, and then the obtained overlapping degree is compared with a preset overlapping degree.
And a substep S1232 of determining whether the overlap is greater than a preset overlap.
The preset overlap degree may be set according to an actual situation (for example, 30%).
If the overlap is greater than the predetermined overlap, go to substep S1233.
And a substep S1233, processing the overlapping region of the first target movement suspicious region and the second target movement suspicious region to obtain a target region to be tracked.
The overlapping degree is larger than a preset overlapping degree, and the coverage area of the first target movement suspicious region can be adjusted according to the second target movement suspicious region. And then filling the cavity of the suspicious region of the first target motion and removing the region with the area smaller than the preset overlapping degree by adopting a morphological technology to obtain the target to be tracked. And then calculating the installation position of the camera, the image resolution, the position of the target to be tracked in the image, the height and width of the pedestrian and other characteristics to preliminarily determine the target area to be tracked.
If the overlap is smaller than the predetermined overlap, go to substep S1234.
And a substep S1234 of stopping the tracking judgment.
And the overlapping degree is less than the preset overlapping degree, the suspicious region of the first target motion is represented as a false region, and subsequent processing is not performed on the false region.
Step S130, detecting whether the target area to be tracked includes a pedestrian.
In this embodiment, a Hog algorithm is used to detect whether the target area to be tracked includes a pedestrian. The Hog (Histogram of Oriented Gradient) feature is a feature descriptor used for object detection in computer vision and image processing. It constructs features by calculating and counting the histogram of gradient direction of local area of image. The main idea is as follows: under the condition that the specific position of the edge is unknown, the distribution of the edge direction can well represent the outline of the pedestrian target. Thereby, it is possible to detect whether or not a pedestrian is included in the obtained image. If the pedestrian is not included, the subsequent steps are not carried out. If the pedestrian is included, the process proceeds to step S140.
And step S140, when the pedestrians are included, tracking the pedestrians, and detecting edges of the pedestrians in the tracking process to obtain a to-be-identified area of the pedestrians from the to-be-tracked target area.
In this embodiment, when pedestrians are included, a Mean-shift algorithm can be adopted to track each pedestrian, thereby monitoring the movement situation of the section. The Mean-shift algorithm is a target tracking algorithm based on Mean shift, and obtains descriptions about a target model and a candidate model by respectively calculating the probability of characteristic values of pixels in a target area and a candidate area, then measures the similarity of the target model of an initial frame and a candidate template of a current frame by using a similarity function, selects the candidate model with the maximum similarity function and obtains a Mean shift vector about the target model, and the vector is the vector of a target moving from an initial position to a correct position. Due to the rapid convergence of the mean shift algorithm, the mean shift vector is continuously and iteratively calculated, and the algorithm is finally converged to the real position of the target, so that the tracking purpose is achieved.
Meanwhile, a Snake algorithm can be adopted to detect the pedestrian edge of the pedestrian in the target area to be tracked in the tracking process, and then the area to be identified of the pedestrian is obtained. The Snake algorithm iterates on the basis of giving an initial contour (for example, a target region to be tracked), so that the contour approaches along the energy reduction direction, and finally an optimized boundary is obtained.
Step S150, inputting the area to be identified of the pedestrian into an identification model to obtain a behavior identification result of the pedestrian.
Referring to fig. 5, fig. 5 is a flowchart illustrating sub-steps included in step S150 in fig. 2. The recognition model includes a first recognition model. The step S150 may include the following sub-steps.
And a substep S151, inputting the to-be-recognized region of the pedestrian into a first recognition model to obtain a first output result, wherein the first output result comprises a plurality of behaviors and the weight corresponding to each behavior.
In this embodiment, the region to be recognized of the pedestrian is input into the first recognition model, the first recognition model is caused to calculate the weight of each behavior in the region, and then the first output result is obtained. The first output result comprises a plurality of behaviors and a weight corresponding to each behavior. The behaviors of the first output result may be arranged in a descending order or an ascending order according to the weights corresponding to the behaviors. Such as: normally walking for 0.5; stride 0.3, etc.
And a substep S152, taking the behavior corresponding to the weight greater than the preset weight threshold value as a first candidate result.
In this embodiment, the preset weight threshold may be set according to actual conditions.
And a substep S153, if the first candidate result includes only one behavior, the behavior is used as the recognition result.
In sub-step S154, if the first candidate result includes a plurality of behaviors, a recognition result is determined from the plurality of behaviors.
In one implementation of this embodiment, the behaviors in the first output result are arranged in descending order. When a difference between the weights of a preset number (for example, 2) of behaviors (for example, a difference between the maximum weight and a weight adjacent to the maximum weight) in the first output result is greater than a preset weight difference (for example, 0.4), a behavior corresponding to the maximum weight is taken as the recognition result. And when the difference value between the weights of the preset quantity of behaviors in the first output result is smaller than the preset weight difference value, determining the identification result from the preset quantity of behaviors.
Referring to fig. 6, fig. 6 is a flowchart illustrating sub-steps included in sub-step S154 in fig. 5. The recognition model further comprises a second recognition model. The sub-step S154 may include a sub-step S1541, a sub-step S1542, a sub-step S1543 and a sub-step S1544.
And S1541, extracting a first feature set of the to-be-identified pedestrian region according to the edge and the centroid of the to-be-identified pedestrian region.
In this embodiment, the edge and the centroid of the to-be-recognized area of the pedestrian are calculated to obtain the distance from the edge to the centroid. And taking the horizontal direction of the centroid as a starting point, rotating clockwise, taking every 30 degrees as an interval, and calculating the average value of the distances from the edge to the centroid in the interval, thereby obtaining the first feature set of the region to be identified of the pedestrian. Wherein the first set of features may include 12 features.
Wherein, the centroid is for abstract geometry, and for a solid object with uniform density, the centroid and the centroid coincide.
And S1542, calculating the geometric moment of the to-be-recognized pedestrian region, and extracting a second feature set of the to-be-recognized pedestrian region.
The moment features mainly characterize the geometric features of the image region, which are also called geometric moments, and the geometric moments are invariant features with characteristics of rotation, translation, scale and the like. Wherein the geometric moments comprise the hu moments, and the hu moments construct seven invariant matrices by using the second-order and third-order central moments, which can keep translation, scaling and rotation unchanged under continuous image conditions, thereby obtaining a second feature set. Wherein the second set of features may comprise 7 features.
And S1543, inputting the first feature set and the second feature set into a second recognition model to obtain a second output result.
Substep S1544, determining a recognition result from the plurality of behaviors according to the second output result.
In an implementation manner of this embodiment, a behavior corresponding to the maximum weight in the second output result is taken as the second candidate result. And if the second alternative result is one of the behaviors, taking the behavior as a recognition result. And if the second alternative result does not comprise any behavior in the behaviors, taking the target in the image as a key monitoring target, and continuing monitoring and judging in the next image.
And repeating the preset times (for example, 3 times) to perform behavior recognition, and if the preset times are repeated, still not obtaining a behavior recognition result, generating an alarm prompt. The preset times can be set according to actual conditions.
Referring to fig. 7, fig. 7 is a second schematic flow chart of the behavior recognition method according to the preferred embodiment of the invention. The method may further include step S160.
And step S160, executing a preset strategy according to the identification result.
In this embodiment, whether a preset abnormal behavior condition is satisfied is determined according to the recognition result. The preset abnormal behavior may include, but is not limited to, lighter ignition, kick-off, and the like. And if the recognition result does not meet the preset abnormal behavior condition, stopping performing key monitoring on the behavior corresponding to the recognition result. And if the identification result meets the preset abnormal behavior condition, representing that the pedestrian corresponding to the identification result has a damage tendency, performing key monitoring on the pedestrian corresponding to the identification result, and generating an alarm prompt.
For example, in a fire fighting system, the recognition model is used for performing behavior recognition on the behaviors of pedestrians. When the damage tendency of the pedestrian in the scene is judged according to the identification result, monitoring personnel are prompted in time, and therefore the pedestrian corresponding to the identification result is focused and/or protective measures are taken.
Referring to fig. 8, fig. 8 is a third schematic flow chart of a behavior recognition method according to a preferred embodiment of the invention. The method may further include step S110 before step S120.
And step S110, training to obtain a recognition model.
Referring to fig. 9, fig. 9 is a flowchart illustrating a part of sub-steps included in step S110 in fig. 8. The step S110 may include a substep S111, a substep S112, a substep S113, and a substep S114.
And a substep S111 of collecting behavior images of different behaviors of the pedestrian as sample images.
In the embodiment, different behavior images can be acquired according to the monitored occasions and targets. For example, when applied to a fire fighting system, different behaviors including normal walking, walking by steps, single-leg jumping, double-leg jumping, stooping, arm extension, leg kicking, punch, body collision, etc. may be collected for fire fighting.
And a substep S112, performing image processing on the sample image to obtain a first pedestrian region including pedestrians.
In the present embodiment, a pedestrian section in the sample image is detected by the Hog algorithm, thereby obtaining a first pedestrian region. The first pedestrian zone may also be corrected by manual detection.
And a substep S113, detecting the pedestrian edge of the pedestrian in the first pedestrian region, obtaining a second pedestrian region based on the pedestrian edge, and taking the second pedestrian region as the preprocessed sample image.
In this embodiment, a Snake model may be used to further detect the pedestrian edge of the pedestrian, so as to remove the interference of the background area, and the background area may be set to 0. The pedestrian edges may also be corrected by manual detection.
And the horizontal and vertical projection technology can be adopted to determine the positions of the upper, lower, left and right intervals of the pedestrian. The interval position can be represented by biAnd (i is 0, 1, 2, 3).
And a substep S114 of establishing a model base by the preprocessed sample images, and training according to the preprocessed sample images corresponding to each behavior in the model base to obtain a first recognition model.
In this embodiment, an edge position information normalization model library is formed by sample images after preset processing. Establishing different behavior characteristic sample model database Data according to sample images in the normalized model databasej(j ═ 0, …, N-1), N denotes the total number of behavior sample types. Multiple samples (e.g., 100) per class may be taken. Store each model library DatajPart of the sample (e.g., model library Data)j4/5 total number of samples) into the CNN model for training to obtain an initial first recognition model. The trained CNN model can extract the features of the image.
Then passes through the model library DatajThe initial first identification model is verified by the remaining part of samples, and if the first identification error rate is greater than the first error rate threshold value, the initial first identification model is adjusted according to the samples of the verification part to obtain the first identification model meeting the requirements. The first error rate threshold may be set according to actual conditions.
Referring to fig. 10, fig. 10 is a schematic flowchart of another part of sub-steps included in step S110 in fig. 8. The step S110 may further include a substep S116 and a substep S117.
And a substep S116, extracting a first feature set and a second feature set of the preprocessed sample image respectively.
Obtaining Data of each model libraryj(j-0, …, N-1) and the edge of the sample and the centroid of the image, and calculating the edge-to-centroid distance. Clockwise rotating by taking the horizontal direction of the centroid as a starting point, taking every 30 degrees as an interval, calculating the average value of the distances from the edge to the centroid in the interval, and taking the average value as a sample Sij(i 0-M, j 0-N), i.e., a first set of features, which may include 12 features, as represented by fi(i ═ 0, …, 11). Wherein, M is the number of samples in each class, and N is the total number of sample classes.
And calculating the geometric moments of the sample to obtain a second feature set of the sample. The second set of features may comprise 7 features, denoted as fi(i＝12，…，18)。
And a substep S117, training according to the first feature set and the second feature set corresponding to each behavior to obtain a second recognition model.
Feature f to be composed of a first set of features and a second set of featuresi(i-0, …, 18) training samples. Using each model library DatajPart of the sample (e.g., model library Data)j4/5 total amount of samples) training a random forest to obtain a random forest model. Then passes through the model library DatajThe random forest model is verified and adjusted by the remaining part of the samples to obtain a second recognition model.
Referring to fig. 11, fig. 11 is a block diagram illustrating a behavior recognition device 200 according to a preferred embodiment of the invention. The behavior recognition apparatus 200 is applied to the computing device 100. Wherein the computing device 100 comprises a recognition model for performing behavior recognition. The behavior recognizing apparatus 200 includes: a processing module 220, a detection module 230, and an identification module 250.
And the processing module 220 is configured to perform image processing on the collected visible light image and infrared image to obtain a target area to be tracked.
In the present embodiment, the processing module 220 is configured to execute step S120 in fig. 2, and the detailed description about the processing module 220 may refer to the description of step S120 in fig. 2.
A detecting module 230, configured to detect whether the target area to be tracked includes a pedestrian.
In this embodiment, the detecting module 230 is configured to execute step S130 in fig. 2, and the detailed description about the detecting module 230 may refer to the description of step S130 in fig. 2.
The processing module 220 is further configured to track the pedestrian when the pedestrian is included, and detect a pedestrian edge in the tracking process to obtain a to-be-identified pedestrian region from the to-be-tracked target region.
In this embodiment, the processing module 220 is further configured to execute step S140 in fig. 2, and the detailed description about the processing module 220 may refer to the description of step S140 in fig. 2.
And the identification module 250 is used for inputting the to-be-identified region of the pedestrian into an identification model to obtain a behavior identification result of the pedestrian.
In this embodiment, the identification module 250 is configured to execute step S150 in fig. 2, and the detailed description about the identification module 250 may refer to the description of step S150 in fig. 2.
In summary, the present invention provides a behavior recognition method and apparatus. The method is applied to a computing device comprising a recognition model for performing behavior recognition. And after the visible light image and the infrared image are obtained, performing image processing on the obtained images to obtain a target area to be tracked. When the target area to be tracked comprises the pedestrian, the pedestrian is tracked, and the edge of the pedestrian is detected in the tracking process to obtain the area to be identified of the pedestrian. And inputting the to-be-recognized area of the pedestrian into a recognition model to obtain a behavior recognition result of the pedestrian. Therefore, the pedestrian in the image is automatically identified, so that the occurrence of emergency can be prevented, and the harm is reduced.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A behavior recognition method applied to a computing device, wherein the computing device comprises a recognition model for behavior recognition, and the method comprises the following steps:
carrying out image processing on the collected visible light image and infrared image to obtain a target area to be tracked;
detecting whether the target area to be tracked comprises a pedestrian or not;
when the pedestrians are included, the pedestrians are tracked, and edges of the pedestrians are detected in the tracking process so as to obtain a to-be-identified area of the pedestrians from the to-be-tracked target area;
inputting the area to be recognized of the pedestrian into a recognition model to obtain a behavior recognition result of the pedestrian;
the identification model comprises a first identification model and a second identification model, and the step of inputting the region to be identified of the pedestrian into the identification model to obtain the behavior identification result comprises the following steps: inputting the to-be-recognized area of the pedestrian into a first recognition model to obtain a first output result, wherein the first output result comprises a plurality of behaviors and the weight corresponding to each behavior; taking the behavior corresponding to the weight larger than the preset weight threshold value as a first alternative result; if the first candidate result comprises a plurality of behaviors, determining a recognition result from the plurality of behaviors;
if the first candidate result includes a plurality of behaviors, the step of determining the recognition result from the plurality of behaviors includes: extracting a feature set of the region to be identified of the pedestrian, and inputting the feature set into the second identification model to obtain a second output result; and determining a recognition result from the plurality of behaviors according to the second output result.
2. The method according to claim 1, wherein the step of performing image processing on the collected visible light image and infrared image to obtain the target area to be tracked comprises:
collecting multiple frames of visible light background images to establish a first background image, and processing a current frame image and the first background image to obtain a first target motion suspicious region;
collecting multiple frames of infrared background images to establish a second background image, and processing the current infrared frame image and the second background image to obtain a second target movement suspicious region;
and obtaining a target area to be tracked through the first target movement suspicious area and the second target movement suspicious area.
3. The method according to claim 2, wherein the step of obtaining the target area to be tracked through the first target motion suspicious region and the second target motion suspicious region comprises:
calculating the overlapping degree of the first target movement suspicious region and the second target movement suspicious region;
if the overlapping degree is smaller than the preset overlapping degree, stopping tracking judgment;
and if the overlapping degree is greater than the preset overlapping degree, processing the overlapping area of the first target movement suspicious area and the second target movement suspicious area to obtain a target area to be tracked.
4. The method according to claim 1, wherein the step of inputting the pedestrian region to be recognized into a recognition model to obtain a behavior recognition result further comprises:
and if the first alternative result only comprises one behavior, the behavior is used as a recognition result.
5. The method according to claim 4, wherein the feature sets comprise a first feature set and a second feature set, and the step of extracting the feature set of the pedestrian to-be-identified region comprises:
extracting a first feature set of the to-be-identified region of the pedestrian according to the edge and the centroid of the to-be-identified region of the pedestrian;
and calculating the geometric moment of the to-be-recognized region of the pedestrian, and extracting a second feature set of the to-be-recognized region of the pedestrian.
6. The method according to claim 5, wherein the action corresponding to the largest weight in the second output result is taken as a second candidate result, and the step of determining the recognition result from the plurality of actions according to the second output result comprises:
if the second alternative result is one of the behaviors, taking the behavior as a recognition result;
if the second alternative result does not comprise any behavior in the plurality of behaviors, repeating the preset times to perform behavior identification;
and if the behavior recognition result cannot be obtained after the preset times are repeated, generating an alarm prompt.
7. The method of claim 1, further comprising:
executing a preset strategy according to the identification result;
the step of executing a preset strategy according to the recognition result comprises:
judging whether the recognition result meets a preset abnormal behavior condition or not;
and if the recognition result meets the preset abnormal behavior condition, performing key monitoring on the pedestrian corresponding to the recognition result, and generating an alarm prompt.
8. The method of claim 1, further comprising:
training to obtain a recognition model;
the step of training to obtain the recognition model comprises the following steps:
acquiring behavior images of different behaviors of pedestrians as sample images;
performing image processing on the sample image to obtain a first pedestrian area including pedestrians;
detecting the pedestrian edge of a pedestrian in the first pedestrian area, obtaining a second pedestrian area based on the pedestrian edge, and taking the second pedestrian area as a preprocessed sample image;
and establishing a model base by the preprocessed sample images, and training according to the preprocessed sample images corresponding to each behavior in the model base to obtain a first recognition model.
9. The method of claim 8, wherein the step of training the derived recognition model further comprises:
respectively extracting a first feature set and a second feature set of the preprocessed sample image;
and training according to the first feature set and the second feature set corresponding to each behavior to obtain a second recognition model.
10. A behavior recognition apparatus applied to a computing device, wherein the computing device comprises a recognition model for performing behavior recognition, the apparatus comprising:
the processing module is used for carrying out image processing on the collected visible light image and infrared image to obtain a target area to be tracked;
the detection module is used for detecting whether the target area to be tracked comprises a pedestrian or not;
the processing module is further used for tracking the pedestrians when the pedestrians comprise the target area, and detecting edges of the pedestrians in the tracking process to obtain the target area to be identified of the pedestrians from the target area to be tracked;
the identification module is used for inputting the to-be-identified region of the pedestrian into an identification model to obtain a behavior identification result of the pedestrian;
the identification model comprises a first identification model and a second identification model, and the mode that the identification module inputs the region to be identified of the pedestrian into the identification model to obtain the behavior identification result comprises the following steps: inputting the to-be-recognized area of the pedestrian into a first recognition model to obtain a first output result, wherein the first output result comprises a plurality of behaviors and the weight corresponding to each behavior; taking the behavior corresponding to the weight larger than the preset weight threshold value as a first alternative result; if the first candidate result comprises a plurality of behaviors, determining a recognition result from the plurality of behaviors;
when the first candidate result comprises a plurality of behaviors, the identifying module determines a recognition result from the plurality of behaviors by: extracting a feature set of the region to be identified of the pedestrian, and inputting the feature set into the second identification model to obtain a second output result; and determining a recognition result from the plurality of behaviors according to the second output result.
CN201710544459.1A 2017-07-06 2017-07-06 Behavior recognition method and device Active CN107247946B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710544459.1A CN107247946B (en) 2017-07-06 2017-07-06 Behavior recognition method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710544459.1A CN107247946B (en) 2017-07-06 2017-07-06 Behavior recognition method and device

Publications (2)

Publication Number Publication Date
CN107247946A CN107247946A (en) 2017-10-13
CN107247946B true CN107247946B (en) 2021-01-26

Family

ID=60013968

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710544459.1A Active CN107247946B (en) 2017-07-06 2017-07-06 Behavior recognition method and device

Country Status (1)

Country Link
CN (1) CN107247946B (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108983956B (en) * 2017-11-30 2021-07-06 成都通甲优博科技有限责任公司 Somatosensory interaction method and device
CN108229337B (en) * 2017-12-14 2021-03-30 创新先进技术有限公司 Data processing method, device and equipment
CN109377769B (en) * 2018-10-24 2022-03-15 东北林业大学 Pedestrian signal lamp timing system control method based on infrared thermal imaging technology
CN110059531B (en) * 2018-12-19 2021-06-01 浙江宇视科技有限公司 Frame-fighting behavior detection method and device based on video images
CN109709546B (en) * 2019-01-14 2021-11-16 珠海格力电器股份有限公司 Pet state monitoring method and device
CN109870250A (en) * 2019-01-27 2019-06-11 武汉星巡智能科技有限公司 Region exception body temperature monitoring method, device and computer readable storage medium
CN110223325B (en) * 2019-06-18 2021-04-27 北京字节跳动网络技术有限公司 Object tracking method, device and equipment
CN110751034B (en) * 2019-09-16 2023-09-01 平安科技（深圳）有限公司 Pedestrian behavior recognition method and terminal equipment
CN112084882A (en) * 2020-08-18 2020-12-15 深圳英飞拓科技股份有限公司 Behavior detection method and device and computer readable storage medium

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102096803B (en) * 2010-11-29 2013-11-13 吉林大学 Safe state recognition system for people on basis of machine vision
JP5592441B2 (en) * 2012-06-26 2014-09-17 本田技研工業株式会社 Light distribution control device
CN106327461B (en) * 2015-06-16 2019-11-15 浙江大华技术股份有限公司 A kind of image processing method and device for monitoring
CN105469054B (en) * 2015-11-25 2019-05-07 天津光电高斯通信工程技术股份有限公司 The model building method of normal behaviour and the detection method of abnormal behaviour
CN106571014A (en) * 2016-10-24 2017-04-19 上海伟赛智能科技有限公司 Method for identifying abnormal motion in video and system thereof

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Object Recognition for Human Behavior Analysis;Enes Dayangac 等;《IEEE》;20150209;第64-68页 *

Also Published As

Publication number Publication date
CN107247946A (en) 2017-10-13

Similar Documents

Publication Publication Date Title
CN107247946B (en) Behavior recognition method and device
US10007850B2 (en) System and method for event monitoring and detection
CN107644204B (en) Human body identification and tracking method for security system
US10059569B2 (en) Monitoring system of a passenger conveyor, a passenger conveyor, and a monitoring method thereof
Vishwakarma et al. Automatic detection of human fall in video
Lie et al. Human fall-down event detection based on 2D skeletons and deep learning approach
CN105279484B (en) Method for checking object and object test equipment
US9317765B2 (en) Human image tracking system, and human image detection and human image tracking methods thereof
CN113011367A (en) Abnormal behavior analysis method based on target track
US20200394384A1 (en) Real-time Aerial Suspicious Analysis (ASANA) System and Method for Identification of Suspicious individuals in public areas
CN110633643A (en) Abnormal behavior detection method and system for smart community
CN116563776A (en) Method, system, medium and equipment for warning illegal behaviors based on artificial intelligence
WO2012081969A1 (en) A system and method to detect intrusion event
CN107016350A (en) A kind of Falls Among Old People detection method based on depth camera
CN114943873B (en) Method and device for classifying abnormal behaviors of staff on construction site
KR102544492B1 (en) Apparatus and method of managing safety of swimming pool
CN116342645A (en) Multi-target tracking method for natatorium scene
CN115830536A (en) Method and device for detecting staying object and storage medium
Chong et al. Visual based fall detection with reduced complexity horprasert segmentation using superpixel
JP7491472B2 (en) Abnormality determination device, abnormality determination method, and abnormality determination program
Hagui et al. Coupled Hidden Markov Model for video fall detection
WO2020139071A1 (en) System and method for detecting aggressive behaviour activity
Aljarrah A computer vision system to detect diving cases in soccer
JP7422572B2 (en) Moving object tracking device, moving object tracking method, and moving object tracking program
Liu et al. Fire detection in radiant energy domain for video surveillance

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CP01 Change in the name or title of a patent holder

Address after: 100080 Room 301, 3 / F, 8 caihefang Road, Haidian District, Beijing

Patentee after: Beijing anningwell emergency fire safety technology Co.,Ltd.

Address before: 100080 Room 301, 3 / F, 8 caihefang Road, Haidian District, Beijing

Patentee before: BEIJING ANYWELL TECHNOLOGY DEVELOPMENT Co.,Ltd.

CP01 Change in the name or title of a patent holder