CN112580450B - Fast forward strategy-based method for rapidly detecting animal state in video - Google Patents
Fast forward strategy-based method for rapidly detecting animal state in video Download PDFInfo
- Publication number
- CN112580450B CN112580450B CN202011411815.0A CN202011411815A CN112580450B CN 112580450 B CN112580450 B CN 112580450B CN 202011411815 A CN202011411815 A CN 202011411815A CN 112580450 B CN112580450 B CN 112580450B
- Authority
- CN
- China
- Prior art keywords
- animal
- video
- frame
- image
- fast forward
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Human Computer Interaction (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a fast detection method for animal states in a video based on a fast forward strategy, which comprises the following steps: (1) acquiring an original video by an acquisition end; (2) the modeling end receives the original video; (3) Preprocessing the original video received in the step (2); (4) Establishing an animal position identification model in an image introducing a fast forward strategy; (5) Transmitting the model in the step (4) to a recognition end; (6) The identification end receives a new video and starts a rapid animal state detection method; (7) setting each parameter value in the fast-forward strategy; (8) And (4) detecting the first frame of the video by using the model in the step (4) and judging the position of the animal in the frame. (9) And (5) detecting key frames in the video according to the parameter values in the step (7), and finally judging the animal state. The animal position recognition model in the image is trained by using image data of a plurality of animal position labels as a training set.
Description
Technical Field
The invention belongs to the technical field of video image processing, and particularly relates to a method and a device for rapidly detecting animal states in a video based on a fast forward strategy, a terminal device and a computer readable storage medium.
Background
At present, when ecological or animal behavioural experiments are carried out, a 24-hour closed-circuit monitoring video is often used for collecting the behaviors of target animals, and then experimenters manually watch the video to carry out statistics on specific behaviors required by the experiments (such as statistics on eating frequency, sleeping practices and the like). When special parameters such as night sleeping time length are processed, the video image picture is almost unchanged because animals keep the same behavior for a long period of time. Either the traditional manual detection or the newly emerging artificial intelligence detection methods consume a lot of time and cost. Especially for artificial intelligence algorithm, detection can be carried out only frame by frame at present, and a great deal of useless work is carried out. Meanwhile, for manual detection, the method is high in subjectivity and poor in stability and reliability.
Disclosure of Invention
The invention aims to overcome the defects in the prior art, and provides a method, a device, terminal equipment and a computer-readable storage medium for rapidly detecting animal states in a video based on a fast forward strategy, which are used for detecting animal promptness rhythms in the video. The closed-circuit video information of the experimental animal is collected through the device, the data are transmitted to the terminal equipment, the terminal equipment processes the video, then the animal position identification model in the image is established, and the model is stored in a computer readable storage medium. And after the terminal receives the new video, the animal state detection can be carried out by using the model.
The purpose of the invention is realized by the following technical scheme:
a method for rapidly detecting animal states in videos based on a fast forward strategy comprises the following steps:
(1) Acquiring an original video by an acquisition end;
(2) The modeling end receives an original video;
(3) Preprocessing the original video received in the step (2), specifically converting the video into a single-frame image, adjusting the size of the image, and performing feature extraction and feature selection on the image;
(4) Establishing an animal position identification model in the image;
(5) Transmitting the animal position identification model to an identification end;
(6) The identification end receives a new video and starts a rapid animal state detection method;
(7) Setting each parameter value in the fast forward strategy;
(8) Detecting a first frame of the video by using the animal position identification model in the step (4), and judging the position of an animal in the frame;
(9) Detecting related key frames in the video according to each parameter value in the fast forward strategy in the step (4), and finally judging whether the animal state is awake or sleeping;
the animal position recognition model in the image is trained by using image data of a plurality of animal position labels as a training set.
Furthermore, the resolution of a single frame of the original video is 1280 pixels by 720 pixels, and the frame rate is 15 frames/second; the video format is the. Mp4,. Avi encoding format.
Further, the acquisition end uses 1080P, gathers and passes through 3 passageway camera equipment and transmit the video to the end of modelling through wired.
Further, the animal position identification model in the image in the step (4) is an improved Faster RCNN model, and the Faster RCNN model comprises a correction layer for correcting the score of each suggestion box.
Further, each suggestion box score is modified by the following equation:
y′ i (t)＝(1+μ i (t))y i (t)
in the formula y i (t) and y' i (t) scores before and after correction of the ith suggestion box in the t frame respectively; s (B) i (t)) and
the area of the ith suggestion box in the tth frame and the area of the part of the ith suggestion box intersected with the prediction box are respectively.
Further, the parameters in the fast forward strategy include: a current acceleration level CL (current level); frame skip number FS (frame skip); whether to enter a next acceleration level judgment value SU (speed up); the label CF of the current detection frame; detecting the total frame number totalfame of a video currently; detection threshold MAXMOVE: when the distance between the positions of the animal in the two previous and next detections is less than the threshold, the animal is considered not to move, and MINDURATION is a minimum duration threshold: and when the detected frequency of the distance between the positions of the animals between the two frames is less than the MAXMOVE is more than the threshold value, performing frame skipping detection.
The invention also provides a fast detection device for animal states in videos based on a fast forward strategy, which comprises:
the acquisition end is used for acquiring animal behavior and action videos;
the modeling end is used for processing the collected videos, labeling and the like to obtain an animal position identification model in the image;
and the identification end is used for storing the animal position identification model in the image and identifying the animal state in the input video.
The invention also provides a terminal device for rapidly detecting the animal state in the video based on the fast forward strategy, which comprises a memory, a processor and a computer program which is stored in the memory and can run on the processor, and is characterized in that the processor can realize a method for rapidly detecting the animal state when executing the computer program.
The invention also provides a computer readable storage medium, which stores a computer program, wherein the computer program is capable of implementing a method for rapidly detecting an animal state when executed by a processor.
Compared with the prior art, the technical scheme of the invention has the following beneficial effects:
1. the method provides a complete method flow for detecting the animal state, the deep learning model established by the identification end can automatically detect the animal state, the human intervention is not needed, the human and time cost can be reduced, and the detection efficiency is greatly improved.
2. The animal state identification and detection by manpower has subjectivity, the identification result is unstable, and different people have different identification results. The model established by the embodiment of the invention can detect the animal state according to the self algorithm logic, and the reliability and the repeatability of the detection result are ensured.
3. The fast-forward strategy provided by the invention can effectively improve the video detection speed, and is particularly suitable for videos with animal states kept unchanged for a long time, such as field monitoring videos, laboratory night monitoring and the like. Through testing, the algorithm can save 46.23% of time on average on the premise of not losing precision.
Drawings
FIG. 1 is a schematic structural diagram of a device for rapidly detecting an animal status in a video according to the present invention;
fig. 2 is a schematic flow chart of a method for establishing an animal position identification model in an image according to an embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the figures and specific examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As shown in fig. 1, an apparatus for rapidly detecting an animal status in a video based on a fast forward strategy includes: the acquisition end is used for acquiring animal behavior and action videos; the modeling end is used for processing the collected videos, labeling and the like to obtain an animal position identification model in the image; and the identification end is used for storing the animal position identification model in the image and identifying the animal state in the input video.
The method for rapidly detecting the animal state in the video based on the fast forwarding strategy is provided based on the detection device, and comprises the following steps:
(1) Acquiring an original video by an acquisition end; the camera carries out video recording according to an experimental plan, and video files with 1280 pixels by 720 pixels, a frame rate of 15 frames/second and video formats of mp4, avi and other encoding formats are obtained. And after a period of recording, transmitting the recorded video to the modeling terminal.
(2) The modeling end receives an original video;
(3) Preprocessing the original video received in the step (2), specifically converting the video into a single-frame image, adjusting the size of the image to be 320 pixels by 240 pixels, extracting and selecting the features of the image, and marking the position of an animal in the single-frame image;
(4) Establishing an animal position identification model in an image, as shown in fig. 2, a schematic flow chart of a method for establishing an animal position identification model in an image in an embodiment of the present invention specifically includes the following steps S11 to S14:
step S11: establishing a training sample set according to the image and the marked animal position, inputting the training sample set into an original image, and outputting the training sample set into an image with a label;
step S12: setting network layer parameters such as an input layer, a convolution layer and the like according to the size of the image;
step S13: and establishing an animal position identification model based on fast RCNN. Adding a correction layer to correct the suggestion box score, and for a video frame F (t) at the time t, wherein the approximate position of the living being
The prediction can be made from the position and velocity of the previously detected frame:
v (t) is the speed of the biological current frame; l (t-1) and L (t-2) are respectively the animal positions detected by the animal position identification model at the t-1 moment and the t-2 moment; t is the time interval between two detection frames. Its prediction box size can be expressed as:
w (t) and H (t) are the width and height of the current frame prediction frame, respectively, and can be calculated from the average value of the width and height of the detection frame in the n detection frames before the current detection frame, and W (i) and H (i) are the width and height of the ith detection frame in the n detection frames before the current detection frame, respectively, and n =3 is taken in the present study. For each suggestion box B in the ROI posing layer i The score may be modified as follows:
y′ i (t)＝(1+μ i (t))y i (t)
in the formula y i (t) and y' i (t) scores before and after correction of the ith suggestion box in the t frame respectively; s (B) i (t)) and
the area of the ith suggestion box in the tth frame and the area of the part of the ith suggestion box intersected with the prediction box are respectively.
Step S14: model parameters were trained using the adam algorithm.
(5) Transmitting the animal position identification model in the step (4) to an identification end;
(6) The identification end receives a new video and starts a rapid animal state detection method;
(7) Setting each parameter value in the fast-forward strategy, wherein CL is the current acceleration level; FS is the number of skipped frames (frame skip); SU is whether to enter a next acceleration level judgment value (speed up); CF is the label of the current detection frame; totalfame is the total frame number of the current detection video; MAXMOVE is the detection threshold: when the distance between the positions of the animal in the two previous and next detections is less than the threshold, the animal is considered not to move, and MINDURATION is a minimum duration threshold: and when the detected frequency that the distance between the positions of the animals between the two frames is smaller than the MAXMOVE is larger than the threshold value, frame skipping detection can be carried out. In the present embodiment, CL =0 is set; FS =1; SU =0; CF =1.
(8) And (4) detecting a first frame of the video by using the animal position identification model in the step (4), and judging the position of the animal in the frame.
(9) Detecting related key frames in the video according to each parameter value in the fast forward strategy in the step (4), wherein an algorithm flow chart is as follows, and the algorithm can be understood as follows: when the animal is detected to be present at a certain position within the error range (< = MAXMOVE) for a plurality of times (> = mindetermination), the animal is considered not to move (or no animal is present in the video), and the detection speed can be increased to perform frame skipping detection (FS frame skipping). The animal status (awake/sleeping) is finally judged.
The present invention is not limited to the embodiments described above. The foregoing description of the specific embodiments is intended to describe and illustrate the technical solutions of the present invention, and the specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make various changes in form and details without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (6)
1. A method for rapidly detecting animal states in videos based on a fast forward strategy is characterized by comprising the following steps:
(1) Acquiring an original video by an acquisition end;
(2) The modeling end receives an original video;
(3) Preprocessing the original video received in the step (2), specifically converting the video into a single-frame image, adjusting the image size, and performing feature extraction and feature selection on the image;
(4) Establishing an animal position identification model in an image introducing a fast forward strategy; the animal position identification model in the image is an improved Faster RCNN model, and the Faster RCNN model comprises a correction layer for correcting scores of all the suggestion frames; each suggestion box score is modified using the following equation:
y′ i (t)＝(1+μ i (t))y i (t)
in the formula y i (t) and y' i (t) scoring before and after the modification of the ith suggestion box in the tth frame, respectively; s (B) i (t)) and
the area of the part where the ith suggestion frame intersects with the prediction frame and the area of the part where the ith suggestion frame intersects with the prediction frame in the tth frame are respectively the area of the ith suggestion frame;
(5) Transmitting the animal position identification model to an identification end;
(6) The identification end receives a new video and starts a rapid animal state detection method;
(7) Setting each parameter value in the fast forward strategy; the parameters in the fast forward strategy include: a current acceleration level CL (current level); frame skip number FS (frame skip); whether to enter a next acceleration level judgment value SU (speed up); the label CF of the current detection frame; detecting the total frame number totalfame of a video currently; detection threshold MAXMOVE: when the distance between the positions of the animal in the two previous and next detections is less than the threshold, the animal is considered not to move, and MINDURATION is a minimum duration threshold: when the detected frequency of the distance between the positions of the animals between the two frames is less than the MAXMOVE and is more than the threshold value, frame skipping detection is carried out;
(8) Detecting a first frame of the video by using the animal position identification model in the step (4), and judging the position of an animal in the frame;
(9) Detecting related key frames in the video according to the parameter values in the fast forward strategy in the step (4), and finally judging whether the animal state is awake or asleep;
the animal position recognition model in the image is trained by using image data of a plurality of animal position labels as a training set.
2. The method according to claim 1, wherein the original video has a single frame resolution of 1280 pixels by 720 pixels and a frame rate of 15 frames/sec; the video format is the. Mp 4. Avi encoding format.
3. The method for rapidly detecting the animal state in the video based on the fast-forwarding strategy as claimed in claim 1, wherein the collection terminal collects the animal state through a 3-channel camera device and transmits the video to the modeling terminal through a wire by using 1080P.
4. An animal state rapid detection device in video based on a fast forward strategy, based on any one of the animal state rapid detection method of claims 1 to 3, characterized by comprising:
the acquisition end is used for acquiring animal behavior and action videos;
the modeling end is used for processing the collected videos, labeling and the like to obtain an animal position identification model in the image;
and the identification end is used for storing the animal position identification model in the image and identifying the animal state in the input video.
5. A terminal device for fast detecting animal state in video based on fast forward strategy, comprising a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor when executing the computer program implements the steps of the method for fast detecting animal state according to any one of claims 1 to 4.
6. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out the steps of the animal status rapid detection method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011411815.0A CN112580450B (en) | 2020-12-03 | 2020-12-03 | Fast forward strategy-based method for rapidly detecting animal state in video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011411815.0A CN112580450B (en) | 2020-12-03 | 2020-12-03 | Fast forward strategy-based method for rapidly detecting animal state in video |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112580450A CN112580450A (en) | 2021-03-30 |
| CN112580450B true CN112580450B (en) | 2022-11-18 |
Family
ID=75127363
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011411815.0A Active CN112580450B (en) | 2020-12-03 | 2020-12-03 | Fast forward strategy-based method for rapidly detecting animal state in video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112580450B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117152799B (en) * | 2023-10-31 | 2024-02-09 | 深圳市盛航特科技有限公司 | Animal identification method, apparatus, terminal device and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108241854A (en) * | 2018-01-02 | 2018-07-03 | 天津大学 | A kind of deep video conspicuousness detection method based on movement and recall info |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110147702B (en) * | 2018-07-13 | 2023-05-23 | 腾讯科技（深圳）有限公司 | Method and system for detecting and identifying target of real-time video |
| EP3739503B1 (en) * | 2019-05-14 | 2023-10-25 | Nokia Technologies Oy | Video processing |
| CN110309792B (en) * | 2019-07-04 | 2022-07-01 | 电子科技大学 | Indoor person detection method based on component template |
| CN110765951B (en) * | 2019-10-24 | 2023-03-10 | 西安电子科技大学 | Remote sensing image airplane target detection method based on bounding box correction algorithm |
-
2020
- 2020-12-03 CN CN202011411815.0A patent/CN112580450B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108241854A (en) * | 2018-01-02 | 2018-07-03 | 天津大学 | A kind of deep video conspicuousness detection method based on movement and recall info |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112580450A (en) | 2021-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108710865B (en) | Driver abnormal behavior detection method based on neural network | |
| CN108288015B (en) | Human body action recognition method and system in video based on time scale invariance | |
| CN111738218B (en) | Human body abnormal behavior recognition system and method | |
| CN109472226B (en) | Sleeping behavior detection method based on deep learning | |
| CN114004866B (en) | Mosquito recognition system and method based on image similarity difference | |
| CN110348349A (en) | A kind of method and system collected, analyze pig behavior video data | |
| CN112580450B (en) | Fast forward strategy-based method for rapidly detecting animal state in video | |
| CN110096945B (en) | Indoor monitoring video key frame real-time extraction method based on machine learning | |
| CN115049966A (en) | GhostNet-based lightweight YOLO pet identification method | |
| CN114359791B (en) | Group macaque appetite detection method based on Yolo v5 network and SlowFast network | |
| CN110942045A (en) | Intelligent fish tank feeding system based on machine vision | |
| CN113963298A (en) | Wild animal identification tracking and behavior detection system, method, equipment and storage medium based on computer vision | |
| CN107992937A (en) | Unstructured data decision method and device based on deep learning | |
| CN116824726A (en) | Campus environment intelligent inspection method and system | |
| CN116824626A (en) | Artificial intelligent identification method for abnormal state of animal | |
| CN106529460A (en) | Object classification identification system and identification method based on robot side | |
| CN112184771B (en) | Method and device for tracking personnel track of community | |
| CN116886869A (en) | Video monitoring system and video tracing method based on AI | |
| CN112396033A (en) | Bird background rhythm detection method and device, terminal equipment and storage medium | |
| CN115830078A (en) | Live pig multi-target tracking and behavior recognition method, computer equipment and storage medium | |
| CN115497021A (en) | Method for identifying fine granularity of sow lactation behavior based on computer vision | |
| CN113192106B (en) | Livestock tracking method and device | |
| CN114677614A (en) | Single sow lactation time length calculation method based on computer vision | |
| CN111160224B (en) | High-speed rail contact net foreign matter detection system and method based on FPGA and horizon line segmentation | |
| CN211956492U (en) | Image processing system for visual navigation robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |