CN111898518A - Tumble detection method, electronic device and storage medium - Google Patents
Tumble detection method, electronic device and storage medium Download PDFInfo
- Publication number
- CN111898518A CN111898518A CN202010737111.6A CN202010737111A CN111898518A CN 111898518 A CN111898518 A CN 111898518A CN 202010737111 A CN202010737111 A CN 202010737111A CN 111898518 A CN111898518 A CN 111898518A
- Authority
- CN
- China
- Prior art keywords
- frame
- human body
- fall
- detection
- temporary representative
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 120
- 230000009471 action Effects 0.000 claims abstract description 35
- 230000033001 locomotion Effects 0.000 claims description 49
- 238000004590 computer program Methods 0.000 claims description 5
- 239000000126 substance Substances 0.000 claims 1
- 230000036544 posture Effects 0.000 description 46
- 238000000034 method Methods 0.000 description 14
- 210000003423 ankle Anatomy 0.000 description 13
- 230000003044 adaptive effect Effects 0.000 description 8
- 230000006835 compression Effects 0.000 description 7
- 238000007906 compression Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 238000012216 screening Methods 0.000 description 5
- 230000006399 behavior Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000007547 defect Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000005452 bending Methods 0.000 description 2
- 230000001427 coherent effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000009191 jumping Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
Abstract
The embodiment of the invention relates to the field of computer vision, and discloses a tumble detection method, electronic equipment and a storage medium. The invention obtains temporary representative frames from the video stream to detect the human body frame; if the human body frame is detected, adding the temporary representative frame into a queue, acquiring a frame which is away from the temporary representative frame by a preset interval frame number, and taking the frame as an updated temporary representative frame to detect the human body frame; if the human body frame is not detected, acquiring the next frame of the temporary representative frame as an updated temporary representative frame to detect the human body frame; detecting the human body posture of the frames in the queue; and detecting the falling action according to the result of the human body posture detection. The frame skipping detection mode avoids high time overhead caused by frame-by-frame detection, reduces the frequency of frame target detection failure compared with fixed interval frame skipping detection, and in addition, fully utilizes human body posture information, so that the characteristics are richer and more comprehensive, and the influence of the background environment and the falling posture on the model identification capability is weakened.
Description
Technical Field
The embodiment of the invention relates to the field of computer vision, in particular to a tumble detection method, electronic equipment and a storage medium.
Background
The falling behavior detection has wide application in practice, is mainly suitable for the elderly living alone, the young children and the weak patients in the family environment, detects the abnormal falling behavior in real time by monitoring the activity condition of the human body, and acquires the abnormal falling early warning.
The inventor finds that at least the following problems exist in the prior art: the current falling behavior detection method processes video information flow, frame-by-frame or fixed interval frame skipping processing images to acquire human body conditions and detect falling behaviors, however, the frame-by-frame processing images can increase the time overhead of image processing, the fixed interval frame skipping processing images can cause frame target detection failure, and the requirements of high real-time performance and high stability of data during model building cannot be met.
Disclosure of Invention
An object of the embodiments of the present invention is to provide a fall detection method, an electronic device, and a storage medium, which can reduce the time overhead of processing a frame picture of a video stream and reduce the frequency of frame target detection failure when performing fall detection on the video stream, thereby reducing the negative effect caused by the frame target detection failure, and can fully utilize human body posture information to detect a fall action, and reduce the influence of a background environment and a fall posture on model recognition capability.
In order to solve the above technical problem, an embodiment of the present invention provides a fall detection method, including: detecting a human body frame of the temporary representative frame, wherein the temporary representative frame is derived from a frame image of the video stream; if the human body frame is detected, adding the temporary representative frame into a queue, acquiring a frame which is away from the temporary representative frame by a preset interval frame number, and taking the frame as an updated temporary representative frame to detect the human body frame; if the human body frame is not detected, acquiring the next frame of the temporary representative frame as an updated temporary representative frame to detect the human body frame; detecting the human body posture of the frames in the queue; and detecting the falling action according to the result of the human body posture detection.
An embodiment of the present invention further provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executable by the at least one processor to enable the at least one processor to perform the fall detection method applied to the terminal device.
Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which when executed by a processor implements the fall detection method described above.
Compared with the prior art, the embodiment of the invention selects the temporary representative frame from the frame information of the video stream to detect the human body frame; if the human body frame is detected, adding the temporary representative frame into a queue, acquiring a frame which is away from the temporary representative frame by a preset interval frame number, and taking the frame as an updated temporary representative frame to detect the human body frame; if the human body frame is not detected, acquiring the next frame of the temporary representative frame as an updated temporary representative frame to detect the human body frame; detecting the human body posture of the frames in the queue; and detecting the falling action according to the result of the human body posture detection. The frame skipping detection mode avoids high time overhead brought by frame-by-frame detection, reduces the frequency of frame target detection failure compared with a frame skipping detection image with fixed intervals, further reduces the problem that the established tumble model lacks real-time data due to the frame target detection failure, and improves the stability of the established tumble model.
In addition, detecting a falling motion according to a result of the human posture detection includes: and detecting the tumbling action according to the human body posture detection results of two continuous frames in the queue. In the implementation, the falling action is detected by combining the posture information of two continuous frames, the coherent motion posture of the human body is effectively expressed, and the defect of indefinite motion trend in single-frame detection is avoided.
In addition, detecting a falling motion according to a result of the human posture detection includes: calculating motion information according to the human body posture detection results of two continuous frames in the queue, wherein the motion information comprises transverse motion information and longitudinal motion information; and detecting the falling action according to the motion information. In the implementation, the transverse motion information and the longitudinal motion information are calculated by using the human body posture detection result in the falling detection process, the longitudinal information and the transverse information are considered, so that the false detection rate of similar actions such as squatting, sitting, bending and the like is reduced, and the detection precision of the model is improved.
In addition, detecting a fall action from the movement information includes: taking the frame meeting the preset condition in the queue as a falling frame, wherein the preset condition comprises the following steps: if the transverse motion information is in a preset transverse motion parameter range, and the longitudinal motion information is in a preset longitudinal motion parameter range; and detecting a falling action according to the falling frame and the frame adjacent to the falling frame in the queue. In the implementation, the continuity of the tumbling action is considered by detecting two adjacent frames, and the uncertainty of single-frame detection is avoided.
In addition, detecting a fall action according to the fall frame and the frame in the queue adjacent to the fall frame includes: and if the adjacent frames meet the preset condition and the frame distance between the falling frames and the last falling frame in the queue exceeds a preset frame number, judging that the falling action is detected. In this realization, judge whether the frame interval of present fall frame and last fall frame satisfies the requirement, can reduce to take place for the condition of a plurality of fall actions with a repeated detection of fall action.
Before detecting the human body frame for the temporary representative frame, the method further includes: and acquiring a frame image of the video stream from the monocular camera. In the implementation, the characteristics are constructed by using the information acquired by the monocular camera, so that the cost of hardware equipment is reduced.
In addition, after acquiring the frame image of the video stream from the monocular camera, the method further includes: and compressing the frame image according to the real image proportion. The real distance is mapped into the pixel distance by using an image compression mode, so that the defect that the monocular camera is lack of depth information is overcome.
Drawings
One or more embodiments are illustrated by the corresponding figures in the drawings, which are not meant to be limiting.
FIG. 1 is a flow chart of a thread for adaptive interval frame skipping frame screening of frames with human body frames in a first embodiment of the present application;
fig. 2 is a flow chart of a thread for implementing 2D posture detection and fall discrimination in the first embodiment of the present application;
fig. 3 is a flow chart of a thread for implementing 2D posture detection and fall discrimination in a second embodiment of the present application;
FIG. 4 is a flow chart of a thread for adaptive interval frame skipping frame screening of frames with human body frames according to a third embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device in a fourth embodiment of the present application.
DETAILED DESCRIPTION OF EMBODIMENT (S) OF INVENTION
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, it will be appreciated by those of ordinary skill in the art that in various embodiments of the invention, numerous technical details are set forth in order to provide a better understanding of the present application. However, the technical solution claimed in the present application can be implemented without these technical details and various changes and modifications based on the following embodiments. The following embodiments are divided for convenience of description, and should not constitute any limitation to the specific implementation manner of the present invention, and the embodiments may be mutually incorporated and referred to without contradiction.
A first embodiment of the invention relates to a fall detection method. In this embodiment, the fall detection method may be applied to fall detection equipment, which may be embedded equipment or a computer communicatively connected to a camera, or may be monitoring equipment with a camera. The embodiment comprises the following steps: detecting a human body frame of the temporary representative frame, wherein the temporary representative frame is derived from a frame image of the video stream; if the human body frame is detected, adding the temporary representative frame into a queue, acquiring a frame which is away from the temporary representative frame by a preset interval frame number, and taking the frame as an updated temporary representative frame to detect the human body frame; if the human body frame is not detected, acquiring the next frame of the temporary representative frame as an updated temporary representative frame to detect the human body frame; detecting the human body posture of the frames in the queue; and detecting the falling action according to the result of the human body posture detection. When the video stream is subjected to tumble detection, the time overhead of processing the frame picture of the video stream can be reduced, the frequency of frame target detection failure is reduced, the negative effect caused by the frame target detection failure is weakened, the human posture information can be fully utilized to detect the tumble action, and the influence of the background environment and the tumble posture on the model recognition capability is reduced. The implementation details of the fall detection method of the present embodiment are specifically described below, and the following description is only provided for the convenience of understanding, and is not necessary for implementing the present embodiment.
In this embodiment, two threads are initialized, one thread is used to read frame-by-frame information of a surveillance video stream, perform adaptive frame skipping at intervals to screen frames with human frames, and initialize a queue, a flow chart of the thread is shown in fig. 1, and the other thread is used to implement 2D posture detection and fall discrimination, and a flow chart of the thread is shown in fig. 2. The flow of fig. 1 and 2 will be described in detail below.
The following is a detailed description of a thread for reading frame-by-frame information of a surveillance video stream and performing adaptive frame skipping at intervals to screen frames with human frames, as shown in fig. 1.
And 101, acquiring a temporary representative frame from the video stream to detect a human body frame.
Specifically, a frame of a video stream shot from a camera is taken as a temporary representative frame, for example, the first frame in the video stream is taken as the temporary representative frame, and the detection of a human body frame is performed on the temporary representative frame.
And 103, acquiring the next frame of the temporary representative frame as the updated temporary representative frame, and continuing to detect the human body frame.
And 104, adding the temporary representative frames into a queue, acquiring frames which are far away from the temporary representative frames by preset interval frame numbers, taking the frames as updated temporary representative frames, and continuously detecting the human body frame. It should be noted that if the number of frames between the temporary representative frame and the last frame of the video stream is less than the preset interval frame number, the last frame of the video stream is used as the updated temporary representative frame, and the detection of the human body frame is continued.
In one example, the fall detection device starts a thread to read frame-by-frame information of the video stream. Selecting a first frame of a video stream as a temporary representative frame; performing target detection on the extracted temporary representative frame based on a human body posture detection algorithm YoloV 3; judging whether a human body frame exists in the temporary representative frame; if the human body frame is detected, adding the frame into the queue Q, acquiring a frame which is a distance from the temporary representative frame with the preset interval frame number, and performing human body frame detection as an updated temporary representative frame; and if the human body frame is not detected in the temporary representative frame, selecting the next frame adjacent to the temporary representative frame as the temporary representative frame for human body frame detection, and if the human body frame is detected, adding the frame into the queue Q. In one example, if no frame is detected in the temporary representative frame, selecting the next frame next to the temporary representative frame as the temporary representative frame for frame detection, and if a frame is detected, adding the frame to the queue Q, including: if it is subsequent to the temporary representative frame
If the frame with the detected human body frame exists in the frame, the frame with the detected human body frame is added into a queue Q, and if the frame temporarily represents the follow-up of the frame
If no frame is detected, taking gap integer from initial frame
Is detected, wherein,
the time threshold is preset and is used for indicating the upper limit time of continuously selecting the next frame as a temporary representative frame. E.g. a preset number threshold
If the value is 2 and the gap value is 5, selecting a first frame to detect the human body frame, and if the 1 st frame detects the human body frame, adding the 1 st frame into a queue and selecting a 6 th frame to detect the human body frame; if the first frame does not detect the human body frame, selecting a 2 nd frame to detect the human body frame; if the frame 2 detects the human body frame, adding the frame 2 into a queue and selecting a frame 6 to detect the human body frame; if the human body frame is not detected in the 2 nd frame, selecting a 3 rd frame to detect the human body frame; if the human body frame is detected in the 3 rd frame, adding the 3 rd frame into a queue and selecting the 6 th frame to detect the human body frame; if the human body frame is not detected, because the operation of selecting the next frame to detect the human body frame is performed twice, the set time threshold 2 is reached, and therefore continuous frame skipping is not performed any more, and at this time, the 6 th frame to detect the human body frame is selected.
The following is a detailed description of the thread for realizing 2D posture detection and fall discrimination as shown in fig. 2.
In one example, the fall detection device starts another thread to perform human pose detection on the frames in the created queue, takes a picture from the queue Q, and bases on the human poseThe state detection algorithm AlphaPose performs human body posture detection on the human body, and performs human body posture detection on the human body in a specified data format, for example: openpos, which outputs 18 pieces of node information. Acquiring horizontal and vertical pixel coordinates of a neck node (node 1), a waist node (nodes 8 and 11) and an ankle node (nodes 10 and 13) of a current frame from a human body posture detection result, such as 18 pieces of node information, wherein the pixel coordinates of the neck, left waist, right waist, left ankle and right ankle are respectively marked as (x)1,y1)、(x8,y8)、(x11,y11)、(x10,y10)、(x13,y13) (ii) a The neck node coordinates are (x)neck,yneck)＝(x1,y1) (ii) a In calculating waist node (x)hip,yhip) When the pixel coordinates are detected, if the pixel coordinates of the left waist node and the right waist node are detected, the coordinate average value is taken; if only one node of the left waist node and the right waist node is detected, taking the coordinate of the node; if the left waist node and the right waist node are not detected, the waist node pixel coordinate is (0, 0); at the calculation of ankle node (x)ankle,yankle) When the pixel coordinates are detected, if the pixel coordinates of the left ankle node and the right ankle node are detected, the average value of the coordinates is taken; if only one node of the left ankle node and the right ankle node is detected, taking the coordinate of the node; if neither the left ankle node nor the right ankle node is detected, the ankle node pixel coordinate is (0, 0).
Step 106: and detecting the falling action according to the human body posture detection result.
In one example, the relative position of the human body node is determined according to the posture detection result of the single frame image, for example, the coordinates of the 18 nodes of the human body, and the fall is detected according to the relative position of the human body node.
In the embodiment, the frame skipping detection mode avoids high time overhead brought by frame-by-frame detection, and compared with a frame skipping detection image at a fixed interval, the frequency of frame target detection failure is reduced, so that the problem that the established tumble model lacks real-time data due to the frame target detection failure is solved, and the stability of the established tumble model is improved.
The second embodiment of the invention relates to a fall detection method, which is substantially the same as the first embodiment, and two threads need to be initialized, wherein one thread is used for reading frame-by-frame information of a monitoring video stream, performing adaptive frame skipping at intervals, screening frames with human body frames, and initializing a queue, and the other thread is used for realizing 2D posture detection and fall judgment.
The thread for reading frame-by-frame information of a surveillance video stream and performing adaptive frame skipping at intervals to screen frames with human frames in this embodiment is the same as the steps of fig. 1 in the first embodiment, and the thread step includes: acquiring a temporary representative frame from the video stream to detect a human body frame; judging whether the temporary representative frame has a human body frame or not, if not, acquiring the next frame of the temporary representative frame as an updated temporary representative frame, and continuing to detect the human body frame; and if the human body frame exists, adding the temporary representative frame into the queue, acquiring a frame which is away from the temporary representative frame by a preset interval frame number, taking the frame as an updated temporary representative frame, and continuously detecting the human body frame. The present embodiment is different from the first embodiment in the thread for realizing 2D posture detection and fall discrimination, and the steps thereof are specifically described below as shown in fig. 3.
And step 302, calculating motion information according to the human body posture results of two continuous frames.
In one example, the lateral motion information is calculated according to 18 node information obtained by human body posture detection, for example, the lateral coordinate transformation situation of two adjacent frames of waist nodes is obtained, and the module value | x of the lateral coordinate difference of two adjacent frames of waist nodes is takenhipL as the lateral velocity value
Further, get the currentThe modulus of the difference between the transverse coordinates of the previous frame and the transverse coordinates of the current frame is obtained
Obtaining a modulus of a difference between the transverse coordinates of the current frame and the transverse coordinates of a frame next to the current frame
Get
And
is used as the horizontal average speed value of the current frame
The formula is as follows:
acquiring longitudinal distance according to 18 node information
The longitudinal distance is the longitudinal coordinate of ankle node
Subtracting the longitudinal coordinates of the neck node
The formula is as follows:
acquiring longitudinal motion information according to 18 node information: calculating the longitudinal distance of the current frame
Longitudinal distance from the previous frameSeparation device
The ratio of the distance to the distance is used as the distance change rate of the current frame
The formula is as follows:
In one example, a fall action is detected based on the motion information. Firstly, initializing a counter; judging whether the longitudinal motion information and the transverse motion information of the current frame meet the tumbling condition, if the longitudinal motion information of the current frame is in the longitudinal motion parameters, for example, the distance change rate is in
With transverse movement information within transverse movement parameters, e.g. transverse waist node offset speed
The counter count is incremented by 1; and if the current frame does not meet the speed requirement, resetting the counter count. If the counter count of two consecutive frames is increased by 1 and the frame interval from the last detected frame with the falling motion is greater than the preset frame number, for example, 10 frames, the falling motion is considered to occur.
In step 304, no fall action is detected.
In one example, when no fall is detected, the frames in the queue are continuously taken for fall detection until all the frames in the queue are detected.
In step 305, a fall action is detected.
In one example, the frames in which the fall is detected are saved, and the frames in the queue are continuously taken for fall detection until all the frames in the queue are detected.
In one example, a counter count is initialized; detecting human body postures of frames in the queue, calculating transverse motion information and longitudinal motion information according to human body posture detection results of two continuous frames, such as 18 node information, and judging whether the longitudinal motion information and the transverse motion information of the current frame meet a tumbling condition, specifically, if the longitudinal motion information of the current frame, such as a distance change rate
And lateral movement information, e.g. lateral deflection speed of waist node
The counter count is incremented by 1; and if the current frame does not meet the speed requirement, resetting the counter count. If only one frame counter is added with 1, but the longitudinal distance between the neck node and the ankle node of the current frame
The fall was considered to have occurred. And if the falling action is detected, resetting the counter.
In the embodiment, the posture information of two continuous frames is combined for detecting the falling action, so that the coherent movement posture of the human body is effectively represented, and the defect of unclear movement trend in single-frame detection during falling detection is avoided. In the falling detection process, the transverse motion information and the longitudinal motion information are calculated by using the human body posture detection result, the longitudinal information and the transverse information are considered, so that the false detection rate of similar actions such as squatting, sitting, bending and the like is reduced, and the detection precision of the model is improved.
The third embodiment of the present invention relates to a fall detection method, which is substantially the same as the first embodiment, and two threads need to be initialized, one thread is used for reading frame-by-frame information of a surveillance video stream, performing adaptive frame skipping at intervals, screening frames with human frames, and initializing a queue, and the other thread is used for realizing 2D posture detection and fall discrimination.
In this embodiment, the steps of the thread for implementing 2D posture detection and fall discrimination in this embodiment are the same as those in fig. 2 in the first embodiment, and the thread steps include: and detecting the human body posture of the frames in the queue, and detecting the falling action according to the human body posture detection result.
The difference between this embodiment and the first embodiment is that the thread for reading frame-by-frame information of a surveillance video stream and performing adaptive frame skipping frame-by-frame screening for human body frames in this embodiment is different from the first embodiment, and as shown in fig. 4, the following steps are specifically described.
Step 401: frame images of a video stream are acquired from a monocular camera.
In one example, taking a fall detection device as an embedded device or computer communicatively connected to a monocular camera, the fall detection device obtains frame images of a video stream from the monocular camera, reading the frame images frame by frame.
Step 402: and compressing the frame images of the video stream according to the real image scale.
In one example, a compression ratio is preset, the frame image is compressed proportionally, and if the length l of the transverse side and the longitudinal side of the frame image acquired from the falling detection device is larger than the preset compression ratiowidth、lheightAccording to the side length of the frame image: the side length ratio of the compressed image is 2: 1, the length of the transverse and longitudinal sides after compression is lwidth/2、lheight/2。
In one example, if the length of the horizontal and vertical sides l of the frame picture obtained from the fall detection devicewidth、lheightRecord its long side as lmaxThe short side is recorded asminSet a fixed value
The long side lmaxIs set to the fixed value
The short edges in the frame picture are compressed proportionally, and the compression formula of the short edges is as follows
And step 403, acquiring a temporary representative frame from the compressed frame image to detect the human body frame.
In addition, if the temporary representative frame is the last frame of the frame image of the video stream, the frame is not acquired from the frame image for detecting the human body frame, unless an instruction is given to detect the frame image of the video again.
And step 405, acquiring the next frame of the temporary representative frame as the updated temporary representative frame, and continuing to detect the human body frame.
And 405, adding the temporary representative frames into a queue, acquiring frames which are distant from the temporary representative frames by preset interval frame numbers, taking the frames as updated temporary representative frames, and continuously detecting the human body frame. It should be noted that, if the number of frames in the interval between the temporary representative frame and the last frame of the video stream is less than the preset interval frame number, the next frame of the temporary representative frame is detected until the last frame of the video stream is detected.
In one example, the fall detection device initiates two threads, one of which acquires a video stream from a monocular camera and reads frame images frame by frame, and compresses the frame images in real scale, for example, if the fall detection device acquires a frame image with a length l of the horizontal and vertical sideswidth、lheightRecord its long side as lmaxThe short side is recorded asminSet a fixed value
The long side lmaxIs set to the fixed value
The short edges in the frame picture are compressed proportionally, and the compression formula of the short edges is as follows
Selecting a first frame from the compressed frame image for human body framingIf the human body frame is detected, adding the temporary representative frame into the queue Q, acquiring a frame which is far away from the temporary representative preset interval frame number gap, taking the frame as an updated temporary representative frame, and continuously detecting the human body frame; if the human body frame is not detected, acquiring the next frame of the temporary representative frame as an updated temporary representative frame, and continuing to detect the human body frame; and the other thread is used for carrying out 2D human body posture detection on the frames in the queue Q and detecting the falling action according to the human body posture detection result.
In this embodiment, an embedded device or a computer in communication connection with a monocular camera acquires a frame image from the monocular camera; proportionally reducing the frame image, and detecting a human body frame of the reduced frame image; judging whether the temporary representative frame has a human body frame; if the human body frame exists, acquiring the next frame of the temporary representative frame as an updated temporary representative frame, and continuing to detect the human body frame; and if no human body frame exists, adding the temporary representative frame into the queue, acquiring a frame which is away from the temporary representative frame by a preset interval frame number, taking the frame as an updated temporary representative frame, continuously detecting the human body frame, detecting the human body posture of the frame in the queue, and detecting the tumbling action according to the human body posture detection result. In the embodiment, the frame image information acquired by the monocular camera is used for constructing the fall detection specification, and the frame image is screened from the video stream of the monocular camera for fall detection, so that the cost of hardware equipment is reduced compared with a method for detecting the fall action by using expensive cameras such as a depth camera or a multi-view camera. Meanwhile, frame images are compressed in proportion, the real distance is mapped into the pixel distance by the image compression mode, and the defect that a monocular camera lacks depth information is overcome.
The steps of the above methods are divided for clarity, and the implementation may be combined into one step or split some steps, and the steps are divided into multiple steps, so long as the same logical relationship is included, which are all within the protection scope of the present patent; it is within the scope of the patent to add insignificant modifications to the algorithms or processes or to introduce insignificant design changes to the core design without changing the algorithms or processes.
A fourth embodiment of the invention relates to an electronic device comprising at least one processor 501; and a memory 502 communicatively coupled to the at least one processor 501; the memory 502 stores instructions executable by the at least one processor 501, and the instructions are executed by the at least one processor 501, so that the at least one processor 501 can execute the fall detection method in the above embodiments.
Where the memory and processor are connected by a bus, the bus may comprise any number of interconnected buses and bridges, the buses connecting together one or more of the various circuits of the processor and the memory. The bus may also connect various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor is transmitted over a wireless medium via an antenna, which further receives the data and transmits the data to the processor.
The processor is responsible for managing the bus and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And the memory may be used to store data used by the processor in performing operations.
A fifth embodiment of the present invention relates to a computer-readable storage medium storing a computer program. The computer program realizes the above-described method embodiments when executed by a processor.
That is, as can be understood by those skilled in the art, all or part of the steps in the method for implementing the embodiments described above may be implemented by a program instructing related hardware, where the program is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific embodiments for practicing the invention, and that various changes in form and details may be made therein without departing from the spirit and scope of the invention in practice.
Claims (10)
1. A fall detection method, comprising:
detecting a human body frame of the temporary representative frame, wherein the temporary representative frame is derived from a frame image of the video stream;
if the human body frame is detected, adding the temporary representative frame into a queue, acquiring a frame which is away from the temporary representative frame by a preset interval frame number, and taking the frame as an updated temporary representative frame to detect the human body frame;
if the human body frame is not detected, acquiring the next frame of the temporary representative frame as an updated temporary representative frame to detect the human body frame;
detecting the human body posture of the frames in the queue;
and detecting the falling action according to the result of the human body posture detection.
2. The fall detection method according to claim 1, characterized in that said detection of the fall according to the result of said human posture detection comprises:
and detecting the tumbling action according to the human body posture detection results of two continuous frames in the queue.
3. The fall detection method according to claim 2, characterized in that said detection of the fall according to the result of said detection of the posture of the human body comprises:
calculating motion information according to the human body posture detection results of two continuous frames in the queue, wherein the motion information comprises transverse motion information and longitudinal motion information;
and detecting the falling action according to the motion information.
4. The fall detection method according to claim 3, characterized in that said detection of a fall according to said movement information comprises:
taking the frame meeting the preset condition in the queue as a falling frame, wherein the preset condition comprises the following steps: if the transverse motion information is in a preset transverse motion parameter range, and the longitudinal motion information is in a preset longitudinal motion parameter range;
and detecting a falling action according to the falling frame and the frame adjacent to the falling frame in the queue.
5. The fall detection method according to claim 4, characterized in that said detection of a fall from said fall frame and the frame of said queue adjacent to said fall frame comprises:
and if the adjacent frames meet the preset condition, and the frame distance between the falling frames and the frame where the falling action is detected last time exceeds a preset frame number, judging that the falling action is detected.
6. The fall detection method according to claim 4, characterized in that said detection of a fall from said fall frame and the frame of said queue adjacent to said fall frame comprises:
if the adjacent frames do not meet the preset condition, judging whether the longitudinal distance of the adjacent frames is within a preset range;
and if the current time is within the preset range, judging that the falling action is detected.
7. The fall detection method according to any one of claims 1 to 6, further comprising, before the detecting the body frame for the provisional representative frame:
and acquiring a frame image of the video stream from the monocular camera.
8. The fall detection method according to claim 7, further comprising, after said acquiring of the frame image of the video stream from the monocular camera:
and compressing the frame image according to the real image proportion.
9. An electronic device, comprising:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the fall detection method of any one of claims 1 to 8.
10. A computer-readable storage medium, storing a computer program, characterized in that the computer program, when being executed by a processor, implements the fall detection method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010737111.6A CN111898518A (en) | 2020-07-28 | 2020-07-28 | Tumble detection method, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010737111.6A CN111898518A (en) | 2020-07-28 | 2020-07-28 | Tumble detection method, electronic device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111898518A true CN111898518A (en) | 2020-11-06 |
Family
ID=73190160
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010737111.6A Pending CN111898518A (en) | 2020-07-28 | 2020-07-28 | Tumble detection method, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111898518A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113408390A (en) * | 2021-06-11 | 2021-09-17 | 广东工业大学 | Human behavior real-time identification method, system, device and storage medium |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160217326A1 (en) * | 2013-07-03 | 2016-07-28 | Nec Corporation | Fall detection device, fall detection method, fall detection camera and computer program |
| CN108960056A (en) * | 2018-05-30 | 2018-12-07 | 西南交通大学 | A kind of fall detection method based on posture analysis and Support Vector data description |
| CN110287923A (en) * | 2019-06-29 | 2019-09-27 | 腾讯科技（深圳）有限公司 | Human body attitude acquisition methods, device, computer equipment and storage medium |
| CN110399842A (en) * | 2019-07-26 | 2019-11-01 | 北京奇艺世纪科技有限公司 | Method for processing video frequency, device, electronic equipment and computer readable storage medium |
| CN110458061A (en) * | 2019-07-30 | 2019-11-15 | 四川工商学院 | A kind of method and company robot of identification Falls in Old People |
| CN110633643A (en) * | 2019-08-15 | 2019-12-31 | 青岛文达通科技股份有限公司 | Abnormal behavior detection method and system for smart community |
| CN110765860A (en) * | 2019-09-16 | 2020-02-07 | 平安科技（深圳）有限公司 | Tumble determination method, tumble determination device, computer apparatus, and storage medium |
| CN110991287A (en) * | 2019-11-23 | 2020-04-10 | 深圳市恩钛控股有限公司 | Real-time video stream face detection tracking method and detection tracking system |
| CN111178261A (en) * | 2019-12-30 | 2020-05-19 | 武汉恩智电子科技有限公司 | Face detection acceleration method based on video coding technology |
| CN111209774A (en) * | 2018-11-21 | 2020-05-29 | 杭州海康威视数字技术股份有限公司 | Target behavior recognition and display method, device, equipment and readable medium |
| CN111294524A (en) * | 2020-02-24 | 2020-06-16 | 中移（杭州）信息技术有限公司 | Video editing method and device, electronic equipment and storage medium |
-
2020
- 2020-07-28 CN CN202010737111.6A patent/CN111898518A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160217326A1 (en) * | 2013-07-03 | 2016-07-28 | Nec Corporation | Fall detection device, fall detection method, fall detection camera and computer program |
| CN108960056A (en) * | 2018-05-30 | 2018-12-07 | 西南交通大学 | A kind of fall detection method based on posture analysis and Support Vector data description |
| CN111209774A (en) * | 2018-11-21 | 2020-05-29 | 杭州海康威视数字技术股份有限公司 | Target behavior recognition and display method, device, equipment and readable medium |
| CN110287923A (en) * | 2019-06-29 | 2019-09-27 | 腾讯科技（深圳）有限公司 | Human body attitude acquisition methods, device, computer equipment and storage medium |
| CN110399842A (en) * | 2019-07-26 | 2019-11-01 | 北京奇艺世纪科技有限公司 | Method for processing video frequency, device, electronic equipment and computer readable storage medium |
| CN110458061A (en) * | 2019-07-30 | 2019-11-15 | 四川工商学院 | A kind of method and company robot of identification Falls in Old People |
| CN110633643A (en) * | 2019-08-15 | 2019-12-31 | 青岛文达通科技股份有限公司 | Abnormal behavior detection method and system for smart community |
| CN110765860A (en) * | 2019-09-16 | 2020-02-07 | 平安科技（深圳）有限公司 | Tumble determination method, tumble determination device, computer apparatus, and storage medium |
| CN110991287A (en) * | 2019-11-23 | 2020-04-10 | 深圳市恩钛控股有限公司 | Real-time video stream face detection tracking method and detection tracking system |
| CN111178261A (en) * | 2019-12-30 | 2020-05-19 | 武汉恩智电子科技有限公司 | Face detection acceleration method based on video coding technology |
| CN111294524A (en) * | 2020-02-24 | 2020-06-16 | 中移（杭州）信息技术有限公司 | Video editing method and device, electronic equipment and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113408390A (en) * | 2021-06-11 | 2021-09-17 | 广东工业大学 | Human behavior real-time identification method, system, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108038422B (en) | Camera device, face recognition method and computer-readable storage medium | |
| CN103632352B (en) | Method for time domain noise reduction of noise image and related device | |
| CN106296578B (en) | Image processing method and device | |
| KR102277048B1 (en) | Preview photo blurring method and device and storage medium | |
| US8041075B2 (en) | Identifying spurious regions in a video frame | |
| JP4653235B2 (en) | Composition of panoramic images using frame selection | |
| US20130343604A1 (en) | Video processing apparatus and video processing method | |
| US20100045799A1 (en) | Classifying an Object in a Video Frame | |
| CN110782433B (en) | Dynamic information violent parabolic detection method and device based on time sequence and storage medium | |
| CN111163259A (en) | Image capturing method, monitoring camera and monitoring system | |
| JP6016332B2 (en) | Image processing apparatus and image processing method | |
| EP3732875A1 (en) | Method for outputting a signal from an event-based sensor, and event-based sensor using such method | |
| CN110472612A (en) | Human bodys' response method and electronic equipment | |
| US20120134596A1 (en) | Image processing device, image processing method, integrated circuit, and program | |
| CN110310301B (en) | Method and device for detecting target object | |
| CN111105434A (en) | Motion trajectory synthesis method and electronic equipment | |
| KR20190041929A (en) | Image processing apparatus, image processing method, and non-transitory computer-readable storage medium | |
| CN111241928A (en) | Face recognition base optimization method, system, equipment and readable storage medium | |
| CN110910449B (en) | Method and system for identifying three-dimensional position of object | |
| CN111898518A (en) | Tumble detection method, electronic device and storage medium | |
| CN111639582A (en) | Living body detection method and apparatus | |
| CN109754419B (en) | Position estimation system | |
| CN112446254A (en) | Face tracking method and related device | |
| CN108282674B (en) | Video transmission method, terminal and system | |
| JP7243372B2 (en) | Object tracking device and object tracking method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |