TW202145957A - Motion detection method and device, electronic equipment and computer readable storage medium - Google Patents

Abstract

The invention relates to a motion detection method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: photographing the motion of a detected person, forming a picture frame sequence, and enabling the detected person to wear a calibration assembly; detecting the calibration component in the picture frame sequence to obtain position information of the calibration component; tracking the calibration component in the sequence of picture frames according to a tracking algorithm; and measuring the moving distance of the calibration assembly. According to the motion detection method and device, the electronic equipment and the computer readable storage medium, more accurate and faster target detection, tracking and measurement can be realized.

Description

Motion detection method and device, electronic device, and computer-readable recording medium with stored program

The invention belongs to the field of detection equipment, and in particular relates to a motion detection method and device, an electronic device and a computer-readable recording medium with stored programs.

Ranging exists in all aspects of human life and plays a very important role in the progress and development of human beings, and the social development of human beings promotes the development and improvement of ranging technology. With the development of human society, ranging from the most primitive estimation to the generation of measuring tools to the birth of modern high-tech measuring instruments, the theory of ranging technology has tended to be complete. There are many methods of measuring distance, which can be divided into two types from the level of contact: contact measurement and non-contact measurement. Non-contact measurement does not need to be in contact with the surface to be measured, and the final measurement distance is generally obtained through optical, electrical, imaging and other technologies. Contact measurement has high ranging accuracy and good stability, but it is not widely used due to factors such as volume, quality, installation conditions, structure, and inconvenient operation; although non-contact measurement is less accurate and stable than contact measurement However, it has the advantages of high degree of automation, fast measurement speed, rich information, and large dynamic range, and has gradually attracted people's attention. Especially with the rapid development of technology, non-contact measurement in terms of measurement accuracy and stability has been Approaching contact measurement. A common method used in non-contact measurement is image measurement. Image measurement is a measurement method that acquires an image through an image acquisition device, and then uses image processing technology to correlate the image to obtain the final ranging result. This method has no special requirements for measuring tools and objects to be measured, and is more suitable for applications where traditional contact measurement cannot be implemented.

Ranging can be applied to all aspects of life, especially in human motion scenarios, it is often necessary to detect, track and measure body movements. Detection, tracking and measurement of limb movement scenes, such as sports scoring (gymnastics, etc.), fitness (such as fast pedaling in place, detection of its frequency inflection point), rehabilitation, measurement of body function, dancing, playing the piano (such as violin) ), drumming, etc.

Although there are some motion detection systems and methods in the prior art, the accuracy and speed of these motion detection systems and methods need to be further improved.

The present invention provides a motion detection method and device, an electronic device, and a computer-readable recording medium with stored programs, which can further improve the accuracy and speed of motion detection, and the provided method and device can be applied to various motion detection scenarios.

According to a first aspect of the present invention, a motion detection method is provided, comprising:

Photograph the motion of the detected person to form a picture frame sequence, the detected person wears a calibration element;

Detecting the calibration element in the picture frame sequence to obtain position information of the calibration element;

tracking the calibration element in the sequence of picture frames according to a tracking algorithm; and

Measure the moving distance of the calibration element.

According to a second aspect of the present invention, a motion detection device is provided, comprising:

a photographing module for photographing the movement of the subject to form a picture frame sequence, the subject wearing a calibration element;

a detection module for detecting the calibration element in the picture frame sequence to obtain position information of the calibration element;

a tracking module for tracking the calibration element in the sequence of picture frames according to a tracking algorithm; and

A measuring module is used for measuring the moving distance of the calibration element.

According to the third party aspect of the present application, an electronic device is provided, comprising: processor;

The memory stores the computer program. When the computer program is executed by the processor, the processor executes the motion detection method described in the first aspect.

According to a fourth aspect of the present application, there is provided a computer-readable recording medium storing a program, on which computer-readable instructions are stored, and when the instructions are executed by a processor, the processor executes the first aspect. The described motion detection method.

According to the motion detection method and device, the electronic device and the computer-readable recording medium with the stored program according to the present invention, the position of the marking element can be roughly identified by detecting the color of the target and the moving object through the moving target detection process. After dilation erosion and connected domain screening processing, it can also remove noise and interference in the picture, and then through edge detection and corresponding transformation of the extracted edge, the marker element can be accurately identified and the location information of the marker element can be obtained. . Finally, tracking and distance measurements are performed after obtaining the location information of the markers. The motion detection device of the present invention can realize more precise and faster target detection, tracking and measurement.

In the following, only certain exemplary embodiments are briefly described. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature and not restrictive.

In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "top", "bottom", "front", " Rear", "Left", "Right", "Straight", "Level", "Top", "Bottom", "Inside", "Outside", "Clockwise", "Counterclockwise" etc. The positional relationship is based on the orientation or positional relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, Therefore, it should not be construed as a limitation of the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be understood as indicating or implying relative importance or implying the number of indicated technical features. Thus, features defined as "first", "second" may expressly or implicitly include one or more of said features. In the description of the present invention, "plurality" means two or more, unless otherwise expressly and specifically defined.

In the description of the present invention, it should be noted that, unless otherwise expressly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it may be a fixed connection or a detachable connection Connection, or integral connection: it can be a mechanical connection, an electrical connection or can communicate with each other; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal communication of two elements or the interaction of two elements relation. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

In the present invention, unless otherwise expressly specified and limited, a first feature "on" or "under" a second feature may include the first and second features in direct contact, or may include the first and second features Not directly but through additional features between them. Also, the first feature being "above", "over" and "over" the second feature includes that the first feature is directly above and obliquely above the second feature, or simply means that the first feature level is higher than the second feature. The first feature is "below", "below" and "below" the second feature includes the first feature is directly above and diagonally above the second feature, or simply means that the first feature level is less than the second feature.

The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. In order to simplify the disclosure of the present invention, the components and arrangements of specific examples are described below. Of course, they are only examples and are not intended to limit the invention. Furthermore, the present disclosure may repeat reference numerals and/or reference letters in different instances, such repetition is for the purpose of simplicity and clarity and does not in itself indicate a relationship between the various embodiments and/or arrangements discussed. In addition, the present disclosure provides examples of various specific processes and materials, but one of ordinary skill in the art will recognize the application of other processes and/or the use of other materials.

The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, but not to limit the present invention.

As shown in FIG. 1, an embodiment of the present invention provides a motion detection assistance system. The motion detection assistance system includes a calibration element 100 and a detection terminal support 400 , and the motion detection assistance system cooperates with the detection terminal 200 to detect parameters in the process of motion. The detection terminal 200 may be a smart device such as a mobile phone and an iPad. According to an optional technical solution of the present disclosure, the detection terminal 200 includes a camera, a processing module and a communication module, and can be connected to the server 300 through a data network for data exchange.

As shown in FIG. 2 , the calibration element 100 for motion detection includes a fixing part 101 and a marking part 102 . When exercising, the fixing component 101 is used to fix the calibration element 100 on the wrist, leg, waist and the like of the athlete. The marking part 102 is arranged on the fixing part 101, and is formed as a figure with predetermined size parameters as an optical beacon, which is convenient for identification in the video image of the marking element 100, for example, the marking part 102 can be a kind of logo. The marking member 102 includes a self-luminous structure and/or a reflective structure.

The graphics and the size of the graphics formed by the marking component 102 are pre-stored in the detection terminal, so that the detection terminal can perform subsequent processing. In this embodiment, the figure formed by the marking part 102 is a circle, the diameter of the circle is determined, and the diameter of the circle is pre-stored in the detection terminal. In another solution, the shape of the marking component 102 may be a rectangle or a square, and the side lengths of the rectangle or square are pre-stored in the detection terminal. The graphics of the marking component 102 may also be in other specific shapes, and only need to have a certain size.

According to an optional technical solution of the present invention, the number of the marking parts 102 is multiple, and they are arranged on the fixing part 101 in sequence, so as to facilitate the identification of the spatial posture of the calibration element 100 . The plurality of marking components 102 can also be set to different colors to improve the accuracy of spatial gesture recognition.

The calibration element 100 can be a wristband, and the wristband is worn on the wrist of an athlete. The calibration element 100 can also be a device that can be fixed relative to the arm, such as a watch or an arm guard. For example, when performing cardiopulmonary resuscitation, the calibration element 100 is preferably used to be worn on the wrist, because in this case the error between the movement distance of the bracelet and the movement distance of the palm is minimal.

In the system according to FIG. 1, the detection terminal 200 photographs the process of movement. There are many factors that affect the accuracy of ranging. Among them, the influence of hardware factors can be reduced by selecting high-quality hardware such as high-resolution CCD (Charge Coupled Device) cameras and high-sampling frequency image acquisition cards. Limitations of environmental factors. Because the styles of the detection terminals (eg, mobile phones) are different, the method of improving the ranging accuracy of the system through software algorithms is relatively the most effective way. The detection terminal 200 obtains video input during the process of photographing the movement. In order to perform target detection, target tracking and distance measurement on the calibration element 100, it is first necessary to convert the video input into a single-frame picture, so that the video input becomes a sequence of picture frames. In an optional embodiment, in order to extract the target in a simple and efficient way, convert the RGB (Red Green Blue) color space to the HSV (Hue, Saturation, Value, Hue, Saturation, Lightness) color space , which is convenient for color screening and extraction. Compared with the RGB color space used by normal pictures, the description method of the HSV color space is closer to the recognition method of the human eye, and can better describe the color and brightness. In addition, due to the complexity of the target background, after the color is rich, there are multiple targets in the segmentation, resulting in the inability to locate the image and the corner detection error. The H of the HSV space model can be represented by a circle.

According to one aspect of the present invention, a motion detection method is provided. As shown in Figure 3, the method includes the following steps:

In step S31, the motion of the detected person is photographed to form a picture frame sequence, and the detected person wears the calibration element.

The detected person wears a calibration element, which can be a wristband, a foot ring, or the like. After the detected person wears the calibration element, the detection terminal can photograph the motion of the detected person to form a video input, and then convert the video input into a picture frame sequence to form the picture frame sequence.

In step S32, the calibration element in the picture frame sequence is detected, and the position information of the calibration element is obtained.

After converting the video input into the picture frame sequence, it is first necessary to identify the position of the calibration component in the picture. After obtaining the position information of the calibration position, it is convenient for subsequent tracking of the calibration component and the tracking result according to the tracking result. A measure of the distance traveled by the calibration element. Step S32 will be described in more detail in FIGS. 4-6.

Step S33, tracking the calibration element in the picture frame sequence according to a tracking algorithm.

In order to ensure the speed of ranging, it is necessary to perform target detection and tracking on the calibration element. After the position information of the calibration element is obtained through step S32, since the calibration element is in a state of movement, the tracking of the pair of calibration elements needs to be maintained. Through the tracking algorithm, it is ensured that a good tracking effect is ensured when the target only has weak deformation and moves rapidly, and the trajectory of the movement of the calibration element can be extracted.

In a preferred embodiment, the tracking algorithm may include a KCF algorithm (Kernel Correlation Filter, kernel correlation filter algorithm). The KCF algorithm has the following advantages: simple implementation, good effect and fast speed. The problem is solved by generating a large number of samples through the displacement of the loop matrix, and by the derivation of the discrete Fourier transform, the calculation speed is extremely fast in the frequency domain. In the case of lack of samples, a simple method is used to detect and connect kernel operations to track, which ensures the generalization ability of tracking.

Step S34, measuring the moving distance of the calibration element.

First, the detection terminal 200 knows the predetermined size of the marking part 102 on the calibration element 100 in advance, and after capturing the marking part 102 , knows the pixel value occupied by the marking part 102 . Then, in the process of tracking the calibration element, the position information of the extreme point during the movement of the calibration element is extracted, and the position information of the extreme point includes the position information of the uppermost position and the position information of the lowermost position of the up and down movement , or include the position information of the leftmost position and the position information of the rightmost position of the left and right movement. Of course, the position information of the extreme point may also include information of other extreme positions of the movement. By taking the position information of the extreme point during the movement of the calibration element, the pixel value occupied during the movement of the calibration element is obtained, and then according to the size of the marking part 102 and the pixel value occupied by the marking part 102, the pixel value occupied by the marking part 102 is obtained. The movement distance of the calibration element. Step S32 will be described in more detail in FIGS. 7-8.

FIG. 4 is a flowchart of step S32 shown in FIG. 3 according to an embodiment of the present invention. As shown in Figure 4, step S32 includes the following sub-steps:

In sub-step S41, moving object detection is performed on the picture frame sequence.

Moving object detection is used to extract the information of the target element in the picture frame. There are many kinds of moving target detection algorithms, including background subtraction, optical flow and frame difference. In a preferred embodiment, a frame difference method is used to implement moving object detection. Among them, the frame difference method is used to implement moving target detection as shown in Figures 5 to 6, which will be described in detail below.

In an optional embodiment, in order to reduce the amount of calculation, step S32 may further include the following sub-steps:

In sub-step S42, binarization processing is performed on the result of the moving object detection.

Through binarization, redundant information is filtered out, and color information is processed into black and white, which can greatly reduce the amount of calculation.

In another optional embodiment, in order to reduce and filter out small-area noise interference except the calibration element, step S32 may further include the following sub-steps:

Sub-step S43, performing dilation erosion processing on the binarized result; and

In sub-step S44, the connected domain screening process is performed on the result after expansion and erosion.

Among them, an appropriate size of dilation erosion can be set, and an area of connected domain screening can also be set, for example, a connected domain less than 300 pixels can be filtered out. After completing the screening of moving object detection, the object of interest can be roughly obtained in the picture frame, including the marker element, but still a small part of the interference cannot be excluded. Therefore, the specific location of the marker element will be identified based on the edge features of the marker element. Thus, step S32 includes:

In sub-step S45, edge detection processing is performed on the result of the moving object detection. During the edge detection processing, the extracted edge includes the edge of the calibration element.

In a specific embodiment, the algorithm for edge detection includes a canny algorithm.

After the edge detection process is performed, the extracted edge includes the edge of the calibration element, and may also include the edge of other objects that are not the calibration element. Therefore, in order to more accurately determine the position of the calibration element, step S32 includes:

Sub-step S46, according to the preset shape of the calibration element, correspondingly transform the extracted edge, so as to obtain the position information of the calibration element.

For example, if the calibration element is a wristband, then its preset shape is a rectangle, and each straight line segment of the rectangle can be extracted through a corresponding transformation algorithm, such as Hough transform, so as to determine that the extracted edge is the edge of the wristband. More specifically, set a threshold for the number of straight line segments, such as 2 or 3. When the number of straight line segments extracted by Hough transform is greater than or equal to the threshold, it can be confirmed that the edge is a rectangular edge, that is, the wristband is detected. edge.

For another example, if the preset shape of the editing element is a circle or an ellipse, a corresponding transformation algorithm, such as another Hough transformation, can be used to extract a circle or an ellipse, so as to determine that the extracted edge is the edge of the marking element. or edge components.

When extracting the edge or edge component of the calibration component, set the corresponding conversion algorithm according to the characteristics of the preset shape of the Hough transform calibration component, and extract the edge or edge component of the Hough transform calibration component. What kind of conversion algorithm is not limited in the present invention, as long as the conversion algorithm can extract and determine the edge of the calibration element.

After acquiring the edge of the component marked by the transformation algorithm, the position information of the component marked by the transformation algorithm can be known. For example, the transformation algorithm marks the component as a wristband, and the position of the wristband can be represented by the coordinates [X, Y, H, W]. In a specific embodiment, X and Y respectively represent the horizontal direction of the upper left corner of the wristband rectangle. Coordinates and vertical coordinates, while H and W represent the height and width of the bracelet rectangle, respectively.

FIG. 5 is a flowchart of moving object detection according to an embodiment of the present invention. For the picture frame sequence of video input, two adjacent picture frames (preferably the first two adjacent picture frames of the picture frame sequence) are selected for moving object detection. The moving target detection process includes:

Step S51 , acquiring a first picture frame and a second picture frame adjacent to each other in the picture frame sequence.

Step S52, extract the target color from the first picture frame and the second picture frame respectively.

The target color is the color of the marking element, and the target color is extracted to facilitate the detection of the marking element. The target color is preferably a color different from human skin color, clothing, and video background, such as red or green, etc.

In step S53, the target color extracted from the first picture frame and the second picture frame is differentiated to obtain a moving target detection result.

FIG. 6 is a flowchart of moving object detection according to another embodiment of the present invention. The difference from Figure 5 is that, for the picture frame sequence input by the video, three adjacent picture frames (these three picture frames are preferably the first three adjacent picture frames of the picture frame sequence) are selected for motion. Target Detection. The moving target detection process includes:

Step S61, acquiring a first picture frame, a second picture frame, and a third picture frame adjacent to each other in the picture frame sequence.

Step S62, extract the target color from the first picture frame, the second picture frame and the third picture frame respectively.

The target color is the color of the marking element, and the target color is extracted to facilitate the detection of the marking element. The target color is preferably a color different from human skin color, clothing, and video background, such as red or green, etc.

Step S63: Differentiate the target color extracted from the first picture frame and the second picture frame to obtain a first difference result.

In step S64, the target color extracted from the second picture frame and the third picture frame is differentiated to obtain a second difference result.

The order in which steps S63 and S64 are performed is not limited, and step S63 may be performed before step S64, may be performed after step S64, or may be performed simultaneously with step S64.

Step S65, taking the intersection of the first difference result and the second difference result to obtain a moving target detection result.

The moving object detection process shown in Figure 6 can effectively reduce the noise existing in the image compared to Figure 5. Under the teaching of the embodiments shown in FIGS. 5 to 6, those skilled in the art can think that, for the picture frame sequence inputted by the video, more adjacent picture frames can also be selected for moving object detection, all of which are fall within the scope of this application.

In addition, in a preferred embodiment, to ensure the reliability of the tracking, after the tracking picture frame reaches the set number, it is necessary to perform the detection of the calibration element again to obtain the position information of the calibration element. Therefore, the motion detection method of the present invention may further comprise: acquiring the number of frames tracked by the picture frame sequence; and in response to the number of the tracking frames reaching a preset value, performing calibration in the pair of video input picture frame sequences again The component is detected, and the position information of the calibration component is acquired, that is, step S32 or step S41 is performed again.

Figures 7-8 describe the measurement of the distance traveled by the calibration element. As shown in FIG. 7, it is assumed that the shape of the figure of the marking member 102 is a square as an example. The side length L of the marking member 102 is 2 cm. The detection terminal 200 determines from the video image of the calibration element 100 that the first number of pixels corresponding to the sides of the square is 100, and the actual distance corresponding to the pixels in the video image of the calibration element 100 is 0.02 cm.

Take exercise as an example to implement cardiopulmonary resuscitation. During cardiopulmonary resuscitation, the rescuer presses, and when the calibration element 100 moves, the detection terminal 200 determines the upper limit image frame and the lower limit image frame from the video image of the calibration element. The second pixel number corresponding to the moving distance of the calibration element is determined according to the upper limit image frame and the lower limit image frame.

As shown in FIG. 8 , when the calibration element 100 moves, the detection terminal 200 can identify the upper limit image frame and the lower limit image of a pressing operation from each image frame of the video image of the calibration element 100 frame. The number of pixels between the upper limit position and the lower limit position of the marking component 102 in the upper limit image frame and the lower limit image frame is taken as the second pixel number corresponding to the moving distance of the calibration element 100 .

When the first pressing is performed, the image frame in the image when the calibration element 100 is not moved is regarded as the upper limit image frame of the first pressing; with the downward movement of the calibration element 100, the marking part 102 in the video image corresponds to When the pixel corresponding to the marking part 102 no longer moves downward, the detection terminal 200 takes an image frame at this time as the lower limit image frame of the first pressing. With the upward movement of the calibration element 100, the second sub-module of the processing module recognizes that the pixel corresponding to the marking component 102 is no longer moved upward, and takes an image frame at this time as the upper limit of the second pressing image frame; then the calibration element 100 moves down, and the detection terminal 200 recognizes that the pixel corresponding to the marking part 102 no longer moves down, and takes an image frame at this time as the lower limit image frame for the second pressing. In this way, the detection terminal 200 can identify the upper limit image frame and the lower limit image frame corresponding to each pressing. Through the upper limit image frame and the lower limit image frame, the second number of pixels corresponding to the moving distance of the calibration element 100 in each pressing is determined. The detection terminal 200 determines the moving distance of the calibration element according to the actual distance corresponding to the pixel in the video image of the calibration element and the second number of pixels.

For example, in one press, the detection terminal 200 determines that the second pixel number corresponding to the moving distance S of the calibration element 100 is 200, and the actual distance corresponding to the pixels in the video image of the calibration element is 0.02 cm, and the calibration element is calculated to obtain The moving distance S is 4 cm. Let 4cm be the compression depth for this time.

In an actual scene, the normal line of the marking part 102 on the calibration element may not point to the detection terminal 200 , that is, the detection terminal 200 shoots the calibration element from an oblique direction. At this time, the processing module can determine which direction the detection terminal 200 shoots the calibration element 100 according to the image deformation amount of the marking component; according to the conversion of the cosine relationship of the corresponding angle, the detection terminal 200 determines the video image of the calibration element The actual distance corresponding to the pixels in the image. This design mainly involves that when a person wears the calibration element 100 for rescue, the plane where the marking part 102 of the calibration element 100 is located is not necessarily perpendicular to the direction of the connection between the calibration element 100 and the detection terminal 200 . When it is not vertical, the marking part 102 detected by the detection terminal 200 will form a certain distortion. For example, when the marking part is originally circular, when the plane where the marking part of the marking element is located is not perpendicular to the direction of the connection between the marking part and the detection terminal, the detected marking part takes on an oval shape. At this time, although the detection terminal knows the size of the marking component, for example, the diameter of the circle is 2 cm, the long axis of the ellipse in the detected image is 2 cm, and the short axis is less than 2 cm. At this time, the detection terminal 200 needs to identify the actual size of the marking component according to the actual image captured by the detection terminal 200, for example, the long axis of the ellipse represents the diameter of the circle. In the case of a circle becoming an ellipse, it is also relatively simple, while in the case of eg using rectangles, polygons, etc., the distortion becomes complicated. For example, a rectangle contains long and short sides. When it is not vertical, it may be a parallelogram. At this time, it is necessary to determine the angle between the plane where the rectangle is located and the connection line (between the calibration element and the detection terminal) according to the angle between the two adjacent sides of the parallelogram, and then convert the captured parallelism according to the cosine relationship. The length of the real object represented by each side of the quadrilateral, and then the actual distance represented by each pixel in the pressing direction is calculated.

When the plane where the marking part 102 of the calibration element 100 is located is not perpendicular to the direction of the connection between the calibration element 100 and the detection terminal 200 , in addition to the above-mentioned detection terminal 200 correcting the detected marking part 102 according to the specific shape of the marking part 102 The actual reference size of the marking component 102 can be obtained by correcting the reference coordinate system in which the calibration element 100 and the detection terminal 200 are located to obtain the actual reference size of the marking component 102.

In addition, regarding the accuracy of the distance measured by the detection terminal 200, on the one hand, the higher the resolution of the camera used by the detection terminal 200 during the shooting motion, the smaller the loss of accuracy, that is, the higher the accuracy; on the other hand, according to the following The formula for the loss of precision:

和

分別表示測量時的邊緣選取造成的誤差和像素本身的誤差，

和

分別實際待測目標距離的誤差和參考距離的誤差，

和

分別代表該參考距離和該實際待測目標的距離，該參考距離表示運動過程中第一極限位置點的標定元件100的第一設定參考基準線至第二極限位置點的標定元件100的第二設定參考基準線的距離，而該實際待測目標的距離表示運動過程中第一極限位置點的標定元件100的第一設定參考基準線至第二極限位置點的標定元件100的該第一設定參考基準線的距離。例如，對於進行上下運動（例如心肺複運動）過程來說，參考距離表示運動過程最高點的標定元件100（例如，手環）的上邊沿至最低點的標定元件100的下邊沿的距離，而該實際待測目標的距離表示運動過程最高點的標定元件100（例如，手環）的上邊沿至最低點的標定元件100的上邊沿的距離。in,

and

respectively represent the error caused by the edge selection during measurement and the error of the pixel itself,

and

The error of the actual target distance to be measured and the error of the reference distance, respectively,

and

Represent the reference distance and the distance of the actual target to be measured, respectively, and the reference distance represents the first set reference reference line of the calibration element 100 at the first limit position point during the movement to the second limit position of the calibration element 100 at the second limit position point. The distance of the reference reference line is set, and the distance of the actual target to be measured represents the first setting of the reference reference line of the calibration element 100 at the first limit position point during the movement process to the first setting of the calibration element 100 at the second limit position point The distance from the reference baseline. For example, for a process of performing up and down movements (such as cardiopulmonary rehabilitation), the reference distance represents the distance from the upper edge of the calibration element 100 (eg, wristband) at the highest point of the exercise process to the lower edge of the calibration element 100 at the lowest point, while The distance of the actual target to be measured represents the distance from the upper edge of the calibration element 100 (eg, wristband) at the highest point in the movement process to the upper edge of the calibration element 100 at the lowest point.

It can be seen from the above formula that when the reference distance is larger, the accuracy loss E will gradually become smaller. It can be seen that the pixel value of the camera during shooting and the measurement of the actual distance will cause the accuracy error during the experiment. On the one hand, by using the high-definition camera, on the other hand, by increasing the reference distance, the error in the experiment can be effectively reduced.

Although the above embodiment describes a specific algorithm in conjunction with CPR detection, the solution is not limited to this scenario, but can also be applied to tracking and measurement of limbs and body movements, and further can be applied to fitness, rehabilitation, etc. Motion detection in various limb movement scenarios, such as: sports scoring (gymnastics, etc.), fitness (such as fast pedaling in place, detection of its frequency inflection point), rehabilitation, physical function measurement, dancing, playing the piano (such as violin) , drumming, etc.

This program can not only collect upper body exercise data during cardiopulmonary resuscitation, but also collect in-situ training data of the human body at ordinary times, such as calculating the frequency, frequency, and depth of rapid leg lifts and push-ups in situ. Put calibration devices on the parts of the body that the recovered person needs to exercise, such as arms, legs, and waistbands, such as wristbands, leg rings, etc., use the mobile phone camera to record and monitor the movement process, monitor the movement amplitude, frequency, trajectory, etc., and can give Amplitude measurement record (resolution 2mm, precision 5mm), frequency record, track drawing on video and other functions.

According to the motion detection method of the present invention, through the moving target detection process, the position of the marking element can be roughly identified by detecting the target color and moving object, and after the expansion erosion and the connected domain screening process, the noise in the picture can also be removed. Then, through edge detection and corresponding transformation of the extracted edge, the marking element can be accurately identified and the position information of the marking element can be obtained. Finally, tracking and distance measurements are performed after obtaining the location information of the markers. The motion detection method of the present invention can realize more accurate and faster target detection, tracking and measurement.

According to another aspect of the present invention, a motion detection apparatus is provided. As shown in Figure 9, the device includes the following modules:

A photographing module 91 is used for photographing the movement of the detected person to form a picture frame sequence, and the detected person wears a calibration element.

The detected person wears the calibration element, and the calibration element can be a wristband, a foot ring, or the like. After the detected person wears the calibration element, a detection terminal can photograph the motion of the detected person to form a video input, and then convert the video input into a picture frame sequence to form a picture frame sequence.

A detection module 92 is used for detecting the calibration element in the picture frame sequence to obtain position information of the calibration element.

After converting the video input into a picture frame sequence, it is first necessary to identify the position of the calibration component in the picture. After obtaining the position information of the calibration position, it is convenient for subsequent tracking of the calibration component and the calibration according to the tracking result. A measure of how far a component has moved. The detection module 92 will be described in more detail in Figures 10-12.

A tracking module 93 for tracking the calibration element in the picture frame sequence according to a tracking algorithm.

In order to ensure the speed of ranging, it is necessary to perform target detection and tracking on the calibration element. After the position information of the calibration element is obtained through the detection module 92, since the calibration element is in a moving state, it is necessary to keep track of the calibration element. Through the tracking algorithm, a good tracking effect is ensured when the target only has weak deformation and moves rapidly, and the trajectory of the movement of the calibration element can be extracted.

In a preferred embodiment, the tracking algorithm may include a KCF algorithm (Kernel Correlation Filter, kernel correlation filter algorithm). The KCF algorithm has the following advantages: simple implementation, good effect and fast speed. The problem is solved by generating a large number of samples through the displacement of the loop matrix, and by the derivation of the discrete Fourier transform, the calculation speed is extremely fast in the frequency domain. In the case of lack of samples, a simple method is used to detect and connect kernel operations to track, which ensures the generalization ability of tracking.

A measuring module 94 is used for measuring the moving distance of the calibration element.

First, the detection terminal 200 knows the predetermined size of the marking part 102 on the calibration element 100 in advance, and after capturing the marking part 102 , knows the pixel value occupied by the marking part 102 . Then, in the process of tracking the calibration element, the position information of the extreme point during the movement of the calibration element is extracted, and the position information of the extreme point includes the position information of the uppermost position and the position information of the lowermost position of the up and down movement , or include the position information of the leftmost position and the position information of the rightmost position of the left and right movement. Of course, the position information of the extreme point may also include the information of other extreme positions of the movement. By taking the position information of the extreme point during the movement of the calibration element, the pixel value occupied during the movement of the calibration element is obtained, and then the calibration is obtained according to the size of the marking part 102 and the pixel value occupied by the marking part 102 The movement distance of the element.

FIG. 10 is a schematic diagram of the detection module 92 shown in FIG. 9 according to an embodiment of the present invention. As shown in FIG. 10, the detection module 92 includes the following units:

A moving object detection unit 921, configured to perform moving object detection on the picture frame sequence.

The moving object detection is used to extract the information of the target element in the picture frame. There are many kinds of moving target detection algorithms, including background subtraction, optical flow and frame difference. In a preferred embodiment, a frame difference method is used to implement moving object detection. Among them, the frame difference method is used to implement moving target detection as shown in Figures 11 to 12, which will be described in detail below.

In an optional embodiment, in order to reduce the amount of calculation, the detection module 92 may further include the following units:

A binarization processing unit 922, configured to perform binarization processing on the result of the moving object detection.

Through binarization, redundant information is filtered out, and color information is processed into black and white, which can greatly reduce the amount of calculation.

In another optional embodiment, in order to filter out small-area noise interference except for the calibration element, the detection module 92 may further include the following units:

a dilation erosion processing unit 923, configured to perform dilation erosion processing on the binarized result; and

A connected domain filtering processing unit 924, configured to perform connected domain filtering processing on the result after dilation and erosion.

Among them, an appropriate size of dilation erosion can be set, and an area of connected domain screening can also be set, for example, a connected domain less than 300 pixels can be filtered out.

After completing the screening of moving object detection, the object of interest can be roughly obtained in the picture frame, including the marker element, but still a small part of the interference cannot be excluded. Therefore, the specific location of the marker element will be identified based on the edge features of the marker element. Thus, the detection module 92 includes:

An edge processing unit 925, configured to perform edge detection processing on the result of the moving object detection. During the edge detection processing, the extracted edge includes the edge of the calibration element.

In a specific embodiment, the algorithm for edge detection includes a canny algorithm. After the edge detection process is performed, the extracted edge includes the edge of the calibration element, and may also include the edge of other objects that are not the calibration element. Therefore, in order to more accurately determine the position of the calibration element, the detection module 92 includes:

An edge transformation unit 926, configured to perform corresponding transformation on the extracted edge according to the preset shape of the calibration element, so as to obtain the position information of the calibration element.

For example, if the calibration element is a wristband, then its preset shape is a rectangle, and each straight line segment of the rectangle can be extracted through a corresponding transformation algorithm, such as Hough transform, so as to determine that the extracted edge is the edge of the wristband. More specifically, set a threshold for the number of straight line segments, such as 2 or 3. When the number of straight line segments extracted by Hough transform is greater than or equal to the threshold, it can be confirmed that the edge is a rectangular edge, that is, the wristband is detected. edge.

For another example, if the preset shape of the marking element is a circle or an ellipse, a corresponding transformation algorithm, such as another Hough transformation, can be used to extract a circle or an ellipse, so as to determine that the extracted edge is the marking element the edge or part of the edge.

When extracting the edge or edge component of the calibration component, set the corresponding conversion algorithm according to the characteristics of the preset shape of the calibration component, and extract the edge or edge component of the calibration component. The conversion algorithm is not limited in the present invention, as long as the conversion algorithm can extract and determine the edge of the calibration element.

After the edge of the marking element is acquired, the position information of the marking element can be known. For example, the marking element is a wristband, and the position of the wristband can be represented by coordinates [X, Y, H, W]. In a specific embodiment, X and Y respectively represent the abscissa and the upper left corner of the wristband rectangle The vertical coordinate, while H and W represent the height and width of the bracelet rectangle, respectively.

FIG. 11 is a schematic diagram of a moving target detection module according to an embodiment of the present invention. For the picture frame sequence of video input, two adjacent picture frames (preferably the first two adjacent picture frames of the picture frame sequence) are selected for moving object detection. The moving target detection unit includes:

A first picture frame obtaining subunit 111 is configured to obtain an adjacent first picture frame and a second picture frame in the picture frame sequence.

A first color extraction subunit 112 for extracting target colors from the first picture frame and the second picture frame, respectively.

The target color is the color of the marking element, and the target color is extracted to facilitate the detection of the marking element. The target color is preferably a color different from human skin color, clothing, and video background, such as red or green, etc.

A first difference sub-unit 113, configured to perform a difference between the extracted target colors in the first picture frame and the second picture frame to obtain a result of moving object detection.

FIG. 12 is a schematic diagram of a moving object detection module according to an embodiment of the present invention. The difference from Figure 11 is that for the video input picture frame sequence, select three adjacent picture frames (these three picture frames are preferably the first three adjacent picture frames of the picture frame sequence) for motion. Target Detection. The moving target detection unit includes:

A second picture frame obtaining subunit 121, configured to obtain adjacent first picture frames, second picture frames and a third picture frame in the picture frame sequence.

A second color extraction subunit 122 for extracting the target color from the first picture frame, the second picture frame and the third picture frame, respectively.

The target color is the color of the marking element, and the target color is extracted to facilitate the detection of the marking element. The target color is preferably a color different from human skin color, clothing, and video background, such as red or green, etc.

A second difference sub-unit 123 is configured to perform a difference between the first picture frame and the target color extracted from the second picture frame to obtain a first difference result.

A third difference sub-unit 124 is configured to perform a difference between the second picture frame and the target color extracted from the third picture frame to obtain a second difference result.

An intersection subunit 125 is configured to take the intersection of the first difference result and the second difference result to obtain a moving target detection result.

The moving object detection process shown in Figure 12 can effectively reduce the noise existing in the image compared to Figure 11. Under the teachings of the embodiments shown in FIGS. 11 to 12, those skilled in the art can think that, for the video input picture frame sequence, more adjacent picture frames can also be selected for moving object detection, all of which are fall within the scope of this application.

In addition, in a preferred embodiment, to ensure the reliability of the tracking, after the tracking picture frame reaches the set number, it is necessary to perform the detection of the calibration element again to obtain the position information of the calibration element. Therefore, the motion detection apparatus of the present invention may further comprise: an acquisition module for acquiring the number of frames tracked in the picture frame sequence. In this way, after the number of tracking frames reaches the preset value, the detection module 92 is made to perform the detection of the calibration element in the video input picture frame sequence again, and obtain the position information of the calibration element.

According to the motion detection device of the present invention, through the moving target detection process, by detecting the object color and moving object, the position of the marking element can be roughly identified, and after the expansion erosion and the connected domain screening process, the noise in the picture can also be removed. Then, through edge detection and corresponding transformation of the extracted edge, the marking element can be accurately identified and the position information of the marking element can be obtained. Finally, tracking and distance measurements are performed after obtaining the location information of the markers. The motion detection device of the present invention can realize more precise and faster target detection, tracking and measurement.

According to another aspect of the present application, an electronic device is provided, which includes a processor and a memory, the memory stores the computer program, and when the computer program is executed by the processor, causes the processor to execute any of the above A motion detection method according to an embodiment.

According to another aspect of the present application, there is provided a computer-readable recording medium with a stored program, on which computer-readable instructions are stored, and when the instructions are executed by a processor, the processor can be made to execute any one of the above The motion detection method described in the embodiment. Specifically, the motion detection method includes: photographing the motion of the detected person to form a picture frame sequence, the detected person wearing a calibration element; detecting the calibration element in the picture frame sequence to obtain the position information of the calibration element ; tracking the calibration element in the sequence of picture frames according to a tracking algorithm; and determining the moving distance of the calibration element.

Wherein, the detection of the calibration element in the picture frame sequence and the acquisition of the position information of the calibration element include: performing moving target detection on the picture frame sequence; During the processing, the extracted edge includes the edge of the calibration element; and according to the preset shape of the calibration element, the extracted edge is correspondingly transformed, so as to obtain the position information of the calibration element.

Wherein, performing moving target detection on the picture frame sequence includes: acquiring adjacent first picture frames and second picture frames in the picture frame sequence; extracting targets from the first picture frame and the second picture frame respectively Color; the target color extracted from the first picture frame and the second picture frame is differentiated to obtain the result of moving target detection.

Wherein, performing moving target detection on the picture frame sequence includes: acquiring a first picture frame, a second picture frame and a third picture frame adjacent to the picture frame sequence; The target color is extracted from the second picture frame and the third picture frame respectively; the target color extracted from the first picture frame and the second picture frame is differentiated to obtain a first difference result; the second picture frame and the The target colors extracted from the three picture frames are differentiated to obtain a second difference result; the intersection of the first difference result and the second difference result is obtained to obtain a moving target detection result.

Wherein, the detecting the calibration element in the picture frame sequence of the video input, and acquiring the position information of the calibration element further includes: performing binarization processing on the result of the moving object detection.

Wherein, the detection of the calibration element in the picture frame sequence of the video input, and the acquisition of the position information of the calibration element further includes: performing dilation and erosion processing on the result after the binarization processing; and performing a connected domain on the result after dilation and erosion Filter processing.

Wherein, the tracking algorithm includes KCF algorithm, the edge detection process includes canny algorithm, and/or the moving object detection includes background subtraction method, optical flow method and frame difference method.

Further, the motion detection method also includes: obtaining and tracking the number of frames tracked in the picture frame sequence; and in response to the number of the tracked frames reaching a preset value, performing the calibration element in the video input picture frame sequence again. Detect to obtain the position information of the calibration element.

In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product including a computer program carried on a computer-readable medium, the computer program including code for performing the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via its communication component, and/or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above-described functions defined in the method of the present application are performed. It should be noted that the computer-readable medium of the present application may be a computer-readable signal medium or a computer-readable recording medium storing a program, or any combination of the above two. The computer-readable recording medium storing the program can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the above. More specific examples of computer-readable recording media with stored programs may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), only Read Memory (ROM), Erasable Programmable Read-Only Memory (EPROM or Flash Memory), Optical Fiber, Portable Compact Disc Read-Only Memory (CD-ROM), Optical Memory Devices, Magnetic memory device, or any suitable combination of the above. In this application, a computer-readable recording medium storing a program can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, however, a computer-readable signal medium may comprise a data signal in baseband or propagating as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take a variety of forms including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable recording medium storing a program, which can be sent, propagated, or transmitted for use by the instruction execution system, apparatus, or device or programs used in conjunction with it. Code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out the operations of the present application may be written in one or more programming languages, including object-oriented programming languages—such as Java, Smalltalk, C++, but also conventional procedural programs, or a combination thereof Design language such as "C" language or similar programming language. The code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or run on the server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network including a local area network (LAN) or wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect via the Internet).

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more functions for implementing the specified Executable instructions for logical functions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented using dedicated hardware-based hardware that performs the specified functions or operations. system, or can be implemented using a combination of dedicated hardware and computer instructions.

The units involved in the embodiments of the present application may be implemented in a software manner, and may also be implemented in a hardware manner. The described unit can also be set in the processor, for example, it can be described as: a processor includes a receiving unit, a searching unit, and a sending unit. Among them, the names of these units do not constitute a limitation of the unit itself under certain circumstances. For example, the receiving unit can also be described as "a unit that receives user requests for requesting blockchain account address information".

As another aspect, the present application also provides a computer-readable medium. The computer-readable medium may be included in the apparatus described in the above embodiments, or may exist alone without being assembled into the apparatus. The above-mentioned computer-readable medium carries one or more programs, which, when executed by the apparatus, cause the apparatus to perform the method as described above.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the scope of the present invention. within the scope of protection. Finally, it should be noted that the above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, the The technical solutions described in the foregoing embodiments may be modified, or some technical features thereof may be equivalently replaced. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

﹝this invention﹞ 91: Shooting module 92: Detection module 921: Moving target detection unit 922: Binarization processing unit 923: Expansion Erosion Processing Unit 924: Connected Domain Screening Processing Unit 925: Edge Processing Unit 926: Edge Transform Unit 93: Tracking Module 94: Measurement module 111: The first picture frame acquisition subunit 112: The first color extraction subunit 113: The first difference subunit 121: second picture frame acquisition subunit 122: Second color extraction subunit 123: Second difference subunit 124: The third difference subunit 125: Take intersection subunit 100: Calibration element 101: Fixed parts 102: Marking parts 200: Detect terminal 400: Detect terminal bracket S31～S34: Steps S41～S46: Steps S51～S53: Steps S61～S65: Steps

[Fig. 1] A schematic diagram of a motion detection assistance system according to an embodiment of the present invention. [Fig. 2] A schematic diagram of the calibration element in the embodiment of the present invention. [FIG. 3] A flowchart of a motion detection method according to an embodiment of the present invention. [FIG. 4] The flowchart of step S32 shown in FIG. 3. [FIG. 5] A flowchart of moving target detection according to an embodiment of the present invention. [Fig. 6] A flowchart of moving target detection according to another embodiment of the present invention. [Fig. 7] A schematic diagram of a square marking member according to an embodiment of the present invention. [Fig. 8] A schematic diagram of the up and down movement of the marking member according to an embodiment of the present invention. [Fig. 9] A schematic diagram of a motion detection device according to an embodiment of the present invention. [Fig. 10] A schematic diagram of the detection module shown in Fig. 9. [FIG. 11] A schematic diagram of a moving target detection module according to an embodiment of the present invention. [FIG. 12] A schematic diagram of a moving target detection module according to another embodiment of the present invention.

S31~S34: Steps

Claims (11)

A motion detection method comprising: Photograph the motion of the detected person to form a picture frame sequence, the detected person wears a calibration element; Detecting the calibration element in the picture frame sequence to obtain position information of the calibration element; tracking the calibration element in the sequence of picture frames according to a tracking algorithm; and Measure the moving distance of the calibration element. The motion detection method of claim 1, wherein the calibration element in the picture frame sequence is detected, and obtaining the position information of the calibration element includes performing moving object detection on the picture frame sequence; Edge detection processing, in the process of edge detection processing, the extracted edge includes the edge of the calibration element; and according to the preset shape of the calibration element, the extracted edge is correspondingly transformed, so as to obtain the position of the calibration element Information. The motion detection method of claim 2, wherein performing moving object detection on the picture frame sequence includes acquiring a first picture frame and a second picture frame adjacent to the picture frame sequence; The target color is respectively extracted from the second picture frame; the difference between the first picture frame and the target color extracted from the second picture frame is performed to obtain the result of moving target detection. The motion detection method of claim 2, wherein performing moving object detection on the sequence of picture frames comprises acquiring adjacent first, second and third picture frames in the sequence of picture frames; The target color is extracted from the first picture frame, the second picture frame and the third picture frame respectively; the target color extracted from the first picture frame and the second picture frame is differentiated to obtain a first difference result; the The second picture frame is differentiated from the target color extracted from the third picture frame to obtain a second difference result; the intersection of the first difference result and the second difference result is obtained to obtain a moving target detection result. The motion detection method of claim 2, wherein detecting a calibration element in a picture frame sequence of video input, and acquiring the position information of the calibration element further includes binarizing the result of the moving object detection. The motion detection method of claim 5, wherein detecting a calibration element in a picture frame sequence of video input, and obtaining the position information of the calibration element further includes performing dilation and erosion processing on the binarized result; The eroded results are processed by connected domain filtering. The motion detection method of claim 2, wherein the tracking algorithm includes a KCF algorithm, the edge detection process includes a canny algorithm, and/or the moving object detection includes a background subtraction method, an optical flow method, and a frame difference method. The motion detection method of claim 1, further comprising acquiring the number of frames tracked for tracking the picture frame sequence; and in response to the number of the tracked frames reaching a preset value, executing the calibration element in the pair of video input picture frame sequences again Carry out detection to obtain the position information of the calibration element. A motion detection device, comprising: a photographing module for photographing the movement of the subject to form a picture frame sequence, the subject wearing a calibration element; a detection module for detecting the calibration element in the picture frame sequence to obtain position information of the calibration element; a tracking module for tracking the calibration element in the sequence of picture frames according to a tracking algorithm; and A measuring module is used for measuring the moving distance of the calibration element. An electronic device comprising: a processor; and A memory stores the computer program, and when the computer program is executed by the processor, makes the processor execute the motion detection method according to any one of request items 1 to 8. A computer-readable recording medium with a stored program, on which computer-readable instructions are stored, when the instructions are executed by a processor, the processor is made to execute the motion detection method of any one of claims 1 to 8 .

Images