CN112101273B - Data preprocessing method based on 2D framework - Google Patents

Abstract

The invention discloses a data preprocessing method based on a 2D skeleton, which screens skeleton data according to human body characteristics and mainly comprises the steps of 1) filtering and detecting inaccurate joint points by using skeleton length according to the bilateral symmetry characteristics of a human body; 2) and filtering and detecting inaccurate joint points by using frame difference according to the characteristic that the human body acceleration has an upper limit. By the preprocessing method, the reliability of the whole skeleton data can be improved.

Description

Data preprocessing method based on 2D framework

Technical Field

The invention relates to the technical field of data processing of 2D frameworks, in particular to a data preprocessing method based on a 2D framework.

Background

With the development of economy, cameras are increasingly popular, and are not only used for road monitoring, but also used for monitoring at home or in schools. Human action recognition exposes the corners in the fields of monitoring and the like, and in addition to utilizing traditional RGB image data as input data, the appearance of depth cameras and human posture estimation algorithms provides another form of input data, namely skeleton data. For example, although the human body posture estimation method such as openpos can provide reliable skeleton data, there are some cases where the detection point is inaccurate.

In response to this need, we propose to use human body properties to filter out detected inaccurate joint points to improve the overall confidence of the data.

Aiming at data preprocessing, a plurality of schemes are provided in the academic circles at home and abroad. The technical scheme which is closer to the invention comprises the following steps: the invention patent (application number: 201911268767.1, name: a general flow method and system for data preprocessing) describes a method for processing data by determining a data calculation mode according to a data mapping relation; the invention patent (application number: 201911348312.0, name: data preprocessing method, apparatus, computer device, and storage medium) describes a method for replacing missing values with historical data averages for weather data. The prior art has no preprocessing method aiming at the skeleton data and does not fully utilize the characteristics of the human body.

In summary, the current data preprocessing method based on the 2D skeleton has the following disadvantages: 1) there is no pretreatment method for the scaffold; 2) the characteristics of the human body are not fully utilized.

Disclosure of Invention

Aiming at the defects of the current data preprocessing method based on the 2D framework, the invention provides a data preprocessing method based on the 2D framework.

The technical scheme of the invention is as follows:

a data preprocessing method based on a 2D framework is characterized by comprising the following steps:

step 1: the known skeleton Data is Data ═ v_i|i＝1,2,…,N},v_i＝{(x_it,y_it) 1,2, …, T, joint symmetry Map<v_l,v_r>|l＝1,2,…,N，r＝1,2,…,N，v_lAnd v_rIs a pair of joints representing bilateral symmetry, and the inherent joint relationship E ═ arch pitch<v_i,v_j>1,2, …, N, j 1,2, …, N and i ≠ j and joint v_iAnd v_jThere is a physical bone connection between }, v_jThe j-th joint is shown as being,<v_i,v_j>denotes the ith joint v_iAnd j-th joint v_jThe edge on the space (x) existing between them_it,y_it) Denotes the ith joint v_iCoordinates in the image collected at the T-th moment, wherein T represents the number of image sequences, and N represents the total number of joints;

step 2: the method for filtering and detecting inaccurate joint points by using the characteristics of unchanged length and bilateral symmetry of human bones comprises the following specific steps:

step 2.1: for the<v_i,v_j>E, calculating the bone length B by using the formula (1)_it；

Step 2.2: calculating the standard deviation BS of the bone length_i；

BS_i＝σ({B_it|t＝1,2,…,T}) (2)

In the formula, σ (·) represents a standard deviation calculation;

step 2.3: for the<v_l,v_r>E.g. Map, if BS_l<BS_rCalculating v_lAverage bone length of, wherein BS_lIs a joint v_lStandard deviation of bone length, BS_rIs a joint v_rStandard deviation of bone length of (a);

BA_l＝mean({B_lt|t＝1,2,…,T}) (3)

where mean (-) represents the mean calculation, B_ltRepresents a joint v_lBone length at time t;

step 2.4: if B is present_rt<α_min*BA_lOr B_rt>α_max*BA_lThen x is set_rt＝0，y_rt0, wherein α_minAnd alpha_maxRespectively represent upper and lower threshold values, B_rtIs a joint v_rBone length at time t, x_rtIs a joint v_rAbscissa value, y, in the image at time t_rtIs a joint v_rOrdinate values in the image at time t;

step 2.5: for the<v_l,v_r>E.g. Map, if BS_r<BS_lCalculating v_rAverage bone length of (d);

BA_r＝mean({B_rt|t＝1,2,…,T}) (4)

in the formula, mean (-) represents the mean calculation;

step 2.7: if B is present_lt<α_min*BA_rOr B_lt>α_max*BA_rThen x is set_lt＝0，y_lt＝0，x_ltIs a joint v_lAbscissa value, y, in the image at time t_ltIs a joint v_lOrdinate values in the image at time t;

and step 3: the method filters points with inaccurate detection by utilizing the characteristic that the human motion acceleration has an upper limit, so the frame difference cannot be suddenly increased, and comprises the following specific steps:

step 3.1: calculating the frame difference M using equation (5)_it：

Step 3.2: if M is_it>β*M_i(t-1)Then x is set_it＝0，y_itWhere β is a threshold coefficient, M_i(t-1)Is a joint v_iFrame difference, x, at time t-1_itIs a joint v_iAbscissa value, y, in the image at time t_itIs a joint v_iOrdinate values in the image at time t.

The invention has the advantages that: aiming at the 2D skeleton data, the data is preprocessed by fully utilizing the human body characteristics. Obviously wrong joint points in the joint point detection result are removed according to the left-right symmetry of the human body and the characteristic that the acceleration has the upper limit, so that the overall reliability of the data is improved, and the accuracy of motion recognition is finally improved.

Detailed Description

The present invention will be further described with reference to the following examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

A data preprocessing method based on a 2D framework comprises the following specific steps:

step 1: the known skeleton Data is Data ═ v_i|i＝1,2,…,N},v_i＝{(x_it,y_it) 1,2, …, T, joint symmetry Map<v_l,v_r>|l＝1,2,…,N，r＝1,2,…,N，v_lAnd v_rIs a pair of joints representing bilateral symmetry, and the inherent joint relationship E ═ arch pitch<v_i,v_j>1,2, …, N, j 1,2, …, N and i ≠ j and joint v_iAnd v_jThere is a physical bone connection between }, v_jThe j-th joint is shown as being,<v_i,v_j>denotes the ith joint v_iAnd j-th joint v_jThe edge on the space (x) existing between them_it,y_it) Denotes the ith joint v_iCoordinates in the image collected at the T-th moment, wherein T represents the number of image sequences, and N represents the total number of joints;

step 2: the method for filtering and detecting inaccurate joint points by using the characteristics of unchanged length and bilateral symmetry of human bones comprises the following specific steps:

step 2.1: for the<v_i,v_j>E, calculating the bone length B by using the formula (1)_it；

Step 2.2: calculating the standard deviation BS of the bone length_i；

BS_i＝σ({B_it|t＝1,2,…,T}) (2)

In the formula, σ (·) represents a standard deviation calculation;

step 2.3: for the<v_l,v_r>E.g. Map, if BS_l<BS_rCalculating v_lAverage bone length of, wherein BS_lIs a joint v_lStandard deviation of bone length, BS_rIs a joint v_rStandard deviation of bone length of (a);

BA_l＝mean({B_lt|t＝1,2,…,T}) (3)

where mean (-) represents the mean calculation, B_ltRepresents a joint v_lBone length at time t;

step 2.4: if B is present_rt<α_min*BA_lOr B_rt>α_max*BA_lThen x is set_rt＝0，y_rt0, wherein α_minAnd alpha_maxRespectively represent upper and lower threshold values, B_rtIs a joint v_rBone length at time t, x_rtIs a joint v_rAbscissa value, y, in the image at time t_rtIs a joint v_rOrdinate values in the image at time t;

in the present embodiment, α_min＝0.4，α_max＝1.6；

Step 2.5: for the<v_l,v_r>E.g. Map, if BS_r<BS_lCalculating v_rAverage bone length of (d);

BA_r＝mean({B_rt|t＝1,2,…,T}) (4)

in the formula, mean (-) represents the mean calculation;

step 2.7: if B is present_lt<α_min*BA_rOr B_lt>α_max*BA_rThen x is set_lt＝0，y_lt＝0，x_ltIs a joint v_lAbscissa value, y, in the image at time t_ltIs a joint v_lOrdinate values in the image at time t; and step 3: the method filters points with inaccurate detection by utilizing the characteristic that the human motion acceleration has an upper limit, so the frame difference cannot be suddenly increased, and comprises the following specific steps:

step 3.1: calculating the frame difference M using equation (5)_it：

Step 3.2: if M is_it>β*M_i(t-1)Then x is set_it＝0，y_itWhere β is a threshold coefficient, M_i(t-1)Is a joint v_iFrame difference, x, at time t-1_itIs a joint v_iAbscissa value, y, in the image at time t_itIs a joint v_iOrdinate values in the image at time t;

in the present embodiment, β ═ 5.

Claims (1)

1. A data preprocessing method based on a 2D framework comprises the following technical scheme:

step 1: the known skeleton Data is Data ═ v_i|i＝1,2,…,N},v_i＝{(x_it,y_it) 1,2, …, T, joint symmetry Map<v_l,v_r>|l＝1,2,…,N，r＝1,2,…,N，v_lAnd v_rIs a pair of joints representing bilateral symmetry, and the inherent joint relationship E ═ arch pitch<v_i,v_j>|i＝1,2,…,N，j＝1,2, …, N and i ≠ j and joint v_iAnd v_jThere is a physical bone connection between }, v_jThe j-th joint is shown as being,<v_i,v_j>denotes the ith joint v_iAnd j-th joint v_jThe edge on the space (x) existing between them_it,y_it) Denotes the ith joint v_iCoordinates in the image collected at the T-th moment, wherein T represents the number of image sequences, and N represents the total number of joints;

step 2: the method for filtering and detecting inaccurate joint points by using the characteristics of unchanged length and bilateral symmetry of human bones comprises the following specific steps:

step 2.1: for the<v_i,v_j>E, calculating the bone length B by using the formula (1)_it；

Step 2.2: calculating the standard deviation BS of the bone length_i；

BS_i＝σ({B_it|t＝1,2,…,T}) (2)

In the formula, σ (·) represents a standard deviation calculation;

step 2.3: for the<v_l,v_r>E.g. Map, if BS_l<BS_rCalculating v_lAverage bone length of, wherein BS_lIs a joint v_lStandard deviation of bone length, BS_rIs a joint v_rStandard deviation of bone length of (a);

BA_l＝mean({B_lt|t＝1,2,…,T}) (3)

where mean (-) represents the mean calculation, B_ltRepresents a joint v_lBone length at time t;

step 2.4: if B is present_rt<α_min*BA_lOr B_rt>α_max*BA_lThen x is set_rt＝0，y_rt0, wherein α_minAnd alpha_maxRespectively represent upper and lower threshold values, B_rtIs a joint v_rBone length at time t, x_rtIs a joint v_rAbscissa value, y, in the image at time t_rtIs a joint v_rOrdinate values in the image at time t;

step 2.5: for the<v_l,v_r>E.g. Map, if BS_r<BS_lCalculating v_rAverage bone length of (d);

BA_r＝mean({B_rt|t＝1,2,…,T}) (4)

in the formula, mean (-) represents the mean calculation;

step 2.7: if B is present_lt<α_min*BA_rOr B_lt>α_max*BA_rThen x is set_lt＝0，y_lt＝0，x_ltIs a joint v_lAbscissa value, y, in the image at time t_ltIs a joint v_lOrdinate values in the image at time t;

and step 3: the method filters points with inaccurate detection by utilizing the characteristic that the human motion acceleration has an upper limit, so the frame difference cannot be suddenly increased, and comprises the following specific steps:

step 3.1: calculating the frame difference M using equation (5)_it：

Step 3.2: if M is_it>β*M_i(t-1)Then x is set_it＝0，y_itWhere β is a threshold coefficient, M_i(t-1)Is a joint v_iFrame difference, x, at time t-1_itIs a joint v_iAbscissa value, y, in the image at time t_itIs a joint v_iOrdinate values in the image at time t.