CN112085675B

CN112085675B - Depth image denoising method, foreground segmentation method and human motion monitoring method

Info

Publication number: CN112085675B
Application number: CN202010894752.2A
Authority: CN
Inventors: 李元媛; 何飞; 何凌; 朱婷; 熊熙; 孟雨璇; 周格屹
Original assignee: Sichuan University
Current assignee: Sichuan University
Priority date: 2020-08-31
Filing date: 2020-08-31
Publication date: 2023-07-04
Anticipated expiration: 2040-08-31
Also published as: CN112085675A

Abstract

The invention discloses a depth image denoising method, a foreground segmentation method and a human motion monitoring method. According to the invention, the depth image is segmented, the first characteristic point and the second characteristic point of each block are marked based on the base plane, then the first parameter carrying the isolated information and the space distribution information is calculated, each block is clustered by representing the attribute parameter of each block as a characteristic set, basin area blocks are further filtered, the plateau area blocks are reserved, and the difference of gray level of the first characteristic point and the second characteristic point of the nearest extension block of the plateau area is judged to carry out edge protection, so that denoising is completed. For the denoised ROI, foreground segmentation is performed using multi-level contours. For the segmented human body object, the motion is monitored by carrying out plane fitting on the segmented part and calculating the included angle between the fitting plane and the basic plane. The invention does not need modeling or a large number of iterative operations, and has simple operation process, high processing efficiency and high accuracy.

Description

Depth image denoising method, foreground segmentation method and human motion monitoring method

Technical Field

The invention relates to the field of image processing, in particular to a depth image denoising method, a foreground segmentation method and a human motion monitoring method.

Background

The depth image can reflect the depth information of the research object, and has good data support for the research of the moving object.

Depth image acquisition tools (such as Kinect sensors) typically calculate depth images by transmitting and receiving infrared reflections from infrared sensors in space, but if the object under test is too far from or is reflected multiple times at the target, a large amount of non-uniform noise appears around the subject, in the depth images, the noise also has depth information, and the noise has irregularities and uncertainties, so that the noise has a great influence on subsequent analysis and calculation of the depth information of the subject. Most directly, the analysis of the study object generally requires extracting/dividing depth information of the study object from the depth image, and noise can affect the extraction and division of the study object because the noise also has the depth information.

In the prior art, there are schemes for denoising through image entropy of a depth image, for example, an image denoising method based on anisotropic diffusion of PCNN and image entropy disclosed in CN105005975A, a small target infrared image processing method based on weighted local image entropy disclosed in CN104268844A, an infrared image non-uniformity parameterization correction optimization method based on image entropy disclosed in CN111047521a, and the like. However, the methods all need a large amount of data processing work such as modeling, iterative operation and the like, are complex, have low efficiency and have unsatisfactory denoising effect.

Disclosure of Invention

The invention aims at: in order to solve the above problems, a depth image denoising method is provided to provide a simple and efficient denoising method for accurately denoising a depth image.

The technical scheme adopted by the invention is as follows:

a depth image denoising method for denoising a noise region of a depth image, the noise region being equally sized into a plurality of blocks, the depth image denoising method comprising the steps of:

the following A-B calculations are performed for each block separately:

A. and respectively determining a first characteristic point and a second characteristic point based on the base plane in the block, wherein the first characteristic point is a pixel point with a pixel value below the base plane, and the second characteristic point is a pixel point with a pixel value above the base plane. The purpose of this step is to mark the first and second feature points of each block.

B. The first parameter is calculated by matching the point weight of each second feature point with the minimum position corresponding to each second feature point. The pixel value of the second feature point is located above the base plane, so that the second feature point carries a more prominent feature, and the feature of the block can be further described by applying the feature.

C. Taking the first parameter of each block and the attribute parameters of the first feature point and the second feature point as feature sets of each block, and adopting a clustering method to divide each block into three types: a first class corresponding to a low gray scale range, a second class carrying abrupt values, and a third class having a relatively flat surface; and flattening the first type of block, reserving the third type of block, and determining to reserve or flatten the corresponding second type of block according to the gray level difference between the second characteristic point and the first characteristic point of the nearest extended block of the third type of block.

The low gray scale range block belongs to the background area, does not carry the depth information of the study object, can be directly subjected to background processing, the block with the gray scale value carrying the abrupt change value is quite likely to be an edge block between the study object and the background, further judgment needs to be carried out according to the characteristics of the block, and the block with the relatively flat surface belongs to the interior of the study object body, carries the depth information of the study object, and can be directly reserved. Therefore, through simple statistical analysis, the noise can be removed more accurately, and the whole operation process is simple and efficient.

Further, in the step a, the value of the base plane is an average value of gray values of pixels in the block. The base plane of each block is calculated based on the gray value of the base plane, so that the more prominent characteristic of each block can be highlighted.

Further, in the step B, the method for calculating the point weight of the second feature point includes:

pixels of the first feature point and the second feature point in the block are described by

logic

1 and 0 respectively, and corresponding point weights are calculated for the NxN area around the neighborhood of the second feature point by the following formula:

where w is the point weight of the second feature point, a _ij The second feature point is a pixel logic value in the NxN neighborhood, and N is an integer greater than or equal to 3. So-called secondThe region around the feature point neighborhood is a region centered on the second feature point. The second characteristic points are used as the salient points of the gray values in the blocks, and the pixel gray values of the characteristic points are logically replaced, so that the isolation degree of the second characteristic points in the blocks can be estimated by the method, and the later unified logic operation is facilitated.

Further, in the step B, the calculating method of the minimum position matching corresponding to the second feature point includes:

P _M ＝min{(|(m-x _i )|+|(n-y _i )|)|i＝1,2,…,N _o }

wherein P is _M Is the calculated minimum position match; (m, n) represents the coordinate position of a second feature point in the block, (x) _i ,y _i ) Coordinate values representing other second feature points than the (m, N) point, min being a function of calculating the minimum value, N _o Is the number of other second feature points than the current second feature point. The minimum position matching can reflect the spatial distribution relation among all second characteristic points in the block, and the characteristic description complementation is realized with the point weight.

Further, in the step B, the calculating method of the first parameter is as follows: and respectively calculating products of point weights corresponding to the second feature points and minimum position matching, and taking the average value of all the products.

Because the pixel gray levels of the first and second feature points are logically replaced, the point weights can participate in logic operation, the point weights are multiplied by the minimum position matching, the isolation degree of each second feature point and the spatial distribution relation between the second feature points and the rest of second feature points can be described, and the corresponding features of the block can be described after the average is carried out. Corresponding to the complete scheme, the noise points can be described more accurately.

Further, in the step C, attribute parameters of the first feature point and the second feature point are respectively: and the gray average value of the first characteristic point and the second characteristic point.

The invention also provides a foreground segmentation method, which is used for segmenting the foreground from the ROI (region of interest ) with depth information, wherein the ROI is denoised by applying the depth image denoising method; the foreground segmentation method comprises the following steps: and according to the distribution of the pixels of the ROI on the gray level, respectively extracting the main contour of the ROI by utilizing a plurality of levels of contour lines, merging the contour extracted by each level of contour lines, and dividing the ROI by utilizing the merged contour to obtain the prospect.

The contour extraction method based on the contour lines can improve the continuity and the integrity of the extracted edges.

Further, the method for segmenting the ROI by using the merged contour includes:

filling the combined outline area: filling a logic 1 into an area in the edge of the outline and filling a logic 0 into an area beyond the edge to obtain a filled image;

multiplying the ROI with the fill image.

After the region in the contour edge is logically filled, the filled region can participate in logic operation. The foreground region can be directly extracted by directly multiplying the region after logic filling with the ROI (operation object), complex operation is not needed, the consumed calculation force is small, and the operation efficiency is high.

The invention also provides a human motion monitoring method which is used for analyzing the human foreground of the human image with depth information, wherein the human foreground in the human image is obtained by segmentation through the foreground segmentation method; the human motion monitoring method comprises the following steps:

A. a body part segmentation step;

B. performing plane fitting on each body part to be segmented, and calculating an included angle between a fitting plane and a basic plane;

C. and locating the abrupt change value of the included angle on the time axis to record as effective movement.

The depth information can reflect the change of the distance or angle of the change area, when the distance or the azimuth of the observed object is changed in the depth image, the distribution of the depth points in the three-dimensional space is changed, the motion state of the monitored object can be monitored by monitoring the change of the angle in the three-dimensional space, and the three-dimensional motion state can be monitored.

Further, the step C includes:

c1, counting included angles along a time axis progress;

c2, reserving an included angle value larger than the average value of the included angles as a salient value, and setting the rest included angle values to 0;

and C3, recording the effective movement according to the following rule:

when the salient value appears in the continuous K1 frames after one salient value, the motion is regarded as a continuation of one motion;

when the included angle value exceeding the K2 frame is set to 0 after one protruding value, the motion is recorded as the end of one motion, K1 and K2 are positive integers, and K1 is less than K2.

One action often extends over multiple frames, i.e., when one action occurs, there is often a similar plane angle in successive multiple frames, and statistics of the motion amount needs to be distinguished as to whether the motion belongs to successive actions. The method can set the corresponding threshold value according to the corresponding monitoring object (or set according to the experience value), thereby accurately and efficiently completing the statistics of the motion quantity.

In summary, due to the adoption of the technical scheme, the beneficial effects of the invention are as follows:

1. according to the depth image denoising method, a brand-new denoising design concept is adopted according to inherent characteristics of the depth image, noise is filtered according to the distribution characteristics of noise points, data processing work with extremely large calculation amount such as modeling, large iterative data amount and the like in the traditional method is saved, the calculation efficiency is high, and denoising rules accord with the noise distribution characteristics, so that denoising is more targeted and the denoising effect is better.

2. The foreground segmentation method adopts the multi-level contour lines to carry out edge extraction on the main body, improves the continuity and the integrity of the contour, and can further filter noise. In addition, in the foreground segmentation process, a logic replacement method is adopted, so that direct logic operation can be performed on the ROI, the operation efficiency is high, and the fidelity is high.

3. According to the human body movement monitoring method, the plane angle change of the main body is monitored according to the movement distribution characteristics of the three-dimensional space, the monitoring accuracy of the movement state is high, and the statistical result of the movement amount is accurate.

4. All the methods designed by the invention do not need to carry out early modeling, and a large amount of early sample collection work and model training work are saved. And a large amount of iterative operation is not required to be performed on the monitoring data, so that the calculation force resource and the calculation time are greatly saved, and the accuracy of the calculation result and the reliability of the calculation resource are improved.

Drawings

The invention will now be described by way of example and with reference to the accompanying drawings in which:

fig. 1 is a schematic diagram of a ROI extraction flow based on depth images and bone images.

Fig. 2 is a diagram of noise distribution characteristics.

Fig. 3 is a flow diagram of locating noise regions by entropy distribution.

Fig. 4 is two embodiments of the base plane distinguishing the first feature point from the second feature point.

Fig. 5 is a schematic diagram of a logical replacement of a first feature point and a second feature point in a block.

Fig. 6 is a point weight calculation embodiment of the second feature point in the block.

Fig. 7 is a first parameter calculation embodiment of 4 blocks.

FIG. 8 is a schematic diagram of a partition block denoising process in combination with mean clustering.

FIG. 9 is a schematic illustration of contour extraction of a body contour at different levels.

Fig. 10 is a schematic diagram of ROI region extraction flow.

Fig. 11 is a schematic diagram of a head and limb segmentation process.

Fig. 12 is one embodiment of head area angle statistics.

Detailed Description

All of the features disclosed in this specification, or all of the steps in a method or process disclosed, may be combined in any combination, except for mutually exclusive features and/or steps.

Any feature disclosed in this specification (including any accompanying claims, abstract) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. That is, each feature is one example only of a generic series of equivalent or similar features, unless expressly stated otherwise.

Kinect calculates depth images by transmitting and receiving infrared reflections from infrared sensors in space. It has a certain field of view at the fixation site, so if the object to be measured is too far from the target or multiple reflections occur at the target, uneven noise appears around the subject, as shown in fig. 1 (a), and noise appears around the person to varying degrees. For the subsequent study of the study object, to ensure accuracy, noise needs to be filtered in advance, and first, a noise area in the depth image needs to be located. In this regard, the present embodiment proposes a depth image noise region positioning method.

The method firstly extracts the ROI from the depth image, taking a natural person as a research object as an example, and the ROI is the region where the human body is located.

The depth image acquired by Kinect has the same coordinate space as the bone image. Thus, as shown in fig. 1, the person in both images can be located by the same coordinates. In fig. 1, (a) is an original depth image, (b) is a bone image, and (c) is an extracted ROI. And (3) carrying out row and column projection on a depth region where the object is located in the depth image in comparison with the skeleton image, so that the object can be positioned in the depth image, and further extracting the positioning region to obtain the ROI.

The distribution of noise in the depth image has irregularity and uncertainty, and is shown in fig. 2. Based on the characteristics of the irregularity and uncertainty, the noise domain can be located, and the entropy can reflect the uncertainty of information quantity and data point distribution in one area, so that the embodiment locates the noise domain based on the image entropy distribution.

In the present noise localization method, the ROI area is divided into blocks of 4x4 pixel size (or other sizes, denoted by N x N, the following, and the like), and for each block, an alternative representation is performed by its entropy value. The calculation formula of entropy is as follows (1):

where H (x) is the entropy calculated for all pixels of the current block, P (x) is the probability of occurrence of an element with a specific gray value in the block, a _i Representing the gray value of each pixel.

As shown in fig. 3, by the above calculation, an entropy distribution map of the ROI region can be obtained, from which it can be found that the entropy value calculated by the noise distribution region is higher. And carrying out row projection on the entropy distribution diagram, wherein the noise area appears as different prominent peaks in the projection curve, and the noise area can be determined through determining the peak value interval. For a human body research object, according to the edge characteristics of a human body, a noise area is directly determined by the average value level, and a block with an entropy value higher than the average value is positioned in the noise area by calculating the average value of entropy.

And (5) carrying out further noise filtering processing on the calibrated noise area. To this end, the present embodiment provides a depth image denoising method, which includes:

the following A-B calculations are performed separately for each block of the noise region:

A. the first feature point and the second feature point are respectively determined based on the base plane in the block.

Noise in the depth image appears as some kind of geographical distribution features such as "isolated peaks", "basin" and "plains". In the noise region, some "plains" are between "solitary peaks", while the subject split region is characterized as "plateau-like". Based on this feature, the key to extracting the edges of the complete object is to background the "isolated peaks" and "basin" blocks, highlighting the depth region where the object is located. Based on the characteristic that the 'solitary peak' and the 'basin' have differences in gray level distribution, a base plane can be established based on gray level distribution to distinguish the 'solitary peak' and the 'basin', and the calculation method of the base plane comprises the following steps:

wherein B is _m Is the value of the base plane, N is the number of pixels in a selected region, G is the gray value of each pixel in the selected region, M ₁ And M ₂ Respectively the gray average value of the first characteristic point and the second characteristic point in the selected area, N ₁ And N ₂ The number of first feature points and second feature points within the selected region are represented, respectively. Fig. 4 gives two examples of base planes.

As shown in fig. 4, each pixel point in the 4x4 block is divided into a first feature point and a second feature point, a point (pixel point) having a pixel value located above the base plane is the second feature point, and a point below the pixel value is the first feature point. By using the base plane as an interface, the first feature point and the second feature point in each block can be distinguished.

B. A first parameter of the block is calculated.

The gray value of the first feature point in the block is represented by logic 1, the gray value of the second feature point is represented by logic 0, and as shown in fig. 5, the gray value of each pixel point in a certain block is replaced by a logic value, and in each block, the first feature point is marked with 1, and the second feature point is marked with 0. The calculation process of the first parameter of the block includes:

B1. a point weight is calculated for each second feature point in the block.

The gray value of each pixel of the divided block is marked (represented by logic 1 or 0), the marking result of each pixel can be used to calculate the isolation degree of a second feature point in a 3x3 (or other larger square size, taking 3x3 as an example here and also for better matching with the resolution of the depth image) template with the center, the judgment is made by counting the number of first feature points around the second feature point in the 3x3 template, and the calculation is performed by the formula (5).

Where w is the point weight of each second feature point, a _ij Is a pixel around the neighborhood of the second feature point (the second feature point is the center of the 3x3 region) (the pixels not belonging to the selected region of the 8 neighborhoods are set to zero), w also represents the effective first feature point number around the second feature point because all the second feature points are 0. Fig. 6 shows an example of calculating the point weights of the second feature points in one block. The point weight can reflect the degree of isolation of each second feature point, but cannot reflect the spatial distribution relationship between all the second feature points. Therefore, the spatial distribution relationship of the second feature points also needs to be described.

B2. And calculating the minimum position matching of each second characteristic point in the block and other second characteristic points.

Calculating a minimum position match for each second feature point in each block using equation (6):

P _M ＝min{(|(m-x _i )|+|(n-y _i )|)|i＝1，2，...，N _o } (6)

wherein P is _M Is the calculated minimum position match; (m, n) represents the coordinate position of a second feature point in the block, (x) _i ,y _i ) Coordinate values representing other second feature points than the (m, N) point, min being a function of calculating the minimum value, N _o Is the number of other second feature points than the current second feature point.

B3. The first parameter is calculated by matching the point weight of each second feature point with the minimum position corresponding to each second feature point.

The first parameter (represented by MPD) is calculated by equation (7):

where N is the number of second feature points in one block (4 x4 region). An example of the calculation of the first parameters of the four blocks is shown in fig. 7, respectively.

As shown in fig. 7, the first parameter may reflect a positional distribution relationship of the second feature points. In addition, it has a great advantage in protecting edges because the blocks having edge-like characteristics have smaller first parameter values, as shown in fig. 7 (a) and (b). By using the first parameter and combining the isolated existence characteristic of the noise point, whether the current area is a noise block or not can be judged by evaluating the distribution characteristic of the pixels.

C. Noise in the noise region is filtered out using a clustering method.

Three features (MPD, M1, and M2) are used as feature sets with which the features of each block are described.

The blocks are divided into 3 groups, feature sets of the 3 blocks are randomly selected as initial clustering centers, then the distance between each feature set and each clustering center is calculated, and the feature set participating in the current calculation is distributed to the closest clustering center. The cluster centers and the feature sets assigned to them represent a cluster. Every time a sample (feature set) is allocated, the clustering center of the cluster is recalculated according to the existing objects in the cluster until the clustering center is not changed any more and the square sum of errors is minimum locally, so that the clustering is completed once. The blocks involved in the calculation can be divided into three types: a "basin" of low gray scale, a "plains" carrying "isolated peaks", and a "plateau" with a relatively flat surface. The depth information area of the subject is mostly "plateau", while other classified areas are flattened or preserved for possible edge blocks by the subject block extension.

As shown in FIG. 8It is shown that by clustering, blocks with different distribution features can be distinguished in combination with the extracted feature set. According to the block label (clustering result), the basin block is replaced by a flat background, the detected plateau region is reserved, and whether the gray level of the second characteristic point and the first characteristic point of the plain block carrying the isolated peak adjacent to the plateau region is significantly different or not is judged (M ₁ <M ₂ 2) edge protection (if M ₁ <M ₂ 2, then remain, otherwise replace with a flat background). Then, other noise block disturbances around the observed object are eliminated. Thus, the denoising of the ROI is completed.

For the study object, after denoising the depth image, depth information (i.e. foreground segmentation) in the edge of the study object needs to be extracted for further study, and edge extraction is an important link for foreground object segmentation in the depth image. Since kinect is the measurement of distance by transmitting and receiving infrared reflections, noise at the foreground object and background boundary forms an edge, and there are many break points at the edge of the observed object. Conventional edge extraction methods are not applicable in this case, since they do not take into account the continuity and integrity of the edges, the extracted edges may have multiple break points, thereby affecting the segmentation of the object. In this regard, the present embodiment provides a foreground segmentation method that employs contours to extract edges of objects. In the contour calculation, linear interpolation is used to maintain edge continuity. Extracting edges using contours of different depth levels may improve edge continuity and integrity.

Based on the ROI in the above embodiment, the contour of the subject is extracted using 4 levels of contour lines, the values of which are 50, 100, 150, 200, respectively. In other embodiments, the level, value of the contour line may be adaptively adjusted according to the distribution of the ROI pixels on the gray scale. The contour information of the subject region on the contour lines of different levels is shown in fig. 9.

Fig. 10 shows the ROI region extraction process, fig. 10 (b) shows the result of merging (projecting onto the same plane) the contours extracted from the contours of each class, on the basis of which the region between the edges is filled with a logical value of 1, and the other regions beyond the edges are marked with a logical value of 0, resulting in a filled image as shown in fig. 10 (c), and finally the depth information of the study object can be extracted by multiplying the denoised ROI depth image with the filled image, as shown in fig. 10 (d).

Based on the extraction of the edge depth information of the research object, the embodiment also discloses a human motion monitoring method. The method can be applied to the analysis of the status of ADHD symptomatic patients to provide more accurate and objective information to the clinician. The human motion monitoring method comprises the following steps:

A. a body part segmentation step.

The parts of the human body that are active include the head, torso and limbs, where the invention divides the body area into five parts:

according to the characteristics of human organ proportion, the head proportion is smaller, and according to the line projection of the trunk, the head region can be rapidly positioned from the logic image, so that the human head in the depth image is segmented according to the positioning. The head region in the logical image is then set to zero and the body centroid, except for the head, is calculated. The human body region is divided into 4 parts according to the position of the centroid (x, y). The pixels in the x-th row and y-th column are set to zero, dividing the body into four parts, as shown in fig. 11, corresponding to extracting depth information of each part from the depth image.

B. And (5) plane fitting and included angle calculation.

The depth information may reflect a change in the distance or angle of the change region. In the depth image, when the distance or azimuth of the observed object changes, the distribution of the depth points in the three-dimensional space also changes. By a linear regression method, plane fitting can be performed on the distribution of depth points to reflect the motion change condition.

And randomly selecting a basic plane, newly fitting the regional plane by using a linear regression method, and calculating an included angle between the newly fitted plane and the basic plane by using a formula (8).

Where θ is the calculated angle, a is the normal vector to the newly fitted plane, and b is the normal vector to the base plane. The change in the included angle may reflect the magnitude of the movement of the subject. An example of the time course of the angle in the head region is shown in fig. 12.

As shown in fig. 12, the line is a locus of angles calculated from the head region in the time series. The results show that most angles have similar values, while some angles in the track have significant values. This means that the distribution of depth information is changed compared to the base plane. Consequently, the angle between the base plane and the newly fitted plane changes significantly. The intensity of the motion is reflected by counting the amount of motion (the number of times of motion occurrence) in the region where the change occurs.

The statistics of the motion quantity is based on a curve of the included angle, the motion area is positioned by utilizing mutation of a time sequence according to the change of the included angle on a time axis, the mutation values are detected by the statistical average value of all points on the curve, and the mutation values are regarded as mutation when the mutation values are higher than the average value. The method for judging whether the action occurs is as follows:

1) The included angle value is larger than the average value level of the overall included angle track curve, the values are reserved as the salient values, and the included angle value lower than the average value level is set to be 0;

2) For the salient values, when one action occurs, a similar plane included angle is usually formed in a plurality of continuous frames, each salient value is searched for the salient value along the time axis, and when the salient value occurs in a continuous K1 frame (K1 <10, K1 is a positive integer), the action is considered to be continuous of the same action, and the action is counted as one movement; continuing to judge from the detected new salient value to the rear, when a continuous K2 frame (K2 > =10, and K2 is a positive integer) is set to a value of 0, the current motion state is considered to be changed, and the change of the current motion state is regarded as one motion.

For example, for each salient value, the salient value search is performed for its backward direction, and when the salient value occurs within 5 consecutive frames, it is considered to be consecutive of the same action; when more than 10 frames of the value 0 are found to the right by the salience value, the motion is recorded as two motions, because the mutation of the included angle occurs twice. Of course, the corresponding threshold can be set according to the characteristics of the study object, so long as the basic requirements are not separated.

By the method, the movement occurrence condition of each part (head and limbs) can be judged, gesture detection on each part is not needed, and the method is more efficient and easier to realize.

The invention is not limited to the specific embodiments described above. The invention extends to any novel one, or any novel combination, of the features disclosed in this specification, as well as to any novel one, or any novel combination, of the steps of the method or process disclosed.

Claims

1. A depth image denoising method for denoising a noise region of a depth image, the noise region being equally sized into a plurality of blocks, the depth image denoising method comprising: the following A-B calculations are performed for each block separately:

A. respectively determining a first characteristic point and a second characteristic point based on a base plane in the block, wherein the first characteristic point is a pixel point with a pixel value positioned below the base plane, and the second characteristic point is a pixel point with a pixel value positioned above the base plane;

B. calculating a first parameter by matching the point weight of each second feature point with the minimum position corresponding to each second feature point;

the calculation method of the point weight of the second feature point comprises the following steps:

pixels of the first feature point and the second feature point in the block are described by logic 1 and 0 respectively, and corresponding point weights are calculated for the NxN area around the neighborhood of the second feature point by the following formula:

where w is the point weight of the second feature point, a _ij Is the logical value of the pixel of the second feature point in the NxN neighborhood, N isAn integer of 3 or more;

the calculation method of the minimum position matching corresponding to the second feature point comprises the following steps:

P _M ＝min{(|(m-x _i )|+|(n-y _i )|)|i＝1,2,…,N _o }

wherein P is _M Is the calculated minimum position match; (m, n) represents the coordinate position of a second feature point in the block, (x) _i ,y _i ) Coordinate values representing other second feature points than the (m, N) point, min being a function of calculating the minimum value, N _o Is the number of other second feature points than the current second feature point;

the first parameter calculating method comprises the following steps: calculating products of point weights and minimum position matching corresponding to each second characteristic point respectively, and taking the average value of all the products;

2. The method of denoising a depth image according to claim 1, wherein in the step a, the value of the base plane is an average value of gray values of pixels in a block.

3. The depth image denoising method according to claim 1, wherein in the step C, the attribute parameters of the first feature point and the second feature point are respectively: and the gray average value of the first characteristic point and the second characteristic point.

4. A foreground segmentation method for segmenting a foreground from an ROI with depth information, wherein the ROI is denoised by applying the depth image denoising method according to any one of claims 1 to 3; the foreground segmentation method comprises the following steps: and according to the distribution of the pixels of the ROI on the gray level, respectively extracting the main contour of the ROI by utilizing a plurality of levels of contour lines, merging the contour extracted by each level of contour lines, and dividing the ROI by utilizing the merged contour to obtain the prospect.

5. The foreground segmentation method of claim 4, wherein the method of segmenting the ROI using the merged contour comprises:

filling the combined outline: filling logic 1 in the area between the contour edges and filling logic 0 in the area beyond the edges to obtain a filling image;

multiplying the ROI with the fill image.

6. A human motion monitoring method for analyzing human foreground of a human image with depth information, characterized in that the human foreground in the human image is segmented by the foreground segmentation method according to claim 4 or 5; the human motion monitoring method comprises the following steps:

A. a body part segmentation step;

7. The method of claim 6, wherein the step C comprises:

c1, counting included angles along a time axis progress;

and C3, recording the effective movement according to the following rule: