CN109711302B - Model parameter calibration method, device, computer equipment and storage medium - Google Patents

Model parameter calibration method, device, computer equipment and storage medium Download PDF

Info

Publication number
CN109711302B
CN109711302B CN201811550990.0A CN201811550990A CN109711302B CN 109711302 B CN109711302 B CN 109711302B CN 201811550990 A CN201811550990 A CN 201811550990A CN 109711302 B CN109711302 B CN 109711302B
Authority
CN
China
Prior art keywords
data
posture
skeleton
inertial sensor
luminous point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811550990.0A
Other languages
Chinese (zh)
Other versions
CN109711302A (en
Inventor
马浩
刘维甫
刘昊扬
戴若犁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Nuo Yiteng Science And Technology Ltd
Original Assignee
Beijing Nuo Yiteng Science And Technology Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Nuo Yiteng Science And Technology Ltd filed Critical Beijing Nuo Yiteng Science And Technology Ltd
Priority to CN201811550990.0A priority Critical patent/CN109711302B/en
Publication of CN109711302A publication Critical patent/CN109711302A/en
Application granted granted Critical
Publication of CN109711302B publication Critical patent/CN109711302B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

This application involves a kind of model parameter calibration methods, device, computer equipment and storage medium, by obtaining historical data, historical data includes the historical position and posture information of skeleton, the historical measurement data of history image and inertial sensor comprising luminous point, obtain the current frame data of the current measurement data including present image and inertial sensor, the static estimation of "current" model parameter is calculated according to historical data and confidence level corresponding with historical data, historical data is updated according to the static estimation of model parameter, the confidence level of current frame data is calculated according to historical data, the dynamic compensation of "current" model parameter is calculated according to the confidence level of current frame data and updated historical data, the calibration accuracy of lift scheme parameter.

Description

Model parameter calibration method, device, computer equipment and storage medium
Technical field
This application involves technical field of computer vision more particularly to a kind of model parameter calibration methods, device, computer Equipment and storage medium.
Background technique
With the development of computer and electronic technology, optics motion capture technology based on computer vision and with micro- Inertia action capturing technology based on electromechanical inertial sensor is applied to each technical field.For optics and inertia action Capture system needs accurately to determine skeleton size, and be mounted on the view of human body surface to accurately capture human motion Feel the position of positioning spot and the posture of inertial sensor.In addition, being mounted on the vision positioning luminous point and inertia biography of human body surface Sensor position can change with the movement of human body, and positional accuracy is caused to reduce.The prior art lacks for human body ruler The method that very little parameter, vision positioning luminous point and inertial sensor installation parameter carry out joint calibration and optimization.
Summary of the invention
In order to solve the above-mentioned technical problem, this application provides a kind of model parameter calibration methods, device, computer equipment And storage medium.
A kind of model parameter calibration method, comprising:
Historical data is obtained, historical data includes the historical position and posture information, the history comprising luminous point of skeleton The historical measurement data of image and inertial sensor, luminous point and inertial sensor are mounted on human body；
Obtain the current frame data of the current measurement data including present image and inertial sensor；
The static estimation of "current" model parameter is calculated according to historical data and confidence level corresponding with historical data；
The confidence level of current frame data is calculated according to historical data；
It is mended according to the dynamic that current frame data, the confidence level of current frame data and historical data calculate "current" model parameter It repays.
A kind of model parameter calibrating installation, comprising:
Historical data obtains module, and for obtaining historical data, historical data includes the historical position and appearance of skeleton The historical measurement data of state information, the history image comprising luminous point and inertial sensor, luminous point and inertial sensor are mounted on people On body；
Current data obtains module, for obtaining the current of the current measurement data including present image and inertial sensor Frame data；
The static estimation computing module of model parameter, based on according to historical data and confidence level corresponding with historical data Calculate the static estimation of "current" model parameter；
Confidence level computing module, for calculating the confidence level of current frame data according to historical data；
The dynamic compensation calculation module of model parameter, for according to the confidence level of current frame data, current frame data and going through History data calculate the dynamic compensation of "current" model parameter.
A kind of computer equipment can be run on a memory and on a processor including memory, processor and storage Computer program, the processor perform the steps of when executing the computer program
Historical data is obtained, historical data includes the historical position and posture information, the history comprising luminous point of skeleton The historical measurement data of image and inertial sensor, luminous point and inertial sensor are mounted on human body；
Obtain the current frame data of the current measurement data including present image and inertial sensor；
The static estimation of "current" model parameter is calculated according to historical data and confidence level corresponding with historical data；
The confidence level of current frame data is calculated according to historical data；
It is mended according to the dynamic that current frame data, the confidence level of current frame data and historical data calculate "current" model parameter It repays.
A kind of computer readable storage medium, is stored thereon with computer program, and the computer program is held by processor It is performed the steps of when row
Historical data is obtained, historical data includes the historical position and posture information, the history comprising luminous point of skeleton The historical measurement data of image and inertial sensor, luminous point and inertial sensor are mounted on human body；
Obtain the current frame data of the current measurement data including present image and inertial sensor；
The static estimation of "current" model parameter is calculated according to historical data and confidence level corresponding with historical data；
The confidence level of current frame data is calculated according to historical data；
It is mended according to the dynamic that current frame data, the confidence level of current frame data and historical data calculate "current" model parameter It repays.
Above-mentioned model parameter calibration method, device, computer equipment and storage medium, by obtaining historical data, history Data include the historical position of skeleton and the history measurement of posture information, the history image comprising luminous point and inertial sensor Data, luminous point and inertial sensor are mounted on human body, obtain the current measurement data including present image and inertial sensor Current frame data, the static estimation of "current" model parameter is calculated according to historical data and confidence level corresponding with historical data, The confidence level that current frame data is calculated according to historical data calculates current mould according to the confidence level of current frame data and historical data The dynamic of shape parameter compensates, the calibration accuracy of lift scheme parameter.
Detailed description of the invention
The drawings herein are incorporated into the specification and forms part of this specification, and shows and meets implementation of the invention Example, and be used to explain the principle of the present invention together with specification.
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, for those of ordinary skill in the art Speech, without any creative labor, is also possible to obtain other drawings based on these drawings.
Fig. 1 is the application scenario diagram of model parameter calibration method in one embodiment；
Fig. 2 is the flow diagram of model parameter calibration method in one embodiment；
Fig. 3 is the flow diagram of model parameter calibration method in another embodiment；
Fig. 4 is the flow diagram of data-optimized step in one embodiment；
Fig. 5 is the flow diagram of data-optimized step in another embodiment；
Fig. 6 is the flow diagram of the confidence level calculating step in one embodiment；
Fig. 7 is the flow diagram of the dynamic compensation calculation step of model parameter in one embodiment；
Fig. 8 is the scheme of installation of luminous point and inertial sensor in one embodiment；
Fig. 9 is the schematic diagram of different human body posture in one embodiment；
Figure 10 is the structural block diagram of model parameter calibrating installation in one embodiment；
Figure 11 is the structural block diagram of model parameter calibrating installation in another embodiment；
Figure 12 is the structural block diagram of data update module in one embodiment；
Figure 13 is the structural block diagram of the static estimation computing module of model parameter in one embodiment；
Figure 14 is the structural block diagram of confidence level computing module in one embodiment；
Figure 15 is the structural block diagram of the dynamic compensation calculation module of model parameter in one embodiment；
Figure 16 is the internal structure block diagram of computer equipment in one embodiment.
Specific embodiment
To keep the purposes, technical schemes and advantages of the embodiment of the present application clearer, below in conjunction with the embodiment of the present application In attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is A part of the embodiment of the application, instead of all the embodiments.Based on the embodiment in the application, ordinary skill people Member's every other embodiment obtained without making creative work, shall fall in the protection scope of this application.
Fig. 1 is the applied environment figure of model parameter calibration method in one embodiment.Referring to Fig.1, the application that calibrates for error In error system.The error system includes terminal 110 and server 120.Terminal 110 or server 120 obtain historical data, go through History data include that the historical position of skeleton and the history of posture information, the history image comprising luminous point and inertial sensor are surveyed Data are measured, luminous point and inertial sensor are mounted on human body, obtain the current measurement number including present image and inertial sensor According to current frame data, estimated according to the static state that historical data and confidence level corresponding with historical data calculate "current" model parameter Meter calculates the confidence level of current frame data according to historical data, according to the confidence level and history of current frame data, current frame data Data calculate the dynamic compensation of "current" model parameter.Terminal 110 and server 120 pass through network connection.Terminal 110 specifically can be with It is terminal console or mobile terminal, mobile terminal specifically can be at least one of mobile phone, tablet computer, laptop etc.. Server 120 can be realized with the server cluster of the either multiple server compositions of independent server.
As shown in Fig. 2, in one embodiment, providing a kind of model parameter calibration method.The present embodiment is mainly with this Method is applied to the terminal 110 (or server 120) in above-mentioned Fig. 1 to illustrate.Referring to Fig. 2, the specific packet that calibrates for error Include following steps:
Step S201 obtains historical data.
In this embodiment, historical data include skeleton historical position and posture information, comprising luminous point The historical measurement data of history image and inertial sensor, luminous point and inertial sensor are mounted on human body.
Step S202 obtains the current frame data of the current measurement data including present image and inertial sensor.
Specifically, historical data refers to the data before current data, each human body bone including measuring obtained human body The historical position and posture information of bone, historical position and posture information include but is not limited to the location information, each of skeleton Relative position and posture information between skeleton etc..Luminous point refers to that the active on human body shines or passive reflective object Body.Inertial sensor is used to detect acceleration and posture of each limbs of human body etc..History image comprising luminous point is to pass through The image comprising luminous point of capture apparatus shooting, historical measurement data refer to the acceleration data and posture number of inertial sensor measurement According to etc..Wherein historical data includes the historical position of the skeleton of at least one human body and posture information, history image and goes through History measurement data.
Present image refers to that the image data for being ready for compensating for the dynamic of computation model parameter, current measurement data are Refer to the dynamic compensation measurement data being ready for for computation model parameter that inertial sensor obtains.Obtain capture apparatus shooting History image and present image obtain the historical measurement data and current measurement data of inertial sensor acquisition.Wherein history figure It seem the image first with current image shot, historical measurement data is prior to the collected measurement data of current measurement data.When Previous frame data include present image and current measurement data.
Step S203 estimates according to the static state that historical data and confidence level corresponding with historical data calculate "current" model parameter Meter updates historical data according to the static estimation of model parameter.
Specifically, the static estimation of model parameter refers to calculating of the calibration method using accumulation data for model parameter, The static estimation of "current" model parameter is the static estimation for the model parameter being just calculated.Static state refers to the ruler of skeleton Very little, luminous point installation site, the model parameters such as Installation posture of inertial sensor are under default human body attitude, not with human body The value that position and posture change.Confidence level refers to the whether reliable evaluation index of each frame data, according to the confidence level of historical data Historical data is screened, can be used for the static estimation of computation model parameter using the historical data after screening.Wherein, it counts When calculating the static estimation of "current" model parameter, using in the nearest preset time period apart from current data or the data of default frame Calculate the static estimation of "current" model parameter.The measurement data and image data in a period of time can be accumulated, or is being acquired After the measurement data and image data of the threshold value screening conditions of several frame data, according to the measurement data of accumulation and image data meter Calculate the static estimation of "current" model parameter.For each frame data all static estimation of computation model parameter, improve Computational efficiency, and can guarantee the stability and accuracy of parameter.
Step S204 calculates the confidence level of current frame data according to historical data.
Specifically, the confidence level of current frame data refers to the evaluation index of the reliability for measuring current data.According to Historical data obtains prediction model, is predicted using prediction model current frame data to obtain corresponding prediction data, according to Prediction data and the diversity factor of current frame data determine the reliability of current frame data.Such as when diversity factor be greater than it is pre-set can When the diversity factor threshold value of reliability, current frame data is insincere, otherwise credible.
In one embodiment, prediction data and the diversity factor of current frame data can be subtracted directly currently with prediction data Frame data obtain difference, can also be handled according to custom algorithm difference.
Step S205 is calculated current according to current frame data, the confidence level of current frame data and updated historical data The dynamic of model parameter compensates.
Specifically, the installation site and Installation posture that the dynamic error of model parameter refers to inertial sensor and luminous point are in people Due to the error that muscle deformation generates in body movement, which eliminates in the case where human body attitude returns to default posture."current" model ginseng Several dynamic compensation refers to that the model parameter dynamic being calculated using the historical data of current frame data and current frame data is missed The compensation of difference.The size of current human's bone, the peace of luminous point are calculated separately according to current frame data and updated historical data The Installation posture of holding position and inertial sensor is compensated according to the dynamic that the difference of the two data calculates "current" model parameter.
In one embodiment, the confidence level of current frame data can measure number with current according to the confidence level of present image According to confidence level determine.When present image it is credible when, the dynamic of "current" model parameter is calculated using present image and historical data State compensation；When current measurement data is credible, "current" model parameter can be calculated using current measurement data and historical data Dynamic compensates.
In one embodiment, dynamic compensation or the root of "current" model parameter are calculated according to present image and historical data When calculating the dynamic compensation of "current" model parameter according to current measurement data and historical data, historical data can be screened, The data for meeting screening conditions are filtered out from historical data, according to the historical data and current measurement data after screening or currently Image calculates the dynamic compensation of "current" model parameter.
Above-mentioned model parameter calibration method, by obtaining historical data, historical data includes the historical position of skeleton With the historical measurement data of posture information, the history image comprising luminous point and inertial sensor, luminous point and inertial sensor installation On human body, the current frame data of the current measurement data including present image and inertial sensor is obtained, according to historical data The static estimation that "current" model parameter is calculated with confidence level corresponding with historical data calculates current frame data according to historical data Confidence level, compensated according to the dynamic that the confidence level of current frame data and historical data calculate "current" model parameter, lift scheme The calibration accuracy of parameter.
In one embodiment, as shown in figure 3, step S201, further includes:
Step S301 obtains the initial makeup location of the original dimension and luminous point of skeleton relative to skeleton.
Step S302 obtains skeleton when static, and skeleton is in image and inertia sensing when different postures The measurement data of device.
Specifically, when the original dimension of skeleton refers to that people is in static, the human body that is obtained by measuring device measurement The size of bone.When luminous point refers to that human body is in static relative to the initial makeup location of skeleton, surveyed by measuring device The luminous point measured is mounted on the location information on human body, the bone that position letter can be nearest by the human body of luminous point and installation Relative position information indicate.Different postures refers to locating posture after human body makes different movements, such as erects upright, double Hand is downward vertically or arm is opened and the postures such as shoulder maintains an equal level.Obtain the measurement number of the image and inertial sensor under various postures According to.
Step S303 calculates the reference posture of the skeleton in different postures according to initial position message and image.
Specifically, human body is calculated according to the image data that the original dimension of skeleton and capture apparatus acquire and is in difference Reference posture when posture refers to reference to posture by carrying out feature extraction to the image of shooting, according to the characteristic information of extraction With the original dimension of skeleton be calculated human body different postures when skeleton location information.
Step S304 according to the measurement data of inertial sensor and obtains inertial coodinate system with reference to Attitude Calculation and luminous point is sat Mark the rotation relationship of system and the initial Installation posture of inertial sensor.
Specifically, the location information and posture that skeleton is calculated by the measurement data of inertial sensor, obtain human body The posture of skeleton under inertial coodinate system.Due to being the skeleton posture of the same human body in world coordinate system, therefore The posture of skeleton under inertial coodinate system can be transformed into the posture of skeleton under optical coordinate system by rotation mode.And The posture of skeleton is with reference to posture under optical coordinate system.By reference to the posture of skeleton under posture and inertial coodinate system The rotation relationship between inertial coodinate system and optical coordinate system is calculated.The initial Installation posture of inertial sensor refers to inertia The relative attitude of sensor and the skeleton installed.Inertial sensor is calculated according to the measurement data of inertial sensor Initial attitude under inertial coodinate system is obtained by the initial attitude under inertial sensor inertial coodinate system by way of rotation Initial attitude under inertial sensor optical coordinate system, based under reference posture and inertial sensor optical coordinate system The initial Installation posture of inertial sensor can be calculated in initial attitude.
Step S305 obtains static constraint condition, according to static constraint condition, the original dimension of skeleton, with reference to appearance The initial makeup location of the initial makeup location of state and luminous point corresponding with reference posture update luminous point.
Specifically, static constraint condition refers to, when human body is still in some posture, between each bone of human body and human body Existing relative attitude and positional relationship between environment.According to each with reference to the corresponding static constraint condition of posture, human body The original dimension of bone, with reference to the initial makeup location of posture and luminous point corresponding with reference posture, update each luminous point and people Relative position between body bone, the i.e. initial makeup location of luminous point.
Step S306, obtain human body execute deliberate action when inertial sensor measurement data and image, respectively as Dynamic measuring data and dynamic image.
Specifically, deliberate action refers to pre-set movement, and human body is walked according to pre-set movement.Obtain human body The measurement data of inertial sensor is as dynamic measuring data when walking, and the image of capture apparatus shooting is as dynamic image.
Step S307 obtains dynamic constrained condition, is updated according to dynamic constrained condition, dynamic measuring data and dynamic image Original dimension, the initial Installation posture of inertial sensor and the initial makeup location of luminous point of skeleton.
Specifically, dynamic constrained condition refers between human skeleton when human body makes deliberate action and human body and environment Between relative positional relationship.Skeleton is updated according to dynamic constrained condition, dynamic measuring data and dynamic image Original dimension, the initial Installation posture of inertial sensor and the initial makeup location of luminous point, until according to dynamic measuring data and The posture for the skeleton that the initial Installation posture of inertial sensor is calculated, and according to dynamic image and inertial sensor Difference between the posture for the skeleton that initial Installation posture is calculated meets default difference.
By the original dimension of updated skeleton, the initial installation of the initial Installation posture of inertial sensor and luminous point Position brings dynamic measuring data and dynamic image into, so that be calculated using mixed dynamic measuring data and dynamic image The position of skeleton and posture calculate position and the posture of skeleton according to dynamic measuring data, calculate above two meter The initial position of the skeleton of calculation method and the difference of posture are passed according to the original dimension of difference update skeleton, inertia The initial Installation posture of sensor and the initial makeup location of luminous point, until the initial bit of the skeleton of above two calculation method It sets and meets pre-set range with the difference of posture.
In one embodiment, as shown in figure 4, step S307, comprising:
Step S3071 calculates the dynamic posture of skeleton according to dynamic measuring data and dynamic image.
Step S3072 updates the initial installation appearance of rotation relationship and inertial sensor according to dynamic posture and constraint condition State.
Step S3073, by the initial peace of the original dimension of skeleton, the initial makeup location of luminous point and inertial sensor Fill posture and the data-optimized model of dynamic constrained condition entry, the original dimension of the skeleton after optimize, luminous point at the beginning of The initial Installation posture of beginning installation site and inertial sensor.
Specifically, according to the rotation relationship of inertial coodinate system and optical coordinate system by dynamic measuring data optics coordinate system It indicates.The dynamic posture of the skeleton under optical coordinate system is calculated according to dynamic image and dynamic measuring data.Wherein, it moves State posture refers to the posture of people when walking.Inertial coodinate system and optical coordinate system are updated according to dynamic posture and constraint condition Rotation relationship calculates the initial Installation posture of inertial sensor according to updated rotation relationship.Since human body is when calibrating It is different to wait the accuracy that calibration actions are done due to everyone, causes the positive direction of people and pitch angle to have deviation, when human body is towards just When front is walked, capturing result will appear the deviation of positive direction and pitch angle, be calculated based on dynamic measuring data and dynamic image Pitch angle in position and attitude reduction result and just towards deviation, updates the rotation relationship of optical coordinate system and inertial coodinate system, The pitch angle in position and attitude reduction result that is calculated based on dynamic measuring data and dynamic image and just towards deviation, is updated and is used to The Installation posture of property sensor.By the initial of the original dimension of skeleton, the initial makeup location of luminous point and inertial sensor Installation posture and the data-optimized model of dynamic constrained condition entry, the original dimension of the skeleton after being optimized, luminous point The initial Installation posture of initial makeup location and inertial sensor.Data-optimized model is pre-established for optimizing data Model, the model optimize measurement data by dynamic constrained condition, until meeting the optimal conditions of Optimized model, obtain The initial Installation posture of the original dimension of skeleton after optimization, the initial makeup location of luminous point and inertial sensor.
In one embodiment, as shown in figure 5, step S203, comprising:
Step S2031 obtains the historical measurement data for meeting the first preset threshold as first from historical measurement data Data.
Step S2032 obtains the second data of conduct for meeting the image of the second preset threshold from history image.
Step S2033, by the initial installation of the first data, the second data, the original dimension of skeleton, inertial sensor Posture, the initial makeup location of luminous point and static constraint condition entry static optimization model, skeleton after being optimized Original dimension, the initial Installation posture of inertial sensor and the initial makeup location of luminous point.
Step S2034, according to the original dimension of the skeleton after optimization, the initial Installation posture and light of inertial sensor The static estimation of "current" model parameter is calculated in the initial makeup location of point, is updated according to the static estimation of "current" model parameter Historical data.
Specifically, the first preset threshold be for measuring the whether believable critical value of the measurement data of inertial sensor, when It when greater than the first preset threshold, indicates credible, the history that confidence level is greater than the first preset threshold is obtained from historical measurement data Measurement data, as the first data.Second preset threshold is all believable critical value for measuring image data, from history image Middle the second data of conduct for obtaining confidence level and being greater than the image of the second preset threshold.Static optimization model is joined for Optimized model The model of several static estimations, by the first data, the second data, the original dimension of skeleton, the initial peace of inertial sensor The initial makeup location for filling posture and luminous point inputs static optimization model, passes through the initial of static optimization model optimization skeleton Size, the initial Installation posture of inertial sensor and the initial makeup location of luminous point, until meeting the static state of static optimization model Optimal conditions, the original dimension of the skeleton after being optimized, the initial Installation posture of inertial sensor and luminous point it is initial Installation site.
In one embodiment, as shown in fig. 6, step S204, comprising:
Step S2041 establishes prediction model by historical data.
The defeated prediction model of current frame data is obtained the prediction of present frame skeleton position and posture by step S2042 Value.
Step S2043, the present frame skeleton position being calculated according to predicted value and based on current frame data and appearance The diversity factor of the end value of state calculates the confidence level of current frame data.
Specifically, the historical frames Data Data of current frame data refers to an at least frame number nearest apart from current frame data According to.When obtaining history frame data, data screening condition can be set, screen and obtain from the data frame before history frame data History frame data.Prediction model is the model established according to historical data, which can be according to historical data prediction pair Current frame data input prediction model is obtained the predicted value of present frame skeleton position and posture by the historical data answered, meter The diversity factor of the end value of present frame skeleton position and posture calculating predicted value and being calculated based on current frame data, when When diversity factor is spent greater than preset forecasted variances, current frame data is insincere, otherwise credible.It is carried out by history frame data pre- It surveys, current frame data confidence level is determined according to the difference of preset value and current frame data, guarantees the reliability of data, to improve The accuracy that data calculate.
In one embodiment, as shown in fig. 7, step S205, comprising:
Step S2051 obtains the first preset threshold and the second preset threshold.
Step S2052, when the confidence level of current measurement data does not meet the first preset threshold, the confidence level of present image is full The second preset threshold of foot, the posture of skeleton is calculated according to present image, measures number according to present image, updated history It is compensated according to the dynamic of the Attitude Calculation inertial sensor Installation posture with skeleton.
Step S2053, when the confidence level of current measurement data meets the first preset threshold, the confidence level of present image is less than The second preset threshold of foot calculates the posture of skeleton, the relative position letter of luminous point and skeleton according to current measurement data Breath calculates the dynamic compensation of luminous point installation site.
Specifically, the first preset threshold and the first preset threshold of step S2031 are the same preset threshold, and second is default Threshold value and the second preset threshold of step S2031 are the same preset threshold.When the confidence level of current measurement data does not meet One preset threshold, i.e., the confidence level of current measurement data is less than the first preset threshold.It is pre- that the confidence level of present image meets second If threshold value, i.e., the confidence level of preceding image is greater than the second preset threshold.Since current measurement data is insincere, present image is credible, Therefore the posture of present image calculating skeleton is used, in the posture according to present image, historical measurement data and skeleton Calculate the dynamic compensation of inertial sensor Installation posture.When confidence level the first preset threshold of satisfaction of current measurement data, that is, work as The confidence level of preceding measurement data is greater than the first preset threshold, and the confidence level of present image does not meet the second preset threshold, current to scheme The confidence level of picture is less than the second preset threshold.Since current measurement data is credible, when present image is insincere, according to current measurement Data calculate the posture of skeleton and the relative position information of luminous point and skeleton.It is calculated according to current measurement data Posture, the dynamic of the relative position information calculating luminous point installation site of luminous point and skeleton of the skeleton arrived compensate.Root The position of skeleton, inertia are updated according to the dynamic compensation of luminous point installation site and the dynamic compensation of inertial sensor Installation posture The posture of sensor and the location information of luminous point obtain the current location of skeleton, the current pose of inertial sensor and light The current location information of point.
In a specific embodiment, above-mentioned model parameter calibration method, comprising:
In mixing motion capture, pass through fusion calculation skeleton position to inertial sensor and luminous point data and appearance State.As shown in figure 8, providing an application scenario diagram, user is equipped with the multiple inertial sensors of inertia and multiple luminous points with it. Wherein each luminous point is separately mounted to the left and right instep of human body, left and right wrist top, waist, back and head.
Data acquisition: measure human body dimension data and each luminous point human body installation site.Measure the main portion of human body The geometric dimension of position, according to the initial length for measuring size all bones in calculating human cinology's model.It is opposite to measure luminous point In three-dimensional distance (offset of the luminous point relative to ankle central point in front and rear, left and right, up and down direction on such as foot of human body feature point Amount, offset etc. of the forearm luminous point relative to wrist), calculate initial makeup location of the luminous point in skeleton coordinate system.
Static calibration: acquisition static data, as shown in figure 9, acquisition human body is in A posture 002, human body is in S posture 004 and human body processing T posture 006 when inertial sensor measurement data and optical measurement data.Wherein A posture refers to people Body whole body standing, vertically downward, S posture refers to that both legs are squatted down to both arms, lifts before both arms are horizontal, T posture, which refers to, lifts torso up erect It stands, both arms level is lifted to body two sides, and the palm of the hand is downward.Wherein the measurement data of inertial sensor includes that inertial sensor is opposite In the posture of inertial coodinate system, optical measurement data includes position of the luminous point in optical coordinate system, when luminous point be light rigid body then Measure the position relative to the posture and its central point of optical coordinate system in optical coordinate system of light rigid body.Pass through optical data When calculating human body is in A posture, S posture and T posture, people is calculated according to the position of hand luminous point in the position of hand luminous point Around body, upper and lower three directions optical coordinate system expression.Human body processing is calculated by inertial sensor measurement data When A posture, S posture and T posture, the relative attitude of inertial sensor and arm on arm, according to inertial sensor and arm Relative attitude be calculated human body around, expression of upper and lower three directions in inertial coodinate system, according to inertial coordinate The relationship of system and human body, obtains human body.By human body in the expression of optical coordinate system and inertial coodinate system, optics seat is calculated The rotation relationship of mark system and inertial coodinate system, the rotation relationship can be indicated with the data of four dimensions.
Assuming that in A posture and T posture the posture of each bone of human body true value can by around human body, up and down Expression of three axial directions in optical coordinate system is calculated, and posture is referred to as bone, in conjunction with human body in A posture and T posture The measurement posture of each inertial sensor in the reference posture and A posture and T posture of each bone, is calculated all inertia and passes Installation posture initial estimate of the sensor relative to installed bone.
Update luminous point installation site initial value.Under static calibration poses, there are following correlativities: A for partes corporis humani position When posture, double-legged ankle spacing is identical as hip breadth, and when T posture, both hands wrist height is identical as shoulder height, when S posture, both hands wrist Horizontal distance is identical as shoulder breadth.Based on the above relativeness, in conjunction with the size of skeleton, human body in A posture, S posture and T appearance The reference posture of each bone of human body, the measurement position of each luminous point solve the position for the bone that luminous point is installed relative to place when state It sets, as updated luminous point installation site initial value.
Dynamic calibration: acquisition walking calibration actions data: designing a set of dynamic calibration movement, such as to nature immediately ahead of human body Walking and substantially front and back swing arm, so that the relative attitude of each inertial sensor on human body generates larger amplitude variation, while as far as possible Guarantee that limb muscle deformation is unobvious in dynamic calibration action process, so that the dynamic for not introducing inertial sensor Installation posture is missed Difference.The measurement data and optical measurement data of inertia sensing in acquisition walking calibration process.
Optimize the Installation posture value of inertial sensor: when doing calibration actions, everyone does the amplitude of calibration actions to human body Etc. different, there are deviations for the positive direction and pitch angle for leading to people, and when user walks towards front, capturing result will appear positive court To the deviation with pitch angle, in the installation site posture reduction result by the inertial sensor of walking calibration actions data calculating Pitch angle and just towards deviation, update the rotation relationship of optical coordinate system and inertial coodinate system and the peace of all inertial sensors Fill posture initial value.
The Installation posture of the size of combined optimization skeleton, the installation site of luminous point and inertial sensor: it is based on standard Data and revised inertial sensor Installation posture, by the size of skeleton, the installation site of luminous point and inertial sensor Installation posture be described as two optimization problems about upper limb and lower limb, using gradient descent method, optimization light The installation site of point, the crucial length of bone and the Installation posture of inertial sensor, so that based on luminous point installation inverse kinematics Gap between the posture for each bone calculated and the bone posture calculated based on inertial sensor measurement data is minimum.
By the Installation posture band reversion of the size of the skeleton after optimization, the installation site of luminous point and inertial sensor Calibration actions data are walked, position and posture using mixing motion capture data calculating fusion skeleton pass through calculating Obtained position and posture is compared with the posture being calculated according to inertia measurement, is examined the people after optimization after optimizing The reliability of the Installation posture of the size of body bone, the installation site of luminous point and inertial sensor.
When the variance of the error of the two is larger, then reject what part inertia measurement data in walking calibration actions data calculated The incredible data in position that posture or optical measurement calculate, the again installation of the size of optimization skeleton, luminous point The Installation posture of position and inertial sensor obtains size, the light of skeleton until error variance converges in preset range The installation site of point and the Installation posture of inertial sensor.
On-line Estimation module.Calibration process can obtain the size of skeleton, the installation site of luminous point and inertia sensing The initial estimation of the Installation posture of device, so that the installation appearance of the size of skeleton, the installation site of luminous point and inertial sensor State realizes that inertia is consistent with two model results of optics for calibration data.The length of obtained bone is calibrated in motion capture It is no longer changed in the process, but during motion capture, sensor mounting location and posture may become because of muscle Shape or wearing shift and generate variation, generate the static estimation of the dynamic compensation and model parameter of model parameter.In motion capture In the process by accumulation data, the reasonable assumption based on measurement and constraint solves instant skeleton using the method for optimization The optimal solution of the Installation posture of size, the installation site of luminous point and inertial sensor, the static estimation and mould of correction model parameter The dynamic of shape parameter compensates.Specific makeover process is as follows:
Each frame data are calculated, current human's posture and muscular soft tissues effect priori knowledge is based on, calculates all inertia The confidence level of sensor and optical measurement.Wherein human body attitude is calculated by current frame data, human muscle's soft tissue effect Priori knowledge is calculated by historical data, and present frame measurement data input human muscle's soft tissue effect model is obtained To human body gesture prediction value.By comparing current human's posture and its difference of predicted value, calculate present frame measurement data can Reliability.Reasonable threshold value is set, when inertial sensor measures or optical measurement confidence level is greater than threshold value, it is believed that the inertia sensing Device/bone posture or optics/position measure credible.The measurement number of the inertial sensor of human upper limb and lower limb is investigated respectively According to the confidence level with optical measurement data, if there are the measurement data of inertial sensor or optics to survey in the bone that lower limb are included Measure the incredible situation of data, then it is assumed that the measurement data of the lower limb is insincere.Similarly handle the measurement data of upper limb.
The data of the static estimation On-line Estimation of parameter model parameter are accumulated.For each frame data, according to human upper limb It is credible to be added into upper limb respectively by the measurement data for meeting preset threshold with the confidence level of lower limb measurement measurement data for the frame data Data and lower limb trust data.For example, certain frame data only has the inertia measurement of left thigh insincere, then the frame data are added into Upper limb trust data.The exercise data of a period of time is acquired, the confidence level based on measurement data, obtain 2 groups of data: upper limb is credible Data and lower limb trust data.
The On-line Estimation of the static estimation of the Installation posture model parameter of the installation site and inertial sensor of luminous point.For The lower limb trust data for accumulating a period of time, by the peace of the installation site and lower limb inertial sensor of trunk (waist) and foot's luminous point The estimation of dress posture is described as an optimization problem, using gradient descent method, data-optimized according to walking calibration actions The Installation posture of the installation site and lower limb inertial sensor that obtain foot's luminous point is nearby searched for optimal solution and is solved, so that being based on body The lower limb skeletons posture of dry (waist) and foot's light spot position the computation of inverse- kinematics measures Attitude Calculation with based on inertial sensor Gap between lower limb skeletons posture is minimum, and calibration actions data are walked in the lower limb skeletons attitude tape reversion after optimization, utilizes Position and the posture for mixing motion capture data calculating fusion human body, by walking, human body is calculated in calibration actions data Position and the posture that is calculated with the measurement data according to inertial sensor of posture compare, determined according to comparing result The reliability of parameter estimation result.If the variance of resultant error is big, the confidence level of the measurement data of inertial sensor is improved Judgment threshold further screens lower limb inertia trust data, again the installation position of optimization trunk (waist) and foot's luminous point It sets and the Installation posture of lower limb inertial sensor, until error variance converges in preset range；
Upper limb trust data is similarly handled, the installation site and upper limb inertia for obtaining trunk (back and head) and upper limb luminous point pass The optimal estimation of the Installation posture of sensor.The new estimated value of the Installation posture of the installation site and inertial sensor that obtain luminous point it Afterwards, calibration result, the static estimation of the model parameter of corrected parameter are updated.
The On-line Estimation that the dynamic of luminous point installation site and inertial sensor Installation posture model parameter compensates.Consider current Frame lower limb measurement data reliability assessment is as a result, if the measurement data of present frame lower limb inertial sensor is insincere, and optics is surveyed It is credible to measure data, obtains human body lower limbs using soft tissue effect priori knowledge and n- quasi- kinematic data calculating fusion Bone posture, and then the posture for combining the measurement data of inertial sensor to calculate, estimate the mould of the Installation posture of inertial sensor The dynamic of shape parameter compensates, the size of real-time update human body lower limbs bone, the installation site of lower limb luminous point and lower limb inertia sensing Installation posture.
Consider present frame lower limb measurement data reliability assessment as a result, if the measurement data of present frame lower limb inertial sensor It is credible, and optical measurement data is insincere, utilizes soft tissue effect priori knowledge and n- quasi- kinematic data calculating fusion The position at human body acra and optics installation point is obtained, estimates the dynamic of the model parameter of trunk (waist) and foot's luminous point installation site State compensation, the Installation posture of the size of real-time update human body lower limbs bone, the installation site of lower limb luminous point and lower limb inertia sensing.
Similarly, according to present frame upper limb measurement data reliability assessment as a result, estimating trunk (back and head) and upper limb in turn The compensation of the dynamic of luminous point installation site and the model parameter of upper limb inertial sensor Installation posture, the ruler of real-time body's upper limb bone The Installation posture of very little, upper limb luminous point installation site and upper limb inertia sensing.The dynamic of the model parameter of estimation is compensated and worked as Soft tissue effect data collection is added in the posture of preceding human body, for updating soft tissue effect knowledge.
Fig. 2-7 is the flow diagram to calibrate for error in one embodiment.Although should be understood that the process of Fig. 2-7 Each step in figure is successively shown according to the instruction of arrow, but these steps are not the inevitable sequence indicated according to arrow Successively execute.Unless expressly stating otherwise herein, there is no stringent sequences to limit for the execution of these steps, these steps can To execute in other order.Moreover, at least part step in Fig. 2-7 may include multiple sub-steps or multiple ranks Section, these sub-steps or stage are not necessarily to execute completion in synchronization, but can execute at different times, this The execution sequence in a little step perhaps stage be also not necessarily successively carry out but can be with other steps or other steps Sub-step or at least part in stage execute in turn or alternately.
In one embodiment, as shown in Figure 10, a model parameter calibrating installation 200 is provided, comprising:
Historical data obtains module 201, for obtaining historical data, historical data include skeleton historical position and The historical measurement data of posture information, the history image comprising luminous point and inertial sensor, luminous point and inertial sensor are mounted on On human body.
Current data obtains module 202, for obtaining the current measurement data including present image and inertial sensor Current frame data.
The static estimation computing module 203 of model parameter, for according to historical data and corresponding with historical data credible Degree calculates the static estimation of "current" model parameter, updates historical data partially according to current static.
Confidence level computing module 204, for calculating the confidence level of current frame data according to historical data.
The dynamic compensation calculation module 205 of model parameter, for according to the confidence level of current frame data, current frame data and Updated historical data calculates the dynamic compensation of "current" model parameter.
In one embodiment, as shown in figure 11, above-mentioned model parameter calibrating installation 200 further include:
Primary data obtains module 301, and the original dimension and luminous point for obtaining skeleton are relative to skeleton Initial makeup location obtains skeleton when static, the image and inertial sensor when skeleton is in different postures Measurement data.
With reference to Attitude Calculation module 302, for calculating the human body bone in different postures according to initial position message and image The reference posture of bone.
Data processing module 303 obtains inertia seat for the measurement data according to inertial sensor and with reference to Attitude Calculation Mark system and the rotation relationship of luminous point coordinate system and the initial Installation posture of inertial sensor；
Luminous point optimization module 304, for obtaining static constraint condition, according to static constraint condition, skeleton it is initial Size, the initial makeup location that luminous point is updated with reference to the initial makeup location of posture and luminous point corresponding with reference posture.
Dynamic data acquisition module 305, for obtain human body execute deliberate action when inertial sensor measurement data And image, respectively as dynamic measuring data and dynamic image.
Data update module 306, for obtaining dynamic constrained condition, according to dynamic constrained condition, dynamic measuring data and Original dimension, the initial Installation posture of inertial sensor and the initial makeup location of luminous point of dynamic image update skeleton.
In one embodiment, as shown in figure 12, data update module 306, comprising:
Dynamic attitude calculation unit 3061, for calculating the dynamic of skeleton according to dynamic measuring data and dynamic image Posture.
Data updating unit 3062, for updating rotation relationship and inertial sensor according to dynamic posture and constraint condition Initial Installation posture.
Data-optimized unit 3063, for passing the initial makeup location of the original dimension of skeleton, luminous point and inertia The initial Installation posture and the data-optimized model of dynamic constrained condition entry of sensor, the initial ruler of the skeleton after being optimized The initial Installation posture of very little, luminous point initial makeup location and inertial sensor.
In one embodiment, as shown in figure 13, the static estimation computing module 203 of model parameter, comprising:
Data screening unit 2031, for obtaining the history measurement number for meeting the first preset threshold from historical measurement data According to as the first data, the second data of conduct for meeting the image of the second preset threshold are obtained from history image.
Static data optimizes unit 2032, for by the first data, the second data, the original dimension of skeleton, inertia The initial Installation posture of sensor, the initial makeup location of luminous point and static constraint condition entry static optimization model, obtain excellent Original dimension, the initial Installation posture of inertial sensor and the initial makeup location of luminous point of skeleton after change.
The static estimation computing unit 2033 of model parameter, for according to the original dimension of the skeleton after optimization, used The static estimation of "current" model parameter is calculated in the initial Installation posture of property sensor and the initial makeup location of luminous point, according to The static estimation of "current" model parameter updates historical data.
In one embodiment, as shown in figure 14, confidence level computing module 204, comprising:
Historical frames acquiring unit 2041, for obtaining the history frame data of current frame data.
Predicting unit 2042, for obtaining the predicted value of current frame data for history frame data input prediction model.
Confidence level computing unit 2043, for calculating current frame data according to the diversity factor of predicted value and current frame data Confidence level.
In one embodiment, as shown in figure 15, the dynamic compensation calculation module 205 of model parameter, comprising:
Threshold value acquiring unit 2051, for obtaining the first preset threshold and the second preset threshold.
Sensor bias computing unit 2052 does not meet the first preset threshold for the confidence level when current measurement data, The confidence level of present image meets the second preset threshold, and the posture of skeleton is calculated according to present image, according to present image, The dynamic of the Attitude Calculation inertial sensor Installation posture of updated historical measurement data and skeleton compensates.
Luminous point deviation computing unit 2053 meets the first preset threshold for the confidence level when current measurement data, currently The confidence level of image does not meet the second preset threshold, and posture, luminous point and the human body of skeleton are calculated according to current measurement data The relative position information of bone, according to the relative position information of the posture of skeleton, luminous point and skeleton calculate luminous point with The dynamic of the model parameter of the relative position of skeleton compensates.
Figure 16 shows the internal structure chart of computer equipment in one embodiment.The computer equipment specifically can be figure Terminal 110 (or server 120) in 1.As shown in figure 16, it includes passing through system which, which includes the computer equipment, Processor, memory, network interface, input unit and the display screen of bus connection.Wherein, memory includes non-volatile memories Medium and built-in storage.The non-volatile memory medium of the computer equipment is stored with operating system, can also be stored with computer Program when the computer program is executed by processor, may make processor realization to calibrate for error.It can also be stored in the built-in storage There is computer program, when which is executed by processor, processor execution may make to calibrate for error.Computer equipment Display screen can be liquid crystal display or electric ink display screen, and the input unit of computer equipment can be to be covered on display screen The touch layer of lid is also possible to the key being arranged on computer equipment shell, trace ball or Trackpad, can also be external key Disk, Trackpad or mouse etc..
It will be understood by those skilled in the art that structure shown in Figure 16, only part relevant to application scheme The block diagram of structure, does not constitute the restriction for the computer equipment being applied thereon to application scheme, and specific computer is set Standby may include perhaps combining certain components or with different component layouts than more or fewer components as shown in the figure.
In one embodiment, model parameter calibrating installation provided by the present application can be implemented as a kind of computer program Form, computer program can be run in computer equipment as shown in figure 16.Group can be stored in the memory of computer equipment At each program module of the model parameter calibrating installation, for example, A module shown in Fig. 10, B module and C module.Each program The computer program of module composition makes processor execute calibrating for error for each embodiment of the application described in this specification In step.
For example, computer equipment shown in Figure 16 can pass through the history in model parameter calibrating installation as shown in Figure 10 Data acquisition module 201, which executes, obtains historical data, and historical data includes the historical position of skeleton and posture information, includes The history image of luminous point and the historical measurement data of inertial sensor, luminous point and inertial sensor are mounted on human body.Computer Equipment can be obtained module 202 by current data and execute current measurement data of the acquisition including present image and inertial sensor Current frame data.Computer equipment can by the static estimation computing module 203 of model parameter execute according to historical data and with The corresponding confidence level of historical data calculates the static estimation of "current" model parameter, is updated according to the static estimation of "current" model parameter Historical data.Computer equipment can by confidence level computing module 204 execute according to historical data calculate current frame data can Reliability.Computer equipment can be executed by the dynamic compensation calculation module 205 of model parameter according to current frame data, current frame number According to confidence level and updated historical data calculate "current" model parameter dynamic compensation.
In one embodiment, a kind of computer equipment is provided, including memory, processor and storage are on a memory And the computer program that can be run on a processor, processor performs the steps of when executing computer program obtains history number According to historical data includes the historical position and posture information, history image and inertial sensor comprising luminous point of skeleton Historical measurement data, luminous point and inertial sensor are mounted on human body；Obtaining includes the current of present image and inertial sensor The current frame data of measurement data calculates the quiet of "current" model parameter according to historical data and confidence level corresponding with historical data State estimation updates historical data according to the static estimation of model parameter, the confidence level of current frame data is calculated according to historical data, It is mended according to the dynamic that current frame data, the confidence level of current frame data and updated historical data calculate "current" model parameter It repays.
In one embodiment, before obtaining historical data, processor also performs the steps of when executing computer program The initial makeup location of the original dimension and luminous point of skeleton relative to skeleton is obtained, obtains skeleton static When, the measurement data of image and inertial sensor when skeleton is in different postures, according to initial position message and image The reference posture for calculating the skeleton in different postures is obtained according to the measurement data of inertial sensor and with reference to Attitude Calculation Inertial coodinate system and the rotation relationship of luminous point coordinate system and the initial Installation posture of inertial sensor obtain static constraint condition, According to static constraint condition, the original dimension of skeleton, with reference to the initial installation of posture and luminous point corresponding with reference posture The initial makeup location of location updating luminous point, obtain human body execute deliberate action when inertial sensor measurement data and figure Picture obtains dynamic constrained condition respectively as dynamic measuring data and dynamic image, is measured according to dynamic constrained condition, dynamic Data and dynamic image update the original dimension of skeleton, inertial sensor initial Installation posture and luminous point initial installation Position.
In one embodiment, skeleton is updated according to dynamic constrained condition, dynamic measuring data and dynamic image Original dimension, the initial Installation posture of inertial sensor and the initial makeup location of luminous point, comprising: according to dynamic measuring data and Dynamic image calculates the dynamic posture of skeleton, updates rotation relationship and inertial sensor according to dynamic posture and constraint condition Initial Installation posture, by the initial installation of the original dimension of skeleton, the initial makeup location of luminous point and inertial sensor Posture and the data-optimized model of dynamic constrained condition entry, the original dimension of the skeleton after being optimized, luminous point it is initial The initial Installation posture of installation site and inertial sensor.
In one embodiment, "current" model parameter is calculated according to historical data and confidence level corresponding with historical data Static estimation updates historical data according to the static estimation of "current" model parameter, comprising: obtain and meet from historical measurement data The historical measurement data of first preset threshold obtains the image for meeting the second preset threshold as the first data from history image The second data of conduct, by the first data, the second data, the original dimension of skeleton, the initial installation appearance of inertial sensor State, the initial makeup location of luminous point and static constraint condition entry static optimization model, skeleton after being optimized just Beginning size, the initial Installation posture of inertial sensor and the initial makeup location of luminous point, according to the first of the skeleton after optimization The static state of "current" model parameter is calculated in the initial makeup location of beginning size, the initial Installation posture of inertial sensor and luminous point Estimation updates historical data according to the static estimation of "current" model parameter.
In one embodiment, the confidence level of current frame data is calculated according to historical data, comprising: obtain current frame data History frame data history frame data input prediction model is obtained into the predicted value of current frame data, according to predicted value and current The diversity factor of frame data calculates the confidence level of current frame data.
In one embodiment, the dynamic compensation of "current" model parameter includes the model parameter of inertial sensor Installation posture Dynamic compensation and luminous point and skeleton relative position information model parameter dynamic compensation, according to current frame data, The confidence level of current frame data and updated historical data calculate the dynamic compensation of "current" model parameter, comprising: obtain first Preset threshold and the second preset threshold, when the confidence level of current measurement data does not meet the first preset threshold, present image can Reliability meets the second preset threshold, the posture of skeleton is calculated according to present image, according to present image, updated history The dynamic of the model parameter of the Attitude Calculation inertial sensor Installation posture of measurement data and skeleton compensates, when current measurement The confidence level of data meets the first preset threshold, and the confidence level of present image does not meet the second preset threshold, according to current measurement Data calculate the relative position information of the posture of skeleton, luminous point and skeleton, posture, luminous point and the human body of skeleton The dynamic that the relative position information of bone calculates the model parameter of the relative position of luminous point and skeleton compensates.
In one embodiment, a kind of computer readable storage medium is provided, computer program is stored thereon with, is calculated Machine program performs the steps of acquisition historical data when being executed by processor, historical data includes the historical position of skeleton With the historical measurement data of posture information, the history image comprising luminous point and inertial sensor, luminous point and inertial sensor installation On human body, the current frame data of the current measurement data including present image and inertial sensor is obtained, according to historical data The static estimation that "current" model parameter is calculated with confidence level corresponding with historical data, according to the static estimation of "current" model parameter Update historical data, according to historical data calculate current frame data confidence level, according to current frame data, current frame data can Reliability and updated historical data calculate the dynamic compensation of "current" model parameter.
In one embodiment, before obtaining historical data, processor also performs the steps of when executing computer program The initial makeup location of the original dimension and luminous point of skeleton relative to skeleton is obtained, obtains skeleton static When, the measurement data of image and inertial sensor when skeleton is in different postures, according to initial position message and image The reference posture for calculating the skeleton in different postures is obtained according to the measurement data of inertial sensor and with reference to Attitude Calculation Inertial coodinate system and the rotation relationship of luminous point coordinate system and the initial Installation posture of inertial sensor obtain static constraint condition, According to static constraint condition, the original dimension of skeleton, with reference to the initial installation of posture and luminous point corresponding with reference posture The initial makeup location of location updating luminous point, obtain human body execute deliberate action when inertial sensor measurement data and figure Picture obtains dynamic constrained condition respectively as dynamic measuring data and dynamic image, is measured according to dynamic constrained condition, dynamic Data and dynamic image update the original dimension of skeleton, inertial sensor initial Installation posture and luminous point initial installation Position.
In one embodiment, skeleton is updated according to dynamic constrained condition, dynamic measuring data and dynamic image Original dimension, the initial Installation posture of inertial sensor and the initial makeup location of luminous point, comprising: according to dynamic measuring data and Dynamic image calculates the dynamic posture of skeleton, updates rotation relationship and inertial sensor according to dynamic posture and constraint condition Initial Installation posture, by the initial installation of the original dimension of skeleton, the initial makeup location of luminous point and inertial sensor Posture and the data-optimized model of dynamic constrained condition entry, the original dimension of the skeleton after being optimized, luminous point it is initial The initial Installation posture of installation site and inertial sensor.
In one embodiment, "current" model parameter is calculated according to historical data and confidence level corresponding with historical data Static estimation updates historical data according to the static estimation of "current" model parameter, comprising: obtain and meet from historical measurement data The historical measurement data of first preset threshold obtains the image for meeting the second preset threshold as the first data from history image The second data of conduct, by the first data, the second data, the original dimension of skeleton, the initial installation appearance of inertial sensor State, the initial makeup location of luminous point and static constraint condition entry static optimization model, skeleton after being optimized just Beginning size, the initial Installation posture of inertial sensor and the initial makeup location of luminous point, according to the first of the skeleton after optimization The static state of "current" model parameter is calculated in the initial makeup location of beginning size, the initial Installation posture of inertial sensor and luminous point Estimation updates historical data according to the static estimation of "current" model parameter,.
In one embodiment, the confidence level of current frame data is calculated according to historical data, comprising: obtain current frame data History frame data history frame data input prediction model is obtained into the predicted value of current frame data, according to predicted value and current The diversity factor of frame data calculates the confidence level of current frame data.
In one embodiment, the dynamic compensation of "current" model parameter includes the model parameter of inertial sensor Installation posture Dynamic compensation and luminous point and skeleton relative position information model parameter dynamic compensation, according to current frame data, The confidence level and historical data of current frame data calculate the dynamic compensation of "current" model parameter, comprising: obtain the first preset threshold With the second preset threshold, when the confidence level of current measurement data does not meet the first preset threshold, the confidence level of present image meets Second preset threshold calculates the posture of skeleton according to present image, according to present image, updated historical measurement data Compensated with the dynamic of the model parameter of the Attitude Calculation inertial sensor Installation posture of skeleton, when current measurement data can Reliability meets the first preset threshold, and the confidence level of present image does not meet the second preset threshold, calculated according to current measurement data The relative position information of the posture of skeleton, luminous point and skeleton, posture, the phase of luminous point and skeleton of skeleton Dynamic compensation to the model parameter of the relative position of positional information calculation luminous point and skeleton.
Those of ordinary skill in the art will appreciate that realizing all or part of the process in above-described embodiment method, being can be with Relevant hardware is instructed to complete by computer program, the program can be stored in a non-volatile computer and can be read In storage medium, the program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.Wherein, provided herein Each embodiment used in any reference to memory, storage, database or other media, may each comprise non-volatile And/or volatile memory.Nonvolatile memory may include that read-only memory (ROM), programming ROM (PROM), electricity can be compiled Journey ROM (EPROM), electrically erasable ROM (EEPROM) or flash memory.Volatile memory may include random access memory (RAM) or external cache.By way of illustration and not limitation, RAM is available in many forms, such as static state RAM (SRAM), dynamic ram (DRAM), synchronous dram (SDRAM), double data rate sdram (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronization link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) directly RAM (RDRAM), straight Connect memory bus dynamic ram (DRDRAM) and memory bus dynamic ram (RDRAM) etc..
It should be noted that, in this document, the relational terms of such as " first " and " second " or the like are used merely to one A entity or operation with another entity or operate distinguish, without necessarily requiring or implying these entities or operation it Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant are intended to Cover non-exclusive inclusion, so that the process, method, article or equipment for including a series of elements not only includes those Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or setting Standby intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that There is also other identical elements in the process, method, article or apparatus that includes the element.
The above is only a specific embodiment of the invention, is made skilled artisans appreciate that or realizing this hair It is bright.Various modifications to these embodiments will be apparent to one skilled in the art, as defined herein General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, of the invention It is not intended to be limited to the embodiments shown herein, and is to fit to and applied principle and features of novelty phase one herein The widest scope of cause.

Claims (9)

1. a kind of model parameter calibration method, which comprises
Historical data is obtained, the historical data includes the historical position and posture information, the history comprising luminous point of skeleton The historical measurement data of image and inertial sensor, the luminous point and the inertial sensor are mounted on human body；
Obtain the current frame data of the current measurement data including present image and inertial sensor；
The static estimation of "current" model parameter, root are calculated according to the historical data and confidence level corresponding with the historical data The historical data is updated according to the static estimation of the model parameter；
The confidence level of the current frame data is calculated according to the historical data；
"current" model ginseng is calculated according to the current frame data, the confidence level of the current frame data and updated historical data Several dynamic compensation；
The confidence level that the current frame data is calculated according to the historical data, comprising:
Prediction model is established according to the historical data；
By the current frame data input prediction model, the predicted value of present frame skeleton position and posture is obtained；
According to the knot of the predicted value and the present frame skeleton position and posture that are calculated based on the current frame data The diversity factor of fruit value calculates the confidence level of the current frame data.
2. the method according to claim 1, wherein before the acquisition historical data, further includes:
Obtain the initial makeup location of the original dimension and the luminous point of the skeleton relative to the skeleton；
The skeleton is obtained when static, the skeleton is in image and the inertial sensor when different postures Measurement data；
The reference posture of the skeleton in different postures is calculated according to the initial makeup location and described image；
Inertial coodinate system and luminous point coordinate system are obtained according to the measurement data of the inertial sensor and the Attitude Calculation that refers to Rotation relationship and the inertial sensor initial Installation posture；
Obtain static constraint condition, according to the static constraint condition, the original dimension of the skeleton, it is described refer to posture With with the initial makeup location of luminous point described in the updating location information with reference to the corresponding luminous point of posture；
The measurement data and image for obtaining the inertial sensor when human body executes deliberate action, are surveyed respectively as dynamic Measure data and dynamic image；
Dynamic constrained condition is obtained, is updated according to the dynamic constrained condition, the dynamic measuring data and the dynamic image The initial makeup location of the original dimension of the skeleton, the initial Installation posture of the inertial sensor and the luminous point.
3. according to the method described in claim 2, it is characterized in that, described survey according to the dynamic constrained condition, the dynamic Amount data and the dynamic image update the original dimension of the skeleton, the inertial sensor initial Installation posture and The initial makeup location of the luminous point, comprising:
The dynamic posture of the skeleton is calculated according to the dynamic measuring data and the dynamic image；
The initial installation of the rotation relationship and the inertial sensor is updated according to the dynamic posture and the constraint condition Posture；
By the original dimension of the skeleton, the initial installation of the initial makeup location of the luminous point and the inertial sensor Posture and the data-optimized model of dynamic constrained condition entry, the original dimension of the skeleton after being optimized, institute State the initial makeup location of luminous point and the initial Installation posture of the inertial sensor.
4. according to the method described in claim 2, it is characterized in that, it is described according to the historical data and with the historical data Corresponding confidence level calculates the static estimation of "current" model parameter, according to the update of the static estimation of the "current" model parameter Historical data, comprising:
The historical measurement data for meeting the first preset threshold is obtained from the historical measurement data as the first data；
The second data of conduct for meeting the image of the second preset threshold are obtained from the history image；
By first data, second data, the original dimension of the skeleton, the initial peace of the inertial sensor The initial makeup location and the static constraint condition entry static optimization model for filling posture, the luminous point, after being optimized The initial makeup location of the original dimension of the skeleton, the initial Installation posture of the inertial sensor and the luminous point, Wherein, the static optimization model is the model for the static estimation of Optimized model parameter；
According to the original dimension of the skeleton after optimization, the initial Installation posture and the luminous point of the inertial sensor Initial makeup location the static estimation of the "current" model parameter is calculated, estimated according to the static state of the "current" model parameter Meter updates the historical data.
5. the method according to claim 1, wherein the dynamic compensation of "current" model parameter includes that the inertia passes The dynamic compensation of the installation site of the dynamic compensation and luminous point of the Installation posture of sensor, it is described according to the current frame number The dynamic compensation of "current" model parameter is calculated according to, the confidence level of the current frame data and updated historical data, comprising:
Obtain the first preset threshold and the second preset threshold；
When the confidence level of the current measurement data does not meet first preset threshold, the confidence level of the present image meets Second preset threshold, the posture of the skeleton is calculated according to the present image, according to the present image, update The dynamic of the Installation posture of inertial sensor described in the Attitude Calculation of historical measurement data and the skeleton afterwards compensates；
When the confidence level of the current measurement data meets first preset threshold, the confidence level of the present image does not meet Second preset threshold calculates the posture of the skeleton, the installation position of the luminous point according to the current measurement data It sets, is compensated according to the dynamic that the installation site of the posture of the skeleton, the luminous point calculates the installation site of the luminous point.
6. a kind of model parameter calibrating installation, which is characterized in that described device includes:
Historical data obtains module, and for obtaining historical data, the historical data includes the historical position and appearance of skeleton The historical measurement data of state information, the history image comprising luminous point and inertial sensor, the luminous point and the inertial sensor It is mounted on human body；
Current data obtains module, for obtaining the current frame number of the current measurement data including present image and inertial sensor According to；
The static estimation computing module of model parameter, for according to the historical data and corresponding with the historical data credible Degree calculates the static estimation of "current" model parameter, updates the historical data according to the static estimation of the "current" model parameter；
Confidence level computing module, for calculating the confidence level of the current frame data according to the historical data；
The dynamic compensation calculation module of model parameter, for the confidence level according to the current frame data, the current frame data The dynamic compensation of "current" model parameter is calculated with updated historical data；
The confidence level computing module is specifically used for establishing prediction model according to the historical data；The current frame data is defeated Enter prediction model, obtains the predicted value of present frame skeleton position and posture；According to the predicted value and based on described The diversity factor of the end value of present frame skeleton position and posture that current frame data is calculated calculates the current frame number According to confidence level.
7. device according to claim 6, which is characterized in that described device further include:
Primary data obtains module, and the original dimension and the luminous point for obtaining the skeleton are relative to the human body The initial makeup location of bone obtains the skeleton when static, and the skeleton is in image when different postures With the measurement data of the inertial sensor；
With reference to Attitude Calculation module, for calculating the people in different postures according to the initial makeup location and described image The reference posture of body bone；
Data processing module, for obtaining inertia seat according to the measurement data and the Attitude Calculation that refers to of the inertial sensor Mark system and the rotation relationship of luminous point coordinate system and the initial Installation posture of the inertial sensor；
Luminous point optimization module, for obtaining static constraint condition, according to the static constraint condition, the skeleton it is initial Size, it is described with reference to posture and with the initial makeup location with reference to the corresponding luminous point of posture update the luminous point just Beginning installation site；
Dynamic data acquisition module, for obtaining the measurement data of the inertial sensor when human body executes deliberate action And image, respectively as dynamic measuring data and dynamic image；
Data update module, for obtaining dynamic constrained condition, according to the dynamic constrained condition, the dynamic measuring data and The dynamic image updates the initial Installation posture and the luminous point of the original dimension of the skeleton, the inertial sensor Initial makeup location.
8. a kind of computer equipment including memory, processor and stores the meter that can be run on a memory and on a processor Calculation machine program, which is characterized in that the processor realizes any one of claims 1 to 5 institute when executing the computer program The step of stating method.
9. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program quilt The step of processor realizes method described in any one of claims 1 to 5 when executing.
CN201811550990.0A 2018-12-18 2018-12-18 Model parameter calibration method, device, computer equipment and storage medium Active CN109711302B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811550990.0A CN109711302B (en) 2018-12-18 2018-12-18 Model parameter calibration method, device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811550990.0A CN109711302B (en) 2018-12-18 2018-12-18 Model parameter calibration method, device, computer equipment and storage medium

Publications (2)

Publication Number Publication Date
CN109711302A CN109711302A (en) 2019-05-03
CN109711302B true CN109711302B (en) 2019-10-18

Family

ID=66256664

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811550990.0A Active CN109711302B (en) 2018-12-18 2018-12-18 Model parameter calibration method, device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN109711302B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112001413B (en) * 2020-07-13 2023-06-06 中国标准化研究院 Human body appearance data prediction system based on human body size database
CN112033432A (en) * 2020-07-31 2020-12-04 东莞市易联交互信息科技有限责任公司 Motion capture method, system and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106445130A (en) * 2016-09-19 2017-02-22 武汉元生创新科技有限公司 Motion capture glove for gesture recognition and calibration method thereof
CN108107592A (en) * 2014-01-06 2018-06-01 欧库勒斯虚拟现实有限责任公司 The calibration of virtual reality system
CN108939512A (en) * 2018-07-23 2018-12-07 大连理工大学 A kind of swimming attitude measurement method based on wearable sensor

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2171688A2 (en) * 2007-07-10 2010-04-07 Koninklijke Philips Electronics N.V. Object motion capturing system and method
US9983709B2 (en) * 2015-11-02 2018-05-29 Oculus Vr, Llc Eye tracking using structured light
CN106153077B (en) * 2016-09-22 2019-06-14 苏州坦特拉智能科技有限公司 A kind of initialization of calibration method for M-IMU human motion capture system
CN108268129B (en) * 2016-12-30 2021-03-12 北京诺亦腾科技有限公司 Method and apparatus for calibrating a plurality of sensors on a motion capture glove and motion capture glove
CN107563295B (en) * 2017-08-03 2020-07-28 中国科学院自动化研究所 Multi-Kinect-based all-dimensional human body tracking method and processing equipment

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108107592A (en) * 2014-01-06 2018-06-01 欧库勒斯虚拟现实有限责任公司 The calibration of virtual reality system
CN106445130A (en) * 2016-09-19 2017-02-22 武汉元生创新科技有限公司 Motion capture glove for gesture recognition and calibration method thereof
CN108939512A (en) * 2018-07-23 2018-12-07 大连理工大学 A kind of swimming attitude measurement method based on wearable sensor

Also Published As

Publication number Publication date
CN109711302A (en) 2019-05-03

Similar Documents

Publication Publication Date Title
US11182599B2 (en) Motion state evaluation system, motion state evaluation device, motion state evaluation server, motion state evaluation method, and motion state evaluation program
KR102097190B1 (en) Method for analyzing and displaying a realtime exercise motion using a smart mirror and smart mirror for the same
CN1960674B (en) System and method for ergonomic tracking for individual physical exertion
CN106153077B (en) A kind of initialization of calibration method for M-IMU human motion capture system
KR20100002803A (en) Apparatus and method for capturing a motion of human
CN109711302B (en) Model parameter calibration method, device, computer equipment and storage medium
WO2018087853A1 (en) Stereoscopic image generation system, stereoscopic image generation method, and stereoscopic image generation program
Bleser et al. Using egocentric vision to achieve robust inertial body tracking under magnetic disturbances
JP6288858B2 (en) Method and apparatus for estimating position of optical marker in optical motion capture
CN106648088A (en) Inertial motion capture pose transient calibration method and inertial motion capture system
CN109284006B (en) Human motion capturing device and method
CN108106614B (en) A kind of inertial sensor and visual sensor data melt algorithm
CN111681281B (en) Calibration method and device for limb motion capture, electronic equipment and storage medium
CN111194122A (en) Somatosensory interactive light control system
Salehi et al. Body-IMU autocalibration for inertial hip and knee joint tracking
CN110609621B (en) Gesture calibration method and human motion capture system based on microsensor
JP2020151267A (en) Body motion estimation system for person or the like
De Rosario et al. Correction of joint angles from Kinect for balance exercising and assessment
Truppa et al. An innovative sensor fusion algorithm for motion tracking with on-line bias compensation: Application to joint angles estimation in yoga
JP2016081121A (en) Information processing unit, information processing method and program
CN109297507B (en) Human body limb movement capturing method free of alignment action based on inertial sensor
US11497962B2 (en) System and method for human motion detection and tracking
JP6205387B2 (en) Method and apparatus for acquiring position information of virtual marker, and operation measurement method
CN109814714A (en) The Installation posture of motion sensor determines method, apparatus and storage medium
Romantsova et al. Metrological parameters of information and measurement systems for the study of a kinematic portrait of a person

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant