CN108830861A - A kind of hybrid optical motion capture method and system - Google Patents
A kind of hybrid optical motion capture method and system Download PDFInfo
- Publication number
- CN108830861A CN108830861A CN201810519253.8A CN201810519253A CN108830861A CN 108830861 A CN108830861 A CN 108830861A CN 201810519253 A CN201810519253 A CN 201810519253A CN 108830861 A CN108830861 A CN 108830861A
- Authority
- CN
- China
- Prior art keywords
- module
- mark point
- data
- infrared
- capture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/155—Segmentation; Edge detection involving morphological operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Abstract
The invention discloses a kind of hybrid optical motion capture method and system, the method carries out color correction to every capture camera first；Each capture camera positions the mark point in pure infrared image again；Further according to the number of corresponding position mark point in fixation and recognition visible images；Then 2-D data repairing is carried out to the mark point that may be present blocked or lost；Then 2-D data is redeveloped into three-dimensional data；The reconstruction three-dimensional data is finally sent to application end.The system comprises color correction module, infrared capture to take, and acquisition module, identification module, repairing module, rebuilds module, transmission module, application module at locating module.The present invention can solve the mark point in capture-process and block, and avoid the post-processing that movement capturing data is many and diverse, while also reducing optical motion and capturing use cost.
Description
Technical field
The present invention relates to a kind of hybrid optical motion capture method and systems, belong to field of human-computer interaction more particularly to light
Capture data processing under learning motion capture and blocking.
Background technique
In recent years with the continuous upgrading of the sustainable development of machine vision technique and computer hardware, motion capture technology
It has been widely deployed in the fields such as production of film and TV, training, interactive game.According to the difference of working principle, fortune
Dynamic capture can be divided into following a few classes substantially：Mechanically, electromagnetic type, acoustics formula, inertia-type, optical profile type etc..Optics among this
Formula it is motion-captured with its high-precision, limitation of movement is small, sample frequency is high the advantages that be widely used in the professional field such as film making
In conjunction.When progress optics is moved and caught, the major joint point for capturing object is labeled with some special infrared markers points, infrared camera
Its position in two dimensional image is determined according to the infrared light of label point reflection.According to principle of stereoscopic vision, if a label
Point seen in two infrared cameras, can calculate this mark point in three-dimensional simultaneously according to the location parameter demarcated between camera
Three-dimensional position in space.
Because infrared markers point is generally indiscriminate grey witch ball, when being captured, system can not directly distinguish mark
The corresponding space coordinate of note point, each mark point are expressed as a series of unordered three-dimensional coordinates.So optical motion captures
It needs performer to make preset posture, usually T-pose when beginning, is closed to establish the mapping of mark point and human synovial rigid body
System.Often require to use prefabricated template among these, with preset human geometry's constraint shapes to each mark point matched come
Obtain the specificity information of mark point.And on the other hand, mark point is easy to appear circumstance of occlusion during the motion, causes pair
The detection of joints failure answered, or be the excessive failure of restrained deformation between mark point each in compound movement, cause to mark
Point information is obscured, while the processing for influencing follow-up data calculates.It will lead to that optical profile type is dynamic to catch these the occurrence of
Real time demonstration is ineffective, and then needs manually to carry out a large amount of data cleansing repair in the post-processing of data.It solves
Such issues that direct method be to increase the quantity of infrared capture camera, enhance the covering power of camera as much as possible, but in this way
The operation expense and use cost of system are increased again.
Summary of the invention
In view of the defects existing in the prior art, the purpose of the present invention is to provide a kind of hybrid optical motion capture method and
System, while visible images and infrared image are utilized, it is avoided with the number that color encodes directly detection infrared markers point at random
The matching of data, with internal interpolation repairing missing data, with overcome current optical formula motion-captured vulnerable to block influence, data
Post-process many and diverse, the high problem of use cost.
In order to achieve the above objectives, the present invention adopts the following technical scheme that：
A kind of hybrid optical motion capture method, includes the following steps：
101. pair every capture camera carries out color correction；
102. every capture camera positions the mark point in pure infrared image；
103. according to the number of corresponding position mark point in the fixation and recognition visible images；
104. pair mark point that may be present blocked or lost carries out 2-D data repairing；
105. the 2-D data that all capture cameras obtain is redeveloped into three-dimensional data；
106. the three-dimensional data is sent to application end.
Color correction is carried out to every capture camera in the step 101, including：
A. using the otherwise visible light color image of this camera acquisition Standard colour board；
B. the Standard colour board image is calculated in the lower color characteristic of this camera environment；
C. the mapping function of the color characteristic of the color characteristic and Standard colour board under standard environment is calculated；
D. color correction is carried out to this camera image by the mapping function.
In the step 102 locating mark points be by carrying out gray level threshold segmentation and Morphological scale-space to infrared image,
It is quickly found two-dimensional position of all mark points in infrared image.
Mark point is identified by the position of infrared image acquisition unit Yu visible images acquisition unit in the step 103
Calibration relationship is set, the position of mark point in visible images is obtained, visible images have been subjected to color correction at this time, further quasi-
The number of mark point really is identified according to color.
Data modification is the mark point missing which number is learnt by detecting in the step 104, is transported using mark point
Dynamic continuity in time, the 2-D data of missing is calculated using local interpolation method.
Data reconstruction is the three-dimensional information that each mark point is reconstructed using principle of stereoscopic vision in the step 105, is used
By repairing after 2-D data it is always complete.
A kind of hybrid optical motion capture system, including color correction module, infrared capture clothes, acquisition module, positioning mould
Block, identification module repair module, rebuild module, transmission module, application module；Wherein,
Color correction module, for the color correction of each capture camera under various circumstances；
Infrared capture clothes, for marking each position of human body using infrared reflecting mark point；
Acquisition module, for acquiring infrared image and visible images simultaneously；
Locating module, for positioning the mark point position in infrared image；
Identification module, for identification in visible images mark point number；
Module is repaired, for carrying out 2-D data repairing to missing mark point that may be present；
Module is rebuild, for the 2-D data of infrared camera to be redeveloped into three-dimensional data；
Transmission module, for the reconstruction three-dimensional data to be transmitted to application module；
Application module, for the three-dimensional data to be carried out concrete application.
The infrared capture clothes mark partes corporis humani position using the infrared reflecting mark point of different colours, not according to color
Together, each mark point has unique corresponding number.
The acquisition module includes infrared image acquisition unit and visible images acquisition unit, can acquire infrared figure simultaneously
As and visible images, the positional relationship between two kinds of field of view pass through camera calibration acquisition.
The present invention compared with the existing technology has the following advantages that and effect：
The method of the present invention carrys out matched indicia point without carrying out post-processing matching to data, without using prefabricated posture and template, can
The specific number of each infrared markers point of Direct Recognition, avoids mark point from obscuring；Mark point is blocked, can directly be existed in real time
It is repaired on 2-D data, guarantees the integrality of data；And then the quantitative requirement for capturing camera is reduced, reduce optical profile type movement
The use cost of capture.
Detailed description of the invention
Fig. 1 is a kind of hybrid optical formula motion capture method flow diagram described in the embodiment of the present invention.
Fig. 2 is infrared markers space of points distribution map described in the embodiment of the present invention.
Fig. 3 is human joint points spatial distribution map described in the embodiment of the present invention.
Fig. 4 is a kind of hybrid optical formula motion capture method system schematic described in the embodiment of the present invention.
Specific embodiment
To make the object, technical solutions and advantages of the present invention clearer, below in conjunction with the preferred embodiment of the present invention and
Technical solution of the present invention is clearly and completely described in attached drawing.
As shown in Figure 1, a kind of hybrid optical motion capture method, includes the following steps：
101. pair every capture camera carries out color correction；
102. every capture camera positions the mark point in pure infrared image；
103. according to the number of corresponding position mark point in the fixation and recognition visible images；
104. pair mark point that may be present blocked or lost carries out 2-D data repairing；
105. the 2-D data that all capture cameras obtain is redeveloped into three-dimensional data；
106. the three-dimensional data is sent to application end.
Color correction is carried out to every capture camera in the step 101, including：
A. using the otherwise visible light color image of this camera acquisition Standard colour board；
B. the Standard colour board image is calculated in the lower color characteristic of this camera environment；
C. the mapping function of the color characteristic of the color characteristic and Standard colour board under standard environment is calculated；
D. color correction is carried out to this camera image by the mapping function.
The step 101 image-context different for each video camera, i.e., different illumination, equipment color difference etc., to figure
The color of picture adjusts compensation to offset it to influence caused by color identification.Color correction is made, mistake can be passed through
Poor reverse transmittance nerve network trains correction maps function, by image of the Standard colour board under standard environment and non-standard environmental
As training sample.Input layer uses 3 neuron nodes, corresponding triple channel rgb value to be corrected；Hidden layer uses 12 minds
Through first node, Sigmoid function is activation primitive；Output layer uses 3 neuron nodes, the triple channel rgb value of corresponding correction；
And adaptive adjustment ground is carried out using momentum method and learning rate to learn mapping function.
In the step 102 locating mark points be by infrared image carry out gray level threshold segmentation, then by intermediate value filter
Wave and etching operation remove picture noise, obtain the image of clean two-value, are quickly found all mark points in infrared image
In two-dimensional position.
Mark point is identified by the position of infrared image acquisition unit Yu visible images acquisition unit in the step 103
Calibration relationship is set, the position for obtaining mark point in visible images takes and its Euclidean distance then according to the rgb value of this position
Number of the smallest standard color coding as this point.
Data modification can learn that the mark point of which number lacks by detecting in the step 104, be transported using mark point
Dynamic continuity in time carries out interpolation to the mark point lacked in infrared image using cubic spline interpolation, has obtained
Whole mark point 2-D data.
Data reconstruction is the three-dimensional information that each mark point is reconstructed using principle of stereoscopic vision in the step 105, is used
By repairing after 2-D data it is always complete.
As shown in figure 4, a kind of hybrid optical motion capture system, including color correction module 21, infrared capture take 22, adopt
Collect module 23, locating module 24, identification module 25 repairs module 26, rebuilds module 27, transmission module 28, application module 29；Its
In,
Color correction module 21, for the color correction of each capture camera under various circumstances；
Infrared capture takes 22, for marking each position of human body using infrared reflecting mark point；
Acquisition module 23, for acquiring infrared image and visible images simultaneously；
Locating module 24, for positioning the mark point position in infrared image；
Identification module 25, for identification in visible images mark point number；
Module 26 is repaired, for carrying out 2-D data repairing to missing mark point that may be present；
Module 27 is rebuild, for the 2-D data of infrared camera to be redeveloped into three-dimensional data；
Transmission module 28, for the reconstruction three-dimensional data to be transmitted to application module 29；
Application module 29, for the three-dimensional data to be carried out concrete application.
The infrared capture takes 22 and marks partes corporis humani position using the infrared reflecting mark point of different colours, according to color
Difference, each mark point have unique corresponding number.32 mark points are shared in the present embodiment, throughout head, arm, chest,
Shoulder, waist, leg, foot etc., spatial distribution is as shown in Fig. 2, its standard color coding RGB is as shown in the table：
The acquisition module 23 includes infrared image acquisition unit 231 and visible images acquisition unit 232, be can be groups of
Infrared camera and Visible Light Camera are placed with fixed mode, can acquire infrared image and visible images, two kinds of images simultaneously
Positional relationship between the visual field is obtained by camera calibration.
The three-dimensional information rebuild module 27 and reconstruct each mark point using principle of stereoscopic vision, the artis of composition are 16
A, spatial distribution is as shown in Figure 3.
Claims (9)
1. a kind of hybrid optical motion capture method, which is characterized in that include the following steps：
101. carrying out color correction to every capture camera；
102. every capture camera positions the mark point in pure infrared image；
103. according to the number of corresponding position mark point in the fixation and recognition visible images；
104. pair mark point that may be present blocked or lost carries out 2-D data repairing；
105. the 2-D data that all capture cameras obtain is redeveloped into three-dimensional data；
106. the three-dimensional data is sent to application end.
2. hybrid optical motion capture method as described in claim 1, which is characterized in that caught in the step 101 to every
It catches camera and carries out color correction, including：
A. using the otherwise visible light color image of this camera acquisition Standard colour board；
B. the Standard colour board image is calculated in the lower color characteristic of this camera environment；
C. the mapping function of the color characteristic of the color characteristic and Standard colour board under standard environment is calculated；
D. color correction is carried out to this camera image by the mapping function.
3. hybrid optical motion capture method as described in claim 1, which is characterized in that mark point is fixed in the step 102
Position is to be quickly found all mark points in infrared image by carrying out gray level threshold segmentation and Morphological scale-space to infrared image
In two-dimensional position.
4. hybrid optical motion capture method as described in claim 1, which is characterized in that mark point is known in the step 103
It is not that visible images acceptance of the bid is obtained by the location position relationship of infrared image acquisition unit and visible images acquisition unit
Remember the position of point, visible images have been subjected to color correction at this time, and the number of mark point is further accurately identified according to color.
5. hybrid optical motion capture method as described in claim 1, which is characterized in that data modification in the step 104
It is that the mark point missing which number is learnt by detecting is inserted using the continuity of mark point movement in time using part
Value method calculates the 2-D data of missing.
6. hybrid optical motion capture method as described in claim 1, which is characterized in that data reconstruction in the step 105
It is the three-dimensional information that each mark point is reconstructed using principle of stereoscopic vision, the used 2-D data after repairing is always complete
's.
7. a kind of hybrid optical motion capture system, which is characterized in that including color correction module（21）, infrared capture clothes
（22）, acquisition module（23）, locating module（24）, identification module（25）, repair module（26）, rebuild module（27）, transmit mould
Block（28）, application module（29）；Wherein,
Color correction module（21）, for the color correction of each capture camera under various circumstances；
Infrared capture clothes（22）, for marking each position of human body using infrared reflecting mark point；
Acquisition module（23）, for acquiring infrared image and visible images simultaneously；
Locating module（24）, for positioning the mark point position in infrared image；
Identification module（25）, for identification in visible images mark point number；
Repair module（26）, for carrying out 2-D data repairing to missing mark point that may be present；
Rebuild module（27）, for the 2-D data of infrared camera to be redeveloped into three-dimensional data；
Transmission module（28）, for the reconstruction three-dimensional data to be transmitted to application module（29）；
Application module（29）, for the three-dimensional data to be carried out concrete application.
8. hybrid optical motion capture system as claimed in claim 7, which is characterized in that the infrared capture clothes（22）It uses
The infrared reflecting mark point label partes corporis humani position of different colours, according to the difference of color, each mark point has unique corresponding
Number.
9. hybrid optical motion capture system as claimed in claim 7, which is characterized in that the acquisition module（23）Including red
Outer image acquisition units（231）With visible images acquisition unit（232）, infrared image and visible images can be acquired simultaneously,
Positional relationship between two kinds of field of view is obtained by camera calibration.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810519253.8A CN108830861A (en) | 2018-05-28 | 2018-05-28 | A kind of hybrid optical motion capture method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810519253.8A CN108830861A (en) | 2018-05-28 | 2018-05-28 | A kind of hybrid optical motion capture method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108830861A true CN108830861A (en) | 2018-11-16 |
Family
ID=64145733
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810519253.8A Pending CN108830861A (en) | 2018-05-28 | 2018-05-28 | A kind of hybrid optical motion capture method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108830861A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110442153A (en) * | 2019-07-10 | 2019-11-12 | 佛山科学技术学院 | A kind of passive optical is dynamic to catch system video cameras Corrective control method and system |
| CN110796701A (en) * | 2019-10-21 | 2020-02-14 | 深圳市瑞立视多媒体科技有限公司 | Identification method, device and equipment of mark points and storage medium |
| CN113012126A (en) * | 2021-03-17 | 2021-06-22 | 武汉联影智融医疗科技有限公司 | Mark point reconstruction method and device, computer equipment and storage medium |
| CN113505637A (en) * | 2021-05-27 | 2021-10-15 | 成都威爱新经济技术研究院有限公司 | Real-time virtual anchor motion capture method and system for live streaming |
| CN114969623A (en) * | 2022-07-28 | 2022-08-30 | 江西财经大学 | Data processing method and system for lepidoptera insect motion capture |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106175780A (en) * | 2016-07-13 | 2016-12-07 | 天远三维（天津）科技有限公司 | Facial muscle motion-captured analysis system and the method for analysis thereof |
| CN106600627A (en) * | 2016-12-07 | 2017-04-26 | 成都通甲优博科技有限责任公司 | Rigid body motion capturing method and system based on mark point |
-
2018
- 2018-05-28 CN CN201810519253.8A patent/CN108830861A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106175780A (en) * | 2016-07-13 | 2016-12-07 | 天远三维（天津）科技有限公司 | Facial muscle motion-captured analysis system and the method for analysis thereof |
| CN106600627A (en) * | 2016-12-07 | 2017-04-26 | 成都通甲优博科技有限责任公司 | Rigid body motion capturing method and system based on mark point |
Non-Patent Citations (2)
| Title |
|---|
| 佟颖: ""基于红外与可见光双波段图像的立体视觉关键技术研究"", 《中国博士学位论文全文数据库 信息科技辑》 * |
| 李青: ""光学式人体运动捕捉数据处理研究"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110442153A (en) * | 2019-07-10 | 2019-11-12 | 佛山科学技术学院 | A kind of passive optical is dynamic to catch system video cameras Corrective control method and system |
| CN110442153B (en) * | 2019-07-10 | 2022-03-25 | 佛山科学技术学院 | Camera correction control method and system for passive optical dynamic capturing system |
| CN110796701A (en) * | 2019-10-21 | 2020-02-14 | 深圳市瑞立视多媒体科技有限公司 | Identification method, device and equipment of mark points and storage medium |
| CN110796701B (en) * | 2019-10-21 | 2022-06-07 | 深圳市瑞立视多媒体科技有限公司 | Identification method, device and equipment of mark points and storage medium |
| CN113012126A (en) * | 2021-03-17 | 2021-06-22 | 武汉联影智融医疗科技有限公司 | Mark point reconstruction method and device, computer equipment and storage medium |
| CN113012126B (en) * | 2021-03-17 | 2024-03-22 | 武汉联影智融医疗科技有限公司 | Method, device, computer equipment and storage medium for reconstructing marking point |
| CN113505637A (en) * | 2021-05-27 | 2021-10-15 | 成都威爱新经济技术研究院有限公司 | Real-time virtual anchor motion capture method and system for live streaming |
| CN114969623A (en) * | 2022-07-28 | 2022-08-30 | 江西财经大学 | Data processing method and system for lepidoptera insect motion capture |
| CN114969623B (en) * | 2022-07-28 | 2022-10-25 | 江西财经大学 | Data processing method and system for lepidoptera insect motion capture |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108830861A (en) | A kind of hybrid optical motion capture method and system | |
| CN109255813B (en) | Man-machine cooperation oriented hand-held object pose real-time detection method | |
| CN101853528B (en) | Hand-held three-dimensional surface information extraction method and extractor thereof | |
| EP4042101A1 (en) | Systems and methods for surface normals sensing with polarization | |
| CN103337094A (en) | Method for realizing three-dimensional reconstruction of movement by using binocular camera | |
| CN109758756B (en) | Gymnastics video analysis method and system based on 3D camera | |
| CN103839277A (en) | Mobile augmented reality registration method of outdoor wide-range natural scene | |
| CN106996748A (en) | A kind of wheel footpath measuring method based on binocular vision | |
| CN110246124A (en) | Target size measurement method and system based on deep learning | |
| CN106952312B (en) | Non-identification augmented reality registration method based on line feature description | |
| CN114119739A (en) | Binocular vision-based hand key point space coordinate acquisition method | |
| CN108154066B (en) | Three-dimensional target identification method based on curvature characteristic recurrent neural network | |
| CN101833762A (en) | Different-source image matching method based on thick edges among objects and fit | |
| CN115272271A (en) | Pipeline defect detecting and positioning ranging system based on binocular stereo vision | |
| CN113077519B (en) | Multi-phase external parameter automatic calibration method based on human skeleton extraction | |
| CN111583386B (en) | Multi-view human body posture reconstruction method based on label propagation algorithm | |
| CN112365578A (en) | Three-dimensional human body model reconstruction system and method based on double cameras | |
| CN103093426B (en) | Method recovering texture and illumination of calibration plate sheltered area | |
| CN111738971B (en) | Circuit board stereoscopic scanning detection method based on line laser binocular stereoscopic vision | |
| WO2023280082A1 (en) | Handle inside-out visual six-degree-of-freedom positioning method and system | |
| Xompero et al. | Multi-view shape estimation of transparent containers | |
| CN105488802A (en) | Fingertip depth detection method and system | |
| WO2023193763A1 (en) | Data processing method and apparatus, and tracking mark, electronic device and storage medium | |
| Haker et al. | Self-organizing maps for pose estimation with a time-of-flight camera | |
| CN109410272B (en) | Transformer nut recognition and positioning device and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181116 |
|
| RJ01 | Rejection of invention patent application after publication |