CN114140598A - Double-camera human body scanning method and system - Google Patents
Double-camera human body scanning method and system Download PDFInfo
- Publication number
- CN114140598A CN114140598A CN202111437531.3A CN202111437531A CN114140598A CN 114140598 A CN114140598 A CN 114140598A CN 202111437531 A CN202111437531 A CN 202111437531A CN 114140598 A CN114140598 A CN 114140598A
- Authority
- CN
- China
- Prior art keywords
- human body
- camera
- depth
- point cloud
- depth camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000005259 measurement Methods 0.000 claims abstract description 16
- 238000012545 processing Methods 0.000 claims abstract description 10
- 230000008569 process Effects 0.000 claims description 21
- 238000005457 optimization Methods 0.000 claims description 9
- 230000009977 dual effect Effects 0.000 claims description 8
- 238000009499 grossing Methods 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 6
- 230000002159 abnormal effect Effects 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 238000012805 post-processing Methods 0.000 claims description 4
- 239000013598 vector Substances 0.000 claims description 4
- 238000004422 calculation algorithm Methods 0.000 claims description 3
- 201000002161 intrahepatic cholestasis of pregnancy Diseases 0.000 claims 2
- 238000013461 design Methods 0.000 abstract description 4
- 230000036541 health Effects 0.000 abstract description 3
- 238000004519 manufacturing process Methods 0.000 abstract description 3
- 230000008901 benefit Effects 0.000 abstract description 2
- 238000009616 inductively coupled plasma Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 208000012886 Vertigo Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004576 sand Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 231100000889 vertigo Toxicity 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a double-camera human body scanning method and a double-camera human body scanning system, wherein the method comprises the following steps: s1, receiving a signal for starting measurement input by a user, controlling the rotation of the turntable and controlling the first depth camera and the second depth camera to alternately acquire depth images corresponding to each part of a measured person, wherein when the turntable rotates, the measured person rotates along with the turntable, and the first depth camera and the second depth camera are fixed on the bracket and do not rotate along with the turntable; s2, processing the collected depth image to form a three-dimensional human body model corresponding to the measured person; and S3, extracting point cloud data in the three-dimensional human body model, and calculating the key size of the measured person. The invention can automatically synthesize the human body three-dimensional data by utilizing the scanning data without manual additional calibration, and the equipment has stronger flexibility. The method can be applied to the fields of physical health management, garment design and manufacture, online shopping, virtual fitting and the like, and has certain economic benefit and wide market prospect.
Description
Technical Field
The invention relates to the technical field of human body scanning, in particular to a double-camera human body scanning system.
Background
The traditional contact type human body measuring method has the defects of low measuring efficiency, high labor cost, uncontrollable error and the like, is easy to cause discomfort of a measured person, and cannot well provide high-quality service for customers. Therefore, a human body scanning system with no contact, high speed, small error, less workload and simple structure is urgently needed in the technical field of human body scanning. With the rapid development of computer vision, various human body scanning systems are present at present, and are widely applied to the fields of physical health management, garment design and manufacture, online shopping, virtual fitting and the like.
The human body scanning system is a non-contact automatic measuring system which utilizes an optical measuring technology, a point cloud processing technology and the like to carry out three-dimensional human body surface contour processing. Generally, there are several techniques for implementing human body scanning as follows:
the first technology is as follows: based on a slidable single depth camera. Usually, a single depth camera is fixed on a slide block, and the information of a human body is scanned in a mode that the slide block moves up and down along a slide rail. The equipment comprises an upright post, a rotary table, a connecting plate and a depth camera, wherein one end of the connecting plate is connected with the upright post, and the other end of the connecting plate is connected with the rotary table; a base, a sliding rail and a sliding block are arranged in the upright column shell along the length direction of the upright column shell, and the depth camera is fixed on the sliding block; the motor drive sliding block moves for the camera can slide from top to bottom along the slide rail, and the revolving stage can rotate for the connecting plate, carries out human scanning more comprehensively, acquires human information. The flexibility of data acquisition is improved, the camera does not need to be calibrated again after the equipment is moved, the position can be moved randomly, and the measurement precision is not influenced.
The second technology is as follows: four-column human body scanning system. Four columns are erected on polygon vertexes around the human body measurement space, 2 to 4 depth cameras are installed on each column, and the depth cameras are configured such that the field angle in the horizontal direction is smaller than the field angle in the vertical direction; then calibrating internal parameters and external parameters of the camera, alternately acquiring depth data acquired by the multi-depth camera, fusing the depth data to generate a three-dimensional human body model, and further extracting the key size of the human body. The whole scanning system can complete human body scanning within several seconds, and the efficiency is high.
The existing human body scanning system mainly adopts a slidable single-depth camera device or a four-upright-column device to collect human body data. The slidable single-depth camera device is characterized in that a single-depth camera is fixed on a sliding block, and human body information is scanned in a mode that the sliding block moves up and down along a sliding rail; the depth camera moving up and down inevitably generates noise when scanning a human body, and influences the sensory experience of a user; due to the fact that the starting time and the running speed of the rotary table motor and the sliding block motor are different, the photographing position of the depth camera slightly deviates, and the stability of model synthesis is poor; and the information collected by the single camera lacks contrast error correction process, so the accuracy of the scanning model is not high. 2-4 depth cameras are fixed on each column of the four-column device, 8-16 depth cameras are required for the whole set of equipment, the hardware cost is high, and the wiring is complex; moving any one upright post needs to recalibrate the whole device, so that the complexity of the mobile device is increased; dead angles of shooting can exist, for example, the oxter part is often lost, so that the comprehensive information of the human body can not be acquired, and the reconstruction and measurement accuracy is influenced; in addition, the four-column human body scanning equipment needs a large space to arrange hardware equipment, occupies a large area, limits the application range, and is not beneficial to movement and transportation.
Disclosure of Invention
The present invention is directed to a dual-camera human body scanning system to solve the above problems.
In order to achieve the purpose, the invention provides the following technical scheme: a double-camera human body scanning method is characterized by comprising the following steps: s1, receiving a signal for starting measurement input by a user, controlling a turntable to rotate and controlling a first depth camera and a second depth camera to alternately acquire depth images corresponding to each part of a measured person, wherein when the turntable rotates, the measured person rotates along with the turntable, and the first depth camera and the second depth camera are fixed on a support and do not rotate along with the turntable; s2, processing the collected depth image to form a three-dimensional human body model corresponding to the measured person; and S3, extracting point cloud data in the three-dimensional human body model, and calculating the key size of the measured person.
In some embodiments, the following features are also included:
in step S2, the processing the acquired depth image to form the three-dimensional human body model corresponding to the measured person includes: s2-1, reading in configuration parameters, wherein the configuration parameters comprise internal and external parameters of the first depth camera and the second depth camera, depth image parameters and threshold parameters in a reconstruction algorithm; s2-2, respectively carrying out initialization rigid registration on the depth images acquired by the first depth camera and the second depth camera; s2-3, respectively carrying out global rigid registration on the point clouds after the first depth camera and the second depth camera are initialized and registered; s2-4, performing point cloud registration and splicing aiming at the global rigid registration output of the first depth camera and the second depth camera according to the initial external parameters; s2-5, performing Poisson reconstruction: meshing the result of the two point clouds, inputting the point clouds and normal vectors thereof, and outputting a three-dimensional mesh; s2-6, carrying out three-dimensional human body model post-processing: the method comprises the steps of extracting a maximum connected domain, smoothing and removing abnormal data.
The rigid registration operation is initialized as follows: s2-2-1, circularly reading in the depth image, and converting the read-in depth image into point cloud; s2-2-2, point cloud preprocessing and downsampling are carried out, wherein relative motion exists between a human body rotating in front of a camera and the ground, ground point cloud is mainly removed in the point cloud preprocessing, so that a rigid matching relation between front and rear frame point clouds is guaranteed as far as possible, and uniform grid downsampling is mainly used in the downsampling to improve performance; s2-2-3, performing point cloud registration to solve the camera pose, using the result of the previous frame as an initial value, performing point cloud registration, solving primary ICP, and reconstructing or updating a point cloud matcher; and S2-2-4, point cloud combination is carried out, and the down-sampled point clouds are combined by using the pose obtained by solving.
The global rigid registration operates as follows: s2-3-1, selecting key frames by using the preliminarily solved camera pose, and ensuring that the key frames are uniformly distributed in the angle as much as possible; s2-3-2, establishing adjacent key frames with the interval of 1 or 2 as a camera pair; s2-3-3, traversing the camera pair, and performing ICP registration to obtain an initial relative pose; s2-3-4, traversing the camera pair, and performing point cloud neighbor matching by using the initial relative pose; s2-3-5, establishing least square optimization for all point cloud matching, and solving a relative pose; s2-3-6, judging whether the global optimization is converged, if yes, ending; otherwise, the step S2-3-4 is returned.
The method further comprises displaying the three-dimensional human body point cloud and the size extraction result on the interface.
The invention also provides a double-camera human body scanning system, which comprises: a host; the first depth camera is connected with the host and is used for shooting lower body data; the second depth camera is connected with the host and is used for shooting upper half body data; the turntable is connected with the host machine by using a serial port line and is used for a tested person to stand; the host computer has a processor and a memory, the memory having stored therein a computer program executable to implement the method as described above.
Optionally, the method further comprises: a bracket for fixing two depth cameras; a handrail; the button is embedded into the armrest and is in wireless connection with the host; and the display screen displays the human body acquisition process, the three-dimensional human body model and the human body key size.
Optionally, the computer program interface comprises: preview page, scan start page, countdown page, collect page, wait page, measurement success page, and measurement failure page.
The invention has at least the following beneficial effects:
the structure of the invention is provided with a turntable for a person to be measured to stand on, a rotary driving mechanism is arranged in the turntable, two depth cameras are fixed on a bracket at one side of the turntable and are connected to a host through a USB, and under the condition of avoiding contact with the person to be measured, 360-degree all-directional scanning of a human body is realized through software control. Because the camera is fixed on the bracket, and the distance between the bracket and the turntable is short, the human body scanning device saves the occupied space, reduces the use cost of the device and enlarges the application range of the device. The invention can enable the person to be measured to independently complete the measurement through software control, thereby well protecting the personal privacy.
In addition, the invention can automatically synthesize the human body three-dimensional data by utilizing the scanning data without manual additional calibration, and the equipment has stronger flexibility.
The invention can be applied to the fields of physical health management, garment design and manufacture, online shopping, virtual fitting and the like, and has certain economic benefit and wide market prospect.
Drawings
FIG. 1 is an overall flow chart of the present invention;
FIG. 2 is an overall hardware assembly diagram of the present invention;
FIG. 3 is a schematic view of a camera scan according to the present invention;
FIG. 4 is a scan profile of FIG. 3;
FIG. 5 is a reconstruction diagram of the overall process of the present invention;
FIG. 6 is a flow chart of the global rigid registration of the present invention;
fig. 7 is an overall forming diagram of the human body scanning of the present invention.
In the figure: 1-a host; 2-a first depth camera; 3-a second depth camera; 4-a scaffold; 5-a turntable; 6-arm rest; 7-a button; 8-display screen.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1-7, a dual camera human body scanning method and system is shown, the method comprising the steps of:
s1, completing the assembly of the scanning system hardware device, and operating the scanning system control software;
s2, scanning is started, a measurer presses a switch of scanning hardware, and personnel rotate along with the hardware;
s3, in S2, the two depth cameras start to collect data alternately, scan each part of the human body and obtain a plurality of depth images;
s4, the personnel leave the hardware equipment which stops rotating, and the scanning is finished;
s5, analyzing the collected depth image to form a three-dimensional human body model;
s6, extracting point cloud data in the three-dimensional human body model, calculating key dimensions of the human body, such as height, neck circumference, chest circumference, waist circumference, hip circumference and the like, and displaying the three-dimensional human body point cloud and dimension extraction results on an interface.
The human body scanning system device includes:
the host is provided with a windows system and a display card above NVIDIA GeForce GTX 1050;
the first depth camera shoots the data of the lower body and is connected with the host by using a USB (universal serial bus) line;
the second depth camera shoots upper body data and is connected with the host computer by using a USB (universal serial bus) line;
the height of the bracket is 172cm, and the bracket is used for fixing two depth cameras, the first depth camera is 53cm away from the ground, and the distance between the two depth cameras is 89 cm; the distance between the support and the center of the rotary table is 125 cm;
the turntable is connected with the host machine by using a serial port line, and is used for a tested person to stand and bear 150kg of load;
the distance between the two handrails is 80cm, and the height expansion range is 70 cm-115 cm;
the button is embedded into the armrest and is in wireless connection with the host;
the display screen displays the human body acquisition process, the three-dimensional human body model and the human body key size and is connected with the host machine by an HDMI data line.
The scanning software interface in S1 includes: preview page, scan start page, countdown page, collect page, wait page, measurement success page, and measurement failure page.
In S5, the acquired depth image is analyzed and processed, and the overall reconstruction process includes the following steps:
1, reading configuration parameters. The method comprises the steps of obtaining internal and external parameters of a depth camera, depth image parameters, related threshold values in a reconstruction algorithm and the like;
respectively carrying out initialization rigid registration on the depth maps acquired by the first depth camera and the second depth camera;
3, respectively carrying out global rigid registration on the point clouds subjected to the initialization registration of the first depth camera and the second depth camera;
4, performing point cloud registration and splicing aiming at global rigid registration output of the first depth camera and the second depth camera according to the initial external parameters;
and 5, performing Poisson reconstruction. Meshing the result of the two point clouds, inputting the point clouds and normal vectors thereof, and outputting a three-dimensional mesh;
and 6, carrying out post-processing on the three-dimensional human body model. Including operations of extracting the maximum connected domain, smoothing, removing abnormal data, and the like.
The rigid registration operation is initialized as follows:
1, circularly reading in a depth image, and converting the read-in depth image into point cloud;
2, point cloud preprocessing and downsampling are carried out, wherein relative motion exists between a human body rotating in front of the camera and the ground, ground point cloud is mainly removed in the point cloud preprocessing, so that a rigid matching relation between front and rear frame point clouds is guaranteed as much as possible, and uniform grid downsampling is mainly used in the downsampling to improve performance;
3, registering and solving the camera pose by the point cloud, using the result of the previous frame as an initial value, registering the point cloud, solving primary ICP (inductively coupled plasma), and reconstructing or updating a point cloud matcher;
and 4, point cloud merging is carried out, and the down-sampled point clouds are merged by using the pose obtained by solving.
The global rigid registration operates as follows:
extracting an original point cloud key frame, selecting the key frame by using a preliminarily solved camera pose, and ensuring that the key frame is uniformly distributed on an angle as much as possible;
2, establishing a camera pair to form a loop, and establishing adjacent key frames with the interval of 1 or 2 as the camera pair;
3, the camera registers the initial ICP, traverses the camera pair, and performs ICP registration to obtain an initial relative pose;
4, performing point cloud neighbor matching by the camera, traversing the camera pair, and performing point cloud neighbor matching by using the initial relative pose;
optimizing global point-surface errors, establishing a least square optimization problem aiming at all point cloud matching, and solving a relative pose;
6, judging whether the global optimization is converged, if so, ending; otherwise, returning to the step 4.
The invention relates to a double-camera human body scanning system, which mainly solves the technical problems in the following two aspects:
(1) a body scanning device based on a dual depth camera;
the problem that the stability of human body model synthesis is poor due to the fact that a slidable single-depth camera collects a depth map in a moving mode is mainly solved; the four-column human body scanning equipment has the problems of complicated wiring, shooting dead angles, large occupied area and the like. Through the two depth cameras, in the rotating process of the rotary table, the depth data of the upper half body and the lower half body of the human body on the rotary table are respectively collected for three-dimensional human body reconstruction. The human body information acquisition device can completely acquire all human body information, is convenient to disassemble and assemble, is easy to carry, and occupies a small area.
(2) A human body reconstruction method of automatic calibration;
the problem that the existing four-column human body scanning equipment needs to be calibrated again after moving is mainly solved. The method comprises the steps of utilizing two depth cameras to collect data of an upper half body and a lower half body of a human body respectively, generating two parts of point clouds after initial registration and overall registration, then carrying out point cloud splicing optimization according to initial external parameters, and finally obtaining a three-dimensional human body model through a Poisson reconstruction process. The automatic calibration can be completed according to the depth data without calibrating the precise external parameters between the two depth cameras in advance, and the service efficiency of the whole equipment is improved.
The human body scanning device has a simple structure, does not need additional calibration, and is more convenient to transport, disassemble and assemble.
The invention discloses a double-camera human body scanning system. The system mainly comprises: a body scanning system device based on a dual depth camera; an automatic calibration human body reconstruction method. FIG. 1 is an overall flow chart of the present invention.
The detailed technical decomposition is as follows:
1. human body scanning device based on double-depth camera
The human body scanning device comprises a hardware device and control software.
1.1) hardware device of scanning system
The system comprises a host, two depth cameras, a support, a rotary table, armrests, buttons and a display screen, and the general assembly schematic diagram is shown in figure 2.
1-host computer, installing windows system and having NVIDIA GeForce GTX 1050 above display card.
2-a first depth camera for shooting lower body data, connected with the host computer using a USB cable.
And 3, a second depth camera for shooting upper body data and connecting the upper body data with the host by using a USB (universal serial bus) line.
4-a bracket, 172cm high, for fixing two depth cameras, the first depth camera being 53cm from the ground, the distance between the two depth cameras being 89 cm; the support is 125cm from the center of the turntable.
And 5, connecting the turntable with a host machine by using a serial port line, so that the tested personnel can stand and bear 150kg of load.
6-handrails, wherein the distance between the two handrails is 80cm, and the height expansion range is 70 cm-115 cm.
And 7-the button is embedded into the armrest and is in wireless connection with the host.
And 8, displaying the human body acquisition process, the three-dimensional human body model and the human body key size by using a display screen, and connecting the HDMI data line with the host.
Since the field angle of a single depth camera is limited at a distance of 125cm and is not sufficient to cover the human body, the human body is photographed by two depth cameras. And under the condition that the first depth camera can shoot foot data and the second depth camera can shoot head data, the visual angle between the two depth cameras needs to have certain overlap ratio. The overall coverage height must reach 2m to ensure that the field of view formed by the two depth cameras can cover the height of the human body, as shown in fig. 3.
The purpose of designing the rotary table is to enable the human body to rotate 360 degrees, carry out all-round human body coverage, avoid the appearance of shooting dead angles, and influence the three-dimensional human body reconstruction precision and the size calculation precision.
The purpose of designing the armrest is to keep the posture of a user unchanged, ensure that two arms are opened when the human body is scanned and facilitate the measurement of the body. And the telescopic device can be telescopic to adapt to human bodies with different heights, and the vertigo feeling generated in the rotating process of the human body is relieved.
The purpose of designing the button is to enable a measured person to control the scanning to start, and to avoid the uncomfortable feeling of the measured person caused by the presence of other people under the condition of scanning without dressing or tightly dressing.
1.2) scanning System control software
The scanning software interface includes: previewing a page, scanning a starting page, counting down a page, collecting a page, waiting for a page, measuring a successful page, measuring a failed page and the like, wherein software control corresponding to each page is detailed as follows.
(1) Preview page
The depth maps of the current two depth cameras are displayed, as well as the cylindrical outline of the region of interest (ROI). The ROI refers to a region where a human body is located, and specifically, a cylindrical range with a center of the human body as a central axis and R as a radius, where R is 40cm, as shown in fig. 4. The cylinders of the two depth cameras are independent and can be adjusted independently. The depth data of the turntable can be removed by adjusting the height of the bottom of the cylinder of the first depth camera, so that the turntable is prevented from being reconstructed and adhered to the three-dimensional human body model; by adjusting the height of the bottom of the cylinder of the second depth camera, the human body data of the boundary of the depth map can be prevented from being collected, the size of the overlapped area of the upper part and the lower part is adjusted, and the three-dimensional human body reconstruction precision is improved. Through the cylinder height of adjusting first degree of depth camera, can get rid of the degree of depth data of handrail, avoid coming out the handrail reconstruction, with three-dimensional manikin hand adhesion, influence the critical dimension and draw the precision. Generally speaking, a shot depth map is converted into point clouds according to camera internal parameters, target human body point clouds are screened according to a cylindrical ROI in a point cloud space, and then the point clouds are converted back into depth data according to the camera internal parameters to be previewed and displayed.
And adjusting the parameters according to the image displayed on the preview page until the displayed human body does not include other interference.
(2) Scanning for a start page
After the parameters of the preview depth map are adjusted, the human body can be scanned. The scanning starting page comprises a human body standing posture requirement prompting word and a 'start measuring' button, and the button can be triggered by clicking the button by a mouse or pressing the button on the armrest.
(3) Countdown page
After triggering "begin to measure" button, scanning system can not begin work immediately, but will begin to control the revolving stage rotation and two degree of depth cameras collection data after countdown a few seconds, and this kind of design is favorable to promoting user experience.
(4) Collection page
After the countdown is finished, the rotating platform starts to rotate, the two cameras start to collect data, and the two cameras enter a collection page. And displaying the depth map acquired in the human body rotation process on a page. Note that the process needs to control the two depth cameras to alternately acquire information, so that the two depth cameras are prevented from interfering with each other and affecting the accuracy of the depth map. The specific control method comprises the following steps: starting to close the laser spot projectors of the two depth cameras, opening the laser spot projector of the first depth camera while the rotary table starts to rotate, capturing depth data after waiting for 80ms, and then closing the laser spot projectors; the laser spot projector of the second depth camera is then turned on, the depth data is captured after 80ms, and then the laser spot projector is turned off. And alternating the two cameras in turn until the turntable stops rotating for 360 degrees, and stopping collecting the depth data of the two cameras.
(5) Waiting page
After the depth data are collected, three-dimensional human body reconstruction and human body key size extraction are required, and the operation enters a waiting page to wait for a calculation result.
(6) Measure success page
After the three-dimensional human body is successfully reconstructed and the human body key size is successfully extracted, a three-dimensional human body point cloud model is displayed on the left side of the interface, and key parts and numerical values of the human body are displayed on the right side of the interface.
(7) Measurement failure page
If the three-dimensional human body reconstruction fails or the human body key size extraction fails, a failure prompt pops up. And (3) adjusting hardware or software parameters according to the failure reason, reopening the scanning software, and executing the step (1).
1.3) human body scanning implementation step
The specific implementation steps of the human body scanning are described as follows:
(1) and after the system hardware is assembled, the scanning software is opened to preview the picture.
Guiding a person to be measured to stand on the rotary table, and adjusting the height of the handrail to a proper position; the tested person stands at the center of the rotary table, raises the head, lifts the chest, stands straight, is still, holds the armrests with both hands, and can completely see the oxter outline of the tested person from the front; and adjusting the ROI parameters of the human body cylinder, and removing other noise interferences such as a rotary table, a handrail and the like.
(2) The scanning starting page is entered, a measured person presses an armrest button, a motor is started after counting down for several seconds, a rotary table starts to rotate stably, meanwhile, two depth cameras start to collect data alternately, each part of a human body is scanned, a plurality of depth images are obtained, and the body is kept still as much as possible in the collecting process.
(3) The rotary table stops rotating, the two depth cameras stop collecting images, and a measured person can leave the rotary table.
(4) And analyzing and processing the acquired depth image to form a three-dimensional human body model.
(5) Extracting point cloud data in the three-dimensional human body model, calculating key dimensions of the human body, such as height, neck circumference, chest circumference, waist circumference, hip circumference and the like, and displaying the three-dimensional human body point cloud and size extraction result on an interface.
The whole measuring process is simple to operate, and the measured personnel can finish measurement independently, so that the labor cost is saved, and the individual privacy is protected.
2. Human body reconstruction method with automatic calibration
According to the invention, the Orbbec AstraPro depth camera is used, and a three-dimensional human body model can be reconstructed according to 360-degree human body data captured by the first depth camera and the second depth camera, internal references corresponding to the two depth cameras and initial external references between the two depth cameras, and the reconstructed model is shown in fig. 7.
1.1) Overall reconstruction Process
The overall reconstruction flow is shown in fig. 5. The process is described as follows:
(1) and reading in configuration parameters. Including the inside and outside parameters and reconstruction parameters of the depth camera.
The internal parameter K is as follows:
wherein f isx、fyFocal lengths in x and y directions are respectively, and the unit is a pixel; (c)x,cy) Is the principal point, the center of the image, in pixels. (f)x,fy,cx,cy) The four values represent camera parameters that the astraror depth camera can read out automatically.
The external reference RT is as follows:
wherein,
as a rotation between two camerasTurning to the matrices, all may be initially set to 0; t ═ tx,ty,tzIs a translation matrix between two cameras, t can be initially definedy,tzSet to 0, txThe value is set to the approximate distance between the two cameras in meters.
The depth image parameters include: the width and height of an image (the actual width and height of a depth map, W: 640, H: 480), and a sampling interval T (in an acquired depth image, one depth map is selected for reconstruction every T frames of images, and is generally set to T: 10).
The reconstruction parameters include: an image frame number threshold N is abnormally skipped (if the pose difference between consecutive N frames is smaller than the pose threshold, the collected data is considered incorrect, and the reconstruction is terminated. N is generally set to 10), and a pose threshold Y (the minimum pose difference between two images, generally set to Y0.02)
(2) Respectively carrying out initialization rigid registration on the depth maps acquired by the first depth camera and the second depth camera:
the continuously collected point clouds are transformed into a uniform world coordinate system through rigid alignment, and are generally used for initializing global rigid alignment. The input is a depth image sequence which is continuously acquired under the same coordinate system.
The specific process is as follows:
and (2.1) circularly reading in the depth image and converting the read depth image into point cloud.
And (2.2) carrying out point cloud pretreatment and downsampling. Relative motion exists between a human body rotating in front of the camera and the ground, and the point cloud preprocessing is mainly used for removing ground point clouds to ensure the rigid matching relationship between front and back frame point clouds as much as possible. Downsampling is mainly to use uniform grid downsampling to improve performance.
And (2.3) registering the point cloud to solve the pose of the camera. And (3) using the result of the last frame as an initial value, performing Point cloud registration, solving an Iterative Closest Point (ICP) Point once, and reconstructing or updating the Point cloud matcher.
And (2.4) point cloud merging. And merging the down-sampled point clouds by using the pose obtained by solving.
(3) And respectively carrying out global rigid registration on the point clouds after the first depth camera and the second depth camera are initialized and registered. The method comprises the steps of inputting a point cloud sequence which is continuously collected under the same coordinate system, obtaining a unified model with the minimum error through globally solving rigid transformation of key frame point clouds, and outputting the unified model. The flow chart is shown in fig. 6, and the specific process is as follows:
and (3.1) extracting the key frame of the original point cloud. And selecting key frames by using the preliminarily solved camera pose, and ensuring that the key frames are uniformly distributed in the angle as much as possible.
(3.2) creating a camera pair to form a loop. Adjacent key frames with an interval of 1 or 2 are established as a camera pair.
(3.3) the camera registers for initial ICP. And traversing the camera pair, and performing ICP registration to obtain an initial relative pose.
And (3.4) matching the point-to-point cloud neighbors by the camera. And traversing the camera pair, and performing point cloud neighbor matching by using the initial relative pose.
And (3.5) optimizing the global point-surface error. And establishing a least square optimization problem aiming at all point cloud matching, and solving a relative pose.
And (3.6) judging whether the global optimization converges. If yes, ending; otherwise, returning to the step (3.4).
(4) And according to the initial external parameters, performing point cloud registration aiming at the global rigid registration output of the two cameras. Point cloud registration is the transformation relationship of two three-dimensional data point set spaces from different coordinate systems, so that the two point sets can be unified into the same coordinate system, and the intersection areas between the two point sets are completely overlapped.
(4.1) extracting the key points of the scenes in the two data sets.
(4.2) at each keypoint, computing a feature descriptor.
(4.3) estimating correspondences based on similarities between features and locations from the feature descriptor sets and their coordinate locations in the two data sets.
And (4.4) obtaining rigid body transformation which enables the corresponding point to have minimum Root Mean Square (RMS) and obtaining rotation parameters and translation parameters.
And (4.5) converting the target point cloud by using the obtained conversion matrix.
(4.6) iterating (4.3) until a condition for terminating the iteration is met (the number of iterations or the error is less than a threshold).
(5) And performing Poisson reconstruction. And meshing the result of the two point clouds, inputting the point clouds and normal vectors thereof, and outputting a three-dimensional mesh. Poisson reconstruction is a reconstruction method of a hidden function, and an approximation surface is directly reconstructed by defining that the value inside a model is larger than zero, the value outside the model is smaller than zero, and then extracting an isosurface with the value of zero. And (3) obtaining a hidden equation represented by the surface information described by the point cloud model by solving a Poisson equation in a hidden fitting mode, and obtaining the surface model with the geometric entity information by extracting an isosurface of the equation. The reconstructed model has good geometric surface characteristics and detail characteristics.
(6) And carrying out post-processing on the three-dimensional human body model. Including operations of extracting the maximum connected domain, smoothing, removing abnormal data, and the like.
(6.1) extracting the maximum connected component
Setting the value of the coordinate position where the point cloud exists in the space as 1, and setting the value of the coordinate position where the point cloud does not exist as 0; and dividing the connected point clouds into the same block according to whether the point clouds exist at the upper, lower, front, rear, left and right positions of the point clouds, and taking the block with the largest point number after the division is finished to obtain the target human body point cloud.
(6.2) Point cloud smoothing
Setting point neighborhood number N of point cloud smoothingsAnd traversing each point cloud, solving the mean value of the point neighborhood and assigning the mean value to the current point cloud to realize smoothing. General setting NsIs 8 to 20, NsThe larger the value, the better the smoothness.
(6.3) removing abnormal data
Dividing the point cloud into different small blocks, and calculating the isolated value P of each small blockn，PnThe point cloud number of the local block where the point is located is the proportion of the whole point cloud number (the range is (0, 1)). The smaller the isolated value, the greater the probability of an isolated term; points with an isolation value less than 0.1 are considered to be isolated terms, and the points are removed.
The technical key points of the above embodiment are as follows:
human scanning device based on two degree of depth cameras, including supplying the revolving stage that the surveyor stood, two degree of depth cameras are fixed in on the support of revolving stage one side, are connected to the host computer through USB, under the condition of avoiding contacting with the surveyor, have realized 360 all-round scans to the human body through software control. The automatic calibration human body reconstruction method obtains a three-dimensional human body model through the processes of initial registration, global registration, point cloud splicing, poisson reconstruction and the like according to an acquired depth map, and extracts the key size of a human body.
In summary, the claimed application is a dual camera body scanning system, comprising: a body scanning device based on a dual depth camera; an automatic calibration human body reconstruction method.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (10)
1. A double-camera human body scanning method is characterized by comprising the following steps:
s1, receiving a signal for starting measurement input by a user, controlling a turntable to rotate and controlling a first depth camera and a second depth camera to alternately acquire depth images corresponding to each part of a measured person, wherein when the turntable rotates, the measured person rotates along with the turntable, and the first depth camera and the second depth camera are fixed on a support and do not rotate along with the turntable;
s2, processing the collected depth image to form a three-dimensional human body model corresponding to the measured person;
and S3, extracting point cloud data in the three-dimensional human body model, and calculating the key size of the measured person.
2. The dual-camera human body scanning method as claimed in claim 1, wherein said step S2, wherein said processing the acquired depth image to form the three-dimensional human body model corresponding to the measured person comprises:
s2-1, reading in configuration parameters, wherein the configuration parameters comprise internal and external parameters of the first depth camera and the second depth camera, depth image parameters and threshold parameters in a reconstruction algorithm;
s2-2, respectively carrying out initialization rigid registration on the depth images acquired by the first depth camera and the second depth camera;
s2-3, respectively carrying out global rigid registration on the point clouds after the first depth camera and the second depth camera are initialized and registered.
3. The dual-camera human body scanning method as claimed in claim 2, wherein said step S2, wherein said processing the collected depth image to form a three-dimensional human body model corresponding to the measured person further comprises;
s2-4, performing point cloud registration and splicing aiming at the global rigid registration output of the first depth camera and the second depth camera according to the initial external parameters;
s2-5, performing Poisson reconstruction: and meshing the result of the two point clouds, inputting the point clouds and normal vectors thereof, and outputting a three-dimensional mesh.
4. The dual-camera human body scanning method as claimed in claim 3, wherein said step S2, wherein the processing the acquired depth image to form the three-dimensional human body model corresponding to the measured person further comprises:
s2-6, carrying out three-dimensional human body model post-processing: the method comprises the steps of extracting a maximum connected domain, smoothing and removing abnormal data.
5. The dual-camera human body scanning method as claimed in claim 4, wherein: the rigid registration operation is initialized as follows:
s2-2-1, circularly reading in the depth image, and converting the read-in depth image into point cloud;
s2-2-2, point cloud preprocessing and downsampling are carried out, wherein relative motion exists between a human body rotating in front of a camera and the ground, ground point cloud is mainly removed in the point cloud preprocessing, so that a rigid matching relation between front and rear frame point clouds is guaranteed as far as possible, and uniform grid downsampling is mainly used in the downsampling to improve performance;
s2-2-3, performing point cloud registration to solve the camera pose, using the result of the previous frame as an initial value, performing point cloud registration, solving primary ICP, and reconstructing or updating a point cloud matcher;
and S2-2-4, point cloud combination is carried out, and the down-sampled point clouds are combined by using the pose obtained by solving.
6. The dual-camera human body scanning method as claimed in claim 4, wherein: the global rigid registration operates as follows:
s2-3-1, selecting key frames by using the preliminarily solved camera pose, and ensuring that the key frames are uniformly distributed in the angle as much as possible;
s2-3-2, establishing adjacent key frames with the interval of 1 or 2 as a camera pair;
s2-3-3, traversing the camera pair, and performing ICP registration to obtain an initial relative pose;
s2-3-4, traversing the camera pair, and performing point cloud neighbor matching by using the initial relative pose;
s2-3-5, establishing least square optimization for all point cloud matching, and solving a relative pose;
s2-3-6, judging whether the global optimization is converged, if yes, ending; otherwise, the step S2-3-4 is returned.
7. The dual-camera human body scanning method as claimed in claim 1, further comprising displaying the three-dimensional human body point cloud and the size extraction result on an interface.
8. A dual camera body scanning system, comprising:
a host;
the first depth camera is connected with the host and is used for shooting lower body data;
the second depth camera is connected with the host and is used for shooting upper half body data;
the turntable is connected with the host machine by using a serial port line and is used for a tested person to stand;
the host computer has a processor and a memory, the memory having stored therein a computer program executable to implement the method of any one of claims 1-7.
9. The dual camera body scanning system of claim 8, wherein: further comprising:
a bracket for fixing two depth cameras;
a handrail;
the button is embedded into the armrest and is in wireless connection with the host;
and the display screen displays the human body acquisition process, the three-dimensional human body model and the human body key size.
10. The dual camera body scanning system of claim 8, wherein: the computer program interface includes: preview page, scan start page, countdown page, collect page, wait page, measurement success page, and measurement failure page.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111437531.3A CN114140598A (en) | 2021-11-29 | 2021-11-29 | Double-camera human body scanning method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111437531.3A CN114140598A (en) | 2021-11-29 | 2021-11-29 | Double-camera human body scanning method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114140598A true CN114140598A (en) | 2022-03-04 |
Family
ID=80389291
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111437531.3A Pending CN114140598A (en) | 2021-11-29 | 2021-11-29 | Double-camera human body scanning method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114140598A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115414031A (en) * | 2022-10-19 | 2022-12-02 | 深圳仙库智能有限公司 | Human body parameter measuring device |
| CN116721104A (en) * | 2023-08-10 | 2023-09-08 | 武汉大学 | Live three-dimensional model defect detection method and device, electronic equipment and storage medium |
| CN117372647A (en) * | 2023-10-26 | 2024-01-09 | 天宫开物(深圳)科技有限公司 | Rapid construction method and system of three-dimensional model for building |
-
2021
- 2021-11-29 CN CN202111437531.3A patent/CN114140598A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115414031A (en) * | 2022-10-19 | 2022-12-02 | 深圳仙库智能有限公司 | Human body parameter measuring device |
| CN116721104A (en) * | 2023-08-10 | 2023-09-08 | 武汉大学 | Live three-dimensional model defect detection method and device, electronic equipment and storage medium |
| CN116721104B (en) * | 2023-08-10 | 2023-11-07 | 武汉大学 | Live three-dimensional model defect detection method and device, electronic equipment and storage medium |
| CN117372647A (en) * | 2023-10-26 | 2024-01-09 | 天宫开物(深圳)科技有限公司 | Rapid construction method and system of three-dimensional model for building |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114140598A (en) | Double-camera human body scanning method and system | |
| JP6392756B2 (en) | System and method for obtaining accurate body size measurements from a two-dimensional image sequence | |
| Li et al. | 3D self-portraits | |
| CN111292364B (en) | Method for rapidly matching images in three-dimensional model construction process | |
| CN108053476B (en) | Human body parameter measuring system and method based on segmented three-dimensional reconstruction | |
| US8823775B2 (en) | Body surface imaging | |
| Or-El et al. | Rgbd-fusion: Real-time high precision depth recovery | |
| CN105046746B (en) | A kind of digital speckle human body three-dimensional fast scanning method | |
| Bouguet et al. | 3D photography on your desk | |
| Kang et al. | 3-D scene data recovery using omnidirectional multibaseline stereo | |
| US9418475B2 (en) | 3D body modeling from one or more depth cameras in the presence of articulated motion | |
| CN106204718A (en) | A kind of simple and efficient 3 D human body method for reconstructing based on single Kinect | |
| CN109523635A (en) | A kind of non-rigid reconstruction of 3D anthropometric scanning and measurement method and device | |
| Yu et al. | A portable stereo vision system for whole body surface imaging | |
| CN110120071A (en) | A kind of depth estimation method towards light field image | |
| Yang et al. | Surface reconstruction via fusing sparse-sequence of depth images | |
| CN111292239A (en) | Three-dimensional model splicing equipment and method | |
| CN103247074A (en) | 3D (three dimensional) photographing method combining depth information and human face analyzing technology | |
| CN111340959B (en) | Three-dimensional model seamless texture mapping method based on histogram matching | |
| Gu et al. | A 3D reconstruction system for human body modeling | |
| JP5555193B2 (en) | Data processing apparatus, data processing system, and program | |
| JP2015198406A (en) | Image reader, method, and program | |
| CN101661623A (en) | Three-dimensional tracking method of deformable body based on linear programming | |
| Ran et al. | High-precision human body acquisition via multi-view binocular stereopsis | |
| Kumara et al. | Real-time 3D human objects rendering based on multiple camera details |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20221123 Address after: 518000 13 / F, joint headquarters building, high tech Zone, 63 Gaoxin South 10th Road, Binhai community, Yuehai street, Nanshan District, Shenzhen City, Guangdong Province Applicant after: XTOP 3D TECHNOLOGY (SHENZHEN) Co.,Ltd. Address before: 710000 room 1101, building C2, yunhuigu (phase II project of Xi'an software new town R & D base), 156 Tiangu 8th Road, software new town, Xi'an high tech Zone, Xi'an, Shaanxi Province Applicant before: Xi'an aobi Tuojiang Technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right |