WO2021077721A1

WO2021077721A1 - Method, apparatus and system for reconstructing three-dimensional model of human body, and readable storage medium

Info

Publication number: WO2021077721A1
Application number: PCT/CN2020/089885
Authority: WO
Inventors: 张吉; 张遥; 李竹; 王琳
Original assignee: 深圳奥比中光科技有限公司
Priority date: 2019-10-25
Filing date: 2020-05-12
Publication date: 2021-04-29
Also published as: CN110874851A

Abstract

The present application is applicable to the technical field of computer vision. Provided is a method for reconstructing a three-dimensional model of a human body, which method is applied to a depth camera. The method comprises: collecting a sequence of multiple frames of infrared images which comprise various parts of a human body; processing the sequence of infrared images to acquire a corresponding parallax image sequence or a depth image sequence; compressing and encoding the parallax image sequence and a first parameter, or compressing and encoding the depth image sequence and a second parameter; and uploading compressed data which has been compressed and encoded to a server, wherein the compressed data is used for instructing the server to decompress and decode the received compressed data, and then to reconstruct a real three-dimensional model of a real human body. By means of the present application, the efficient reconstruction of a three-dimensional model of a human body is realized.

Description

Method, device, system and readable storage medium for reconstructing three-dimensional model of human body

Technical field

This application relates to the field of computer vision technology, and in particular to a method, device, system and readable storage medium for reconstructing a three-dimensional human body model.

Background technique

Three-dimensional reconstruction is the future core basic technology for the development of computer vision. The current development and application is aimed at groups with specific shapes and characteristics, such as the human body, in film and television entertainment and life applications.

Most of the existing human body 3D reconstruction technologies are based on a large amount of depth data of the human body to be measured, so the reconstruction efficiency of the human body 3D model is not high.

Summary of the invention

The embodiments of the present application provide a method, device, system, and readable storage medium for reconstructing a three-dimensional human body model, and provide an efficient solution for reconstructing a three-dimensional human body model.

In the first aspect, an embodiment of the present application provides a method for reconstructing a three-dimensional human body model, including:

Collect multiple frames of infrared image sequences including various parts of the human body;

Processing the infrared image sequence to obtain a corresponding parallax image sequence or depth image sequence;

Performing compression coding on the disparity image sequence and the first parameter, or performing compression coding on the depth image sequence and the second parameter;

Upload the compressed data that has been compressed and encoded to a server, where the compressed data is used to instruct the server to decompress and decode the received compressed data to reconstruct a real three-dimensional human body model.

By first processing the infrared image sequence, the corresponding parallax image sequence or depth image sequence is obtained, and the parallax image sequence or depth image sequence is compressed and encoded and then uploaded to the server to reconstruct the human body 3D model. On the one hand, the depth camera converts the infrared image sequence Disparity image sequence or depth image sequence, so that the server does not need to directly process the infrared image sequence, reduces the amount of data calculation of the server, reduces the system resource occupation, and greatly improves the efficiency of human body 3D model reconstruction; on the other hand, the depth camera will By uploading the compressed data after compression and encoding to the server, the data transmission efficiency is improved, and the efficiency of the reconstruction of the three-dimensional human body model is further improved.

In the second aspect, an embodiment of the present application provides an apparatus for reconstructing a three-dimensional human body model, including:

The image acquisition unit is used to acquire multiple frames of infrared image sequences including various parts of the human body;

An image processing unit, configured to process the infrared image sequence to obtain a corresponding parallax image sequence or depth image sequence;

A compression coding unit, configured to perform compression coding on the disparity image sequence and the first parameter, or perform compression coding on the depth image sequence and the second parameter;

The data uploading unit is configured to upload compressed data that has been compressed and encoded to a server, and the compressed data is used to instruct the server to decompress and decode the received compressed data to reconstruct a real three-dimensional human body model.

In the third aspect, an embodiment of the present application provides a depth camera, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, Implement the method as described in the first aspect.

In a fourth aspect, an embodiment of the present application provides a depth camera, including the device as described in the second aspect.

In a fifth aspect, an embodiment of the present application provides a system for reconstructing a three-dimensional human body model, including a server, and the depth camera according to the third aspect or the fourth aspect, and the server is configured to compare the received compressed data After decompression and decoding, a real three-dimensional model of the human body is reconstructed.

In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium that stores a computer program that implements the method described in the first aspect when the computer program is executed by a processor.

In a seventh aspect, embodiments of the present application provide a computer program product, which when the computer program product runs on an electronic device, causes the electronic device to execute the method described in the first aspect.

It can be understood that the beneficial effects of the above second aspect to the seventh aspect can be referred to the related description in the above first aspect, which will not be repeated here.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only of the present application. For some embodiments, those of ordinary skill in the art can obtain other drawings based on these drawings without creative labor.

Fig. 1 is a schematic diagram of a system for reconstructing a three-dimensional human body model provided by an embodiment of the present application;

2 is a schematic diagram of a distribution network of a system for reconstructing a three-dimensional human body model provided by an embodiment of the present application;

3 is a schematic structural diagram of an apparatus for reconstructing a three-dimensional human body model provided by an embodiment of the present application;

4 is a schematic structural diagram of an apparatus for reconstructing a three-dimensional human body model provided by an embodiment of the present application;

Fig. 5 is a schematic structural diagram of an apparatus for reconstructing a three-dimensional human body model provided by an embodiment of the present application;

Fig. 6 is a schematic structural diagram of an apparatus for reconstructing a three-dimensional human body model provided by an embodiment of the present application;

FIG. 7 is a schematic structural diagram of an apparatus for reconstructing a three-dimensional human body model provided by an embodiment of the present application;

FIG. 8 is a schematic structural diagram of an apparatus for reconstructing a three-dimensional human body model provided by an embodiment of the present application.

Detailed ways

In order to illustrate the technical solution described in the present application, the following description will be made with reference to the drawings and in conjunction with the embodiments.

In order to enable those skilled in the art to better understand the solution of the application, the technical solutions in the embodiments of the application will be clearly and completely described below in conjunction with the drawings in the embodiments of the application. Obviously, the described embodiments are only It is a part of the embodiments of this application, not all the embodiments. Based on the embodiments in this application, for those of ordinary skill in the art, all other embodiments obtained without creative labor should fall within the protection scope of this application. It should be noted that the embodiments in this application and the features in the embodiments can be combined with each other if there is no conflict.

In the following description, for the purpose of illustration rather than limitation, specific details such as a specific system structure and technology are proposed for a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from obstructing the description of this application.

It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is said to be "connected" to another element, it can be wired to the other element or wirelessly connected to the other element, and the connection is used for data transmission.

In addition, the descriptions involving "first" or "second" in the terms in the specification, claims, and drawings of this application are only used to distinguish similar objects, and cannot be understood as indicating or implying their relative importance. Or implicitly specify the number of the indicated technical features, that is, these descriptions do not have to be used to describe a specific order or sequence. In addition, it should be understood that these descriptions can be interchanged under appropriate circumstances in order to describe the embodiments of the present application.

Please refer to FIG. 1, which shows a system for reconstructing a three-dimensional human body model provided by the present application, including a depth camera 101 connected in two-to-two communication, a client (mobile phone shown in FIG. 1) 102, and a server 103. The measurement principle of the system is: the client 102 initiates a measurement instruction to the depth camera 101, and after receiving the measurement instruction, the depth camera 101 takes a picture of the human body to collect multiple frames of parallax image sequences or depth image sequences including various parts of the human body and upload them To the server 103, the server 103 performs real three-dimensional (3D) reconstruction of the human body according to the received parallax image sequence or depth image sequence, and selects key parts on the 3D model for measurement, so as to obtain the corresponding data of the measured human body. After the measurement is completed, The final three-dimensional data is transmitted to the client 102 that initiated the measurement instruction.

It should be noted that the depth camera 101, the client 102, and the server 103 must transmit data through the network in each pair. Therefore, the client 102 needs to configure the network for the three before sending the measurement instruction to the depth camera 101. Please refer to FIG. 2, which is an implementation diagram of the distribution network process in one of the embodiments of this application. The specific process is as follows: the client 102 starts the network configuration and searches for connectable Bluetooth devices. When it finds the Deepin Camera 101, it connects to it with Bluetooth. After the connection is successful, the Deepin Camera 101 will scan the nearby WiFi QR code and generate it. The WiFi list is transmitted to the client 102 through a Bluetooth unit (not shown). The client 102 selects a WiFi and enters the WiFi password. If the connection is successful, the network configuration is completed. At this time, the Deepin Camera can use its WiFi unit (not shown) ) Access the server 103. It should be noted that, in FIG. 2, the WiFi distribution network is taken as an example for description, and the basis is only an exemplary description, and cannot be construed as a specific limitation to the present application.

In some embodiments of the present application, the depth camera may be a depth camera based on structured light, binocular, and time of flight (TOF) technology. In addition, the depth camera may also be a depth camera including a color camera module, such as a depth camera including an RGB camera module. In this way, both depth images containing depth information and color images containing rich texture information can be obtained.

In some embodiments of the present application, the client may also be a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR)/virtual reality (VR) device, a notebook computer, a super mobile personal computer ( For terminal devices such as ultra-mobile personal computer (UMPC), netbooks, and personal digital assistant (personal digital assistant, PDA), the embodiments of this application do not impose any restrictions on the specific types of electronic devices.

In some embodiments of the present application, the server may also be an independent server, a server cluster, or a distributed server, etc. The embodiments of the present application do not impose any restriction on the specific type of the server.

It is understandable that those skilled in the art can deploy according to actual needs, and the illustrations in the embodiments of the present application and the explanations corresponding to the illustrations do not constitute a limitation on the specific deployment form thereof.

FIG. 3 shows an implementation flowchart of a method for reconstructing a three-dimensional human body model provided by an embodiment of the present application. The method includes steps S110 to S130. This method is suitable for situations where the human body needs to be reconstructed in three dimensions. This method can be applied to the depth camera shown in Figure 1. The specific implementation principle of each step is as follows.

S110: Collect multiple frames of infrared image sequences including various parts of the human body.

The depth camera collects infrared images of the human body from different angles to form an infrared image sequence including various parts of the human body.

Continuing to refer to FIG. 1, in one embodiment, the image acquisition unit of the depth camera 101 includes a binocular IR camera module, wherein the baseline distance of the left and right IR cameras is 150 mm. When taking pictures of the human body, place the depth camera 101 vertically and stick it on a vertical wall, about 0.8m to 1.2m from the ground. The subject keeps standing at a preset posture, distance and position and at a preset angle Rotate, for example, the subject’s hands hang down at a certain angle in A-pose, and rotate at a distance of 1m to 2m from the depth camera. During the rotation, the depth camera uses the image acquisition unit to continuously take pictures to obtain multiple frames (eg 300 frames) Sequence of infrared images from different angles.

In one embodiment, the image acquisition unit further includes a color camera module (not shown in FIG. 1), such as an RGB camera module. In this way, both depth images containing depth information and color images containing rich texture information can be obtained. image. It is understandable that, in order to collect more accurate depth images containing various parts of the human body, the above-mentioned image collection process is preferably carried out indoors to avoid interference from ambient light and strong infrared light. It is best for the subject to wear tights, or Naked body, stand in the preset position and follow the prescribed actions to complete the measurement process.

In one embodiment, the image acquisition unit further includes a laser projection module (not shown in Figure 1). For example, the laser projection module emits a laser with a wavelength of 825 nm. At this time, the image acquisition unit collects infrared speckles. image. Therefore, when the laser projection module is in the working state, it should also detect whether there is an object within the dangerous distance range. Once an object intrusion is detected, the laser projection module should be turned off. In one embodiment, the depth camera further includes a distance measurement unit. Specifically, when the image acquisition unit is turned on, the distance measurement unit is simultaneously turned on, and when an object is detected within 1 m, the laser module is turned off. In an embodiment, the distance measuring unit may be a distance sensor or a proximity sensor or the like.

It is understandable that not only when the human body is close to the laser projection module, it will cause harm to the human body, but also when the laser power emitted by the laser projection module is high, it may also cause harm to the human body. Among them, the reason for the high power of the laser projection module may be that its diffractive optical elements (DOE) are damaged and the zero-order beam is not effectively diffracted. Therefore, before the image collection of the human body, the The integrity of the DOE should be tested. In one embodiment, a photo-diode (PD) can be placed near the DOE, for example, it can be placed obliquely 45 degrees above the top angle of the DOE to detect the amount of light (luminous intensity), where the amount of light is equal to that of the PD. The voltage across the PD is proportional. When the voltage across the PD exceeds the threshold, it is judged that the DOE is destroyed, and the laser projection module needs to be turned off at this time.

S120: Process the infrared image sequence to obtain a corresponding parallax image sequence or depth image sequence.

The depth camera processes the infrared image sequence to obtain the corresponding parallax image sequence or depth image sequence.

Specifically, the image processing unit of the depth camera processes the infrared image sequence to obtain the corresponding parallax image sequence or depth image sequence.

In one embodiment, the image processing unit includes a parallax image acquisition unit and a depth image acquisition unit.

The parallax image acquisition unit is used to process the above-mentioned infrared image sequence to obtain the parallax image sequence, for example, the deviation of the spatial point in the two infrared images can be calculated according to the stereo matching algorithm to obtain a series of parallax images, or to calculate the reference speckle The deviation between the image and the acquired infrared speckle image to obtain a series of parallax images.

The depth image acquiring unit is configured to process the disparity image sequence to acquire the depth image sequence, for example, the disparity image sequence may be further converted into a depth image sequence according to the mapping relationship between the disparity and the depth.

S130: If it is determined that the human body is at a preset posture and distance according to the parallax image sequence or the depth image sequence, send the parallax image sequence or the depth image sequence to a server, and the parallax image sequence or the depth image sequence The depth image sequence is used to instruct the server to reconstruct a three-dimensional model of a real human body.

The depth camera judges that the human body is in a preset posture and distance according to the parallax image sequence or the depth image sequence. If it is determined that the human body is in a preset posture and distance according to the parallax image sequence or the depth image sequence, the parallax image sequence or the depth image sequence is sent to the server, and the server performs the reconstruction of the human body 3D model according to the parallax image sequence or the depth image sequence sent by the depth camera.

The depth camera includes a position detection unit and a data upload unit. The pose detection unit judges whether the human body is in a preset posture and distance based on the first frame or the first few depth images, so that when the human body is in the preset posture and distance, the data upload unit will The parallax image sequence or the depth image sequence is sent to a server, and the server receives the parallax image sequence or the depth image sent by a depth camera to reconstruct a three-dimensional human body model.

In an example, the position detection unit mainly detects whether the following two conditions meet the requirements: 1) Whether the human body is in the central area of the depth image and occupies more than 80% of the screen; 2) Whether the human body is hanging down at a certain angle with both hands Standing in a posture. For condition 1), the position detection unit can use the image segmentation algorithm to segment the target area (including the area of the human body) and the background area, and calculate the distance between the center of the target area and the geometric center of the depth image, when the distance is less than the preset value , It is judged that the human body is in the central area of the depth image, and at the same time, the proportion value of the target area in the entire depth image is calculated, and it is judged whether it is greater than 80%; for condition 2), the position detection unit can process the first frame or Perform key point detection (including but not limited to head, waist, hand, elbow, shoulder joint points and soles of feet, etc.) in the first few frames of depth images to extract human bone data and calculate the angle between the arm and the torso. Within a preset range, for example, 15 to 30 degrees, condition 2) is satisfied. When the above two conditions are met, it can be determined that the human body is in the preset posture and distance. At this time, the image acquisition unit can continue to image the human body.

In some other embodiments, the depth camera further includes a reminder unit. When the posture detection unit determines that the human body is not in the preset posture and distance, the reminder unit sends out an adjustment reminder to adjust the posture and distance of the human body until it continues to collect data. The sequence of parallax images or the sequence of depth images determines that the human body is in a preset posture and distance.

In one embodiment, when the human body is not in the preset posture and distance, the reminder unit will issue a related reminder according to the current posture and/or distance of the human body, for example, a broadcast: "Please step forward/back/left/right" Or "open your arms and keep your posture", the subjects can perform corresponding operations according to the broadcast content. For example, when the distance between the standing position of the human body and the depth camera (or laser projection module) is within the dangerous distance range, the reminding unit will remind the subject to move backward through a broadcast. In this process, the image acquisition unit will continuously collect infrared images of the subject to judge the rationality of the current subject’s standing position. Specifically, based on the continuously collected parallax image sequence or depth image sequence, in one embodiment, the reminding unit may be a speaker.

In the embodiment of this application, by first processing the infrared image sequence, the corresponding parallax image sequence or depth image sequence is obtained, and then the parallax image sequence is determined according to the parallax image sequence or depth image sequence when the human body is in a preset posture and distance. Or the depth image sequence is sent to the server for human body 3D model reconstruction. On the one hand, the depth camera converts the infrared image sequence into a parallax image sequence or a depth image sequence, so that the server does not need to directly process the infrared image sequence, reducing the amount of data calculation on the server. The system resource occupation is reduced, and the efficiency of human body 3D model reconstruction is greatly improved. On the other hand, the depth camera sends the parallax image sequence or depth image sequence that determines that the human body is in the preset posture and distance to the server for human body 3D model reconstruction. The accuracy and completeness of data collection are improved, and the accuracy of the reconstruction of the three-dimensional model of the human body is further improved.

It is understandable that errors will inevitably occur during the manufacturing and assembly of depth camera components, which will also bring systematic errors to the measured depth value. Therefore, based on the method embodiment shown in FIG. 2 above Before sending the disparity image sequence or the depth image sequence to the server, the method further includes: performing multi-distance calibration on the depth data in the depth image sequence.

In an embodiment, the depth camera further includes a multi-distance calibration unit, which is used to perform multi-distance calibration on the depth data in the depth image sequence to reduce the systematic error of the measurement.

It should be noted that not all collected image frames are suitable for three-dimensional human body reconstruction. Therefore, based on the method embodiment shown in FIG. 2, before sending the parallax image sequence or the depth image sequence to the server, it also includes: Multi-distance calibration is performed on the depth data in the depth image sequence; the calibrated depth image sequence is screened to obtain the screened depth image sequence.

In one embodiment, in addition to the multi-distance calibration unit, the depth camera also includes a valid frame detection unit. The valid frame detection unit is used to screen the calibrated depth image sequence to remove redundant frames and further reduce The amount of data for subsequent 3D reconstruction.

In order to further reduce the amount of data for subsequent 3D reconstruction calculations and increase the calculation speed, on the basis of the foregoing method embodiment, after obtaining the selected depth image sequence, the method further includes: masking the selected depth image sequence Process to obtain a sequence of deep human mask images.

In one embodiment, the depth camera further includes a depth human body mask image acquisition unit for performing mask processing on the above-mentioned filtered depth image sequence to acquire a depth human body mask image sequence. Specifically, a pre-made sensory image sequence may be used. The interest region mask is multiplied by the above-mentioned depth image sequence to remove the background area to obtain the depth human body mask image sequence.

On the basis of the foregoing method embodiment, after acquiring the depth human body mask image sequence, the method further includes: calculating the depth human body mask image sequence to obtain the parallax human body mask image sequence.

In one embodiment, the above-mentioned depth camera further includes a parallax human body mask image acquisition unit, configured to calculate the above-mentioned depth human body mask image to obtain a parallax human body mask image sequence. Compared with the depth image data, the dynamic range of the parallax image data is relatively small (each pixel can be expressed in 12 bits or less), the change between neighboring pixels is small, a lower bit rate can be obtained, and the transmission speed can be further improved.

On the basis of the foregoing method embodiment, after obtaining the screened depth image sequence, the method further includes: calculating the screened depth image sequence to obtain skeleton information of the human body.

In an embodiment, the depth camera further includes a skeleton acquisition unit for calculating the above-mentioned filtered depth image sequence to obtain human body skeleton information, which is mainly used for subsequent three-dimensional reconstruction.

In some other embodiments, in order to shorten the data transmission time and further improve the efficiency of the three-dimensional human body model, the depth camera may upload the acquired data to the server through compression coding.

Specifically, the depth camera also includes a compression encoding unit for compressing and encoding the depth human mask image sequence, the second parameters (including the internal parameters of the depth camera), and the human skeleton information, and compressing them to 10% of the original data size. % Is then uploaded to the server through the data upload unit, which shortens the data transmission time, thereby further improving the efficiency of the three-dimensional human body model. It should be noted that uploading the depth image data does not need to upload multiple parameters at the same time. The system design is relatively simple, but the dynamic range is relatively large, and the adjacent pixels change greatly at a long distance, which is not conducive to compression coding.

Therefore, in some other embodiments, the compression encoding unit compresses and encodes the parallax human body mask image sequence, the first parameters (including the internal parameters of the depth camera, the parallax conversion depth parameter, and the multi-distance calibration parameter), and the human skeleton information, and It is compressed to 10% of the original data size and then uploaded to the server through the data upload unit. Compared with depth image data, the dynamic range of parallax image data is relatively small (each pixel can be expressed in 12bit or less), the change between neighboring pixels is small, and lower bit rates can be obtained, but multi-distance calibration needs to be uploaded Parameters and other additional parameters.

When the depth camera uploads the acquired data to the server through compression and encoding, the server first decodes and decompresses the received compressed data to obtain the parallax human body mask image sequence, the first parameter, and the human body skeleton information; or the deep human body The mask image sequence, the second parameter, and the human body skeleton information are used to perform three-dimensional reconstruction of the human body through the data obtained after the above-mentioned decoding and decompression.

In one embodiment, the server includes a decoding and decompression unit and a three-dimensional reconstruction unit. After the server receives the compressed data, the received compressed and encoded data is decoded and decompressed by the decoding and decompression unit to obtain the parallax human body mask image sequence, the first parameters, and the human body skeleton information; or the depth human body mask image sequence, The second parameter and the human skeleton information, and the data obtained after the above-mentioned decoding and decompression are subjected to the three-dimensional reconstruction of the human body by the three-dimensional reconstruction unit. It is understandable that when the server receives a parallax human mask image sequence, it also needs to convert the parallax human mask image sequence into a depth human mask image sequence according to the parallax conversion depth parameter, and then according to the depth camera internal parameters and multi-distance The calibration parameters correct the depth data in the depth human body mask image sequence to reduce the system measurement error.

In other embodiments of the present application, the server further includes a data measuring unit, which is used to measure the dimensions of the required body part and push the measurement result to the client. In one embodiment, the measurement locations include, but are not limited to: chest circumference, waist circumference, hip circumference, upper arm circumference, lower arm circumference, thigh circumference, calf circumference, and the like.

In this way, in the entire system, data collection, data processing, and data display are carried out in three different devices, which can improve the speed and accuracy of three-dimensional data measurement.

It should be understood that the size of the sequence number of each step in the foregoing embodiment does not mean the order of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

Corresponding to the method for reconstructing a three-dimensional human body model described in the above embodiment, FIG. 4 shows a structural block diagram of a device for reconstructing a three-dimensional human body model provided by an embodiment of the present application, and the device for reconstructing a three-dimensional human body model is configured in a depth camera. For ease of description, only the parts related to the embodiments of the present application are shown.

Referring to Figure 4, the device includes:

The image acquisition unit 41 is used to acquire multiple frames of infrared image sequences including various parts of the human body;

The image processing unit 42 is configured to process the infrared image sequence to obtain a corresponding parallax image sequence or depth image sequence;

The data uploading unit 43 is configured to send the parallax image sequence or the depth image sequence to the server if it is determined that the human body is at a preset posture and distance according to the parallax image sequence or the depth image sequence. The image sequence or the depth image sequence is used to instruct the server to reconstruct a three-dimensional model of a real human body.

Optionally, on the basis of the embodiment shown in FIG. 4, as shown in FIG. 5, the device further includes:

The position detection unit 44 is configured to determine whether the human body is in a preset posture and distance according to the parallax image sequence or the depth image sequence;

The reminding unit 45 is configured to send an adjustment reminder if it is determined according to the parallax image sequence or the depth image sequence that the human body is not in the preset posture and distance, until according to the continuously collected parallax image sequence or the depth The image sequence determines that the human body is in a preset posture and distance.

Optionally, on the basis of the embodiment shown in FIG. 5, as shown in FIG. 6, the device further includes:

The multi-distance calibration unit 46 is configured to perform multi-distance calibration on the depth data in the depth image sequence.

Optionally, on the basis of the embodiment shown in FIG. 5, as shown in FIG. 7, the device further includes:

The multi-distance calibration unit 46 is configured to perform multi-distance calibration on the depth data in the depth image sequence;

The effective frame detection unit 47 is configured to screen the calibrated depth image sequence to obtain the screened depth image sequence.

Optionally, on the basis of the embodiment shown in FIG. 7, as shown in FIG. 8, the device further includes:

The skeleton obtaining unit 48 is configured to calculate the depth image sequence after screening to obtain skeleton information of the human body.

The depth human body mask image sequence acquiring unit 49 is configured to perform mask processing on the screened depth image sequence to acquire the depth human body mask image sequence.

The parallax human body mask image sequence acquiring unit 50 is configured to calculate the depth human body mask image sequence to obtain the parallax human body mask image sequence.

The compression encoding unit 51 is used for compressing and encoding the deep human body mask image sequence, the second parameter, and the human body skeleton information; or for compressing and encoding the parallax human body mask image sequence, the first parameter, and the human body skeleton information, Get compressed coded data.

Correspondingly, the data uploading unit 43 is configured to upload the compressed coded data to the server, and the compressed coded data is used to instruct the server to reconstruct the three-dimensional model of the human body.

It should be noted that the information exchange and execution process between the above-mentioned units are based on the same idea as the method embodiment of this application, and its specific functions and technical effects can be found in the method embodiment section. Go into details again.

Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the division of the above functional units and modules is used as an example. In practical applications, the above functions can be allocated to different functional units and modules as needed. Module completion, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist alone physically, or two or more units can be integrated into one unit. The above-mentioned integrated units can be hardware-based Formal realization can also be realized in the form of software functional units. In addition, the specific names of the functional units and modules are only used to facilitate distinguishing from each other, and are not used to limit the protection scope of the present application. For the specific working process of the units and modules in the foregoing system, reference may be made to the corresponding process in the foregoing method embodiment, which will not be repeated here.

It should be noted that the information interaction and execution process between the above-mentioned devices/units are based on the same concept as the method embodiment of this application, and its specific functions and technical effects can be found in the method embodiment section. I won't repeat it here.

Those skilled in the art can clearly understand that for the convenience and conciseness of description, only the division of the above functional units and modules is used as an example. In practical applications, the above functions can be allocated to different functional units and modules as required. Module completion, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist alone physically, or two or more units can be integrated into one unit. The above-mentioned integrated units can be hardware-based Formal realization can also be realized in the form of software functional units. In addition, the specific names of the functional units and modules are only used to facilitate distinguishing from each other, and are not used to limit the protection scope of the present application. For the specific working process of the units and modules in the foregoing system, reference may be made to the corresponding process in the foregoing method embodiment, which will not be repeated here.

The embodiments of the present application also provide a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in each of the foregoing method embodiments can be realized.

The embodiments of the present application provide a computer program product. When the computer program product runs on an electronic device, the electronic device can realize the steps in the foregoing method embodiments when the electronic device is executed.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the implementation of all or part of the processes in the above-mentioned embodiment methods in this application can be accomplished by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by the processor, the steps of the foregoing method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms. The computer-readable medium may at least include: any entity or device capable of carrying computer program code to the photographing device/terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (Random Access Memory, RAM), electric carrier signal, telecommunications signal, and software distribution medium. Such as U disk, mobile hard disk, floppy disk or CD-ROM, etc. In some jurisdictions, in accordance with legislation and patent practices, computer-readable media cannot be electrical carrier signals and telecommunication signals.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail or recorded in an embodiment, reference may be made to related descriptions of other embodiments.

A person of ordinary skill in the art may realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.

In the embodiments provided in this application, it should be understood that the disclosed apparatus/network equipment and method may be implemented in other ways. For example, the device/network device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units. Or components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, a person of ordinary skill in the art should understand that it can still implement the foregoing The technical solutions recorded in the examples are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in Within the scope of protection of this application.

Claims

A method for reconstructing a three-dimensional human body model, which is characterized in that it includes:

Collect multiple frames of infrared image sequences including various parts of the human body;

Processing the infrared image sequence to obtain a corresponding parallax image sequence or depth image sequence;

Performing compression coding on the disparity image sequence and the first parameter, or performing compression coding on the depth image sequence and the second parameter;

Upload the compressed data that has been compressed and encoded to a server, where the compressed data is used to instruct the server to decompress and decode the received compressed data to reconstruct a real human three-dimensional model.
The method according to claim 1, wherein if it is determined that the human body is at a preset posture and distance according to the parallax image sequence or the depth image sequence, then the pairing of the parallax image sequence and the first The parameters are compressed and encoded, or the depth image sequence and the second parameter are compressed and encoded.
The method according to claim 1 or 2, characterized in that, after acquiring the corresponding disparity image sequence or depth image sequence, the method further comprises:

If it is determined according to the parallax image sequence or the depth image sequence that the human body is not in the preset posture and distance, an adjustment reminder is issued until the human body is determined according to the continuously collected parallax image sequence or the depth image sequence In a preset position and distance.
The method according to claim 1 or 2, wherein the compressed data is further used to instruct the server to measure the three-dimensional model of the real human body after reconstructing the three-dimensional model of the real human body to obtain the three-dimensional model of the human body. data.
The method according to claim 1 or 2, characterized in that, before compressing and encoding the disparity image sequence and the first parameter, or compressing and encoding the depth image sequence and the second parameter, the method further comprises:

Performing multi-distance calibration on the depth data in the depth image sequence;

or

Multi-distance calibration is performed on the depth data in the depth image sequence; the calibrated depth image sequence is screened to obtain the screened depth image sequence.
The method according to claim 5, characterized in that, after obtaining the selected depth image sequence, the method further comprises:

Calculating the selected depth image sequence to obtain skeleton information of the human body;

or

Performing mask processing on the selected depth image sequence to obtain a depth human body mask image sequence;

or

Mask processing is performed on the selected depth image sequence to obtain a depth human body mask image sequence; the depth human body mask image sequence is calculated to obtain a parallax human body mask image sequence.
The method according to claim 1 or 2, wherein the first parameter includes: internal parameters of the depth camera, parallax to depth parameters, and multi-distance calibration parameters; the second parameter includes the depth camera Internal reference.
A device for reconstructing a three-dimensional human body model, characterized in that it comprises:

The image acquisition unit is used to acquire multiple frames of infrared image sequences including various parts of the human body;

An image processing unit, configured to process the infrared image sequence to obtain a corresponding parallax image sequence or depth image sequence;

A compression coding unit, configured to perform compression coding on the disparity image sequence and the first parameter, or perform compression coding on the depth image sequence and the second parameter;

The data uploading unit is configured to upload compressed data that has been compressed and encoded to a server, and the compressed data is used to instruct the server to decompress and decode the received compressed data to reconstruct a real three-dimensional human body model.
A system for reconstructing a three-dimensional human body model, comprising a server and a depth camera, the server is used to decompress and decode the received compressed data to reconstruct a real three-dimensional human body model, and the depth camera includes 8 said device.
A computer-readable storage medium storing a computer program, wherein the computer program implements the method according to any one of claims 1 to 7 when the computer program is executed by a processor.