CN114187343A - 3D data acquisition method and device and electronic equipment - Google Patents

3D data acquisition method and device and electronic equipment Download PDF

Info

Publication number
CN114187343A
CN114187343A CN202111547657.6A CN202111547657A CN114187343A CN 114187343 A CN114187343 A CN 114187343A CN 202111547657 A CN202111547657 A CN 202111547657A CN 114187343 A CN114187343 A CN 114187343A
Authority
CN
China
Prior art keywords
bone
vectors
vector
data acquisition
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111547657.6A
Other languages
Chinese (zh)
Inventor
孙梦南
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Ezviz Network Co Ltd
Original Assignee
Hangzhou Ezviz Network Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Ezviz Network Co Ltd filed Critical Hangzhou Ezviz Network Co Ltd
Priority to CN202111547657.6A priority Critical patent/CN114187343A/en
Publication of CN114187343A publication Critical patent/CN114187343A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the application provides a 3D data acquisition method, a device and electronic equipment, wherein the method comprises the following steps: acquiring a plurality of 3D key points of a data acquisition object and a plurality of groups of key point coordinate information of the data acquisition object; each group of key point coordinate information comprises coordinate information of each 3D key point of the same time point data acquisition object; for each group of key point coordinate information, determining a group of initial skeleton vectors of the corresponding data acquisition object; performing coordinate conversion processing on each group of initial skeleton vectors to obtain a group of corresponding target skeleton vectors; and generating bone assembly data and bone rotation data of the data acquisition object according to the obtained multiple groups of target bone vectors. According to the embodiment of the application, through coordinate conversion processing, a plurality of skeleton vectors of the data acquisition object can realize operations such as movement, assembly, rotation and the like, so that a large amount of 3D data of the data acquisition object in different postures is obtained by utilizing a plurality of groups of key point coordinate information with small data volume.

Description

3D data acquisition method and device and electronic equipment
Technical Field
The present application relates to the field of data generation, the field of 3D, and the field of attitude estimation, and in particular, to a method and an apparatus for acquiring 3D data, and an electronic device.
Background
With the development of science and technology, the 3D technology is more and more widely applied. In the application of 3D technology, it is often necessary to generate a large amount of 3D data for a data acquisition object, so as to improve the 3D effect of the data acquisition object.
However, acquiring 3D data requires high performance of the data acquisition device, and acquiring 3D data by acquiring images of the data acquisition object in different postures is inefficient, and the number of 3D data that can be acquired is small.
Disclosure of Invention
An object of the embodiments of the present application is to provide a method, an apparatus, and an electronic device for acquiring 3D data, so as to solve the problems of low efficiency and small quantity of 3D data acquisition.
In order to solve the above technical problem, the embodiment of the present application is implemented as follows:
in a first aspect, an embodiment of the present application provides a 3D data acquisition method, including:
acquiring a plurality of 3D key points of a data acquisition object and a plurality of groups of key point coordinate information of the data acquisition object;
each group of key point coordinate information comprises coordinate information of each 3D key point of the same time point data acquisition object;
for each group of key point coordinate information, determining a group of initial skeleton vectors of the corresponding data acquisition object;
performing coordinate conversion processing on each group of initial skeleton vectors to obtain a group of corresponding target skeleton vectors;
and generating bone assembly data and bone rotation data of the data acquisition object according to the obtained multiple groups of target bone vectors.
In a second aspect, an embodiment of the present application provides a 3D data acquisition apparatus, including:
the processor is used for acquiring a plurality of 3D key points of the data acquisition object and a plurality of groups of key point coordinate information of the data acquisition object; each group of key point coordinate information comprises coordinate information of each 3D key point of the same time point data acquisition object; for each group of key point coordinate information, determining a group of initial skeleton vectors of the corresponding data acquisition object; performing coordinate conversion processing on each group of initial skeleton vectors to obtain a group of corresponding target skeleton vectors; and generating bone assembly data and bone rotation data of the data acquisition object according to the obtained multiple groups of target bone vectors.
In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory electrically connected with the processor. The memory stores a computer program, and the processor is used for calling and executing the computer program from the memory to realize the steps of the 3D data acquisition method.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program, when executed by a processor, implements the steps of the 3D data acquisition method described above.
In an embodiment of the present application, a 3D data acquisition method includes: acquiring a plurality of 3D key points of a data acquisition object and a plurality of groups of key point coordinate information of the data acquisition object; each group of key point coordinate information comprises coordinate information of each 3D key point of the same time point data acquisition object; for each group of key point coordinate information, determining a group of initial skeleton vectors of the corresponding data acquisition object; performing coordinate conversion processing on each group of initial skeleton vectors to obtain a group of corresponding target skeleton vectors; and generating bone assembly data and bone rotation data of the data acquisition object according to the obtained multiple groups of target bone vectors. According to the embodiment of the application, through coordinate conversion processing, a plurality of skeleton vectors of the data acquisition object can realize operations such as movement, assembly, rotation and the like, so that a large amount of 3D data of the data acquisition object in different postures is obtained by utilizing a plurality of groups of key point coordinate information with small data volume.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without any creative effort.
Fig. 1 is a schematic flowchart of a 3D data acquisition method provided in an embodiment of the present disclosure;
fig. 2A is a schematic diagram of a first set of target bone vectors and a second set of target bone vectors to be assembled in a 3D data acquisition method provided in an embodiment of the present disclosure;
FIG. 2B is a first schematic diagram of a set of bone assembly vectors provided by an embodiment of the present disclosure;
fig. 2C is a second schematic diagram of a set of bone assembly vectors provided by an embodiment of the present disclosure;
fig. 3 is a schematic flow chart of another 3D data acquisition method provided in the embodiments of the present disclosure;
fig. 4 is a schematic diagram illustrating a module composition of a 3D data acquisition apparatus according to an embodiment of the present disclosure;
fig. 5 is a schematic composition diagram of an electronic device provided in an embodiment of the present specification.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Fig. 1 is a schematic flow chart of a 3D data acquisition method provided in one or more embodiments of the present disclosure, where the method specifically includes the following steps:
102, acquiring a plurality of 3D key points of a data acquisition object and a plurality of groups of key point coordinate information of the data acquisition object; each set of the key point coordinate information includes coordinate information of each 3D key point of the same time point data acquisition object.
The data acquisition object can be a human body, an animal or other movable objects.
The method comprises the steps of obtaining a plurality of 3D key points of a data acquisition object, wherein during specific implementation, shooting operation can be carried out through a camera, the 3D key points of the data acquisition object are detected from a frame of image obtained through shooting by using an algorithm model of the 3D posture key points, and the number and coordinate information of the 3D key points are corrected manually.
In an embodiment, after step 102 is executed, any one set of acquired coordinate information of the key points may be modified by a marking tool according to a user operation, so that the values of the acquired coordinate information of the key points are more accurate. The modification method may be to modify the coordinate information of any one 3D key point, to increase the 3D key point, to decrease the 3D key point, or the like.
The method includes the steps of obtaining a plurality of 3D key points of a data collection object, and during specific implementation, the data collection object can be provided with wearable equipment on the body, so that the plurality of 3D key points are obtained.
The method comprises the steps of obtaining a plurality of 3D key points of a data acquisition object, wherein in the concrete implementation, shooting operation can be carried out through a camera with a depth information acquisition function, a plurality of 2D key points of the data acquisition object are detected from a frame of image obtained through shooting by utilizing an algorithm model of the 2D posture key points, and then the 2D key points are converted into the 3D key points by combining depth information obtained when the camera with the depth information acquisition function shoots the image.
Cameras with depth information collection functionality include, but are not limited to, binocular cameras, RGBD depth cameras, and laser cameras.
In one embodiment, the camera with the depth information acquisition function can be calibrated. Calibrating a camera refers to acquiring internal parameters of the camera. The internal parameters include, but are not limited to, camera focal length, camera offset, distortion parameters of the camera.
The number of the 3D key points may be 17, or may also be a preset number, and the embodiment of the present application does not specially limit the number of the 3D key points.
The method includes the steps that a plurality of 3D key points of a data acquisition object and a plurality of groups of key point coordinate information of the data acquisition object are obtained, during specific implementation, shooting operation can be carried out through a camera with a depth information acquisition function, a section of video data including a plurality of frames of images is obtained, in the plurality of frames of images included in the video data, the plurality of 3D key points of the data acquisition object can be detected in each frame of image, and each 3D key point is located, so that the coordinate information of each 3D key point is obtained.
The coordinate information of the 3D keypoint may be coordinate information in a camera coordinate system, and the coordinate information of the 3D keypoint may be coordinate information in a cartesian coordinate system, for example, the coordinate information of the 3D keypoint P is (x1, y1, z 1).
Taking the data acquisition object as a human body as an example, the designated person photographed by the camera can swing a plurality of preset postures in the photographing process, for example, a posture that both hands lift up, or a posture that both hands fork and waist, or a posture that one foot stands, and the like. The video data includes each frame of image corresponding to a pose of a designated person being photographed. In the video data, when an interval between time points corresponding to the two frames of images is smaller than a preset time interval threshold, the postures of the designated person in the two frames of images may be regarded as the same posture.
In the multiple sets of key point coordinate information of the data acquisition object, each set of key point coordinate information includes coordinate information of each 3D key point of the data acquisition object at the same time point, that is, each set of key point coordinate information includes coordinate information of each 3D key point of the data acquisition object in a frame of image corresponding to the set of key point coordinate information in the video data.
Taking the data acquisition object as a human body as an example, the plurality of 3D key points of the human body may include, but are not limited to: neck, head, left shoulder, left elbow, right shoulder, right elbow, left hip, left knee, left foot, right hip, right knee, right foot. The set of key point coordinate information of the human body may be coordinate information of the neck, head, left shoulder … … and right foot of the human body under the camera coordinate system at the same time point.
And step 104, determining a group of initial skeleton vectors of the corresponding data acquisition object for each group of key point coordinate information.
The initial set of bone vectors of the data acquisition object may be a plurality of bone vectors of the data acquisition object in the camera coordinate system at the same time point, the plurality of bone vectors in the initial set of bone vectors being determined by a corresponding set of the keypoint coordinate information.
Optionally, for each set of keypoint coordinate information, determining a set of initial bone vectors of the corresponding data acquisition object comprises: determining a father node set and a child node set in a plurality of 3D key points of a data acquisition object; determining a plurality of main trunk vectors and a plurality of sub trunk vectors of the data acquisition object according to the parent node set, the child node set and the coordinate information of each 3D key point in each group of key point coordinate information, and using the main trunk vectors and the sub trunk vectors as a group of initial skeleton vectors of the data acquisition object corresponding to each group of key point coordinate information; each main torso vector is associated with at least one sub-torso vector.
A parent node set and a child node set are determined from a plurality of 3D key points of a data acquisition object.
Taking a data acquisition object as a human body as an example, 9 trunks can be defined according to a plurality of 3D key points of the human body, each trunk is composed of a father node and a child node, and the father node points to the child node. One trunk corresponds to one skeleton vector, and the skeleton vector corresponding to one trunk can be obtained by calculating the coordinate information of the parent node and the coordinate information of the child node.
The 9 trunks are respectively:
(1) neck & head
(2) Left shoulder-
(3) Left elbow, left wrist
(4) Right shoulder-
(5) Right elbow (wrist) ─ Right wrist
(6) Left hip-
(7) Left knee (left knee) ─ left foot
(8) Right hip-
(9) Right Knee (right knee) ─ Right foot
The left elbow can be a child node in the torso, namely 'left shoulder > left elbow', or a father node in the torso, namely 'left elbow > left wrist', and the right elbow, the left knee and the right knee are similar to the left elbow and are not spider-threaded.
The set of parent nodes may be: neck, left shoulder, left elbow, right shoulder, right elbow, left hip, left knee, right hip, right knee; the set of child nodes may be: head, left elbow, left wrist, right elbow, right wrist, left knee, left foot, right knee, right foot.
Of the 9 torsos, a main torso and sub-torsos can be defined, the main torso can include "left shoulder- > left elbow," "right shoulder- > right elbow," "left hip- > left knee," and "right hip- > right knee"; the sub-torso may include "left elbow- > left wrist," right elbow- > right wrist, "" left knee- > left foot, "" right knee- > right foot, "and" neck- > head.
Wherein, the main torso "left shoulder- > left elbow" can be associated with the sub-torso "left elbow- > left wrist"; main torso "right shoulder- > right elbow" may be associated with sub-torso "right elbow- > right wrist"; main torso "left hip- > left knee" may be associated with sub-torso "left knee- > left foot"; the main torso "right hip- > right knee" may be associated with the sub-torso "right knee- > right foot".
Under the condition that the sub-trunk changes, the associated main trunk cannot be influenced, but under the condition that the main trunk changes, the associated sub-trunk is driven to change.
According to the parent node set, the child node set and the coordinate information of each 3D key point in each set of key point coordinate information, the coordinate information of each parent node and the coordinate information of each child node of the same time point data acquisition object can be determined, for example, the coordinate information of the left shoulder of the parent node is (x2, y2, z2), the coordinate information of the left elbow of the child node is (x3, y3, z3), and then according to the coordinate information of the parent node and the corresponding coordinate information of the child node, a bone vector corresponding to a trunk formed by the parent node pointing to the child node can be determined, for example, the bone vector corresponding to the trunk "left shoulder- > left elbow" is (x3-x2, y3-y2, z3-z 2).
If the trunk is a main trunk, the corresponding skeleton vector is a main trunk vector; if the trunk is a sub-trunk, the corresponding skeleton vector is a sub-trunk vector.
According to the father node set, the child node set and the coordinate information of each 3D key point in each group of key point coordinate information, a plurality of main trunk vectors and a plurality of child trunk vectors of the same time point data acquisition object can be determined and used as a group of initial skeleton vectors of the data acquisition object corresponding to each group of key point coordinate information.
And 106, performing coordinate conversion processing on each group of initial skeleton vectors to obtain a corresponding group of target skeleton vectors.
The target bone vector may be a bone vector in a local coordinate system, and the target bone vector may be a bone vector in a spherical coordinate system.
The initial skeleton vector is converted from a cartesian coordinate system in a camera coordinate system to a target skeleton vector in a spherical coordinate system in a local coordinate system, and two coordinate conversion processes are required.
A set of target bone vectors may acquire multiple bone vectors of a spherical coordinate system of the subject in the local coordinate system for the same point-in-time data. Each set of target bone vectors is obtained from a set of initial bone vectors through a coordinate transformation process, so that each set of initial bone vectors corresponds to a set of target bone vectors.
Taking a data acquisition object as a human body as an example, by adopting a local coordinate system, the operations of moving, splicing, rotating and the like of limbs can be performed through the relative positions of joints of the human body. If a camera coordinate system is employed, it is difficult to achieve the aforementioned various operations.
Optionally, the initial bone vector comprises a bone vector in a camera coordinate system; performing coordinate transformation processing on each group of initial bone vectors to obtain a corresponding group of target bone vectors, wherein the coordinate transformation processing comprises the following steps: constructing at least one local coordinate system according to the data acquisition object; converting each bone vector in each set of initial bone vectors from a camera coordinate system to a local coordinate system; normalizing each skeleton vector to obtain an azimuth angle and a polar angle of the skeleton vector under a spherical coordinate system with a radius of a preset numerical value; and counting the azimuth angle and the polar angle of each bone vector to obtain a group of target bone vectors of the data acquisition object corresponding to each group of initial bone vectors.
The initial bone vector comprises a bone vector in a camera coordinate system.
At least one local coordinate system is constructed from the data acquisition object. Taking the data acquisition object as a human body as an example, a local coordinate system of the upper body and a local coordinate system of the lower body can be constructed. After the local coordinate system of the upper body and the local coordinate system of the lower body are constructed, the bone vector corresponding to the upper body of the human body in each set of initial bone vectors may be converted from the camera coordinate system to the local coordinate system of the upper body, and the bone vector corresponding to the lower body of the human body in each set of initial bone vectors may be converted from the camera coordinate system to the local coordinate system of the lower body.
The local coordinate system of the upper body may be:
A1vector quantity: left shoulder to right shoulder;
B1vector quantity: hip to neck (or spine to neck, adjustable according to the actual 3D keypoint location);
C1vector quantity: a. the1Vector sum B1The vector product of the vectors.
The local coordinate system of the upper body is then orthogonalized by Schmidt orthogonalization (Schmidt orthogonalization).
Schmidt orthogonalization is adopted to solve a method of Euclidean space orthogonal basis. An orthogonal vector set beta 1, beta 2, … …, beta m is obtained from vector sets alpha 1, alpha 2, … …, alpha m which are independent of arbitrary linearity in Euclidean space, the vector sets beta 1, beta 2, … …, beta m are equivalent to the vector sets alpha 1, alpha 2, … …, alpha m, and each vector in the orthogonal vector set is unitized to obtain a standard orthogonal vector set.
In obtaining an orthogonalized local coordinate system (A)1,B1,C1) And then, converting the bone vector under the camera coordinate system to be converted into a bone vector under the local coordinate system of the upper body, wherein the conversion formula is as follows:
bone vector under local coordinate system ═ (A)1,B1,C1) Multiplied by the skeleton vector in the camera coordinate system.
Similar to the local coordinate system of the upper body, the local coordinate system of the lower body may be:
A2vector quantity: from left hip to right hip;
B2vector quantity: hip to neck (or spine to neck, adjustable according to the actual 3D keypoint location);
C2vector quantity: a. the2Vector sum B2The vector product of the vectors.
Then, the local coordinate system of the lower body is orthogonalized by Schmidt orthogonalization (Schmidt orthogonalization).
In obtaining an orthogonalized local coordinate system (A)1,B1,C1) And then, converting the bone vector under the camera coordinate system to be converted into a bone vector under a local coordinate system of the lower body, wherein the conversion formula is similar to the conversion formula of the bone vector under the camera coordinate system into the bone vector under the local coordinate system of the upper body, and the description is omitted here.
Normalizing each bone vector means that, in the case of converting the bone vector in the local coordinate system from the cartesian coordinate system to the spherical coordinate system, the spherical radius of the spherical coordinate system may be set to a preset value, and the preset value may be 1.
In a spherical coordinate system, the azimuth angle refers to the azimuth angle between the connecting line from the origin of the middle finger in the spherical surface to the point P, the projection line on the xy plane and the positive x-axis; polar angle, refers to the zenith angle between the line connecting the origin to point P and the positive z-axis. In the case where the spherical radii of the spherical coordinate system are all 1, a unique skeletal vector can be determined in the spherical coordinate system by a set of azimuth and polar angles.
The transformation relationship between the spherical coordinate system and the cartesian coordinate system can be referred to as follows:
spherical coordinate system
Figure BDA0003416184040000081
The conversion relation with the rectangular coordinate system (x, y, z) is as follows:
Figure BDA0003416184040000082
Figure BDA0003416184040000083
z＝rcosθ (a3)
conversely, the rectangular coordinate system (x, y, z) and the spherical coordinate system
Figure BDA0003416184040000084
The conversion relationship is as follows:
Figure BDA0003416184040000085
θ＝arccos(z/r)(a5)
Figure BDA0003416184040000086
each of the bone vectors may be normalized by equations (a4) - (a6) to obtain an azimuth angle and a polar angle of the bone vector in a spherical coordinate system having a radius of a preset value.
Taking a data acquisition object as a human body as an example, a spherical coordinate system is adopted, so that the length and the angle of a bone vector formed between joints of the human body can be expressed, but a Cartesian coordinate system cannot express the length and the angle of the bone vector.
And counting the azimuth angle and the polar angle of each bone vector to obtain a group of target bone vectors of the same time point data acquisition object corresponding to each group of initial bone vectors. In particular implementations, the azimuth and polar angles of each bone vector may be stored in historical data to determine the maximum amplitude motion that may be performed by the data collection subject.
It should be noted that at one point in time, the pose of the data acquisition object is fixed, so the azimuth and polar angles of each skeletal vector of the data acquisition object are also unique.
And step 108, generating bone assembly data and bone rotation data of the data acquisition object according to the obtained multiple groups of target bone vectors.
The bone assembly data can be obtained by splicing and assembling bone vectors of the same data acquisition object under different postures. The bone rotation data may be obtained by varying an azimuth and a polar angle of at least one bone vector of the data acquisition subject.
Optionally, generating bone assembly data of the data acquisition object according to the obtained sets of target bone vectors, including: determining a first group of target bone vectors and a second group of target bone vectors which are to be assembled in multiple groups of obtained target bone vectors; determining at least one first bone vector in the first set of target bone vectors; determining a second bone vector in the second set of target bone vectors corresponding to the first bone vector; the at least one first skeleton vector comprises a target skeleton vector corresponding to a sub-trunk vector, or a main trunk vector and a target skeleton vector corresponding to a sub-trunk vector associated with the main trunk vector; replacing the first bone vector with the second bone vector in the first set of target bone vectors to obtain a set of bone assembly vectors; and carrying out statistics on the obtained multiple groups of bone assembly vectors to obtain bone assembly data of the data acquisition object.
Fig. 2A is a schematic diagram of a first set of target bone vectors and a second set of target bone vectors to be assembled in a 3D data acquisition method provided in an embodiment of the present disclosure; FIG. 2B is a first schematic diagram of a set of bone assembly vectors provided by an embodiment of the present disclosure; fig. 2C is a second schematic diagram of a set of bone assembly vectors provided by an embodiment of the present disclosure; how to obtain bone assembly data of a data acquisition subject may be described below in conjunction with fig. 2A, 2B, and 2C.
Among the obtained sets of target bone vectors, a first set of target bone vectors and a second set of target bone vectors that are to be assembled in multiple pairs may be determined. Wherein, a pair of the first and second target bone vectors to be assembled can refer to fig. 2A, in fig. 2A, the left side of "+" shows the position information and connection relationship of each 3D keypoint in the first target bone vector, and the right side of "+" shows the position information and connection relationship of each 3D keypoint in the second target bone vector.
In one embodiment, the corresponding sub-torsos of the same data acquisition object may be interchanged. At least one first bone vector is determined in the first set of target bone vectors, as shown in fig. 2A, the at least one first bone vector may be a target bone vector corresponding to one sub-torso vector, such as target bone vector 201, and a second bone vector determined in the second set of target bone vectors corresponding to target bone vector 201 may be a target bone vector corresponding to one sub-torso vector, such as target bone vector 203.
The target bone vector 201 is located at the same position in the first set of target bone vectors as the target bone vector 203 is located in the second set of target bone vectors. As shown in fig. 2B, a set of bone assembly vectors may be obtained by replacing target bone vector 201 with target bone vector 203 in a first set of target bone vectors.
In another embodiment, the torso set formed by the main torso and the sub-torso corresponding to the same person may be interchanged. At least one first skeletal vector is determined in the first set of target skeletal vectors, as shown in fig. 2A, the at least one first skeletal vector may be a main trunk vector and a target skeletal vector corresponding to a sub-trunk vector associated with the main trunk vector, such as vector set 202 of target skeletal vectors, and a second skeletal vector determined in the second set of target skeletal vectors corresponding to vector set 202 of target skeletal vectors may be a main trunk vector and a target skeletal vector corresponding to a sub-trunk vector associated with the main trunk vector, such as vector set 204 of target skeletal vectors.
The vector set 202 of target bone vectors has the same location in the first set of target bone vectors as the vector set 204 of target bone vectors has in the second set of target bone vectors. As shown in fig. 2C, a set of bone assembly vectors may be obtained by replacing a set of vectors 202 of a target bone vector with a set of vectors 204 of the target bone vector in a first set of target bone vectors.
And carrying out statistics on a plurality of groups of obtained bone assembly vectors, and taking the plurality of groups of bone assembly vectors as bone assembly data of a data acquisition object.
Taking the data acquisition object as a human body as an example, a designated person photographed by the camera can take a plurality of preset postures in the photographing process, for example, a posture that both hands are lifted, or a posture that both hands face down, but the designated person does not take a posture that one hand is lifted and the other hand faces down. By generating the skeleton assembly data of the data acquisition object, the limbs of the designated personnel under different postures can be spliced together to obtain the skeleton assembly data with one hand lifting and the other hand downwards, so that the posture information of the data acquisition object is enriched, and more 3D data are obtained.
Optionally, acquiring multiple sets of key point coordinate information of the data acquisition object includes: acquiring multiple groups of key point coordinate information of a data acquisition object through video data comprising multiple frames of images; each set of key point coordinate information corresponds to a frame of image; determining a first set of target bone vectors and a second set of target bone vectors to be assembled in multiple sets of the obtained target bone vectors, including: determining a plurality of pairs of first images and second images in a plurality of frames of images included in video data according to a preset time interval; for each pair of the first image and the second image, in the obtained multiple sets of target bone vectors, determining a set of target bone vectors corresponding to the first image and a set of target bone vectors corresponding to the second image as a first set of target bone vectors and a second set of target bone vectors to be assembled.
Acquiring multiple groups of key point coordinate information of a data acquisition object, wherein the multiple groups of key point coordinate information of the data acquisition object can be acquired through video data comprising multiple frames of images; each set of keypoint coordinate information corresponds to a frame of image. For example, a video is taken by a depth camera for 1 minute for a designated person who changes posture every 3 seconds during the shooting. After removing the repeated image frames, the 1 minute video may include a plurality of frames with a frame number greater than 20, and a set of key point coordinate information may be obtained through each frame of image.
In the multiple sets of obtained target bone vectors, a first set of target bone vectors and a second set of target bone vectors to be assembled are determined, wherein the two sets of target bone vectors with larger attitude difference of the data acquisition object are used as a pair of the first set of target bone vectors and the second set of target bone vectors to be assembled.
In specific implementation, a plurality of pairs of first images and second images may be determined according to a preset time interval in a plurality of frames of images included in video data, for example, a video of a period of 1 minute is taken by a depth camera for a designated person, where the preset time interval may be 30 seconds, a frame of image corresponding to 10 seconds may be used as the first image, and a frame of image corresponding to 40 seconds may be used as the second image; one frame image corresponding to 20 seconds may be used as the first image and one frame image corresponding to 50 seconds may be used as the second image.
In the obtained multiple groups of target bone vectors, a group of key point coordinate information corresponding to the first image can be determined, then a group of target bone vectors of the data acquisition object corresponding to the group of key point coordinate information is determined according to the group of key point coordinate information corresponding to the first image, and the group of target bone vectors are used as the first group of target bone vectors.
In the obtained multiple groups of target bone vectors, a group of key point coordinate information corresponding to the second image can be determined, then a group of target bone vectors of the data acquisition object corresponding to the group of key point coordinate information is determined according to the group of key point coordinate information corresponding to the second image, and the group of target bone vectors are used as a second group of target bone vectors.
Optionally, generating bone rotation data of the data acquisition object according to the obtained plurality of sets of target bone vectors, including: determining a first numerical range interval of the azimuth angle of each bone vector and a second numerical range interval of the polar angle of each bone vector according to the azimuth angle and the polar angle of each bone vector in the obtained multiple groups of target bone vectors; for each bone vector, rotating the bone vector by taking a value in a first numerical range interval according to a first preset interval and/or taking a value in a second numerical range interval according to a second preset interval to generate rotation data of the bone vector; and (5) counting the rotation data of each bone vector to obtain the bone rotation data of the data acquisition object.
In a specific implementation, the azimuth and polar angles of each bone vector stored in step 106, i.e., the azimuth and polar angles of each bone vector in the plurality of target bone vectors obtained in step 106, may be read from historical data. From the read azimuth angles of each bone vector, a maximum value and a minimum value of each bone vector can be determined, thereby determining a first numerical range interval of the azimuth angle of each bone vector. In the case that the azimuth angle of the bone vector is within the first range of values, the rotation of the bone vector is reasonable, and the data acquisition object can actually make a corresponding gesture. Under the condition that the azimuth angle of the skeleton vector is located outside the first numerical range interval, the rotation of the skeleton vector possibly exceeds the action limit of the data acquisition object, so that in the process of generating skeleton rotation data, each azimuth angle in historical data can be traversed, and the azimuth angle can also be taken within the first numerical range interval according to a first preset interval, so that reasonable skeleton rotation data can be obtained.
Similarly, a second range of numerical values for the polar angle of each bone vector may be determined. In the process of generating the bone rotation data, each polar angle in the historical data can be traversed, and the polar angle can also be valued in a second numerical value range interval according to a second preset interval, so that reasonable bone rotation data can be obtained.
The first preset interval may be 5 degrees or other values, and the second preset interval may be 5 degrees or other values. The first preset interval and the second preset interval may be the same or different.
For each bone vector, in the process of generating the rotation data of the bone vector, the bone vector can be rotated only by taking values in a first numerical range interval according to a first preset interval to generate the rotation data of the bone vector; the bone vector can be rotated only by taking values in a second numerical value range interval according to a second preset interval to generate rotation data of the bone vector; the bone vector can be rotated by taking a value in a first numerical range interval according to a first preset interval and taking a value in a second numerical range interval according to a second preset interval, and the rotation data of the bone vector can be generated.
Optionally, each group of target skeleton vectors includes target skeleton vectors of a plurality of main trunk vector objects and target skeleton vectors corresponding to a plurality of sub trunk vectors; each main trunk vector is associated with at least one sub-trunk vector; and under the condition that the skeleton vector is the target skeleton vector corresponding to the main trunk vector, the rotation data of the target skeleton vector corresponding to the sub-trunk vector associated with the main trunk vector is determined according to the values of the azimuth angle and the polar angle of the skeleton vector and the values of the azimuth angle and the polar angle of the target skeleton vector corresponding to the sub-trunk vector associated with the main trunk vector.
It should be noted that, under the condition that the skeleton vector to be rotated is the target skeleton vector corresponding to the sub-trunk vector, the rotation data of the skeleton vector is only taken according to the azimuth angle and the polar angle of the skeleton vector; and under the condition that the skeleton vector to be rotated is the target skeleton vector corresponding to the main trunk vector, the rotation data of the target skeleton vector corresponding to the sub-trunk vector associated with the main trunk vector is determined according to the values of the azimuth angle and polar angle of the skeleton vector and the values of the azimuth angle and polar angle of the target skeleton vector corresponding to the sub-trunk vector associated with the main trunk vector.
The target skeleton vector corresponding to the main trunk vector is rotated to drive the target skeleton vector corresponding to the associated sub-trunk vector to rotate, so that the target skeleton vector corresponding to the main trunk vector and the target skeleton vector corresponding to the associated sub-trunk vector need to be evaluated simultaneously under the condition that the skeleton vector to be rotated is the target skeleton vector corresponding to the main trunk vector.
After step 108, the obtained bone assembly data and bone rotation data of the data acquisition object may also be converted from a spherical coordinate system to a cartesian coordinate system, the conversion formula may refer to the aforementioned formulas (a1) - (a 3). After the obtained bone assembly data and bone rotation data of the data acquisition object are converted from the spherical coordinate system to the cartesian coordinate system, the obtained data may also be converted from the local coordinate system to the camera coordinate system.
Converting the data from the local coordinate system to the camera coordinate system may refer to the aforementioned procedure of converting the data from the camera coordinate system to the local coordinate system, except that after the orthogonalized local coordinate system is obtained, the skeletal vector in the camera coordinate system is obtained by multiplying the skeletal vector in the local coordinate system by the orthogonalized local coordinate system.
After the bone assembly data and the bone rotation data of the data acquisition object in the camera coordinate system are obtained, they may be stored as 3D data of the data acquisition object.
After acquiring and storing 3D data of the data acquisition object as a 3D data set, a corresponding 2D data set may also be generated from the stored 3D data set. In specific implementation, 3D data in a camera coordinate system can be converted into 2D data in a world coordinate system according to a camera imaging principle.
For example, the coordinate information of a point in the 3D data in the camera coordinate system is known as (X)c，Yc，Zc) The correspondence between the two coordinate systems of the camera coordinate system and the image coordinate system can refer to the following formula (a 7):
Figure BDA0003416184040000141
wherein f isx，fyIs the focal length of the camera, cxAnd cyFor the camera offset, γ is the camera distortion parameter, (u ', v ', w ') is a point (X) of the 3D datac，Yc，Zc) And 3D coordinate information in the world coordinate system is obtained after the camera coordinate system is converted into the world coordinate system.
The camera parameters such as the focal length of the camera, the offset of the camera, and the distortion parameter of the camera may be the camera parameters obtained by the aforementioned step of calibrating the depth camera.
Then, a point (X) can be calculated from u ', v', and wc，Yc，Zc) Corresponding 2D coordinate information (u, v), wherein,
Figure BDA0003416184040000142
Figure BDA0003416184040000143
the 3D data of the data acquisition object can be used for dynamically displaying the posture, for example, the data acquisition object is reconstructed through mesh assistance and the like, so that various postures of the data acquisition object are dynamically displayed; or generating a corresponding 2D data set according to the stored 3D data set, thereby establishing a 2D and 3D data set pair, wherein the 2D and 3D data set pair can be used for training a 3D posture network; and a 3D posture data set of the data acquisition object can be constructed based on the 3D data of the data acquisition object, wherein the data set comprises 3D data of which each bone vector is within the bone stretching limit range of the data acquisition object, and whether any 3D posture data is effective or not, namely whether the 3D posture data exceeds the bone stretching limit range of the data acquisition object or not can be judged through the 3D posture data set of the data acquisition object.
In the method embodiment as shown in fig. 1, the 3D data acquisition method comprises: acquiring a plurality of 3D key points of a data acquisition object and a plurality of groups of key point coordinate information of the data acquisition object; each group of key point coordinate information comprises coordinate information of each 3D key point of the same time point data acquisition object; for each group of key point coordinate information, determining a group of initial skeleton vectors of the corresponding data acquisition object; performing coordinate conversion processing on each group of initial skeleton vectors to obtain a group of corresponding target skeleton vectors; and generating bone assembly data and bone rotation data of the data acquisition object according to the obtained multiple groups of target bone vectors. According to the embodiment of the application, through coordinate conversion processing, a plurality of skeleton vectors of the data acquisition object can realize operations such as movement, assembly, rotation and the like, so that a large amount of 3D data of the data acquisition object in different postures is obtained by utilizing a plurality of groups of key point coordinate information with small data volume.
Based on the same technical concept, one or more embodiments of the present specification further provide a 3D data acquisition method, in which a data acquisition object may be a human body. Fig. 3 is a schematic flowchart of another 3D data acquisition method provided in an embodiment of the present disclosure, and as shown in fig. 3, the method includes:
step 302, calibrating a depth camera.
And step 304, acquiring 3D key points of the human body.
And step 306, splitting the nine limbs of the human body into an upper half body and a lower half body, and respectively establishing local coordinate systems.
Step 308, the skeleton vector of the local coordinate system is converted into a spherical coordinate system.
And 310, generating the 3D data spliced by the human body limbs and the 3D data rotated by the human body posture.
After step 310 is performed, various applications of 3D data may also be implemented using the generated 3D data of the human body. For example, human body mesh reconstruction is performed by using the 3D data spliced by the human body limbs and the 3D data rotated by the human body posture, so that various postures of the human body are dynamically displayed. For another example, a data set of human body posture is constructed from 3D data of a human body, where the data set includes 3D data of which each bone vector is within a human body bone stretching limit range, and then, through the data set of the human body posture, it can be determined whether any 3D posture data is valid, that is, whether the 3D posture data exceeds the human body bone stretching limit range. For another example, the corresponding 2D data may also be generated from the 3D data of the human body, thereby constructing a pair of 2D and 3D data sets, which may be used to train the 3D pose network.
The application of the 3D data is not limited to the above examples, and the embodiments of the present application are not listed.
In addition, as for the embodiment of the 3D data acquisition method shown in fig. 3, since it is basically similar to the embodiment of the 3D data acquisition method shown in fig. 1, the description is relatively simple, and relevant points can be referred to the partial description of the embodiment of the 3D data acquisition method shown in fig. 1.
Based on the same technical concept, one or more embodiments of the present specification further provide a 3D data acquisition apparatus, and fig. 4 is a schematic diagram of a module composition of the 3D data acquisition apparatus provided in the embodiments of the present application, as shown in fig. 4, the apparatus includes:
a processor 401, configured to obtain a plurality of 3D key points of a data acquisition object and a plurality of sets of key point coordinate information of the data acquisition object; each group of key point coordinate information comprises coordinate information of each 3D key point of the same time point data acquisition object; for each group of key point coordinate information, determining a group of initial skeleton vectors of the corresponding data acquisition object; performing coordinate conversion processing on each group of initial skeleton vectors to obtain a group of corresponding target skeleton vectors; and generating bone assembly data and bone rotation data of the data acquisition object according to the obtained multiple groups of target bone vectors.
Optionally, the initial bone vector comprises a bone vector in a camera coordinate system; the processor 401 is further configured to:
performing coordinate transformation processing on each group of initial bone vectors to obtain a corresponding group of target bone vectors, wherein the coordinate transformation processing comprises the following steps:
constructing at least one local coordinate system according to the data acquisition object;
converting each bone vector in each set of initial bone vectors from a camera coordinate system to a local coordinate system;
normalizing each skeleton vector to obtain an azimuth angle and a polar angle of the skeleton vector under a spherical coordinate system with a radius of a preset numerical value;
and counting the azimuth angle and the polar angle of each bone vector to obtain a group of target bone vectors of the data acquisition object corresponding to each group of initial bone vectors.
Optionally, the processor 401 is further configured to:
for each set of keypoint coordinate information, determining a set of initial bone vectors for the corresponding data acquisition object, comprising:
determining a father node set and a child node set in a plurality of 3D key points of a data acquisition object;
determining a plurality of main trunk vectors and a plurality of sub trunk vectors of the data acquisition object according to the parent node set, the child node set and the coordinate information of each 3D key point in each group of key point coordinate information, and using the main trunk vectors and the sub trunk vectors as a group of initial skeleton vectors of the data acquisition object corresponding to each group of key point coordinate information; each main torso vector is associated with at least one sub-torso vector.
Optionally, the processor 401 is further configured to:
generating bone assembly data of the data acquisition object according to the obtained multiple groups of target bone vectors, comprising:
determining a first group of target bone vectors and a second group of target bone vectors which are to be assembled in multiple groups of obtained target bone vectors;
determining at least one first bone vector in the first set of target bone vectors; determining a second bone vector in the second set of target bone vectors corresponding to the first bone vector; the at least one first skeleton vector comprises a target skeleton vector corresponding to a sub-trunk vector, or a main trunk vector and a target skeleton vector corresponding to a sub-trunk vector associated with the main trunk vector;
replacing the first bone vector with the second bone vector in the first set of target bone vectors to obtain a set of bone assembly vectors;
and carrying out statistics on the obtained multiple groups of bone assembly vectors to obtain bone assembly data of the data acquisition object.
Optionally, the processor 401 is further configured to:
generating bone rotation data of the data acquisition object according to the obtained multiple groups of target bone vectors, comprising:
determining a first numerical range interval of the azimuth angle of each bone vector and a second numerical range interval of the polar angle of each bone vector according to the azimuth angle and the polar angle of each bone vector in the obtained multiple groups of target bone vectors;
for each bone vector, rotating the bone vector by taking a value in a first numerical range interval according to a first preset interval and/or taking a value in a second numerical range interval according to a second preset interval to generate rotation data of the bone vector;
and (5) counting the rotation data of each bone vector to obtain the bone rotation data of the data acquisition object.
Optionally, each group of target skeleton vectors includes target skeleton vectors of a plurality of main trunk vector objects and target skeleton vectors corresponding to a plurality of sub trunk vectors; each main trunk vector is associated with at least one sub-trunk vector;
and under the condition that the skeleton vector is the target skeleton vector corresponding to the main trunk vector, the rotation data of the target skeleton vector corresponding to the sub-trunk vector associated with the main trunk vector is determined according to the values of the azimuth angle and the polar angle of the skeleton vector and the values of the azimuth angle and the polar angle of the target skeleton vector corresponding to the sub-trunk vector associated with the main trunk vector.
Optionally, the processor 401 is further configured to:
acquiring multiple groups of key point coordinate information of a data acquisition object, comprising:
acquiring multiple groups of key point coordinate information of a data acquisition object through video data comprising multiple frames of images; each set of key point coordinate information corresponds to a frame of image;
determining a first set of target bone vectors and a second set of target bone vectors to be assembled in multiple sets of the obtained target bone vectors, including:
determining a plurality of pairs of first images and second images in a plurality of frames of images included in video data according to a preset time interval;
for each pair of the first image and the second image, in the obtained multiple sets of target bone vectors, determining a set of target bone vectors corresponding to the first image and a set of target bone vectors corresponding to the second image as a first set of target bone vectors and a second set of target bone vectors to be assembled.
In the 3D data acquisition apparatus provided in the embodiment of the present application, the 3D data acquisition method includes: acquiring a plurality of 3D key points of a data acquisition object and a plurality of groups of key point coordinate information of the data acquisition object; each group of key point coordinate information comprises coordinate information of each 3D key point of the same time point data acquisition object; for each group of key point coordinate information, determining a group of initial skeleton vectors of the corresponding data acquisition object; performing coordinate conversion processing on each group of initial skeleton vectors to obtain a group of corresponding target skeleton vectors; and generating bone assembly data and bone rotation data of the data acquisition object according to the obtained multiple groups of target bone vectors. According to the embodiment of the application, through coordinate conversion processing, a plurality of skeleton vectors of the data acquisition object can realize operations such as movement, assembly, rotation and the like, so that a large amount of 3D data of the data acquisition object in different postures is obtained by utilizing a plurality of groups of key point coordinate information with small data volume.
In addition, for the embodiment of the 3D data acquisition apparatus, since it is basically similar to the embodiment of the 3D data acquisition method, the description is relatively simple, and for relevant points, refer to the partial description of the embodiment of the 3D data acquisition method. Further, it should be noted that, of the respective components of the 3D data acquisition apparatus of the present invention, the components thereof are logically divided according to the functions to be implemented, but the present invention is not limited thereto, and the respective components may be newly divided or combined as necessary.
Based on the same technical concept, the embodiment of the application also provides an electronic device, as shown in fig. 5. Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure, and referring to fig. 5, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required by other services. The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to form the 3D data acquisition device on the logic level. Of course, besides the software implementation, the present application does not exclude other implementations, such as logic devices or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, and may also be hardware or logic devices.
The network interface, the processor and the memory may be interconnected by a bus system. The bus may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 5, but this does not indicate only one bus or one type of bus.
The memory is used for storing programs. In particular, the program may include program code including computer operating instructions. The memory may include both read-only memory and random access memory, and provides instructions and data to the processor. The Memory may include a Random-Access Memory (RAM) and may also include a non-volatile Memory (non-volatile Memory), such as at least 1 disk Memory.
The processor is used for executing the program stored in the memory and specifically executing the following steps:
acquiring a plurality of 3D key points of a data acquisition object and a plurality of groups of key point coordinate information of the data acquisition object;
each group of key point coordinate information comprises coordinate information of each 3D key point of the same time point data acquisition object;
for each group of key point coordinate information, determining a group of initial skeleton vectors of the corresponding data acquisition object;
performing coordinate conversion processing on each group of initial skeleton vectors to obtain a group of corresponding target skeleton vectors;
and generating bone assembly data and bone rotation data of the data acquisition object according to the obtained multiple groups of target bone vectors.
The 3D data acquisition method performed by the 3D data acquisition apparatus according to the embodiment shown in fig. 3 of the present application may be applied to or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also audio processing Devices (DSP), Application Specific Integrated Circuits (ASIC), Field Programmable Gate Arrays (FPGA) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the 3D data acquisition method disclosed in the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory, and the processor reads information in the memory and completes the steps of the 3D data acquisition method by combining hardware of the processor.
Based on the same technical concept, embodiments of the present application also provide a computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, cause the electronic device to perform the 3D data acquisition method provided by the foregoing method embodiments.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims (10)

1. A method of 3D data acquisition, comprising:
acquiring a plurality of 3D key points of a data acquisition object and a plurality of groups of key point coordinate information of the data acquisition object; each group of the key point coordinate information comprises coordinate information of each 3D key point of the data acquisition object at the same time point;
for each set of the keypoint coordinate information, determining a set of initial bone vectors of the corresponding data acquisition object;
performing coordinate conversion processing on each group of initial skeleton vectors to obtain a group of corresponding target skeleton vectors;
and generating bone assembly data and bone rotation data of the data acquisition object according to the obtained multiple groups of target bone vectors.
2. The method of claim 1, wherein the initial bone vector comprises a bone vector in a camera coordinate system; the performing coordinate transformation processing on each group of initial bone vectors to obtain a corresponding group of target bone vectors includes:
constructing at least one local coordinate system according to the data acquisition object;
converting each bone vector in each set of the initial bone vectors from the camera coordinate system to the local coordinate system;
normalizing each skeleton vector to obtain an azimuth angle and a polar angle of the skeleton vector under a spherical coordinate system with a radius of a preset numerical value;
and counting the azimuth angle and the polar angle of each bone vector to obtain a group of target bone vectors of the data acquisition object corresponding to each group of initial bone vectors.
3. The method of claim 1, wherein said determining, for each set of said keypoint coordinate information, an initial set of bone vectors for the corresponding data acquisition object comprises:
determining a parent node set and a child node set in a plurality of 3D key points of the data acquisition object;
determining a plurality of main trunk vectors and a plurality of sub trunk vectors of the data acquisition object according to the father node set, the child node set and the coordinate information of each 3D key point in each group of key point coordinate information, wherein the main trunk vectors and the sub trunk vectors are used as a group of initial skeleton vectors of the data acquisition object corresponding to each group of key point coordinate information; each of the main torso vectors is associated with at least one of the sub-torso vectors.
4. The method of claim 3, wherein generating bone assembly data of the data acquisition subject from the obtained sets of target bone vectors comprises:
determining a first group of target bone vectors and a second group of target bone vectors which are to be assembled in multiple groups of the obtained target bone vectors;
determining at least one first bone vector in the first set of target bone vectors; determining a second bone vector in the second set of target bone vectors corresponding to the first bone vector; the at least one first skeleton vector comprises a target skeleton vector corresponding to the sub-trunk vector, or a target skeleton vector corresponding to the main trunk vector and the sub-trunk vector associated with the main trunk vector;
replacing said first bone vector with said second bone vector in said first set of target bone vectors to obtain a set of bone assembly vectors;
and carrying out statistics on the obtained multiple groups of bone assembly vectors to obtain the bone assembly data of the data acquisition object.
5. The method of claim 2, wherein generating bone rotation data of the data acquisition object from the obtained sets of target bone vectors comprises:
determining a first numerical range interval of the azimuth angle of each bone vector and a second numerical range interval of the polar angle of each bone vector according to the azimuth angle and the polar angle of each bone vector in the obtained multiple groups of target bone vectors;
for each bone vector, rotating the bone vector by taking values in the first numerical range interval according to a first preset interval and/or taking values in the second numerical range interval according to a second preset interval to generate rotation data of the bone vector;
and counting the rotation data of each bone vector to obtain the bone rotation data of the data acquisition object.
6. The method of claim 5, wherein each set of said target bone vectors comprises target bone vectors of a plurality of main torso vector objects and target bone vectors corresponding to a plurality of sub-torso vectors; each said main torso vector associated with at least one said sub-torso vector;
and under the condition that the skeleton vector is the target skeleton vector corresponding to the main trunk vector, determining the rotation data of the target skeleton vector corresponding to the sub-trunk vector associated with the main trunk vector according to the values of the azimuth angle and polar angle of the skeleton vector and the values of the azimuth angle and polar angle of the target skeleton vector corresponding to the sub-trunk vector associated with the main trunk vector.
7. The method of claim 4, wherein obtaining sets of keypoint coordinate information of the data acquisition object comprises:
acquiring multiple groups of key point coordinate information of the data acquisition object through video data comprising multiple frames of images; each set of the key point coordinate information corresponds to one frame of image;
determining a first group of target bone vectors and a second group of target bone vectors to be assembled in multiple groups of the obtained target bone vectors, wherein the steps comprise:
determining a plurality of pairs of first images and second images in a plurality of frames of images included in the video data according to a preset time interval;
for each pair of the first image and the second image, in the obtained multiple sets of the target bone vectors, determining a set of the target bone vectors corresponding to the first image and a set of the target bone vectors corresponding to the second image as a first set of the target bone vectors and a second set of the target bone vectors to be assembled.
8. A 3D data acquisition apparatus, comprising:
the system comprises a processor, a data acquisition object and a data processing module, wherein the processor is used for acquiring a plurality of 3D key points of the data acquisition object and a plurality of groups of key point coordinate information of the data acquisition object; each group of the key point coordinate information comprises coordinate information of each 3D key point of the data acquisition object at the same time point; for each set of the keypoint coordinate information, determining a set of initial bone vectors of the corresponding data acquisition object; performing coordinate conversion processing on each group of initial skeleton vectors to obtain a group of corresponding target skeleton vectors; and generating bone assembly data and bone rotation data of the data acquisition object according to the obtained multiple groups of target bone vectors.
9. An electronic device, comprising: a processor, a memory electrically connected with the processor. The memory stores a computer program for calling and executing the steps of the computer program in the 3D data acquisition method of any of the claims 1 to 7 from the memory.
10. A computer-readable storage medium, characterized in that a computer program is stored thereon, which computer program, when being executed by a processor, carries out the steps of the 3D data acquisition method of any of the preceding claims 1 to 7.
CN202111547657.6A 2021-12-16 2021-12-16 3D data acquisition method and device and electronic equipment Pending CN114187343A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111547657.6A CN114187343A (en) 2021-12-16 2021-12-16 3D data acquisition method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111547657.6A CN114187343A (en) 2021-12-16 2021-12-16 3D data acquisition method and device and electronic equipment

Publications (1)

Publication Number Publication Date
CN114187343A true CN114187343A (en) 2022-03-15

Family

ID=80605440

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111547657.6A Pending CN114187343A (en) 2021-12-16 2021-12-16 3D data acquisition method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN114187343A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116719416A (en) * 2023-08-07 2023-09-08 海马云（天津）信息技术有限公司 Gesture motion correction method and device for virtual digital person, electronic equipment and storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116719416A (en) * 2023-08-07 2023-09-08 海马云（天津）信息技术有限公司 Gesture motion correction method and device for virtual digital person, electronic equipment and storage medium
CN116719416B (en) * 2023-08-07 2023-12-15 海马云（天津）信息技术有限公司 Gesture motion correction method and device for virtual digital person, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
CN104363986B (en) A kind of image processing method and equipment
CN110599540B (en) Real-time three-dimensional human body shape and posture reconstruction method and device under multi-viewpoint camera
CN109448090B (en) Image processing method, device, electronic equipment and storage medium
Bok et al. Geometric calibration of micro-lens-based light field cameras using line features
US20210044787A1 (en) Three-dimensional reconstruction method, three-dimensional reconstruction device, and computer
CN101998136B (en) Homography matrix acquisition method as well as image pickup equipment calibrating method and device
JP2018129007A (en) Learning data generation apparatus, learning apparatus, estimation apparatus, learning data generation method, and computer program
CN112401369A (en) Body parameter measuring method, system, equipment, chip and medium based on human body reconstruction
CN114187343A (en) 3D data acquisition method and device and electronic equipment
CN114022645A (en) Action driving method, device, equipment and storage medium of virtual teacher system
CN111310590B (en) Action recognition method and electronic equipment
CN111681302A (en) Method and device for generating 3D virtual image, electronic equipment and storage medium
CN103955928B (en) Method and device for determining camber lens distortion parameter and electronic equipment
Malm et al. Simplified intrinsic camera calibration and hand-eye calibration for robot vision
CN111339969B (en) Human body posture estimation method, device, equipment and storage medium
CN109531578B (en) Humanoid mechanical arm somatosensory control method and device
CN110874820B (en) Material simulation deformation data acquisition method and device
Park et al. Improvement on Zhang's camera calibration
KR20200057572A (en) Hand recognition augmented reality-intraction apparatus and method
CN112562071A (en) Method, device and equipment for calculating motion difference and storage medium
CN118120246A (en) Learning device, learning method, learning program, camera parameter calculation device, camera parameter calculation method, and camera parameter calculation program
CN115311472A (en) Motion capture method and related equipment
CN114821791A (en) Method and system for capturing three-dimensional motion information of image
CN110838182B (en) Method and system for attaching image to mannequin
Lobo et al. Fusing of image and inertial sensing for camera calibration

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination