CN113205591B

CN113205591B - Method and device for acquiring three-dimensional reconstruction training data and electronic equipment

Info

Publication number: CN113205591B
Application number: CN202110485357.3A
Authority: CN
Inventors: 韩忠涛
Original assignee: Beijing QIYI Century Science and Technology Co Ltd
Current assignee: Beijing QIYI Century Science and Technology Co Ltd
Priority date: 2021-04-30
Filing date: 2021-04-30
Publication date: 2024-03-08
Anticipated expiration: 2041-04-30
Also published as: CN113205591A

Abstract

The embodiment of the invention provides a method and a device for acquiring three-dimensional reconstruction training data and electronic equipment, wherein the method comprises the following steps: arranging a plurality of virtual cameras in a virtual three-dimensional environment to obtain external parameters of each virtual camera; generating a plurality of virtual three-dimensional point coordinates within the field of view of the plurality of virtual cameras; for each virtual three-dimensional point coordinate, determining that the virtual three-dimensional point coordinate is mapped to a target two-dimensional point coordinate in a two-dimensional image acquired by each virtual camera according to an external reference and a preset internal reference of each virtual camera, and obtaining a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point coordinate; and determining each virtual three-dimensional point coordinate and the corresponding target two-dimensional point coordinate set as training data of a three-dimensional reconstruction algorithm. In the scheme, a real camera and a real object do not need to be deployed, and a real image shooting process does not need to be carried out, so that the complexity of acquiring training data can be reduced, and the acquisition efficiency of the training data is improved.

Description

Method and device for acquiring three-dimensional reconstruction training data and electronic equipment

Technical Field

The present invention relates to the field of image three-dimensional reconstruction technology, and in particular, to a method and an apparatus for acquiring three-dimensional reconstruction training data, and an electronic device.

Background

Three-dimensional reconstruction is understood to be the establishment of a mathematical model suitable for computer representation and processing of a three-dimensional object, which is the basis for processing and analyzing a three-dimensional object in a computer environment, and is one of techniques for establishing virtual reality expressing an objective world in a computer.

Three-dimensional reconstruction algorithms are typically used in three-dimensional reconstruction schemes. Generally, more training data is needed, and the three-dimensional reconstruction algorithm is subjected to iterative optimization to obtain the three-dimensional reconstruction algorithm with higher accuracy. The three-dimensional reconstruction algorithm is a process of calculating three-dimensional points in a three-dimensional space according to two-dimensional points in an image, so that training data comprises a mapping relationship between the three-dimensional points in the three-dimensional space and the two-dimensional points in the two-dimensional image.

The scheme for acquiring the training data generally includes: arranging a plurality of objects in a real physical space, respectively acquiring more two-dimensional images of each object from different angles, and further determining the mapping relation between the three-dimensional coordinates of the objects in the real physical space and the two-dimensional coordinates in the two-dimensional images through processes of feature point matching, camera parameter estimation and the like, and taking the mapping relation as training data. In order to acquire more training data, a plurality of objects and cameras with multiple angles are required to be arranged, a large number of two-dimensional images of different angles of each object are shot, and therefore the scheme needs to manually arrange the objects and the cameras and manually control the cameras to shoot, so that the complexity is high, and the acquisition efficiency of the training data is low.

Disclosure of Invention

The embodiment of the invention aims to provide a method, a device and equipment for acquiring three-dimensional reconstruction training data so as to improve the acquisition efficiency of the training data. The specific technical scheme is as follows:

to achieve the above objective, an embodiment of the present invention provides a method for acquiring three-dimensional reconstruction training data, including:

arranging a plurality of virtual cameras in a virtual three-dimensional environment to obtain external parameters of each virtual camera;

generating a plurality of virtual three-dimensional point coordinates within the field of view of the plurality of virtual cameras;

for each virtual three-dimensional point coordinate, determining that the virtual three-dimensional point coordinate is mapped to a target two-dimensional point coordinate in a two-dimensional image acquired by each virtual camera according to the external reference and the preset internal reference of each virtual camera, and obtaining a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point;

and determining each virtual three-dimensional point coordinate and the corresponding target two-dimensional point coordinate set as training data of a three-dimensional reconstruction algorithm.

Optionally, the deploying a plurality of virtual cameras in the virtual three-dimensional environment to obtain external parameters of each virtual camera includes:

determining a coordinate point in the virtual three-dimensional environment as a reference point;

Determining the positions of a plurality of virtual cameras according to a preset mode by taking the datum point as a reference;

and adjusting the main optical axis direction of each virtual camera so that the field of view range of the plurality of virtual cameras has a superposition area.

Optionally, the determining the positions of the plurality of virtual cameras according to a preset mode with the reference point as a reference includes:

determining the fiducial point as a determined deployment center point;

and deploying a plurality of virtual cameras around the deployment center point in a preset array shape to obtain the position of each virtual camera.

Optionally, disposing a plurality of virtual cameras around the deployment center point in a preset array shape to obtain a position of each virtual camera includes:

the position coordinates of each virtual camera are calculated using the following formula:

wherein, camera _center [i]And N is the total number of the virtual cameras, and distance is the distance from the virtual camera to the deployment center point.

Optionally, adjusting the main optical axis direction of each virtual camera includes:

and respectively adjusting the main optical axis direction of each virtual camera to be towards the deployment center point.

Optionally, the determining, according to the external parameter and the preset internal parameter of each virtual camera, that the virtual three-dimensional point coordinate maps to the target two-dimensional point coordinate in the two-dimensional image acquired by each virtual camera includes:

Calculating a mapping matrix corresponding to each virtual camera according to the external parameters and the preset internal parameters of the virtual camera, wherein the mapping matrix is used for representing the mapping relation between the three-dimensional coordinates of the three-dimensional points and the two-dimensional coordinates of the two-dimensional points corresponding to the three-dimensional points in the two-dimensional image;

multiplying the three-dimensional homogeneous coordinates corresponding to the virtual three-dimensional point coordinates by the mapping matrix to obtain two-dimensional homogeneous coordinates of the target two-dimensional point in the two-dimensional image acquired by the virtual camera.

Optionally, the calculating the mapping matrix corresponding to the virtual camera according to the external parameters and the preset internal parameters of the virtual camera includes:

obtaining a translation vector corresponding to the virtual camera according to the external parameters of the virtual camera, wherein the translation vector is used for representing the translation relation from the three-dimensional coordinate transformation of the three-dimensional point to the three-dimensional camera coordinate system of the virtual camera;

and determining a mapping matrix corresponding to the virtual camera based on the translation vector, the preset internal parameter of the virtual camera and the main optical axis direction included by the external parameter of the virtual camera.

Optionally, after determining each virtual three-dimensional point coordinate and the corresponding target two-dimensional point coordinate set thereof as training data of the three-dimensional reconstruction algorithm, the method further includes:

Determining three-dimensional reconstruction point coordinates corresponding to the virtual three-dimensional point coordinates by a three-dimensional reconstruction algorithm based on a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point coordinates aiming at each virtual three-dimensional point coordinate;

and carrying out iterative training on the three-dimensional reconstruction algorithm according to the difference between the three-dimensional reconstruction point coordinates and the virtual three-dimensional point coordinates.

To achieve the above object, an embodiment of the present invention further provides an apparatus for acquiring three-dimensional reconstructed training data, including:

the deployment module is used for deploying a plurality of virtual cameras in the virtual three-dimensional environment to obtain external parameters of each virtual camera;

the generating module is used for generating a plurality of virtual three-dimensional point coordinates in the field of view range of the plurality of virtual cameras;

the mapping module is used for mapping the virtual three-dimensional point coordinates to target two-dimensional point coordinates in the two-dimensional image acquired by each virtual camera according to the external parameters and the preset internal parameters of each virtual camera respectively to obtain a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point coordinates;

and the determining module is used for determining each virtual three-dimensional point coordinate and the corresponding target two-dimensional point coordinate set as training data of a three-dimensional reconstruction algorithm.

Optionally, the deployment module includes:

a reference point determination unit configured to determine one coordinate point as a reference point in the virtual three-dimensional environment;

a camera position determining unit, configured to determine positions of a plurality of virtual cameras according to a preset manner with the reference point as a reference;

and the optical axis direction adjusting unit is used for adjusting the main optical axis direction of each virtual camera so that the field of view range of the plurality of virtual cameras has a superposition area.

Optionally, the camera position determining unit includes:

a center point determination subunit configured to determine the reference point as a deployment center point;

and the camera position determining subunit is used for deploying a plurality of virtual cameras around the deployment center point in a preset array shape to obtain the position of each virtual camera.

Optionally, the camera position determining subunit includes:

the position coordinate calculation subunit is used for calculating the position coordinate of each virtual camera by using the following formula:

Optionally, the optical axis direction adjustment unit includes:

And the optical axis direction adjusting subunit is used for respectively adjusting the main optical axis direction of each virtual camera to be towards the deployment center point.

Optionally, the mapping module includes:

the mapping matrix calculation unit is used for calculating a mapping matrix corresponding to each virtual camera according to the external parameters and the preset internal parameters of the virtual camera, wherein the mapping matrix is used for representing the mapping relation between the three-dimensional coordinates of the three-dimensional points and the two-dimensional coordinates of the two-dimensional points corresponding to the three-dimensional points in the two-dimensional image;

and the coordinate mapping unit is used for multiplying the three-dimensional homogeneous coordinate corresponding to the virtual three-dimensional point coordinate by the mapping matrix to obtain the two-dimensional homogeneous coordinate of the target two-dimensional point in the two-dimensional image acquired by the virtual camera.

Optionally, the mapping matrix calculation unit includes:

a translation vector calculation operator unit, configured to calculate a translation vector corresponding to the virtual camera according to an external parameter of the virtual camera, where the translation vector is used to represent a translation relationship from a three-dimensional coordinate of a three-dimensional point to a three-dimensional camera coordinate system of the virtual camera;

and the mapping matrix calculating subunit is used for determining a mapping matrix corresponding to the virtual camera based on the translation vector, the preset internal parameter of the virtual camera and the main optical axis direction included by the external parameter of the virtual camera.

Optionally, the apparatus further includes:

the building point coordinate determining module is used for determining three-dimensional reconstruction point coordinates corresponding to the virtual three-dimensional point coordinates through a three-dimensional reconstruction algorithm based on the target two-dimensional point coordinate set corresponding to the virtual three-dimensional point coordinates for each virtual three-dimensional point coordinate after determining each virtual three-dimensional point coordinate and the target two-dimensional point coordinate set corresponding to the virtual three-dimensional point coordinate as training data of the three-dimensional reconstruction algorithm;

and the optimization module is used for carrying out iterative training on the three-dimensional reconstruction algorithm according to the difference between the three-dimensional reconstruction point coordinates and the virtual three-dimensional point coordinates.

In order to achieve the above object, an embodiment of the present invention further provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;

a memory for storing a computer program;

and the processor is used for realizing any one of the steps of the method for acquiring the three-dimensional reconstruction training data when executing the program stored in the memory.

By applying the embodiment of the invention, the electronic equipment can deploy a plurality of virtual cameras in a virtual three-dimensional environment to obtain the external parameters of each virtual camera, generate a plurality of virtual three-dimensional point coordinates in the field of view of the plurality of virtual cameras, and determine, for each virtual three-dimensional point coordinate, that the virtual three-dimensional point coordinates are mapped to target two-dimensional point coordinates in a two-dimensional image acquired by each virtual camera according to the external parameters and preset internal parameters of each virtual camera, so as to obtain a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point, and further determine each virtual three-dimensional point coordinate and the corresponding target two-dimensional point coordinate set as training data of a three-dimensional reconstruction algorithm.

In the scheme, a plurality of virtual cameras are deployed in a virtual three-dimensional environment, a plurality of virtual three-dimensional point coordinates are generated to represent the position of an object, and then a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point can be determined according to the external parameters of the virtual cameras, the preset internal parameters and the imaging principle, a real camera and a real object do not need to be deployed, and a real image shooting process does not need to be carried out, so that the complexity of acquiring training data can be reduced, and the acquisition efficiency of the training data is improved. Of course, it is not necessary for any one product or method of practicing the invention to achieve all of the advantages set forth above at the same time.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are necessary for the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention and that other embodiments may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flowchart of a method for acquiring three-dimensional reconstruction training data according to an embodiment of the present invention;

FIG. 2 is a specific flowchart of step S101 in the embodiment shown in FIG. 1;

FIG. 3 is a schematic diagram of a deployment of virtual cameras based on the embodiment shown in FIG. 2;

FIG. 4 is a specific flowchart of step S103 in the embodiment shown in FIG. 1;

FIG. 5 is a schematic diagram of a pinhole imaging system according to an embodiment of the present invention;

FIG. 6 is a flow chart of an iterative training mode based on the three-dimensional reconstruction algorithm of the embodiment shown in FIG. 1;

fig. 7 is a schematic structural diagram of an apparatus for acquiring three-dimensional reconstructed training data according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, those of ordinary skill in the art will be able to devise all other embodiments that are obtained based on this application and are within the scope of the present invention.

In order to achieve the above objective, the embodiments of the present invention provide a method, an apparatus, an electronic device, a computer readable storage medium, and a computer program product for acquiring three-dimensional reconstruction training data, where the method and the apparatus can be applied to various electronic devices that need to acquire three-dimensional reconstruction training data, for example, a computer, a processor, a processing chip, etc., and are not limited in particular, and the method for acquiring three-dimensional reconstruction training data is described in detail below.

Fig. 1 is a flow chart of a method for acquiring three-dimensional reconstruction training data according to an embodiment of the present invention, including:

s101, arranging a plurality of virtual cameras in a virtual three-dimensional environment to obtain external parameters of each virtual camera;

s102, generating a plurality of virtual three-dimensional point coordinates in the field of view of the plurality of virtual cameras;

s103, aiming at each virtual three-dimensional point coordinate, determining a target two-dimensional point coordinate which is mapped to a two-dimensional image acquired by each virtual camera by the virtual three-dimensional point coordinate according to the external reference and the preset internal reference of each virtual camera, and obtaining a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point;

and S104, determining each virtual three-dimensional point coordinate and the corresponding target two-dimensional point coordinate set as training data of a three-dimensional reconstruction algorithm.

In the scheme provided by the embodiment of the invention, the electronic device can deploy a plurality of virtual cameras in the virtual three-dimensional environment to obtain the external parameters of each virtual camera, generate a plurality of virtual three-dimensional point coordinates in the field of view of the plurality of virtual cameras, and determine that the virtual three-dimensional point coordinates are mapped to target two-dimensional point coordinates in the two-dimensional image acquired by each virtual camera according to the external parameters and preset internal parameters of each virtual three-dimensional point coordinate respectively to obtain a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point, and further determine each virtual three-dimensional point coordinate and the corresponding target two-dimensional point coordinate set as training data of a three-dimensional reconstruction algorithm. In the scheme, a plurality of virtual cameras are deployed in a virtual three-dimensional environment, a plurality of virtual three-dimensional point coordinates are generated to represent the position of an object, and then a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point can be determined according to the external parameters of the virtual cameras, the preset internal parameters and the imaging principle, a real camera and a real object do not need to be deployed, and a real image shooting process does not need to be carried out, so that the complexity of acquiring training data can be reduced, and the acquisition efficiency of the training data is improved.

When the three-dimensional reconstruction training data needs to be acquired, the electronic device may execute the step S101, that is, deploy a plurality of virtual cameras in the virtual three-dimensional environment, to obtain external parameters of each virtual camera. Wherein a virtual camera may be understood as a model for simulating the process of capturing images by the camera in a virtual environment. The number of deployed virtual cameras may be 6, 8, 10, etc., and the number of specific deployed virtual cameras is not limited.

When the virtual cameras are deployed, the positions and the main optical axis directions of the virtual cameras need to be determined, namely the external parameters of the virtual cameras are determined, so that a plurality of virtual cameras are deployed in a virtual three-dimensional environment, and the external parameters of each virtual camera can be obtained. As a real-time manner, the electronic device may deploy the virtual camera at a preset position in the virtual three-dimensional environment, and may determine a main optical axis direction of the virtual camera as a preset direction. The preset positions and preset directions can be preset according to the number of virtual cameras to be deployed and the like, and are not particularly limited herein.

After determining the external parameters of each virtual camera, the electronic device may perform step S102 described above, that is, generate a plurality of virtual three-dimensional point coordinates within the field of view of the plurality of virtual cameras. The field of view range may be an area in the virtual three-dimensional environment that can be acquired by the virtual camera, and if the virtual camera is corresponding to a real camera in the real world, the field of view range of the virtual cameras is equivalent to the world area that can be acquired by the real cameras. If the object position exceeds the field of view range, the object position cannot be acquired by the virtual camera, so in order to simulate the process of acquiring the object image of the real scene, the electronic device can generate a plurality of virtual three-dimensional point coordinates in the field of view range of the plurality of virtual cameras, wherein the plurality of virtual three-dimensional point coordinates are positions of points on the object in the simulated real scene.

As one implementation, the electronic device may randomly generate a plurality of virtual three-dimensional point coordinates within a field of view of a plurality of virtual cameras. For example, a predetermined number of virtual three-dimensional point coordinates may be randomly generated within the field of view of the plurality of virtual cameras. The preset number may be 6, 8, 10, etc., and is not particularly limited.

As another embodiment, since the real object generally has a certain shape, the electronic device may generate virtual three-dimensional point coordinates according to a preset shape within the field of view of the plurality of virtual cameras. For example, the electronic device may generate eight virtual three-dimensional point coordinates, which are coordinates of eight vertices of one cube, in terms of the cubes within the field of view of the plurality of virtual cameras. The preset shape may be a cube, a cuboid, a tetrahedron, or some irregular shapes, and the like, and is not particularly limited.

According to the imaging principle of the camera, the external reference matrix of the camera is used for identifying the conversion relation between the world coordinate system and the camera coordinate system, and the internal reference matrix of the camera is used for identifying the conversion relation between the camera coordinate system and the image coordinate system, so that the two-dimensional coordinates of the three-dimensional point coordinates in the two-dimensional image acquired by the camera can be calculated according to the external reference matrix, the internal reference matrix and the three-dimensional point coordinates of the camera.

Similarly, after the electronic device obtains the external parameters of each virtual camera and the virtual three-dimensional point coordinates, the electronic device can determine that the virtual three-dimensional point coordinates are mapped to target two-dimensional point coordinates in the two-dimensional image acquired by each virtual camera according to the external parameters and preset internal parameters of each virtual camera according to each virtual three-dimensional point coordinate, and then a target two-dimensional point set corresponding to the virtual three-dimensional point can be obtained. The external parameter matrix of the virtual camera is a rotation translation matrix, and can be determined according to the position of the virtual camera and the direction of the main optical axis.

The internal parameters of the virtual camera may include: the horizontal direction pixel focal length, the vertical direction pixel focal length, the abscissa of imaging of the principal point of the virtual camera in the pixel coordinate system, and the ordinate of imaging of the principal point of the virtual camera in the pixel coordinate system. Since the virtual camera is a virtual model in a virtual three-dimensional environment and is not a real camera, the electronic device may set its internal parameters in advance. In one embodiment, the internal reference matrix of the virtual camera may be set as:

K＝[f _x ，0，c _x ；0，f _y ，c _y ；0，0，1]

wherein K represents an internal reference matrix of the virtual camera, f _x Representing the horizontal pixel focal length of the virtual camera, c _x An abscissa representing imaging of a principal point of the virtual camera in an image coordinate system, f _y Representing the sag of a virtual cameraStraight direction pixel focal length, c _y Representing the ordinate of the principal point of the virtual camera imaged in the image coordinate system.

In step S104, after determining the target two-dimensional point coordinate set corresponding to each virtual three-dimensional point coordinate, the electronic device may determine each virtual three-dimensional point coordinate and the target two-dimensional point coordinate set corresponding to each virtual three-dimensional point coordinate as training data of the three-dimensional reconstruction algorithm, and use the training data to train the three-dimensional reconstruction algorithm to obtain an algorithm capable of accurately performing three-dimensional reconstruction.

As shown in fig. 2, the step of deploying a plurality of virtual cameras in the virtual three-dimensional environment to obtain the external parameters of each virtual camera may include:

s201, determining a coordinate point in a virtual three-dimensional environment as a reference point;

to facilitate determining the locations of multiple virtual cameras, the electronic device may determine a coordinate point as a reference point in the virtual three-dimensional environment. For example, an origin of a three-dimensional coordinate system corresponding to the virtual three-dimensional environment may be used as a reference point, so that the position of the virtual camera may be conveniently determined. Of course, a preset coordinate point in the three-dimensional coordinate system corresponding to the virtual three-dimensional environment can be taken as a reference point, which is reasonable.

S202, determining the positions of a plurality of virtual cameras according to a preset mode by taking the datum point as a reference;

after determining the reference points, the electronic device may determine the positions of the plurality of virtual cameras in a preset manner. In one embodiment, the position of a first virtual camera may be determined in terms of distance from a reference point, then the position of a second virtual camera is determined in terms of distance from the first virtual camera, and so on until the positions of all virtual cameras are determined.

S203, adjusting the main optical axis direction of each virtual camera so that the field of view range of the plurality of virtual cameras has a superposition area.

Because the field of view scope of the camera has the coincidence area and can collect the picture of the object in this coincidence area at the same time, so the electronic equipment can adjust the main optical axis direction of each virtual camera, in order to make the field of view scope of multiple virtual cameras have the coincidence area, in this way, produce the coordinate of virtual three-dimensional point in this overlapping area, can simulate the course of the real camera gathering the real picture of the real object in the real scene accurately, guarantee the accuracy of the training data.

In this embodiment, the electronic device may determine one coordinate point in the virtual three-dimensional environment as a reference point, determine the positions of the plurality of virtual cameras according to a preset manner with the reference point as a reference point, and further adjust the main optical axis direction of each virtual camera so that the field of view ranges of the plurality of virtual cameras have overlapping areas. Therefore, the external parameters of the virtual camera can be accurately determined, and the accuracy of the training data determined later is ensured.

In one embodiment, the step of determining the positions of the plurality of virtual cameras in a preset manner based on the reference point may include: determining the datum point as a deployment center point; and deploying a plurality of virtual cameras around the deployment center point in a preset array shape to obtain the position of each virtual camera.

The electronic device may determine the reference point as a deployment center point, and further deploy a plurality of virtual cameras around the deployment center point in a preset array shape, so as to obtain a position of each virtual camera. In one embodiment, the reference point may be an origin of a three-dimensional coordinate system corresponding to the virtual environment, that is, the electronic device may determine the origin of the three-dimensional coordinate system as a deployment center point, and then may deploy a plurality of virtual cameras in a preset array shape around the origin of the three-dimensional coordinate system. The preset array shape may be a circle, a square, a cube, a sphere, or the like, and the embodiment of the present invention is not particularly limited.

As an implementation manner, multiple virtual cameras can be deployed around a deployment center point according to a circle with a preset radius, so as to obtain the position of each virtual camera. For example, the electronic device may deploy 8 virtual cameras in the x-z plane of the three-dimensional coordinate system in a circle with a radius of 10 meters using the origin of the three-dimensional coordinate system as a deployment center point.

In this embodiment, the electronic device may determine the reference point as a deployment center point, and further deploy a plurality of virtual cameras around the deployment center point in a preset array shape, so as to obtain a position of each virtual camera. Therefore, the position of each virtual camera can be simply, conveniently and quickly determined, and the acquisition efficiency of training data can be further improved.

As an implementation manner of the embodiment of the present invention, the step of deploying a plurality of virtual cameras around the deployment center point in a preset array shape to obtain a position of each virtual camera may include:

wherein, camera _center [i]The position coordinates of the ith virtual camera are represented, N represents the total number of virtual cameras, and distance represents the distance of the virtual cameras from the deployment center point. By adopting the formula, the virtual cameras can be deployed in a circular array in an x-z plane of a three-dimensional coordinate system corresponding to the virtual three-dimensional environment, so that the distance from the virtual camera to a deployment center point is the radius of the circle, namely the distance is the radius of the circle, and N virtual cameras can be uniformly deployed on the circumference of the circle with the radius of the distance and the origin of the three-dimensional coordinate system as the center of the circle.

It can be seen that, in this embodiment, the electronic device may calculate the position coordinate of each virtual camera according to the above formula, and according to the formula, N virtual cameras may be uniformly deployed on a circle with a radius being distance and an origin of the three-dimensional coordinate system being a center of a circle, so as to conveniently determine the position of the virtual camera.

As an implementation manner of the embodiment of the present invention, the step of adjusting the main optical axis direction of each virtual camera may include: the main optical axis direction of each virtual camera is respectively adjusted to be towards the deployment center point.

Because the virtual cameras are deployed in a preset array shape around the deployment center point, the deployment center point is positioned at the center position of the plurality of virtual cameras, and the electronic equipment can adjust the main optical axis direction of each virtual camera to face the deployment center point, so that the main optical axis directions of the plurality of virtual cameras face the same position, and the field of view ranges of the virtual cameras can be ensured to be overlapped as much as possible.

In one embodiment, the initial direction of the primary optical axis of the virtual camera may be the same, and the electronic device may adjust the primary optical axis direction of the virtual camera using a lookahead function to adjust the primary optical axis direction of the virtual camera toward the deployment center point.

In the three-dimensional coordinate system corresponding to the virtual three-dimensional environment, with the origin as the deployment center point, in the case where N is 8, the electronic device may determine the position coordinates of each virtual camera according to the above formula, such as the virtual camera 310 shown in fig. 3. The electronic device may adjust the main optical axis direction (the direction indicated by the arrow in fig. 3) of each virtual camera toward the deployment center point, and may further generate a plurality of virtual three-dimensional point coordinates 320 within the field of view of the plurality of virtual cameras.

The rotation matrix R of the virtual camera may represent a process of adjusting the direction of the main optical axis of the virtual camera, where the rotation matrix R is a 3×3 matrix, and each column of the matrix corresponds to a rotation component, for example, a first column corresponds to a rotation component in the x direction, a second column corresponds to a rotation component in the y direction, and a third column corresponds to a rotation component in the z direction. Therefore, when the main optical axis direction of the virtual camera is adjusted, the electronic device can determine the rotation matrix R of the virtual camera according to the rotation angles of the main optical axis direction of the virtual camera, which correspond to the x axis, the y axis and the z axis of the three-dimensional coordinate system respectively.

It can be seen that, in this embodiment, the electronic device may adjust the main optical axis direction of each virtual camera toward the deployment center point, so that the overlapping area of the field of view ranges of the multiple virtual cameras may be larger. Therefore, when the virtual three-dimensional point is generated later, as many virtual cameras as possible can acquire images of the virtual three-dimensional point, namely images of more angles of the three-dimensional point can be acquired, so that the acquired three-dimensional reconstruction training data is richer.

As shown in fig. 4, the step of determining, according to the external parameter and the preset internal parameter of each virtual camera, that the virtual three-dimensional point coordinate maps to the target two-dimensional point coordinate in the two-dimensional image acquired by each virtual camera may include:

s401, calculating a mapping matrix corresponding to each virtual camera according to the external parameters and the preset internal parameters of the virtual camera;

and S402, multiplying the three-dimensional homogeneous coordinates corresponding to the virtual three-dimensional point coordinates by a mapping matrix to obtain two-dimensional homogeneous coordinates of the two-dimensional point of the target in the two-dimensional image acquired by the virtual camera.

The mapping matrix corresponding to the virtual camera is an imaging matrix corresponding to the virtual camera and is used for representing the mapping relationship between the three-dimensional coordinates of the three-dimensional point and the two-dimensional coordinates of the two-dimensional point corresponding to the three-dimensional point in the two-dimensional image. According to the imaging principle of the camera, the mapping matrix of the camera is the product of the internal reference matrix and the external reference matrix of the camera, so that the mapping matrix of the virtual camera can be calculated by using the following formula:

P[i]＝K[i][R[i]|t[i]]

wherein, pi represents the mapping matrix of the ith virtual camera, ki represents the internal reference matrix of the ith virtual camera, ri represents the main optical axis direction of the ith virtual camera, that is, the rotation matrix of the ith virtual camera, t i represents the translation vector of transforming the point in the virtual three-dimensional coordinate system to the three-dimensional camera coordinate system of the ith virtual camera, and Ri is the rotation translation matrix of the virtual camera, that is, the external reference matrix of the virtual camera.

In an embodiment, the step of calculating the mapping matrix corresponding to the virtual camera according to the external parameters and the preset internal parameters of the virtual camera may include:

calculating according to the external parameters of the virtual camera to obtain a translation vector corresponding to the virtual camera; and determining a mapping matrix corresponding to the virtual camera based on the translation vector, the preset internal parameter of the virtual camera and the main optical axis direction included by the external parameter of the virtual camera.

The electronic device may calculate a translation vector of a point in the virtual three-dimensional coordinate system transformed to the three-dimensional camera coordinate system of the virtual camera from the external parameters of the virtual camera, i.e., the position coordinates and the rotation matrix of the virtual camera. Wherein the translation vector is used for representing the translation relation of the three-dimensional coordinates of the three-dimensional point to the three-dimensional camera coordinate system of the virtual camera, and then, for the three-dimensional point B, the corresponding coordinates in the three-dimensional camera coordinate system converted to the virtual camera are A, and according to the physical meaning represented by the translation vector t [ i ], A=R [ i ] ×B+t [ i ], t [ i ] =A-R [ i ] ×B.

If the coordinate position of the virtual Camera in the virtual three-dimensional coordinate system is Camera _center [i]The corresponding coordinates a in the three-dimensional camera coordinate system transformed to the virtual camera itself are (0, 0), so the translation vector of the point transformed to the three-dimensional camera coordinate system of the virtual camera in the virtual three-dimensional coordinate system can be calculated by using the following formula:

t[i]＝-R[i]×Camera _center [i]

Wherein, ti]Translation vector representing transformation of point in virtual three-dimensional coordinate system to three-dimensional camera coordinate system of ith virtual camera, ri]Rotation matrix representing ith virtual Camera, camera _center [i]And representing the position coordinates of the ith virtual camera in the virtual three-dimensional coordinate system.

After determining the translation vector corresponding to the virtual camera, the electronic device may determine the mapping matrix corresponding to the virtual camera, that is, pi=ki|r|t|based on the translation vector, the preset internal parameter of the virtual camera, and the main optical axis direction included in the external parameter of the virtual camera.

And then, according to the imaging principle, the electronic equipment multiplies the three-dimensional homogeneous coordinate corresponding to the virtual three-dimensional point coordinate by the mapping matrix, so that the two-dimensional homogeneous coordinate of the target two-dimensional point in the two-dimensional image acquired by the virtual camera can be obtained. That is, or, the electronic device may calculate the two-dimensional homogeneous coordinates of the virtual three-dimensional point mapped to the target two-dimensional point in the two-dimensional image captured by the virtual camera using the following formula:

x[i，j]＝P[i]×X[j]

wherein X [ i, j ] represents the two-dimensional homogeneous coordinates of the target two-dimensional point of the jth virtual three-dimensional point in the two-dimensional image acquired by the ith virtual camera, pi represents the mapping matrix of the ith virtual camera, and X [ j ] represents the three-dimensional homogeneous coordinates of the jth virtual three-dimensional point.

It can be seen that, in this embodiment, the electronic device may determine, by calculating a mapping matrix corresponding to the virtual camera, a mapping relationship between a three-dimensional coordinate of the virtual three-dimensional point and a two-dimensional coordinate of a target two-dimensional point in the two-dimensional image acquired by the virtual camera, and according to this mapping relationship, may simply and quickly calculate the two-dimensional coordinate of the target two-dimensional point of the virtual three-dimensional point in the two-dimensional image acquired by the virtual camera.

In another embodiment, the electronic device may further calculate, according to the three-dimensional coordinates of the virtual three-dimensional point, the position of the virtual camera, the main optical axis direction, and the internal reference, the two-dimensional coordinates of the target two-dimensional point of the virtual three-dimensional point in the two-dimensional image acquired by the virtual camera using the principle of pinhole imaging.

Referring to fig. 5, a represents three-dimensional coordinates of a virtual three-dimensional point, C represents a position of a virtual camera, a straight line BC represents a horizontal plane in which the virtual camera is located, a perpendicular line is drawn from a passing point a to the straight line BC, a foot B, a line segment BC represents a position of the virtual camera and a horizontal distance of the virtual three-dimensional point, ECB represents a main optical axis direction of the virtual camera, D represents two-dimensional coordinates of a target two-dimensional point in a two-dimensional image, a straight line ED represents an imaging plane of the virtual camera, a perpendicular line is drawn from a passing point C to the straight line ED, a foot E, and a line segment CE represents a focal length in an internal reference of the virtual camera.

Wherein, the angle ABC and the angle CED are right angles, and then the three-dimensional coordinate point A of the virtual three-dimensional point, the position C of the virtual camera and the main optical axis direction ECB can form a right triangle ABC; the internal reference of the virtual camera and the two-dimensional coordinates D of the target two-dimensional point in the two-dimensional image acquired by the virtual camera can form a right triangle DEC; according to the principle of aperture imaging, the right triangle ABC is similar to the right triangle DEC, and according to the similarity, the two-dimensional coordinates of the target two-dimensional point in the two-dimensional image acquired by the virtual camera can be calculated by utilizing the coordinates of the virtual three-dimensional point, the position of the virtual camera, the main optical axis direction and the internal reference.

As shown in fig. 6, after the step of determining each virtual three-dimensional point coordinate and the corresponding target two-dimensional point coordinate set as training data of the three-dimensional reconstruction algorithm, the method may further include:

s601, determining three-dimensional reconstruction point coordinates corresponding to the virtual three-dimensional point coordinates through a three-dimensional reconstruction algorithm based on a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point coordinates;

after each virtual three-dimensional point coordinate and the corresponding target two-dimensional point coordinate set are determined as training data of the three-dimensional reconstruction algorithm, the electronic device can determine the three-dimensional reconstruction point coordinate corresponding to the virtual three-dimensional point coordinate through the three-dimensional reconstruction algorithm based on the target two-dimensional point coordinate set corresponding to the virtual three-dimensional point coordinate.

S602, performing iterative training on the three-dimensional reconstruction algorithm according to the difference between the three-dimensional reconstruction point coordinates and the virtual three-dimensional point coordinates.

Because the three-dimensional reconstruction algorithm may not be accurate, the three-dimensional reconstruction point coordinate corresponding to the virtual three-dimensional point coordinate determined by the three-dimensional reconstruction algorithm may have a certain difference from the corresponding virtual three-dimensional point coordinate, so the electronic device may compare the three-dimensional reconstruction point coordinate with the virtual three-dimensional point, and determine the difference between the three-dimensional reconstruction point coordinate and the virtual three-dimensional point coordinate.

Furthermore, the electronic device can perform iterative training on the three-dimensional reconstruction algorithm according to the difference, and the accuracy of the three-dimensional reconstruction algorithm is gradually improved. Any training method in the three-dimensional reconstruction field can be adopted for the iterative training method of the three-dimensional reconstruction algorithm, for example, countermeasure training can be performed on the three-dimensional reconstruction algorithm, so that iterative optimization and the like are performed on the three-dimensional reconstruction algorithm, and the embodiment of the invention is not particularly limited and described herein.

The three-dimensional reconstruction algorithm may be SfM (Structure from Motion, motion restoration structure) algorithm, MC (Marching Cubes) algorithm, or the like, and the specific three-dimensional reconstruction algorithm is not limited. For example, for performing iterative optimization on the SfM algorithm, a target two-dimensional point set corresponding to each virtual three-dimensional point coordinate may be input to the SfM algorithm to obtain a three-dimensional reconstruction point coordinate, and the three-dimensional reconstruction point coordinate and the virtual three-dimensional point coordinate are compared to obtain a difference between the two points, so that the SfM algorithm is subjected to countermeasure training based on the difference, thereby performing iterative optimization on the SfM algorithm.

It can be seen that, in this embodiment, the electronic device may further determine, for each virtual three-dimensional point coordinate, a three-dimensional reconstruction point coordinate corresponding to the virtual three-dimensional point coordinate through a three-dimensional reconstruction algorithm based on a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point coordinate, and further perform iterative training on the three-dimensional reconstruction algorithm according to a difference between the three-dimensional reconstruction point coordinate and the virtual three-dimensional point coordinate. Thus, a three-dimensional reconstruction algorithm with higher accuracy can be trained.

In some related schemes, a plurality of objects are arranged in a real physical space, more two-dimensional images are acquired for each object from different angles, and the mapping relation between the three-dimensional coordinates of the object in the real physical space and the two-dimensional coordinates in the two-dimensional images is determined and used as training data. And optimizing a three-dimensional reconstruction algorithm by using the training data. However, in these schemes, three-dimensional coordinates of an object in a real physical space need to be measured as a training sample of countermeasure training, and since the training sample needs to be measured manually, the accuracy of obtaining the training sample of countermeasure training required in the process of optimizing the three-dimensional reconstruction algorithm is low.

However, in this embodiment, the virtual three-dimensional point coordinates and the two-dimensional point coordinate sets corresponding to the virtual three-dimensional point coordinates may be determined as training data of the three-dimensional reconstruction algorithm, and images of different angles of the real object do not need to be acquired.

By applying the scheme of the embodiment of the invention, a plurality of virtual cameras are deployed in a virtual three-dimensional environment to obtain the external parameters of each virtual camera; generating a plurality of virtual three-dimensional point coordinates within the field of view of the plurality of virtual cameras; for each virtual three-dimensional point coordinate, determining that the virtual three-dimensional point coordinate is mapped to a target two-dimensional point coordinate in a two-dimensional image acquired by each virtual camera according to an external reference and a preset internal reference of each virtual camera, and obtaining a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point coordinate; and determining each virtual three-dimensional point coordinate and the corresponding target two-dimensional point coordinate set as training data of a three-dimensional reconstruction algorithm. Therefore, in the scheme, the virtual camera is deployed to generate the virtual three-dimensional point, and the position of the virtual three-dimensional point can be directly determined, that is, in the scheme, the real camera and the real object do not need to be deployed, and the real image shooting process does not need to be carried out, so that the complexity of acquiring the training data can be reduced, the acquisition efficiency of the training data is improved, and meanwhile, the cost of acquiring the training data is also reduced.

Corresponding to the above method embodiment, the embodiment of the present invention further provides an apparatus for acquiring three-dimensional reconstruction training data, as shown in fig. 7, including:

A deployment module 710, configured to deploy a plurality of virtual cameras in a virtual three-dimensional environment, to obtain external parameters of each virtual camera;

a generating module 720, configured to generate a plurality of virtual three-dimensional point coordinates within a field of view of the plurality of virtual cameras;

the mapping module 730 is configured to determine, for each virtual three-dimensional point coordinate, that the virtual three-dimensional point coordinate maps to a target two-dimensional point coordinate in the two-dimensional image acquired by each virtual camera according to the external parameter and the preset internal parameter of each virtual camera, so as to obtain a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point;

the determining module 740 is configured to determine each virtual three-dimensional point coordinate and the corresponding target two-dimensional point coordinate set as training data of the three-dimensional reconstruction algorithm.

In one embodiment, the deployment module 710 may include:

In an embodiment, the camera position determining unit may include:

In an embodiment, the camera position determining subunit may comprise:

In one embodiment, the optical axis direction adjustment unit may include:

In one embodiment, the mapping module 730 may include:

the mapping matrix calculation unit is used for calculating a mapping matrix of each virtual camera according to the external parameters and the preset internal parameters of the virtual camera;

the mapping matrix is used for representing the mapping relation between the three-dimensional coordinates of the three-dimensional point and the two-dimensional coordinates of the two-dimensional point corresponding to the three-dimensional point in the two-dimensional image.

In one embodiment, the mapping matrix calculation unit may include:

the translation vector calculation operator unit is used for calculating and obtaining a translation vector corresponding to the virtual camera according to the external parameters of the virtual camera;

the translation vector is used for representing the translation relation of the three-dimensional coordinates of the three-dimensional point transformed to the three-dimensional camera coordinate system of the virtual camera.

In one embodiment, the apparatus may further include:

The embodiment of the present invention further provides an electronic device, as shown in fig. 8, including a processor 801, a communication interface 802, a memory 803, and a communication bus 804, where the processor 801, the communication interface 802, and the memory 803 complete communication with each other through the communication bus 804,

a memory 803 for storing a computer program;

The processor 801 is configured to implement any one of the above-described three-dimensional reconstruction training data acquisition methods when executing the program stored in the memory 803.

The communication bus mentioned above for the electronic devices may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.

The communication interface is used for communication between the electronic device and other devices.

The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

In yet another embodiment of the present invention, a computer readable storage medium is provided, in which a computer program is stored, the computer program implementing the steps of any of the above-mentioned three-dimensional reconstruction training data acquisition methods when executed by a processor.

In the scheme provided by the embodiment of the invention, when the computer program stored in the computer readable storage medium is executed by the processor, a plurality of virtual cameras can be deployed in a virtual three-dimensional environment to obtain the external parameters of each virtual camera, a plurality of virtual three-dimensional point coordinates are generated in the field of view of the plurality of virtual cameras, the virtual three-dimensional point coordinates are respectively determined according to the external parameters and preset internal parameters of each virtual camera aiming at each virtual three-dimensional point coordinate, the virtual three-dimensional point coordinates are mapped to target two-dimensional point coordinates in a two-dimensional image acquired by each virtual camera, a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point is obtained, and then each virtual three-dimensional point coordinate and the target two-dimensional point coordinate set corresponding to each virtual three-dimensional point coordinate are determined as training data of a three-dimensional reconstruction algorithm. In the scheme, a plurality of virtual cameras are deployed in a virtual three-dimensional environment, a plurality of virtual three-dimensional point coordinates are generated to represent the position of an object, and then a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point can be determined according to the external parameters of the virtual cameras, the preset internal parameters and the imaging principle, a real camera and a real object do not need to be deployed, and a real image shooting process does not need to be carried out, so that the complexity of acquiring training data can be reduced, and the acquisition efficiency of the training data is improved.

In yet another embodiment of the present invention, a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of acquiring three-dimensional reconstruction training data of any of the above embodiments is also provided.

In the scheme provided by the embodiment of the invention, when the computer program product runs on a computer, a plurality of virtual cameras can be deployed in a virtual three-dimensional environment to obtain the external parameters of each virtual camera, a plurality of virtual three-dimensional point coordinates are generated in the field of view of the plurality of virtual cameras, the virtual three-dimensional point coordinates are respectively mapped to target two-dimensional point coordinates in a two-dimensional image acquired by each virtual camera according to the external parameters and preset internal parameters of each virtual camera for each virtual three-dimensional point coordinate, a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point is obtained, and then each virtual three-dimensional point coordinate and the target two-dimensional point coordinate set corresponding to the virtual three-dimensional point coordinate are determined as training data of a three-dimensional reconstruction algorithm. In the scheme, a plurality of virtual cameras are deployed in a virtual three-dimensional environment, a plurality of virtual three-dimensional point coordinates are generated to represent the position of an object, and then a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point can be determined according to the external parameters of the virtual cameras, the preset internal parameters and the imaging principle, a real camera and a real object do not need to be deployed, and a real image shooting process does not need to be carried out, so that the complexity of acquiring training data can be reduced, and the acquisition efficiency of the training data is improved.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for apparatus embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments, the description is relatively simple, as relevant to the method embodiments being referred to in the description of the method embodiments.

The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. The method for acquiring the three-dimensional reconstruction training data is characterized by comprising the following steps of:

generating a plurality of virtual three-dimensional point coordinates within the field of view of the plurality of virtual cameras, the plurality of virtual three-dimensional point coordinates being locations of points on objects in a simulated real scene;

for each virtual three-dimensional point coordinate, determining a target two-dimensional point coordinate of the virtual three-dimensional point coordinate mapped to a two-dimensional image acquired by each virtual camera according to the external reference and the preset internal reference of each virtual camera, and obtaining a target two-dimensional point coordinate set corresponding to the virtual three-dimensional point, wherein the target two-dimensional point coordinate set comprises a plurality of target two-dimensional point coordinates;

2. The method of claim 1, wherein deploying a plurality of virtual cameras in the virtual three-dimensional environment, obtaining the extrinsic parameters of each virtual camera, comprises:

3. The method of claim 2, wherein determining the positions of the plurality of virtual cameras in a predetermined manner based on the reference point comprises:

determining the datum point as a deployment center point;

4. The method of claim 3, wherein deploying a plurality of virtual cameras in a preset array shape around the deployment center point, resulting in a position of each virtual camera, comprises:

5. A method according to claim 3, wherein said adjusting the main optical axis direction of each virtual camera comprises:

6. The method of claim 1, wherein determining, based on the external parameters and the preset internal parameters of each virtual camera, that the virtual three-dimensional point coordinates map to target two-dimensional point coordinates in the two-dimensional image acquired by each virtual camera comprises:

7. The method of claim 6, wherein the calculating the mapping matrix corresponding to the virtual camera according to the external parameters and the preset internal parameters of the virtual camera comprises:

8. The method of any of claims 1-7, wherein after said determining each virtual three-dimensional point coordinate and its corresponding set of target two-dimensional point coordinates as training data for a three-dimensional reconstruction algorithm, the method further comprises:

9. An acquisition device for three-dimensional reconstruction training data, comprising:

a generation module, configured to generate a plurality of virtual three-dimensional point coordinates within a field of view of the plurality of virtual cameras, where the plurality of virtual three-dimensional point coordinates are positions of points on an object in a simulated real scene;

and the determining module is used for determining each virtual three-dimensional point coordinate and the corresponding target two-dimensional point coordinate set as training data of a three-dimensional reconstruction algorithm, wherein the target two-dimensional point coordinate set comprises a plurality of target two-dimensional point coordinates.

10. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;

a memory for storing a computer program;

a processor for carrying out the method steps of any one of claims 1-8 when executing a program stored on a memory.