JP2015197374A - Three-dimensional shape estimation device and three-dimensional shape estimation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a three-dimensional shape estimation device that estimates a three-dimensional shape of an object from an image photographed with one camera.SOLUTION: A three-dimensional shape estimation method includes the steps of: acquiring first image data obtained by photographing an object (S601); acquiring second image data obtained by reproducing a reproduction model including parameters showing a three-dimensional shape of the object in computer graphics (S607); and comparing the pieces of image data while changing the parameters (S608) to adjust the parameters of the reproduction model.

Description

  The present invention relates to a technique for estimating a three-dimensional shape of an object.

  Conventionally, as a technique for acquiring a three-dimensional shape of an object, there is a technique for imaging a subject from a plurality of viewpoints with a marker attached to the subject, and estimating the shape from the positional relationship of the markers in each captured image.

  Patent Document 1 discloses a technique for estimating a depth, that is, a three-dimensional shape, by generating an interference fringe on a subject, imaging a plurality of times from different angles, and considering the interference fringe as a contour line.

  In Patent Document 2, a subject is imaged by a video camera while controlling the rotation angle of the stepping motor. Based on the obtained images for each rotation angle and the rotation angle of the stepping motor, the movement of the feature point on the surface of the subject is detected. A technique is disclosed in which a rotation axis of a subject is searched based on the detected movement of feature points, and three-dimensional shape information of the subject is created based on the rotation axis.

Japanese Patent Laid-Open No. 11-118442 JP-A-8-14858

  However, in the prior art, it is necessary to use a plurality of cameras or perform imaging from a plurality of viewpoints while moving one camera.

  A three-dimensional shape estimation apparatus according to the present invention generates a reproduction model including first acquisition means for acquiring first image data obtained by imaging an object and a parameter indicating the three-dimensional shape of the object Means, second acquisition means for acquiring second image data obtained by reproducing the reproduction model by computer graphics, and comparing the first image data and the second image data Adjusting means for adjusting the parameters of the reproduction model.

  According to the present invention, it is possible to estimate the three-dimensional shape of an object from an image captured by one imaging device without moving the camera.

It is a figure which shows the example of a structure in the system which estimates the three-dimensional shape in the Example of this invention. The block diagram which shows the system configuration example of the main processing apparatus in the Example of this invention. The conceptual diagram which simplified paper. The figure which showed the outline | summary of CG reproduction in the Example of this invention. The block diagram which shows the structural example which performs the three-dimensional shape estimation process in the Example of this invention. 5 is a flowchart illustrating an example of processing for acquiring a three-dimensional shape according to the first embodiment. FIG. 3 is a diagram illustrating an example of a real image and a CG image according to the first embodiment. The figure which shows the mode of the update of the vertex coordinate in Example 1. FIG. FIG. 6 is a diagram illustrating how a vertex is added in the first embodiment. FIG. 10 is a diagram illustrating an example of area division in the second embodiment. 10 is a flowchart illustrating an example of processing for acquiring a three-dimensional shape according to the second embodiment. The figure which shows the addition result of the vertex in Example 2. FIG. FIG. 10 is a diagram illustrating a method for setting an initial value of a reproduction model according to a third embodiment. 10 is a flowchart illustrating an example of processing for acquiring a three-dimensional shape according to a third embodiment.

  Hereinafter, the present invention will be described in detail based on preferred embodiments with reference to the accompanying drawings. In addition, the structure shown in the following Examples is only an example, and this invention is not limited to the structure shown in figure.

  In this embodiment, a method for estimating the three-dimensional shape of a non-rigid object will be described. In the present embodiment, processing when paper is used as the non-rigid object will be described. A non-rigid object is an object whose three-dimensional shape changes.

  FIG. 1 is a diagram illustrating a configuration example of a system in the present embodiment. FIG. 1A is a diagram illustrating a system configuration example including the image processing apparatus according to the present exemplary embodiment. This system includes a main processing device 101 that is a three-dimensional shape estimation device and a camera device 102. A camera apparatus 102 that captures paper is connected to a main processing apparatus 101 that performs various calculation processes and device control. The camera apparatus 102 and the main processing apparatus 101 are connected by a USB line (Universal Serial Bus). Using this connection line, an imaging command signal is transmitted from the main processing apparatus 101 to the camera apparatus 102, and captured image data is transmitted from the camera apparatus 102 to the main processing apparatus 101. In this system, a pedestal 103 is installed, and a paper 104 can be placed thereon.

  FIG. 1B is a diagram showing a coordinate system in the present embodiment. The center part of the pedestal 103 is the origin O, and an x, y, z coordinate system is configured. The coordinate system is in meters.

  FIG. 2 is a block diagram illustrating a configuration example of the main processing device 101. The main processing device 101 includes a CPU 201, a main memory 202, a storage device 203, a GPU 204, and a camera input board 205. The CPU 201 executes arithmetic processing and various programs. The main memory 202 provides the CPU 201 with programs, data, work areas, and the like necessary for processing. The storage device 203 is a device that accumulates various data and programs, and usually uses a hard disk. A GPU (Graphic processing unit) 204 is used when performing CG (Computer Graphics) rendering. Since the GPU 204 specializes in parallel operation processing, rendering can be executed faster than the CPU 201. The camera input board 205 is a base device connected to the camera device 102 via a USB line, and transmits various signal commands to the camera device 102. The camera input board 205 also receives captured image data from the camera device 102.

  FIG. 3 is a conceptual diagram in which the paper 104 is simplified. As shown in the reproduction model 301, the paper 104 has a mesh structure composed of a plurality of vertices and polygons formed by the plurality of vertices. Each vertex has a three-dimensional coordinate value, and the size and shape of the paper, that is, the three-dimensional shape is determined by these vertex coordinates.

FIG. 4 is a diagram showing an outline of CG reproduction in the present embodiment. FIG. 4A is a diagram illustrating a method for displaying the paper reproduction model 301 described in FIG. 3 on the CG. First, it is necessary to determine the coordinate system, and this uses the coordinate system of FIG. 1B as it is. That is, an xyz coordinate system having the origin of the center of the base model 403 generated with the same size as the base 103 is taken. There, the virtual camera 401 is installed on the CG space at the same position as the camera device 102. At this time, the angle of view of the virtual camera 401 is set to the same value as that of the camera device 102. Next, the base model 403 is installed on the CG with the same size as the base 103. Then, a reproduction model 301 having the same size, the same color, the same pattern, and the same content as the paper 104 is set on the CG at the same position as the paper 104. At this time, as shown in FIG. 4B, a spherical vertex is not drawn, and a quadrangle surrounded by four vertices is divided into two polygons (triangles) to perform display processing. Thereby, a display closer to the paper 104 can be performed. When the installation of the camera or model is completed, the video viewed from the virtual camera 401 is rendered. The rendering process is executed according to Equation 1 below.
Image = Screen (Proj * View * Pos (x, y, z)) Equation 1
In Equation 1, Pos (x, y, z) indicates the position of each model in the xyz coordinate system. View is a viewpoint conversion matrix, and indicates the position and imaging direction of the virtual camera 401. Here, the position and imaging direction of the virtual camera 401 are matched with those of the camera device 102. Proj is a projection matrix from 3D to 2D, and indicates the angle of view of the virtual camera 401 and the clip area. Here, the angle of view of the virtual camera 401 is matched with that of the camera device 102. The clip area represents a trapezoid area surrounded by the front clip surface 404 and the rear clip surface 405, and only this region is a rendering target. Screen () converts the coordinate system according to the image data storing the rendering result. Through the above operation, the video viewed from the virtual camera can be converted into image data. In addition to this, as a process for bringing a CG image closer to a real image, a lighting process for shading a model may be performed. At this time, a CG image closer to a real image can be generated by matching the light source positions in the real image environment and the CG space.

  FIG. 5 is a diagram illustrating an example of a block that performs processing for estimating the three-dimensional shape of an object in the present embodiment. The main processing apparatus 101 has a control unit 506. The control unit 506 includes a real image data acquisition unit 501, a reproduction model generation unit 502, a CG image data generation unit 503, a comparison unit 504, and a parameter adjustment unit 505. In this embodiment, a program corresponding to each unit shown in FIG. The CPU 201 and the GPU 204 execute these programs on the main memory 202 to function as the respective units illustrated in FIG.

  FIG. 6 is a flowchart showing a flow of acquiring a three-dimensional shape of paper using the main processing apparatus in the present embodiment. It is assumed that a program or the like necessary for this processing is preliminarily developed from the storage device 203 to the main memory 202 and is in an execution standby state. The size of the paper 104 is assumed to be given as data in advance. The processing of this embodiment will be described with reference to the block diagram of FIG. 5 and the flowchart of FIG.

  In step S601, the photographed image data acquisition unit 501 acquires photographed image data as the first image data. For example, the control unit 506 transmits an imaging command to the camera apparatus 102 to execute imaging. When the imaging is completed, the control unit 506 transmits an image acquisition command to the camera apparatus 102, receives the real image data, and develops it in the main memory 202. The photographed image data acquisition unit 501 obtains photographed image data expanded in the main memory. Note that the control unit 506 also performs lens correction processing for correcting lens distortion in the camera apparatus 102. The photographed image data acquisition unit 501 obtains photographed image data expanded in the main memory. An example of a real image at this time is shown in FIG. In the live-action image 701, the deformed paper 104 and the pedestal 103 are captured.

  In step S <b> 602, the reproduction model generation unit 502 configures the reproduction model 301 on the main memory 202 according to the size of the paper 104. At this time, the number of mesh vertices constituting the reproduction model 301 and the initial value of each vertex coordinate are also determined and stored in the main memory 202.

  In step S <b> 603, the control unit 506 sets a threshold value Th for the error between the real image data and the CG image data in the main memory 202. As the value of Th is smaller, the estimation accuracy of the three-dimensional shape increases, but the possibility that the solution does not converge increases. Therefore, the maximum number of trials N (second number) of the estimation process is determined. Further, a variable n for recording the number of trials is generated in the main memory 202.

  In step S604, the control unit 506 inputs the maximum change count S (first count) when changing the vertex coordinates. As will be described below, if the error is not reduced even if the vertex coordinates set at the initial stage are updated to some extent, processing for adding a vertex is performed in this embodiment. Therefore, the maximum change count S is set as a reference for updating to some extent. In addition, a variable s that records the number of changes in vertex coordinates is generated in the main memory 202. In step S605, the control unit 506 initializes the variable n to 0. In step S606, the control unit 506 initializes the variable s to 0.

  In step S607, the CG image data generation unit 503 generates CG image data that is second image data from the virtual camera viewpoint. The CG image data generation unit 503 acquires data necessary for rendering. Data necessary for rendering is as described with reference to FIG. 4. The data is once read from the storage device 203 into the main memory 202, transferred to the GPU 204, and acquired by the CG image data generation unit 503 functioning by the GPU 204. At this time, each vertex coordinate in the reproduction model 301 is also acquired. Thereafter, as described in FIG. 4, CG rendering is performed by the virtual camera 401, CG image data is generated, and is developed on the main memory 202. At this time, the number of pixels of the CG image data and the actual image data are made to coincide. An example of the generated CG image is shown in FIG. In the CG image 702, a reproduction model 301 and a pedestal model 403 that are deformed in the same manner as the real image 701 are rendered. As illustrated in FIG. 7, the CG image at this time does not sufficiently reproduce the three-dimensional shape. In the following, it is possible to estimate the three-dimensional shape by calculating the parameters of the reproduction model that reduces the difference between the photographed image and the CG image.

  In step S608, the comparison unit calculates an error E between the real image data representing the real image 701 generated in step S601 and the CG image data representing the CG image 702 generated in step S607. As a calculation method of the error E, a square error between the image data is calculated. As described above, since the same size, the same color, the same pattern, and the same content as the paper 104 are set in the reproduction model 301, the error can be evaluated by calculating the difference. At this time, the calculation time can be shortened by calculating the difference value of only the image area corresponding to the vertex portion of the reproduction model 301.

  In step S609, the comparison unit 504 determines whether the error E is smaller than a predetermined threshold Th. If yes, the process ends. If no, the process proceeds to step S610. The process is also terminated when the number of trials n is greater than the maximum number of trials N. When the process is completed under the condition that the number of trials is n> N, the vertex coordinate value in the case of the smallest error among the tried comparison results is used as a guess solution. When the processing is completed under the condition of E <Th, the vertex coordinate value at that time is used as a guess solution.

  In step S610, the comparison unit 504 determines whether the vertex coordinate change count s is smaller than the maximum change count S. If yes, then continue with step S611, otherwise, continue with step S613. In step S611, the control unit 506 increments the vertex coordinate change count s by one.

  In step S612, the parameter adjustment unit 505 sets a new coordinate value for the three-dimensional coordinates of each vertex of the reproduction model 301, changes the coordinate position, and returns to step S607 again. A new coordinate value is determined using a random number or the like. Here, since the paper size and area are given as data in advance, new values are set using these as constraint conditions. Thereafter, CG image data is regenerated based on the new coordinate value, and the above-described processing is repeated. FIG. 8 shows how the vertex coordinates are changed. As the vertex coordinate change count s increases, the three-dimensional coordinates of each vertex are updated at random. In the example of FIG. 8, when s = 10, the error E is less than the threshold value Th, so that the end condition is satisfied in the conditional expression in step S609. Therefore, the estimated value in this case is the coordinate value of each vertex when s = 10. The coordinate value of each vertex is a parameter indicating the three-dimensional shape of the paper.

  On the other hand, if the vertex coordinate change count s is equal to or greater than the maximum change count S, the control unit increments the trial count n by 1 in step S613.

  In step S614, the parameter adjustment unit 505 adds a vertex to the reproduction model 301, and returns to step S606 again. FIG. 9 shows how a vertex is added. In this way, for example, vertices are added by adding more vertices between the vertices. The coordinates of each added vertex are stored in the main memory 202 like other coordinates. Then, as described above, the coordinate value of each vertex including the added vertex is changed to obtain the coordinate value of the vertex with less error.

  Thus, the three-dimensional shape of the paper can be acquired by comparing the photographed image and the CG image.

  In this embodiment, the method of searching for the estimated solution while updating the coordinates of each vertex at random is described. However, it is expected that such a method takes time for convergence. Therefore, it is also possible to use a method for obtaining an estimated solution at a higher speed using a search algorithm such as a genetic algorithm. For example, in the genetic algorithm, estimation can be performed by representing the coordinates of each vertex as an individual and expressing the gene.

  In the first embodiment, the example in which the error is evaluated using the entire image data is described. In the second embodiment, a method for reducing the estimation time and reducing the data size of the three-dimensional shape data by performing error evaluation for each region will be described. Since the apparatus and the like are the same as those in the first embodiment, description thereof is omitted.

  FIG. 10 is a diagram illustrating an example of setting the evaluation area in the present embodiment. As shown in FIG. 10, four evaluation areas of area A, area B, area C, and area D are set for the reproduction model 301. In the present embodiment, when calculating the error between the photographed image data and the CG image data, the error is calculated for each evaluation region and evaluated. In the present embodiment, the number of set areas is described as four. However, the present invention is not limited to this. Other setting numbers or region division methods may be used.

  FIG. 11 is a flowchart showing a flow for acquiring the three-dimensional shape of the paper in this embodiment. It is assumed that a program or the like necessary for this processing is preliminarily developed from the storage device 203 to the main memory 202 and is in an execution standby state. The size of the paper 104 is assumed to be given as data in advance.

  Steps S1101 and S1102 are the same as steps S601 and S602 in FIG.

  In step S1103, the control unit 506 sets an evaluation area for the paper reproduction model as shown in FIG. In the step of calculating an error in step S1109 described later, the error is calculated for each evaluation region set here.

  Since the processing from step S1104 to step S1108 is the same as the processing from step S603 to S607, description thereof will be omitted.

  In step S1109, the comparison unit 504 calculates an error Ei for each evaluation region set in step S1103 with respect to the real image data generated in step S1101 and the CG image data generated in step S1108. Here, the subscript i indicates the index of the evaluation area. As a calculation method of the error Ei, a square error between image data of portions corresponding to the evaluation region is calculated. As described in the first embodiment, since the same size, the same color, the same pattern, and the same content as the paper 104 are set in the reproduction model 301, the error can be evaluated by calculating the difference. At this time, the calculation time can be shortened by calculating the difference value of only the image area corresponding to the vertex portion of the reproduction model 301.

  In step S1110, the comparison unit 504 determines whether the error Ei is smaller than the threshold value Th for all the evaluation regions. If this condition is satisfied in all evaluation regions, the process is terminated. If not, the process proceeds to step S1111. The process is also terminated when the number of trials n is greater than the maximum number of trials N. When the processing is completed under the condition that the number of trials is n> N, the vertex coordinate value having the smallest error among trials is used as a guess solution. When the process is completed under the condition of Ei <Th, the vertex coordinate value at that time is used as a guess solution.

  In step S <b> 1111, the comparison unit 504 determines whether the vertex coordinate change count s is smaller than the maximum change count S. If yes, then continue with step S1112, otherwise continue with step S1114. In step S1112, the control unit 506 increments the vertex coordinate change count s by one.

  In step S <b> 1113, the parameter adjustment unit 505 sets a new coordinate value for the three-dimensional coordinate of each vertex of the reproduction model 301 for the region where the error Ei of the evaluation region is larger than the threshold Th in step S <b> 1110. Then, the process returns to step S1108 again. In step S1110, the vertex coordinates are not updated for the region where the error Ei of the evaluation region is smaller than the threshold value Th. The new coordinate value is determined using, for example, a random number. Here, since the paper size and area are given as data in advance, new values are set using these as constraint conditions. Details of the vertex coordinate update processing have been described in the first embodiment, and are omitted here.

  In step S1114, the control unit 506 increments the number of trials n by 1. In step S1115, the parameter adjustment unit 505 adds vertices to the reproduction model 301 for the region where the error Ei of the evaluation region is larger than the threshold Th in step S1110, and returns to step S606 again. In step S1110, the vertex addition process is not performed on the area where the error Ei of the evaluation area is smaller than the threshold value Th. The coordinates of each added vertex are stored in the main memory 202 like other coordinates. FIG. 12 is a diagram illustrating a vertex addition result when the error in the region B is larger than the threshold value Th as an example.

  In this way, by setting an evaluation region and comparing the photographed image and the CG image for each region, the three-dimensional shape of the paper can be estimated at high speed, and the data size of the three-dimensional shape data can be reduced.

  As described in the first embodiment, a search algorithm such as a genetic algorithm may be used as the estimated solution search method.

  In this embodiment, a method for shortening the estimation time when estimating the three-dimensional shape of the paper for each frame when the paper moves continuously in time will be described. Since the apparatus and the like are the same as those in the first embodiment, description thereof is omitted.

  It is expected that the three-dimensional shape of the paper in each frame when the paper moves continuously in time has a very high correlation with the previous and subsequent frames. Therefore, in the present embodiment, the method described in the first or second embodiment is used for the process of estimating the three-dimensional shape of the first frame. In the subsequent frame estimation process, as shown in FIG. 13, the estimation result of the previous frame is set as the initial value of the reproduction model 301. This shortens the estimated time. It is assumed that the three-dimensional shape data estimated in each frame is stored in the main memory 202.

  FIG. 14 is a flowchart showing a flow for estimating the three-dimensional shape of paper in a plurality of temporally continuous frames in the present embodiment.

  In step S1401, the control unit 506 transmits an imaging command from the main processing apparatus 101 to the camera apparatus 102, and causes the t-th frame image to be captured. When the imaging is completed, the control unit 506 transmits an image acquisition command to the camera apparatus 102, receives the actual image data of the t-th frame, and develops it in the main memory 202. The control unit 506 also performs lens correction processing for correcting lens distortion in the camera apparatus 102. The captured image data acquisition unit 501 acquires the t-th frame image as captured image data.

  In step S1402, the control unit 506 acquires the estimation result of the three-dimensional shape of the (t−1) th frame from the main memory 202.

  In step S1403, the reproduction model generation unit 502 sets the estimation result of the three-dimensional shape of the (t−1) -th frame acquired in step S1402 as the initial value of each vertex of the reproduction model 301, and configures it on the main memory 202. .

  The subsequent steps are the same as those in the flowchart of FIG.

  In this way, by using the estimation result of the three-dimensional shape of the previous frame as the initial value of the reproduction model, the three-dimensional shape of the paper for each frame when the paper moves continuously in time. The estimation time can be shortened.

<Other examples>
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

  Further, the program code for realizing the function of the present embodiment may be executed by one computer (CPU, MPU), or may be executed by a plurality of computers cooperating. Also good. Further, the program code may be executed by a computer, or hardware such as a circuit for realizing the function of the program code may be provided. Alternatively, a part of the program code may be realized by hardware and the remaining part may be executed by a computer.

Claims (16)

First acquisition means for acquiring first image data obtained by imaging an object;
Generating means for generating a reproduction model including a parameter indicating the three-dimensional shape of the object;
Second acquisition means for acquiring second image data obtained by reproducing the reproduction model by computer graphics;
An apparatus for estimating a three-dimensional shape, comprising: adjusting means for adjusting parameters of the reproduction model by comparing the first image data and the second image data.   The three-dimensional shape estimation apparatus according to claim 1, wherein the generation unit regenerates a reproduction model based on the parameter adjusted by the adjustment unit.   The three-dimensional shape estimation apparatus according to claim 2, wherein the adjustment unit adjusts the parameters of the reproduction model by repeatedly comparing the second image data based on the adjusted parameters and the first image data. .   When the difference between the first image data and the second image data is smaller than a predetermined threshold as a result of the comparison, the adjustment means is a parameter of the reproduction model that is the source of the second image data. The three-dimensional shape estimation apparatus according to any one of claims 1 to 3, wherein the parameter is estimated as a parameter indicating the shape of the object. The parameter indicates vertex coordinates,
When the difference between the first image data and the second image data is not smaller than the threshold value and the number of comparisons is less than the first number, the adjusting means is configured to adjust the coordinates of the vertex coordinates. Change the position,
The three-dimensional shape estimation apparatus according to claim 4, wherein the generation unit regenerates a reproduction model at the changed coordinate position. When the difference between the first image data and the second image data is not smaller than the threshold and the number of comparisons is not less than the first number, the adjustment means Add the number of
The three-dimensional shape estimation apparatus according to claim 5, wherein the generation unit regenerates a reproduction model with vertex coordinates including the added vertex coordinates. The adjusting means changes the coordinate position of the added vertex coordinate,
The three-dimensional shape estimation apparatus according to claim 6, wherein the generation unit regenerates a reproduction model at a coordinate position of the added and changed vertex coordinates.   When the number of comparisons reaches the second number, the adjustment means determines that the difference between the first image data and the second image data in the comparison results obtained so far is The three-dimensional shape estimation apparatus according to any one of claims 1 to 7, wherein a parameter of a reproduction model that is a source of the second image data in a small case is estimated as a parameter indicating the shape of the object.   The three-dimensional shape estimation apparatus according to any one of claims 1 to 8, wherein the generation unit generates a reproduction model in which the same size, the same color, the same pattern, and the same content as the first image data are set. .   The three-dimensional shape estimation apparatus according to any one of claims 1 to 9, wherein the second image data acquisition unit acquires second image data having the same angle of view as the first image data. A setting means for setting an evaluation region for the reproduction model is further provided,
The three-dimensional shape estimation apparatus according to any one of claims 1 to 10, wherein the adjustment unit adjusts a parameter for each evaluation region by comparing image data for each evaluation region.   12. The three-dimensional image according to claim 11, wherein, as a result of the comparison, the adjustment unit adjusts a parameter of an evaluation region in which a difference between the first image data and the second image data is higher than a predetermined threshold. Shape estimation device. A three-dimensional shape estimation apparatus for estimating a three-dimensional shape of an object in a plurality of temporally continuous frames,
first acquisition means for acquiring first image data obtained by imaging an object in the t-th frame;
generating means for generating a reproduction model including a parameter indicating the three-dimensional shape of the (t-1) th frame;
Second acquisition means for acquiring second image data obtained by reproducing the reproduction model by computer graphics;
An apparatus for estimating a three-dimensional shape, comprising: adjusting means for adjusting parameters of the reproduction model by comparing the first image data and the second image data. A first acquisition step of acquiring first image data obtained by imaging an object;
Generating a reproduction model including a parameter indicating a three-dimensional shape of the object;
A second acquisition step of acquiring second image data obtained by reproducing the reproduction model by computer graphics;
A three-dimensional shape estimation method comprising: an adjustment step of adjusting a parameter of the reproduction model by comparing the first image data and the second image data. A three-dimensional shape estimation method for estimating a three-dimensional shape of an object in a plurality of temporally continuous frames,
a first step of acquiring first image data obtained by imaging an object in a t-th frame;
a generation step of generating a reproduction model including a parameter indicating the three-dimensional shape of the (t-1) th frame;
A second acquisition step of acquiring second image data obtained by reproducing the reproduction model by computer graphics;
A three-dimensional shape estimation method comprising: an adjustment step of adjusting a parameter of the reproduction model by comparing the first image data and the second image data.   The program for functioning a computer as a three-dimensional shape estimation apparatus as described in any one of Claims 1-13.

Images