JPWO2021048107A5

JPWO2021048107A5 -

Info

Publication number: JPWO2021048107A5
Application number: JP2022516051A
Authority: JP
Publication date: 2024-03-21
Anticipated expiration: 2040-09-08

Claims

a memory circuit that stores a scene model;
Generate virtual capture images for multiple camera poses of the camera configuration,
generating the virtual capture image by rendering an image for the camera pose based on the model;
a capture circuit,
a depth generation circuit that generates model depth data of the virtual captured image from the model;
A first synthesis circuit that executes a first process,
The first processing processes the virtual captured image based on the model depth data to generate a first view image for a plurality of test poses within a region of the scene.
a first synthesis circuit;
a depth estimation circuit that generates estimated depth data for the virtual capture image based on the virtual capture image;
a second synthesis circuit that executes a second process,
The second processing processes the virtual capture image based on the estimated depth data to generate a second view image for the plurality of test poses.
a second synthesis circuit;
a reference circuit that generates reference images for the plurality of test poses by rendering images for the plurality of test poses based on the model;
A quality circuit that generates a quality metric for at least one of the camera configuration, the first process , and the second process , the quality circuit comprising:
The first processing and the second processing use a comparison of the first view image, the second view image, and the reference image,
A device comprising a quality circuit .

At least one of the first processing and the second processing includes generating a depth map model for a first virtual capture image of the virtual capture image,
At least one of the first process and the second process view-shifts the first virtual captured image to a test pose of the plurality of test poses using the depth map model. 2. The apparatus of claim 1, comprising:

At least one of the first process and the second process includes determining a set of 3D points using at least one depth model;
the depth model is determined from the virtual captured image;
At least one of the first process and the second process includes determining a color of each 3D point using at least one of the virtual capture images;
At least one of the first processing and the second processing includes synthesizing a new image for a test pose of the plurality of test poses based on the projection of the 3D points. A device according to claim 1.

2. The apparatus of claim 1, wherein the quality metrics include a first quality metric for the first view image and a second quality metric for the second view image.

the quality circuit determines quality metrics for a plurality of camera configurations;
5. The apparatus of claim 4, wherein the quality circuit selects among the plurality of camera configurations depending on both the first quality metric and the second quality metric.

The quality circuit determines that the first quality metric satisfies a first criterion , that the second quality metric satisfies a second criterion, and that the first quality metric and the first quality metric meet a second criterion. 6. The apparatus of claim 5 , selecting a camera configuration from the plurality of camera configurations in response to at least one of a difference measurement value with a quality metric of 2 satisfying a third criterion. .

the quality circuit generates a signal-to-noise measurement for each second view image;
2. The apparatus of claim 1 , wherein the quality circuit generates the quality metric in response to the signal-to-noise measurements for the second view image.

At least one of the first synthesis circuit and the second synthesis circuit performs encoding and decoding of the virtual capture image before image synthesis based on the encoded and decoded virtual capture image. 2. The apparatus of claim 1 , comprising:

At least one of the first process and the second process is associated with the virtual captured image prior to image synthesis based on at least one of the model depth data and the estimated depth data. The apparatus of claim 1 , comprising encoding and decoding at least one of the model depth data and the estimated depth data.

The apparatus of claim 8 , wherein the encoding comprises performing lossy encoding.

2. The apparatus of claim 1 , wherein at least some camera poses are the same as at least one of the plurality of test poses .

2. The apparatus of claim 1 , wherein there are ten times more test poses than the camera poses.

The camera positions form a one-dimensional arrangement ,
2. The apparatus of claim 1, wherein the test locations form a two-dimensional or three-dimensional arrangement.

A method of evaluating the quality of image capture, the method comprising:
storing a model of the scene;
generating virtual captured images for a plurality of camera poses of a camera configuration by rendering images for the camera poses based on the model;
generating model depth data for the virtual captured image from the model;
processing the virtual captured image based on the model depth data to generate a first view image for a plurality of test poses within the region of the scene;
generating estimated depth data for the virtual capture image based on the virtual capture image;
processing the virtual capture image based on the estimated depth data to generate a second view image for the plurality of test poses;
generating reference images for the plurality of test poses by rendering images for the plurality of test poses based on the model;
generating a quality metric for at least one of the camera configuration, the process of generating the first view image, and the process of generating the second view image , the step of: The process of generating and the process of generating the second view image includes the step of generating a quality metric using a comparison of the first view image, the second view image, and the reference image.
A method having .

15. A computer program stored on a non-transitory medium which, when executed on a processor, performs the method of claim 14 .

At least one of the processing by the first synthesis circuit and the processing by the second synthesis circuit includes generating a depth map model for the first virtual capture image of the virtual capture image,
At least one of the processing by the first synthesis circuit and the processing by the second synthesis circuit uses the depth map model to apply the first synthesis circuit to the test pose of the plurality of test poses. 15. The method of claim 14, comprising view shifting the virtual captured image.

At least one of the processing by the first synthesis circuit and the processing by the second synthesis circuit includes determining a set of 3D points using at least one depth model;
the depth model is determined from the virtual captured image;
At least one of the processing by the first synthesis circuit and the processing by the second synthesis circuit determines the color of each 3D point using at least one of the virtual capture images. including doing;
At least one of the processing by the first synthesis circuit and the processing by the second synthesis circuit generates a new image for a test pose of the plurality of test poses based on the projection of the 3D points. 15. The method of claim 14, comprising synthesizing.

15. The method of claim 14, wherein the quality metrics include a first quality metric for the first view image and a second quality metric for the second view image.

determining quality metrics for multiple camera configurations;
19. The method of claim 18, further comprising selecting among the plurality of camera configurations depending on both the first quality metric and the second quality metric.

the first quality metric satisfies a first criterion; the second quality metric satisfies a second criterion; and the first quality metric and the second quality metric 20. The method of claim 19, further comprising selecting a camera configuration from the plurality of camera configurations in response to at least one of the difference measurements of satisfying a third criterion.