JP2021196915A - Stereoscopic image depth control device and program thereof - Google Patents

Abstract

To provide a stereoscopic image depth control device capable of adjusting the ratio between the distortion of space and the distortion of shape.SOLUTION: A stereoscopic image depth control device 4 includes: a static vertex coordinate calculation unit 41 that calculates static vertex coordinates, which are coordinates of each vertex included in a three-dimensional model, by static depth compression; a dynamic vertex coordinate calculation unit 42 that calculates dynamic vertex coordinates, which are coordinates of each vertex included in a three-dimensional model according to a viewing position, by dynamic depth compression; a vertex coordinate adjustment unit 43 that adjusts coordinates of each vertex between the static vertex coordinates and the dynamic vertex coordinates based on an adjustment ratio; and an element image generation unit 44 that generates an element image from the three-dimensional model after the coordinates of each vertex are adjusted.SELECTED DRAWING: Figure 3

Description

The present invention relates to a stereoscopic image depth control device for depth compression of a three-dimensional model of a subject and a program thereof.

The integral 3D display is a flat display on which an element image is displayed and a lens array is superposed on the display, and it is possible to display as if there is an object there without special glasses. The integral method can naturally present horizontal and vertical binocular parallax and motion parallax without spectacles by reproducing light rays in three-dimensional space. Further, since the integral method reproduces the light beam emitted from the object, it is possible to avoid the problem that the congestion and the adjustment are inconsistent, which is the main problem of the 3D display.

On the other hand, a spatial image reproduction type 3D display such as an integral system has a problem that it is difficult to present a scene with a wide depth. Specifically, since the spatial frequency decreases as the distance from the lens array surface increases in the depth direction, when an image is displayed at a distant position, the image is greatly blurred. As shown in FIG. 8, the resolution (spatial frequency γ) of the stereoscopic image reproduced by the integral method maintains a predetermined resolution (Nyquist frequency) within a constant depth reproduction range in the vicinity of the lens array position (z = 0). However, it has the property of dropping sharply when it exceeds that range. This predetermined resolution is specified by the spacing (pitch) of the element lenses constituting the lens array, and the stereoscopic image exceeding this depth reproduction range becomes blurred as the distance from the position of the lens array increases. This means that in the integral method, in principle, there is a depth reproduction range in which a stereoscopic image can be displayed without blurring.

In order to widen the area (depth reproduction range) where blurring does not occur, such as an image of the same subject, it is necessary to reduce the pixel pitch of the element image, which is a burden on system development (Non-Patent Document 1). Therefore, as shown in FIG. 9, a conventional method of compressing the depth of a 3D scene and avoiding blurring has been proposed (Non-Patent Document 2). In this conventional method, as shown in the following equation (1), the vertex coordinates p (x, y, z) constituting the object (three-dimensional model M) in the scene are set to the vertex coordinates p ₀ '(x ₀ ', Convert to y ₀ ', z ₀ '). At that time, the scene is moved from the back of the screen to the front when viewed from the viewer 9, and the scene with a wide depth is compressed into a narrow range that can be displayed without blurring.

In addition, in the formula (1), as shown in FIG. 10, f (z) moves a certain apex coordinate from the depth position z on the back side to the depth position z'on the front side when viewed from the viewer 9. It is a function. Further, in the equation (1), _{the conversion of x 0'and} y _0'is a conversion in which the apparent size of the object does not change before and after the compression when viewed from the origin of the depth compression.

In addition, p _v0 (0,0,0) is taken as an initial viewing position (standard viewing position). Further, let (i, j) be the horizontal position and the vertical position of the point whose vertex coordinates are projected on the display, respectively. The subscript character pre represents the projection of the vertices before compression, and the subscript character post represents the projection of the vertices after compression. The subscript number 0 represents the projection of the initial viewing position. The subscript number 1 represents the projection of the viewing position after movement.

A method for determining a non-linear depth compression function has been proposed by obtaining the pixel spacing and element lens spacing of an integral 3D display and the depth range (depth reproduction range) that can be displayed without blurring (Patent Document 1). .. Hereinafter, statically compressing the depth of the scene without considering the movement of the viewing position as in this conventional technique is referred to as "static depth compression".

In this static depth compression, the shape distortion of the object becomes remarkable when the viewing position is moved. As shown in FIG. 11, at the initial viewing position p _v0 (0, 0, 0), the points p ₀ ′ (x ₀ ′, y ₀ ′, z ₀ ′), p ₁ ′ before and after the depth compression. (X ₁ ', y ₁ ', z ₁ ') remain in the same position. On the other hand, at the viewing position p _v1 (x _v1 , y _v1 , z _{v1 ) after movement} , the projection points (i pre1, j _pre1 ) and (i _post1 , j _post1 ) on the display shift before and after the depth compression. The shape of the object looks distorted. In this way, with static depth compression, it is difficult to notice that an object whose depth has been crushed is crushed even when viewed from the front, but when it is turned sideways, the distortion of the shape due to crushing becomes conspicuous.

Here, the distortion of the shape, as shown in the following equation (2), in each of the viewing position after the initial viewing position and movement, the deviation of the point obtained through projection of the integral 3D display _{D shape0,} D _shape1 Is defined as.

As described above, in the static depth compression, the shape is not distorted at the initial viewing position, and D _shape 0 = 0. On the other hand, in static depth compression, when the viewing position moves, D _shape1 > 0, and the distortion of the shape increases.

Therefore, as shown in FIG. 12, a method of dynamically compressing the depth of the scene in consideration of the movement of the viewing position has been proposed, and hereinafter referred to as "dynamic depth compression" (Non-Patent Document 3). ). In this dynamic depth compression, in order to measure the viewing position in real time, the viewer 9 is photographed with an infrared camera or an RGB camera and image processing is performed. In the dynamic depth compression, the vertex coordinates p (x, y, z) of the object are converted _{to p 1} ′ (x ₁ ′, y ₁ ′, z _{1 ′) based on the following equation (3).} do.

In this dynamic depth compression, as shown in FIG. 12, since the projection points (i _pre1 , j _pre1 ) and (i _post1 , j _post1 ) always match before and after the depth compression, shape distortion does not occur. Similarly, the projection points (i _pre0 , j _pre0 ) and (i _post0 , j _post0 ) also match before and after the depth compression. Therefore, D _hape0 = 0 and D _happe1 = 0.

On the other hand, with this dynamic depth compression, when the viewing position is moved, an object that should originally appear to be stationary in the air may slide in the same direction as the viewing position. .. It is known that spatial distortion according to the movement of the viewing position occurs when the distance between the shooting cameras on the two-lens stereoscopic display does not match the distance between the eyes of the viewer 9 (the distance between the eyes of the viewer 9 does not match). Non-Patent Document 4). Here, the distortion of space is defined as shown in the following equation (4).

There is a trade-off relationship between the spatial distortion and the shape distortion described above. For example, in static depth compression, D _shear1 = 0 and D _shape1 > 0, and in dynamic depth compression, D _shear1 > 0 and D _shape1 = 0.

Japanese Unexamined Patent Publication No. 2017-11520

Hoshino H, Okano F, Isono H, Yuyama, Analysis of resolution limitation of integral photography, Opt Soc Am, 1998,15 (8), 2059-2065 Sawahata Y, Morita, Estimating Depth Range Required for 3-D Displays to Show Depth-Compressed Scenes Without Inducing Sense of Unnaturalness, IEEE Trans Broadcast, 2018, 64 (2), 488-497 Miyashita Y, Sawahata Y, Katayama M, Komine, Depth boost: Extended depth reconstruction capability on volumetric display, In: ACM SIGGRAPH 2019 Talks, SIGGRAPH 2019, 2019 Wartell Z, Hodges LF, Ribarsky, Balancing fusion, image depth and distortion in stereoscopic head-tracked displays, Proc 26th Annu Conf Comput Graph Interact Tech SIGGRAPH 1999, 1999, 351-358

However, in the above-mentioned conventional technique, there is no mechanism for adjusting the distortion of space and the distortion of shape in the depth compression expression of a three-dimensional image, and only one of the states in which the distortion is maximum can be selected.

Therefore, it is an object of the present invention to provide a stereoscopic image depth control device and a program thereof that can adjust the ratio between the distortion of space and the distortion of shape.

In order to solve the above-mentioned problems, the stereoscopic image depth control device according to the present invention performs depth compression of a three-dimensional model of a subject so as to fall within a depth reproduction range satisfying a preset resolution, and then depth-compresses the three-dimensional model. It is a stereoscopic image depth control device that generates an element image from, and has a configuration including a static vertex coordinate calculation unit, a dynamic vertex coordinate calculation unit, a vertex coordinate adjustment unit, and an element image generation unit.

According to this configuration, the static vertex coordinate calculation unit calculates the static vertex coordinates, which are the coordinates of each vertex constituting the three-dimensional model, by static depth compression that does not consider the movement of the viewing position of the stereoscopic image. do.
The dynamic vertex coordinate calculation unit calculates the dynamic vertex coordinates, which are the coordinates of each vertex constituting the three-dimensional model according to the viewing position, by dynamic depth compression in consideration of the movement of the viewing position.
The vertex coordinate adjustment unit adjusts the vertex coordinates of the three-dimensional model between the static vertex coordinates and the dynamic vertex coordinates based on the adjustment ratio between the static depth compression and the dynamic depth compression.
The element image generation unit generates an element image from the three-dimensional model after adjusting the vertex coordinates.

That is, the above-mentioned adjustment ratio represents the ratio between the distortion of the shape caused by static depth compression and the distortion of space caused by dynamic depth compression. Therefore, the stereoscopic image depth control device can adjust the ratio of both the spatial distortion and the shape distortion in the vertex coordinate adjusting unit.

The present invention can also be realized by a program for making a computer function as the above-mentioned stereoscopic image depth control device.

The present invention can adjust the ratio of spatial distortion to shape distortion.

It is a schematic block diagram of the stereoscopic image display system which concerns on 1st Embodiment. It is explanatory drawing explaining an example of the adjustment ratio input device in 1st Embodiment. It is a block diagram which shows the structure of the stereoscopic image depth control apparatus which concerns on 1st Embodiment. It is explanatory drawing explaining the adjustment of the vertex coordinates of the 3D model in 1st Embodiment. In the first embodiment, it is explanatory drawing explaining the relationship between the setting of the adjustment ratio, the distortion of a shape, and the distortion of a space. It is a flowchart which shows the operation of the stereoscopic image depth control apparatus which concerns on 1st Embodiment. It is a block diagram which shows the structure of the stereoscopic image depth control apparatus which concerns on 2nd Embodiment. It is explanatory drawing explaining the depth reproduction range in the conventional integral 3D display. It is explanatory drawing explaining the conventional static depth compression. It is a graph which shows an example of the function which compresses the depth in the conventional static depth compression. It is explanatory drawing explaining the distortion of the shape in the conventional static depth compression. It is explanatory drawing explaining the conventional dynamic depth compression.

Hereinafter, each embodiment of the present invention will be described with reference to the drawings. However, each embodiment described below is for embodying the technical idea of the present invention, and the present invention is not limited to the following unless otherwise specified. Further, in each embodiment, the same means may be designated by the same reference numerals and description thereof may be omitted.

(First Embodiment)
[Structure of stereoscopic image display system]
With reference to FIG. 1, the configuration of the stereoscopic image display system 100 according to the first embodiment will be described.
The stereoscopic image display system 100 generates an element image from a three-dimensional model of the subject and displays the generated element image. At this time, the stereoscopic image display system 100 depth-compresses the three-dimensional model so as to be within the depth reproduction range of the stereoscopic image display device 1 in the stereoscopic image depth control device 4 described later. As shown in FIG. 1, the stereoscopic image display system 100 includes a stereoscopic image display device 1, a viewing position measuring device 2, an adjustment ratio input device 3, and a stereoscopic image depth control device 4.

The stereoscopic image display device 1 is a general integral 3D display that displays an integral stereoscopic image. The stereoscopic image display device 1 inputs an element image from the stereoscopic image depth control device 4, and displays the element image via a lens array (not shown) so that the stereoscopic image T is visually recognized by the viewer 9. Let me.

The viewing position measuring device 2 measures a position (viewing position) at which the viewer 9 views the stereoscopic image T. The viewing position measuring device 2 can be configured by an infrared camera or an RGB camera. Specifically, the viewing position measuring device 2 uses the scleral reflex method or the corneal / pupillary reflex method from the camera images of the viewer 9 taken by two RGB cameras to the left and right corneas of the viewer 9. , The three-dimensional viewing position may be obtained by the principle of triangular measurement, and the viewing distance from the stereoscopic image display device 1 to the viewer 9 may be measured. Then, the viewing position measuring device 2 inputs the measured viewing position and viewing distance to the stereoscopic image depth control device 4.

In the adjustment ratio input device 3, the viewer 9 sets the adjustment ratio s, and the adjustment ratio s is input to the stereoscopic image depth control device 4 (vertex coordinate adjustment unit 43). For example, as shown in FIG. 2, the adjustment ratio input device 3 is a slider that can arbitrarily set the adjustment ratio s between 0 and 1 (0 ≦ s ≦ 1). This adjustment ratio s represents the ratio between static depth compression and dynamic depth compression. That is, the adjustment ratio s represents the ratio between the distortion of the shape caused by static depth compression and the distortion of space caused by dynamic depth compression.

The stereoscopic image depth control device 4 depth-compresses the three-dimensional model of the subject so as to fall within the depth reproduction range that satisfies the resolution of the stereoscopic image display device 1 set in advance, and extracts the element image from the three-dimensional model after the depth compression. It is what is generated. At this time, the stereoscopic image depth control device 4 adjusts the ratios of both the spatial distortion and the shape distortion based on the adjustment ratio input from the adjustment ratio input device 3.
In the stereoscopic image depth control device 4, the three-dimensional model is input from the three-dimensional model generation device (not shown), and the type and generation method of the three-dimensional model are not particularly limited.

[Structure of stereoscopic image depth control device]
The configuration of the stereoscopic image depth control device 4 will be described with reference to FIG.
As shown in FIG. 3, the stereoscopic image depth control device 4 includes a depth compression unit 40 and an element image generation unit 44.

The depth compression unit 40 inputs a three-dimensional model of the subject and compresses the input three-dimensional model in depth. As shown in FIG. 3, the depth compression unit 40 includes a static vertex coordinate calculation unit 41, a dynamic vertex coordinate calculation unit 42, and a vertex coordinate adjustment unit 43.

The static vertex coordinate calculation unit 41 calculates the static vertex coordinates, which are the coordinates of each vertex constituting the three-dimensional model, by static depth compression that does not consider the movement of the viewing position. Here, the static vertex coordinate calculation unit 41 performs static depth compression as in the conventional case. For example, the static vertex coordinate calculation unit 41 converts the vertex coordinate p of the three-dimensional model into the static vertex coordinate p _{0 ′ by using the above equation (1).} Then, the static vertex coordinate calculation unit 41 outputs the calculated static vertex coordinates p ₀ ′ of the three-dimensional model to the vertex coordinate adjustment unit 43.

The dynamic vertex coordinate calculation unit 42 calculates the dynamic vertex coordinates, which are the coordinates of each vertex constituting the three-dimensional model, according to the viewing position by dynamic depth compression considering the movement of the viewing position. It is a thing. Here, the dynamic vertex coordinate calculation unit 42 performs dynamic depth compression similar to the conventional one according to the viewing position input from the adjustment ratio input device 3. For example, the dynamic vertex coordinate calculation unit 42 converts the vertex coordinates p of the three-dimensional model into the dynamic vertex coordinates p _{1 ′ by using the above equation (3).} Then, the dynamic vertex coordinate calculation unit 42 outputs the calculated dynamic vertex coordinate p ₁ ′ of the three-dimensional model to the vertex coordinate adjustment unit 43.

_{The vertex coordinate adjustment unit 43 inputs the static vertex coordinates p 0} ′ input from the static vertex coordinate calculation unit 41 and the dynamic vertex coordinate calculation unit 42 based on the adjustment ratio s input from the adjustment ratio input device 3. The coordinates of the vertices of the three-dimensional model are adjusted with the dynamic coordinates p _1'. Then, the vertex coordinate adjustment unit 43 outputs the three-dimensional model after adjusting the vertex coordinates to the element image generation unit 44.

<Adjustment of vertex coordinates>
With reference to FIGS. 4 and 5, the adjustment of the vertex coordinates by the vertex coordinate adjustment unit 43 will be described in detail.
Specifically, the vertex coordinate adjustment unit 43 uses the following equation (5) to set the vertex coordinate p _{mod 1} ′ of the three-dimensional model between the static vertex coordinate p ₀ ′ and the dynamic vertex coordinate p ₁ ′. _{This vertex coordinate p mod1'is} calculated so as to be located at. At this time, the vertex coordinate adjusting unit 43 may shift the _{vertex coordinate p mod1 ′ in the direction opposite to the moving direction of the viewing position.}

As shown in FIG. 4, the apex coordinate p _{mod 1} ′ is shifted between _{the static apex coordinate p 0} ′ and the dynamic apex coordinate p ₁ ′ in the x-axis direction and the y-axis direction according to the adjustment ratio s. do. Specifically, the closer the adjustment ratio s is to 0, the closer the vertex coordinate p _mod1 ′ is to the static vertex coordinate p ₀ ′, while the closer the adjustment ratio s is to 1, the closer the vertex coordinate p _mod1 ′ is to the dynamic vertex. It becomes close to the coordinates p _1'. In FIG. 4, the projection points (i _mod1 , j _mod1 ) are points where the vertex coordinates p _mod1 ′ are projected onto the display.

As shown in FIG. 5, as the adjustment ratio s approaches 0, the distortion of the shape becomes larger, while the distortion of the space becomes smaller. In the example of FIG. 5, as the adjustment ratio s approaches 0, the straight line on the right side of the subject A is distorted in a curved shape. When the adjustment ratio s = 0, it is synonymous with static depth compression, and only shape distortion occurs, and spatial distortion does not occur.
On the other hand, as the adjustment ratio s approaches 1, the distortion of the space becomes larger and the distortion of the shape becomes smaller. In the example of FIG. 5, the closer the adjustment ratio s is to 1, the more the subject B slides to the right. When the adjustment ratio s = 1, it is synonymous with dynamic depth compression, and only spatial distortion occurs, and shape distortion does not occur.

Here, it is considered that the value of the optimum adjustment ratio s differs depending on the viewing environment of the stereoscopic image display device 1. For example, in a large-screen viewing environment such as a movie theater, the viewer 9 rarely observes while moving his head. Therefore, since it is better to relax the distortion of the shape rather than the distortion of the space, the value of the adjustment ratio s is increased. On the other hand, in a small screen viewing environment such as a tablet, since the screen can be easily moved with both hands, the relative positional relationship between the head and the display of the viewer 9 is likely to change significantly. Therefore, since it is better to alleviate the distortion of the space rather than the distortion of the shape, the value of the adjustment ratio s is reduced.

Further, it is considered that the value of the adjustment ratio s differs not only depending on the viewing environment but also depending on the characteristics of the content. For example, in a content having a large camera motion, it is considered that the viewer 9 does not actively move the head because the motion parallax is sufficiently included. In this case, since it is better to preferentially alleviate the distortion of the shape, the value of the adjustment ratio s is increased. On the other hand, in the content to be viewed from various directions, it is considered that the viewer 9 actively moves his / her head. In this case, since it is better to preferentially relax the distortion of the space, the value of the adjustment ratio s is reduced. For example, by carrying out a visual psychological experiment using an adjustment method or the like, it is possible to obtain the value of the optimum adjustment ratio s according to the viewing environment and the type of content.

Returning to FIG. 3, the description of the stereoscopic image depth control device 4 will be continued.
The element image generation unit 44 generates an element image from the three-dimensional model input from the vertex coordinate adjustment unit 43. Here, the element image generation unit 44 can generate an element image from the three-dimensional model by using a conventional method such as a ray tracing method.

The ray tracing method extends a straight line connecting each element lens constituting the lens array of the stereoscopic image display device 1 and each pixel of the element image, and obtains color information at the intersection of the extended straight line and the surface of the three-dimensional model. It is acquired as the pixel value of the pixel. For example, the ray tracing method is described in detail in References 1 and 2 below, and further description thereof will be omitted.

Reference 1: Katayama, "Conversion method from 3D model to integral stereoscopic image", Japan Broadcasting Corporation, NHK Science & Technical Research Laboratories R & D, No. 128, July 2011 Reference 2: "Technology for generating integral stereoscopic images from 3D models", Japan Broadcasting Corporation, NHK Science & Technical Research Laboratories R & D, No. 123, September 2010

Further, the element image generation unit 44 may generate an element image by a method other than the ray tracing method. For example, the element image generation unit 44 may generate a multi-viewpoint image obtained by capturing a three-dimensional model with a virtual camera arranged in an array, and convert the multi-viewpoint image into an element image (for example, Reference 3). ..
Reference 3: Iketani, "Method for generating integral stereoscopic images using a multi-view camera", Japan Broadcasting Corporation, NHK Science & Technical Research Laboratories R & D, No. 146, August 2014

After that, the element image generation unit 44 outputs the generated element image to the stereoscopic image display device 1. Then, the stereoscopic image display device 1 displays the element image via the lens array, and makes the viewer 9 visually recognize the stereoscopic image T.

[Operation of stereoscopic image depth control device]
The operation of the stereoscopic image depth control device 4 will be described with reference to FIG.
As shown in FIG. 6, in step S1, the static vertex coordinate calculation unit 41 calculates each static vertex coordinate of the three-dimensional model by static depth compression. For example, the static vertex coordinate calculation unit 41 converts the vertex coordinate p of the three-dimensional model into the static vertex coordinate p _{0 ′ by using the above equation (1).}

In step S2, the dynamic vertex coordinate calculation unit 42 calculates each dynamic vertex coordinate of the three-dimensional model according to the viewing position by dynamic depth compression. For example, the dynamic vertex coordinate calculation unit 42 converts the vertex coordinates p of the three-dimensional model into the dynamic vertex coordinates p _{1 ′ by using the above equation (3).}

In step S3, the vertex coordinate adjustment unit 43 has the static vertex coordinates p ₀ ′ calculated in step S1 and the dynamic vertex coordinates p _{1 calculated in step S2 based on the adjustment ratio s input from the adjustment ratio input device 3.} Adjust the coordinates of each vertex of the 3D model to and from ´. _{Specifically, the vertex coordinate adjustment unit 43 calculates the vertex coordinates p mod1} ′ of the three-dimensional model using the above equation (5).

In step S4, the element image generation unit 44 generates an element image from the three-dimensional model whose vertex coordinates are adjusted in step S3. For example, the element image generation unit 44 generates an element image from a three-dimensional model by using a ray tracing method.

[Action / Effect]
The stereoscopic image depth control device 4 according to the first embodiment adjusts both ratios so that both spatial distortion and shape distortion can be alleviated based on the adjustment ratio s set by the viewer 9. can do. That is, the stereoscopic image depth control device 4 compresses the depth of the scene in order to avoid blurring that occurs when a scene with a wide depth is displayed on the stereoscopic image display device 1, and the distortion and space of the shape generated at that time. The distortion of the viewer 9 can be arbitrarily adjusted.

(Second Embodiment)
With reference to FIG. 7, the stereoscopic image display system 100B according to the second embodiment will be described as different from the first embodiment.
As shown in FIG. 7, the stereoscopic image display system 100B includes a stereoscopic image display device 1, a viewing position measuring device 2, an adjustment ratio input device 3, and a stereoscopic image depth control device 4B. Since each means of the stereoscopic image depth control device 4B is the same as that of the first embodiment, the description thereof will be omitted.

[Structure of stereoscopic image depth control device]
In the second embodiment, the stereoscopic image depth control device 4B is different from the first embodiment in that it learns the optimum adjustment ratio s. Therefore, the stereoscopic image depth control device 4B includes a depth compression unit 40B, an element image generation unit 44, a history DB storage unit 45, a learning model generation unit 46, an adjustment ratio acquisition unit 47, and a switch 48. Further, the depth compression unit 40B includes a static vertex coordinate calculation unit 41, a dynamic vertex coordinate calculation unit 42, and a vertex coordinate adjustment unit 43B.
Since the static vertex coordinate calculation unit 41, the dynamic vertex coordinate calculation unit 42, and the element image generation unit 44 are the same as those in the first embodiment, the description thereof will be omitted.

The history DB storage unit 45 stores the history database. This history database is a database showing the relationship between the adjustment ratio s, the depth position of the stereoscopic image T, and the viewing distance. For example, a relational database can be mentioned as a history database.

The learning model generation unit 46 refers to the history database and generates a learning model in which the adjustment ratio s is represented by the relationship between the depth position of the stereoscopic image T and the viewing distance. Here, the learning model generation unit 46 stores the learning model in a memory (not shown).

<Generation of learning model>
Hereinafter, the generation of the learning model by the learning model generation unit 46 will be described in detail.
For example, the learning model is a linear model in which _{the adjustment ratio s is expressed by the relationship between the depth position t 1} and the viewing distance t ₂ of the stereoscopic image T, as shown in the following equation (6). In equation (6), α, β ₁ , and β ₂ represent predetermined coefficients.

In this case, the learning model generation unit 46 may obtain _{the coefficients α, β 1} and β _{2 of the equation (6) as an optimization problem.} Specifically, the learning model generation unit 46 solves the following equation (7) by the least squares method to obtain the coefficients α, β ₁ , and β ₂ . In equation (7), _{the subscript n of s (n)} , t _{1 (n)} , and t _{2 (n)} represents the nth record stored in the history database (however, n = 0). , 1, 2, ...).

Returning to FIG. 7, the description of the stereoscopic image depth control device 4B will be continued.
The adjustment ratio acquisition unit 47 inputs the depth position and the viewing distance, and acquires the adjustment ratio s corresponding to the input depth position and the viewing distance from the learning model. Here, the adjustment ratio acquisition unit 47 inputs the viewing distance from the viewing position measuring device 2, and the viewer 9 inputs the depth position designated in advance by the video creator. In this way, the adjustment ratio acquisition unit 47 inputs the depth position and the viewing distance corresponding to the viewing environment of the viewer 9 into the learning model, so that the adjustment ratio s suitable for the viewing environment of the viewer 9 Can be obtained from the learning model.

The switch 48 selects either the adjustment ratio s input from the adjustment ratio input device 3 or the adjustment ratio s acquired by the adjustment ratio acquisition unit 47, and sets the selected adjustment ratio s as the vertex coordinate adjustment unit 43B. It is output to. Here, the viewer 9 can manually set which adjustment ratio s the switch 48 selects. In the example of FIG. 7, the switch 48 selects the adjustment ratio s acquired by the adjustment ratio acquisition unit 47.

The vertex coordinate adjustment unit 43B adjusts the vertex coordinates of the three-dimensional model based on the adjustment ratio s input from the switch 48. That is, the apex coordinate adjustment unit 43B adjusts the apex coordinates of the three-dimensional model based on the adjustment ratio s input from the adjustment ratio input device 3 or the adjustment ratio s acquired by the adjustment ratio acquisition unit 47. Since the adjustment of the vertex coordinates itself is the same as that of the first embodiment, further description thereof will be omitted.

[Action / Effect]
Since the stereoscopic image depth control device 4B according to the second embodiment can acquire the adjustment ratio s suitable for the viewing environment of the viewer 9, the distortion and shape of the space are matched to the viewing environment of the viewer 9. The ratio with the distortion of is automatically adjusted. Thereby, the stereoscopic image depth control device 4B can improve the convenience.

(Modification example)
Although each embodiment of the present invention has been described in detail above, the present invention is not limited to this, and includes design changes and the like within a range that does not deviate from the gist of the present invention.
In the above-described embodiment, the learning model has been described as a linear model, but the learning model is not limited to this. For example, a learning model may be learned by machine learning or deep learning.

In the above-described embodiment, the stereoscopic image depth control device has been described as independent hardware, but the present invention is not limited thereto. Hardware resources such as a CPU, a memory, and a hard disk included in a computer can also be realized by a program that operates as the stereoscopic image depth control device described above. These programs may be distributed via a communication line, or may be written and distributed on a recording medium such as a CD-ROM or a flash memory.

1 Solid image display device 2 Viewing position measuring device 3 Adjustment ratio input device 4, 4B Solid image depth control device 40 Depth compression unit 41 Static vertex coordinate calculation unit 42 Dynamic vertex coordinate calculation unit 43, 43B Vertex coordinate adjustment unit 44 Element image generator 100 Stereoscopic image display system

Claims (5)

It is a stereoscopic image depth control device that compresses the depth of a 3D model of a subject so as to be within the depth reproduction range that satisfies a preset resolution, and generates an element image from the 3D model after the depth compression.
A static vertex coordinate calculation unit that calculates static vertex coordinates, which are the coordinates of each vertex constituting the 3D model, by static depth compression that does not consider the movement of the viewing position of the stereoscopic image.
With a dynamic vertex coordinate calculation unit that calculates dynamic vertex coordinates that are the coordinates of each vertex constituting the three-dimensional model according to the viewing position by dynamic depth compression considering the movement of the viewing position. ,
A vertex coordinate adjustment unit that adjusts the vertex coordinates of the three-dimensional model between the static vertex coordinates and the dynamic vertex coordinates based on the adjustment ratio between the static depth compression and the dynamic depth compression. When,
An element image generation unit that generates the element image from the three-dimensional model after adjusting the vertex coordinates,
A stereoscopic image depth control device characterized by being equipped with. The vertex coordinate adjustment unit is as shown in the following equation (5).

_{Claim 1} is characterized in that _{the vertex coordinates p mod 1} ′ of the three-dimensional model _{are adjusted between the static vertex coordinates p 0} ′ and the dynamic vertex coordinates p 1 ′ based on the adjustment ratio s. The stereoscopic image depth control device described in 1. A learning model generation unit that generates a learning model in which the adjustment ratio is represented by the relationship between the depth position of the stereoscopic image and the viewing distance.
It is provided with an adjustment ratio acquisition unit for inputting the depth position and the viewing distance and acquiring the adjustment ratio corresponding to the input depth position and the viewing distance from the learning model.
The stereoscopic image depth control according to claim 1 or 2, wherein the vertex coordinate adjustment unit adjusts the vertex coordinates of the three-dimensional model based on the adjustment ratio acquired by the adjustment ratio acquisition unit. Device. The three-dimensional image according to claim 1 or 2, wherein the vertex coordinate adjusting unit inputs the adjustment ratio and adjusts the vertex coordinates of the three-dimensional model based on the input adjustment ratio. Depth control device. A program for causing a computer to function as the stereoscopic image depth control device according to any one of claims 1 to 4.

Images