JP6880410B2 - Video display system, video display method and video display program - Google Patents

Description

The present invention relates to a technique for allowing an observer to perceive the movement of a spatial image in the depth direction.

By arranging a transparent screen in front of a three-dimensional real space that is the background of a stage or the like and projecting a two-dimensional image on the screen, the two-dimensional image is a three-dimensional and floating space image in the air. There is a video display system that makes it look as if.

Such an image display system has a feature that there are few clues to make the observer perceive that the image is flat, such as not making the observer feel the existence of the image frame. The psychological sense of depth that a spatial image exists at a predetermined position in space can be expressed more easily than a three-dimensional display device with a naked eye having a complicated structure.

JP-A-2017-49354

Since the conventional image display system has the above-mentioned projection mechanism, it is possible to give a sense of depth to the planar image projected on the transparent screen. However, it is difficult to give the plane image movement in the depth direction, that is, to make the observer perceive the movement of the spatial image in the depth direction.

Regarding this problem, Patent Document 1 employs a method of adjusting the position of the spatial image in the depth direction by controlling the transmittance of the variable transmittance plate. However, in the case of this method, the movement of the spatial image in the depth direction can be expressed only discretely.

The present invention has been made in view of the above circumstances, and the first is to make the observer perceive that the spatial image is moving in the depth direction from each viewpoint even if there are a plurality of viewpoints to be observed. The second purpose is to perceive that the spatial image is continuously moving in the depth direction.

In order to solve the above problems, the spatial image moving direction determining device according to claim 1 uses a two-dimensional image of the three-dimensional space viewed from a predetermined viewpoint position, and moves the depth of the spatial image displayed in the three-dimensional space. When the direction is different from the direction of the depth side of the three-dimensional space, a calculation unit that calculates a predetermined movement direction allowable range that can be perceived as the space image moving in the depth direction of the three-dimensional space, and a plurality of different viewpoint positions. A determination unit for determining a movement direction on a two-dimensional image for moving the spatial image in the depth direction of the three-dimensional space by using a common range common to a plurality of movement direction allowable ranges corresponding to the above. It is a feature.

The spatial image moving direction determining device according to claim 2 is the spatial image moving direction determining device according to claim 1, wherein the three-dimensional space is a rectangular parallelepiped space, and the calculation unit sets the moving direction allowable range. The angle of the depth side of the rectangular parallelepiped space with respect to the horizontal axis of the two-dimensional image when the rectangular parallelepiped space is viewed from the viewpoint position on the predetermined side, and the line segment from the space image to the apex of the rectangular parallelepiped space forming the depth side. It is characterized in that the angle range between the angle and the angle is calculated.

The spatial image moving direction determining device according to claim 3 is the spatial image moving direction determining device according to claim 2, wherein the determination unit has an angle range in which the angle of the spatial image with respect to the horizontal axis in the depth moving direction is the angle range. When it is included in the common range according to the above, it is characterized in that the depth moving direction is determined as the moving direction on the two-dimensional image.

The spatial image moving direction determining device according to claim 4 is the spatial image moving direction determining device according to claim 2 or 3, wherein the determination unit has an angle of the spatial image with respect to the horizontal axis in the depth moving direction. When it is not included in the common range related to the angle range, the moving direction to the upper limit angle or the lower limit angle of the common range is determined as the moving direction on the two-dimensional image.

The spatial image moving direction determining device according to claim 5 is the spatial image moving direction determining device according to any one of claims 2 to 4, wherein the spatial image moving direction is moved among the four depth sides forming the rectangular space. A search unit for searching a depth side having a minimum relative angle with respect to a direction inclination is further provided, and the calculation unit is characterized in that the movement direction allowable range is calculated using the minimum depth side.

The spatial image display device according to claim 6 sets the moving direction of at least one or more spatial images displayed in the three-dimensional space in the depth direction for each frame that is continuous in time, according to any one of claims 1 to 5. A changing unit that changes the moving direction determined by the spatial image moving direction determining device described in the above, and a display unit that continuously displays the spatial image at a position on the changed depth moving direction for the frame. , Is provided.

In the method for determining the moving direction of a spatial image according to claim 7, the spatial image moving direction determining device uses a two-dimensional image of the three-dimensional space viewed from a predetermined viewpoint position to move the depth of the spatial image displayed in the three-dimensional space. When the direction is different from the direction of the depth side of the three-dimensional space, a step of calculating a predetermined movement direction allowable range that can be perceived as the space image moving in the depth direction of the three-dimensional space, and a plurality of different viewpoint positions. It is characterized in that a step of determining a movement direction on a two-dimensional image for moving the spatial image in the depth direction of the three-dimensional space is performed using a common range common to a plurality of corresponding movement direction allowable ranges. To do.

The spatial image moving direction determining program according to claim 8 is characterized in that a computer functions as the spatial image moving direction determining device according to any one of claims 1 to 5.

According to the present invention, even if there are a plurality of viewpoints to be observed, the observer can perceive that the spatial image is moving in the depth direction at each viewpoint. In addition, it is possible to perceive that the spatial image is continuously moving in the depth direction.

It is a figure which shows the whole structure of a video display system. It is a figure which shows the example of the space image. It is a figure which shows the functional block composition of the space image movement direction determination apparatus 1. It is a figure which shows the functional block composition of a space image output device 2. It is a figure which shows the determination processing flow of the depth movement direction of a space image. It is a figure which shows the example of the bounding box. It is a figure which shows the example of the real space area and the observation viewpoint range. It is a figure which shows the example of a viewpoint group and a vertex viewpoint. It is a figure which shows the example of the 2D image taken from the apex viewpoint. It is a figure which shows the example of the 2D image taken from the apex viewpoint. It is a figure which shows the display processing flow of a space image.

<Outline of the invention>
Among the moving directions of the planar image (spatial image in the three-dimensional space) projected on the screen, there is a moving direction that is psychologically perceived as moving in the depth direction due to the relationship with the three-dimensional space that is the background.

Therefore, in the present invention, when the depth movement direction of the spatial image is different from the direction (the original movement direction) that is psychologically perceived as the movement in the depth direction from the viewpoint of the observer, all viewpoints of the observer. The common movement direction tolerance that is allowed to be interpreted as movement in the depth direction is obtained, and the movement in the direction within the tolerance is projected on the screen as the movement in the depth direction of the spatial image.

As a result, even if there are a plurality of viewpoints to be observed, it is possible to make the observer perceive that the spatial image is moving in the depth direction at each viewpoint.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

<Configuration example of video display system>
FIG. 1 is a diagram showing an overall configuration of a video display system according to the present embodiment. As shown in FIG. 1, this video display system includes a spatial image moving direction determining device 1 that determines the depth moving direction of a spatial image displayed in a three-dimensional space, and a spatial image output device 2 that outputs a spatial image to the three-dimensional space. A background image display device 3 that displays a background image in a three-dimensional space, and a transparent space image display device 4 that displays a space image output from the space image output device 2 are provided.

In this video display system, as shown in FIG. 1, the spatial image display device 4 is arranged so as to be tilted to the rear side at an angle of about 45 degrees when viewed from the observer 100, and further, the back side of the spatial image display device 4 is arranged. By arranging the background image display device 3 in the space image display device 3 and arranging the space image output device 2 above the space image display device 4, the planar image (2D) projected from the space image output device 2 onto the space image display device 4 Image) is expressed as a spatial image in a three-dimensional and floating state in the air. For example, as shown in FIG. 2, the ball can be three-dimensionally shown to the observer 100 on an empty stage. Further, various contents can be provided by appropriately changing the plane image and the background image.

The configuration of the video display system shown in FIG. 1 is an example. For example, the spatial image moving direction determining device 1 is used as a preprocessing unit for determining the depth moving direction of the spatial image in advance, and the spatial image output device 2 is placed at a position on the moving direction determined by the preprocessing unit. It may be realized by one device as a sequential processing unit that outputs a spatial image. In addition, the spatial image output device 2 and the spatial image display device 4 may be used as one spatial image display device.

<Function of spatial image moving direction determining device 1>
Next, the function of the spatial image moving direction determining device 1 will be described. FIG. 3 is a diagram showing a functional block configuration of the spatial image moving direction determining device 1. As shown in FIG. 3, the spatial image moving direction determining device 1 includes a background information storage unit 11, an observation viewpoint range storage unit 12, a dominant line perspective cue search unit 13, a depth perception allowance calculation unit 14, and a depth perception allowance calculation unit 14. The optimum movement direction vector calculation unit 15 and the optimum movement direction vector storage unit 16 are provided.

The background information storage unit 11 is a functional unit that regards a real space area that is the background of a spatial image as a rectangular parallelepiped and records the values of the width, height, and depth of the rectangular parallelepiped. An arbitrary shape can be considered as the real space area of the background, but in the case of a shape other than a rectangular parallelepiped (for example, a shape having unevenness in a part), the area of that shape is set as the floor surface by simple user input or the like. After constructing a rectangular parallelepiped by dividing it into a ceiling surface, left and right wall surfaces, and a background surface, record the values of the width, height, and depth of the rectangular parallelepiped. In addition, left-handed coordinates with the horizontal, vertical, and depth ridges (sides) of the rectangular parallelepiped as the x-axis, y-axis, and z-axis, respectively, are set, and the surface facing the background surface is the image surface on which the spatial image is displayed. And.

For information on how to construct a rectangular parallelepiped from a shape other than a rectangular parallelepiped, for example, "Satoshi Iizuka," An Efficient Modeling System for 3D Scenes from a Single Image ", IEEE Computer Graphics and Applications, vol.32, no.6, 2012 Year, p.18-p.25 ".

The observation viewpoint range storage unit 12 is a functional unit that records the range in which the viewpoint position of the observer observing the spatial image exists as the observation viewpoint range. Specifically, the observation viewpoint range storage unit 12 obtains a set of coordinate values (x, y, z) of the observer's viewpoint position in the Cartesian coordinate system when an appropriate point in the real space is defined as the origin. The area in the bounding box surrounding the set is set as the observation viewpoint range, and the set of the coordinate values of the eight vertices of the bounding box is recorded.

The observer always observes the image plane of the spatial image in one direction from the front side, and does not observe it from the back side. Regarding the three-dimensional space (rectangular parallelepiped) in the background, the ridgelines of the upper right, lower right, upper left, and lower left parallel to the z-axis are the upper right line, lower right line, upper left line, and lower left line, respectively, when viewed from the observer's viewpoint position. Call it, and let the position of each center of gravity be the position of each ridgeline. Furthermore, the vertices on the ceiling surface of the rectangular parallelepiped are called the upper left front point, upper left back point, upper right back point, and upper right front point clockwise from the left front. Similarly, for the floor surface of a rectangular parallelepiped, the apex of the floor surface is called the lower left front point, the lower left back point, the lower right back point, and the lower right front point clockwise from the left front.

The dominant line perspective cue search unit 13 is a functional unit that searches for a dominant line perspective cue when the background three-dimensional space (rectangular parallelepiped) is viewed from each viewpoint position recorded in the observation viewpoint range storage unit 12. The dominant line perspective cue is a component of a three-dimensional space that can be perceived as a spatial image moving in the depth direction in the three-dimensional space, and is psychologically perceived as moving in the depth direction.

When the three-dimensional space is a rectangular parallelepiped, such a dominant line perspective cue has four patterns depending on the relative position between the viewpoint position and the spatial image, and each is uniquely determined by one ridgeline. Specifically, if the viewpoint position is higher than the spatial image and to the left, it is the lower right line, if it is higher than the spatial image and to the right, it is the lower left line, if it is lower than the spatial image and to the left, it is the upper right line, and if it is lower than the spatial image, it is to the right. In some cases, the upper left line is the dominant line perspective queue.

The depth perception allowance calculation unit 14 (calculation unit) uses a two-dimensional image of the three-dimensional space viewed from a predetermined viewpoint position by a virtual camera, and the depth movement direction of the spatial image displayed in the three-dimensional space is the depth ridgeline of the three-dimensional space. It is a functional unit that calculates a predetermined allowable range of movement direction in which it can be perceived that the spatial image is moving in the depth direction of the three-dimensional space when it is different from the direction of (dominant line perspective cue) (movement direction that should be originally).

Specifically, when the three-dimensional space is a rectangular parallelepiped, the depth perception allowance calculation unit 14 (calculation unit) is dominant at the predetermined viewpoint position with respect to the horizontal axis of the two-dimensional image when the rectangular parallelepiped is viewed from the predetermined viewpoint position. The angle range between the angle of the line perspective cue and the angle of the line segment from the position of the center of gravity of the spatial image to the apex of the rectangular parallelepiped space forming the dominant line perspective cue is calculated. Then, this angle range is defined as the depth perception permissible range (= movement direction permissible range) allowed for perceiving the depth movement at the predetermined viewpoint position.

The optimum movement direction vector calculation unit 15 (determination unit) moves a spatial image in the depth direction of a three-dimensional space by using a common range common to a plurality of depth permissible ranges corresponding to a plurality of different viewpoint positions. It is a functional unit that determines the moving direction on the image.

Specifically, the optimum movement direction vector calculation unit 15 (determination unit) obtains a common range of the permissible depth perception range at all the viewpoint positions to be observed, and moves the depth of the spatial image as seen from all the viewpoint positions or each viewpoint position. When the angle of direction is included in the common range, the depth movement direction of the spatial image viewed from each viewpoint position is linearly interpolated in the depth movement direction of the spatial image, and the linearly interpolated depth movement direction vector is optimally moved direction vector storage unit. Save at 16.

On the other hand, when the angle in the depth moving direction of the spatial image is not included in the common range, the optimum moving direction vector calculation unit 15 (determining unit) exceeds the upper limit value of the common range in the angle in the depth moving direction of the spatial image. If it is, the depth movement direction is updated to the upper limit value, and if it is below the lower limit value, the depth movement direction is updated to the lower limit value, and the updated depth movement direction vector is stored in the optimum movement direction vector storage unit 16.

Further, when there is no common range in each depth perception allowable range in a plurality of viewpoint positions such that the upper limit value at a predetermined viewpoint position is lower than the lower limit value in another viewpoint position, the depth movement direction of the spatial image at each viewpoint position. The vector value obtained by averaging the true values (described later) of is stored in the optimum moving direction vector storage unit 16.

The optimum movement direction vector storage unit 16 is a functional unit that stores the movement direction vector in the depth direction calculated by the optimum movement direction vector calculation unit 15 as the optimum movement direction vector.

<Function of spatial image output device 2>
Next, the function of the spatial image output device 2 will be described. FIG. 4 is a diagram showing a functional block configuration of the spatial image output device 2. As shown in FIG. 4, the spatial image output device 2 includes an optimum movement direction vector storage unit 21, an object information storage unit 22, an object depth movement component update unit 23, and an object output unit 24. Will be done.

The optimum movement direction vector storage unit 21 synchronizes with the optimum movement direction vector storage unit 16 included in the spatial image movement direction determination device 1, and is optimal for moving the spatial image in the depth direction obtained by the spatial image movement direction determination device 1. It is a functional part that stores the movement direction vector. The spatial image output device 2 may use the optimum movement direction vector acquired from the optimum movement direction vector storage unit 16 of the spatial image movement direction determination device 1 as it is without providing the optimum movement direction vector storage unit 21.

The object information storage unit 22 is a functional unit that stores a plurality of frame data that are continuous in time. Specifically, the object information storage unit 22 stores the texture information of the object in each frame to be displayed as a spatial image and the position information (x, y, z) on the spatial image display device 4 from which the object is output. Remember.

The object depth movement component update unit 23 (change unit) extracts only the position information of the movement direction component (z component) in the depth direction from the movement directions of the object output to the spatial image display device 4, and the depth direction thereof. It is a functional part that updates the moving direction component to with the optimum moving direction vector.

The object output unit 24 is a functional unit that outputs an object to the spatial image display device 4 at a position on the depth movement direction updated by the optimum movement direction vector.

<Background image display device 3 and spatial image display device 4>
Next, the background image display device 3 and the spatial image display device 4 will be described.

The background image display device 3 is a device that displays a real space area as a background. As shown in FIG. 1, it is arranged behind the spatial image display device 4 when viewed from the observer 100, and an image of a real space region is displayed via the spatial image display device 4. For example, it is realized by using a display device having a recording / playback function.

The spatial image display device 4 is a device that transmits an image of a real space region from the background image display device 3 as a background and reflects a plane image (2D image) from the spatial image output device 2. As a result, the observer can be made to perceive that there is a spatial image in the real space of the background image. For example, it is realized by using a half mirror whose transmittance and reflectance are almost equal to each other.

<Operation example of video display system>
Next, the operation of the video display system will be described. In order to easily understand the features of the present invention, an outline of the entire operation will be described first, and then an operation of determining the depth moving direction of the spatial image and an operation of outputting the spatial image will be described.

(Outline of operation)
First, an outline of the overall operation of the video display system will be described.

First, the spatial image moving direction determining device 1 uses a two-dimensional image at each viewpoint position included in the observation viewpoint range, and the depth moving direction of the spatial image object (hereinafter, object) with respect to the horizontal axis of the two-dimensional image. The inclination θ _{0 of, the inclination θ 1} of the dominant line perspective cue at each viewpoint position, _{and the inclination θ 2 of the} line segment from the object toward the apex of the background surface forming the dominant line perspective cue are obtained and recorded. (Step S1).

Next, the spatial image movement direction determining device 1 uses the two inclinations θ ₁ and θ ₂ obtained in step S1 to allow the depth perception permissible range (θ 1 to _{θ 1) for perceiving the depth movement for each viewpoint.} θ ₂ ) is calculated (step S2).

Next, the aerial image movement direction determining apparatus 1 takes the common range (θ ₁ '~θ _2') of depth perception tolerance at all viewpoint position included in the observation position range (theta ₁ through? _2), the _{It is determined whether or not the inclination θ 0} in the depth moving direction at each viewpoint position is within the common range, and a new depth moving direction vector is obtained and recorded based on the determination result (step S3).

Finally, the spatial image output device 2 changes the depth direction of the object projected on the spatial image display device 4 by using the new depth movement direction vector obtained in step 3, and sets the position on the changed depth direction. Output the object (step S4).

(Operation to determine the depth movement direction of the spatial image)
Next, the operation of determining the depth movement direction of the spatial image will be described. FIG. 5 is a diagram showing an operation flow of a method of determining the depth moving direction of the spatial image.

Step S101;
First, the spatial image moving direction determining device 1 considers a real space region that is the background of the spatial image as a rectangular parallelepiped, and records the values of the width, height, and depth of the rectangular parallelepiped in the background information storage unit 11. At this time, the left-handed coordinates are set so that the horizontal, vertical, and depth ridges of the rectangular parallelepiped are the x-axis, y-axis, and z-axis, respectively, and the surface facing the background surface is set as the image plane of the spatial image.

Step S102;
Next, the spatial image moving direction determining device 1 defines the range in which the viewpoint position of the observer observing the spatial image exists as the origin at an appropriate point in the real space, and the coordinate values (x,) in the Cartesian coordinate system. It is obtained as a set of y, z), the area in the bounding box surrounding the set is set as the observation viewpoint range, and the set of the coordinate values of the eight vertices of the observation viewpoint range is recorded in the observation viewpoint range storage unit 12. As shown in FIG. 6, the bounding box is a rectangular body composed of sides tangent to only the outermost viewpoint (+) among a plurality of viewpoints (+) existing in space.

Here, FIG. 7 shows an example of the real space region and the observation viewpoint range formed based on step S101 and step S102. In FIG. 7, for easy understanding, the coordinate system of the rectangular parallelepiped, which is the real space region, and the coordinate system of the observation viewpoint space are the same. The observer shall always observe from the front side of FIG. 7 with respect to the image plane of the spatial image. From the observer's point of view, the rectangular parallelepiped in the real space area has an upper right line, a lower right line, an upper left line, and a lower left line parallel to the z-axis, and the upper left front point, upper left back point, upper right back point, and upper right front point are on the ceiling surface. It has a lower left front point, a lower left back point, a lower right back point, and a lower right front point on the floor.

Step S103;
Next, the spatial image moving direction determining device 1 creates a rectangular parallelepiped of the same scale as the rectangular parallelepiped recorded in the background information storage unit 11 in the virtual space, and creates an object (object) to be displayed as a spatial image in the virtual space. Put. After that, it is assumed that the objects are placed at all the spatial positions in the rectangular parallelepiped of the background, and the subsequent processing is performed for each position. At this time, when processing is performed by a computer having a low processing capacity, a predetermined number of positions may be appropriately sampled from all the positions in the rectangular parallelepiped, and only the sampled sample positions may be set as the object positions.

In addition, the object is placed at all positions in the space inside the rectangular parallelepiped of the background, and the spatial image moving direction determining device 1 converts the rectangular parallelepiped and the object into a two-dimensional image at an arbitrary position in the observation viewpoint range in the virtual space. Install a virtual camera that converts perspective. The orientation of the virtual camera is such that the rectangular parallelepiped and the object can be photographed from the front side.

There are two possible vertical and horizontal positions for the virtual camera: when the virtual camera is higher or lower than the object in the square, when it is to the right of the object, and when it is to the left of the object. When the positions of are combined, the positions of the virtual cameras can be divided into four groups. A set of viewpoint positions in which the virtual cameras belong to the same position group is defined as a "viewpoint group".

After that, the spatial image moving direction determining device 1 executes the loop B shown in FIG. 5 for each viewpoint group, and in the loop B, performs steps S104 to S106 described later for each position of the object. Since the processing operation of each viewpoint group is common, the case where the virtual camera is included in the viewpoint group higher to the left than the object will be described below.

Objectives of steps S104 to S106;
As shown in FIG. 8, when the virtual camera VR is located in the viewpoint group G1 on the left side higher than the object OBJ, the background rectangular body and the two-dimensional image of the object OBJ taken by the virtual camera VR are as shown in FIG. It becomes an image.

At this time, when the object OBJ moves in the depth direction of the rectangular body which is the background, that is, in the z-axis direction, considering the position of the center of gravity of the object OBJ on the image plane, the object OBJ is the viewpoint of the viewpoint shown in FIG. In the coordinate system of the two-dimensional image, it always moves in the negative direction of the x-axis and the positive direction of the y-axis. For reference, when the position of the virtual camera is higher than the object and to the right, the object is always moving in the positive direction of the x-axis and the positive direction of the y-axis. Similarly, if the virtual camera is lower than the object and to the left, the object is always moving in the negative x-axis and negative y-axis, and if the virtual camera is lower than the object and to the right. , The object is always moving in the positive direction of the x-axis and the negative direction of the y-axis.

By comparing the movement trajectory of this object with the line perspective clues of the background, the observer perceives that the object is moving in the depth direction in the real space of the background. The object advances in the direction of the vanishing point formed by the background space. For example, in the situation shown in FIG. 9, this traveling direction has a difference in inclination with respect to the ridgeline of the lower right line among the four ridgelines constituting the rectangular parallelepiped of the background. The smallest. The ridgeline that minimizes the difference in inclination (relative angle) from the depth movement direction of the object, such as the lower right line in this situation, is called the dominant line perspective cue at this viewpoint.

The slope of the dominant line perspective cue changes according to the position of the viewpoint, as described above, the lower right line when the viewpoint position is higher than the object and to the left, the lower left line when the viewpoint position is higher than the object and to the right, and lower than the object. If it is on the left, the upper right line is the dominant line perspective queue, and if it is lower than the object, the upper left line is the dominant line perspective queue.

Here, when the spatial image is displayed as a single 2D image (two-dimensional image), only one moving direction can be expressed. Therefore, the depth movement direction on the two-dimensional image should be optimized so that the observer can interpret that the object is moving in the depth direction even when observing from an arbitrary viewpoint in a given viewpoint position range. think of. In the following, the magnitude of the inclination is the angle formed by, for example, the dominant line perspective queue at each viewpoint position and the horizontal axis in the x-axis direction in the screen coordinate system (x, y) of the virtual camera at that viewpoint position. Of these, the size of the acute angle.

Step S104;
There are innumerable observation viewpoints in the observation viewpoint range, but when considering the viewpoint in which the inclination of the dominant line perspective cue is the maximum or the minimum, only the viewpoint of the apex in the observation viewpoint range needs to be considered. Therefore, the dominant line perspective cue search unit 13 acquires the viewpoint position existing on the apex of the observation viewpoint range and sets it as the apex viewpoint, and further searches for the dominant line perspective cue corresponding to the apex viewpoint. Specifically, in the viewpoint group G1 shown in FIG. 8, the upper left vertex T is set as the apex viewpoint, and the lower right line of the rectangular parallelepiped shown in FIG. 9 is determined as the dominant line perspective queue.

Step S105;
Next, the dominant line perspective cue search unit 13 uses a two-dimensional image (FIG. 10) of the rectangular body of the background viewed from the apex viewpoint, and the depth movement direction (O) of the object OBJ with respect to the x-axis of the two-dimensional image. the inclination theta ₀ of -O '), the inclination theta ₁ of the dominant linear perspective queue (O-a), the inclination theta ₂ of the line segment (O-B) connecting the centroid position and the lower right rear point of the object OBJ , And record them as a set of two-dimensional vectors (x, y) in the screen coordinate system of the corresponding virtual camera.

Step S106;
Next, the depth perception allowance calculation unit 14 _{determines whether the inclination θ 0 in the depth movement direction of the object matches or differs from the inclination θ 1} of the dominant line perspective cue, and if the inclinations are different from each other, The depth movement direction on the two-dimensional image is optimized so that the observer can interpret that the object is moving in the depth direction. Specifically, the depth perception allowance calculation unit 14 determines the angle range (θ _{1) between the slope θ 1 of the dominant line perspective cue and the slope θ 2 of the} line segment connecting the position of the center of gravity of the object and the lower right back point. ~ Θ ₂ ) is calculated, and the angle range is set as the permissible depth perception range with respect to the depth movement direction of the object at the corresponding viewpoint vertex.

After that, steps S104 to S106 described above are repeated by changing the position of the object in the rectangular parallelepiped and further changing the viewpoint group. Of course, in one viewpoint group, it may be carried out at a viewpoint position other than the apex viewpoint.

Step S107;
Finally, the optimal movement direction vector calculation unit 15, a common range sought _{(θ 1 '~θ 2')} , all vertices that common range of depth perception tolerance in all vertices viewpoint (θ ₁ ~θ ₂₎ The permissible range of depth perception of the viewpoint.

Then, the optimal movement direction vector calculation unit 15, there is a common range and the inclination theta ₀ in the depth direction of movement of the object in all vertices viewpoint or each vertex viewpoint to the common range (θ ₁ '~θ _2') If it fits, the depth movement direction of the object viewed from each vertex viewpoint is linearly interpolated in the depth movement direction of the object, and the linearly interpolated depth movement direction vector is stored in the optimum movement direction vector storage unit 16.

On the other hand, even if there is a common range, the angle of the upper limit value in the case where the depth direction of movement of the tilt theta ₀ objects exceeds the common range (θ ₁ '~θ _2') the upper limit value of theta _{2 '} direction, if below the lower limit theta _{1 'is} calculated in the depth direction of movement vectors which moves in the angular direction of the lower limit value, and stores the optimum moving direction vector storage unit 16.

In addition, when there is no common range, the average value of the true values of the depth movement directions of the objects at all the vertex viewpoints is stored in the optimum movement direction vector storage unit 16 as the optimum movement direction vector. The true value of the depth movement direction of an object is the depth movement direction observed when the object actually exists in the three-dimensional space.

In the above description, the case where the relative position of the virtual camera is higher than the object and is on the left is described as an example, but the optimum moving direction of the object can be obtained by the same method for the other three patterns.

(Operation to display a spatial image)
Next, the operation of displaying the spatial image will be described. FIG. 11 is a diagram showing an operation flow of a method of displaying a spatial image. Here, the optimum movement direction vector recorded by the spatial image movement direction determination device 1 is input, and an aerial image that makes the observer perceive the depth movement is displayed. It is assumed that the observation viewpoint range of the observer is the same as the observation viewpoint range used in the spatial image moving direction determining device 1.

Step S201;
First, the object depth movement component updating unit 23 acquires the texture information of the observation target object to be displayed as a spatial image and the position information (x, y, z) of the object from the object information storage unit 22.

Step S202;
Next, the object depth movement component update unit 23 extracts only the movement direction component (z coordinate) in the depth direction from the acquired position information of the object, and the optimum movement direction stored in the optimum movement direction vector storage unit 21. Update with vector. That is, of the three-dimensional movement directions of the object to be observed, the movement in the x direction and the y direction is moved in the original movement direction, and the original movement direction is optimally moved only in the movement in the z direction, which is the depth direction. Change to a position on the direction vector.

Step S203;
Finally, the object output unit 24 outputs the observation target object to the spatial image display device 4 at the position updated in step S202.

The above-mentioned steps S201 to S203 are repeated frame by frame. When a plurality of objects are included in one frame, steps S201 to S203 are repeatedly executed at each position of each object. By repeating this process, the space image displayed in the three-dimensional space can be displayed continuously and simultaneously without any limitation on the number of space images, even though it is a single plane image (2D image). The observer can perceive the movement of the image in the depth direction.

<Effect>
According to the present embodiment, the spatial image moving direction determining device 1 uses a two-dimensional image of the background rectangular space viewed from the front apex viewpoint position, and the depth moving direction of the spatial image displayed in the rectangular space is a rectangular body. Dominant line of space When different from the direction of the perspective cue, a predetermined tolerance range that can be perceived as the spatial image moving in the depth direction of the rectangular space is calculated, and a plurality of tolerance ranges corresponding to a plurality of different viewpoint positions are obtained. Since the movement direction on the two-dimensional image that moves the spatial image in the depth direction of the rectangular space is determined using the common range common to both, the spatial image is in the depth direction at each viewpoint even if there are multiple viewpoints to be observed. It is possible to make the observer perceive that the space is moving to (first purpose).

Further, according to the present embodiment, the spatial image output device 2 determines the moving direction of at least one spatial image displayed in the rectangular space in the depth direction for each frame that is continuous in time. The movement direction is changed to the movement direction determined by the movement direction determination device 1, and the spatial image is continuously displayed at the position on the changed depth movement direction for the continuous frames. , Simultaneous movement of a plurality of spatial images in different depth directions and depth movement with respect to a plurality of viewpoint positions can be realized at the same time (second purpose).

Finally, the video display system described in this embodiment can be realized by a computer. It is also possible to create a video display program (for example, a spatial image moving direction determination program, a spatial image output program, a spatial image display program) for operating a computer as a video display system, and a storage medium for the video display program. is there.

1 ... Spatial image movement direction determination device 11 ... Background information storage unit 12 ... Observation viewpoint range storage unit 13 ... Dominant line perspective cue search unit 14 ... Depth perception allowance calculation unit 15 ... Optimal movement direction vector calculation unit 16 ... Optimal movement Direction vector storage unit 2 ... Spatial image output device 21 ... Optimal movement direction Vector storage unit 22 ... Object information storage unit 23 ... Object depth movement component update unit 24 ... Object output unit 3 ... Background image display device 4 ... Spatial image display device 100 … Observer

Claims (8)

In a video display system including a spatial image moving direction determining device and a spatial image display device,
The spatial image moving direction determining device is
When the depth movement direction of the spatial image displayed in the stereoscopic space is different from the direction of the depth side of the stereoscopic space using the two-dimensional image of the stereoscopic space viewed from the viewpoint position on the predetermined side, the spatial image is the stereoscopic space. A calculation unit that calculates a predetermined movement direction allowable range that can be perceived as moving in the depth direction of
A determination unit for determining a movement direction on a two-dimensional image for moving the spatial image in the depth direction of the stereoscopic space by using a common range common to a plurality of movement direction tolerance ranges corresponding to a plurality of different viewpoint positions. , With
The spatial image display device is
A changing unit for changing the moving direction of the spatial image displayed in the three-dimensional space in the depth direction to the moving direction determined by the spatial image moving direction determining device is provided.
A video display system characterized by this. The three-dimensional space is a rectangular parallelepiped space,
The calculation unit
As the movement direction allowable range, the angle of the depth side of the rectangular parallelepiped space with respect to the horizontal axis of the two-dimensional image when the rectangular parallelepiped space is viewed from the viewpoint position on the predetermined side, and the inner side of the rectangular parallelepiped space forming the depth side from the space image. The image display system according to claim 1, wherein the angle of the line segment to the apex and the angle range between the two are calculated. The decision unit
When the angle of the spatial image with respect to the horizontal axis in the depth moving direction is included in the common range related to the angle range, the claim is characterized in that the depth moving direction is determined as the moving direction on the two-dimensional image. Item 2. The video display system according to item 2. The decision unit
When the angle of the spatial image in the depth movement direction with respect to the horizontal axis is not included in the common range related to the angle range, the movement direction of the common range to the upper limit angle or the lower limit angle is moved on the two-dimensional image. The video display system according to claim 2 or 3, wherein the direction is determined. The spatial image moving direction determining device is
Of the four depth sides forming the rectangular parallelepiped space, a search unit for searching the depth side having the smallest relative angle with respect to the inclination of the space image in the depth movement direction is further provided.
The calculation unit
The video display system according to any one of claims 2 to 4, wherein the movement direction allowable range is calculated using the minimum depth side. The spatial image display device is
For frames that are continuous in time, a display unit that continuously displays the spatial image at a position on the changed depth moving direction is further provided.
The video display system according to any one of claims 1 to 5 . In the image display method performed by the spatial image moving direction determining device and the spatial image display device,
The spatial image moving direction determining device
When the depth movement direction of the spatial image displayed in the stereoscopic space is different from the direction of the depth side of the stereoscopic space using the two-dimensional image of the stereoscopic space viewed from the viewpoint position on the predetermined side, the spatial image is the stereoscopic space. The step of calculating a predetermined movement direction tolerance that can be perceived as moving in the depth direction of
A step of determining a moving direction on a two-dimensional image for moving the spatial image in the depth direction of the stereoscopic space by using a common range common to a plurality of moving direction allowable ranges corresponding to a plurality of different viewpoint positions. And
The spatial image display device
A step of changing the moving direction of the spatial image displayed in the three-dimensional space in the depth direction to the moving direction determined by the spatial image moving direction determining device is performed.
A video display method characterized by this. A video display program comprising operating a computer as the video display system according to any one of claims 1 to 6.

Images