JP2020127133A - Magnification display system and display device - Google Patents

Abstract

To cause an image a viewer is gazing at to be easily and inexpensively magnified and displayed when a VR image is displayed in a display region of a display device that is wearable on the viewer's head.SOLUTION: A magnification display system includes a display device that is wearable on a viewer's head and a distribution server that distributes a VR image captured from a predetermined shooting viewpoint to the display device. The display device includes a display unit that displays an image in a predetermined display region, a display controller for projecting the VR image onto a spherical image projection surface centered on the shooting viewpoint in a virtual space, and causes the display unit to display, in the display region, an image corresponding to the display region included in the projection image projected on the image projection surface, and a state determination unit that determines whether or not a viewer is in a gazing state where the viewer is gazing at a part of the display region. When it is determined by the state determination unit that the viewer is in the gazing state, the display control unit causes the display unit to magnify and display in the display region, at a predetermined magnification ratio, an image corresponding to a region of the projection image at which the viewer is gazing.SELECTED DRAWING: Figure 1

Description

The present invention relates to a magnified display system and a display device that display a VR image in a display area of a display device that can be worn on a viewer's head.

Conventionally, as described in Patent Document 1 or the like, a so-called VR image obtained by shooting with a wide angle of view such as 360 degrees from a predetermined shooting viewpoint is viewed by a viewer such as a head mounted display (hereinafter, HMD). There is known a technique of displaying in a display area of a display device that can be worn on the head of a person.

Specifically, as shown in FIG. 7, the display device projects a VR image on a spherical image projection surface centered on the shooting viewpoint in a virtual space, and the VR image is projected from the image projected on the image projection surface. An image (hereinafter referred to as a display image) of an area corresponding to the posture and the viewing angle of the display area is cut out, and the cut out display image is displayed in the display area.

JP, 2006-285482, A

Using the above-mentioned conventional technique, for example, as shown in section (A) of FIG. 8, it is assumed that a VR image obtained by photographing three subjects SB1 to SB3 having the same shape is displayed in the display area VDA. In this case, as shown in section (B) of FIG. 8, in the display image displayed in the display area VDA, the images of the subjects SB1 and SB3 existing farther than the image of the subject SB2 existing near the shooting viewpoint. Becomes smaller.

Therefore, in order for the viewer to gaze at the subjects SB1 and SB3 that are present far away from the shooting viewpoint, it is necessary for the viewer to perform an operation of enlarging and displaying the subjects SB1 and SB3. Further, in order to allow a viewer wearing the display device to perform the operation, a hard key is provided on a side portion of the display device, or a remote controller capable of communicating with the display device is provided, It was necessary to provide an operating device separately.

The present invention has been made in view of the above circumstances, and when the VR image is displayed in the display area of the display device that can be worn on the viewer's head, the image watched by the viewer is separately displayed. It is an object of the present invention to provide an inexpensive enlargement display system and a display device that can easily perform enlargement display without providing an operating device.

The magnified display system according to the present invention is a magnified display system including a display device that can be worn on the head of a viewer and a distribution server that distributes a VR video imaged from a predetermined photographing viewpoint to the display device. The display device controls the display unit by projecting the VR image onto a display unit that displays an image in a predetermined display area and a spherical image projection surface centered on the shooting viewpoint in the virtual space. A display control unit for displaying a display image corresponding to the display area included in the projection image projected on the image projection surface on the display area, and the viewer gazing at a part of the display area. A state determination unit that determines whether or not the viewer is in a gaze state, the display control unit, if the viewer is determined to be in the gaze state by the state determination unit, the display unit, By controlling, the gaze image included in the projection image and corresponding to the gaze region gazed by the viewer is enlarged and displayed in the display region at a predetermined enlargement ratio.

A display device according to the present invention is a display device that can be mounted on a viewer's head, and includes a storage unit that stores a VR video imaged from a predetermined shooting viewpoint and a display unit that displays the video image in a predetermined display area. And displaying the VR image on a spherical image projection surface centered on the shooting viewpoint in the virtual space, controlling the display unit, and including the display image included in the projection image projected on the image projection surface. A display control unit that displays a display image corresponding to a region in the display region; and a state determination unit that determines whether or not the viewer is in a gaze state of gazing at a part of the display region. The display control unit, when the state determination unit determines that the viewer is in the gaze state, controls the display unit to be included in the projection image, and is watched by the viewer. The gaze image corresponding to the existing gaze area is enlarged and displayed in the display area at a predetermined enlargement ratio.

According to these configurations, when the display image corresponding to the display area included in the projection image projected on the image projection surface is displayed in the display area of the display device that can be worn on the viewer's head. It is determined whether or not the viewer is gazing at a part of the display area. Then, when the viewer is determined to be in the gaze state, the gaze image corresponding to the gaze area in which the viewer gazes is automatically displayed in the display area at a predetermined enlargement ratio without requiring the viewer's operation. It is enlarged and displayed.

Therefore, according to this configuration, it is not necessary to separately provide an operating device for performing an operation of enlarging and displaying the gaze image on the display device that can be worn on the viewer's head, and the viewer can gaze at low cost. It is possible to enlarge and display the watching image that is present on the display area of the display device.

Further, the display device further comprises a posture detection unit that detects the posture of the display area, the state determination unit, the state that the amount of change in the posture detected by the posture detection unit is less than a predetermined amount, It may be determined that the viewer is in the gazing state when the watching is continued for a predetermined time or longer.

According to this configuration, when the viewer gazes at a part of the display area, the state in which the amount of change in the posture of the display region detected by the posture detection unit is less than the predetermined amount continues for the predetermined gaze time or longer. In this case, it is possible to appropriately determine that the viewer is in the gaze state.

In addition, the display device further includes a viewpoint detection unit that detects the viewpoint of the viewer in the display area, and the state determination unit is configured such that the change amount of the viewpoint detected by the viewpoint detection unit is less than a predetermined amount. When a certain state continues for a predetermined gaze time or more, it may be determined that the viewer is in the gaze state.

According to this configuration, when the viewer gazes at a part of the display area, the state in which the amount of change in the viewpoint of the viewer detected by the viewpoint detection unit is less than the predetermined amount continues for the predetermined gaze time or longer. In this case, it is possible to appropriately determine that the viewer is in the gaze state.

The distribution server may distribute the gaze time information indicating the gaze time to the display device together with the VR video, and the state determination unit may use the gaze time indicated by the gaze time information.

According to this configuration, it is determined whether or not the viewer is in the gaze state using the gaze time indicated by the gaze time information distributed from the distribution server together with the VR video. Therefore, according to this configuration, whether or not the viewer is in the gaze state can be individually determined according to the content of the VR video delivered from the delivery server.

Further, the state determination unit, after the gaze video is enlarged display, further determines whether the viewer does not continue the gaze state, the display control unit, by the state determination unit, When it is determined that the viewer does not continue the gaze state, the display unit is controlled, the enlarged display of the gaze image is ended, and the display image corresponding to the display area included in the projection image is displayed. It may be displayed in the display area.

According to this configuration, when the state determination unit determines that the viewer does not continue the gaze state after the gaze image is enlarged and displayed, the gaze image enlargement display ends and the display included in the projection image is displayed. A display image corresponding to the area is displayed in the display area.

Therefore, the viewer can easily display the display image corresponding to the display area on the display area simply by stopping the fixation of the fixation area by moving the viewpoint to a position different from the fixation area in the display area. can do.

Further, the enlargement ratio may be a constant.

According to this configuration, when the state determination unit determines that the viewer is in the gaze state, the gaze image can be enlarged and displayed at a constant enlargement ratio.

Further, the enlargement ratio may be a function that changes according to the elapsed time from the time when the viewer determines that the viewer is in the gaze state by the state determination unit.

According to this configuration, when the state determination unit determines that the viewer is in the gaze state, the enlarged display is performed while changing the size of the gaze image according to the elapsed time from the time when the determination is made. You can

Further, the display control unit may not display the gaze image in an enlarged manner when the gaze image is not an image projected in a predetermined expandable area on the image projection surface.

According to this configuration, when the state determination unit determines that the viewer is in the gaze state, the gaze image is enlarged and displayed when the gaze image is not an image projected in the expandable area on the image projection surface. You can avoid that.

Further, the display control unit is configured such that a gazing point, which is a point on the video projection surface corresponding to a viewpoint in the display area when the viewer is in the gazing state, is a predetermined expandable area on the video projection surface. If it does not exist, the gaze image may not be displayed in an enlarged manner.

According to this configuration, when the state determination unit determines that the viewer is in the gaze state, the gaze point on the image projection plane corresponding to the viewpoint in the display area when the viewer is in the gaze state is When it does not exist in the magnified area on the projection surface, it is possible to avoid magnifying and displaying the gaze image.

Further, when the gaze image is an image projected in a first expandable area on the image projection surface, the display control unit enlarges and displays the gaze image at a first enlargement ratio. Is an image projected in a second expandable region that does not overlap with the first expandable region on the image projection surface, the gaze image may be enlarged and displayed at the second enlargement ratio.

According to this configuration, when the state determination unit determines that the viewer is in the gaze state, the gaze image can be enlarged and displayed at an enlargement ratio according to the position of the gaze image on the image projection surface.

Further, the display control unit enlarges and displays the gaze image at a first enlargement ratio when the gaze point exists in a first enlargeable region on the image projection surface, and the gaze point projects the image. When it exists in the second expandable area that does not overlap with the first expandable area on the surface, the gaze image may be enlarged and displayed at the second enlargement ratio.

According to this configuration, when the state determination unit determines that the viewer is in the gaze state, the gaze image can be enlarged and displayed at an enlargement ratio according to the position of the gaze point on the image projection plane.

Further, the distribution server distributes area specifying information for specifying the expandable area to the display device together with the VR video, and the display control unit displays the expandable area specified by the area specifying information. You may use.

According to this configuration, the gaze image is enlarged and displayed when the gaze image or the gaze point is included in the expandable area specified by the area specifying information distributed from the distribution server together with the VR image. On the other hand, when the gaze image or the gaze point is not included in the enlargeable area, the gaze image is not enlarged and displayed. Therefore, according to the present configuration, it is possible to individually determine whether or not to enlarge and display the gaze video according to the content of the VR video delivered from the delivery server.

Further, the distribution server may distribute the enlargement ratio information indicating the enlargement ratio to the display device together with the VR video, and the display control unit may use the enlargement ratio indicated by the enlargement ratio information.

According to this configuration, the watched image is enlarged and displayed at the enlargement ratio indicated by the enlargement ratio information distributed together with the VR video from the distribution server. Therefore, according to this configuration, it is possible to magnify and display the gaze image at an enlargement ratio according to the content of the VR image distributed from the distribution server.

Further, the display device further includes a setting unit that sets whether or not to allow the enlarged display of the gaze image, and the display control unit sets that the enlarged display of the gaze image is not permitted by the setting unit. If so, it is not necessary to enlarge and display the gaze image.

According to this configuration, by setting the setting unit to not permit the enlarged display of the gaze image, the gaze image is enlarged and displayed when the state determination unit determines that the viewer is in the gaze state. Can be avoided.

According to the present invention, when the VR image is displayed in the display area of the display device that can be worn on the viewer's head, the image gazed at by the viewer can be easily provided without providing a separate operation device. An inexpensive enlarged display system and display device capable of enlarged display can be provided.

It is a block diagram showing an example of functional composition of an expansion display system concerning one embodiment of the present invention. It is a figure which shows an example of the display control of VR image|video by a display control part. It is a figure which shows an example of the enlarged display control of a gaze image by a display control part. It is a figure which shows an example of the gaze image displayed on the display area, and the image which expanded and displayed the gaze image. It is a flowchart which shows an example of operation|movement of the expansion display system which concerns on one Embodiment of this invention. It is a figure which shows an example of an expandable area|region. It is a figure which shows an example of the display control of VR image by a prior art. FIG. 10 is a diagram showing an example of capturing and displaying a VR image according to a conventional technique.

Hereinafter, an enlarged display system according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of a functional configuration of an enlarged display system 100 according to an embodiment of the present invention.

(Structure of enlarged display system 100)
As shown in FIG. 1, the enlarged display system 100 includes a distribution server 1 and a display device 2. The distribution server 1 and the display device 2 are communicably connected via a LAN, the Internet, a network (not shown) such as a broadcast data network.

(Structure of distribution server 1)
The distribution server 1 distributes the VR video imaged from a predetermined imaging viewpoint to the display device 2. Specifically, the distribution server 1 includes a storage unit 11 and a distribution unit 12.

The storage unit 11 is configured by a storage device such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive). The storage unit 11 stores video data indicating a VR video to be distributed. The VR image refers to an image such as a still image or a moving image obtained by shooting with a wide angle of view (for example, 360 degrees) from a predetermined shooting viewpoint with an omnidirectional camera (360 degree camera) or the like. Hereinafter, video data indicating a VR video is abbreviated as VR video.

Further, the storage unit 11 stores the distribution information related to the VR video in association with the VR video to be distributed. The distribution information includes the identification information, the title, and the URL of the VR video to be distributed. The distribution information further includes gaze time information, enlargement ratio information, enlargement display permission information, and the like, which will be described later.

The distribution unit 12 is composed of an encoder and a communication interface circuit that communicates with the display device 2 via the network. The distribution unit 12 encodes the VR video stored in the storage unit 11 by a predetermined method (eg, MPEG-4, HLS (HTTP Live Streaming), etc.), and converts the VR video into the VR video in the storage unit 11. The associated delivery information is delivered to the display device 2 via the network.

Here, distributing VR video and distribution information includes two modes of so-called on-demand distribution and live distribution. That is, in the on-demand distribution, the distribution unit 12 transmits the distribution information via the network to the display device 2 that is allowed to display the VR video to be distributed. Then, in the display device 2, when the URL of the VR video included in the distribution information is accessed, the distribution unit 12 returns the VR video to the display device 2.

On the other hand, in the live distribution, the distribution unit 12, like the broadcast of the video data of the television program, at the predetermined time, the distribution target VR video and the distribution information associated with the VR video in the storage unit 11, Broadcast using a broadcast data network included in the network.

The storage unit 11 and the distribution unit 12 may be provided in one server device, or may be appropriately provided in a plurality of server devices that are communicably connected to each other. In addition, the distribution unit 12 may include a distribution unit 12 that distributes a VR video and a distribution unit 12 that distributes distribution information associated with the VR video, which are distributed to different server devices. ..

(Structure of display device 2)
The display device 2 is configured by a device such as a head mount display (hereinafter, HMD) that can be mounted on the head of a viewer. Specifically, the display device 2 includes a microcomputer including a CPU, a RAM, a ROM, and the like, a communication interface circuit that communicates with the distribution server 1 via the network, and decodes encoded video. A decoder, a display such as a liquid crystal display for displaying an image in a flat display area, and a headband that supports a housing that houses these and is worn on the viewer's head are provided. The display device 2 also includes an angular velocity (gyro) sensor that detects the orientation of the display area.

The HMD is not limited to such a configuration, and the headband may detachably support the smartphone including the microcomputer, the communication interface circuit, the decoder, and the display.

The display device 2 acquires the VR video and the distribution information related to the VR video from the distribution server 1, and displays the acquired VR video in the display area based on the acquired distribution information. Specifically, the display device 2 functions as the posture detection unit 91, the display unit 21, the acquisition unit 22, the display control unit 23, the state determination unit 24, and the setting unit 25, as shown by the solid line portion in FIG. 1.

The posture detection unit 91 periodically detects the posture of the display area. Specifically, the posture detection unit 91 is composed of the angular velocity (gyro) sensor. Note that the present invention is not limited to this, and the posture detection unit 91 may include a plurality of pressure sensors that are arranged on a plate material parallel to the display area and that detects movement of the center of gravity. Alternatively, the posture detection unit 91 is arranged at the four corners of the room and performs wireless communication with an infrared sensor that detects the posture of the head of the wearer of the display device 2 by using infrared rays, whereby the display area The posture may be detected. Alternatively, the posture detection unit 91 is arranged at four corners of a room and performs wireless communication with a camera that captures the surroundings of the wearer of the display device 2, and the head of the wearer of the display device 2 is detected based on the image captured by the camera. Image recognition of the posture of the part may be performed.

The display unit 21 is configured by the display unit and displays an image in the display area under the control of the microcomputer.

The acquisition unit 22 includes the microcomputer and the communication interface circuit, and acquires the VR video and the distribution information related to the VR video by communicating with the distribution server 1.

Specifically, when the distribution unit 12 distributes the VR video and the distribution information related to the VR video in the above-described on-demand distribution mode, the acquisition unit 22 first receives the distribution information transmitted by the distribution unit 12. To do. The acquisition unit 22 stores the received distribution information in a RAM, a ROM or the like included in the microcomputer. After that, when the viewer performs a display operation of the VR video, the acquisition unit 22 accesses the URL of the VR video included in the distribution information stored in the RAM, the ROM, or the like, and the VR video returned from the distribution unit 12 is displayed. get.

On the other hand, when the distribution unit 12 distributes the VR video and the distribution information in the above-described live distribution mode, the acquisition unit 22 causes the distribution unit 12 to broadcast the VR video and the VR video broadcast by using the broadcast data network. Distribution information related to the VR video is acquired.

The display control unit 23 includes the decoder and the microcomputer, and decodes the VR image acquired by the acquisition unit 22. FIG. 2 is a diagram showing an example of display control of VR video by the display control unit 23. Then, the display control unit 23 executes a well-known VR image projection process so that, as shown in FIG. 2, on the spherical image projection surface VSP centered on the shooting viewpoint SVP of the VR image in the virtual space, The decoded VR image is projected. Then, the display control unit 23 cuts out the display image projected on the area VDAa corresponding to the display area VDA from the projection image projected on the image projection surface VSP. Then, the display control unit 23 controls the display unit 21 to display the cutout display image in the display area VDA.

Further, when the viewer wearing the display device 2 changes the posture of the head to change the posture of the display area VDA, the posture detection unit 91 detects the posture after the change of the display area VDA. To do. In this case, the display control unit 23 moves the area of the display image to be cut out from the projection image according to the change in the posture of the display area VDA, as shown by the alternate long and short dash line in FIG. Then, the display control unit 23 cuts out the display image projected in the area corresponding to the display area VDA after the posture change, controls the display unit 21, and displays the cut-out display image in the display area VDA. ..

Referring back to FIG. The state determination unit 24 is configured by the microcomputer, and determines whether or not the viewer is in a gaze state of gazing at a part of the display area VDA.

Specifically, the state determination unit 24 acquires the gaze time information included in the distribution information acquired by the acquisition unit 22. The gaze time information is the time that is considered to be required from when the viewer starts to visually recognize a part of the display area VDA to when the viewer gazes at a part of the display area VDA (hereinafter, gaze time). ) Is information. Then, when the state in which the amount of change in the posture of the display area VDA detected by the posture detection unit 91 is less than the predetermined amount continues for at least the gaze time indicated by the gaze time information, the state determination unit 24 performs viewing. It is determined that the person is in a gaze state.

As a result, when the viewer gazes at a part of the display area VDA, the state in which the change amount of the posture of the display area VDA detected by the posture detection unit 91 is less than the predetermined amount continues for the predetermined gaze time or longer. In this case, it is possible to appropriately determine that the viewer is in the gaze state. Further, it is determined whether or not the viewer is in the gaze state using the gaze time indicated by the gaze time information distributed from the distribution server 1 together with the VR video. Therefore, whether or not the viewer is in the gaze state can be individually determined according to the content of the VR video distributed from the distribution server 1.

Note that the gaze time information may be stored in advance in the RAM, ROM, or the like included in the display device 2. In this case, the state determination unit 24 does not acquire the gaze time information included in the distribution information, but uses the gaze time indicated by the gaze time information stored in advance in the RAM, the ROM, etc. It may be determined whether or not there is.

In addition, when the state determination unit 24 determines that the viewer is in the gaze state, the display control unit 23 controls the display unit 21 to include the gaze area in the projection image, which is gazed by the viewer. The gaze image corresponding to is enlarged and displayed in the display area VDA at a predetermined enlargement ratio.

FIG. 3 is a diagram illustrating an example of enlarged display control of the gaze video FD by the display control unit 23. FIG. 4 is a diagram showing an example of a gaze image FD displayed in the display area VDA and an image EFD in which the gaze image FD is enlarged and displayed. Specifically, the display control unit 23 first acquires the enlargement ratio information included in the distribution information acquired by the acquisition unit 22. The magnifying power information is information indicating the magnifying power when the gazing image FD is magnified and displayed in the display area VDA.

The enlargement ratio is set to, for example, a constant (example: 2). In this case, when the state determination unit 24 determines that the viewer is in the gaze state, the gaze image FD can be enlarged and displayed at a constant enlargement ratio. In addition, the enlargement ratio is, for example, a function F(t) that changes according to the elapsed time t from the time when the state determination unit 24 determines that the viewer is in the gaze state (for example, 2×t (0 second< t<1 second), 2(t>1 second)). In this case, when the state determination unit 24 determines that the viewer is in the gaze state, the gaze image FD is enlarged and displayed while changing the size of the gaze image FD according to the elapsed time t from the time when the determination is made. You can

Then, the display control unit 23, as shown in the section (A) of FIG. 3 and FIG. 4, the central portion of the area VDAa corresponding to the display area VDA on the image projection surface VSP onto which the VR image is projected in the virtual space. It is assumed that a virtual camera VC for shooting the CVDA from the shooting viewpoint SVP at a predetermined angle of view is arranged. Then, the display control unit 23 identifies the area FA on the video projection surface VSP that can be captured by the virtual camera VC as the gazing area (hereinafter, gazing area FA) that the viewer is gazing.

Next, as shown in section (B) of FIG. 3, the display control unit 23 causes the virtual camera VC to display the enlargement ratio information in a direction from the photographing viewpoint SVP toward the central portion CVDA (direction of a dotted arrow). It is moved by the distance corresponding to the indicated magnification. The display control unit 23 uses, for example, the product of the predetermined coefficient and the enlargement ratio as the distance corresponding to the enlargement ratio.

Then, as shown in section (B) of FIG. 3 and FIG. 4, the display control unit 23 acquires the image projected on the area EFA on the image projection surface VSP that can be captured by the virtual camera VC after the movement. .. Then, the display control unit 23 enlarges the acquired image to the size of the display area VDA and displays the enlarged image EFD in the display area VDA, as shown in section (D) of FIG. 4. As a result, the gaze image FD displayed in the display area VDA as shown in section (C) of FIG. 4 is enlarged and displayed as shown in section (D) of FIG.

According to the above configuration, the watched image FD is enlarged and displayed at the enlargement ratio indicated by the enlargement ratio information distributed from the distribution server 1 together with the VR image. Therefore, the watching image FD can be enlarged and displayed at an enlargement ratio according to the content of the VR image distributed from the distribution server 1.

The enlargement ratio information may be stored in advance in a RAM, a ROM, or the like included in the display device 2. In this case, the display control unit 23 does not acquire the enlargement ratio information included in the distribution information, but uses the enlargement ratio indicated by the enlargement ratio information stored in advance in the RAM, the ROM, or the like to enlarge and display the gazing image FD. You may do so.

Referring back to FIG. The setting unit 25 is configured by the microcomputer and sets whether or not to allow the enlarged display of the gaze video FD by the display control unit 23. Specifically, the distribution information acquired by the acquisition unit 22 includes enlarged display permission information indicating whether to permit enlarged display of the gaze video FD. The setting unit 25 first acquires the enlarged display permission information included in the distribution information acquired by the acquisition unit 22.

The enlargement display permission information may be stored in advance in the RAM, the ROM or the like included in the display device 2. In this case, the display control unit 23 may acquire the enlarged display permission information stored in advance in the RAM, the ROM, or the like, without acquiring the enlarged display permission information from the distribution information acquired by the acquisition unit 22. ..

Then, when the acquired enlarged display permission information indicates that the enlarged display of the gazing image FD is permitted, the setting unit 25 sets to permit the enlarged display of the gazing image FD. On the other hand, when the acquired enlarged display permission information indicates that the enlarged display of the gazing image FD is not permitted, the setting unit 25 sets that the enlarged display of the gazing image FD is not permitted. Specifically, the setting unit 25 stores the data (for example, “1”) indicating that the enlarged display of the gazing image FD is permitted in the RAM, the ROM, or the like to permit the enlarged display of the gazing image FD. Set things up. Similarly, the setting unit 25 stores the data (for example, "0") indicating that the enlarged display of the gazing image FD is not permitted in the RAM, the ROM, or the like, so that the enlarged display of the gazing image FD is not permitted. Set things up.

(Operation flow)
The operation of the enlarged display system 100 will be described below. FIG. 5 is a flowchart showing an example of the operation of the enlarged display system 100 according to the embodiment of the present invention.

As shown in FIG. 5, in the distribution server 1, the distribution unit 12 distributes the VR video stored in the storage unit 11 and the distribution information associated with the VR video to the display device 2 via the network. (Step S10). In this case, in the display device 2, the acquisition unit 22 acquires the distribution information related to the VR video distributed by the distribution unit 12 (step S11), and also acquires the VR video distributed by the distribution unit 12. (Step S12).

Next, the display control unit 23 decodes the VR video acquired in step S12 (step S13), and projects the decoded VR video on the video projection surface VSP (FIG. 2) (step S14). Then, when the attitude detection unit 91 detects the attitude of the display area VDA (step S15), the display control unit 23 performs step S16.

In step S16, the display control unit 23 cuts out the display image projected in the area VDAa corresponding to the display area VDA of the posture detected in step S15 from the projection image projected on the image projection surface VSP in step S14. Then, the display control unit 23 controls the display unit 21 to display the cutout display image in the display area VDA (step S16).

After step S16, it is assumed that the state determination unit 24 determines that the viewer is in the gaze state (YES in step S17). In this case, as described above, the setting unit 25 acquires the enlarged display permission information from the distribution information acquired in step S11 or the ROM or the RAM, and uses the acquired enlarged display permission information to display the control unit 23. It is set whether or not to allow the enlarged display of the gazing image FD by (step S18).

In step S18, it is assumed that the setting unit 25 sets to permit the enlarged display of the gaze video FD (YES in step S18). In this case, as described above, the display control unit 23 controls the display unit 21 to predetermined the gaze image FD corresponding to the gaze area FA included in the projection image projected on the image projection surface VSP in step S14. The display area VDA is enlarged and displayed at the enlargement ratio (step S19). After step S19, step S17 is performed again.

It is assumed that it is determined in step S17 that the viewer is not in the gaze state (NO in step S17). Alternatively, it is assumed that the setting unit 25 sets not to allow enlarged display of the gaze video FD in step S18 (NO in step S18). In these cases (NO in step S17 or NO in step S18), if a predetermined operation for ending the viewing of the VR video is performed (YES in step S20), the operation of enlarged display system 100 shown in FIG. 5 ends. .. On the other hand, when the predetermined operation for ending the viewing of the VR video has not been performed (NO in step S20), the processes of step S15 and thereafter are performed.

That is, the state determination unit 24 determines whether or not the viewer does not continue the gaze state in step S17 after the gaze video FD is enlarged and displayed in step S19. In step S17, it is determined that the viewer has stopped maintaining the gaze state (NO in step S17), and if the predetermined operation for ending the viewing of the VR video has not been performed (NO in step S20), step S15 and thereafter. Is processed. Thereby, the display control unit 23 controls the display unit 21 to end the enlarged display of the gaze image FD and display the display image corresponding to the display region VDA included in the projection image in the display region VDA (step S15). ~S16). As a result, the viewer can easily watch the display image corresponding to the display area VDA simply by stopping watching the watching area FA by moving the viewpoint to a position different from the watching area FA in the display area VDA. It can be displayed in the display area VDA.

As described above, according to the configuration of the above-described embodiment, the display area VDA of the display device 2 that can be worn on the viewer's head corresponds to the display area VDA included in the projection image projected on the video projection surface VSP. When the display image is displayed, it is determined whether or not the viewer is in the gaze state of gazing at a part of the display area VDA. Then, when the viewer is determined to be in the gazing state, the gazing image FD corresponding to the gazing area FA in which the viewer is gazing does not require a viewer's operation, and a predetermined enlargement ratio is applied to the display area VDA. Is automatically enlarged. Therefore, it is not necessary to separately provide the display device 2 that can be mounted on the viewer's head with an operation device for performing an operation of enlarging and displaying the watched image FD, and the watched image FD watched by the viewer can be inexpensively obtained. Can be enlarged and displayed in the display area VDA of the display device 2.

Further, the setting unit 25 sets that the enlarged display of the gaze image FD is not permitted, so that the gaze image FD is enlarged and displayed when the state determination unit 24 determines that the viewer is in the gaze state. Can be avoided.

(Modified embodiment)
The above embodiment is merely an example of the embodiment according to the present invention, and is not intended to limit the present invention to the above embodiment. For example, the following modified embodiments may be used.

(1) The display device 2 further includes a line-of-sight tracking sensor that detects the viewpoint of the viewer in the display area VDA by using infrared rays or the like, and further functions as the viewpoint detection unit 92 as shown by the broken line portion in FIG. You may do it. The viewpoint detection unit 92 is composed of the line-of-sight tracking sensor and periodically detects the viewpoint of the viewer in the display area VDA.

In accordance with this, when the state determination unit 24 determines that the amount of change in the viewpoint detected by the viewpoint detection unit 92 is less than the predetermined amount for the continuous watching time or more, the viewer is in the watching state. May be determined. In this case, when the viewer gazes at a part of the display area VDA, the state in which the amount of change in the viewpoint of the viewer detected by the viewpoint detection unit 92 is less than the predetermined amount continues for the watching time or longer. In this case, it can be appropriately determined that the viewer is in the gaze state.

In this case, the display control unit 23, when enlarging and displaying the gaze image FD, instead of the central part CVDA, the viewpoint detected by the viewpoint detection unit 92 when the state determination unit 24 determines that the state is the gaze state ( After that, the gaze point) may be used. That is, the display control unit 23 assumes that the virtual camera VC (FIG. 3) shoots the gaze point from the shooting point SVP at a predetermined angle of view in the virtual space, and the image projection that can be taken by the virtual camera VC is performed. The area on the surface VSP may be specified as the gazing area FA. Then, the display control unit 23 moves the virtual camera VC in the direction from the photographing viewpoint SVP to the gazing viewpoint by a distance corresponding to a predetermined enlargement ratio, so that the gazing image projected in the gazing area FA. The FD should be enlarged and displayed.

(2) FIG. 6 is a diagram showing an example of the expandable areas EPA1 to EPA3. In step S19 (FIG. 5), the display control unit 23 gazes when the gaze image FD is not an image projected in the predetermined expandable areas EPA1 to EPA3 on the image projection surface VSP, as shown in FIG. You may make it perform the process after step S20, without enlarging and displaying the image FD.

According to this configuration, when the state determination unit 24 determines that the viewer is in the gaze state, when the gaze image FD is not an image projected in the expandable areas EPA1 to EPA3 on the image projection surface VSP, It is possible to avoid enlarging and displaying the gazing image FD.

Alternatively, the display controller 23 allows the viewer to gaze at the central portion CVDA (section (A) in FIGS. 3 and 4) of the area VDAa corresponding to the display area VDA on the image projection surface VSP onto which the VR image is projected. May be a point on the video projection surface VSP (hereinafter, the gazing point) corresponding to the viewpoint in the display area VDA. Then, in step S19 (FIG. 5 ), the display control unit 23 enlarges and displays the gaze image FD when the gaze point does not exist within the predetermined expandable areas EPA1 to EPA3 (FIG. 6) on the image projection surface VSP. Instead, the processing after step S20 may be performed.

Similarly, in the modified embodiment (1), the display control unit 23 causes the state determination unit 24 to determine the gaze state, and the image projection surface VSP corresponding to the gaze point detected by the viewpoint detection unit 92. The point above may be the gaze point. Then, in step S19 (FIG. 5), the display control unit 23 enlarges the gaze image FD when the gaze point does not exist within the predetermined expandable areas EPA1 to EPA3 (FIG. 6) on the image projection plane VSP. The processing after step S20 may be performed without displaying.

According to these configurations, when the state determination unit 24 determines that the viewer is in the gaze state, the attention on the video projection surface VSP corresponding to the viewpoint in the display area VDA when the viewer is in the gaze state. When the viewpoint does not exist in the expandable areas EPA1 to EPA3 on the image projection surface VSP, it is possible to avoid enlarging and displaying the gaze image FD.

Further, in accordance with these, in step S19 (FIG. 5), the display control unit 23 sets the gazing image FD or the gazing point within the first expandable area EPA1 on the image projection plane VSP, as shown in FIG. When the image is a projected image, the gaze image FD may be enlarged and displayed at the first enlargement ratio. Similarly, in step S19 (FIG. 5), the display control unit 23 projects the gazing image FD or the gazing point into the second expandable area EPA2 that does not overlap the first expandable area EPA1 on the video projection surface VSP. In the case of a captured image, the gazing image FD may be enlarged and displayed at the second enlargement ratio. Further, in step S19 (FIG. 5), the display control unit 23 determines that the gaze image FD or the gaze point does not overlap the first expandable area EPA1 and the second expandable area EPA2 on the image projection surface VSP. When the image is projected in the expandable area EPA3, the gazing image FD may be enlarged and displayed at the third enlargement ratio.

According to this configuration, when the state determination unit 24 determines that the viewer is in the gaze state, the gaze image FD is displayed at the gaze image FD on the image projection plane VSP or the enlargement ratio according to the position of the gaze point. It can be enlarged.

Specifically, the area specifying information for specifying the respective expandable areas EPA1 to EPA3 on the image projection plane VSP may be included in the distribution information related to the VR image distributed from the distribution server 1. For example, when the expandable area is a polygon, the area specifying information may be information indicating the coordinates of the positions of the vertices of the expandable area on the video projection surface VSP. Alternatively, for example, when the expandable area is circular, the area specifying information may be information indicating a coordinate indicating the position of the center of the expandable area on the video projection plane VSP and a radius.

In addition, when the gazing image FD or the gazing point is included in each of the expandable areas EPA1 to EPA3, the respective enlargement ratios (first to third enlargement ratios) used when enlarging the gazing image FD are shown. The enlargement ratio information may be included in the distribution information related to the VR video distributed from the distribution server 1.

Then, in step S19 (FIG. 5), the display control unit 23 may be configured to acquire the area specifying information and the enlargement ratio information from the distribution information acquired in step S11. Then, the display control unit 23 uses each expandable area EPA1 to EPA3 indicated by the acquired area specifying information and each expansion rate (first to third expansion rate) indicated by the acquired expansion rate information. It may be configured as follows.

In these cases, the gaze image FD is enlarged and displayed when the gaze image FD or the gaze point is included in the expandable area specified by the area specifying information distributed from the distribution server 1 together with the VR image. On the other hand, when the gazing image FD or the gazing point is not included in the enlargeable region, the gazing image FD is not enlarged and displayed. Therefore, it is possible to individually determine whether or not to enlarge and display the gaze video FD according to the content of the VR video distributed from the distribution server 1. Further, the watching image FD is enlarged and displayed at the enlargement ratio indicated by the enlargement ratio information distributed from the distribution server 1 together with the VR image. Therefore, the watching image FD can be enlarged and displayed at an enlargement ratio according to the content of the VR image distributed from the distribution server 1.

It should be noted that at least one of the area specifying information and the enlargement ratio information is stored in advance in the RAM, the ROM, or the like included in the display device 2, and the display control unit 23 sets the at least one of the information in step S19 (FIG. 5). The information may be obtained from RAM or ROM.

(3) The display device 2 may not function as the setting unit 25, and step S18 (FIG. 5) may be omitted.

(4) The distribution server 1 and the display device 2 may not communicate with each other via the network. In accordance with this, the distribution unit 12 is configured by an encoder and a recording device that records data and information on a recording medium such as a DVD or a BD (Blu-ray disc), and the distribution unit 12 determines in step S10 (FIG. 5) that The VR image and the distribution information may be recorded on a recording medium. Then, the VR image provider may provide the recording medium to the viewer who owns the display device 2 by mail or the like. Thereby, the VR video and the distribution information may be distributed by the recording medium.

In this case, the acquisition unit 22 may be configured by the microcomputer and an acquisition device that acquires data and information from a recording medium such as a DVD or a BD (Blu-ray disc). Then, the acquisition unit 22 may be configured to acquire the VR video and distribution information from the recording medium provided by the provider in step S11 and step S12 (FIG. 5).

(5) The display device 2 may include a storage unit similar to the storage unit 11. In this case, step S10 (FIG. 5) is omitted, and the acquisition unit 22 acquires the VR video and distribution information from the storage unit without the distribution unit 12 in Steps S11 and S12 (FIG. 5). You can

1: distribution server 11: storage unit 12: distribution unit 100: enlarged display system 2: display device 21: display unit 22: acquisition unit 23: display control unit 24: state determination unit 25: setting unit 91: attitude detection unit 92: Point-of-view detection unit CVDA: Central part of the area corresponding to the display area on the image projection surface onto which the VR image is projected EFA: Area on the image projection surface that can be photographed by the virtual camera after moving EFD: An image of the gaze image enlarged and displayed EPA1 to EPA3: Expandable area FA: Gaze area FD: Gaze image SB1 to SB3: Subject SVP: Shooting viewpoint VC: Virtual camera VDA: Display area VSP: Image projection plane t: Elapsed time

Claims (15)

An enlarged display system comprising: a display device that can be worn on a viewer's head; and a distribution server that distributes a VR video imaged from a predetermined photographing viewpoint to the display device.
The display device is
A display unit that displays an image in a predetermined display area,
The VR image is projected onto a spherical image projection surface centered on the shooting viewpoint in the virtual space, the display unit is controlled, and the VR is included in the projection image projected on the image projection surface. A display control unit for displaying the corresponding display image in the display area;
A state determination unit that determines whether or not the viewer is in a gaze state of gazing at a part of the display area,
Equipped with
When the state determination unit determines that the viewer is in the gaze state, the display control unit controls the display unit to include the gaze that the viewer is gazing in the projection image. A magnified display system for magnifying and displaying a gaze image corresponding to an area in the display area at a predetermined magnification. The display device is
Further comprising a posture detection unit that detects the posture of the display area,
The state determination unit determines that the viewer is in the gaze state when the state in which the amount of change in the posture detected by the posture detection unit is less than a predetermined amount continues for a predetermined gaze time or longer. To do
The enlarged display system according to claim 1. The display device is
Further comprising a viewpoint detection unit that detects the viewpoint of the viewer in the display area,
The state determination unit determines that the viewer is in the gaze state when the state in which the amount of change in the viewpoint detected by the viewpoint detection unit is less than a predetermined amount is continuous for a predetermined gaze time or more. To do
The enlarged display system according to claim 1. The distribution server distributes gaze time information indicating the gaze time together with the VR video to the display device,
The state determination unit uses the gaze time indicated by the gaze time information,
The enlarged display system according to claim 2. The state determination unit, after the gaze video is displayed in an enlarged manner, further determines whether or not the viewer does not continue the gaze state,
The display control unit controls the display unit to end the enlarged display of the gaze image when the state determination unit determines that the viewer does not continue the gaze state, and the projection image is displayed. A display image corresponding to the display area included in the display area,
The enlarged display system according to claim 1. The magnification is a constant,
The enlarged display system according to any one of claims 1 to 5. The enlargement ratio is a function that changes according to the elapsed time from the time when the viewer determines that the viewer is in the gaze state by the state determination unit,
The enlarged display system according to any one of claims 1 to 5. 8. The enlargement according to claim 1, wherein the display control unit does not enlarge and display the gaze image if the gaze image is not an image projected in a predetermined expandable area on the image projection surface. Display system. In the display control unit, a gazing point that is a point on the video projection surface corresponding to a viewpoint in the display area when the viewer is in the gaze state is within a predetermined expandable area on the video projection surface. The enlarged display system according to any one of claims 1 to 7, wherein the gaze image is not displayed in an enlarged manner when it does not exist. When the gaze image is an image projected in a first expandable area on the image projection surface, the display control unit enlarges and displays the gaze image at a first enlargement ratio, and the gaze image is In the case of an image projected in a second expandable region that does not overlap with the first expandable region on the image projection surface, the gaze image is enlarged and displayed at a second enlargement ratio,
The enlarged display system according to claim 8. The display control unit, when the gazing point is present in a first expandable region on the video projection surface, enlarges and displays the gazing image at a first magnification ratio, and the gazing point is on the video projection surface. When present in the second expandable region that does not overlap with the first expandable region, the gaze image is enlarged and displayed at the second enlargement ratio,
The enlarged display system according to claim 9. The distribution server distributes area specifying information for specifying the expandable area to the display device together with the VR video,
The enlargement display system according to any one of claims 8 to 11, wherein the display control unit uses the enlargeable area specified by the area specifying information. The distribution server distributes enlargement ratio information indicating the enlargement ratio to the display device together with the VR video,
The display control unit uses the enlargement ratio indicated by the enlargement ratio information,
The enlarged display system according to any one of claims 1 to 12. The display device is
Further comprising a setting unit for setting whether to allow the enlarged display of the gaze image,
The display control unit does not magnify the gaze image when the setting unit does not allow the magnified display of the gaze image.
The enlarged display system according to any one of claims 1 to 13. A display device that can be worn on a viewer's head,
A storage unit for storing VR images shot from a predetermined shooting viewpoint;
A display unit that displays an image in a predetermined display area,
The VR image is projected onto a spherical image projection surface centered on the shooting viewpoint in the virtual space, the display unit is controlled, and the VR is included in the projection image projected on the image projection surface. A display control unit for displaying the corresponding display image in the display area;
A state determination unit that determines whether or not the viewer is in a gaze state of gazing at a part of the display area,
Equipped with
When the state determination unit determines that the viewer is in the gaze state, the display control unit controls the display unit to include the gaze that the viewer is gazing in the projection image. A display device for enlarging and displaying a gaze image corresponding to an area in the display area at a predetermined enlargement ratio.

Images