JP7123222B1 - Display control device, display control method and display control program - Google Patents

Abstract

An object of the present invention is to appropriately switch image quality in AR content. A display control device according to the present application includes a reproduction unit that reproduces a moving image to be superimposed on a captured image of the surroundings, and a static operation of the moving image by a predetermined operation while the moving image is being reproduced by the reproducing unit. and a display control unit that, when the accepting unit accepts a still operation, switches to a still image of a moving image with higher image quality than the moving image and displays the still image. [Selection drawing] Fig. 1

Description

The present invention relates to a display control device, a display control method and a display control program.

In recent years, augmented reality (AR) is becoming popular as content provided via the Internet. In such AR, for example, an AR object viewed from a virtual viewpoint is superimposed on a captured image (see, for example, Patent Document 1).

Japanese Patent Application Laid-Open No. 2020-95602

However, in the related art, for example, there is room for improvement in appropriately switching the image quality of AR content.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a display control device, a display control method, and a display control program capable of appropriately switching the image quality of AR content.

In order to solve the above-described problems and achieve the object, a display control device according to the present invention includes a reproduction section, a reception section, and a display control section. The reproducing unit reproduces a moving image to be superimposed on a captured image of surroundings. The accepting unit accepts a still operation of the moving image by a predetermined operation while the moving image is being reproduced by the reproducing unit. The display control unit switches to display a still image of the moving image having higher image quality than the moving image when the receiving unit receives the still operation.

According to the present invention, it is possible to optimize the image quality of AR content.

FIG. 1 is a diagram illustrating an example of display control processing according to the embodiment. FIG. 2 is a block diagram illustrating a configuration example of the information processing apparatus according to the embodiment; FIG. 3 is a diagram illustrating an example of information stored in a multi-view video database according to the embodiment; FIG. 4 is a block diagram illustrating a configuration example of the display control device according to the embodiment; FIG. 5 is a diagram illustrating an example of a stationary operation according to the embodiment; FIG. 6 is a diagram illustrating an example of a stationary operation according to the embodiment; 7 is a flowchart illustrating an example of a processing procedure executed by the information processing apparatus according to the embodiment; FIG. 8 is a flowchart illustrating an example of a processing procedure executed by the display control device according to the embodiment; FIG. FIG. 9 is a hardware configuration diagram showing an example of a computer that implements the functions of the display control device according to the embodiment.

EMBODIMENT OF THE INVENTION Below, the form (it is hereafter described as "embodiment".) for implementing the display control apparatus, display control method, and display control program which concerns on this application is demonstrated in detail, referring drawings. Note that the display control device, the display control method, and the display control program according to the present application are not limited to this embodiment.

[Embodiment]
[1. Display control processing]
First, an example of display control processing according to the embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of display control processing according to the embodiment.

As shown in FIG. 1 , the display system S according to the embodiment has, for example, an information processing device 10 and a display control device 50 . The information processing device 10 is an information processing device that distributes various types of AR (Augmented Reality) content to each display control device 50 . The information processing device 10 is implemented by a server device or a cloud system.

As shown in FIG. 1, the information processing apparatus 10 has a multi-view video database, and distributes multi-view videos registered in the multi-view video database as respective multi-view images. Here, the multi-viewpoint video is a video obtained by rendering a three-dimensional model from a plurality of predetermined viewpoints, or a video captured by installing a plurality of shooting devices around an actual subject. Also, each image forming the multi-view video is a multi-view image.

The display control device 50 is, for example, various terminal devices such as a smartphone and an HMD (Head Mount Display), and provides the AR content 100 of multi-view video distributed from the information processing device 10 so that the user can view it.

For example, the display control device 50 provides the AR content 100 in which the virtual object Ov of the multi-view video distributed from the information processing device 10 is superimposed on the captured image G of the surroundings. That is, the display control device 50 provides so-called video see-through type AR content 100 . Note that the AR content 100 provided by the display system S is not limited to the video see-through type, and may be the optical see-through type.

By the way, for example, in providing multi-view video AR content, there is room for improvement in appropriately switching the image quality of virtual objects in the multi-view video. For example, if AR content of moving images with high-quality multi-view video is provided, there is a possibility that the moving images cannot be played back smoothly from the viewpoint of data capacity, various processing loads, and the like.

For example, when providing the AR content 100 as a moving image, the time for viewing each multi-view image is short, so even if the image quality of the multi-view image is reduced, the user does not care about the deterioration of the image quality. hard to become

On the other hand, for example, when the moving image is stopped, the display time of the multi-view image is long. This is a problem in providing high AR contents.

Focusing on this point, the display system S according to the embodiment switches the display to a high-quality virtual object Ov when displaying a still image virtual object Ov in the AR content 100 .

Specifically, as shown in FIG. 1, the display control device 50 starts acquiring multi-viewpoint images from the information processing device 10 (step S01). Subsequently, the display control device 50 reproduces and displays the moving image of the multi-viewpoint video each time the multi-viewpoint video is acquired, that is, the AR content 100 is reproduced by so-called streaming reproduction (step S02).

In the example shown in FIG. 1, the virtual object Ov is an idol group, and a case is shown in which a moving image of singing video of the idol group is superimposed on the captured image G as the virtual object Ov.

Subsequently, when the display control device 50 receives a still operation for the moving image (step S03), it performs switching processing of the virtual object Ov (step S04). The still operation here is, for example, a moving image pause operation, but also includes a still image shooting operation for the AR content 100 as described later.

For example, when the display control device 50 receives the stop operation in step S03, the display control device 50 transmits to the information processing device 10 information about the reproduction time when the moving image is still, and displays the number of high-quality virtual objects Ov at the corresponding reproduction time. A viewpoint image is acquired from the information processing device 10 .

Then, the display control device 50 superimposes the acquired high-quality virtual object Ov on the current captured image G, thereby providing the AR content 100 with the high-quality virtual object Ov.

Further, the display control device 50 changes the viewpoint position with respect to the virtual object Ov by, for example, following the change in the imaging position of the captured image G, that is, following the change in the background image of the virtual object Ov in the AR content 100. to the information processing apparatus 10 .

Based on the request, the display control device 50 acquires the high-quality virtual object Ov whose viewpoint position has been changed from the information processing device 10, and provides the AR content 100 of the acquired virtual object Ov.

As a result, the user can view the high-quality virtual object Ov viewed from multiple viewpoints, so that the user can view the desired point from the desired angle.

As described above, in the display control process according to the embodiment, when the moving image of the virtual object Ov is played, if the still operation is received, the still image of the virtual object Ov with higher image quality than the moving image is displayed.

Therefore, according to the display control process according to the embodiment, the image quality of the moving image and the still image can be properly used, so that the image quality of the AR content 100 can be optimized. It should be noted that the present invention is not limited to multi-viewpoint video, and can be applied to other moving images as well. That is, when the three-dimensional graphic moving image is stopped, it may be switched to a high-quality still image.

[2-1. Configuration example of information processing device]
Next, a configuration example of the information processing apparatus 10 will be described with reference to FIG. FIG. 2 is a block diagram showing a configuration example of the information processing apparatus 10. As shown in FIG. As shown in FIG. 2, the information processing apparatus 10 includes a communication section 20, a storage section 30, and a control section 40.

The communication unit 20 is implemented by, for example, a NIC (Network Interface Card) or the like. The communication unit 20 uses networks such as various wireless communication networks such as 4G (Generation), 5G, LTE (Long Term Evolution), WiFi (registered trademark) or wireless LAN (Local Area Network), or various wired communication networks. Information is sent and received to and from an external device via the network.

The storage unit 30 is realized by, for example, a semiconductor memory device such as a RAM or flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 30 also has a multi-view video database 31 .

The multi-view video database 31 is a database that stores multi-view videos. FIG. 3 is a diagram showing an example of information stored in the multi-view video database 31. As shown in FIG. As shown in FIG. 3, the multi-view video database 31 stores items of information such as "video ID", "image group", "size information", "audio", and "sound source coordinates" in association with each other.

The “video ID” is an identifier for identifying each multi-view video, and the “image group” is a group of images to be provided as multi-view video, still images or moving images shot simultaneously from a plurality of viewpoints. This data summarizes

Also, the "image group" includes, for example, moving images and still images. For example, a moving image is an image group related to a multi-view video distributed as a moving image, and a still image is an image group related to a multi-view video distributed as a still image.

For example, a group of still images has higher image quality than a group of moving images. A still image group is captured by a high-performance camera, for example, compared to a moving image group. It should be noted that the image group of the still image may be obtained by increasing the resolution, bit rate, etc. of the image group of the moving image.

"Size information" is information about the actual size of the virtual object Ov appearing in the multi-view image or multi-view video. “Audio” indicates audio data to be played along with the multi-view video, and “sound source coordinates” is information regarding the coordinates of the sound source of the audio data in the multi-view video.

When an artist's singing video is provided as the AR content 100, the voice is the sound source of the artist singing, and the sound source coordinates are the head (for example, mouth) of the artist.

Returning to the description of FIG. 2, the control unit 40 will be described. The control unit 40 is, for example, a controller, and is stored in a storage device inside the information processing apparatus 10 by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an MPU (Micro Processing Unit), or the like. Various programs are implemented by executing the RAM as a work area.

As shown in FIG. 2, the controller 40 includes a selector 41 and a distributor 42 . The selection unit 41 selects the corresponding multi-view video from the multi-view video database 31 based on the multi-view video distribution request received from each display control device 50 .

For example, the distribution request includes information about the video ID, and the selection unit 41 selects a multi-view video that matches the video ID included in the distribution request from the multi-view video database 31 .

Further, for example, during distribution of multi-view video, the selection unit 41 receives the user's point-of-regard information for the AR content 100 from each display control device 50, and performs viewpoint switching processing for the multi-view video based on the point-of-regard information. .

For example, the point-of-regard information is information about the imaging range and imaging orientation of the captured image G in the real space, and is information about the real space displayed in the AR content 100 . That is, it is information about the gaze point of the user in the AR content 100 . For example, when the user changes the position of the display control device 50 (for example, an outward facing camera 61 described later), the gaze point changes, and information about the changed direction and amount of change becomes the gaze point information.

The selection unit 41 selects a viewpoint position for the multi-view video based on the point-of-regard information, and selects a multi-view image corresponding to the selected viewpoint position from the multi-view video database 31, thereby performing viewpoint switching processing. In addition, the selection unit 41 performs a process of switching the viewpoint position at any time following changes in the point-of-regard information.

Further, when receiving a request for switching the image quality from the display control device 50, the selection unit 41 switches the image quality of the virtual object Ov. Specifically, when the display control device 50 receives an image quality switching request while distributing the moving image of the virtual object Ov, the selection unit 41 selects a high-quality still image of the virtual object Ov at the corresponding reproduction time. . As a result, the still image of the virtual object Ov with high image quality is delivered to the display control device 50, and the AR content 100 of the still image showing the virtual object Ov with high image quality is displayed on the display control device 50. FIG.

Further, when the selection unit 41 receives an image quality switching request while distributing the still image of the virtual object Ov, the selection unit 41 selects the virtual viewpoint image of the moving image. That is, when the pause operation is canceled, the image quality is returned to the image quality for moving images.

The distribution unit 42 distributes the multi-viewpoint video selected by the selection unit 41 to each display control device 50 . The multi-view images distributed by the distribution unit 42 are temporally continuous multi-view images, and the distribution unit 42 distributes multi-view images viewed from viewpoint positions set based on the point-of-regard information as needed. to deliver multi-view video. In addition, the distribution unit 42 distributes the multi-view video together with information about audio and the like to the display control device 50 .

[2-2. Configuration example of display control device]
Next, a configuration example of the display control device 50 according to the embodiment will be described with reference to FIG. FIG. 4 is a block diagram showing a configuration example of the display control device 50. As shown in FIG. As shown in FIG. 4 , the display control device 50 includes a communication section 60 , an outward facing camera 61 , an inward facing camera 62 , a sensor group 63 , a display section 64 , an audio output section 65 , a storage section 70 and a control section 80 . Note that the outward camera 61 , the inward camera 62 , the sensor group 63 , the display section 64 and the audio output section 65 may be devices different from the display control device 50 .

The communication unit 60 is realized by, for example, a NIC (Network Interface Card) or the like. The communication unit 60 uses networks such as various wireless communication networks such as 4G (Generation), 5G, LTE (Long Term Evolution), WiFi (registered trademark) or wireless LAN (Local Area Network), or various wired communication networks. Information is sent and received to and from an external device via the network.

The outward camera 61 is a camera that captures an image of the outside of the display control device 50 (for example, the side opposite to the display surface of the display unit 64), and captures an image of the real space. For example, the captured image captured by the outward facing camera 61 is used as the background image of the virtual object Ov in the AR content 100 . The inward facing camera 62 is a camera that captures an image of the inside of the display control device 50 (for example, the display surface side of the display unit 64), and captures an image of the user viewing the AR content 100. FIG.

The sensor group 63 is composed of various sensors for detecting, for example, the attitude of the display control device 50, the distance to an object existing in the real space, and the three-dimensional shape. For example, the sensor group 63 includes a gyro sensor, lidar, and the like.

The display unit 64 is, for example, a touch panel display, displays various images such as the AR content 100, and receives various user operations. The audio output unit 65 is, for example, a surround speaker or a stereo speaker, and outputs various sounds that flow together with the AR content 100 .

The storage unit 70 is realized by, for example, a semiconductor memory device such as a RAM or flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 70 also has a map information storage unit 71 and a size information storage unit 72 .

The map information storage unit 71 stores map information of the real space. For example, the map information is an environment map indicating the surrounding environment of the display control device 50, and is created based on the processing result by the self-position estimation unit 81, which will be described later.

Next, the controller 80 will be described. The control unit 80 is, for example, a controller, and is stored in a storage device inside the information processing apparatus 10 by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an MPU (Micro Processing Unit), or the like. Various programs are implemented by executing the RAM as a work area.

As shown in FIG. 4 , the control unit 80 has a self-position estimation unit 81 , a display surface setting unit 82 , a playback unit 83 , a reception unit 84 , a display control unit 85 and an audio control unit 86 . Based on various information input from the outward camera 61 and the sensor group 63, the self-position estimation unit 81 creates an environment map of the real space using a technique such as Slam (Simultaneous Localization and Mapping), and , the coordinates (for example, world coordinates) and orientation of the display control device 50 are estimated.

In addition, the self-position estimation unit 81 stores information on the created environment map as map information in the map information storage unit 71, and also stores information on the estimated position and orientation, that is, the above gaze point information, on the display surface setting unit 82. pass to Further, while the AR content 100 is being provided, the point-of-regard information is also transmitted to the information processing device 10 .

The display surface setting unit 82 sets a virtual display surface (hereinafter simply display surface) for displaying the virtual object Ov in the real space based on the environment map created by the self-position estimation unit 81 .

For example, the display surface setting unit 82 searches for a display surface on which the virtual object Ov can be displayed from the environmental map. Here, the display surface that can be displayed corresponds to, for example, the bottom surface of the display space that can contain the virtual object Ov displayed as the AR content 100 .

The display space here is a three-dimensional space that can contain the virtual part Ov in the real space. That is, when displaying the virtual object Ov, a three-dimensional space that can be sufficiently displayed becomes the display space, and the bottom surface thereof becomes the display surface of the virtual object Ov.

As a result, for example, in the AR content 100, it is possible to avoid in advance an erroneous display in which the virtual object Ov is displayed through a wall or ceiling in the real space. Note that, for example, the display surface setting unit 82 may reduce the display size of the virtual object Ov in accordance with the real space so that the virtual object Ov fits in the real space. In other words, the display size of the virtual object Ov may be set according to the real space.

After setting the display space and the display surface, the display surface setting unit 82 writes various coordinate information regarding the display space and the display surface to the map information storage unit 71 . For example, the display surface is the floor of the real space, but it may be a part of the user's body (for example, the palm of the hand) depending on the type of the virtual object Ov. Alternatively, the display space itself may be made to float above the actual floor. Further, the display surface setting unit 82 may independently set the display space and the display surface for each virtual object Ov to be displayed. Further, for example, content in which the virtual object Ov itself moves within the AR content 100 is conceivable, so the display surface setting unit 82 sets the display space and the display surface so as to include the movement range of the virtual object Ov. may

After that, the display surface setting unit 82 updates the display size of the virtual part object Ov according to the imaging position of the captured image G. FIG. For example, the display surface setting unit 82 recognizes the imaging position of the captured image G (for example, the position of the outward facing camera 61) based on the point-of-regard information received from the self-position estimating unit 81. FIG. Then, the display surface setting unit 82 sets the display size based on the positional relationship between the imaging position and the display surface. Specifically, the display surface setting unit 82 sets the display size so that the closer the imaging position and the display surface are, the larger the display size is, and the farther the imaging position is from the display surface, the smaller the display size is.

In this case, assuming that the virtual object Ov actually exists on the display surface, the display size of the virtual object Ov is updated so as to have the actual size shown in the captured image G.

The reproducing unit 83 reproduces a moving image to be superimposed on the captured image G that captures the surroundings. For example, the reproduction unit 83 reproduces the moving image of the virtual object Ov by streaming reproduction of the multi-viewpoint image transmitted from the information processing device 10 as needed. Also, the reproducing unit 83 passes information about the reproduced moving image to the display control unit 85 .

The accepting unit 84 accepts an operation to freeze a moving image by a predetermined operation during playback of the moving image by the playback unit 83 . For example, accepting unit 84 accepts a stationary operation based on a user's operation on display unit 64, which is a touch panel. The accepting unit 84 passes information about the still operation to the display control unit 85 when accepting the still operation.

A specific example of the stationary operation will now be described with reference to FIGS. 5 and 6 are diagrams illustrating an example of a stationary operation according to the embodiment. For example, as shown in FIG. 5, the accepting unit 84 accepts an operation of the pause button B1 displayed on the display unit 64 as a stationary operation.

For example, the user can cause the display unit 64 to display the pause button B1 by performing a tap operation on the display unit 64 during playback of the AR content 100, and tap the displayed pause button B1. Thus, the receiving unit 84 receives the pause operation.

Further, as shown in FIG. 6, the accepting unit 84 may accept the operation of the shooting button B1 displayed on the display unit 64 as the still operation. The user can cause the shooting button B2 to be displayed on the display unit 64 by performing a tap operation on the display unit 64 during playback of the AR content 100. By tapping the displayed shooting button B2, the reception can be performed. A unit 84 accepts a still operation by photographing.

It should be noted that the stationary operation described above is an example, and may be changed arbitrarily. That is, for example, the accepting unit 84 may accept a stationary operation by voice input or gesture operation.

Returning to the description of FIG. 4, the display control unit 85 will be described. When the accepting unit 84 accepts the still operation, the display control unit 85 switches to display a still image of a moving image with higher image quality than the moving image.

For example, the display control unit 85 superimposes the virtual object Ov of the moving image reproduced by the reproducing unit 83 on the captured image G so as to be displayed on the display surface set by the display surface setting unit 82, thereby displaying the moving image. Generate the AR content 100 of the image.

Further, when the accepting unit 84 accepts the still operation, the display control unit 85 requests the information processing apparatus 10 to switch the virtual object Ov to the still image at the reproduction time at which the still operation was performed.

As a result, the high-quality virtual object Ov is distributed from the information processing apparatus 10, and the display control unit 85 displays the still image AR content 100 related to the high-quality virtual object Ov. Further, for example, when a still image shooting operation is accepted as a still operation, the display control unit 85 continues to display the moving image related to the virtual object Ov, and at the same time, the height at the angle of view at which the shooting operation was performed. A still image of the virtual object Ov of image quality is received from the information processing device 10 and stored as a screen shot superimposed on the captured image G, for example.

In this way, the display control unit 85 displays a still image with a higher image quality than a moving image in response to a still operation. You can switch.

The audio control unit 86 controls audio output based on the positional relationship between the imaging position of the captured image G and the audio generation source set in the multi-view image. For example, the virtual object Ov is a person whose head is set as the sound source.

Here, for example, when the imaging position of the outward facing camera 61 changes, the position of the virtual object Ov in the AR content 100 changes, and the position of the sound source changes accordingly.

Therefore, the audio control unit 86 controls the output of the audio output unit 65 based on these positional relationships, thereby performing processing for matching the positions of the audio sources in the AR content 100 . Specifically, for example, when the virtual object Ov is displayed on the right side of the captured image G, the audio output unit 65 is controlled so that the sound can be heard from the user's right side, and the virtual object Ov is displayed on the front side of the captured image G. When displayed, the audio output unit 65 is controlled so that the audio can be heard from the near side.

In this way, the audio control unit 86 can provide highly realistic AR content 100 by controlling audio control according to changes in the position of the audio source. Note that the number of sound generation sources is not limited to one, and may be plural. In this case, the audio control unit 86 performs audio control based on the positional relationship for each audio source.

Further, for example, when there are a plurality of sound sources, the sound control unit 86 may perform various adjustments such as volume for each sound source, or may integrate the sound sources. Also, the audio control unit 86 may accept an operation to change the position of the audio source, for example. In this case, for example, the sound source may be set at arbitrary coordinates different from the initially set sound source.

Also, the audio control unit 86 may delete, add, etc. the audio based on the user's operation, for example. That is, it is also possible to combine arbitrary sounds from a plurality of multi-viewpoint images and output them. In this case, for example, it is possible to provide the AR content 100 as if the voice sung by a different singer is sung by another singer's virtual object Ov.

[3. Processing procedure]
Next, processing procedures executed by the information processing device 10 and the display control device 50 according to the embodiment will be described with reference to FIGS. 7 and 8. FIG. First, a processing procedure executed by the information processing apparatus 10 will be described with reference to FIG. FIG. 7 is a flowchart showing a processing procedure executed by the information processing apparatus 10. As shown in FIG. Note that the processing procedure described below is repeatedly executed by the control unit 40 of the information processing apparatus 10 each time a distribution request is acquired.

As shown in FIG. 7, first, the information processing device 10 receives a distribution request (step S101). Subsequently, the information processing device 10 distributes the multi-view video based on the received distribution request (step S102).

Subsequently, the information processing device 10 determines whether or not there is a request for image quality switching associated with the still operation from the display control device 50 (step S103). When the switching request is received (step S103; Yes), the information processing apparatus 10 distributes a still image with higher image quality than the moving image (step S104).

If the information processing apparatus 10 does not request switching (step S103; No), the process of step S104 is omitted, and the process proceeds to step S105. After that, the information processing apparatus 10 determines whether or not the multi-view video being distributed has ended (step S105).

The information processing apparatus 10 ends the process when the multi-view video is finished (step S105; Yes). Further, when the multi-view video is being continued (step S105; No), the information processing apparatus 10 proceeds to the process of step S103.

Next, a processing procedure executed by the display control device 50 will be described with reference to FIG. FIG. 8 is a flowchart showing an example of a processing procedure executed by the display control device 50. As shown in FIG. The processing procedure described below is executed by the control unit 80 of the display control device 50 each time the AR content 100 is provided.

As shown in FIG. 8, first, the display control device 50 sets a display surface A of the virtual object Ov in the real space based on, for example, a captured image of the surroundings (step S201).

Subsequently, the display control device 50 starts displaying the AR content 100 based on the virtual object Ov of the moving image distributed from the information processing device 10 (step S202). Subsequently, the display control device 50 determines whether or not a still operation for the moving image has been accepted (step S203). If a still operation has been accepted (step S203; screen (step S204).

If the display control device 50 has not received a still operation (step S203; No), the process proceeds to step S205. After that, the information processing apparatus 10 determines whether or not the AR content 100 being provided has ended (step S205), and if the AR content 100 has ended (step S205; Yes), the processing ends. Further, when the AR content 100 is being continued (step S205; No), the display control device 50 proceeds to the process of step S203.

[4. Modification]
The display system S described above may be implemented in various different forms other than the above embodiment. Therefore, other embodiments of the display system S will be described below.

For example, display control device 50 may be configured separately from the cameras (outward facing camera 61 and inward facing camera 62). The content provided by the display control device 50 is not limited to the AR content 100. For example, based on the captured image G, VR (Virtual Reality) content obtained by converting the real space into an animation or the like may be provided.

[5. effect〕
As described above, the display control device 50 according to the embodiment includes the reproduction unit 83 that reproduces a moving image that is superimposed on the captured image that captures the surroundings, and the reproduction unit 83 that reproduces the moving image by a predetermined operation. A receiving part 84 for receiving a still operation of a moving image, and a display control part 85 for switching and displaying a still image of a moving image having higher image quality than the moving image when the receiving part 84 receives the still operation. Therefore, according to the display control device 50 according to the embodiment, it is possible to appropriately switch the image quality of the AR content 100 .

Further, in the display control device 50 according to the embodiment, the reception unit 84 receives a still image photographing operation for a moving image as a still operation, and the display control unit 85 receives a still image of high image quality in the still image photographed by the photographing operation. display the image. Therefore, according to the display control device 50 according to the embodiment, for example, it is possible to take a photograph including a high-quality still image from a moving image.
The service related to the AR content 100 can be improved.

Further, in the display control device 50 according to the embodiment, the reception unit 84 receives a pause operation for the moving image as a still operation, and the display control unit 85 displays a high-quality still image at the playback time stopped by the pause operation. indicate. Therefore, according to the display control device 50 according to the embodiment, it is possible to provide the AR content 100 including the high-quality still image at the playback time that the user wants to watch, so that the service related to the AR content 100 can be improved. can.

In addition, in the display control device 50 according to the embodiment, the reception unit 84 reproduces moving images of multi-viewpoint images that can be viewed from multiple viewpoints, and the display control unit 85 reproduces multi-viewpoint images with higher image quality than the moving images. Switch to a still image and display it. Therefore, according to the display control device 50 according to the embodiment, it is possible to provide the AR content 100 including the still image showing the high-quality virtual object Ov viewed from the angle that the user wants to gaze at. can be improved.

In addition, in the display control device 50 according to the embodiment, the display control unit 85 switches and displays the viewpoint position with respect to the still image in accordance with the imaging position of the captured image. Therefore, according to the display control device 50 according to the embodiment, it is possible to provide the AR content 100 including the still image showing the high-quality virtual object Ov viewed from the angle that the user wants to gaze at. can be improved.

Further, a display control method according to an embodiment is a display control method executed by a computer, and includes a reproduction step of reproducing a moving image to be superimposed on a captured image of the surroundings, and during reproduction of the moving image in the reproducing step. a receiving step of receiving a still operation of a moving image by a predetermined operation; and a display control step of switching to and displaying a still image of a moving image with higher image quality than the moving image when the still operation is received by the receiving step. .

Further, the display control program according to the embodiment performs a reproduction procedure for reproducing a moving image to be superimposed and displayed on a captured image of the surroundings, and a static operation of the moving image by a predetermined operation while the moving image is being reproduced according to the reproduction procedure. A computer is caused to execute a reception procedure for reception and a display control procedure for switching to display a still image of a moving image having a higher image quality than a moving image when a static operation is received by the reception procedure.

[6. Hardware configuration]
Also, the display control device 50 according to the above-described embodiments is implemented by a computer 1000 configured as shown in FIG. 9, for example. FIG. 9 is a hardware configuration diagram showing an example of a computer that implements the functions of the display control device 50 according to the embodiment. Computer 1000 has CPU 1100 , RAM 1200 , ROM 1300 , HDD 1400 , communication interface (I/F) 1500 , input/output interface (I/F) 1600 and media interface (I/F) 1700 .

The CPU 1100 operates based on programs stored in the ROM 1300 or HDD 1400 and controls each section. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started up, a program depending on the hardware of the computer 1000, and the like.

The HDD 1400 stores programs executed by the CPU 1100, data used by these programs, and the like. Communication interface 1500 receives data from other devices via network (communication network) N, sends the data to CPU 1100, and transmits data generated by CPU 1100 to other devices via network N. FIG.

Through an input/output interface 1600, the CPU 1100 connects an output device such as a display and a printer, and an input device such as a keyboard and a mouse (in FIG. 9, the output device and the input device are collectively referred to as an "input/output device" ). CPU 1100 acquires data from an input device via input/output interface 1600 . CPU 1100 also outputs the generated data to an output device via input/output interface 1600 .

Media interface 1700 reads programs or data stored in recording medium 1800 and provides them to CPU 1100 via RAM 1200 . CPU 1100 loads such a program from recording medium 1800 onto RAM 1200 via media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. etc.

For example, when the computer 1000 functions as the display control device 50 according to the embodiment, the CPU 1100 of the computer 1000 implements the functions of the control unit 120 by executing programs loaded on the RAM 1200 . CPU 1100 of computer 1000 reads these programs from recording medium 1800 and executes them, but as another example, these programs may be acquired via network N from another device.

As described above, some of the embodiments of the present application have been described in detail based on the drawings. It is possible to carry out the invention in other forms with modifications.

[7. others〕
Further, among the processes described in the above embodiments and modifications, all or part of the processes described as being performed automatically can be performed manually, or described as being performed manually. All or part of the processing can also be performed automatically by known methods. In addition, information including processing procedures, specific names, various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

Also, each component of each device illustrated is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution/integration of each device is not limited to the illustrated one, and all or part of them can be functionally or physically distributed/integrated in arbitrary units according to various loads and usage conditions. Can be integrated and configured.

Further, the above-described embodiments and modifications can be appropriately combined within a range that does not contradict the processing contents.

Also, the above-mentioned "section, module, unit" can be read as "means" or "circuit". For example, the self-position estimator 81 can be read as self-position estimator means or a self-position estimator circuit.

10 information processing device 20 communication unit 30 storage unit 31 multi-view video database 40 control unit 41 selection unit 42 distribution unit 50 display control unit 70 storage unit 71 map information storage unit 80 control unit 81 self-position estimation unit 82 display surface setting unit 83 Reproduction unit 84 Reception unit 85 Display control unit 86 Audio control unit 100 AR content G Captured image Ov Virtual object

Claims (5)

a reproducing unit that reproduces a moving image superimposed on a captured image of the surroundings;
a reception unit that receives a still operation of the moving image by a predetermined operation while the moving image is being reproduced by the reproducing unit;
a display control unit that switches to and displays a still image of the moving image with higher image quality than the moving image when the static operation is received by the receiving unit ;
The reception unit
Receiving a still image pause operation or a shooting operation for the moving image as the still operation;
The display control unit
When the pause operation is accepted by the accepting unit, the high-quality still image at the playback time when the pause operation was stopped is displayed, and when the shooting operation is accepted by the accepting unit, storing the high-quality still image of the angle of view taken by the shooting operation while continuing to display the moving image;
A display control device characterized by: The playback unit
playing back the moving image of a multi-viewpoint image that can be viewed from multiple viewpoints;
The display control unit
The display control device according to claim 1, wherein the display is switched to the still image of the multi-viewpoint image having a higher image quality than the moving image. The display control unit
3. The display control device according to claim 1, wherein the viewpoint position of the still image is switched according to the imaging position of the captured image. A display control method executed by a computer,
a reproducing step of reproducing a moving image to be displayed superimposed on a captured image of the surroundings;
a receiving step of receiving an operation to freeze the moving image by a predetermined operation while the moving image is being played back by the playing step;
a display control step of switching to and displaying a still image of the moving image having higher image quality than the moving image when the static operation is received by the receiving step ;
The receiving step includes
Receiving a still image pause operation or a shooting operation for the moving image as the still operation;
The display control step includes:
When the pause operation is accepted by the accepting step, displaying the high-quality still image at the playback time at which the pause operation stopped, and when the photographing operation is accepted by the accepting step, storing the high-quality still image of the angle of view taken by the shooting operation while continuing to display the moving image;
A display control method characterized by: a playback procedure for playing back a moving image superimposed on a captured image of the surroundings;
a reception procedure for accepting an operation to freeze the moving image by a predetermined operation while the moving image is being reproduced by the reproducing procedure;
causing a computer to execute a display control procedure for switching to and displaying a still image of the moving image having higher image quality than the moving image when the still operation is accepted by the accepting procedure ;
The reception procedure includes:
Receiving a still image pause operation or a shooting operation for the moving image as the still operation;
The display control procedure includes:
When the pause operation is accepted by the acceptance procedure, the high-quality still image at the playback time stopped by the pause operation is displayed, and when the shooting operation is accepted by the acceptance procedure, storing the high-quality still image of the angle of view taken by the shooting operation while continuing to display the moving image;
A display control program characterized by:

Images