JP2020177488A - Program, terminal apparatus, and content reproduction system - Google Patents

Abstract

To provide a program, a terminal apparatus and the like which can realize synchronization of commencement of contents reproduction with ease when contents including motion pictures obtained by capturing a character in a virtual three-dimensional space are reproduced.SOLUTION: A terminal apparatus includes: a content reproducing unit for reproducing contents including a motion picture generated by capturing, by a virtual camera, a virtual three-dimensional space where at least one character is disposed; and a reading unit for, if a captured image obtained by capturing a reproduction information display object displaying reproduction information including reproduction timing information is acquired, extracting, from the captured image, the reproduction timing information for use in designation of the reproduction start timing for the contents. The content reproducing unit starts reproduction of the contents at the reproduction start timing designated based on the reproduction timing information.SELECTED DRAWING: Figure 7

Description

The present invention relates to a program, a terminal device, a content reproduction system, and the like.

In recent years, a content playback system that allows a virtual idol character to play and watch a MV (Music Video) movie of dancing and singing in a virtual three-dimensional space on a terminal device has been in the limelight. .. In such a MV movie, there is a need to watch not a solo idol but a plurality of idols forming an idol group dancing and singing.

As one method for responding to such needs, for example, by utilizing the technique disclosed in Patent Document 1, for example, each character of a plurality of characters constituting a virtual idle group is displayed on each terminal of a plurality of terminals. A method to do is conceivable. According to this method, it is also possible to realize a system in which a plurality of characters can be watched as an MV moving image of dancing and singing by using a plurality of terminals. Patent Document 2 discloses a technique relating to synchronization and change of camera position in an augmented reality game using an AR marker. Further, Patent Document 3 discloses a technique for group registration using a two-dimensional code.

Japanese Unexamined Patent Publication No. 2005-173291 Japanese Unexamined Patent Publication No. 2013-61870 Japanese Unexamined Patent Publication No. 2014-215924

However, in order to realize a content reproduction system using a plurality of terminals, there is a problem that it is necessary to synchronize the movements of characters and music on the plurality of terminals. There is also a problem that the MV video playback is not started until all the users who are participants gather, and it is necessary to wait for a long time. Further, in an environment in which a plurality of terminals cannot communicate with each other or in an environment in which communication is interrupted, there is also a problem that a plurality of terminals cannot realize synchronized video playback.

According to the present disclosure, it is possible to provide a program, a terminal device, a content reproduction system, and the like that can simply realize synchronization of the start of content reproduction in the reproduction of content including a moving image of a character in a virtual three-dimensional space.

One aspect of the present disclosure is a content reproduction unit that reproduces content including a moving image generated by shooting a virtual three-dimensional space in which at least one character is arranged by a virtual camera, and reproduction information including reproduction timing information. Reproduction information to be displayed The content reproduction includes a reading unit that extracts the reproduction timing information for designating the reproduction start timing of the content from the photographed image when the photographed image of the displayed object is acquired. The unit relates to a terminal device that starts playback of the content at the playback start timing specified based on the playback timing information. Further, one aspect of the present disclosure relates to a program that causes a computer to function as each of the above parts, or a computer-readable information storage medium that stores the program.

According to one aspect of the present disclosure, when a photographed image of a reproduction information display object displaying reproduction information including reproduction timing information is acquired, reproduction timing information is extracted from the photographed image. Then, at the playback start timing specified based on the extracted playback timing information, the playback of the content including the moving image generated by shooting the character in the virtual three-dimensional space with the virtual camera is started. By doing so, the playback timing information that specifies the playback start timing is extracted from the shot image without communication or the like, and the character in the virtual three-dimensional space is shot with the virtual camera to generate the moving image. Content playback can be started at the playback start timing. Therefore, it is possible to provide a terminal device, a program, or the like that can simply realize synchronization of the start of content reproduction in the reproduction of content including a moving image of a character in a virtual three-dimensional space.

Further, in one aspect of the present disclosure, as the at least one character, a character selected by the user is arranged in the virtual three-dimensional space, and the reproduction information is for designating the content to be reproduced by the content reproduction unit. The content reproduction unit includes the content identification information, and determines the motion of the character and the position of the virtual camera according to the position of the content specified by the content identification information and the character selected by the user. You may make settings.

In this way, it is possible to select an appropriate motion and set the position of the virtual camera according to the designated content and the position of the character selected by the user.

Further, in one aspect of the present disclosure, a virtual space setting unit for arranging and setting a plurality of characters as the at least one character in the virtual three-dimensional space and a position of the virtual camera are set based on user input information. A virtual camera control unit may be included (a computer may function as a virtual space setting unit and a virtual camera control unit).

In this way, a plurality of characters can be arranged by reflecting the input information of the user, and the virtual camera can be set at an appropriate position according to the arranged plurality of characters.

Further, in one aspect of the present disclosure, a plurality of characters are arranged in the virtual three-dimensional space as the at least one character, and the content reproduction unit is each of the plurality of characters according to the arrangement of the plurality of characters. The motion of the character may be determined.

In this way, when a plurality of characters are arranged in the virtual three-dimensional space, it is possible to select an appropriate motion according to the arrangement position of the characters and operate the characters.

Further, in one aspect of the present disclosure, the reproduction timing information is information that designates a time after a set time from the reference time as a designated time of the reproduction start timing, and the content reproduction unit performs the above-mentioned at the designated time. You may start playing the content.

By doing so, it is possible to start playing the content at a designated time after the set time by simply specifying the set time, and it is possible to improve the convenience of the user.

Further, in one aspect of the present disclosure, an activation unit that activates the content reproduction unit may be included before the designated time (a computer may function as the activation unit).

By doing so, the content reproduction unit can be activated at the timing before the designated time, and the content can be appropriately reproduced.

Further, one aspect of the present disclosure may include a reproduction information generation unit that generates the reproduction information and an output processing unit that performs a process for outputting the reproduction information display object that displays the reproduction information (reproduction information generation). The computer may function as a unit or output processing unit).

In this way, the reproduction information including the reproduction timing information for designating the reproduction start timing of the content can be transmitted to another terminal via the reproduction information display object.

Further, one aspect of the present disclosure is to reproduce content including a moving image generated by shooting a virtual three-dimensional space having a host terminal and at least one guest terminal and arranging at least one character with a virtual camera. The host terminal generates playback information including playback timing information for designating the playback start timing of the content, and outputs a playback information display object for displaying the playback information. When the at least one guest terminal acquires a photographed image obtained by photographing the reproduction information display object, the reproduction timing information is extracted from the photographed image, and the reproduction start designated based on the reproduction timing information. It relates to a content reproduction system that starts reproduction of the content at a timing.

In one aspect of the present disclosure, the host terminal generates playback information including playback timing information, and the guest terminal captures a playback information display object on which the playback information is displayed, thereby providing playback timing information that specifies the playback start timing. , It becomes possible to transmit from the host terminal to the guest terminal. Then, at the reproduction start timing specified by the reproduction timing information, it becomes possible to start the reproduction of the content including the moving image generated by shooting the character in the virtual three-dimensional space with the virtual camera. In this way, even if the host terminal and the guest terminal are not connected by communication, the playback timing information that specifies the playback start timing is transmitted from the host terminal to the guest terminal, and the video content is played back at the playback start timing. You will be able to start. Therefore, it is possible to provide a content reproduction system that can easily realize synchronization of the start of content reproduction in the reproduction of the content including the moving image of the character in the virtual three-dimensional space.

Further, in one aspect of the present disclosure, the at least one guest terminal includes a first guest terminal and a second guest terminal, and the first guest terminal is a user of the first user of the first guest terminal. Setting information including the information of the selected first character is generated as the first user information, the first reproduction information display object for displaying the first user information is output, and the second guest terminal is , The second reproduction information which generates the setting information including the information of the 2nd character selected by the 2nd user of the 2nd guest terminal as the 2nd user information, and displays the 2nd user information. When the display object is output and the host terminal acquires a photographed image obtained by photographing the first reproduction information display object and the second reproduction information display object, the virtual three-dimensional structure is based on the photographed image. The placement position of the first and second characters in space is determined, the reproduction information including the character arrangement information indicating the arrangement position of the first and second characters is generated, and the reproduction information is displayed. When the reproduced information display object is output and each guest terminal of the first and second guest terminals acquires the photographed image obtained by photographing the reproduction information display object, the character arrangement information is obtained from the photographed image. The first and second characters may be arranged in the virtual three-dimensional space based on the character arrangement information, and the content may be reproduced.

In this way, the host terminal can use the first and second guest terminals to provide setting information including information about the first and second characters selected by the first and second users of the first and second guest terminals. It becomes possible to acquire from the guest terminal and arrange the first and second characters in the virtual three-dimensional space.

Further, in one aspect of the present disclosure, when the host terminal acquires a photographed image obtained by photographing the first reproduction information display object and the second reproduction information display object, the first reproduction information display object in the photographed image is obtained. Based on the positional relationship between the reproduction information display object and the second reproduction information display object, the arrangement positions of the first and second characters in the virtual three-dimensional space are determined, and the character arrangement information is generated. You may.

In this way, the first and second characters can be arranged in the virtual three-dimensional space by utilizing the positional relationship between the first reproduction information display object and the second reproduction information display object in the captured image. Become.

Further, in one aspect of the present disclosure, the host terminal has acquired a first captured image obtained by photographing the first reproduction information display object and a second photographed image obtained by photographing the second reproduction information display object. In this case, even if the placement position of the first and second characters in the virtual three-dimensional space is determined based on the acquisition order of the first and second captured images, and the character placement information is generated. Good.

In this way, the first and second characters can be arranged in the virtual three-dimensional space by using the acquisition order of the first and second captured images.

Further, in one aspect of the present disclosure, the host terminal includes the first and second characters selected by the first and second users of the first and second guest terminals and the user of the host terminal. The arrangement of the selected character in the virtual three-dimensional space may be determined.

In this way, in addition to the first and second characters selected by the users of the first and second guest terminals, the characters selected by the user of the host terminal can also be arranged in the virtual three-dimensional space. Become.

Further, in one aspect of the present disclosure, the host terminal generates the reproduction timing information for designating the time after the set time from the reference time as the designated time of the reproduction start timing, and the at least one guest terminal is the said. Playback of the content may be started at a designated time.

By doing so, it is possible to start playing the content at a designated time after the set time by simply specifying the set time, and it is possible to improve the convenience of the user.

Further, in one aspect of the present disclosure, the host terminal may output the reproduction information display object by blinking the screen of the host terminal.

In this way, the playback information can be transmitted from the host terminal to the first and second guest terminals by blinking the screen of the host terminal.

A configuration example of the content playback system of the present embodiment. Configuration example of the terminal device of this embodiment. Explanatory drawing about shooting of reproduction information display thing. Explanatory drawing about reading of reproduction information using reproduction information display thing. Explanatory drawing of output of reproduction information display object by blinking screen. Explanatory drawing about the synchronization method of this embodiment. The flowchart explaining the processing example of this embodiment. 8 (A) and 8 (B) are explanatory views of a character position and a virtual camera setting method. Explanatory drawing of virtual camera and motion setting according to character arrangement. Explanatory drawing of character arrangement method based on user input information. 11 (A) and 11 (B) are explanatory views of a playback start timing setting method. A flowchart illustrating a processing example of a content playback system. A flowchart illustrating a processing example of a content playback system. 14 (A) and 14 (B) are explanatory views of a character arrangement method based on the positional relationship of the reproduced information display object in the captured image. 15 (A) and 15 (B) are also explanatory views of a character arrangement method based on the positional relationship of the reproduced information display object in the captured image. Explanatory drawing of character arrangement method based on acquisition order of photographed image. Explanatory drawing of a method of displaying a reproduction information display object on a large monitor.

Hereinafter, this embodiment will be described. It should be noted that the present embodiment described below does not unreasonably limit the description of the scope of claims. Moreover, not all of the configurations described in the present embodiment are essential configuration requirements.

1. 1. Content playback system, terminal device FIG. 1 shows a configuration example of the content playback system of the present embodiment. The content reproduction system of FIG. 1 includes a host terminal TMH and guest terminals TM1 to TMn. The host terminal TMH can also be called a master terminal, a host device, or the like, and the guest terminals TM1 to TMn can also be called a slave terminal, a client terminal, or the like. Although a plurality of guest terminals TM1 to TMn are provided in FIG. 1, the number of guest terminals may be one.

The guest terminals TM1 to TMn operate, for example, as a guest role, and the host terminal TMH operates, for example, as a host role. The guest terminals TM1 to TMM and the host terminal TMH can be realized by, for example, a terminal device having a configuration as shown in FIG. 2 described later. However, the roles of the host and the guest can be exchanged, such that the terminal that was the role of the guest becomes the host terminal and the terminal that was the role of the host becomes the guest terminal. Therefore, for example, both the program for operating as a guest and the program for operating as a host are installed in each of the guest terminal and the host terminal. In the case of the application example shown in FIG. 17 described later, the host terminal is realized by, for example, a server device, and the guest terminal is realized by a terminal device serving as a client terminal. In this case, the roles of the host and the guest will not be exchanged.

FIG. 2 shows a configuration example of a terminal device that realizes a guest terminal or a host terminal. The terminal device of the present embodiment is not limited to the configuration shown in FIG. 2, and various modifications such as omitting a part of the constituent elements or adding other constituent elements can be performed.

The terminal device is, for example, a portable communication terminal having an Internet connection function (Internet connection, etc.), such as a smartphone or a mobile phone. However, the terminal device may be a portable terminal such as a portable game device or a tablet PC.

The operation unit 160 is for the user to input various operation information. The operation unit 160 can be realized by, for example, an operation button, a touch panel type display, a slide button, various sensors (angular velocity sensor, acceleration sensor, etc.), a microphone, or the like. In the touch panel type display, for example, a touch panel serving as an operation unit 160 and a display unit 190 serving as a display are integrally provided.

The photographing unit 162 photographs an object, and is realized by an image sensor such as a CCD or CMOS sensor and an optical system composed of a focus lens or the like. The photographing unit 162 may be provided with a main camera provided on the opposite side of the screen and a sub in-camera provided on the screen side.

The storage unit 170 stores various types of information. The storage unit 170 functions as a work area such as the processing unit 100 and the communication unit 196. The program and data necessary for executing the program are stored in the storage unit 170. The function of the storage unit 170 can be realized by a semiconductor memory (DRAM, SRAM, VRAM), an HDD (hard disk drive), an SSD (Solid State Drive), or the like. The storage unit 170 includes a content information storage unit 172, a reproduction information storage unit 174, and a virtual space information storage unit 176. The content information storage unit 172 stores information on the content to be reproduced. The content information includes, for example, information on a moving image generated by shooting a virtual three-dimensional space in which a character is arranged by a virtual camera, information on a musical piece played together with the moving image, and the like. The reproduction information storage unit 174 stores reproduction information which is information for content reproduction. The reproduction information includes the reproduction timing information described later. Further, the reproduction information can include content identification information and character setting information. The virtual space information storage unit 176 stores virtual space information, which is information for constructing a virtual three-dimensional space. For example, as virtual space information, information about a three-dimensional object arranged and set in the virtual three-dimensional space is stored. The storage unit 170 may also include a drawing buffer for rendering a virtual space image.

The information storage medium 180 is a medium that can be read by a computer and stores programs, data, and the like. The information storage medium 180 can be realized by an optical disk (DVD, BD, CD), an HDD, a semiconductor memory (ROM), or the like. The processing unit 100 performs various processes of the present embodiment based on the program (data) stored in the information storage medium 180. That is, the information storage medium 180 is a program for causing a computer (a device including an input device, a processing unit, a storage unit, and an output unit) to function as each part of the present embodiment (a program for causing the computer to execute processing of each part). Is memorized.

The display unit 190 is a display that outputs an image generated by the present embodiment, and its function can be realized by a liquid crystal display (LCD), an organic EL display, or the like. In the case of a touch panel type display, the touch panel is integrally formed with a display such as a liquid crystal display or an organic EL display.

The I / F (interface) unit 194 performs interface processing with the portable information storage medium 195, and the function can be realized by an ASIC or the like for I / F processing. The portable information storage medium 195 is for the user to store various types of information, and is a storage device that retains the storage of such information even when the power is not supplied. The portable information storage medium 195 can be realized by an IC card (memory card), a USB memory, a magnetic card, or the like.

The communication unit 196 communicates with an external device (another device) via a wired or wireless network, and its functions include hardware such as a communication ASIC or a communication processor, and communication firmware. Can be realized by.

The program (data) for operating the computer as each part of the present embodiment is distributed from the information storage medium of the server (host device) to the information storage medium 180 (or storage unit 170) via the network and the communication unit 196. You may. The use of the information storage medium by such a server can also be included in the scope of the present embodiment.

The processing unit 100 (processor) performs content reproduction processing, reading processing, virtual space setting processing, virtual camera control processing, startup processing, reproduction information generation processing, and output processing based on operation input information and programs from the operation unit 160. , Display processing, sound processing, etc.

Each process of the present embodiment performed by each unit of the processing unit 100 can be realized by a processor (a processor including hardware). For example, each process of the present embodiment can be realized by a processor that operates based on information such as a program and a memory that stores information such as a program. In the processor, for example, the functions of each part may be realized by individual hardware, or the functions of each part may be realized by integrated hardware. For example, a processor includes hardware, which hardware can include at least one of a circuit that processes a digital signal and a circuit that processes an analog signal. For example, a processor may be composed of one or more circuit devices (for example, ICs, etc.) mounted on a circuit board, or one or more circuit elements (for example, resistors, capacitors, etc.). The processor may be, for example, a CPU (Central Processing Unit). However, the processor is not limited to the CPU, and various processors such as a GPU (Graphics Processing Unit) or a DSP (Digital Signal Processor) can be used. Further, the processor may be a hardware circuit by ASIC. Further, the processor may include an amplifier circuit, a filter circuit, and the like for processing an analog signal. The memory (storage unit 170) may be a semiconductor memory such as SRAM or DRAM, or may be a register. Alternatively, it may be a magnetic storage device such as a hard disk device (HDD), or an optical storage device such as an optical disk device. For example, the memory stores instructions that can be read by a computer, and when the instructions are executed by the processor, the processing of each part of the processing unit 100 is realized. The instructions here may be an instruction set that constitutes a program, or may be an instruction that instructs the hardware circuit of the processor to operate.

The processing unit 100 includes a content reproduction unit 102, a reading unit 104, a virtual space setting unit 106, a virtual camera control unit 108, an activation unit 109, a reproduction information generation unit 110, an output processing unit 112, a display processing unit 120, and a sound processing unit 130. including. As described above, each process of the present embodiment executed by each of these parts can be realized by a processor (or a processor and a memory). It is possible to carry out various modifications such as omitting a part of these components (each part) or adding other components.

The display processing unit 120 performs processing for displaying an image on the display unit 190. For example, the display processing unit 120 performs drawing processing based on the results of various processing performed by the processing unit 100, thereby generating an image and displaying it on the display unit 190. For example, the display processing unit 120 can generate a virtual space image that is an image that can be seen by a virtual camera in a virtual three-dimensional space by performing drawing processing on the drawing buffer of the storage unit 170.

The sound processing unit 130 performs sound generation processing based on the results of various processing performed by the processing unit 100. Specifically, a musical piece (music, BGM), a sound effect, a voice, or the like is generated and output to the sound output unit 192.

Then, as shown in FIG. 2, the terminal device of the present embodiment includes the content reproduction unit 102 and the reading unit 104. For example, when the terminal device is the guest terminal described with reference to FIG. 1, the content reproduction unit 102 and the reading unit 104 are processed. For example, the terminal device serving as a guest terminal operates based on a program for executing the processing of the content reproduction unit 102 and the processing of the reading unit 104, thereby performing the operation as a guest.

The content reproduction unit 102 performs content reproduction processing. Specifically, the content reproduction unit 102 performs a process of reproducing content including a moving image generated by shooting a virtual three-dimensional space in which at least one character is arranged by a virtual camera. A character is a model object that imitates an idol, a singer, a dancer, or the like. The virtual three-dimensional space is also called an object space. In the virtual three-dimensional space, various objects constituting the virtual three-dimensional space, such as a character object and a background object, are arranged and set. Then, the content reproduction unit 102 performs the reproduction processing of the content including the moving image generated by shooting such a virtual three-dimensional space with a virtual camera. Here, the moving image may be a moving image generated in real time based on the information of a three-dimensional object stored in the virtual space information storage unit 176. For example, a moving image generated in real time is played back like streaming playback. The content reproduction unit 102 can reproduce the moving image generated by the real-time rendering in this way. Alternatively, the content reproduction unit 102 may reproduce a moving image created in advance and stored in the content information storage unit 172.

The reading unit 104 performs information reading processing. For example, the reading unit 104 extracts the reproduction timing information from the photographed image when the photographed image of the reproduction information display object which displays the reproduction information including the reproduction timing information is acquired. For example, in the present embodiment, the photographing unit 162 photographs a reproduction information display object that displays reproduction information including reproduction timing information. For example, the photographing unit 162 of the guest terminal photographs the reproduction information display object displayed on the screen of the host terminal. Then, the reading unit 104 extracts the reproduction timing information from the photographed image acquired by the photographing of the photographing unit 162. For example, when the reproduction information display object is a two-dimensional code such as a QR code (QR code is a registered trademark), the reading unit 104 performs a two-dimensional code decoding process or the like to perform reproduction timing included in the reproduction information. Extract information. The playback information can also include content identification information. In this case, the reading unit 104 extracts the content identification information and the reproduction timing information from the captured image acquired by the photographing of the photographing unit 162. For example, the reading unit 104 performs decoding processing of a two-dimensional code such as a QR code to extract content identification information and reproduction timing information included in the reproduction information.

Here, the reproduction timing information is information for designating the reproduction start timing of the content. It is desirable that the reproduction start timing is, for example, a time representing an absolute time. For example, the playback start timing is a designated time that specifies the start time of content playback. Further, the content identification information is information for designating the content to be reproduced by the content reproduction unit 102. For example, when an ID is assigned to the content, the content identification information is the content ID assigned to the content. By using this content identification information, the content reproduction unit 102 can select the content specified by the content identification information from a plurality of contents to be reproduced by the content reproduction unit 102.

Then, the content reproduction unit 102 starts the reproduction of the content at the reproduction start timing designated based on the reproduction timing information. For example, when the reproduction information includes the content identification information and the reproduction timing information, the content reproduction unit 102 selects one content from a plurality of contents stored in advance based on the content identification information. For example, the content information storage unit 172 stores information on a plurality of contents that can be reproduced by the content reproduction unit 102. The content reproduction unit 102 selects one content to be reproduced from the plurality of contents. Then, the content reproduction unit 102 starts the reproduction of the selected content at the reproduction start timing designated based on the reproduction timing information.

In the following, a case where the reproduction information includes both the reproduction timing information and the content identification information will be described mainly as an example. However, if there is only one content to be reproduced, or if a plurality of contents are stored in the content information storage unit 172 but the content to be reproduced is predetermined, the reproduction information is identified as the content. It is not necessary to include the information, and it is sufficient to include only the reproduction timing information.

For example, the terminal device has a time measuring unit (timer) for acquiring the current time. The timekeeping unit updates the time information on the terminal device, for example, by acquiring the time information from the server via the network. As an example, time information is acquired from the server when logging in to an application program for content playback. The time information may be acquired using GPS or the like. Then, the content reproduction unit 102 compares the current time based on the time information in the time measuring unit of the terminal device with the specified time which is the reproduction start timing specified based on the reproduction timing information, and determines that the current time has reached the specified time. When this is done, playback of the selected content based on the content identification information is started. That is, the reproduction of the moving image generated by shooting the virtual three-dimensional space in which at least one character is arranged by the virtual camera is started. It also starts playing the music associated with the video.

Further, in the present embodiment, a character selected by the user is arranged in the virtual three-dimensional space as at least one character described above. The virtual three-dimensional space is, for example, an object space, and a plurality of objects that are three-dimensional models are arranged and set, and a character selected by the user is arranged with respect to the three-dimensional position of such a virtual three-dimensional space. Then, the content reproduction unit 102 determines the motion of the character and sets the position of the virtual camera according to the content specified by the content identification information and the position of the character selected by the user. Further, the content reproduction unit 102 can set the operation (movement) of the virtual camera.

Here, the character selected by the user is, for example, a user's favorite character. For example, a virtual idle group is composed of a plurality of characters, and one of the plurality of characters constituting the virtual idle group is registered in advance as a user's favorite character, for example. The selected character, which is the character selected by the user, does not necessarily have to be such a favorite character. Then, the content to be played by the virtual idle group is determined by the content identification information, and when the selected character such as the user's favorite character is specified, it is the position of the selected character in the performance of the content on the stage. The character position determines the motion of the selected character and the position of the virtual camera. The motion is motion data for moving the character, and the motion data is data for specifying the time-series movement of the skeleton (bone) of the model object of the character. Then, the virtual camera is arranged, for example, at a position facing the selected character arranged at a determined position (position on the stage). When the operation of the virtual camera (camera work) is determined according to the selected character, the virtual camera control unit 108 controls the virtual camera so that the virtual camera performs such an operation.

The virtual space setting unit 106 performs setting processing of a virtual three-dimensional space (object space) in which objects are arranged. For example, a process of arranging and setting a three-dimensional object such as a character object or a background object in a virtual three-dimensional space is performed. That is, the position and rotation angle (synonymous with orientation and direction) of an object in the world coordinate system are determined, and the rotation angle (rotation angle around the X, Y, Z axis) is used at that position (X, Y, Z). Place the object.

The virtual camera control unit 108 controls the position and direction of the virtual camera in the virtual three-dimensional space. For example, when the camera work is performed according to the choreography of the character, the virtual camera control unit 108 controls the position and direction of the virtual camera so as to perform the operation according to the camera work.

Then, in the present embodiment, the virtual space setting unit 106 arranges and sets a plurality of characters as at least one character in the virtual three-dimensional space based on, for example, user input information. For example, a plurality of characters constituting a virtual idle are arranged and set on a virtual stage in a virtual three-dimensional space. For example, a plurality of characters are arranged and set based on the user's input information input using the operation unit 160. Then, the virtual camera control unit 108 sets the position of the virtual camera in the virtual three-dimensional space in which the plurality of characters are arranged and set in this way. For example, among a plurality of characters arranged and set in the virtual three-dimensional space, the position of the virtual camera is set at a position facing the selected character of the user of the terminal device, and a virtual space image obtained by capturing the selected character is generated. To do so.

Further, in the present embodiment, a plurality of characters are arranged in a virtual three-dimensional space as at least one character, and the content reproduction unit 102 determines the motion of each character of the plurality of characters according to the arrangement of the plurality of characters. .. For example, motion data is associated with a character placement position or a character at that placement position. Then, the content reproduction unit 102 reproduces the arrangement position or the motion data associated with the character at the arrangement position according to the arrangement of the characters. In this way, for each character of the plurality of characters arranged at each position of the plurality of arrangement positions, the movement such as choreography can be controlled by using the motion data corresponding to the arrangement position. For example, when the choreography of the character's dance or the like is different according to each position of the plurality of arrangement positions, the character can be operated by the choreography according to each arrangement position.

Further, the reproduction timing information is information for designating, for example, a time after a set time from the reference time as a designated time for the reproduction start timing. Then, the content reproduction unit 102 starts reproduction of the content at this designated time. For example, in the reproduction timing information extracted from the reproduction information display object, a designated time, which is a time after the set time from the reference time, is set as the reproduction start timing. Then, the content reproduction unit 102 compares the current time timed by the timekeeping unit described above with the designated time which is the reproduction start timing, and starts reproducing the content at the timing when the current time reaches the designated time.

Here, the reference time is, for example, the display time or the generation time of the reproduced information display object. For example, the user of the host terminal specifies a set time for determining the playback start timing. Then, for example, when the set time is 20 minutes, the other user of the guest terminal is informed, such as "start after 20 minutes". And since the current time is timed by the metering unit, for example, it is possible to specify what time is the time after the set time from the reference time, and the specified time, which is the time after the set time from the reference time, is started to be played back. Can be set as timing.

Further, the activation unit 109 performs a process for activating the content reproduction unit 102 before the above-mentioned designated time. For example, the terminal device may shift to the sleep mode after the reading unit 104 extracts the content identification information and the reproduction timing information from the captured image of the reproduction information display object. For example, when, for example, 20 minutes is set as the set time as described above, the terminal device may shift to the sleep mode during the set time of 20 minutes. Even in such a case, the activation unit 109 activates the content reproduction unit 102 (terminal device) before the designated time, which is the reproduction start timing, to shift from the sleep mode to the normal operation mode. As a result, when the current time reaches the designated time, the content reproduction unit 102 can appropriately reproduce the content.

Further, as shown in FIG. 2, the terminal device includes a reproduction information generation unit 110 and an output processing unit 112. For example, when the terminal device is the host terminal described with reference to FIG. 1, the reproduction information generation unit 110 and the output processing unit 112 are processed. For example, the terminal device serving as a host terminal operates based on a program for executing the processing of the reproduction information generation unit 110 and the processing of the output processing unit 112, thereby performing the operation as a host. Since a virtual space image including a character image is generated and displayed on the screen of the host terminal, the host terminal also executes the processing of the content reproduction unit 102.

The reproduction information generation unit 110 generates reproduction information. For example, the reproduction information generation unit 110 generates reproduction information including content identification information for designating the content to be reproduced and reproduction timing information for specifying the reproduction start timing of the content. Then, the output processing unit 112 performs a process for outputting a reproduction information display object for displaying the reproduction information. For example, when the reproduced information display object is a two-dimensional code such as a QR code, the output processing unit 112 performs output processing for displaying the two-dimensional code on the display unit 190. When the reproduction information display object is a printed matter on which the reproduction information is printed, the output processing unit 112 performs a process for printing the reproduction information on a print medium and outputting the reproduction information. Alternatively, as will be described later, when the playback information display object is output due to the blinking of the screen of the host terminal, the output processing unit 112 of the host terminal performs an output process for transmitting the reproduction information to the guest terminal by blinking the screen. Do.

It should be noted that it is not always necessary to include both the content identification information and the reproduction timing information in one reproduction information. For example, the reproduction information generation unit 110 may reproduce the first reproduction information including the content identification information and the second reproduction information including the reproduction timing information. That is, the content identification information and the reproduction timing information are separated to generate two reproduction information such as the first and second reproduction information. Then, the output processing unit 112 is for processing for outputting the first reproduction information display object for displaying the first reproduction information and for outputting the second reproduction information display object for displaying the second reproduction information. The process may be performed.

2. Method of this Embodiment Next, the method of this embodiment will be described in detail. In the following, the case where the character is a virtual female idol character will be mainly described as an example, but the idol of the present embodiment is not limited to such a virtual female idol character. For example, the character may be a male idol character, a non-idol singer character, a dancer character, or the like.

2.1 Synchronization of content reproduction In FIG. 3, the content reproduction system is composed of a host terminal TMH and guest terminals TM1, TM2, TM3, and TM4. The content playback system may include at least one guest terminal, and the number of guest terminals is arbitrary.

As shown in FIG. 4, the host terminal TMH generates playback information including playback timing information for designating the playback start timing of the content. Specifically, the host terminal TMH generates playback information including the playback timing information and content identification information for designating the content. As described above, the reproduction information may include at least the reproduction timing information. Then, the host terminal TMH outputs a reproduction information display object that displays the reproduction information. In FIG. 3, a code QC, which is a two-dimensional code, is displayed on the screen of the host terminal TMH as a reproduction information display object. That is, the host terminal TMH converts the content identification information of the contents such as moving images and music and the reproduction information including the reproduction timing information which is the designated time of the reproduction start into two-dimensional code (QR code) and displays it on the screen.

Note that the content identification information and the reproduction timing information may be generated and displayed in two steps without being included in the same code. For example, the first reproduction information including the content identification information is generated, and the first code in which the first reproduction information is encoded is displayed on the screen of the host terminal TMH. Next, the second reproduction information including the reproduction timing information is generated, and the second code in which the second reproduction information is encoded is displayed on the screen of the host terminal TMH.

As shown in FIG. 3, the guest terminals TM1 to TM4 photograph the code QC displayed on the screen of the host terminal TMH by the camera which is the photographing unit 162. Then, the guest terminals TM1 to TM4 extract the reproduction timing information from the captured image. Specifically, the reproduction timing information is extracted by decoding the code QC. More specifically, the content identification information and the reproduction timing information are extracted by decoding the code QC.

The guest terminals TM1 to TM4 may read and extract the content identification information and the reproduction timing information in two steps. For example, when the host terminal TMH displays the first code that encodes the first playback information including the content identification information on the screen as described above, the guest terminals TM1 to TM4 capture the first code. And read it to extract the content identification information. Further, when the host terminal TMH displays a second code that encodes the second playback information including the playback timing information on the screen, the guest terminals TM1 to TM4 photograph and read the second code. , Extract playback timing information.

Further, in FIG. 3, the code QC is used as the reproduction information display object, but the present embodiment is not limited to this. For example, the host terminal TMH may perform printing processing of a printed matter on which reproduction information including content identification information and reproduction timing information is printed. For example, a process for printing the code QC on a print medium is performed. Then, the guest terminals TM1 to TM4 may read the reproduction information of the printed matter by taking a picture with a camera or the like to extract the content identification information and the reproduction timing information.

Alternatively, as shown in FIG. 5, the host terminal TMH may output the reproduction information display object by blinking the screen of the host terminal TMH. For example, the host terminal TMH blinks the screen such that the lighting state is assigned to the first logic level (for example, logic "1") and the turning state is assigned to the second logic level (for example, logic "0"). By doing so, the playback information is encoded and output. Then, the guest terminals TM1 to TM4 read the reproduction information by detecting the blinking of the screen of the host terminal TMH. For example, the guest terminals TM1 to TM4 can read the playback information by photographing the blinking of the screen of the host terminal TMH with a camera or the like. In this way, the playback information can be transmitted from the host terminal TMH to the guest terminals TM1 to TM4 by blinking the screen of the host terminal. It is also possible to carry out a modification in which the host terminal TMH outputs the reproduction information by outputting the sound using the speaker, and the guest terminals TM1 to TM4 use the microphone to collect and analyze the reproduction information.

Then, the guest terminals TM1 to TM4 select one content from a plurality of contents based on the extracted content identification information, and at the playback start timing specified based on the extracted playback timing information, the selected content Start playback. That is, the playback of the MV movie in which the character dances or sings is started. Further, when displaying the MV moving image of the character on the screen of the host terminal TMH, the host terminal TMH also starts playing the selected content at the playback start timing corresponding to the playback timing information.

By doing so, as shown in FIG. 6, the guest terminals TM1 to TM4 and the host terminal TMH can synchronously start the reproduction of the MV video content by the characters CH1 to CH3 and CHH at the reproduction start timing. .. That is, at the designated time, which is the playback start timing, the MV moving images by the characters CH1 to CH3 and CHH start at the same time and are displayed on the screens of the guest terminals TM1 to TM4 and the host terminal TMH.

FIG. 7 is a flowchart illustrating a processing example of the present embodiment. The content playback system of the present embodiment has a host terminal, at least one guest terminal, and content including a moving image generated by shooting a virtual three-dimensional space in which at least one character is arranged by a virtual camera. Reproduce. Specifically, as shown in FIG. 7, the host terminal generates playback information including content identification information for designating the content and playback timing information for designating the playback start timing of the content (step S1). ). Then, the reproduction information display object for displaying the reproduction information is output (step S2). Specifically, as shown in FIG. 3, a two-dimensional code such as a QR code is displayed on the screen of the host terminal as a reproduction information display object.

Then, the guest terminal captures the reproduced information display object and acquires the captured image (step S11). That is, a captured image is acquired by photographing a two-dimensional code which is a reproduction information display object. Then, the content identification information and the reproduction timing information are extracted from the captured image (step S12). For example, by decoding a two-dimensional code, the content identification information and the reproduction timing information included in the reproduction information are extracted. Then, based on the content identification information, one content is selected from a plurality of contents stored in advance (step S13). That is, the content such as a moving image or a musical piece to be played back is selected. Then, at the playback start timing specified based on the playback timing information, playback of the selected content is started (step S14). That is, as shown in FIG. 6, the reproduction of the content of the MV moving image in which the character dances or sings is started. That is, the content including the moving image generated by shooting the virtual three-dimensional space in which the character is arranged by the virtual camera is reproduced. When the operation is reproduced, the music corresponding to the moving image is also output from the guest terminal or the host terminal.

As described above, according to the present embodiment, the host terminal generates the playback information including the content playback information and the playback timing information, and the guest terminal shoots the playback information display object on which the playback information is displayed, so that the playback start timing It becomes possible to transmit the playback timing information for specifying to the guest terminal from the host terminal. Then, at the reproduction start timing specified by the reproduction timing information, the reproduction of the content including the moving image generated by shooting the character in the virtual three-dimensional space with the virtual camera is started. In this way, even if the host terminal and the guest terminal are not connected by communication, the playback timing information that specifies the playback start timing is transmitted from the host terminal to the guest terminal, and the video content is played back at the playback start timing. You will be able to start at the same time. Then, as shown in FIG. 6, by displaying each corresponding character on each screen of the guest terminal and the host terminal, content reproduction that makes it appear as if a plurality of characters dance or sing in synchronization is performed. realizable. In the following, the guest terminal and the host terminal may be simply referred to as a plurality of terminals as appropriate for the sake of simplification of description.

For example, if there is a time lag between the movements of a plurality of characters displayed on a plurality of terminals and the music output from the plurality of terminals, there is a problem that the user feels unnatural.

In this regard, according to the present embodiment, accurate synchronization can be achieved between a plurality of terminals by using the reproduction timing information. Therefore, there is almost no time lag between the movements of the plurality of characters displayed on the plurality of terminals and the music output from the plurality of terminals. Therefore, it becomes possible to realize content reproduction that makes it appear as if a real idol group dances and sings in unison.

Further, for example, as a method of the comparative example of the present embodiment, a method of connecting a plurality of terminals by communication and synchronizing the moving image playback can be considered. However, this method has a problem that synchronized video reproduction cannot be appropriately realized in an environment in which a plurality of terminals cannot communicate or in an environment in which communication is interrupted.

In this respect, in the present embodiment, the playback timing information can be transmitted from the host terminal to the guest terminal to synchronize the video playback without connecting a plurality of terminals via communication, so that the above problems can be solved. it can.

Further, when a plurality of users gather to play the content as shown in FIG. 6, there is a problem that the moving image playback is not started and must wait until the plurality of users gather. For a small group, the waiting time may be short, but for a large group, the waiting time is long because the reception time is set to some extent and the number of people is gathered or the group is not created until the planned number of people is reached. There is a problem of becoming. In addition, if the reception time is set, access will be concentrated, so it is necessary to design a system that can withstand the load.

In this regard, according to the present embodiment, a given set time elapses after the host terminal displays the code (in a broad sense, the playback information display object) and the guest terminal reads the code as shown in FIG. Playback of the video content can be started at the playback start timing. Therefore, after the code is displayed or read, the user waits until the designated time, which is the playback start timing, and arranges the terminals at the designated time as shown in FIG. 6, so that a large number of people can play the video. You will be able to enjoy.

In addition, the screen of terminals such as smartphones is vertically long. In this case, in the present embodiment, as shown in FIG. 6, a moving image in which a character stands along the long side direction of the vertically long screen of the terminal is played, and these a plurality of terminals are displayed on the short side of the screen. Arrange them side by side in the direction along. This makes it possible to play back a moving image in which five characters appear to be dancing side by side on the stage, and it is possible to realize a content playback system that has never existed before.

Next, the character motion settings and virtual camera settings will be described in detail. In the present embodiment, for example, a character selected by the user as at least one character is arranged in the virtual three-dimensional space. For example, as shown in FIG. 8A, each user of a plurality of users selects his / her favorite character on his / her terminal. As a result, the favorite character of the user of each terminal is displayed on each terminal of the plurality of terminals shown in FIG. In this case, in the present embodiment, the motion of the character is determined and the position of the virtual camera is set according to the content specified by the content identification information and the position of the character selected by the user. For example, as shown in FIG. 8B, it is possible to associate the motion ID of the character with the position and operation settings of the virtual camera with respect to the content ID and the character position. In this way, it is possible to select an appropriate motion and set the position of the virtual camera according to the designated content and the position of the character selected by the user.

As an example, each user of the plurality of users of the plurality of terminals in FIG. 6 selects his / her favorite character. Then, on the user's terminal, only one favorite character selected by the user is placed at the center position of the stage. That is, the character position in FIG. 8B is set to the center position of the stage. For example, the user's favorite character is arranged as the center character CHA in FIG. Then, the virtual camera is set so that the character CHA arranged in the center is projected in the substantially center of the screen. That is, the position of the virtual camera is set to the position of the virtual camera VCA of FIG. 9, and is arranged at a position in front of the character CHA arranged at the center. In this way, the position of the virtual camera is determined by the position of the character. Then, as the motion of the character, the motion for the character at the center position corresponding to the music that is the content is selected. For example, in FIG. 9, motion reproduction of the character CHA is performed using the motion data MTA for the character CHA at the center position. That is, the motion (motion ID) is also determined by the content (content ID) and the character position. Further, even when the camera work of the virtual camera is performed according to the movement of the character, the camera work which is the operation of the virtual camera is the camera work corresponding to the motion for the character at the center position. That is, the operation of the virtual camera is also determined according to the content and the character position.

Further, in the present embodiment, a plurality of characters may be arranged and set in the virtual three-dimensional space based on the input information of the user, and the position of the virtual camera may be set according to the arranged plurality of characters. .. Specifically, as shown in FIG. 10, based on the input information of the user, the arrangement of characters other than the user's favorite character and the arrangement position of the favorite character (in a broad sense, the user's selected character) are changed. In this way, a plurality of characters can be arranged by reflecting the input information of the user, and the virtual camera can be set at an appropriate position according to the arranged plurality of characters.

For example, in the present embodiment, basically, the favorite character selected by the user is placed at the center position of the stage. In this case, a plurality of characters other than the user's favorite character can be arranged on the stage in the virtual three-dimensional space based on the user's input information. For example, in FIG. 10, when the user's favorite character is arranged as the center character CHA, the characters CHB, CHC, CHD, and CHE other than the center position can also be arranged based on the user's input information. To do. The user can also change the arrangement of his / her favorite character, for example, the favorite character can be arranged at a position other than the center position. For example, the user can also place the favorite character as the character CHB next to the center position.

Further, the user can set which character among the plurality of characters is to be photographed by the virtual camera. For example, when the user's favorite character is placed in the center position as the character CHA of FIG. 10, the position of the virtual camera is set to the position of the virtual camera VCA of FIG. 10 so that the favorite character is reflected in the center. Also, when the user places the character CHB next to the center position as a character in the favorite, the position of the normal virtual camera is the position of the virtual camera VCA, but the virtual camera is located in front of the favorite character. , The position of the virtual camera may be changed to the position of the virtual camera VCB. Since the position of the virtual camera can be controlled by the virtual camera control unit 108, such a position of the virtual camera can be set. In this way, the user can dare to select a position other than the center position and place the virtual camera in front of the favorite character. For example, in a real-world idol corresponding to a virtual idol group, the user's favorite idol position may not be the center position. In such a case, the user's request can be met by setting the character position and the virtual camera position as described above.

Further, in the present embodiment, when a plurality of characters are arranged in a virtual three-dimensional space, the motion of each character of the plurality of characters is determined according to the arrangement of the plurality of characters. For example, when a plurality of characters are arranged as shown in FIG. 10, the motion of the characters changes depending on the position corresponding to the music. For example, the character CHA at the center position uses the motion data MTA to reproduce the motion, and the character CHB next to it uses the motion data MTB to reproduce the motion. The same applies to other characters. As described above, in the present embodiment, actions such as motion of each character are determined according to the arrangement positions of the plurality of characters. In this way, when a plurality of characters are arranged in the virtual three-dimensional space, it is possible to select an appropriate motion according to the arrangement position of the characters and operate the characters. When there is only one character arranged in the virtual three-dimensional space, the user may be able to select the action of the character.

Further, in the present embodiment, the reproduction timing information is information that specifies the time after the set time from the reference time as the designated time of the reproduction start timing, and the reproduction of the content is started at this designated time.

For example, in FIG. 11A, the display "Starts in 10 minutes" is displayed on the screen of the host terminal TMH. For example, when the time when the code QC is generated and displayed is set as the reference time, as shown in FIG. 11B, the time after the set time STM = 10 minutes from the reference time TREF is the playback start timing. It is set as the specified time TSP. For example, if the time when the display of FIG. 11A is displayed is 6:00 pm, the designated time TSP is 6:10 pm, which is 10 minutes after 6:00 pm, which is the reference time TREF. That is, in the code QC, the reproduction timing information in which the designated time TSP is 6:10 pm is coded. Then, the content of FIG. 6 is reproduced with the reproduction start timing of 6:10 pm, which is the designated time TSP. If the time when the display in FIG. 11A is displayed is 7:30 pm, the reference time TREF is 7:30 pm, and the set time STM = 10 minutes later, 7:40 pm is designated. It becomes the time TSP. That is, in the code QC, the reproduction timing information in which the designated time TSP is 7:40 pm is coded. Then, the content of FIG. 6 is played back with the designated time TSP of 7:40 pm as the playback start timing.

By doing so, each user of the plurality of guest terminals captures the code QC of FIG. 11 (A), so that the time after the set time from the reference time, which is the display time of FIG. 11 (A), is reproduced. It will be set as the specified time of the start timing. Then, the user can grasp that the content playback is started after 10 minutes, which is the set time, by looking at the display as shown in FIG. 11A, so that the convenience of the user can be improved.

Each terminal updates the current time on the terminal based on the time information acquired from the server via the network. For example, at the time of login, the current time, which is the time in the application program (in the game), is adjusted based on the time on the server. As a result, the time difference between a plurality of terminals can be minimized. Therefore, the designated time for content playback set by the method described with reference to FIG. 11 (A) also minimizes the time lag between a plurality of terminals, and the lag in character movement during content playback and the music. It is possible to prevent the deviation of the performance timing of.

Further, in the present embodiment, the content reproduction unit 102 is activated before the designated time. For example, in FIG. 11B, the content reproduction unit 102 is automatically started at the timing TI before the designated time TSP (for example, the timing immediately before the designated time TSP). By doing so, even when the terminal shifts to the sleep mode after the reference time TREF, the content reproduction unit 102 is activated at the timing TI before the specified time TSP, and the sleep mode is changed to the normal operation mode. Can be migrated to. As a result, when the current time reaches the designated time TSP, the content reproduction unit 102 can properly start the reproduction of the content.

2.2 Details of the content playback system Next, a detailed example of the content playback system of the present embodiment will be described. FIG. 12 is a flowchart illustrating a detailed processing example of the content reproduction system.

In FIG. 12, the content reproduction system of the present embodiment includes a host terminal, a first guest terminal, and a second guest terminal. Then, the first guest terminal generates setting information including the information of the first character selected by the first user of the first guest terminal as the first user information (step S21). Then, the first reproduction information display object for displaying the first user information is output (step S22). Taking FIG. 14A as an example, the guest terminal TM1 which is the first guest terminal displays the two-dimensional code QC1 on the screen of the guest terminal TM1 as the first reproduction information display object.

The second guest terminal generates setting information including the information of the second character selected by the second user of the second guest terminal as the second user information (step S31). Then, a second playback information display object that displays the second user information is output (step S32). Taking FIG. 14A as an example, the guest terminal TM2, which is the second guest terminal, displays the two-dimensional code QC2 on the screen of the guest terminal TM2 as the second reproduction information display object.

Next, the host terminal simultaneously shoots the first and second reproduction information display objects and acquires the shot image (step S41). For example, a captured image IM as shown in FIG. 14 (A) is acquired. In this captured image IM, a code QC1 which is a first reproduction information display object and a code QC2 which is a second reproduction information display object are displayed. Then, the host terminal determines the placement positions of the first and second characters in the virtual three-dimensional space based on the acquired captured image (step S42). Then, the reproduction information including the character arrangement information indicating the arrangement position of the first and second characters is generated (step S43). This reproduction information includes character arrangement information, content identification information and reproduction timing information described with reference to FIG. Then, the host terminal outputs a reproduction information display object that displays the reproduction information (step S44). Specifically, as shown in FIG. 3, a two-dimensional code QC is displayed on the screen of the host terminal TMH as a reproduction information display object. The code QC, which is the reproduction information display object, includes content identification information, reproduction timing information, and character arrangement information.

Next, the first guest terminal captures a reproduction information display object output by the host terminal and acquires a captured image (step S23). Then, the character arrangement information is extracted from the captured image (step S24). Then, based on the character arrangement information, the first and second characters are arranged in the virtual three-dimensional space, and the content is reproduced (step S25). Although omitted here, in step S24, the first guest terminal extracts the content identification information and the reproduction timing information described in FIG. 4 in addition to the character arrangement information. Then, in step S25, the content is reproduced using the content identification information and the reproduction timing information.

Similarly, the second guest terminal captures the reproduction information display object output by the host terminal TMH and acquires the captured image (step S33). Then, the character arrangement information is extracted from the captured image (step S34). Then, based on the character arrangement information, the first and second characters are arranged in the virtual three-dimensional space, and the content is reproduced (step S35). Although omitted here, in step S34, the second guest terminal extracts the content identification information and the reproduction timing information in addition to the character arrangement information. Then, in step S35, the content is reproduced using the content identification information and the reproduction timing information.

According to the processing example shown in FIG. 12, the host terminal sets setting information including information about the first and second characters selected by the first and second users of the first and second guest terminals. It will be possible to acquire from the first and second guest terminals. For example, the first and second users can acquire information about which character is selected as the favorite character. As a result, the host terminal can arrange the first and second characters selected by the first and second users in the virtual three-dimensional space. Therefore, when arranging a plurality of characters in the virtual three-dimensional space, the characters desired by the first and second users of the first and second guest terminals can be arranged.

Further, the host terminal acquires a photographed image of the first reproduction information display object and the second reproduction information display object as shown in step S41 of FIG. 12, and as shown in step S42, the photographed image is used. Based on the positional relationship between the first reproduction information display object and the second reproduction information display object, the placement positions of the first and second characters in the virtual three-dimensional space are determined, and as shown in step S43, the characters Generate placement information. In this way, the first and second characters can be arranged in the virtual three-dimensional space at positions corresponding to the positional relationship between the first reproduction information display object and the second reproduction information display object in the captured image. Will be.

For example, in FIG. 14A, the first and first codes QC1 which are the first reproduction information display objects and the code QC2 which is the second reproduction information display object are first and first depending on the positional relationship in the captured image IM. The first and second characters of two users can be placed. As an example, in the case of the positional relationship of the codes QC1 and QC2 as shown in FIG. 14 (A), the characters CH1 and CH2 are arranged as shown in FIG. 14 (B). Here, the character CH1 is the first character, and the character CH2 is the second character. For example, in FIG. 14B, the character CHH of the host terminal is arranged at, for example, the center position of the stage. On the other hand, as shown in FIG. 14A, the code QC1 of the guest terminal TM1 is located on the left side of the captured image IM, and the code QC2 of the guest terminal TM2 is located on the right side of the captured image IM. Therefore, in this case, the character CH1 of the first user of the guest terminal TM1 is arranged to the left of the center character CHH so as to correspond to the arrangement of the codes QC1 and QC2, and the guest is arranged. The character CH2 of the second user of the terminal TM2 is arranged on the right side when viewed from the front. By doing so, the characters CH1 and CH2, which are the first and second characters, can be arranged in the virtual three-dimensional space by utilizing the positional relationship of the first and second reproduction information display objects in the captured image. become.

Further, the host terminal is arranged in a virtual three-dimensional space with respect to the first and second characters selected by the first and second users of the first and second guest terminals and the characters selected by the user of the host terminal. Can be determined. For example, in FIG. 14B, the user of the host terminal places his / her character CHH in the center position. Alternatively, the user of the host terminal can place his / her character CHH in a position next to the center. In this way, in addition to the characters selected by the users of the first and second guest terminals, the characters selected by the users of the host terminal can also be arranged in the virtual three-dimensional space.

Further, in steps S21 and S31 of FIG. 12, the setting information of the first and second characters generated by the first and second guest terminals is the first and first characters selected by the first and second users as favorite characters. Information that specifies the two characters can be included, and information that specifies the positions of the first and second characters can also be included. If the positions of the first and second characters are determined in the content of the music, the positions are also determined when the favorite character is selected. Further, the positions of the first and second characters may be determined by the user of the host terminal or may be automatically determined according to a predetermined algorithm. For example, the host terminal may accept a desired position from the first and second guest terminals and determine the position of each character. In this case, the first choice, the second choice, etc. regarding the position may be accepted from the first and second guest terminals.

Further, the in-camera on the screen side of the terminal may be activated and each reproduction information display object (code) may be displayed on the screen so that the shooting in step S41 and the shooting in steps S23 and S33 can be performed at the same time. For example, the host terminal takes a picture of step S41 with the in-camera on the screen side while displaying the reproduction information display object (code) on the screen as in step S44, and the first and second guest terminals of the first and second guest terminals, The photographed image of the second reproduction information display object is acquired. The first and second guest terminals take pictures of steps S23 and S33 with the in-camera on the screen side while displaying the first and second reproduction information display objects (codes) as in steps S22 and S32. , Playback information Acquire the captured image of the displayed object. By doing so, each terminal can acquire each photographed image without dividing the photographing into two times.

Further, also in FIG. 14B, the position of the virtual camera can be set to an arbitrary position as shown in VCH, VC1 and VC2. Further, in FIGS. 14A and 14B, the arrangement of the characters CH1 and CH2 is set by using the positional relationship of the codes QC1 and QC2 displayed on the screens of the guest terminals TM1 and TM2. On the other hand, in FIGS. 15 (A) and 15 (B), the character CH1 and the characters CH1 and The arrangement of CH2, CH3, and CH4 is set. As described above, according to the present embodiment, even when the number of characters to be arranged increases, it can be easily dealt with.

FIG. 13 is a flowchart illustrating another detailed processing example of the present embodiment. FIG. 13 is different from FIG. 12 in steps S41a and S41b and steps S42 and S43 of FIG. That is, in FIG. 13, the host terminal captures the first reproduction information display object output by the first guest terminal and acquires the first captured image (step S41a). Further, the second reproduction information display object output by the second guest terminal is photographed to acquire the second photographed image (step S41b). Then, the placement positions of the first and second characters in the virtual three-dimensional space are determined based on the acquisition order of the first and second captured images (step S42). Then, the reproduction information including the character arrangement information indicating the arrangement position of the first and second characters is generated (step S43).

For example, in FIG. 16, the captured image IM1 in which the code QC1 which is the first reproduction information display object is photographed is acquired, and then the photographed image IM2 in which the code QC2 which is the second reproduction information display object is photographed is acquired. There is. Further, the captured image IM3 obtained by photographing the code QC3 which is the third reproduction information display object is acquired, and the photographed image IM4 obtained by photographing the code QC4 which is the fourth reproduction information display object is acquired. Here, the captured images IM1, IM2, IM3, and IM4 are the first, second, third, and fourth captured images, respectively. When the captured images IM1, IM2, IM3, and IM4 are acquired in such an acquisition order, the characters CH1, CH2, CH3, and CH4 are arranged and set, for example, as shown in FIG. 15B. In this way, the characters CH1, CH2, CH3, and CH4 can be arranged in the virtual three-dimensional space by using the acquisition order of the captured images IM1, IM2, IM3, and IM4.

FIG. 17 is an explanatory diagram of a method of displaying a reproduction information display object on a large monitor. In FIG. 17, for example, a two-dimensional code, which is a reproduction information display object, is displayed on a large monitor installed in a building. Then, many users shoot the code of a large monitor with a smartphone, which is a guest terminal. Then, after a predetermined set time, playback of the MV video content is started at the same timing on the guest terminals of these a large number of users. In this case, the host terminal is a server device prepared by the management side. For example, a large monitor as shown in FIG. 17 is installed at a live venue, and a two-dimensional code, which is a reproduction information display object, is displayed on the monitor. The audience then shoots the monitor code with their smartphone. As a result, after a predetermined set time, for example, MV video playback that synchronizes with a live performance can be started at the same timing on the smartphones of a large number of spectators, and a content playback system with a high promotion effect can be realized. It will be possible.

Although the present embodiment has been described in detail as described above, those skilled in the art will easily understand that many modifications that do not substantially deviate from the novel matters and effects of the present disclosure are possible. Therefore, all such modifications are within the scope of the present disclosure. For example, a term (code, etc.) described at least once in a specification or drawing together with a different term (reproduced information display, etc.) in a broader sense or synonymously means the different term in any part of the specification or drawing. Can be replaced. Further, the content reproduction processing, reading processing, virtual space setting processing, virtual camera control processing, reproduction information generation processing, output processing, display processing, sound processing, etc. are not limited to those described in the present embodiment, and are equivalent to these. Methods, processes and configurations are also included within the scope of this disclosure.

TMH ... host terminal, TM1-TMn ... guest terminal, QC ... code,
CH1 to CH4, CHH, CHA to CHE ... Character,
VC1 to VC4, VCH, VCA to VCE ... Virtual camera,
MT1-MT4, MTH, MTA-MTE ... Motion data,
TREF ... Reference time, STM ... Set time, TSP ... Specified time, T1 ... Timing,
100 ... Processing unit, 102 ... Content playback unit, 104 ... Reading unit,
106 ... virtual space setting unit, 108 ... virtual camera control unit, 109 ... activation unit,
110 ... Reproduction information generation unit, 112 ... Output processing unit, 120 ... Display processing unit,
130 ... Sound processing unit, 160 ... Operation unit, 162 ... Shooting unit, 170 ... Storage unit,
172 ... Content information storage unit, 174 ... Reproduction information storage unit,
176 ... Virtual space information storage unit, 180 ... Information storage medium, 190 ... Display unit,
192 ... Sound output section, 194 ... I / F section,
195 ... Portable information storage medium, 196 ... Communication unit,

Claims (15)

A content playback unit that reproduces content including moving images generated by shooting a virtual three-dimensional space in which at least one character is arranged with a virtual camera, and a content playback unit.
Displaying playback information including playback timing information When a shot image of a playback information display object is acquired, the playback timing information for specifying the playback start timing of the content is extracted from the shot image. As a department
Make your computer work
The content playback unit
A program characterized in that playback of the content is started at the playback start timing specified based on the playback timing information. In claim 1,
As the at least one character, a character selected by the user is arranged in the virtual three-dimensional space.
The playback information includes content identification information for designating the content to be played by the content playback unit.
The content playback unit
A program characterized in that the motion of the character is determined and the position of the virtual camera is set according to the content specified by the content identification information and the position of the character selected by the user. In claim 1,
A virtual space setting unit that arranges and sets a plurality of characters as the at least one character in the virtual three-dimensional space based on user input information.
As a virtual camera control unit that sets the position of the virtual camera,
A program characterized by making a computer work. In any one of claims 1 to 3,
As the at least one character, a plurality of characters are arranged in the virtual three-dimensional space.
The content playback unit
A program characterized in that the motion of each character of the plurality of characters is determined according to the arrangement of the plurality of characters. In any of claims 1 to 4,
The reproduction timing information is information that specifies a time after a set time from the reference time as a designated time of the reproduction start timing.
The content playback unit
A program characterized in that playback of the content is started at the designated time. In claim 5,
As an activation unit that activates the content playback unit before the designated time.
A program characterized by making a computer work. In any of claims 1 to 6,
A reproduction information generation unit that generates the reproduction information,
As an output processing unit that performs processing for outputting the reproduction information display object for displaying the reproduction information
A program characterized by making a computer work. A content playback unit that reproduces content including moving images generated by shooting a virtual three-dimensional space in which at least one character is arranged with a virtual camera, and a content playback unit.
Displaying playback information including playback timing information When a shot image of a playback information display object is acquired, the playback timing information for specifying the playback start timing of the content is extracted from the shot image. Department and
Including
The content playback unit
A terminal device characterized in that playback of the content is started at the playback start timing designated based on the playback timing information. It is a content playback system for playing back content including moving images generated by shooting a virtual three-dimensional space in which at least one character is arranged with a host terminal and at least one guest terminal with a virtual camera. hand,
The host terminal
Generates playback information including playback timing information for designating the playback start timing of the content, and outputs a playback information display object that displays the playback information.
The at least one guest terminal is
When a photographed image obtained by photographing the reproduction information display object is acquired, the reproduction timing information is extracted from the photographed image.
A content reproduction system characterized in that reproduction of the content is started at the reproduction start timing designated based on the reproduction timing information. In claim 9.
The at least one guest terminal includes a first guest terminal and a second guest terminal.
The first guest terminal generates setting information including information of the first character selected by the first user of the first guest terminal as the first user information, and generates the first user information. Output the first playback information display object to be displayed,
The second guest terminal generates setting information including the information of the second character selected by the second user of the second guest terminal as the second user information, and generates the second user information. Output the second playback information display object to be displayed,
The host terminal
When the photographed image obtained by photographing the first reproduction information display object and the second reproduction information display object is acquired, the first and second characters in the virtual three-dimensional space are obtained based on the photographed image. Is determined, the reproduction information including the character arrangement information indicating the arrangement position of the first and second characters is generated, and the reproduction information display object for displaying the reproduction information is output.
Each guest terminal of the first and second guest terminals is
When the photographed image obtained by photographing the reproduction information display object is acquired, the character arrangement information is extracted from the photographed image.
A content reproduction system characterized in that the first and second characters are arranged in the virtual three-dimensional space based on the character arrangement information to reproduce the contents. In claim 10,
The host terminal
When the photographed image obtained by photographing the first reproduction information display object and the second reproduction information display object is acquired, the first reproduction information display object and the second reproduction information display object in the photographed image are obtained. A content reproduction system characterized in that the arrangement positions of the first and second characters in the virtual three-dimensional space are determined based on the positional relationship of the characters, and the character arrangement information is generated. In claim 10,
The host terminal
When the first photographed image obtained by photographing the first reproduction information display object and the second photographed image obtained by photographing the second reproduction information display object are acquired, the first and second photographed images A content reproduction system characterized in that the arrangement positions of the first and second characters in the virtual three-dimensional space are determined based on the acquisition order, and the character arrangement information is generated. In any of claims 10 to 12,
The host terminal
Arrangement of the first and second characters selected by the first and second users of the first and second guest terminals and the characters selected by the user of the host terminal in the virtual three-dimensional space. A content playback system characterized by determining. In any of claims 9 to 13,
The host terminal
Generates the reproduction timing information that specifies the time after the set time from the reference time as the designated time of the reproduction start timing.
The at least one guest terminal is
A content playback system characterized in that playback of the content is started at the designated time. In any of claims 9 to 14,
The host terminal
A content reproduction system characterized in that the reproduction information display object is output by blinking the screen of the host terminal.

Images