WO2022244364A1 - Dispositif de traitement d'informations, procédé de traitement d'informations, et programme - Google Patents

Dispositif de traitement d'informations, procédé de traitement d'informations, et programme Download PDF

Info

Publication number
WO2022244364A1
WO2022244364A1 PCT/JP2022/007333 JP2022007333W WO2022244364A1 WO 2022244364 A1 WO2022244364 A1 WO 2022244364A1 JP 2022007333 W JP2022007333 W JP 2022007333W WO 2022244364 A1 WO2022244364 A1 WO 2022244364A1
Authority
WO
WIPO (PCT)
Prior art keywords
content
information
delay
reaction
terminal
Prior art date
Application number
PCT/JP2022/007333
Other languages
English (en)
Japanese (ja)
Inventor
広 岩瀬
賢次 杉原
茜 近藤
文彦 飯田
悠 朽木
Original Assignee
ソニーグループ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ソニーグループ株式会社 filed Critical ソニーグループ株式会社
Priority to DE112022002670.7T priority Critical patent/DE112022002670T5/de
Priority to CN202280029021.9A priority patent/CN117178558A/zh
Publication of WO2022244364A1 publication Critical patent/WO2022244364A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2402—Monitoring of the downstream path of the transmission network, e.g. bandwidth available
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2408—Monitoring of the upstream path of the transmission network, e.g. client requests
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/242—Synchronization processes, e.g. processing of PCR [Program Clock References]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43074—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8547—Content authoring involving timestamps for synchronizing content

Definitions

  • This technology relates to an information processing device, an information processing method, and a program, and particularly to technology for reproducing content.
  • the information processing device receives content from the content distribution server, receives user information that captures an image of the user viewing the content from the user terminal, and generates an image (user image) based on the content and user information.
  • a display unit has been proposed (for example, Patent Document 1).
  • the information processing device it is possible to give the user a sense of unity as if they were viewing the content together with other users viewing the same content at different locations.
  • the purpose of this technology is to suppress the reduction in the sense of unity with other users.
  • An information processing apparatus acquires information about a first communication delay from a content distribution server to a target terminal and a second communication delay from the content distribution server to a target terminal via an edge terminal, Based on the information about the first communication delay and the second communication delay, the content transmitted from the content distribution server and the reproduction of reaction information transmitted from the edge terminal are controlled at the target terminal. It has a control unit. This enables the information processing device to synchronize and reproduce the content and the reaction information on the target terminal.
  • FIG. 1 is an explanatory diagram showing the configuration of a content delivery system according to an embodiment of the present technology
  • FIG. It is the figure which showed the functional structure of the content distribution server. It is the figure which showed the functional structure of a management server.
  • 3 is a diagram showing a functional configuration of a user terminal;
  • FIG. 4 is a diagram for explaining Example 1; 7 is a flow chart showing automatic matching processing of the management server in Embodiment 1.
  • FIG. 4 is a flow chart showing user terminal selection processing according to the first embodiment.
  • FIG. 11 is a diagram for explaining Example 2; 10 is a flow chart showing automatic matching processing of a target terminal in Example 2.
  • FIG. 10 is a flow chart showing user terminal selection processing according to the second embodiment.
  • FIG. 11 is a diagram for explaining Example 3; 14 is a flow chart showing friend matching processing of a management server in Example 3.
  • FIG. FIG. 11 is a diagram for explaining Example 4;
  • FIG. 16 is a flow chart showing friend matching processing of a management server in Example 4.
  • FIG. It is the flowchart which showed the 1st delay provision process. It is the flowchart which showed the 2nd delay provision process.
  • FIG. 4 is a diagram for explaining a replay mode;
  • FIG. FIG. 2 is a diagram showing an example of arrangement of display devices and audio output devices in a user terminal system;
  • FIG. 10 is a diagram illustrating an example of reproducing reaction information; It is a figure showing hardware constitutions.
  • FIG. 1 is an explanatory diagram showing the configuration of a content distribution system 1 according to an embodiment of the present technology.
  • a content distribution system 1 according to an embodiment of the present technology includes a content distribution server system 2, a management server 3, and a plurality of user terminal systems 4.
  • the content distribution server system 2 , the management server 3 and the user terminal system 4 are connected to a network 5 such as the Internet, and can communicate with each other via the network 5 .
  • the content distribution server system 2 is installed at a sports venue, a concert venue, or the like, and distributes video and audio of sports or concerts as content to a plurality of user terminal systems 4 in real time.
  • voice are included in a content
  • the content may include either video or audio only, or may include other data such as still images, for example.
  • the content distribution server system 2 distributes the content in real time, but it may also distribute the content recorded and recorded in advance.
  • the content distribution server system 2 includes a content distribution server 10, an imaging device 11, a sound collection device 12, a display device 13, and an audio output device 14.
  • the content delivery server 10 is a computer having a CPU (Central Processing Unit), RAM (Random Access Memory) and ROM (Read Only Memory), and controls the content delivery server system 2 in an integrated manner.
  • CPU Central Processing Unit
  • RAM Random Access Memory
  • ROM Read Only Memory
  • the imaging device 11 is a video camera that captures images, and includes an imaging device 11a and an imaging device 11b.
  • the imaging device 11a captures images of sports or concerts (including performers Cpf who perform these).
  • the imaging device 11b captures an image of a spectator Cau viewing content such as sports or a concert on site.
  • the sound collector 12 is a microphone and includes a sound collector 12a and a sound collector 12b.
  • the sound collection device 12a collects sounds of sports or concerts (including performers Cpf who perform these).
  • the sound collecting device 12b collects the sound uttered by the audience Cau.
  • the content distribution server 10 distributes the video imaged by the imaging device 11a and the sound collected by the sound collector 12a to the user terminal system 4 via the network 5 as content.
  • the content distribution server 10 distributes the video imaged by the imaging device 11b and the sound collected by the sound collector 12b to the user terminal system 4 via the network 5 as spectator information.
  • the spectator information is information related to the spectators Cau who view the content such as sports or concerts on-site, and includes, for example, images showing the spectators Cau and voices uttered by the spectators Cau.
  • the display device 13 is a liquid crystal display, an OLED (Organic Light Emitting Diode) display, a projector device, or the like, and displays images to the performer Cpf and the audience Cau.
  • the audio output device 14 is a speaker and outputs audio to the performer Cpf and the audience Cau.
  • the management server 3 is a computer having a CPU, RAM, and ROM, and controls the reproduction of content and reaction information (to be described later) on the user terminal system 4 based on delays (communication delays) that occur in the content distribution system 1 .
  • the user terminal system 4 receives and reproduces content distributed from the content distribution server system 2 .
  • the user terminal system 4 includes a user terminal 20 , a display device 21 , an audio output device 22 , an imaging device 23 and a sound collector 24 .
  • the user terminal 20 is a computer equipped with a CPU, RAM and ROM, and controls the user terminal system 4 in an integrated manner.
  • the user terminal 20 receives content via the network 5, for example.
  • the display device 21 is a liquid crystal display, an OLED display, a projector device, or the like, and displays images included in the received content.
  • the audio output device 22 is a speaker and outputs audio included in the received content.
  • the display of video by the display device 21 and the output of audio by the audio output device 22 may be collectively referred to as "playback".
  • the imaging device 23 is a video camera that captures an image of the user's appearance (whole body, upper body, or face).
  • the sound collecting device 24 is a microphone, and collects the voice uttered by the user.
  • the user terminal 20 uses the video imaged by the imaging device 23 and the sound collected by the sound collector 24 as reaction information to transmit the content distribution server system 2 (content distribution server 10) via the network 5, It can be transmitted to the management server 3 and another user terminal system 4 (user terminal 20).
  • the reaction information is information relating to the reaction of the user viewing the content at a remote location (the user terminal system 4), for example, the user's appearance and voice.
  • a user terminal system 4 is provided for each user or home in a different location.
  • the user terminal system 4 for an arbitrary target user T is denoted as target user terminal system 4a
  • the user terminal system 4 for other users R is denoted as other user terminal system 4b.
  • the target user terminal system 4a can be the other user terminal system 4b
  • the other user terminal system 4b can be the target user terminal system 4a.
  • the user terminal 20 of the target user terminal system 4a is referred to as the target terminal 20a
  • the user terminal 20 of the other user terminal system 4b is referred to as the edge terminal 20b.
  • content is distributed from the content distribution server 10 to a plurality of user terminals 20 at the same time.
  • reaction information of users viewing the content distributed from the content distribution server 10 is transmitted to other user terminals 20 .
  • the user terminal 20 reproduces the content distributed from the content distribution server 10 and also reproduces the reaction information transmitted from the other user terminal 20 .
  • the user terminal system 4 will be able to reproduce the reaction of the other user along with the content, making it possible to enjoy the content with a sense of unity with the other user.
  • the content delivery system 1 a communication delay occurs in the network 5 or the like when sending and receiving content and reaction information. Therefore, in the user terminal system 4, the content to be reproduced and the reaction information may not be synchronized, causing a time lag.
  • the synchronization of the content and the reaction information means that the reaction information reproduced on the user terminal system 4 was photographed and collected when the content at the same time as the content reproduced on the user terminal system 4 was viewed. Say that it is reaction information. Note that the same time includes an error within a preset specified value. Therefore, in the user terminal system 4, if the content and the reaction information are synchronized, the reaction of the other user viewing the content at the same time as the content being reproduced will be reproduced simultaneously with the content.
  • the user terminal system 4 reproduces the reaction information so that the user can understand the contents of the content. There is a risk that it will be spoiled (so-called spoilers).
  • spoilers Specifically, in the case of sports content, it is possible that a scene pleasing to other users may be reproduced before a scene in which a score is scored is reproduced. In such a case, the user is made to know that the score has been scored before the content is reproduced.
  • the user terminal system 4 may lose a sense of unity with other users.
  • a scene pleasing to other users may be played back after a period of time has passed since the scene in which the score was scored.
  • a scene that pleases other users will be played after a time interval from the scene that scored, and the user cannot enjoy the pleasure together with other users, and a sense of unity cannot be obtained.
  • FIG. 2 is a diagram showing the functional configuration of the content distribution server 10.
  • the content distribution server 10 includes a network communication unit 31, a content audience information input unit 32, a timestamp addition unit 33, a content audience information distribution unit 34, a content audience information storage unit 35, and a reaction information reception unit 36. and functions as a video/audio output unit 37 .
  • the network communication unit 31 establishes communication between the management server 3 and the user terminal 20 via the network 5, and enables data transmission and reception between the management server 3 and the user terminal 20.
  • the content audience information input unit 32 acquires the video imaged by the imaging device 11a and the sound collected by the sound collector 12a as content. Also, the content audience information input unit 32 acquires the video imaged by the imaging device 11b and the sound collected by the sound collector 12b as audience information.
  • the time stamp adding unit 33 divides the content acquired by the content audience information input unit 32 into packets, which are distribution units, and adds a time stamp indicating the current time to each packet. Further, the time stamp adding unit 33 divides the audience information acquired by the content audience information input unit 32 into packets, which are distribution units, and adds a time stamp indicating the current time to each packet.
  • the content audience information distribution unit 34 transmits (distributes) the packet-by-packet content to which the time stamp is added by the time stamp addition unit 33 to the user terminal system 4 via the network communication unit 31 and the network 5 .
  • the content audience information distribution unit 34 also transmits (distributes) the audience information for each packet to which the time stamp is added by the time stamp addition unit 33 to the user terminal system 4 via the network communication unit 31 and the network 5 .
  • the content audience information storage unit 35 stores the content for each packet to which the time stamp is added by the time stamp addition unit 33 as needed. Also, the content audience information storage unit 35 stores the audience information for each packet to which the time stamp is added by the time stamp adding unit 33 as needed.
  • the reaction information receiving unit 36 receives reaction information transmitted from the management server 3 or the user terminal system 4 via the network 5 and the network communication unit 31.
  • the video/audio output unit 37 reads the content stored in the content audience information storage unit 35 during the replay mode, which will be described later in detail. Then, the video/audio output unit 37 causes the display device 13 to display the video included in the read content, and causes the audio output device 14 to output the audio included in the content.
  • the video/audio output unit 37 causes the display device 13 to display the video included in the reaction information together with the video included in the content.
  • the video/audio output unit 37 causes the audio output device 14 to output the audio included in the reaction information together with the audio included in the content.
  • FIG. 3 is a diagram showing the functional configuration of the management server 3.
  • the management server 3 includes a network communication unit 41, a delay information reception unit 42, a delay synchronization control unit 43, a reaction information communication instruction unit 44, a reaction information reception unit 45, a reaction information storage unit 46, a time stamp It functions as a receiver 47 and a reaction information distributor 48 .
  • the network communication unit 41 establishes communication between the content distribution server 10 and the user terminal 20 via the network 5, and enables data transmission and reception between the content distribution server 10 and the user terminal 20.
  • the delay information receiving unit 42 receives information about communication delays from the plurality of user terminals 20 via the network communication unit 41 and the network 5 respectively.
  • Information related to communication delay is, for example, a delay time or a time stamp, which will be described later.
  • the delay synchronization control unit 43 controls the reproduction of the content and reaction information on the target terminal 20a based on the delay information from the plurality of user terminals 20 received by the delay information receiving unit 42.
  • reaction information communication instruction unit 44 transmits an instruction to transmit reaction information to the edge terminal 20b via the network communication unit 41 and the network 5.
  • the reaction information receiving unit 45 receives reaction information from the user terminal 20 via the network communication unit 41 and the network 5.
  • the reaction information storage unit 46 stores the reaction information received by the reaction information reception unit 45.
  • the time stamp reception unit 47 receives the time stamp of the content to be reproduced in the replay mode from the content distribution server 10 via the network communication unit 41 and the network 5.
  • the reaction information distribution unit 48 reads the reaction information with the same time stamp as the time stamp received by the time stamp reception unit 47 from the reaction information storage unit 46 , and distributes the content via the network communication unit 41 and the network 5 . It transmits to the distribution server 10 and the user terminal 20 .
  • FIG. 4 is a diagram showing the functional configuration of the user terminal 20.
  • the user terminal 20 includes a network communication unit 51, a content receiving unit 52, a content delay adding unit 53, an audio/video generating unit 54, an audio/video output unit 55, a reaction information receiving unit 56, and a reaction information delaying unit.
  • the network communication unit 51 establishes communication with the management server 3 and the content distribution server 10 via the network 5 and enables data transmission and reception between the management server 3 and the content distribution server 10 .
  • the content receiving unit 52 receives content transmitted from the content distribution server 10 via the network 5 and the network communication unit 51.
  • the content delay imparting unit 53 imparts a delay to the content received by the content receiving unit 52 under the control of the delay synchronization control unit 63. That is, the content delay adding unit 53 delays the reproduction of the content received by the content receiving unit 52 by a predetermined delay time (delay amount).
  • the video/audio generation unit 54 generates video and audio based on the content received by the content reception unit 52 or the content delayed by the content delay addition unit 53 .
  • the video/audio output unit 55 causes the display device 21 to display the video generated by the video/audio generation unit 54 and outputs the audio generated by the video/audio generation unit 54 from the audio output device 22 . That is, the video/audio generator 54 and the video/audio output unit 55 reproduce the content distributed from the content distribution server 10 .
  • the reaction information receiving unit 56 receives the reaction information transmitted from the edge terminal 20b (other user terminal system 4b) and the spectator information transmitted from the content distribution server 10 (content distribution server system 2) through the network communication unit 41 and the network communication unit 41. Receive via network 5 .
  • the reaction information delay imparting unit 57 imparts a delay to the reaction information received by the reaction information receiving unit 56. That is, the reaction information delay imparting unit 57 delays the reproduction of the reaction information received by the reaction information receiving unit 56 by a predetermined delay time (delay amount).
  • the video/audio generation unit 54 generates video and audio based on the reaction information or audience information received by the reaction information reception unit 56 or the reaction information delayed by the reaction information delay addition unit 57 .
  • the video/audio output unit 55 causes the display device 21 to display the video generated by the video/audio generation unit 54 and outputs the audio generated by the video/audio generation unit 54 from the audio output device 22 . That is, the video/audio generator 54 and the video/audio output unit 55 reproduce reaction information or audience information.
  • the reaction information input unit 58 acquires the video imaged by the imaging device 23 and the sound collected by the sound collector 24 as reaction information.
  • the time stamping unit 59 divides the reaction information acquired by the reaction information input unit 58 into packets, which are distribution units, and also adds a time stamp indicating the current time to each packet.
  • the reaction information transmission unit 60 transmits the reaction information for each packet to which the time stamp is added by the time stamp addition unit 59 to the management server 3 and the user terminal system 4 via the network communication unit 51 and the network 5.
  • the reaction information storage unit 61 stores reaction information for each packet to which a time stamp has been added by the time stamping unit 59 as needed.
  • the reaction information communication instruction receiving unit 62 receives instructions sent from the management server 3 .
  • the delay synchronization control unit 63 controls each unit regarding reproduction delay of the content and reaction information based on instructions from the management server 3 .
  • the content delay measurement unit 64 measures the delay that occurs with the content distribution server 10, that is, the amount of delay in receiving content.
  • the inter-user terminal communication delay measuring unit 65 measures the delay that occurs with the edge terminal 20b (another user terminal system 4b), that is, the amount of delay related to reception of reaction information.
  • the target user terminal system 4a synchronizes and reproduces the content distributed from the content distribution server system 2 and the reaction information transmitted from the edge terminal 20b.
  • the content delivery system 1 has an automatic matching mode for synchronizing between unspecified user terminals 20 that are not predetermined, and A friend mode for synchronizing between terminals 20 is provided.
  • the content distribution system 1 is provided with a replay mode for replay distribution (replay transmission, replay reproduction) of a part of content such as a sports score scene.
  • Replay mode may occur in both automatic matching mode and friends mode.
  • the automatic matching mode is a mode for synchronously reproducing content and reaction information between unspecified user terminals 20 .
  • the automatic matching mode will be described by exemplifying Example 1 in which synchronization is performed based on the delay time and Example 2 in which synchronization is performed based on the time stamp.
  • Example 1 the content and the reaction information are synchronously reproduced based on the control of the management server 3 .
  • Example 2 the content and reaction information are synchronously reproduced based on the control of the target terminal 20a.
  • FIG. 5 is a diagram for explaining the first embodiment.
  • FIG. 6 is a flow chart showing automatic matching processing of the management server 3 in the first embodiment.
  • FIG. 7 is a flowchart showing user terminal selection processing according to the first embodiment.
  • the content distributed from the content distribution server 10 to the target terminal 20a is delayed by the distribution delay time Dct. It is also assumed that the content distributed from the content distribution server 10 to the edge terminal 20b is delayed by the distribution delay time Dcr. Furthermore, it is assumed that a communication delay of the inter-user terminal delay time Drt occurs from the edge terminal 20b to the target terminal 20a.
  • the target terminal 20a reproduces the content with a delay of the delivery delay time Dct based on the time at which the content is delivered from the content delivery server 10.
  • the edge terminal 20b reproduces the content with a delay of the delivery delay time Dcr based on the time at which the content is delivered from the content delivery server 10 . Therefore, the reaction information of the other user R when viewing the content reproduced with a delay of the delivery delay time Dcr is based on the time when the content is delivered from the content delivery server 10, the reaction delay time Dcrt (delivery delay time Dcr+inter-user-terminal delay time (Drt) is delayed before being played back on the target terminal 20a.
  • the reaction delay time Dcrt delivery delay time Dcr+inter-user-terminal delay time (Drt) is delayed before being played back on the target terminal 20a.
  • the edge terminal selection process is executed in step S1.
  • the delay synchronization control unit 43 requests and acquires the delivery delay time Dct from the target terminal 20a in step S11.
  • the content delay measuring unit 64 compares the time stamp included in the content received by the content receiving unit 52 with the current time to determine the delivery time. A delay time Dct is measured (calculated). The delay synchronization control unit 63 then transmits the measured delivery delay time Dct to the management server 3 . Thereby, the delay synchronization control unit 43 acquires the distribution delay time Dcr (information on the first communication delay) from the target terminal 20a.
  • step S12 the delay synchronization control unit 43 randomly selects the edge terminal 20b and requests and acquires the distribution delay time Dcr and the inter-user terminal delay time Drt from the selected edge terminal 20b.
  • the content delay measuring unit 64 compares the time stamp included in the content received by the content receiving unit 52 with the current time to determine the delivery delay time Dcr. to measure Further, the inter-user-terminal communication delay measuring unit 65 transmits a test packet to the target terminal 20a.
  • the inter-user terminal communication delay measuring unit 65 of the target terminal 20a that received the test packet returns the test packet to the edge terminal 20b that transmitted the test packet. Then, the inter-user-terminal communication delay measuring unit 65 of the edge terminal 20b measures the inter-user-terminal delay time Drt based on the time taken to transmit and receive the test packet.
  • the delay synchronization control unit 63 transmits the measured distribution delay time Dcr and inter-user terminal delay time Drt to the management server 3 .
  • the delay synchronization control unit 43 receives the distribution delay time Dcr and the inter-user terminal delay time Drt from the edge terminal 20b.
  • step S13 the delay synchronization control unit 43 calculates the reaction delay time Dcrt (second communication delay information) by adding the acquired delivery delay time Dcr and the inter-user terminal delay time Drt.
  • step S14 the delay synchronization control unit 43 determines whether the time difference between the delivery delay time Dct and the reaction delay time Dcrt is within a preset specified value.
  • the specified value is set to the longest value (time) at which the user feels that they are synchronized, and is 70 ms as an example. Therefore, here, it is determined whether or not the content transmitted to the target terminal 20a is synchronized with the reaction information.
  • the delay synchronization control unit 43 sends the reaction information to the target terminal 20a for the selected edge terminal 20b. to start the transmission of reaction information. Further, the delay synchronization control unit 43 adds 1 to the number of communications indicating the number of edge terminals 20b transmitting reaction information to the target terminal 20a, and terminates the edge terminal selection process. In the edge terminal 20b that has received the reaction information transmission instruction, the reaction information transmission unit 60 transmits the reaction information to the target terminal 20a.
  • the delay synchronization control unit 43 transmits reaction information from the edge terminal 20b for which the difference between the content delay amount (distribution delay time Dct) and the reaction information delay amount (reaction delay time Dcrt) is within a specified value.
  • step S2 the delay synchronization control unit 43 determines whether the number of communications has reached the preset upper limit number. As a result, when the number of communications reaches the upper limit (Yes in step S2), the automatic matching process is terminated.
  • the upper limit number may be any number as long as it is 1 or more. However, the upper limit number may be changed in consideration of the communication speed of the content distribution server 10 and the user terminal 20 and the like.
  • the delay synchronization control unit 43 causes the target terminal 20a to transmit reaction information from the upper limit number of edge terminals 20b.
  • step S3 the delay synchronization control unit 43 determines whether the number of executions of the edge terminal selection process has reached the preset upper limit. do. As a result, the processing from step S1 to step S3 is repeated until the number of times of execution reaches the upper limit number of times (No in step S3).
  • the upper limit number of times may be any number as long as it is 1 or more.
  • step S3 the delay synchronization control unit 43 determines whether the number of communications is one or more in step S4.
  • the target terminal 20a has an edge terminal 20b capable of transmitting reaction information in synchronization with the content.
  • the automatic matching process is terminated.
  • step S5 the delay synchronization control unit 43 determines the response delay time Dcrt of the edge terminal 20b selected in the edge terminal selection process. is smaller than the delivery delay time Dct.
  • the delay synchronization control unit 43 determines in step S6 whether the delivery delay time Dct is selected, and the selected edge terminal 20b is instructed to transmit reaction information.
  • the reaction information transmission unit 60 transmits the reaction information to the target terminal 20a. That is, the delay synchronization control unit 43 selects, from among the plurality of edge terminals 20b, the edge terminal 20b that transmits reaction information with the delay amount closest to the delay amount of the content, and transmits the reaction information from the selected edge terminal 20b.
  • step S7 the delay synchronization control unit 43 instructs the target terminal 20a to add a delay time (delay amount) obtained by subtracting the reaction delay time Dcrt from the distribution delay time Dct to the reaction information.
  • the reaction information delay addition unit 57 adds a delay time to the reaction information under the control of the delay synchronization control unit 63 . That is, the target terminal 20a delays the reaction information received from the edge terminal 20b by the delay time and then reproduces it. Thereby, the target terminal 20a can synchronize the content and the reaction information.
  • the delay synchronization control unit 43 instructs the content delivery server 10 to send the audience to the target terminal 20a in step S8.
  • a spectator information transmission instruction is given to start information transmission.
  • the delay synchronization control unit 43 causes the content distribution server 10 to transmit the audience information to the target terminal 20a.
  • the content audience information distribution unit 34 transmits the audience information to the target terminal 20a.
  • the delay time to the target terminal 20a is the same. Therefore, in the user terminal system 4, when there is no reaction information that can be synchronized, by reproducing the audience information synchronized with the content, the target user T is given a sense of unity as if he or she were viewing the content together with the audience Cau. becomes possible.
  • FIG. 8 is a diagram for explaining the second embodiment.
  • FIG. 9 is a flow chart showing automatic matching processing of the target terminal 20a in the second embodiment.
  • FIG. 10 is a flowchart showing user terminal selection processing according to the second embodiment. As described above, in the second embodiment, the target terminal 20a executes the automatic matching process, so the management server 3 is unnecessary.
  • the target terminal 20a and the edge terminal 20b When receiving content from the content distribution server 10, the target terminal 20a and the edge terminal 20b receive content packets containing time stamps at any time.
  • the timestamp received by the target terminal 20a is assumed to be a distribution timestamp Tct
  • the timestamp received by the edge terminal 20b is assumed to be a distribution timestamp Tcr.
  • the target terminal 20a executes the edge terminal selection process in step S21 as shown in FIG.
  • the delay synchronization control unit 63 randomly selects the edge terminal 20b and requests the selected edge terminal 20b to transmit a test packet including the distribution time stamp Tcr. go and get.
  • the delay synchronization control unit 63 transmits the test packet including the distribution time stamp Tcr included in the content received by the content receiving unit 52 to the target terminal 20a. .
  • the delay synchronization control unit 63 of the target terminal 20a acquires the delivery time stamp Tcr included in the test packet as a reaction time stamp Tcrt (second communication delay information).
  • step S32 the delay synchronization control unit 63 extracts the delivery time stamp Tct (information on the first communication delay) contained in the content packet acquired at the same timing as the test packet containing the reaction time stamp Tcrt. and the reaction time stamp Tcrt included in the test packet is within a preset specified value.
  • the delivery time stamp Tct information on the first communication delay
  • the delay synchronization control unit 63 causes the edge terminal 20b to start sending reaction information. Instruct to send information. In addition, the delay synchronization control unit 63 adds 1 to the number of communications indicating the number of edge terminals 20b transmitting reaction information, and terminates the edge terminal selection process.
  • step S22 the delay synchronization control unit 63 determines whether the number of communications has reached the preset upper limit number. As a result, when the number of communications reaches the upper limit number (Yes in step S22), the automatic matching process ends.
  • step S23 the delay synchronization control unit 63 determines whether the number of executions of the edge terminal selection process has reached the preset upper limit. do. As a result, the processing from step S21 to step S23 is repeated until the number of times of execution reaches the upper limit number of times (No in step S23).
  • step S23 the delay synchronization control unit 63 determines whether the number of communications is 1 or more in step S24. As a result, if the number of communications is 1 or more (Yes in step S24), the automatic matching process is terminated.
  • step S25 the delay synchronization control unit 63 sets the reaction time stamp Tcrt of the edge terminal 20b selected in the edge terminal selection process. , which is earlier than the delivery time stamp Tct acquired at the same timing.
  • step S26 the delay synchronization control unit 63 selects the delivery time of the reaction time stamps earlier than the delivery time stamp Tct.
  • the edge terminal 20b with the reaction time stamp Tcrt closest to the stamp Tct is instructed to transmit the reaction information.
  • the reaction information transmission unit 60 transmits the reaction information to the target terminal 20a.
  • step S27 the delay synchronization control unit 63 adds a delay time (delay amount) obtained by subtracting the reaction time stamp Tcrt from the delivery time stamp Tct to the reaction information. That is, the target terminal 20a delays the reaction information received from the edge terminal 20b by the delay time and then reproduces it. Thereby, the target terminal 20a can synchronize the content and the reaction information.
  • the delay synchronization control unit 63 instructs the content distribution server 10 to transmit audience information in step S28. Give an instruction to send spectator information to start.
  • the content audience information distribution unit 34 transmits the audience information to the target terminal 20a. Therefore, if there is no reaction information that can be synchronized, the target terminal 20a reproduces the spectator information synchronized with the content, thereby giving the target user T a sense of unity as if they were watching together with the spectator Cau. becomes possible.
  • the friend mode will be described by exemplifying a third embodiment in which synchronization is performed with one specific preset edge terminal 20b and a fourth embodiment in which synchronization is performed with a plurality of preset specific edge terminals 20b.
  • FIG. 11 is a diagram for explaining the third embodiment.
  • FIG. 12 is a flow chart showing friend matching processing of the management server 3 in the third embodiment.
  • the content distributed from the content distribution server 10 to the target terminal 20a is delayed by the distribution delay time Dct. It is also assumed that the content distributed from the content distribution server 10 to one specific edge terminal 20b is delayed by the distribution delay time Dcr. Furthermore, it is assumed that a communication delay of the user terminal delay time Dtr occurs from the target terminal 20a to the edge terminal 20b. Furthermore, it is assumed that a communication delay of the inter-user terminal delay time Drt occurs from the edge terminal 20b to the target terminal 20a.
  • the target terminal 20a reproduces the content with a delay of the delivery delay time Dct based on the time at which the content is delivered from the content delivery server 10 .
  • the edge terminal 20b delays the content by the distribution delay time Dcr based on the time when the content is distributed from the content distribution server 10, and reproduces the content. Therefore, the reaction information of the other user R when viewing the content played back with a delay of the delivery delay time Dcr is delayed by the reaction delay time Dcrt (the delivery delay time Dcr + the delay time between the user terminals Drt) and is displayed on the target terminal 20a. will be played in
  • the delay synchronization control unit 43 requests and acquires the delivery delay time Dct (first communication delay information) from the target terminal 20a in step S41.
  • step S42 the delay synchronization control unit 43 requests and acquires the distribution delay time Dcr and the inter-user terminal delay time Drt from one specific edge terminal 20b. Further, in step S43, the delay synchronization control unit 43 calculates the reaction delay time Dcrt (second communication delay information) by adding the acquired delivery delay time Dcr and the inter-user-terminal delay time Drt.
  • step S44 the delay synchronization control unit 43 determines whether the distribution delay time Dct is smaller than the reaction delay time Dcrt. Then, if the delivery delay time Dct is smaller than the reaction delay time Dcrt (Yes in step S44), in step S45 the delay synchronization control unit 43 gives the content a delay time Ac obtained by subtracting the delivery delay time Dct from the reaction delay time Dcrt. Then, the target terminal 20a is instructed to add a delay. In the target terminal 20 a that has received the delay addition instruction, the content delay addition section 53 adds the delay time Ac to the content under the control of the delay synchronization control section 63 .
  • the target terminal 20a delays the content received from the content distribution server 10 by the delay time Ac and then reproduces the content.
  • the delivery delay time Dct' of the content in the target terminal 20a is the sum of the delivery delay time Dct and the delay time Ac.
  • step S41 to step S45 described above is also executed in one specific edge terminal 20b.
  • the reaction delay time Dcrt' after the application is a time obtained by adding the delay time Ac to the reaction delay time Dcrt.
  • the target terminal 20a and the edge terminal 20b synchronize the content and the reaction information by delaying the content when the delivery delay time is smaller than the reaction delay time.
  • step S46 the delay synchronization control unit 43 requests and acquires the inter-user terminal delay time Dtr from the target terminal 20a. Determine if there is Here, both the target terminal 20a and the edge terminal 20b determine whether the content and reaction information are synchronized.
  • step S46 if the difference between the inter-user terminal delay time Dtr and the inter-user terminal delay time Drt is within the specified value (Yes in step S46), the friend matching process ends. On the other hand, if the difference between the inter-user terminal delay time Dtr and the inter-user terminal delay time Drt is not within the specified value (No in step S46), the delay synchronization control unit 43 determines that the delivery delay time Dct' is equal to the response delay time in step S47. It is determined whether it is greater than the sum of Dcrt' and a specified value.
  • the delay synchronization control unit 43 calculates the reaction delay time Dcrt from the distribution delay time Dct' in step S48. ' and the delay time Ar obtained by subtracting the specified value is given to the reaction information.
  • the reaction information delay addition unit 57 adds the delay time Ar to the reaction information under the control of the delay synchronization control unit 63. FIG. As a result, the target terminal 20a delays the reaction information received from the edge terminal 20b by the delay time Ar and then reproduces it.
  • step S49 the delay synchronization control unit 43 instructs one specific edge terminal 20b to transmit reaction information to the target terminal 20a to start transmitting reaction information.
  • the reaction information transmission unit 60 transmits the reaction information to the target terminal 20a.
  • FIG. 13 is a diagram for explaining the fourth embodiment.
  • FIG. 14 is a flow chart showing friend matching processing of the management server 3 in the fourth embodiment.
  • FIG. 15 is a flow chart showing the first delay imparting process.
  • FIG. 16 is a flow chart showing the second delay imparting process.
  • the content distributed from the content distribution server 10 to the target terminal 20a is delayed by the distribution delay time Dct. It is also assumed that the content distributed from the content distribution server 10 to a plurality of specific edge terminals 20b is delayed by a distribution delay time Dcr. Furthermore, it is assumed that a communication delay of the user terminal delay time Dtr occurs from the target terminal 20a to the edge terminal 20b. Furthermore, it is assumed that a communication delay of the inter-user terminal delay time Drt occurs from the edge terminal 20b to the target terminal 20a.
  • the target terminal 20a reproduces the content with a delay of the delivery delay time Dct based on the time at which the content is delivered from the content delivery server 10 .
  • the edge terminal 20b delays the content by the distribution delay time Dct from the time when the content is distributed from the content distribution server 10, and reproduces the content. Therefore, the reaction information of the other user R when viewing the content played back with a delay of the delivery delay time Dct is delayed by the reaction delay time Dcrt (the delivery delay time Dcr + the delay time between the user terminals Drt) and is displayed on the target terminal 20a. will be played in
  • step S53 the specific edge terminal 20b is instructed to transmit reaction information to the target terminal 20a to start transmitting reaction information.
  • the reaction information transmission unit 60 transmits the reaction information to the target terminal 20a. Details of the first delay imparting process and the second delay imparting process will be described below.
  • step S61 the delay synchronization control unit 43 requests and acquires the delivery delay time Dct (information on the first communication delay) from the target terminal 20a. do.
  • step S62 the delay synchronization control unit 43 selects one of the plurality of specific edge terminals 20b, requests the selected edge terminal 20b for the distribution delay time Dcr and the inter-user terminal delay time Drt, and obtains them. .
  • step S63 the delay synchronization control unit 43 calculates the reaction delay time Dcrt (second communication delay information) by adding the acquired delivery delay time Dcr and the inter-user terminal delay time Drt.
  • step S64 the delay synchronization control unit 43 determines whether the reaction delay time Dcrt has been calculated for all specific edge terminals 20b. Then, if the reaction delay time Dcrt has not been calculated for all the specific edge terminals 20b (No in step S64), the process returns to step S62.
  • the delay synchronization control unit 43 selects the maximum value among the calculated reaction delay times Dcrt in step S65. Extract as the maximum reaction delay time Dcrtmax.
  • step S66 the delay synchronization control unit 43 determines whether the maximum reaction delay time Dcrtmax is longer than the delivery delay time Dct. As a result, if the maximum reaction delay time Dcrtmax is not longer than the distribution delay time Dct (No in step S66), the first delay determination process is terminated. On the other hand, if the maximum reaction delay time Dcrtmax is greater than the distribution delay time Dct (Yes in step S66), in step S67 the delay synchronization control unit 43 subtracts the distribution delay time Dct from the maximum reaction delay time Dcrtmax to obtain the delay time Ac. Gives a delayed addition instruction to be added to the content.
  • the content delay addition section 53 adds the delay time Ac to the content under the control of the delay synchronization control section 63 .
  • the target terminal 20a delays the content received from the content distribution server 10 by the delay time Ac and then reproduces the content. That is, the delay synchronization control unit 43 delays the content based on the largest delay amount among the delay amounts of the plurality of reaction information.
  • the first delay imparting process described above is also executed in all specific edge terminals 20b.
  • the distribution delay time Dct' after the giving is the sum of the delivery delay time Dct and the delay time Ac.
  • the delivery delay time Dcr' after the delivery is a time obtained by adding the delay time Ac to the delivery delay time Dcr
  • the reaction delay time Dcrt' is the reaction delay time. The time is obtained by adding the delay time Ac to Dcrt.
  • step S71 the delay synchronization control unit 43 requests the target terminal 20a to acquire the distribution delay time Dct' after the first delay adding process.
  • step S72 the delay synchronization control unit 43 selects one of a plurality of specific edge terminals 20b, and assigns the selected edge terminal 20b a distribution delay time Dcr' after the first delay addition process and a delay time Drt between user terminals. Request and get. Then, the delay synchronization control unit 43 calculates the reaction delay time Dcrt' by adding the distribution delay time Dcr' and the inter-user terminal delay time Drt.
  • step S73 the delay synchronization control unit 43 determines whether the delivery delay time Dct' is greater than the sum of the reaction delay time Dcrt' and a specified value. As a result, if the delivery delay time Dct' is not greater than the sum of the reaction delay time Dcrt' and the specified value (No in step S73), the process proceeds to step S75.
  • the delay synchronization control unit 43 converts the delivery delay time Dct' to the reaction delay time Dcrt' and A delay addition instruction is given to add the delay time Ar obtained by subtracting the specified value to the reaction information received from the specific edge terminal 20b.
  • the reaction information delay addition unit 57 adds the delay time Ar to the reaction information under the control of the delay synchronization control unit 63. FIG. As a result, the target terminal 20a delays the reaction information received from the edge terminal 20b by the delay time Ar and then reproduces it.
  • step S75 the delay synchronization control unit 43 determines whether all specific edge terminals 20b have been specified. If not specified for all specific edge terminals 20b (No in step S75), the process returns to step S72. On the other hand, if all the specific edge terminals 20b have been specified (Yes in step S75), the second delay imparting process ends. As a result, the target terminal 20a can prevent the reaction information for the content earlier in time than the content to be played back from being played back earlier.
  • FIG. 17 is a diagram for explaining the replay mode.
  • the content distribution server 10 transmits the content to the user terminal 20 in units of packets in real time. Also, the content audience information storage unit 35 of the content distribution server 10 stores content to which time stamps have been added.
  • reaction information with a time stamp is stored in the reaction information storage unit 61. Also, in the content distribution server 10 , the reaction information receiving unit 45 receives the reaction information with the time stamp from the user terminal 20 and stores it in the reaction information storage unit 46 .
  • the content audience information distribution unit 34 transmits the time stamp Tcp to the management server 3 .
  • the user terminal 20 when the content reception unit 52 receives the content, the audio/video generation unit 54 and the audio/video output unit 55 replay the content. As a result, the user terminal 20 can allow the user to view the redistributed content.
  • the reaction information distribution section 48 transmits reaction information corresponding to the time stamp Tcp to the user terminal 20 and the content distribution server 10 .
  • the video/audio output section 37 causes the display device 13 to display the video contained in the reaction information, and the audio contained in the reaction information to the audio output device 14. output from For example, when reproducing a plurality of pieces of reaction information, the video/audio output unit 37 causes the display device 13 to display the images included in the plurality of pieces of reaction information in multi-view. Also, the video/audio output unit 37 mixes, reverberates, or cheers the sounds contained in the plurality of pieces of reaction information, and outputs them from the audio output device 14 . Thus, the reaction information can be viewed by the performer Cpf and the audience Cau.
  • reaction information receiving section 56 when the reaction information receiving section 56 receives the reaction information, the reaction information is reproduced by the audio/video generating section 54 and the audio/video output section 55. FIG. Thereby, the reaction information synchronized with the replay-reproduced content can be viewed.
  • the user terminal 20 may reproduce the reaction information corresponding to the time stamp Tcp taken and collected by itself. Also, the content distribution server 10 may receive reaction information corresponding to the time stamp Tcp from the user terminal 20 .
  • FIG. 18 is a diagram showing an arrangement example of the display device 21 and the audio output device 22 in the user terminal system 4.
  • FIG. 19 is a diagram illustrating an example of reproduction of reaction information.
  • the display device 21a is provided in front of the user, the display device 21b is provided on the left side of the display device 21a, and the display device 21c is provided on the right side of the display device 21a. Then, the image of the content is displayed on the display device 21a, and the image of the reaction information (image of the user) is displayed on the display devices 21b and 21c.
  • the audio output device 22a is provided on the upper right front side of the user, the audio output device 22b is provided on the upper left front side of the user, and the audio output device 22k is provided on the rear side of the user.
  • the audio output device 22c is provided on the upper right of the user, the audio output device 22d is provided on the upper left of the user, the audio output device 22g is provided on the lower right of the user, and the audio output device 22h is provided on the lower left of the user.
  • the voice of the reaction information is output by .
  • the audio output device 22e is provided at the rear upper right of the user, the audio output device 22f is provided at the rear upper left of the user, the audio output device 22i is provided at the rear lower right of the user, and the audio output device 22j is provided at the rear lower left of the user. Audience information audio is output by these.
  • the video-audio generator 54 and the video-audio output unit 55 set the delay time given to the reaction information as shown in FIG. Accordingly, the image of the other user R is changed and displayed. That is, the audio/video generation unit 54 and the audio/video output unit 55 reproduce the reaction information in a manner corresponding to the amount of delay given to the reaction information.
  • the video/audio generation unit 54 and the video/audio output unit 55 display the video of the other user R such that it becomes smaller as the amount of delay (delay time) assigned to the reaction information increases. It can be displayed so that the target distance becomes farther, displayed with increased transparency, or displayed with different colors. This makes it possible for the target user T to easily recognize the other user R whose delay time is long, and makes it possible to make the other user R whose delay time is long and difficult to synchronize with stand out less.
  • the management server 3, the content distribution server 10, and the user terminal 20 can have a hardware configuration shown in FIG. 20, for example.
  • this hardware 900 mainly comprises a CPU 902, a ROM 904, a RAM 906, a host bus 908 and a bridge 910. Further, this hardware includes an external bus 912 , an interface 914 , an input section 916 , an output section 918 , a storage section 920 , a drive 922 , a connection port 924 and a communication section 926 .
  • the CPU 902 executes the above-described processing based on various programs recorded in the ROM 904, RAM 906, storage unit 920, or removable recording medium 928.
  • the ROM 904 stores programs read by the CPU 902, data used for calculation, and the like.
  • the RAM 906 stores, for example, programs to be read by the CPU 902 and various parameters that change when the programs are executed.
  • a host bus 908 capable of high-speed data transmission.
  • the host bus 908 is connected, for example, via a bridge 910 to an external bus 912 with a relatively low data transmission speed.
  • the input unit 916 for example, a mouse, keyboard, touch panel, button, switch, lever, or the like is used.
  • a remote controller capable of transmitting control signals using infrared rays or other radio waves may be used.
  • the output unit 918 is a device capable of outputting acquired information, such as a printer, mobile phone, or facsimile.
  • the storage unit 920 is a device for storing various data.
  • a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like is used.
  • the drive 922 is a device that reads information recorded on a removable recording medium 928 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, or writes information to the removable recording medium 928, for example.
  • the removable recording media 928 are, for example, DVD media, Blu-ray media, HD DVD media, various semiconductor storage media, and the like.
  • the removable recording medium 928 may be, for example, an IC card equipped with a contactless IC chip, an electronic device, or the like.
  • the above IC is an abbreviation for Integrated Circuit.
  • the connection port 924 is a port for connecting an external connection device 930 such as a USB port, IEEE1394 port, SCSI, RS-232C port, optical audio terminal, or the like.
  • the externally connected device 930 is, for example, a printer, a portable music player, a digital camera, a digital video camera, an IC recorder, or the like.
  • USB is an abbreviation for Universal Serial Bus.
  • SCSI is an abbreviation for Small Computer System Interface.
  • a communication unit 926 is a device for connecting to the network 5 .
  • the embodiment is not limited to the specific examples described above, and various modifications can be made.
  • the delay time (delay amount) is added to the reaction information.
  • the delay time may be added to the content.
  • the delay synchronization control unit 43 instructs the content distribution server 10 to transmit the spectator information to the target terminal 20a. good too.
  • the management server 3 controls the reproduction of content and reaction information, but the user terminal 20 may control. Further, although the user terminal 20 controls the reproduction of the content and reaction information in the second embodiment, the management server 3 may control the reproduction.
  • the functional units (control units) of the management server 3, the content distribution server 10, and the user terminal 20 in the above-described embodiment may be grouped as appropriate or separated for each process. Also, the functional units (control units) provided in the management server 3, the content distribution server 10, and the user terminal 20 may be provided in other devices.
  • the information processing apparatus reduces the first communication delay (distribution delay time, distribution time stamp) from the content distribution server 10 to the target terminal 20a and through the edge terminal 20b. acquire information about a second communication delay (response delay time, reaction time stamp) from the content distribution server 10 to the target terminal 20a, and distribute the content based on the information about the first communication delay and the second communication delay
  • a control unit (delay synchronization control unit 43, delay synchronization control unit 63, etc.) that controls the reproduction of the content transmitted from the server 10 and the reaction information transmitted from the edge terminal 20b at the target terminal is provided.
  • the information processing device can synchronize and reproduce the content and the reaction information on the target terminal 20a. Therefore, the information processing apparatus can suppress reduction in the sense of unity given to the target user T with the other user R due to the content and reaction information being out of sync.
  • the control unit may add a delay to the reaction information transmitted from the edge terminal 20b.
  • the information processing apparatus can reproduce the reaction information in synchronization with the content by adding a delay in the target terminal 20a. Therefore, the information processing apparatus can suppress reduction in the sense of unity given to the target user T with the other user R due to the content and reaction information being out of sync.
  • the control unit may delay the content transmitted from the content distribution server 10 .
  • the information processing apparatus can add a delay to the target terminal 20a to reproduce the content synchronized with the reaction information. Therefore, the information processing apparatus can suppress reduction in the sense of unity given to the target user T with the other user R due to the content and reaction information being out of sync.
  • the control unit may acquire the delay amount of the content transmitted from the content distribution server 10 to the target terminal 20a as the first communication delay information. Further, in the information processing device, the control unit may include, as the second communication delay information, a delay amount of the content transmitted from the content distribution server to the edge terminal, and a delay between the edge terminal and the target terminal. It is conceivable to acquire the amount of delay between Thereby, the information processing device can synchronize the content and the reaction information based on the delay amount.
  • the control unit selects the edge terminal 20b that transmits the reaction information with the delay amount closest to the delay amount of the content from among the plurality of edge terminals 20b, and transmits the reaction information from the selected edge terminal 20b. can be sent.
  • the information processing device can reproduce the content and the reaction information so as to minimize the time lag. Therefore, the information processing apparatus can suppress reduction in the sense of unity given to the target user T with the other user R due to the content and reaction information being out of sync.
  • control unit may delay the content based on the largest delay amount among the delay amounts of the plurality of pieces of reaction information. As a result, the information processing apparatus can prevent the reaction information for the content earlier in time than the content to be played back from being played back earlier.
  • the control unit can cause the target terminal 20a to transmit reaction information from the upper limit number of edge terminals 20b.
  • the target terminal 20a can reproduce the maximum number of pieces of reaction information together with the content. Therefore, the information processing apparatus can give the target user T a sense of unity with many other users R.
  • control unit may cause reaction information to be transmitted from the edge terminal 20b for which the difference between the delay amount of the content and the delay amount of the reaction information is within a specified value.
  • the target terminal 20a synchronizes and reproduces the content and the reaction information within the specified value. Therefore, the information processing apparatus can suppress reduction in the sense of unity given to the target user T with the other user R due to the content and reaction information being out of sync.
  • the control unit causes the content distribution server to transmit the spectator information to the target terminal when there is no reaction information such that the difference between the delay amount of the content and the delay amount of the reaction information is within a specified value.
  • the target terminal 20a reproduces the content and the spectator information in synchronism. Therefore, in the information processing apparatus, it is possible to give the target user T a sense of unity with the spectator Cau who is viewing the content locally.
  • the control unit reproduces the reaction information in a mode according to the delay amount given to the reaction information. This makes it possible for the target user T to easily recognize the other user R whose delay time is long, and makes it possible to make the other user R whose delay time is long and difficult to synchronize with stand out less.
  • control unit when content is replay-transmitted from the content distribution server 10, the control unit may cause the target terminal 20a to reproduce reaction information corresponding to the replay-transmitted content. Thereby, the reaction information synchronized with the replay-reproduced content can be viewed.
  • control unit may cause the target terminal 20a to reproduce the reaction information obtained by the target terminal 20a corresponding to the content to be replay-transmitted. This allows the target user T to view his or her own reaction when the replay-reproduced content is reproduced.
  • control unit may cause the content distribution server 10 to reproduce the reaction information corresponding to the content to be replay-transmitted. This allows the performer Cpf and the audience Cau at the content venue to view the reaction of the user at a remote location.
  • the information processing method of the embodiment acquires information about a first communication delay from the content distribution server to the target terminal and a second communication delay from the content distribution server to the target terminal via the edge terminal. and, based on the information about the first communication delay and the second communication delay, reproduce the content transmitted from the content distribution server and the reaction information transmitted from the edge terminal at the target terminal. Control. Further, the program of the embodiment acquires information about a first communication delay from the content distribution server to the target terminal and a second communication delay from the content distribution server to the target terminal via the edge terminal, Based on the information about the first communication delay and the second communication delay, the content transmitted from the content distribution server and the reproduction of reaction information transmitted from the edge terminal are controlled at the target terminal. .
  • Such a program can be recorded in advance in a HDD as a recording medium built in equipment such as a computer device, or in a ROM or the like in a microcomputer having a CPU.
  • a flexible disc a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disc, a DVD (Digital Versatile Disc), a Blu-ray disc (Blu-ray Disc (registered trademark)), a magnetic disc, a semiconductor memory
  • It can be temporarily or permanently stored (recorded) in a removable recording medium such as a memory card.
  • Such removable recording media can be provided as so-called package software.
  • it can also be downloaded from a download site via a network such as a LAN (Local Area Network) or the Internet.
  • LAN Local Area Network
  • Such a program is suitable for widely providing the information processing apparatus of the embodiment.
  • a program for example, by downloading a program to a mobile terminal device such as a smartphone or tablet, a mobile phone, a personal computer, a game device, a video device, a PDA (Personal Digital Assistant), etc., it can function as the information processing device of the present disclosure.
  • the present technology can also adopt the following configuration.
  • An information processing device comprising a control unit.
  • (3) The control unit The information processing apparatus according to (1) or (2), wherein the content transmitted from the content distribution server is delayed.
  • the control unit The information processing apparatus according to any one of (1) to (3), wherein a delay amount of content transmitted from the content distribution server to the target terminal is acquired as the information regarding the first communication delay. (5) The control unit obtaining the delay amount of the content transmitted from the content distribution server to the edge terminal and the delay amount between the edge terminal and the target terminal as the second communication delay information; The information processing device according to any one of (4). (6) The control unit Select from among the plurality of edge terminals the edge terminal that transmits reaction information with a delay amount closest to the delay amount of the content, and cause the selected edge terminal to transmit the reaction information. (1) to (5) ).
  • the control unit The information processing apparatus according to any one of (1) to (6), wherein the content is delayed based on the largest delay amount among the plurality of delay amounts of the reaction information.
  • the control unit The information processing apparatus according to any one of (1) to (7), wherein reaction information is transmitted from an upper limit number of edge terminals to the target terminal.
  • the control unit The information processing according to any one of (1) to (8), wherein the content or the reaction information is delayed so that the difference between the delay amount of the content and the delay amount of the reaction information is within a specified value.
  • the control unit (1) to (9) transmitting spectator information from the content distribution server to the target terminal when there is no reaction information for which the difference between the delay amount of the content and the delay amount of the reaction information is within a specified value;
  • the information processing device according to any one of .
  • the control unit The information processing apparatus according to any one of (1) to (10), wherein the reaction information is reproduced in a mode according to the delay amount given to the reaction information.
  • (12) The control unit The information processing apparatus according to any one of (1) to (11), wherein when the content is replay-transmitted from the content distribution server, the target terminal reproduces the reaction information corresponding to the replay-transmitted content.
  • the control unit The information processing apparatus according to (12), wherein the target terminal reproduces the reaction information acquired by the target terminal corresponding to the content to be replay-transmitted.
  • the control unit The information processing apparatus according to (12) or (13), wherein the content distribution server reproduces the reaction information corresponding to the content to be replay-transmitted.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • General Engineering & Computer Science (AREA)
  • Information Transfer Between Computers (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

Le présent dispositif de traitement d'informations comprend une unité de commande qui acquiert des informations concernant un premier retard de communication d'un serveur de distribution de contenu à un terminal cible et un second retard de communication du serveur de distribution de contenu par l'intermédiaire d'un terminal périphérique au terminal cible, et commande, sur la base des informations concernant le premier retard de communication et le second retard de communication, la reproduction sur le terminal cible du contenu transmis à partir du serveur de distribution de contenu et des informations de réaction transmises à partir du terminal périphérique.
PCT/JP2022/007333 2021-05-19 2022-02-22 Dispositif de traitement d'informations, procédé de traitement d'informations, et programme WO2022244364A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
DE112022002670.7T DE112022002670T5 (de) 2021-05-19 2022-02-22 Informationsverarbeitungsvorrichtung, informationsverarbeitungsverfahren und programm
CN202280029021.9A CN117178558A (zh) 2021-05-19 2022-02-22 信息处理装置、信息处理方法和程序

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021-084748 2021-05-19
JP2021084748 2021-05-19

Publications (1)

Publication Number Publication Date
WO2022244364A1 true WO2022244364A1 (fr) 2022-11-24

Family

ID=84140221

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/007333 WO2022244364A1 (fr) 2021-05-19 2022-02-22 Dispositif de traitement d'informations, procédé de traitement d'informations, et programme

Country Status (3)

Country Link
CN (1) CN117178558A (fr)
DE (1) DE112022002670T5 (fr)
WO (1) WO2022244364A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003235027A (ja) * 2002-02-12 2003-08-22 Matsushita Electric Ind Co Ltd 配信映像の同時再生方法、映像配信システムおよび端末装置
JP2004135062A (ja) * 2002-10-10 2004-04-30 Matsushita Electric Ind Co Ltd 映像コミュニケーションシステム及び映像コミュニケーション方法
JP2005208710A (ja) * 2004-01-20 2005-08-04 Nec Corp 同期制御方法、コミュニケーション同期制御装置および双方向コミュニケーションシステム
JP2006129420A (ja) * 2004-09-30 2006-05-18 Victor Co Of Japan Ltd 情報コミュニケーション端末装置
JP2010232716A (ja) * 2009-03-25 2010-10-14 Nec Corp ビデオコンテンツ受信装置、コメント交換システム、コメントデータの生成方法およびプログラム

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3457705A1 (fr) 2013-05-30 2019-03-20 Sony Corporation Organe de commande d'affichage, procédé de commande d'affichage et programme informatique

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003235027A (ja) * 2002-02-12 2003-08-22 Matsushita Electric Ind Co Ltd 配信映像の同時再生方法、映像配信システムおよび端末装置
JP2004135062A (ja) * 2002-10-10 2004-04-30 Matsushita Electric Ind Co Ltd 映像コミュニケーションシステム及び映像コミュニケーション方法
JP2005208710A (ja) * 2004-01-20 2005-08-04 Nec Corp 同期制御方法、コミュニケーション同期制御装置および双方向コミュニケーションシステム
JP2006129420A (ja) * 2004-09-30 2006-05-18 Victor Co Of Japan Ltd 情報コミュニケーション端末装置
JP2010232716A (ja) * 2009-03-25 2010-10-14 Nec Corp ビデオコンテンツ受信装置、コメント交換システム、コメントデータの生成方法およびプログラム

Also Published As

Publication number Publication date
DE112022002670T5 (de) 2024-03-07
CN117178558A (zh) 2023-12-05

Similar Documents

Publication Publication Date Title
CN103327377B (zh) 用于内容的同步流传输的系统、方法和基础设施
CN103190092B (zh) 用于流数字内容的同步重放的系统和方法
JP5609160B2 (ja) 情報処理システム、コンテンツ合成装置および方法、並びに記録媒体
US8391671B2 (en) Information processing device and method, recording medium, and program
CN1981524B (zh) 信息处理设备和方法
JP6662063B2 (ja) 収録データ処理方法
WO2006011398A1 (fr) Dispositif et procede de traitement d’informations, support d’enregistrement et programme
EP3136713A1 (fr) Dispositif de reproduction d'informations, procédé de reproduction d'informations, dispositif d'enregistrement d'informations, et procédé d'enregistrement d'informations
US8782176B2 (en) Synchronized video system
JP5733321B2 (ja) 自動演奏装置
CN110856009B (zh) 网络k歌系统、网络k歌的音视频播放方法及相关设备
KR101873086B1 (ko) 정보 시스템, 정보 재현 장치, 정보 생성 방법, 및 기록 매체
KR101808452B1 (ko) 하이브리드 ｖｒ 노래방 시스템
JP2012070116A (ja) 情報処理装置、情報処理方法、再生装置、再生方法、およびプログラム
JP2006039917A (ja) 情報処理装置および方法、記録媒体、並びにプログラム
JP7285244B2 (ja) コンピュータプログラム、方法及びサーバ装置
WO2022244364A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations, et programme
JP4534201B2 (ja) 情報コミュニケーション端末装置
JP5109496B2 (ja) データ配信装置、データ配信方法およびプログラム
JP2021021870A (ja) コンテンツ収集・配信システム
TWI287786B (en) A voice- and image-mixing system and method for remote karaoke sets
WO2022014369A1 (fr) Dispositif de traitement d'image, procédé de traitement d'image et programme
JP2009268141A (ja) 端末間における同期制御方法
JPWO2018173312A1 (ja) 仮想現実視聴システム、再生同期方法および仮想現実視聴用プログラム
WO2022018786A1 (fr) Système de traitement sonore, dispositif de traitement sonore, procédé de traitement sonore et programme de traitement sonore

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22804285

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 112022002670

Country of ref document: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22804285

Country of ref document: EP

Kind code of ref document: A1