JP2013150096A - Information processor, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide interactive services of great variety in streaming distribution via a network, thereby improving a sense of unity of users.SOLUTION: In a site 102-3, a display apparatus 160 with a large display is installed, and a trainee 124-1 and a trainee 124-2 of a lecture take the lecture respectively using their small terminal 171-1 and small terminal 171-2. On a part of the display of the display apparatus 160, a code image 161 is displayed and imaged with a digital camera incorporated in each small terminal to identify a terminal ID assigned to the display apparatus 160. Each small terminal accesses a management server regularly to acquire information on a reproduction time, a reproduction state, etc. of the display apparatus 160.

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program, and in particular, can provide a variety of interactive services in streaming distribution via a network, and improve a sense of unity between users. The present invention relates to an information processing apparatus, an information processing method, and a program.

  In recent years, video distribution via networks such as the Internet has been actively performed. For example, image data captured by a camera, audio data collected by a microphone, etc. are encoded by a distribution server and multicast on the network, and these data are received simultaneously by a terminal connected to the network. Streaming distribution for playback is also popular.

  In recent streaming distribution, captions and the like are often superimposed and displayed on an image reproduced on a terminal by multicasting metadata in addition to data such as image and sound.

  For example, an e-learning system and a web learning system are provided using such streaming distribution. As a result, for example, a lecture having the same contents can be simultaneously provided to a person at a remote place, and the degree of understanding of the lecture students can be confirmed via the network.

  In recent years, e-learning systems and Web learning systems are oriented toward “interactive” and “bidirectional” so that students can participate in lectures more actively. Yes. For example, an interactive content such as a chat service is also used together with live content such as images and sounds distributed almost in real time.

  As described above, when the interactive process progresses, for example, it becomes necessary to superimpose and display live content and interactive content. However, unlike the case where content is distributed using broadcast waves or the like, in the case of streaming distribution via a network such as the Internet, the delay between when the data is transmitted and when the data is played cannot be ignored. It will be big.

  For example, a technology has been proposed in which content and metadata that are distributed separately are received and the content and metadata are automatically associated with each other (see, for example, Patent Document 1).

  In addition, for example, a technique has been proposed that reduces anxiety of a user due to a chat comment display waiting time caused by providing interactive content such as a chat service and live content in synchronization (see, for example, Patent Document 2). ).

JP 2005-229509 A JP 2004-350178 A

  However, as described above, recent e-learning systems and web learning systems are becoming more interactive. In such a situation, for example, the sense of unity between the lecturer and the student is improved, and the presence is more realistic. It is expected to provide some lectures.

  Furthermore, not only e-learning systems, but also various services that have been provided by distributing content using broadcast waves, etc., will be realized by streaming content via a network such as the Internet. It is also expected. This is because it is expected to provide interactive services that are more diverse than ever.

  The present invention has been made in view of such a situation, and in streaming delivery via a network, it is possible to provide a variety of interactive services and improve the sense of unity between users. It is to make.

  According to one aspect of the present invention, a plurality of playback times specified at time intervals set in advance are specified based on time stamps attached to content data played back by a plurality of playback devices connected to a network. An information processing apparatus that communicates via a network with a management server that is acquired from each of the apparatuses, and each of the plurality of playback apparatuses that reproduces content that includes data obtained by encoding captured images in real time Identification information acquisition means for acquiring identification information corresponding to one of the reproduction apparatuses, and notification of the identification information of the reproduction apparatus acquired by the identification information acquisition means to the management server in association with own identification information Identification information notifying means to be acquired by the identification information acquiring means from the management server at a preset time interval. An information processing apparatus and a reproduction time obtaining means for obtaining a reproduction time of the reproduction apparatus corresponding to the identification information.

  The transmission apparatus may further include transmission means for transmitting the metadata related to the reproduction time of the content with a time stamp corresponding to the acquired reproduction time to the management server via the network.

  Further provided is metadata reproduction means for reproducing the metadata related to the reproduction time of the content received via the network at the reproduction time of the reproduction device corresponding to the time stamp attached to the metadata. Can be.

  The information processing apparatus is possessed by a user who views the content reproduced by the reproduction apparatus, and a plurality of users can view the content reproduced by one reproduction apparatus.

  By comparing the reproduction time notified by each of the plurality of information processing devices and the actual time by the management server, each delay time of the plurality of information processing devices is specified, and the plurality of information processing devices Display data for displaying information related to each of the delay times can be generated.

  The identification information can be displayed as an image on the display of the playback device.

  According to one aspect of the present invention, a plurality of playback times specified at time intervals set in advance are specified based on time stamps attached to content data played back by a plurality of playback devices connected to a network. An information processing method of an information processing apparatus that communicates via a network with a management server respectively acquired from the apparatus, wherein the identification information acquisition unit includes data obtained by encoding a captured image in real time Identification information corresponding to one of the plurality of playback devices is obtained, and the identification information notifying means uses the identification information of the playback device acquired by the identification information acquiring means as its own information. Notifying the management server in association with the identification information, the reproduction time acquisition means from the management server at a preset time interval, Serial is an information processing method comprising the steps of obtaining the reproduction time of the reproduction apparatus corresponding to the identification information acquired by the identification information acquisition unit.

  According to one aspect of the present invention, a computer is configured to set a reproduction time specified based on a time stamp attached to data of content reproduced by a plurality of reproduction devices connected to a network at a predetermined time interval. An information processing apparatus that communicates via a network with a management server that is acquired from each of a plurality of playback apparatuses, and each of the plurality of contents that includes data obtained by encoding captured images in real time Identification information acquisition means for acquiring identification information corresponding to one of the reproduction apparatuses, and the management information associated with identification information of the reproduction apparatus acquired by the identification information acquisition means in association with own identification information Identification information notifying means for notifying the server, and the identification information acquisition means from the management server at a preset time interval. A program to function as an information processing apparatus and a reproduction time obtaining means for obtaining a reproduction time of the reproduction apparatus corresponding to the acquired identification information by.

  In one aspect of the present invention, identification information corresponding to one playback device among the plurality of playback devices that respectively play back content including data obtained by encoding captured images in real time is acquired, The acquired identification information of the reproduction device is notified to the management server in association with its own identification information, and corresponds to the acquired identification information from the management server at a preset time interval. A playback time of the playback device is acquired.

  According to the present invention, it is possible to provide a variety of interactive services in streaming delivery via a network, and to improve the sense of unity between users.

It is a figure explaining the live delivery in the conventional streaming delivery system. It is a figure which shows the structural example of the streaming delivery system which concerns on one embodiment of this invention. It is a figure which shows the example of the table memorize | stored inside a management server. It is a figure which shows the structural example of the database memorize | stored inside a dedicated terminal. It is a figure which shows the example in the case of presenting the information of the database of FIG. 4 in a graph. It is a figure which shows another example in the case of presenting the information of the database of FIG. 4 in a graph. It is a figure which shows another example in the case of presenting the information of the database of FIG. 4 in a graph. It is a figure which shows another example in the case of presenting the information of the database of FIG. 4 in a graph. It is a figure which shows another example of the table memorize | stored inside a management server. It is a figure explaining the form which takes a lecture using a small terminal and a display apparatus. It is a figure explaining the example of a display of a code image. It is a figure which shows another example of the table memorize | stored inside a management server. It is a flowchart explaining the example of a process of a management server. It is a flowchart explaining the example of a process of a terminal. It is a flowchart explaining the example of a process of a small terminal. It is a flowchart explaining the example of a process of a dedicated terminal. It is a figure which shows the example in the case of applying the streaming delivery system of this invention to the live content provision system regarding sports watching. And FIG. 16 is a block diagram illustrating a configuration example of a personal computer.

  Embodiments of the present invention will be described below with reference to the drawings.

  First, problems of the conventional technology will be described.

  In recent years, video distribution via networks such as the Internet has been actively performed. In streaming distribution, content data is transmitted from a distribution server, and a terminal connected to the network performs playback simultaneously while receiving the data. Also, for example, image data captured by a camera, audio data collected by a microphone, etc. are encoded by a distribution server and multicasted over the network, so that images and sounds can be distributed almost in real time. You can also. Streaming content distribution in near real time is also referred to as live distribution.

  FIG. 1 is a diagram for explaining live distribution in a conventional streaming distribution system. In the figure, for example, content data such as images and sounds shot and collected in the studio 11 is transmitted to the venue 12 and the venue 13 via the Internet 10.

  For example, a terminal such as a personal computer connected to the Internet is installed in the hall 12 and the hall 13, and each terminal plays while receiving content data.

  However, unlike the case where content is distributed using broadcast waves or the like, in the case of streaming distribution via a network such as the Internet, the delay between when the data is transmitted and when the data is played cannot be ignored. It will be big.

  For example, an image photographed at a predetermined time in the studio 11 is reproduced on the terminals of the venue 12 and the venue 13 after several seconds to several minutes have elapsed. This is because the delay time from when the data is transmitted to when the data is reproduced depends on the congestion status of the network from the studio to each venue, the processing capability of each terminal, and the like.

  Also, for example, a time stamp is added to the content data in order to synchronize the content image and sound.

  In the example of FIG. 1, in the studio 11, the time represented by the time elapsed since the start of shooting (referred to as the on-site time) is “10:00: 00 (10 minutes 0.0 seconds”). Yes) ”.

  On the other hand, the time corresponding to the image reproduced at the terminal of the hall 12 (hereinafter referred to as the reproduction time) is “9:58:30”, and corresponds to the image reproduced at the terminal of the hall 13. The time is “9:59:10”. That is, the elapsed time required for shooting up to the point (image) being played back by the terminal is displayed as the playback time with the top time of the content as “00:00:00”. The reproduction time can be specified based on, for example, a time stamp added to content data.

  As described above, when content is distributed by live distribution, a delay occurs between a shooting location such as a studio and a reception location such as a venue, and the delay varies depending on each reception location.

  In recent years, contents distributed lively tend to be “interactive” and “two-way”, and so-called interactive is progressing. For example, an interactive content such as a chat service is also used together with a content distributed live. In this case, it is necessary to superimpose and display the live-distributed content and metadata such as character strings constituting the interactive content. At this time, it is desirable to display the image and the metadata in synchronization. That is, it is preferable that the metadata input at the shooting location at the site time t is displayed on the terminal screen at the reception location at the playback time t.

  That is, the metadata in the interactive content is related to the playback time of the content distributed live.

  Considering the delay in live distribution as described above with reference to FIG. 1, for example, by adding a time stamp for synchronizing interactive content with the content to be distributed live, at the shooting location at the site time t. The input metadata can be displayed on the terminal screen at the receiving location at the reproduction time t.

  However, the metadata of the interactive content can be input even at the reception location. For example, if the metadata input at the reception location is distributed as it is, the image and the metadata cannot be distributed in synchronization. As described above, the conventional technology has a problem that the metadata of the interactive content cannot always be properly synchronized.

  Furthermore, for example, when providing an e-learning system, a web learning system, or the like using a streaming distribution system that uses interactive content together with content that is distributed live, the following problems are also assumed.

  For example, in the e-learning system, it is assumed that instructor's question and the student's answer to the question, the student's question about the contents of the lecture, etc. are displayed by the interactive content metadata. In this case, the lecturer needs to wait for a response such as an answer or question from the student before proceeding with the lecture. This is because if the lecture is not advanced in consideration of the degree of understanding of many students and the degree of concentration in the lecture, it will be almost the same as a simple video lecture.

  In particular, if you want to introduce a debate-type lecture where the lecturer and the student discuss and proceed with the lecture, the lecturer needs to wait for a response from the student and answer the question before proceeding with the lecture. . This is because it is necessary to give each student an opportunity to speak equally, and by doing so, the student's participation in the lecture can be raised and the sense of reality of the lecture can be enhanced. .

  However, the conventional technique has a problem that it is difficult to introduce a discussion-type lecture or the like because the lecturer at the shooting location cannot grasp the delay time at the terminal of the student at each reception location. In other words, with the conventional e-learning system, it was difficult to get a sense of reality like a lecture in an actual classroom.

  Therefore, in the present invention, the above-described problems can be solved.

  FIG. 2 is a diagram illustrating a configuration example of the streaming distribution system 100 according to the embodiment of the present invention. The streaming distribution system 100 shown in the figure is configured as an e-learning system, for example.

  As shown in FIG. 2, in the streaming distribution system 100, content data such as lecture images and voices taken by the instructor 121, which are captured and collected by the studio 101, are sent to the venue 102- via the network 110 such as the Internet. 1 is transmitted to the terminals of the students in the venue 102-2,. In this example, the student 122 is taking a lecture using the terminal 150 in the venue 102-1.

  In this example, there is shown an example in which there is only one student in the venue 102-1, but a large number of students may take a lecture using their respective terminals. In addition, in the figure, the student and the terminal are illustrated only at the venue 102-1, but the student also took a lecture using the terminal at the venue 102-2,. It shall be.

  The terminal 150 is constituted by, for example, a personal computer, and reproduces content data transmitted via the network 110 while receiving it.

  In addition, the streaming distribution system 100 performs streaming distribution of content in almost real time. That is, live distribution of content is performed.

  In other words, the lecture image and sound of the lecturer 121 photographed and collected by the camera 131 are encoded in real time by the encoder 141 and supplied to the video distribution server 142. In this example, only the encoder 141 is shown, but a plurality of encoders may be provided as necessary.

  A time stamp is added to the content data in consideration of synchronization when the content data is reproduced. The time stamp added to the content data is, for example, information for identifying the elapsed time since the image of the first frame of the content image data was taken.

  The video distribution server 142 is configured to transmit content data to each terminal at each venue via the network 110 in order to improve the quality of images and audio during the playback of the content. The terminal 150 reproduces the content data by decoding the received content data.

  Further, in the streaming distribution system 100 shown in FIG. 2, a student at each venue responds to information transmitted from the studio 101, and information transmitted by one of the students is displayed on the screen of another student's terminal. Interactive content has been introduced that allows it to be displayed above. The interactive content is actually composed of metadata such as GUI data transmitted and received on the network 110 via the metadata distribution server 143, for example.

  For example, in the studio 101, a dedicated terminal 132 composed of a personal computer or the like is installed. For example, the lecture assistant 123 inputs a question issued by the lecturer 121 in expectation of a response from the student as text data. To do. The dedicated terminal 132 is configured to generate data such as a GUI that includes text data input by the lecture assistant 123. This GUI is displayed on the student's terminal, for example, with a character string describing the instructor's question and a button to which the question can be answered with “Yes” or “No”.

  In addition, for example, the dedicated terminal 132 generates data such as a GUI including text data for displaying a character string obtained by translating the contents of a lecture (for example, the contents of a statement made by the lecturer 121).

  For example, a content image display area 151 and a metadata display area 152 are provided on the screen of the terminal 150 of the student 122.

  In the content image display area 151, an image of a lecture by the lecturer 121 is displayed. Note that the audio of the lecture is output from a speaker of the terminal 150 (not shown). The metadata display area 152 displays a character string describing the instructor's question and a GUI with a button that can answer the question with “Yes” or “No”. In addition, the metadata display area 152 is configured as metadata configured as a character string in which the content of a question by another student is described, and as a character string translated from the instructor's utterance into another language. Metadata etc. are displayed as needed. Further, in the metadata display area 152, an area for inputting a question to the lecturer as a character string is displayed as necessary.

  The content image display area 151 and the metadata display area 152 may not be provided separately, and the metadata may be displayed superimposed on the content image.

  The management server 144 is configured to manage the playback time of each terminal in the streaming distribution system 100, for example.

  For example, when the student 122 starts taking a lecture, the terminal 150 accesses the management server 144. Then, the management server 144 assigns a unique ID to the terminal 150 and transmits the ID to the terminal 150. Note that the dedicated terminal 132 also accesses the terminal 150 prior to distribution of lecture content, for example. The management server 144 assigns a special ID to the dedicated terminal 132 and transmits the ID to the dedicated terminal 132. For example, the dedicated terminal 132 is not a terminal used by the student such as the terminal 150, and thus a special ID is assigned.

  In addition, the management server 144 analyzes the time stamp of the image data supplied from the encoder 141 through communication with the video distribution server 142 and specifies the local time.

  In addition, when a plurality of encoders are provided, the distribution start time in each encoder may be different. For example, when a plurality of encoders are provided, the management server 144 is configured to specify the distribution start time for each encoder.

  Furthermore, each terminal in the streaming distribution system 100 accesses the management server 144 periodically. At this time, each terminal specifies the reproduction time based on the time stamp corresponding to the image of the content currently reproduced and displayed, and notifies the management server 144 of the reproduction time.

  Further, the dedicated terminal 132 periodically accesses the management server 144 to acquire the local time.

  FIG. 3 is a diagram illustrating an example of a table stored in the management server 144. In the figure, a table in which terminal IDs and reproduction times are associated is described. For example, the terminal ID “0001” is a special ID assigned to the dedicated terminal 132. In this case, the reproduction time “2:10:30” corresponding to the terminal ID “0001” represents the local time.

  The management server 144 manages the reproduction time at each terminal by updating the data in the table as shown in FIG. 3 for each access from each terminal, for example.

  For example, when a lecture assistant 123 inputs a question issued by the lecturer 121 in expectation of a response from the student as text data to the dedicated terminal 132, the dedicated terminal 132 adds a time stamp corresponding to the local time. Data is generated and transmitted to the metadata distribution server 143.

  In addition, for example, when the student 122 operates the GUI displayed in the metadata display area 152 to answer a question from the lecturer, the terminal 150 displays the reproduction time in the metadata transmitted as the answer to the question. A time stamp corresponding to is added. Alternatively, for example, when the student 122 inputs a question about the content of the lecture, the terminal 150 adds a time stamp corresponding to the reproduction time to the metadata transmitted as the question. Then, the metadata with the time stamp is transmitted to the metadata distribution server 143.

  Furthermore, each terminal including the dedicated terminal 132 and the terminal 150 periodically accesses the metadata distribution server 143 to acquire metadata addressed to itself. For example, the terminal 150 displays the metadata addressed to itself acquired by accessing the metadata distribution server 143 in synchronization with the content image based on the time stamp added to the metadata. This is because, for example, metadata such as a question from a lecturer is related to the reproduction time of lecture content that is distributed live.

  In addition, for example, the dedicated terminal 132 accesses the metadata distribution server 143 and displays the metadata addressed to itself acquired and aggregated. Note that the metadata distribution server 143 may be configured to be integrated with the dedicated terminal 132, for example.

  For example, the dedicated terminal 132 obtains data of a table as shown in FIG. 3 from the management server 144 and creates a database relating to the delay time in each student's terminal. Then, the dedicated terminal 132 generates information for identifying the delay time and the like in each student's terminal and presents it to the lecturer 121 and the like. By doing in this way, for example, the lecturer 121 can advance the lecture while giving each student a sufficient opportunity to ask questions and remarks.

  FIG. 4 is a diagram illustrating a configuration example of the database 200 stored inside the dedicated terminal 132.

  In the example shown in the figure, the current time 201 is recorded in the database 200. The current time 201 is, for example, information representing the current time of the studio 101 for 24 hours, and is sequentially updated based on, for example, a timekeeping function inside the dedicated terminal 132.

  Further, the database 200 records start time information 202. The start time information 202 is, for example, information in which the time at which content distribution is started is expressed for 24 hours. The “distribution point” shown in the figure is, for example, information such as a URL for identifying an encoder, and the distribution start time for each of a plurality of encoders (here, distribution point 1 and distribution point 2) is recorded. Has been.

  Furthermore, the student 200 delay information 203 is recorded in the database 200. The student delay information 203 is information generated based on a table in which terminal IDs stored in the management server 144 are associated with playback times. That is, although not shown in the figure, each record constituting the student-specific delay information 203 is associated with a unique terminal ID.

  The “distribution point” in the delay information for each student 203 is, for example, information that identifies an encoder that encodes content data received by a terminal having a terminal ID corresponding to the record. Assume that the association between the terminal ID and the encoder is performed in advance by the management server 144.

  As the “reproduction time” in the delay information 203 for each student, for example, the reproduction time in a table as shown in FIG. 3 is described. That is, it is set as the reproduction time at the terminal having the terminal ID corresponding to the record. As described above, the playback time at each terminal is specified based on the time stamp corresponding to the image of the content that is currently played back and displayed. Therefore, the playback time at each terminal represents, for example, the elapsed time since the image of the first frame when the image of the content that is currently played back and displayed is captured.

  When a plurality of encoders are provided and the distribution start times of the encoders are different, the image of the first frame of the content differs for each encoder. For example, when the distribution start time of the first encoder is “13:10:51”, the image shot at the time “13:10:51” in the studio 101 becomes the image of the first frame of the content. On the other hand, when the distribution start time of the second encoder is “13:21:00”, the image taken at the time “13:21:00” in the studio 101 is the image of the first frame of the content.

  The “time” in the delay information 203 for each student is a time obtained by adding the “reproduction time” described above to the distribution start time of each distribution point described in the start time information 202. For example, by comparing the “time” in the delay information 203 for each student with the time described in the current time 201, the progress of the lecture at each terminal (actually, the playback status of the content) can be realized.

  “Delay” in the delay information for each student 203 is a time obtained as a difference between the “time” of the delay information for each student 203 and the time described in the current time 201.

  For example, the dedicated terminal 132 may present the information in the database 200 as shown in FIG. 4 as it is, or may present the information in a graph as described below.

  FIG. 5 is a diagram illustrating an example in which the information of the database 200 is presented as a graph by the dedicated terminal 132.

  In the graph shown in the figure, the horizontal axis is the delay time, the vertical axis is the number of students, and the number of students for each delay time (actually the number of terminals) is described as a bar graph. For example, by presenting the graph as shown in FIG. 5 to the instructor 121, the instructor 121 can easily grasp the delay of the entire student.

  FIG. 6 is a diagram illustrating another example in the case where the dedicated terminal 132 presents the information of the database 200 as a graph.

  In the graph shown in the figure, the horizontal axis is the delay time, the vertical axis is the number of students, and the number of students for each delay time (actually the number of terminals) is described as a bar graph. Further, in the example of the figure, the occurrence of an event is described by a balloon and a dotted line in the vertical direction. Here, the event means, for example, display of predetermined metadata, switching of a scene, and the like.

  That is, in the terminal on the left side of the dotted line in FIG. 6, it means that an event such as display of predetermined metadata or scene switching has already occurred in the reproduced content, and in the terminal on the right side of the dotted line in the figure, This means that the event has not yet occurred. Therefore, as time passes, the dotted line in the figure moves to the right. In the balloon shown in the figure, an event name (here, “Evnt1”) and a ratio of terminals that have already generated an event (here, “38%”) are displayed.

  For example, by presenting a graph as shown in FIG. 6 to the instructor 121, the instructor 121 can grasp the reaction to the event more accurately.

  FIG. 7 is a diagram showing still another example when the information of the database 200 is presented as a graph by the dedicated terminal 132.

  In the graph shown in the figure, the horizontal axis is the delay time, the vertical axis is the number of students, and the number of students for each delay time (actually the number of terminals) is described as a bar graph. In this example, a part of the bar graph is displayed in a different color. For example, when it is possible to identify a special student based on the terminal ID, a portion corresponding to the number of students of the special student may be displayed in a different color.

  Special participants include, for example, students who have taken lectures for a fee, students who have been given the right to ask questions to lecturers, and students who have been entrusted with writing the contents of lectures. It is said.

  In this example, a special student is displayed in one color, but may be displayed in a plurality of colors according to the type of the special student.

  For example, by presenting the graph as shown in FIG. 7 to the instructor 121, the instructor 121 can accurately grasp the delay situation while being aware of the attendance mode.

  FIG. 8 is a diagram showing still another example when the information of the database 200 is presented as a graph by the dedicated terminal 132.

  In the graph shown in the figure, the horizontal axis is the delay time, the vertical axis is the number of students, and the number of students for each delay time (actually the number of terminals) is described as a bar graph. Further, in the example of the figure, the occurrence of an event is described by a balloon and a dotted line in the vertical direction. In this case, for example, the display of metadata of a question from a lecturer who expects an answer from a student is an event.

  That is, in the terminal on the left side of the dotted line in FIG. 8, it means that an event such as the display of the metadata of the question from the lecturer has already occurred in the reproduced content, and in the terminal on the right side of the dotted line in FIG. Means that the event has not yet occurred. Therefore, as time passes, the dotted line in the figure moves to the right.

  In the example of FIG. 8, a part of the bar graph is displayed in a different color. For example, the part corresponding to the number of students who have answered the question from the instructor described above is displayed in a different color.

  In the balloon shown in the figure, the event name (here, “Question1”) and the percentage of terminals where the event (question from the instructor) has already occurred (here “62%”) / of terminals that have already answered the question The ratio (here “41%”) is displayed.

  For example, by presenting a graph as shown in FIG. 8 to the instructor 121, the instructor 121 can more accurately grasp the reaction to the event.

  Here, it has been described that the database 200 as shown in FIG. 4 is recorded by the dedicated terminal 132 and the information as shown in FIGS. 5 to 8 is presented. For example, the dedicated terminal 132 is connected to the management server 144. It may be configured to be integrated. That is, the database 200 as shown in FIG. 4 may be recorded by the management server 144 and information as shown in FIGS. 5 to 8 may be presented.

  Furthermore, the management server 144 and the metadata distribution server 143 may be integrated.

  Here, an example in which information is presented by the dedicated terminal 132 based on the information recorded in the database 200 of FIG. 4 has been described, but information may be presented at the terminal 150, for example. For example, the time described in “delay” in the delay information 203 for each student described above with reference to FIG. 4 may be distributed to each terminal as metadata. In this way, for example, students at each terminal can participate in the lecture while being aware of the delay of the content being played back on their own terminal.

  Thus, according to the present invention, for example, it is possible to solve the problem that the metadata of interactive content cannot always be properly synchronized. That is, each terminal including the dedicated terminal 132 and the terminal 150 generates the metadata by adding the time stamp based on the time axis of the content image, so that the metadata and the content image are always synchronized appropriately. You can make it.

  Further, according to the present invention, it is possible to solve the problem that it is difficult to introduce a discussion-type lecture. That is, as described above, information as shown in FIG. 5 to FIG. 8 can be presented, so that the lecturer 121 can advance the lecture while giving each student a sufficient opportunity to ask questions and speak. As a result, the sense of unity between the instructor and all the participants is enhanced, and for example, a lecture with a sense of reality can be provided by the e-learning system in the same way as a discussion-type lecture performed in a classroom.

  In the example described above with reference to FIG. 3, it has been described that the terminal ID and the reproduction time are stored as table data. However, for example, the reproduction state of the content in each terminal may be stored together. Good. For example, on the terminal 150 or the like, it is possible to pause the playback during the playback of the content or to play back a scene played back in the past once again by a user operation. Such a content reproduction state at each terminal may be notified from each terminal to the management server 144 and further stored in association with the terminal ID.

  Alternatively, the type of metadata provided to each terminal may be stored as table data. For example, when the translation language is different for each terminal, information for identifying the translation language is the type of metadata. In addition, for example, when special information is provided for a visually handicapped person or a hearing handicapped person, information identifying whether or not the special information needs to be provided is a type of metadata.

  FIG. 9 is a diagram illustrating another example of a table stored in the management server 144. In the example of the figure, a table in which the reproduction time, the reproduction state, the metadata type 1, and the metadata type 2 are associated with the terminal ID is described. Note that the metadata type 1 in the example of the figure is information for identifying a translation language, and the metadata type 2 is information for identifying whether or not special information for visually handicapped persons or hearing impaired persons needs to be provided. It is said. In addition, information may be further stored as metadata type 3, metadata type 4,.

  In the above, the example in which the content image and the metadata are displayed on the same screen on the student's terminal has been described. That is, as described above with reference to FIG. 2, the example in which the content image display area 151 and the metadata display area 152 are provided on the screen displayed on the display of the terminal 150 has been described.

  However, in recent years, small terminals such as smartphones and tablets have become widespread, and content images displayed on such small terminals become small images. Also, for example, unlike a personal computer, a smartphone is easy to carry and can be brought into various venues.

  For example, if a student with a small terminal such as a smartphone displays metadata on a small terminal while viewing the content image on a large display, a more convenient streaming distribution system can be realized. Can do.

  For example, as shown in FIG. 10, a display device 160 having a large display is installed in the venue 102-3. The students 124-1 and 124-2 who take the lecture at the venue 102-3 take the lecture using their small terminals 171-1 and 171-2, respectively.

  Here, the small terminal generally refers to an electronic device or the like suitable for carrying, and is not referred to as “small” in order to distinguish between physical sizes. Further, when it is not necessary to distinguish small terminals individually, they are simply referred to as small terminals 171.

  In the case of the example in FIG. 10, for example, an image of content is displayed on the display of the display device 160. That is, the image displayed in the content image display area 151 on the screen of the terminal 150 in FIG. 2 is displayed on the entire display of the display device 160.

  In the case of the example in FIG. 10, for example, metadata-related screens are displayed on the respective displays of the small terminal 171-1 and the small terminal 171-2. That is, the image displayed in the metadata display area 152 on the screen of the terminal 150 in FIG. 2 is displayed on the entire display of the small terminal 171.

  In the example shown in FIG. 10, since the display device 160 reproduces content, the display device 160 periodically accesses the management server 144 and notifies the reproduction time, the reproduction state, and the like. -1 and the small terminal 171-2 cannot specify the reproduction time on the display device 160.

  Therefore, for example, the terminal ID assigned to the display device 160 is displayed on a part of the display of the display device 160. For example, as shown in FIG. 11, a code image 161 corresponding to the terminal ID assigned to the display device 160 is displayed on a part of the display of the display device 160. The code image 161 is displayed as, for example, a QR code (registered trademark).

  The small terminal 171-1 and the small terminal 171-2 are, for example, terminals assigned to the display device 160 by photographing the code image 161 with a digital camera or the like built in each small terminal and analyzing the code image 161. Specify the ID. Alternatively, the user may input a terminal ID assigned to the display device 160.

  Then, the small terminal 171-1 and the small terminal 171-2 periodically access the management server 144 and acquire information such as a reproduction time and a reproduction state. Thereby, the small terminal 171-1 and the small terminal 171-2 can transmit the metadata with the time stamp corresponding to the reproduction time of the display device 160 to the metadata distribution server 143, for example.

  FIG. 12 is a diagram illustrating an example of a table stored in the management server 144 when a display device and a small terminal are introduced, for example, as described above with reference to FIG. In the example of the figure, an area for displaying a code is provided in the table. In the example of FIG. 12, for example, the terminal ID “0001” is the dedicated terminal 132, the terminal ID “0002” is the display device 160, and the terminal ID “0003” and the terminal ID “0004” are the small terminals 171- 1 and a small terminal 171-2.

  As shown in FIG. 12, in the records corresponding to the terminal IDs “0002” to “0004”, the code “0002” is described. In this case, the terminal ID of the display device 160 is stored as the code. Has been.

  For example, it is also possible to identify the terminal IDs of students who have attended at the venue 102-3 by sorting the records in the table shown in FIG. 12 using codes as keys. For example, by specifying the terminal ID of a student who has attended at a predetermined venue, it is also possible to generate and distribute metadata for only the student at that venue.

  For example, the venue where the display device with the terminal ID “0101” is installed is a venue for persons with hearing impairments, and metadata for displaying a transcript of the lecture contents is displayed on the terminals of the students who attend the venue. It can also be delivered.

  In addition, by doing this, for example, it is possible to easily add up the number of students for each venue. This is because it is considered that one student usually has one small terminal. Thus, for example, the venue where each student is located can be specified by information described as a code in a table stored in the management server 144.

  Next, an example of processing of the management server 144 in the streaming distribution system 100 shown in FIG. 2 will be described with reference to the flowchart of FIG. Note that the management server 144 is configured to have a processor, a memory, and the like, for example, and the processing of each step shown in FIG. 13 is executed by executing the program loaded in the memory by the processor.

  In step S21, the management server 144 analyzes the time stamp of the image data supplied from the encoder, for example, through communication with the video distribution server 142, and specifies the local time.

  In step S22, the management server 144 determines whether there is an access from a terminal such as the terminal 150, the small terminal 171-1, the small terminal 171-2, and waits until it is determined that there is an access.

  If it is determined in step S22 that there is an access, the process proceeds to step S23.

  In step S23, the management server 144 determines whether or not an ID has been assigned to the terminal determined to have been accessed in step S22. For example, the management server 144 separately generates a table in which IP addresses and terminal IDs are associated with each other, and can determine whether or not an ID has been assigned to each terminal.

  If it is determined in step S23 that an ID has not been assigned, the process proceeds to step S24.

  In step S24, the management server 144 assigns an ID to the terminal determined to have been accessed in step S22.

  If it is determined in step S23 that an ID has been assigned, the process in step S24 is skipped.

  In step S25, the management server 144 acquires information such as the reproduction time from the terminal determined to have been accessed in step S22. At this time, for example, information such as reproduction time, reproduction state, metadata type, and code is acquired.

  In step S26, the management server 144 updates the table data based on the information acquired in the process of step S25. At this time, for example, the data in the table described above with reference to FIGS. 3, 4, and 12 is updated.

  And a process returns to step S21 and the process after it is repeatedly performed.

  In this way, the processing of the management server is executed.

  Next, an example of processing of the terminal 150 in the streaming distribution system 100 will be described with reference to the flowchart of FIG. This process is executed in parallel with, for example, a reception / reproduction process for receiving and reproducing content data distributed from the video distribution server 142.

  Note that the terminal 150 is configured to have a processor, a memory, and the like, for example, and the processing of each step shown in FIG. 14 is executed by executing the program loaded in the memory by the processor.

  In step S41, the terminal 150 specifies the reproduction time based on the time stamp corresponding to the image of the content currently reproduced and displayed by the terminal 150.

  In step S42, the terminal 150 accesses the management server 144.

  In step S43, the terminal 150 notifies the management server 144 of information such as the reproduction time. At this time, for example, information such as a reproduction time, a reproduction state, a metadata type, and a code is notified, and this information is acquired by the management server 144 in step S25 of FIG.

  In step S44, the terminal 150 accesses the metadata distribution server 143.

  In step S45, the terminal 150 transmits the metadata with the time stamp to the metadata distribution server 143. At this time, for example, a metadata corresponding to the reproduction time specified in step S41 is added to metadata transmitted as a question about the content of the lecture, metadata transmitted as an answer to the question from the lecturer, and the like. Is done.

  In step S <b> 46, the terminal 150 receives metadata addressed to itself from the metadata distribution server 143. At this time, for example, metadata such as metadata transmitted as a question from the lecturer and metadata for displaying a translated character string of the contents of the lecture are received.

  In step S47, the terminal 150 reproduces the metadata addressed to itself received in step S46 at the reproduction time specified based on the time stamp attached to the metadata (image display, audio output, etc.). .

  Note that the processing in step S45 may be executed after the processing in step S46 and step S47 is executed first.

  In step S48, the terminal 150 determines whether or not a predetermined time has elapsed, and waits until it is determined that the predetermined time has elapsed.

  If it is determined in step S48 that the predetermined time has elapsed, the process returns to step S41, and the subsequent processes are repeatedly executed.

  In this way, the processing of the terminal is executed.

  Next, an example of processing of the small terminal 171 in the streaming distribution system 100 will be described with reference to the flowchart of FIG. Note that the small terminal 171 has, for example, a configuration including a processor, a memory, and the like, and the processing of each step shown in FIG. 15 is executed by executing a program loaded in the memory by the processor.

  In step S61, the small terminal 171 obtains the code of the display device 160 that reproduces the content viewed by the student. At this time, for example, as described above, the terminal ID assigned to the display device 160 is specified by photographing the code image 161 with a digital camera or the like built in the small terminal 171 and analyzing the code image 161. Then, the terminal ID assigned to the display device 160 is acquired as a code.

  In step S <b> 62, the small terminal 171 accesses the management server 144.

  In step S63, the small terminal 171 notifies the management server 144 of the code acquired in step S61.

  In step S64, the small terminal 171 identifies the reproduction time. At this time, for example, the small terminal 171 searches the table stored in the management server 144 for a record of the display device having the terminal ID corresponding to the code acquired in step S61, and reproduces the reproduction time of the display device. Specify as time.

  In step S65, the small terminal 171 notifies the management server 144 of the reproduction time and the like. Note that the above-described code may also be notified in step S65. At this time, for example, information such as reproduction time, metadata type, and code is notified, and this information is acquired by the management server 144 in the process of step S25 of FIG.

  Since the playback time of the small terminal 171 is equal to the playback time of the display device 160 specified by the code, the playback time may not be notified in step S65.

  In step S <b> 66, the small terminal 171 accesses the metadata distribution server 143.

  In step S <b> 67, the small terminal 171 transmits the metadata with the time stamp to the metadata distribution server 143. At this time, for example, a metadata corresponding to the reproduction time specified in step S64 is added to metadata transmitted as a question about the content of the lecture, metadata transmitted as an answer to the question from the lecturer, or the like. Is done.

  In step S <b> 68, the small terminal 171 receives metadata addressed to itself from the metadata distribution server 143. At this time, for example, metadata such as metadata transmitted as a question from the lecturer and metadata for displaying a translated character string of the contents of the lecture are received.

  In step S69, the small terminal 171 reproduces the metadata addressed to itself received in step S68 at the reproduction time specified based on the time stamp attached to the metadata (image display, audio output, etc.). To do.

  In step S70, the small terminal 171 determines whether or not a predetermined time has elapsed, and waits until it is determined that the predetermined time has elapsed.

  If it is determined in step S70 that the predetermined time has elapsed, the process returns to step S61, and the subsequent processes are repeatedly executed.

  In this way, the processing of the small terminal is executed.

  Next, an example of processing of the dedicated terminal 132 in the streaming distribution system 100 will be described with reference to the flowchart of FIG. This process is executed when, for example, information as described above with reference to FIGS. 5 to 8 is presented.

  Note that the dedicated terminal 132 has, for example, a configuration including a processor, a memory, and the like, and the processing of each step illustrated in FIG. 16 is executed by executing a program loaded in the memory by the processor.

  In step S91, the dedicated terminal 132 specifies the current time. As described above, the current time of the studio 101 is information expressed in 24 hours, and is sequentially updated based on, for example, the timekeeping function inside the dedicated terminal 132.

  In step S92, the dedicated terminal 132 specifies the distribution start time for each distribution point. As described above, the distribution point is, for example, information such as a URL for identifying the encoder, and the distribution start time for each of the plurality of encoders is specified.

  In step S93, the dedicated terminal 132 obtains the current time specified in step S91, the distribution start time for each distribution point specified in step S92, and the terminal ID and the reproduction time stored in the management server 144. Create or update a database based on the associated table.

  At this time, for example, each record of the delay information 203 by student is generated or updated, and the database 200 shown in FIG. 4 is generated or updated.

  In step S94, the dedicated terminal 132 generates graph data based on the information recorded in the database generated or updated in the process of step S93.

  In step S95, the dedicated terminal 132 displays a graph based on the graph data generated in the process of step S94. At this time, for example, graphs as shown in FIGS. 5 to 8 are displayed.

  In this way, the processing of the dedicated terminal is executed.

  As described above, for example, the dedicated terminal 132 may be integrated with the management server 144. That is, the process illustrated in FIG. 16 may be executed by the management server 144.

  In the above description, the streaming distribution system 100 according to the present invention is applied to an e-learning system, for example. However, the streaming distribution system 100 can be applied to a live content providing system related to sports watching.

  FIG. 17 is a diagram illustrating an example in which the streaming distribution system 100 is applied to a live content providing system related to sports watching.

  In the live content providing system shown in the figure, image and audio contents of the soccer stadium 105 are distributed live. Although not shown in the figure, in the case of FIG. 17 as well, as described above with reference to FIG. 2, the camera 131, the dedicated terminal 132, the encoder 141, the video distribution server 142, the metadata A distribution server 143 and a management server 144 are provided.

  That is, in the case of the example of FIG. 17, the image and sound of the soccer stadium 105 are photographed and collected by a camera 131 (not shown) and distributed as content data.

  In the case of the example in the figure, in the soccer stadium 105, a spectator having the small terminals 171-3 to 171-5 is watching. Note that the small terminals 171-3 to 171-5 acquire codes based on QR codes (registered trademark) displayed on an electronic bulletin board of the soccer stadium 105. In addition, the playback times of the small terminals 171-3 to 171-5 are all set to the local time.

  Content data transmitted via the network 110 is received and reproduced by the terminal 150 of the user's home 106. The terminal 150 operates in the same manner as described above with reference to FIG.

  Further, content data transmitted via the network 110 is received and reproduced by the display device 160 installed in a restaurant (so-called sports cafe) 107 that can eat and drink while watching sports. At the sports cafe 107, a customer who has the small terminals 171-6 to 171-9 watches a soccer game by looking at the image of the soccer stadium 105 displayed on the display of the display device 160. The small terminals 171-6 to 171-9 operate in the same manner as described above with reference to FIGS.

  In this way, for example, a spectator at a stadium, a user at home, and a customer at a sports cafe can watch the same soccer game and send and receive metadata such as comments. For example, if information indicating the height of emotions in the goal scene is transmitted from each terminal as metadata, and the metadata that aggregates this information is distributed, the sense of unity between users can be improved. Can do. This makes it possible to realize live distribution of content that is even more exciting than conventional soccer watching.

The series of processes described above can be executed by hardware, or can be executed by software. When the above-described series of processing is executed by software, a program constituting the software is installed from a network or a recording medium into a computer incorporated in dedicated hardware. Further, by installing various programs, the program is installed from a network or a recording medium into a general-purpose personal computer 700 as shown in FIG. 18 that can execute various functions.

  In FIG. 18, a CPU (Central Processing Unit) 701 executes various processes according to a program stored in a ROM (Read Only Memory) 702 or a program loaded from a storage unit 708 to a RAM (Random Access Memory) 703. To do. The RAM 703 also appropriately stores data necessary for the CPU 701 to execute various processes.

  The CPU 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output interface 705 is also connected to the bus 704.

  The input / output interface 705 is connected to an input unit 706 composed of a keyboard, a mouse, etc., a display composed of an LCD (Liquid Crystal display), etc., and an output unit 707 composed of a speaker. The input / output interface 705 is connected to a storage unit 708 composed of a hard disk and a communication unit 709 composed of a network interface card such as a modem and a LAN card. The communication unit 709 performs communication processing via a network including the Internet.

  A drive 710 is also connected to the input / output interface 705 as necessary, and a removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is appropriately mounted. Then, the computer program read from these removable media is installed in the storage unit 708 as necessary.

  When the above-described series of processing is executed by software, a program constituting the software is installed from a network such as the Internet or a recording medium such as a removable medium 711.

  The recording medium shown in FIG. 18 is a magnetic disk (including a floppy disk (registered trademark)) on which a program is recorded, which is distributed to distribute the program to the user, separately from the apparatus main body. Removable media consisting of optical disks (including CD-ROM (compact disk-read only memory), DVD (digital versatile disk)), magneto-optical disks (including MD (mini-disk) (registered trademark)), or semiconductor memory It includes not only those configured by 711 but also those configured by a ROM 702 in which a program is recorded, a hard disk included in the storage unit 708, and the like distributed to the user in a state of being incorporated in the apparatus main body in advance.

  Note that the series of processes described above in this specification includes processes that are performed in parallel or individually even if they are not necessarily processed in time series, as well as processes that are performed in time series in the order described. Is also included.

  The embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

  100 streaming distribution system, 101 studio, 102 venue, 131 camera, 132 dedicated terminal, 141 encoder, 142 video distribution server, 143 metadata distribution server, 144 management server, 150 terminal, 160 display device, 161 code image, 171 small terminal

Claims (8)

A management server that obtains, from a plurality of playback devices, playback times specified based on time stamps attached to data of content played back by a plurality of playback devices connected to a network, respectively. And an information processing apparatus that communicates via the network,
Identification information acquisition means for acquiring identification information corresponding to one reproduction device among the plurality of reproduction devices, each of which reproduces content including data obtained by encoding a captured image in real time;
Identification information notifying means for notifying the management server of the identification information of the playback device acquired by the identification information acquiring means in association with its own identification information;
An information processing apparatus comprising: a reproduction time acquisition unit configured to acquire a reproduction time of the reproduction apparatus corresponding to the identification information acquired by the identification information acquisition unit from the management server at a preset time interval. The information according to claim 1, further comprising transmission means for transmitting metadata related to the reproduction time of the content with a time stamp corresponding to the acquired reproduction time to the management server via the network. Processing equipment. The apparatus further comprises metadata playback means for playing back the metadata related to the playback time of the content received via the network at the playback time of the playback device corresponding to the time stamp attached to the metadata. Item 4. The information processing apparatus according to Item 1. The information processing apparatus is possessed by a user who views the content reproduced by the reproduction apparatus,
The information processing apparatus according to claim 1, wherein a plurality of users view content that is played back by one playback apparatus. By the management server,
By comparing the reproduction time notified by each of the plurality of information processing devices and the actual time, the delay time of each of the plurality of information processing devices is identified,
The information processing apparatus according to claim 4, wherein display data for displaying information related to a delay time of each of the plurality of information processing apparatuses is generated. The information processing apparatus according to claim 1, wherein the identification information is displayed as an image on a display of the playback apparatus. A management server that obtains, from a plurality of playback devices, playback times specified based on time stamps attached to data of content played back by a plurality of playback devices connected to a network, respectively. And an information processing method of an information processing apparatus that communicates via the network,
The identification information acquisition means acquires identification information corresponding to one of the plurality of playback devices that plays back each content including data obtained by encoding the captured image in real time,
The identification information notification means notifies the management server of the identification information of the playback device acquired by the identification information acquisition means in association with its own identification information,
An information processing method, comprising: a reproduction time acquisition unit that acquires a reproduction time of the reproduction device corresponding to the identification information acquired by the identification information acquisition unit from the management server at a preset time interval. Computer
A management server that obtains, from a plurality of playback devices, playback times specified based on time stamps attached to data of content played back by a plurality of playback devices connected to a network, respectively. And an information processing apparatus that communicates via the network,
Identification information acquisition means for acquiring identification information corresponding to one reproduction device among the plurality of reproduction devices, each of which reproduces content including data obtained by encoding a captured image in real time;
Identification information notifying means for notifying the management server of the identification information of the playback device acquired by the identification information acquiring means in association with its own identification information;
A program that functions as an information processing device comprising: a playback time acquisition unit that acquires a playback time of the playback device corresponding to the identification information acquired by the identification information acquisition unit from the management server at a preset time interval .

Images