JP2016208488A - Hypervideo reproduction method, system and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide automatic link follow and support navigation for supporting a user in such a manner that the user can watch related parts of hypervideo in an appropriate order.SOLUTION: The hyper video includes a plurality of video segments of a plurality of videos that are linked using a plurality of links. A computation processing system including a central processing unit, a display device and a memory generates a user interface part that reproduces the hypervideo, on the display device and executes the reproduction of the hypervideo in the user interface part by automatically following the plurality of links that link the plurality of video segments in such a manner that at least some of the plurality of video segments are reproduced in the predetermined order. The plurality of links are followed based on a reproduction plan.SELECTED DRAWING: Figure 7

Description

  The present invention relates to a hyper video playback method, system, and program.

  Meetings play an important role in collaborative work, but scheduling is particularly difficult when collaborators are distributed in multiple regions and different time zones. In addition, meeting content often overlaps. Meetings based on what was discussed in the previous meeting are difficult to understand for those who did not attend the previous meeting, and discussing the same topic again is unproductive for those who attended the previous meeting I feel that.

  Meetings that can be joined asynchronously are desired, but currently such meetings are not adequately supported. The system records everything that has the duty to handle the original meeting or to detect and respond to participant discussions as a series of individual discussions, i.e., ignoring interconnections. There are techniques that present solutions that record multiple asynchronous meetings by representing a series of meetings in the form of a navigable video recording called hypervideo. Hypervideo includes multiple video segments that are linked using links.

US Pat. No. 7,483,619

  Hypervideo can be displayed in many different ways. While manual link follow is possible, automatic link follow and other assistive navigation that assists the user to be able to see the relevant parts of the hypervideo in the proper order is desired.

  A first aspect is a method of playing hypervideo, the hypervideo including a plurality of video segments of a plurality of videos linked using a plurality of links, including a central processing unit, a display device, and a memory. A computing system generates a user interface unit for playing back the hyper video on the display device, and links the plurality of video segments so that at least some of the plurality of video segments are played in a predetermined order. By automatically following the plurality of links, the user interface unit performs the playback of the hyper video, and the plurality of links are followed based on a playback plan.

  A second aspect is the method of the first aspect, wherein the playback plan includes at least one rule that follows the plurality of links that link the plurality of video segments while the hyper video is played. .

  A third aspect is the method according to the first aspect, wherein the playback plan is determined based on an interest of a user who watches the playback of the hyper video.

  A fourth aspect is the method according to the first aspect, wherein the playback plan is determined based on an action of a user who watches playback of the hyper video.

  A fifth aspect is the method according to the first aspect, wherein the reproduction plan is selected from a plurality of predetermined reproduction plans based on at least one parameter.

  A sixth aspect is the method according to the first aspect, wherein the playback plan is designated by a user who watches playback of the hyper video.

  A seventh aspect is the method according to any one of the first to seventh aspects, wherein the playback plan is configured to filter the plurality of video segments according to a predetermined criterion.

  An eighth aspect is the method according to the seventh aspect, wherein the playback plan is configured to skip portions of the plurality of video segments that do not have audio.

  A ninth aspect is the method according to any one of the first to eighth aspects, wherein each of the plurality of videos is an audiovisual stream associated with one of a plurality of meetings.

  A tenth aspect is the method according to the ninth aspect, wherein the playback plan is determined based on participation of a user who watches the playback of the hyper video of the plurality of meetings.

  An eleventh aspect is the method according to the ninth aspect, wherein the reproduction plan is configured to reproduce an audiovisual stream corresponding to the entire meeting of the plurality of meetings.

  A twelfth aspect is the method of the eleventh aspect, wherein the replay plan is further configured to link to all of the plurality of meetings following the replayed meeting to be followed. Yes.

  A thirteenth aspect is the method according to the eleventh aspect, wherein the replay plan is configured to link to a meeting immediately after a replayed meeting to be followed among the plurality of meetings.

  A fourteenth aspect is the method of the eleventh aspect, wherein the replay plan is further configured to link to all meetings of the plurality of meetings that precede the replayed meeting to be followed. ing.

  A fifteenth aspect is the method according to the eleventh aspect, wherein the replay plan is configured to link to a meeting immediately before a replayed meeting to be followed among the plurality of meetings.

  A sixteenth aspect is the method of the eleventh aspect, wherein the replay plan is configured such that all links are associated with the replayed meeting to be followed.

  A seventeenth aspect is the method according to the eleventh aspect, wherein the meeting to be reproduced is the oldest meeting among the plurality of meetings.

  An eighteenth aspect is the method according to the eleventh aspect, wherein the meeting to be reproduced is a meeting in which a user who watches the reproduction of the hyper video is absent.

  A nineteenth aspect is the method according to the ninth aspect, wherein the playback plan is determined based on a topic selected by a user who views the playback of the hyper video.

  A twentieth aspect is the method according to the ninth aspect, wherein the reproduction plan is determined based on identification of a speaker in one meeting among a plurality of meetings.

  A twenty-first aspect is a system for playing hypervideo, the hypervideo including a plurality of video segments of a plurality of videos linked using a plurality of links, a central processing unit, a display device, and computer execution A memory for storing the executable instruction set, and when the central processing unit executes the computer-executable instruction set, the central processing unit generates a user interface unit for playing back the hyper video on the display device. The user interface unit generated by automatically following a plurality of links linking the plurality of video segments so that at least some of the plurality of video segments are played in a predetermined order. Perform video playback, and Number of link is followed on the basis of the reproduction plan.

  A twenty-second aspect is a program, wherein the hyper video includes a plurality of video segments of a plurality of videos linked using a plurality of links, and a user interface unit for playing back the hyper video is generated on a display device. The user interface unit generated by automatically following a plurality of links linking the plurality of video segments so that at least some of the plurality of video segments are played in a predetermined order. The reproduction of the hyper video is executed, and the plurality of links are followed based on the reproduction plan to cause the computer to execute a process of reproducing the hyper video.

  The present invention provides support navigation that assists the user to enable automatic link follow and related locations of hypervideo to be viewed in the proper order.

2 is an example of a meeting structure according to one or more embodiments. It is an example of the data structure showing a hyper meeting. It is an example of the table showing a hyper meeting. It is an example of the operation | movement sequence of the process of the recording of the content relevant to a meeting, index addition, and link creation. An example of the operation | movement sequence of the reproduction | regeneration processing of the recorded meeting including synchronous reproduction | regeneration of the content recorded previously is shown. 1 shows an example of a graphical user interface of a system for automatically playing back hypervideo. FIG. 5 illustrates an example of a graphical user interface for recording and playing back a meeting in an example situation that includes three subsequent meeting parts. An example of a timeline of three meeting chains is shown. If the user clicks on a link without enabling automatic link follow or automatic follow on the second link, an example of a replay plan is shown in which both halves of the link are replayed and then stopped. 1 shows an embodiment of a calculation processing system for automatically reproducing hyper video.

  In the following detailed description, reference is made to the accompanying drawings. In the accompanying drawings, components having similar functions are referred to by the same reference numerals. The accompanying drawings are intended to illustrate particular embodiments and implementations consistent with the principles of the invention and are not intended to be limiting. These implementations are described in sufficient detail to enable the invention to be practiced. Other implementations are also available, and structural changes and / or replacement of various components are possible without departing from the scope and spirit of the invention. The following detailed description should not be construed as limiting. Also, the various embodiments of the invention described may be implemented in the form of software running on a general purpose computer, may be implemented in the form of special purpose hardware, or software and hardware It may be implemented in combination.

  In view of the above and other problems associated with the prior art, one or more embodiments described herein implement an approach to automatic playback of hypervideo using a playback plan. According to the approach presented here, the playback plan describes different paths across the hypervideo that are automatically followed during playback. One source of hypervideo is hypermeeting, which supports asynchronous meetings by generating a hypermeetable navigable video recording. For purposes of illustration and not limitation, a system and method for automatically playing back hypervideo is illustrated below using exemplary video content associated with a hypermeeting.

  The example 100 of FIG. 1 illustrates the case where a regular project meeting 101 is held for a project by a distributed team that includes Palo Alto's Andrea 106 and Lynn 107 and College Station's Frank 108. In such cases, remote meeting software is required. Team members also have other projects / tasks and therefore cannot participate in all scheduled meetings. Additional team members are regularly associated with meetings at both sites, and face-to-face meetings associated with the project may occur at one site without coordination with the other site.

  As shown in FIG. 1, the kick-off meeting (meeting 1 indicated by reference numeral 102 in FIG. 1) is performed by Andrea 106, Lynn 107 and Frank 108, each recorded in their own video (Rin and Andrea may join the meeting face-to-face). The next week, Lin 107 has other schedules for normal meeting times, but Andrea 106 and Frank 108 have a meeting, discuss the progress, and record the previous week's meeting to confirm Lin's insights about the design they recalled. (FIG. 1: Meeting 2a: 103). Because the meeting participant plays back the previous week's recording and adds comments about the previous week's recording, a link is automatically generated between the original meeting and the subsequent meeting. Andrea and Frank may also label links to indicate discussion topics. Lin then meets with JD to discuss the project further (Figure 1: Meeting 2b: 104). The meeting 2b is a response set to the meeting 2a initially. In meeting 2b, while using the link generated while meeting 2a is taking place, a new hyperlink is generated, but it also moves to other topics. A topical distributed meeting scheduled in week 3 (Figure 1: Meeting 3: 105) uses topics from recordings and meetings 1, 2a, and 2b when questions are asked, and covers topics for subsequent project meetings. A further link will be generated and made possible to track the discussions.

  As is evident from this example, an important aspect of supporting both meeting participants and meeting absentees is to show the relationship between meetings as new meeting content is recorded. Thus, one or more embodiments described herein may include a plurality of data structures and tables, as illustrated in FIGS. 2A and 2B (not the meeting data structure described in FIG. 1). The metadata describing this content and the recorded content are shown. Specifically, FIGS. 2A and 2B illustrate data structures and tables for hypermeeting. In various embodiments, the exemplary data structure shown in FIGS. 2A and 2B can be in a database such as a relational database well known to those skilled in the art, or any other data structure such as any other data structure. May be stored on a computer readable medium in any suitable form. The concepts described herein are not limited to any particular data structure used to store meeting information. Therefore, the data structure examples shown here should not be construed as limiting.

  According to FIG. 2A, the meeting series 201 represents a recurring meeting or an interrelated meeting. Each of the meetings 202 and 203 of the meeting series 201 includes one or more meeting parts 204, 205, 206, 207, which are synchronous activities that create a logical meeting. It is. Each of the meeting parts 204-207 includes one or more each synchronized video stream 208, 209, 210, 211, 212, 213 (video take) and associated additional data. In various embodiments, meeting parts 204-207 may include a single video stream. For example, one user may record a response to a recorded meeting part of a absent meeting.

  In various embodiments, metadata associated with a meeting series includes a list of participants who have joined each of the meeting parts 204, 205, 206, 207, topics assigned to the corresponding meeting part segment, and within different meeting parts. Or link between different meeting parts. In various embodiments, the meeting part may be segmented based on speaker identification and / or topics being discussed. Techniques for such division are well known to those skilled in the art. In various embodiments, segment identification based on speaker identification and the topic being discussed is used to provide fine-grained access to the meeting record. In various embodiments, automatic, semi-automatic and manual techniques may be used in connection with the meeting parts for segmentation, indexing, and link creation.

According to FIG. 2B, the described system embodiment may use a participant table 220. Participant table 220 stores information regarding each participant in the meeting. This information may be used, for example, for speaker segmentation, which will be described in detail below. Further, a topic table 221 may be provided. This table may contain a list of topics that can be used to index the content of the meeting. Finally, in various embodiments, a link table 222 may be provided. The link table 222 includes a list of links between meeting content. In various embodiments, each of the link records includes information regarding link anchors (start time and end time) and link labels. The tables 220 to 222 may be stored in a database such as a relational database.
"Example Meeting Recording Process"

  FIG. 3 illustrates an operational sequence 300 that illustrates the process of recording, indexing, and creating content related to a meeting. First, at step 301, a meeting is recorded. The recording includes one or more synchronized audiovisual streams. In various embodiments, a meeting may be recorded in a meeting room by capturing audio and video with a camera and microphone, or acquiring audio and video with a camera and microphone of a user's personal device such as a tablet or smartphone. May be recorded. This recorded content is then analyzed at step 302 using known methods for performing speaker splitting. Thereafter, in step 303, the content may be indexed based on the identified speaker. In particular, an index of start time and end time may be generated for each utterance of the speaker in the meeting.

Once the synchronized video content has been segmented based on the speaker (not required, but also indexed), at step 304, assign any of the assigned topics to the segment indicated as overlapping. It is done. In addition, content indexing based on the assigned topic may be performed in step 305. The resulting segmented video stream and attached metadata (eg, topics assigned to segments) are then used during playback (FIG. 4). This generates the next content linked to previously recorded content at step 306.
"Example Meeting Replay Process"

  FIG. 4 illustrates an operational sequence 400 that illustrates the process of playing back a recorded meeting. Playback includes synchronized playback of previously recorded content (using results generated by the process illustrated in FIG. 3). Because multiple users viewing recorded content delivered to different locations need to view the same content at the same time (playback is synchronized), the described system embodiment is performed between people viewing the recording. Get and broadcast all events that change playback. As shown in FIG. 4, during the synchronized playback 401, the described system embodiment captures events 402-405. Event 402 changes the playback speed (increase or decrease speed). Event 403 selects a link. Event 404 selects a topic. Event 405 selects a timeline position. This synchronizes playback of all viewing users to a new position in the same or different video recording set. Further, during synchronized playback 401, one of the viewing users may initiate action 406. Action 406 begins recording activity / discussion among current participants, via process 300 illustrated in FIG. 3, at step 407 of FIG.

In one or more embodiments, the described system automatically creates and presents persistent links between interrelated meeting record segments based on the behavior that a user typically sees and records a meeting. . These generated persistent links are stored for future use. An exemplary user interface will be described first, followed by a description of specific features of the environment.
"Example user interface for meeting recording and playback"

  FIG. 5 illustrates a graphical user interface of a hyper video automatic playback system. Specifically, the graphical user interface 500 of FIG. 5 is a meeting acquisition and review main interface. The upper right portion 501 of the interface 500 allows the user to view and navigate the video stream associated with the playback of previously recorded meetings. The current meeting video stream is displayed below the recorded past meeting video stream, ie, in the lower right portion 502. Video playback of past meetings is a collaborative activity that allows any meeting participant to pause and skip the recorded video of all participants. The left portion 503 of the graphical user interface 500 displays a list of topics 504 associated with the acquired meeting.

  In one or more embodiments, the timeline 505 indicates speaker transitions, for example, with a color code 508. In one or more embodiments, topic tag 506 may be presented below timeline 505. In one or more embodiments, alternative visual information is used for the timeline 505. For example, a color code 508 may be used for the topic and an image may be used to indicate speaker transition. The user can control the playback of the meeting stream using the playback control 507.

  In one or more embodiments, topics are assigned by meeting participants during the initial recording. In the same or different embodiments, topics may also be assigned by users who subsequently watch the video. Additionally or alternatively, topics may be assigned by an automatic mechanism. The automatic mechanism collates elements of a predetermined agenda based on content processing. Initially, a meeting series may have an expandable set of topics. A graphical user interface provides easy access to recently defined and used topics. In addition, meeting participants can be added to the original set of topics through a graphical user interface. The set of topics is stored in a data structure associated with the corresponding meeting, such as the topic table 221 of FIG. 2B.

  The settings that illustrate examples of the described technology include subsequent meeting parts that view previously recorded meeting parts with different participants at different times. Since the participant is watching the previous meeting, pause the previous meeting video stream and start a discussion on the same and related topics. In one or more embodiments, upon receipt of a pause command, the system embodiment automatically generates one or more links between the paused video and the recorded new discussion. In one or more embodiments, the generated link is indicated as the starting point of the timeline.

FIG. 6 illustrates a graphical user interface 600 for recording and playing back an example situation meeting that includes three consecutive meeting parts. The recent meeting part in the lower part 601 of the user interface 600 shows all participants reviewing the previous two meeting parts shown in the upper part 602 and the middle part 603. In the example, link 11 (element 604 in FIG. 6) was recently created in the third part of the meeting. Older links 1-10 (element 605) indicate the first to second part of the meeting. In one or more embodiments, the system allows a user to create a link for an older part of the meeting to the current part. The vertical position of the circle representing the link (elements 604 and 605 in FIG. 6) indicates whether it represents a newer meeting part or an older meeting part. For example, as shown in FIG. 6, links 1-10 of timeline 505 and link 11 of timeline 608 indicate newer meeting parts, but links 1-10 of timeline 608 and link 11 of timeline 609 are Indicates an older meeting part.
"Example link to follow in a meeting"

In various embodiments, the link may be followed bi-directionally by clicking on a circled number that identifies the link (eg, elements 604 and 605 in FIG. 6). Following a link first plays the link anchor for the older meeting to provide context, and then plays the link anchor for the newer meeting. For example, in FIG. 8, after clicking “1” (element 701), first, the starting link anchor 800 near the circle 701 is regenerated, and as indicated by the straight line 803, near “1” (element 802). Followed by destination link anchor 804. Playback stops at the end of the destination link anchor 804. By clicking “1” 802, the same playback sequence is started from the departure link anchor 800. A second click on the link circle proceeds with playback to the destination link anchor.
"Auto link follow"

  Viewing a hypermeeting may be a fully manual interaction in which the user selects which hyperlinks should be followed and which video parts should be skipped. However, it is advantageous to automate navigation through the video based on the system knowledge of the participating user's participation and interest.

  In addition to user-initiated link traversal, links may be traversed automatically upon encounter. Continuous playback of a multi-part meeting can move back and forth between the original video content and the additional video content when a link is encountered. For example, upon encountering a link to a newer meeting, playback automatically links to the original location and after returning to the original location after playing the destination link anchor.

If the linked video contains a link to a more recent meeting, the link will be followed as well. The dashed line 801 in FIG. 8 provides an example of this behavior. While the user is playing the content of the link “1”, the user encounters the link “8” and automatically follows it. Since the end of links “1” and “8” are aligned, playback stops at the end of segment 805 around “8”.
"Example timeline showing topics, participants, links and link labels"

  In one or more embodiments, the timeline 505 is produced or automatically displayed as various visual information and icons to participants based on topics and speakers (eg, circle 605 in FIG. 6). Designed to provide a navigable link between or within the generated records. For example, the topic 606 being discussed may be shown below the timeline 505 in the manner illustrated in FIG. Furthermore, the speaker may be shown using a color code 607 on the timeline 505, as illustrated in FIG.

Finally, the link is indicated by a circled number 605 in the timeline 505. The numbers provide the user with information regarding the destination of the link. As shown in FIG. 6, the link indicated by the circled number 605 on the timeline 505 corresponds to the link having the same number on the timeline 608. In one or more embodiments, additional information regarding the link may be displayed when the mouse cursor is placed over the corresponding icon.
"Hyper video playback plan"

  In one or more embodiments, there are many different ways in which links are automatically followed depending on the information the user needs. According to one embodiment, a replay plan extraction describing automatic behavior is provided. The result of a specific playback plan that applies to a specific hypermeeting is similar to an editing decision for video editing. Includes a sequence of video segments from multiple videos that are played in sequence. Unlike the edit decision list, the playback plan is automatically calculated based on the available links and the user's previous behavior. In one or more embodiments, the replay plan itself encodes the logic that determines whether to follow or not follow the link.

  Some exemplary playback plans are the generation of hypervideo link behavior. The behavior determines what happens if the start or end point of the departure link anchor or destination link anchor is encountered during playback. An alternative playback plan may represent a filter that skips unwanted portions of the video, such as portions without audio. More complex plans may be a combination of behaviors. The playback plan may change the playback speed.

  FIG. 7 shows an example of a timeline of a chain of three meetings. A circle 701 indicates a hyperlink having a number that matches a position where the link is bisected. A colored area 702 around the circle is a link anchor. The superimposed slope line showing the playback progress is the playback plan for the oldest meeting in the chain and automatically follows all links to newer meetings. The link “12” to the newest meeting at 16:52 disturbs the order of playback. After returning from link “9”, link “12” is then followed, followed by links “3”, “10”, “4”, “11”.

  FIG. 8 illustrates a playback plan that plays both halves of link “1” and stops when the user clicks a link that does not have automatic link follow enabled, as shown in element 701 of FIG. . If automatic link follow is enabled, link “8” is played, as shown by dashed line 801.

  In one or more embodiments, all regeneration plans may include filter criteria. The example filter may be configured to skip portions of the video where there is no audio. Such a filter is useful in situations where meeting participants are listening and listening to older meetings at the same time. Future meeting participants may wish to skip periods of no audio. Other exemplary filters may desire to focus on a particular topic or speaker.

In one or more embodiments, the plan may be personalized based on who is speaking in each of the segments. Thus, the exemplary plan may play back any of the responses to the segment for which the current user was the speaker, first playing the original segment followed by the response. Similarly, an exemplary topic-oriented plan may be played through any of the segments tagged with specific topics and the content attached to those segments.
"Navigation within the replay plan"

In one or more embodiments, manual navigation within the playback plan, ie skipping to the part of the video covered by the plan, simply continues the plan from that location in the video. Other user interactions may include clicking on the link label while playing the starting point of the link. Such an interaction skips playback to the destination of the link and plays the plan from there. Navigation to a part of the video that is not covered by the current playback plan defaults to the plan to play the video of the current meeting, and follows a link to all subsequent meetings, so that a new playback that includes that part of the video Lead to plan calculation.
"Regeneration Plan Type"

In one or more embodiments, the replay plan is calculated when the user clicks on a link, navigates to a different meeting in the chain, or navigates outside the current plan. In one or more embodiments, the playback plan may be used to filter recorded content and is interactive (ie, the user can navigate within the playback plan). Identify the appropriate replay plan to use the scenario.

  Table 1 shows details of six exemplary contexts of playback plan usage, user information goals for these contexts, and playback plan behavior. The exemplary context shown in Table 1 is for illustrative purposes and should not be construed as limiting. In one embodiment, the desired system in combination with the audioless part filter is for a more general scenario that includes a link that automatically follows a newer meeting, the entire meeting or a single link. Implement a replay plan.

  In one or more embodiments, selecting a playback plan can take into account user characteristics such as attending a meeting. For users who have not joined the meeting chain, play the video of the meeting that started the chain and automatically follow all links to subsequent meetings so that comments can be viewed in context Is meaningful. Such hypervideo traversal skips parts of subsequent meetings that are not linked. This is because the content of these parts is likely not relevant to older meetings. This is the behavior illustrated in FIG.

  Another exemplary default plan is for those who want to join a meeting series and review comments linked to the meeting they attended. In this case, the departure anchor of the joined meeting is played before playing the linked comment, but any links to previous meetings encountered while playing the departure anchor are ignored.

The technology of reproducing hyper video including a reproduction plan is not limited to hyper video related to hyper meeting, and can be used to reproduce any hyper video content. That is, the example of hypervideo content from a series of asynchronous meetings is for illustrative purposes and should not be construed as limiting other possible types of hypervideo content.
"Speaker split"

  In one or more embodiments, speaker splitting plays a useful role in several parts of the system. Here, it is assumed that each participant has a microphone that is set up so that the audio of the meeting playback or the voice of the speaker at the same location cannot be picked up. That is, the task of dividing speech based on speakers is reduced to determining when each speaker is not silenced. One embodiment uses a low-pass filter and a sampling window for speech energy to remove short noise. One embodiment of the system is configured to compare the speech windows of all participants and request a minimum speech period that allows the speaker to change.

  An example user interface displays a timeline showing when each participant speaks. Thereby, the user can skip the comment by a specific person. The speaker split is visualized as a color code line 607 above the timeline 505 in FIG. In one or more embodiments, there is a “skip silent” widget such as a check box that is configured to allow the user to control whether or not to automatically skip a segment that no one speaks Provided.

In one or more embodiments, speaker splitting is also useful when playing several video streams depicting meeting participants in parallel. If all recorded participants don't wear headphones, audio from earlier meeting replays may be included in the recording. The current echo cancellation system of the web browser removes audio from other terminals in the video conference, but does not remove audio generated by a video player running at the same time. When some of these “damaged” video streams are played in parallel, there is a slight timing shift in echoes that can be suppressed by not muting only the audio of the detected speaker's video player. May cause
"Computer Platform Example"

  FIG. 9 illustrates an embodiment of a computing system for automatic playback of hypervideo, such as hypervideo associated with an asynchronous meeting series. In one or more embodiments, the computing system 900 may be implemented within a well-known desktop computer form factor. In alternative embodiments, the computing system 900 may be implemented based on a laptop, notebook computer, or other mobile computing device such as a smartphone or tablet computer.

  The computing system 900 processes the information, a data bus 904, which is a communication mechanism for communicating or interconnecting the various hardware components of the computing system 900. , May include a central processing unit (CPU or simply processor) 901 that is electrically connected to the data bus 904 for performing other computational and control tasks. The computing system 900 includes a memory 912, such as a random access memory (RAM) or other dynamic storage device, connected to a data bus 904 to store various information and instructions executed by the processor 901. . Memory 912 may include persistent storage devices such as magnetic disks, optical disks, solid state flash memory devices, or other non-volatile solid state storage devices.

  In one or more embodiments, memory 912 may be used to store temporary variables or other intermediate information while executing instructions by processor 901. In addition, the computing system 900 further includes the processor 901, such as static information and firmware necessary for operation of the computing system 900, basic input-output system (BIOS), various configuration parameters of the computing system 900, etc. May include a read-only memory (ROM or EPROM) 902 or other static storage device connected to the data bus 904 to store instructions for storage.

  In one or more embodiments, the computing system 900 may include a display device 911. The display device 911 may be electrically connected to the data bus 904 and displays various information such as a user interface for displaying a hypermeeting part and playing a hypervideo to a user of the computing system 900. . In alternative embodiments, display device 911 may be associated with a graphics controller and / or graphics processor (not shown). The display device 911 may be implemented as a well-known LCD (liquid crystal display), or may be manufactured using TFT (thin-film transistor) or OLED (organic light emitting diode) technology. In various embodiments, the display device 911 may be incorporated into a general housing along with the remaining components of the computing system 900. In an alternative embodiment, the display device 911 may be disposed outside the housing, for example, on a table or desk. In one or more embodiments, the computing system 900 further includes an audio acquisition device 903, such as a microphone, configured to acquire audio components of the meeting and store the audio information recorded in the memory 912. You may go out.

  In one or more embodiments, the computing system 900 is electrically connected to the data bus 904 and, as is well known, is an audio component of a recorded meeting, such as an MPEG-3 file. In addition, an audio playback device 925 may be included that is configured to play audio tracks of various audio files or audio tracks of various video files such as MPEG-4 files. Thus, the computing system 900 may include a waveform (not shown) or speech processor or similar device.

  In one or more embodiments, the computing system 900 communicates direction information and command selections to the processor 901 to control the movement of the cursor on the display device 911, a mouse, trackball, touch pad, or cursor direction. One or more input devices such as a mouse / pointing device 910, such as a key, may be included. Input devices generally have two degrees of freedom in two axes (first axis (eg, x) and second axis (eg, y)), which allows the device to locate on a plane Can do.

  The computing system 900 further includes a camera 926 and a keyboard 906 that capture video and still images of various objects, including meeting videos. Camera 926 and keyboard 906 may be connected to data bus 904 for communicating images, videos, and information including user commands (including gestures) to processor 901. The information is not limited to images, videos, user commands (including gestures) to the processor 901.

  In one or more embodiments, the computing system 900 may further include a communication interface such as a network interface 905 connected to the data bus 904. The network interface 905 connects a connection between the computing system 900 and the Internet 924 to at least one of the WiFi interface 907, the mobile phone network (GSM (registered trademark) or CDMA) adapter 908 and / or the local area network (LAN) adapter 909. May be established using one of them. The network interface 905 may be configured to allow bidirectional data communication between the computing system 900 and the Internet 924. The WiFi adapter 907 operates in accordance with the well-known 802.11a, 802.11b, 802.11g and / or 802.11n protocols and the Bluetooth (registered trademark) protocol. The LAN adapter 909 of the computing system 900 interfaces with the Internet 924 using Internet service provider hardware (not shown), eg, to an integrated services digital network (ISDN) card or a corresponding type of telephone line. A modem may be implemented that provides a data communication connection. As another example, the LAN adapter 909 may be a local area network (LAN) interface card that provides a data communication connection to a compatible LAN and the Internet 924. In the exemplary embodiment, WiFi adapter 907, cellular network (GSM or CDMA) adapter 908, and / or LAN adapter 909 send and receive electrical or electromagnetic signals to represent various types of information. Carries digital data streams.

  In one or more embodiments, the Internet 924 generally provides data communication to other network resources via one or more subnetworks. Accordingly, the computing system 900 can access various network resources located anywhere on the Internet 924, such as remote media servers, web servers, other content servers, and other network data storage resources. In one or more embodiments, the computing system 900 is configured to send and receive messages, media, and other data, including application program code, over various networks, including the Internet 924, via a network interface 905. Is done. In the Internet example, if the computing system 900 operates as a network client, the computing system 900 may request code or data for an application program running on the computing system 900. Similarly, the computing system 900 may send various data or computer code to other network resources.

  In one or more embodiments, the functions described herein are responsive to the processor 901 executing one or more sequences of one or more instructions contained in the memory 912. Such instructions implemented by may be read into memory 912 from other computer-readable media. By executing the instruction sequence contained in memory 912, processor 901 performs the various process steps described herein. In alternative embodiments, hardware circuitry may be used in place of or in combination with software instructions to implement embodiments of the present invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and / or software.

  The term “computer-readable medium” as used herein refers to any medium that is used to provide instructions to processor 901 for execution. A computer-readable medium is an example of a machine-readable medium that may carry instructions for implementing any of the methods and / or techniques described herein. Such a medium may take many forms, including but not limited to, non-volatile media and volatile media.

  Common forms of non-transitory computer readable media include, for example, floppy disks, hard disks, magnetic tapes, or any other magnetic media, CD-ROM, any other optical media, punch cards, paper Tape, physical media with any other hole pattern, RAM, PROM, EPROM, flash EPROM, flash drive, memory card, any other memory chip or cartridge, or any other computer readable Media. Various forms of computer readable media may be used to carry one or more sequences of one or more instructions to executing processor 901. For example, the instructions may first be transferred from a remote computer to a magnetic disk. Alternatively, the remote computer can load instructions into the dynamic memory and send the instructions via the Internet 924. Specifically, computer instructions may be downloaded from a remote computer to the memory of computing system 900 via the Internet 924 using various well-known network data communication protocols.

  In one or more embodiments, the memory 912 of the computing system 900 may store any of the following software programs, applications, or modules.

  1. An operating system (OS) 913. An operating system (OS) 913 implements basic system services and manages various hardware components of the computing system 900. Embodiments of operating system 913 are well known and may include any of the currently known or later developed mobile operating systems.

2. Application 914. The application 914 may include a software application set executed by the processor 901 of the computing system 900, for example. The software application set is executed by the processor 901 to cause the computing system 900 to execute predetermined functions such as displaying a graphical user interface on the display device 911 and recording a video of the meeting using the camera 926. In one or more embodiments, application 914 may include a meeting application 915 that automatically generates and plays hypervideo, as described below.

  3. Data storage 921. The data storage 921 may include a data structure and table 922 that stores various information associated with the hypermeeting, such as illustrated in FIGS. 2A and 2B, for example. Further, the data storage 921 may include media files 923 and index data 927 of the actually recorded media stream corresponding to the meeting.

  In one or more embodiments, the meeting application 915 of the present invention that automatically generates and plays hypervideo is configured to generate one or more user interfaces illustrated in FIGS. A user interface generation module 916. In addition, a meeting acquisition module 917 may be included that acquires video and audio streams using the audio acquisition device 903 and the camera 926. Further, a link generation module 918 that automatically generates a link between video segment parts and stores the generated link in the data storage 921 may be included. In addition, as shown in steps 303 and 305 of FIG. 3, an index module 919 that indexes the speaker and topic information may be included. Finally, a playback module 920 that facilitates automatic playback of the generated hypervideo, for example, according to one of the playback plans shown in Table 1, may be provided.

  Finally, the processes and techniques described herein are not inherently related to any particular apparatus, and may be implemented by any suitable combination of components. In addition, various types of general purpose devices may be used in accordance with the teachings described herein. It is also useful to construct a special purpose device that performs the method described herein. Although the invention has been described in connection with specific examples, the specific examples are intended to be illustrative rather than limiting. Many different combinations of hardware, software, and firmware are suitable for carrying out the present invention. For example, the described software includes various programming languages and script languages such as assembler, C / C ++, Objective-C, Perl, shell, PHP, Java (registered trademark), and programming languages that are currently known or will be developed in the future. Or it can be implemented in a script language.

  Moreover, other implementations of the invention will be apparent from a review of the details of the invention and the practice described herein. Various aspects and / or components of the embodiments described herein may be used alone or in any combination in a system and method for automatically playing back hypervideo according to a playback plan.

901 CPU
903 Audio recording device 911 Display device 912 Memory 924 Internet 926 Camera

Claims (22)

Hypervideo includes multiple video segments of multiple videos linked using multiple links,
A computing system comprising a central processing unit, a display device and a memory,
Generating a user interface unit for reproducing the hyper video on the display device;
By automatically following the plurality of links linking the plurality of video segments such that at least some of the plurality of video segments are played in a predetermined order, Perform playback,
The plurality of links are followed based on a replay plan,
How to play hyper video. The playback plan includes at least one rule that follows the plurality of links linking the plurality of video segments during playback of the hypervideo.
The method of claim 1. The playback plan is determined based on an interest of a user who watches the playback of the hyper video.
The method of claim 1.   The method of claim 1, wherein the playback plan is determined based on an action of a user watching playback of the hypervideo. The playback plan is selected from a plurality of predetermined playback plans based on at least one parameter.
The method of claim 1. The playback plan is specified by a user who watches the playback of the hyper video.
The method of claim 1.   The method according to any one of claims 1 to 6, wherein the playback plan is configured to filter the plurality of video segments according to a predetermined criterion.   The method of claim 7, wherein the playback plan is configured to skip audio free portions of the plurality of video segments. Each of the plurality of videos is an audiovisual stream associated with one of the plurality of meetings;
The method according to any one of claims 1 to 8. The playback plan is determined based on user participation watching the playback of the hypervideo in the plurality of meetings.
The method of claim 9. The playback plan is configured to play an audiovisual stream corresponding to the entire meeting of the plurality of meetings;
The method of claim 9. The replay plan is further configured to link to all meetings of the plurality of meetings that follow the replayed meeting that is followed.
The method of claim 11. The replay plan is configured to link to a meeting of the plurality of meetings immediately following the replayed meeting to be followed;
The method of claim 11. The replay plan is further configured to link to all of the plurality of meetings before the replayed meeting to be followed.
The method of claim 11. The replay plan is configured to link to a meeting immediately before the replayed meeting to be followed of the plurality of meetings;
The method of claim 11. The replay plan is configured so that all links are associated with replayed meetings that are followed.
The method of claim 11. The meeting to be played is the oldest meeting among the plurality of meetings.
The method of claim 11. The meeting to be played is a meeting in which a user who watches the playback of the hyper video is absent.
The method of claim 11. The playback plan is determined based on a topic selected by a user watching the playback of the hypervideo.
The method of claim 9. The playback plan is determined based on speaker identification in one meeting among the plurality of meetings.
The method of claim 9. A system for playing hyper video,
The hyper video includes a plurality of video segments of a plurality of videos linked using a plurality of links;
A central processing unit, a display device, and a memory storing a computer-executable instruction set;
The central processing unit executing the computer-executable instruction set;
The central processing unit is
Generating a user interface unit for reproducing the hyper video on the display device;
The hyper video is generated by the generated user interface unit by automatically following a plurality of links linking the plurality of video segments so that at least some of the plurality of video segments are played in a predetermined order. Perform playback,
The plurality of links are followed based on a replay plan,
A system that plays back hypervideo. Hypervideo includes multiple video segments of multiple videos linked using multiple links,
Generating a user interface unit for reproducing the hyper video on a display device;
The hyper video is generated by the generated user interface unit by automatically following a plurality of links linking the plurality of video segments so that at least some of the plurality of video segments are played in a predetermined order. Perform playback,
The plurality of links are followed based on a replay plan,
A program that causes a computer to execute a process of playing back a hyper video.