WO2013032461A1 - Organisation de flux vidéo - Google Patents

Organisation de flux vidéo Download PDF

Info

Publication number
WO2013032461A1
WO2013032461A1 PCT/US2011/049928 US2011049928W WO2013032461A1 WO 2013032461 A1 WO2013032461 A1 WO 2013032461A1 US 2011049928 W US2011049928 W US 2011049928W WO 2013032461 A1 WO2013032461 A1 WO 2013032461A1
Authority
WO
WIPO (PCT)
Prior art keywords
video streams
site
videoconference
layout
displaying
Prior art date
Application number
PCT/US2011/049928
Other languages
English (en)
Inventor
Mark E. Gorzynski
Michael D. Derocher
Original Assignee
Hewlett-Packard Development Company, L.P.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hewlett-Packard Development Company, L.P. filed Critical Hewlett-Packard Development Company, L.P.
Priority to PCT/US2011/049928 priority Critical patent/WO2013032461A1/fr
Publication of WO2013032461A1 publication Critical patent/WO2013032461A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/15—Conference systems
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/02—Networking aspects
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/14—Display of multiple viewports

Definitions

  • a videoconference typically involves an exchange of video streams that enables participants to see one another.
  • a videoconference involving many video streams may encounter limitations on the available space for displaying the video streams.
  • FIG. 1 and FIGS. 2A-2B are block diagrams of examples of videoconference architectures.
  • FIGS. 3A-3D are schematic diagrams of video streams that illustrate examples of techniques for arranging video streams.
  • FIG. 4A is a flow chart of an example of a process for arranging a display of video streams for a videoconference.
  • FIGS. 4B-4C are schematic diagrams of video streams that illustrate an example of a technique for arranging a display of the video streams according to the process of FIG. 4A.
  • FIG. 5A is a flow chart of an example of a process for arranging a display of video streams for a videoconference.
  • FIGS. 5B-5D are schematic diagrams of video streams that illustrate examples of techniques for arranging a display of the video streams according to the process of FIG. 4A.
  • FIGS. 6A-6D and FIGS. 7A-7C are schematic diagrams of different layouts for displaying video streams from a video conference that are to be displayed at a site from which participants are participating in the videoconference.
  • FIG. 8 is a flow chart of an example of a process for arranging a layout for displaying video streams for a videoconference.
  • FIG. 9A is a flow chart of an example of a process for arranging a layout for displaying video streams for a videoconference.
  • FIG. 9B is a schematic diagram that illustrates an example layout for displaying video streams for a videoconference defined according to the process of FIG. 9A.
  • FIG. 10 is a block diagram of an example of a computing system.
  • a videoconference may involve participants participating from multiple different sites.
  • a central layout of locations representing the different sites may be defined for the videoconference for, among other purposes, coordinating the exchange of video streams between the participating sites.
  • the sites participating in the videoconference may be represented by one or more different locations within the central layout for the videoconference.
  • Each site participating in the videoconference may generate one or more video streams providing views of the site.
  • some sites participating in the videoconference may have multiple video cameras that each capture substantially non-overlapping images of different participants who are participating in the videoconference from the site. Because the video streams generated by the video cameras at such sites essentially are unique and non-redundant, such sites may be represented by multiple locations within the central layout for the videoconference, each location within the central layout corresponding to a separate one of the video streams generated by the video cameras at such sites.
  • other sites participating in the videoconference may have multiple video cameras that capture substantially the same images (e.g., all of the participants who are participating in the videoconference from the site) but from different perspectives.
  • each of these sites may be represented by a single location within the central layout for the videoconference.
  • Still other sites participating in the videoconference may have only one video camera and, therefore, also may be represented by single locations within the central layout for the videoconference.
  • the central layout for the videoconference may be implemented as an ordered list of video streams.
  • the initial assignment of videos streams to particular positions within the ordered list of the central layout may be arbitrary, the ultimate ordering of the video streams within the list may be important in defining local layouts for displaying video streams at the different sites in a manner that provides the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication between participants participating in the videoconference from different sites.
  • one technique for laying out incoming video streams at a particular site that is participating in a videoconference involves arranging the incoming video streams in a horizontal row according to the ordering of the locations within the central layout to which the video streams correspond. Arranging the incoming video streams at the particular site in this manner may provide the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication between participants participating in the videoconference from different sites.
  • the video streams may be displayed at a relatively large scale (e.g., so that the images of the videoconference participants captured in the video streams are displayed at or near life size proportions) creating an immersive effect that simulates all of the participants being located in the same room.
  • the video streams may be arranged in multiple rows in a "serpentine" fashion according to the ordering of the locations within the central layout to which the video streams correspond.
  • Arranging the incoming video streams in multiple rows in such a "serpentine" fashion at the particular site still may provide the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication with participants participating in the videoconference from other sites that may be provided by arranging the video streams in a single row as described above, while also taking better advantage of the screen real estate available for displaying video streams at the particular site and/or enabling more video streams to be displayed at one time.
  • a hybrid approach may be employed for displaying the video streams in which a relatively small scale, continuous view of the video streams is provided according to the single horizontal row or the "serpentine" arrangements described above at the same time that a select number of the video streams are displayed more prominently (e.g., at larger size or scale) in a featured view.
  • a featured video stream may be displayed in both the continuous view and the featured view at the same time.
  • a featured video stream may not be displayed in the continuous view so long as it remains a featured video stream displayed in the featured view.
  • the position at which the featured video stream is displayed within the featured view may be a function of the position of the featured video stream relative to other video streams in the central layout of video streams for the videoconference.
  • the display of the featured video stream within the featured view may be located as closely as possible to the display of the featured video stream in the continuous view.
  • the display of the featured video streams may be arranged to reflect the relative ordering of the featured video streams within the central layout for the videoconference (e.g., according to the single horizontal row or "serpentine" arrangements described above) just like the video streams displayed in the continuous view.
  • FIG. 1 is a block diagram of an example videoconference architecture. As illustrated in FIG. 1 , the architecture 100 includes a network 102 connecting four different sites 104 that are participating in a videoconference.
  • the sites 104 participating in the videoconference may be connected across network 102 in a peer-to-peer configuration.
  • each site 104 may transmit one or more locally-generated video streams to each other site 104 and receive one or more locally-generated video streams from each other site 104.
  • the sites 104 participating in the videoconference may be connected to each other through one or more multipoint control units (MCUs) (not shown) or other centralized computing devices (not shown) configured to connect or bridge sites participating in a videoconference.
  • MCUs multipoint control units
  • each of the sites 104 may transmit one or more locally-generated video streams to the one or more MCU(s) or other computing device(s) and, thereafter, rely on the MCU(s) or similar computing device(s) to transmit the video streams to the other locations 104.
  • Network 102 may provide direct or indirect communication links between individual ones of sites 104 and/or between individual sites 104 and an MCU or other similar centralized computing device configured to connect or bridge sites participating in a videoconference.
  • Examples of network 102 include the Internet, the World Wide Web, wide area networks (WANs) including corporate intranets, local area networks (LANs) including wireless LANs (WLANs), analog or digital wired or wireless telephone networks, radio, television, cable, satellite, and/or any other delivery mechanisms for carrying data, or any combinations thereof.
  • WANs wide area networks
  • LANs local area networks
  • WLANs wireless LANs
  • analog or digital wired or wireless telephone networks radio, television, cable, satellite, and/or any other delivery mechanisms for carrying data, or any combinations thereof.
  • Sites 104 may be implemented and configured in a variety of different forms and the implementations or configurations of individual sites 104 may be the same or different.
  • site 104(0) is implemented as a dedicated videoconference studio.
  • the other sites 104(1 )-104(3) may or may not be dedicated videoconference studios like site 104(0).
  • one or more of the other sites 104(1 )-104(3) may be personal computing devices (e.g., a desktop computer, a laptop or notebook computer, a netbook computer, a tablet computer, a smartphone, a personal digital assistant, or other computing device) configured to support videoconferencing.
  • personal computing devices e.g., a desktop computer, a laptop or notebook computer, a netbook computer, a tablet computer, a smartphone, a personal digital assistant, or other computing device
  • one or more of the other sites 104(1 )- 104(3) may be dedicated videoconference studios that are implemented or configured differently than the dedicated videoconference studio of site 104(0).
  • one or more of the other locations 104(1 )-104(3) may be dedicated videoconference studios having more than one display device and/or more than one video camera.
  • the dedicated videoconference studio at site 104(0) has a display device 106 for displaying incoming videoconference streams from the other sites 104(1 )- 104(3) participating in the videoconference.
  • site 104(0) also includes a video camera 108 for capturing a videoconference video stream of local videoconference participants (e.g., local videoconference participant 110) to be transmitted to the other sites 104(1 )-104(3) participating in the videoconference.
  • local videoconference participant 110 e.g., local videoconference participant 110
  • one or more computing devices receive incoming video streams from sites 104(1 )-104(3) and arrange the display of the received video streams for display device 106 at site 104(0). These computing devices also may coordinate the transmission of the local videoconference video stream captured by camera 108 to the other locations 104(1 )-104(3), for example via a peer-to-peer architecture or via one or more MCUs or other computing devices.
  • a central layout 112 for example in the form of locations around a virtual conference table, is defined for the videoconference.
  • each site 104 participating in the videoconference is represented by a location in the central layout 1 12.
  • Central layout 1 12 may be implemented as an ordered list of these locations representing the sites 104(0)-104(3) participating in the videoconference and reflecting virtual spatial relationships between each of the sites 104(0)-104(3) participating in the videoconference.
  • Such an ordered list may be realized using any of a number of different types of data structures, including, for example, an array, a vector, or a table.
  • a centralized computing device configured to manage the videoconference may define central layout 1 12.
  • a computing device (not shown) located at or associated with a particular one of the sites 104 may define central layout 1 12.
  • the central layout 112 defined for the videoconference may be used to define local layouts for displaying video streams from sites 104 participating in the videoconference in a manner that simulates the participants at the different sites 104 sitting around a conference table.
  • the local layout for each site 104 may be unique reflecting the unique position of the site at the virtual conference table as defined by the central layout 1 12.
  • the local layout for site 104(1 ) may be arranged to create the appearance that (i) the participant at site 104(2) is located to the left of the participant at site 104(1 ), (ii) the participant at site 104(0) is located to the right of the participant at site 104(1 ), and (iii) the participant at site 104(3) is located across from the participant at site 104(1 ).
  • the local layout for site 104(3) may be arranged to create the appearance that (i) the participant at site 104(0) is located to the left of the participant at site 104(3), (ii) the participant at site 104(2) is located to the right of the participant at site 104(3), and (iii) the participant at site 104(1 ) is located across from the participant at site 104(3).
  • FIG. 1 illustrates an example of a local layout for site 104(0).
  • the local layout at site 104(0) for displaying video streams from the other sites 104(1 )-104(3) participating in the videoconference is arranged in a single horizontal row of video streams.
  • the local layout at site 104(0) simulates that the videoconference participants at the other sites 104(1 )-104(3) are sitting around a conference table from the local participant 1 10 at site 104(0) in the arrangement defined by the central layout 112 for the videoconference.
  • the local layout at site 104(0) creates the appearance to the local participant 1 10 at site 104(0) that (i) the participant at site 104(1 ) is located to his left, (ii) the participant at site 104(3) is located to his right, and (iii) the participant at site 104(2) is located across from him.
  • This illusion at site 104(0) that the participants at the other sites 104(1 )- 104(3) participating in the videoconference are sitting around the conference table in the order defined by the central layout 112 is accomplished by arranging the display at site 104(0) of the video streams from the other sites 104(1 )-104(3) in a manner that is based on the order defined by the central layout 112.
  • the local layout for displaying video streams at any particular site 104 participating in the videoconference may be defined by one or more computing devices (not shown) located at or associated with the particular site 104.
  • the local layout at any particular site 104 participating in the videoconference may be defined by one or more centralized computing devices (not shown) configured to manage the videoconference.
  • an ordered list of the video streams from the other sites 104(1 )-104(3) to be displayed at site 104(0) may be derived from the central layout 112. This may be accomplished by removing those locations that represent site 104(0) from the central layout 1 12.
  • the video streams corresponding to the remaining sites 104(1 )-104(3) represented as locations in the central layout 1 12 then may be assigned to positions in the ordered list of video streams to be displayed at site 104(0) in the same order as the sites 104 to which they correspond are represented in the central layout 1 12.
  • the ordered list of the video streams to be displayed at site 104(0) then may be used to assign individual video streams to particular positions for display at site 104(0).
  • the videoconference video stream corresponding to site 104(1 ) is displayed on the left
  • the videoconference video stream corresponding to site 104(2) is displayed in the middle
  • the videoconference video stream corresponding to site 104(3) is displayed on the right on display device 106.
  • the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication with participants participating in the videoconference at the other sites 104(1 )-104(3) may be achieved.
  • an immersive effect may be created that, when coupled with the illusion of being able to maintain eye contact with the other participants in the videoconference, creates the feeling that the other participants are located in the same room.
  • FIGS. 2A-2B are block diagrams of an example videoconference architecture 200.
  • the architecture 200 includes a network 202 connecting seven different sites 204 that are participating in a videoconference instead of only four sites 104 as in the architecture 100 of FIG. 1.
  • different techniques may be employed for defining layouts for displaying video streams at individual sites participating in the videoconference than those described above in connection with FIG. 1.
  • the sites 204 participating in the videoconference may be connected across network 202 in a peer-to-peer configuration.
  • each site 204 may transmit one or more locally-generated video streams to each other site 204 and receive one or more locally-generated video streams from each other site 204.
  • the sites 204 participating in the videoconference may be connected to each other through one or more multipoint control units (MCUs) (not shown) or other centralized computing devices (not shown) configured to connect or bridge sites participating in a videoconference.
  • MCUs multipoint control units
  • each of the sites 204 may transmit one or more locally-generated video streams to the one or more MCU(s) or other computing device(s) and, thereafter, rely on the MCU(s) or similar computing device(s) to transmit the video streams to the other locations 204.
  • Network 202 may provide direct or indirect communication links between individual ones of sites 204 and/or between individual sites 204 and an MCU or other similar centralized computing device configured to connect or bridge sites participating in a videoconference.
  • Examples of network 202 include the Internet, the World Wide Web, wide area networks (WANs) including corporate intranets, local area networks (LANs) including wireless LANs (WLANs), analog or digital wired or wireless telephone networks, radio, television, cable, satellite, and/or any other delivery mechanisms for carrying data, or any combinations thereof.
  • WANs wide area networks
  • LANs local area networks
  • WLANs wireless LANs
  • analog or digital wired or wireless telephone networks radio, television, cable, satellite, and/or any other delivery mechanisms for carrying data, or any combinations thereof.
  • Sites 204 may be implemented and configured in a variety of different forms and the implementations or configurations of individual sites 204 may be the same or different.
  • site 204(0) is implemented as a dedicated videoconference studio.
  • the other sites 204(1 )-204(6) may or may not be dedicated videoconference studios like site 204(0).
  • one or more of the other sites 204(1 )-204(6) may be personal computing devices (e.g., a desktop computer, a laptop or notebook computer, a netbook computer, a tablet computer, a smartphone, a personal digital assistant, or other computing device) configured to support videoconferencing.
  • personal computing devices e.g., a desktop computer, a laptop or notebook computer, a netbook computer, a tablet computer, a smartphone, a personal digital assistant, or other computing device
  • one or more of the other sites 204(1 )-204(6) may be dedicated videoconference studios that are implemented or configured differently than the dedicated videoconference studio of site 204(0).
  • one or more of the other locations 204(1 )-204(n) may be dedicated videoconference studios having more than one display device and/or more than one video camera.
  • the dedicated videoconference studio at site 204(0) has a display device 206 for displaying incoming videoconference streams from the other sites 204(1 )- 204(6) participating in the videoconference.
  • site 204(0) also includes a video camera 208 for capturing a videoconference video stream of local videoconference participants (e.g., local videoconference participant 210) to be transmitted to the other sites 204(1 )-204(6) participating in the videoconference.
  • one or more computing devices receive incoming video streams from sites 204(1 )-204(6) and arrange the display of the received video streams for display device 206. These computing devices also may coordinate the transmission of the local videoconference video stream captured by camera 208 to the other locations 204(1 )-204(6), for example via a peer-to-peer architecture or via one or more MCUs or other computing devices.
  • a central layout 212 for example in the form of locations around a virtual conference table, is defined for the videoconference. As illustrated in FIGS. 2A-2B, each site 204 participating in the videoconference is represented by a location in the central layout 212. Central layout 212 may be implemented as an ordered list of these locations representing the sites 204(0)-204(6) participating in the videoconference and reflecting virtual spatial relationships between each of the sites 204(0)-204(6) participating in the videoconference. Such an ordered list may be realized using any of a number of different types of data structures, including, for example, an array, a vector, or a table. In some implementations, a centralized computing device (not shown) configured to manage the videoconference may define central layout 212. Alternatively, in other implementations, a computing device (not shown) located at or associated with a particular one of the sites 204 may define central layout 212.
  • the central layout 212 defined for the videoconference may be used to define local arrangements for displaying video streams from the sites 204 participating in the videoconference in a manner that simulates the participants at the different sites 204 sitting around a conference table.
  • a local arrangement for a site 204 may be unique reflecting the unique position of the site at the virtual conference table as defined by the central layout 212.
  • a local layout for site 204(3) may be set to create the appearance that (i) the participant at site 204(4) and the participant at site 204(5) are located to the left of the participant at site 204(3), (ii) the participant at site 204(1 ) and the participant at site 204(2) are located to the right of the participant at site 204(3), and (iii) the participant at site 204(6) and the participant at site 204(0) are located across from the participant at site 204(3).
  • a local arrangement for site 204(6) may be set to create the appearance that (i) the participant at site 204(0) and the participant at site 204(1 ) are located to the left of the participant at site 204(6), (ii) the participant at site 204(4) and the participant at site 204(5) are located to the right of the participant at site 204(6), and (iii) the participant at site 204(3) and the participant at site 204(4) are located across from the participant at sites 204(6).
  • FIG. 2A illustrates an example of a local arrangement of the video streams for site 204(0).
  • the local arrangement at site 204(0) for displaying video streams from the other sites 204(1 )-204(6) participating in the videoconference includes two rows and three columns of video streams.
  • the local arrangement at site 204(0) simulates that the videoconference participants at the other sites 204(1 )-204(6) are sitting around a conference table from the local participant 210 at site 204(0) in the arrangement defined by the central layout 212 for the videoconference.
  • the local arrangement at site 204(0) creates the appearance to the local participant 210 at site 204(0) that (i) the participant at site 204(1 ) and the participant at site 204(2) are located to his left, (ii) the participant at site 204(5) and the participant at site 204(6) are located to his right, and (iii) the participant at site 204(3) and the participant at site 204(4) are located across from him.
  • This illusion at site 204(0) that the participants at the other sites 204(1 )-204(6) participating in the videoconference are sitting around the conference table in the order defined by the central layout 212 is accomplished by arranging the display at site 204(0) of the video streams from the other sites 204(1 )-204(6) in a manner that is based on the order defined by the central layout 212.
  • a local arrangement for displaying video streams at any particular site 204 participating in the videoconference may be defined by one or more computing devices (not shown) located at or associated with the particular site 204.
  • a local arrangement at any particular site 204 participating in the videoconference may be defined by one or more centralized computing devices (not shown) configured to manage the videoconference.
  • an ordered list of the video streams from the other sites 204(1 )-204(6) to be displayed at site 204(0) may be derived from the central layout 212. This may be accomplished by removing those locations that represent site 204(0) from the central layout 212.
  • the video streams corresponding to the remaining sites 204(1 )-204(6) represented as locations in the central layout 212 then may be assigned to positions in the ordered list of video streams to be displayed at site 204(0) in the same order as the sites 204 to which they correspond are represented in the central layout 212.
  • the ordered list of the video streams to be displayed at site 204(0) then may be used to assign individual video streams to particular cells within the 2x3 local layout for site 204(0) by alternating the assignment of video streams between the bottom and top rows starting with the left-most column and then moving from left to right across the other columns.
  • the videoconference video stream corresponding to site 204(1 ) is displayed in the lower left cell
  • the videoconference video stream corresponding to site 204(2) is displayed in the upper left cell
  • the videoconference video stream corresponding to site 204(3) is displayed in the lower middle cell
  • the videoconference video stream corresponding to site 204(4) is displayed in the upper middle cell
  • the videoconference video stream corresponding to site 204(5) is displayed in the lower right cell
  • the videoconference video stream corresponding to site 204(6) is displayed in the upper right cell.
  • the local layout for displaying video streams at site 204 illustrated in FIG. 2A may provide the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication with participants participating in the videoconference at the other sites 204(1 )-204(6)
  • the smaller scale of the displayed video streams may detract from efforts to provide an immersive environment that simulates all of the participants being gathered in a single room.
  • 2B employs a hybrid approach that displays a combination of a relatively small scale, continuous view 250 of the video streams arranged in a single horizontal row according to the order of the video streams defined by the central layout 212 and a more prominently displayed featured view 252 of the video streams corresponding to sites 204(2) and 204(5).
  • the local participant 210 at site 210 is able to see the video streams for all of the sites 204(1 )-204(6) participating in the videoconference in a manner that provides the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication with the participants participating in the videoconference at the other sites 204(1 )-204(6), while also being able to experience some manner of an immersive effect relative to the participants participating in the videoconference from sites 204(2) and 204(5).
  • the video streams displayed in the featured view 252 may be selected based on the fact that they correspond to designated presenters or other principal participants in the videoconference.
  • Such designation of presenters or other principal participants may be applied to the videoconference globally, or, in some cases, may be applied locally at a site based on, for example, user input.
  • voice-activated switching may be employed to select the video streams displayed in the featured view 252 such that the video streams displayed in the featured view 252 change as participants from different sites participating in the video conference speak.
  • the location at which a featured video stream is displayed in the featured view 252 may be determined based on the position of the display of its corresponding video stream within the continuous view 250 and/or based on its position within the order for the videoconference defined by the central layout 212. For example, as illustrated in FIG. 2B, the instance of the video stream from site 204(2) displayed in the featured view 252 is displayed over the instance of the video stream from site 204(2) displayed in the continuous view 250, and the instance of the video stream from site 204(5) displayed in the featured view 252 is displayed over the instance of the video stream from site 204(5) displayed in the continuous view 250. Moreover, the order of display of the video streams from sites 204(2) and 204(5) in the featured view 252 is consistent with the ordering defined by the central layout 212 relative to site 204(0).
  • the layout for displaying the video streams illustrated in FIG. 2B may provide the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication with participants participating in the videoconference at the other sites 204(1 )-204(6) even if the local participant 210 at location 204(0) looks at the instances of the video streams of sites 204(2) and 204(5) displayed in the featured view 252 instead of the instances of the video streams of sites 204(2) and 204(5) displayed in the continuous view 250.
  • FIG. 2B illustrates the video streams displayed in the continuous view 250 as being arranged in a single horizontal row
  • a "serpentine" arrangement for example as described above and illustrated in FIG. 2A, may be employed to display the video streams in the featured view 252.
  • FIGS. 3A-3D are schematic diagrams of video streams 302(0)-302(n) corresponding to sites participating in a videoconference that illustrate examples of techniques for arranging a display of the video streams 302(0)-302(n) so as to create an illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication between remotely-located participants in the videoconference.
  • a central layout 304 of the video streams 302(0)- 302(n) corresponding to the participating videoconference sites is defined for the videoconference.
  • This central layout 304 represents the video streams 302(0)-302(n) as locations around a conference table and reflects a global ordering of the video streams 302(0)-302(n) relative to one another.
  • Central layout 304 may be reduced in an elemental form to an ordered list.
  • the initial assignment of the video streams 302(0)-302(n) to particular locations within the central layout 304 may be arbitrary, but, as described further below, after the central layout 304 has been defined, the ordering of the video streams 302(0)-302(n) within the central layout 304 may be used to define local layouts for displaying video streams and, thus, once set, may be important.
  • the central layout 304 also reflects a unique ordering of the other video streams 302 relative to the particular videoconference video stream 302. Stated differently, the central layout 304 reflects a unique virtual spatial relationship between the video streams representing the other sites relative to the particular site. These unique orderings for each of the individual video streams 302 relative to the other video streams 302 defined by the central layout 304 may be useful in defining local layouts for displaying the video streams 302 that provide the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication between the participants at the different videoconference sites. Furthermore, much like the central layout 304 itself, these unique orderings for each of the individual video streams 302 relative to the other video streams 302 defined by the central layout 304 may be reduced to ordered lists of video streams 302.
  • One technique for arranging video streams 302 at a particular videoconference site in a manner that provides the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication with participants at the other videoconference sites involves arranging the video streams 302 corresponding to the other videoconference sites from left to right in a horizontal row starting with the videoconference video stream 302 that is adjacent to the videoconference video stream(s) 302 corresponding to the particular site in the central layout 304 and progressing clockwise through the remaining video streams 302 in the central layout 304.
  • FIG. 3B This technique for arranging video streams based on the central layout 304 for the videoconference is illustrated in FIG. 3B for a site that is represented by "Location 0" in central layout 304 and that corresponds to videoconference video stream 302(0).
  • video streams 302(1 )-302(n) are arranged in a horizontal row 310.
  • FIG. 3A proceeding clockwise through the central layout 304 from the videoconference video stream 302(0) corresponding to the local site, video stream 302(1 ) is adjacent to the videoconference video stream 302(0).
  • videoconference video stream 302(1 ) is assigned to the leftmost (or first) position in horizontal row 310 and the remaining video streams 302(2 )-302(n) are assigned sequentially from left to right to the remaining positions in horizontal row 310 in the order in which they appear in central layout 304 proceeding clockwise from video stream 302(1 ) to video stream 302(n).
  • video streams at a particular site may be arranged to be displayed in two horizontal rows of stacked vertical columns with the video streams being assigned to positions within the two horizontal rows of stacked vertical columns in a manner that essentially preserves the ordering of the other video streams defined by the central layout 304.
  • FIGS. 3C-3D A technique for arranging video streams from a videoconference in two horizontal rows of stacked vertical columns in a manner that preserves an ordering of the video streams defined by a central layout for the videoconference is illustrated in FIGS. 3C-3D. More particularly, FIGS. 3C-3D follow on FIGS. 3A-3B and illustrate a technique for arranging video streams 302(1 )-302(n) at a site that is represented by "Location 0" in the central layout 304 and that corresponds to videoconference video stream 302(0).
  • every other one of video streams 302(1 )- 302(n) is alternately stacked on top of the videoconference video stream 302 immediately preceding it in the ordering of video streams 302(1 )-302(n) defined by central layout 304 and reflected in FIG. 3B. As such, as illustrated in FIG.
  • videoconference video stream 302(2) is stacked on top of videoconference video stream 302(1 ) in a first vertical column 322, videoconference video stream 302(4) is stacked on top of videoconference video stream 302(3) in a second vertical column 322, and so on and so forth until videoconference video stream 302(n) is stacked on top of the videoconference video stream 302(n-1 ) in the final vertical column 322.
  • videoconference video stream 302(n) is stacked on top of the videoconference video stream 302(n-1 ) in the final vertical column 322.
  • 3D effectively simulates the experience that the videoconference participants at the other sites are sitting around a conference table from the videoconference participants at the local site in the order defined by the central layout 304.
  • the videoconference participants in video streams 302(1 ) and 302(2) appear to be sitting to the left of the videoconference participants at the local site
  • the videoconference participants in video streams 302(3) and 302(4) appear to be sitting across from the videoconference participants at the local site
  • the videoconference participants in video streams 302(n-1 ) and 302(n) appear to be sitting to the right of the videoconference participants at the local site.
  • the local layout provides the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication between videoconference participants at the local site and the other sites.
  • the concept of arranging video streams at a particular location into two horizontal rows of stacked vertical columns as described above in connection with FIGS. 3A-3D may be extended to defining an arrangement for displaying an ordered list of video streams in any number of m rows of n stacked vertical columns by assigning the first m video streams in the ordered list to the first column, the second m video streams in the ordered list to the second column, and so on and so forth until all of the video streams in the ordered list have been assigned to appropriate positions within the mxn local layout.
  • the techniques described above both for generating a central layout for sites participating in a videoconference may be generalized for any number of sites participating in a videoconference and their corresponding video streams.
  • a central layout for sites participating in a videoconference e.g., an ordered list of video streams corresponding to sites participating in the videoconference
  • local arrangements based on the central layout e.g., unique orderings of the video streams corresponding to the sites participating in the videoconference relative to one another
  • Table 1 below may be used to determine unique local orderings of the video streams relative to one another for each of the T video streams:
  • T represents the number of video streams representing sites participating in the videoconference
  • V represents an ordered list of the video streams arranged as ( ⁇ -1 , 0, ... T-2)
  • L represents relative positions within the local ordered list corresponding to any one of the T video streams arranged as (0... T-2)
  • S i:j of any cell in Table 1 may be computed programmatically according to Equation 1 below:
  • FIG. 4A is a flow chart 400 of an example of a process for arranging a display of video streams for a videoconference. More particularly, FIG. 4A is a flow chart 400 of an example of a process for arranging a display of video streams for a videoconference at a site participating in the videoconference based on an ordered list of the video streams to be displayed at the site. The ordered list of the video streams to be displayed at the site may be derived from a central layout defined for the videoconference, for example according to the techniques described above in connection with FIGS.
  • the process illustrated in the flowchart 400 of FIG. 4A may be performed by a computing system such as, for example, computing system 1000 illustrated in FIG. 10 and described in greater detail below. More specifically, the process illustrated in the flowchart 400 of FIG. 4A may be performed by one or more of the processor(s) 1006 of the computing system 1000 as a consequence of executing the instructions 1008 illustrated in FIG. 10 and described in greater detail below.
  • the ordered list of video streams to be displayed at the site for the videoconference is accessed.
  • the ordered list of video streams to be displayed may have been derived from a central layout defined for the videoconference and it may reflect a unique ordering of the video streams to be displayed relative to the site as defined by the central layout for the videoconference.
  • the ordered list of video streams may define an ordering of nine video streams 420 corresponding to nine different sites participating in the videoconference (assuming a 1 :1 ratio between videoconference sites and locations in the central layout).
  • videoconference video stream 420(a) is assigned to position 0 in the local ordering
  • videoconference video stream 420(b) is assigned to position 1 in the local ordering
  • videoconference video stream 420(c) is assigned to position 2 in the local ordering
  • videoconference video stream 420(d) is assigned to position 3 in the local ordering
  • videoconference video stream 420(e) is assigned to position 4 in the local ordering
  • videoconference video stream 420(f) is assigned to position 5 in the local ordering
  • videoconference video stream 420(g) is assigned to position 6 in the local ordering
  • videoconference video stream 420(h) is assigned to position 7 in the local ordering
  • videoconference video stream 420(i) is assigned to position 8 in the local ordering.
  • a number of columns M in which to display the video streams at the site is determined.
  • the number of columns M in which to display the video streams may be determined based on the number of display devices available at the site for displaying the video streams, with each display device available at the site corresponding to a column of video streams to be displayed. That is to say, the number of columns M in which to display the video streams may be set equal to the number of display devices available at the site for displaying the video streams.
  • the number of columns M in which to display the video streams may be determined based on the total width available for displaying the video streams at the site and/or a desired aspect ratio for each of the video streams. Additionally or alternatively, the number of columns M in which to display the video streams may be determined based on user-specified settings. Referring to FIG. 4C, in one example, a determination may be made to display the video streams in three columns 422.
  • the number of video streams to be displayed at the site is divided by M to determine a number of video streams Q to be displayed in each column. For example, referring to FIGS. 4B-4C, if there are nine video streams 420 to be displayed at the site in three columns 422, nine may be divided by three to determine that three video streams 420 should be displayed in each column 422.
  • the video streams to be displayed at the site are assigned sequentially to positions in the M columns in increments of Q video streams based on the ordering defined by the ordered list. For example, the first Q video streams in the ordered list are assigned to the first column, the second Q video streams in the ordered list are assigned to the second column, and so on and so forth until all of the video streams have been assigned to one of the M columns. For example, referring to FIGS. 4B-4C and continuing with the example of displaying nine video streams 420 in three columns 422 of three video streams 420 each, the first three video streams 420(a)-420(c) in the ordered list illustrated in FIG. 4B are assigned to the first column 422(a) illustrated in FIG.
  • the second three video streams 420(d)-420(f) in the ordered list illustrated in FIG. 4B are assigned to the second column 422(b) illustrated in FIG. 4C, and the remaining three video streams 420(g)-420(i) are assigned to the third column 422(c) illustrated in FIG. 4C.
  • the video streams 420 are slotted into positions in the columns 422 to which they are assigned by filling the columns 422 with the video streams 420 from bottom to top in the order defined by the ordered list illustrated in FIG. 4B.
  • different techniques may be used to arrange the video streams 420 in positions within the columns 422 to which they have been assigned.
  • the video streams 420 may be slotted into positions in the columns 422 to which they are assigned by filling the columns 422 with the video streams 420 from top to bottom in the order defined by the ordered list illustrated in FIG. 4B.
  • the video streams 420 may be slotted into positions in the columns 422 to which they are assigned in a manner that does not necessarily preserve a bottom to top or top to bottom ordering of the video streams as defined in the ordered list illustrated in FIG. 4B.
  • FIG. 5A is a flow chart 500 of another example of a process for arranging a display of video streams for a videoconference. More particularly, FIG. 5A is a flow chart 500 of an example of a process for arranging a display of video streams for a videoconference at a site participating in the videoconference based on an ordered list of the video streams to be displayed at the site.
  • the ordered list of the video streams to be displayed at the site may be derived from a central layout defined for the videoconference, for example according to the techniques described above in connection with FIGS. 3A-3D, Table 1 , and Equation 1.
  • the process illustrated in the flowchart 500 of FIG. 5A may be performed by a computing system such as, for example, computing system 1000 illustrated in FIG.
  • the process illustrated in the flowchart 500 of FIG. 5A may be performed by one or more of the processor(s) 1000 of the computing system 1000 as a consequence of executing the instructions 1008 illustrated in FIG. 10 and described in greater detail below.
  • the ordered list of video streams to be displayed at the site for the videoconference is accessed.
  • the ordered list of video streams to be displayed may have been derived from a central layout defined for the videoconference and it may reflect a unique ordering of the video streams to be displayed relative to the site as defined by the central layout for the videoconference.
  • the ordered list of video streams may define an ordering of nine video streams 520 corresponding to nine different locations participating in the videoconference. As illustrated in FIG.
  • videoconference video stream 520(a) is assigned to position 0 in the local ordering
  • videoconference video stream 520(b) is assigned to position 1 in the local ordering
  • videoconference video stream 520(c) is assigned to position 2 in the local ordering
  • videoconference video stream 520(d) is assigned to position 3 in the local ordering
  • videoconference video stream 520(e) is assigned to position 4 in the local ordering
  • videoconference video stream 520(f) is assigned to position 5 in the local ordering
  • videoconference video stream 520(g) is assigned to position 6 in the local ordering
  • videoconference video stream 520(h) is assigned to position 7 in the local ordering
  • videoconference video stream 520(i) is assigned to position 8 in the local ordering.
  • a number of rows M in which to display the video streams at the site is determined. Referring to FIG. 5C, in one example, a determination may be made to display the video streams 520 in three rows 522.
  • the video streams to be displayed at the site are assigned sequentially to each of the M rows in succession based on the ordering defined by the ordered list. For example, each of the first M video streams in the ordered list may be assigned to a different one of the M rows in accordance with the order defined by the ordered list. Thereafter, each of the second M video streams in the ordered list may be assigned to a different one of the M rows in accordance with the order defined by the ordered list and so on and so forth until all of the video streams have been assigned to one of the M rows.
  • the first three video streams 520(a)-520(c) in the ordered list illustrated in FIG. 5B are assigned in sequence to Row 1 522(a), Row 2 522(b), and Row 3 522(c), respectively.
  • this arrangement is repeated for the second three video streams 520(d)-520(f) and the final three video streams 520(g)-520(i) in the ordered list illustrated in FIG. 5B.
  • FIGS. 5B-5C this arrangement is repeated for the second three video streams 520(d)-520(f) and the final three video streams 520(g)-520(i) in the ordered list illustrated in FIG. 5B.
  • the video streams 520 are assigned to positions within the rows 522 to which they are assigned that reflect the ordering defined by the ordered list illustrated in FIG. 5B.
  • the first three video streams 520(a)-520(c) in the ordered list illustrated in FIG. 5B are assigned to the first positions in their respective rows
  • the second three video streams 520(d)-520(f) are assigned to the second positions in their respective rows
  • the final three video streams 520(g)-520(i) are assigned to the final positions in their respective rows.
  • the video streams 520 are stacked in aligned columns within the rows 522 to which they are assigned.
  • the video streams 520 may be assigned to positions within the rows 522 that do not result in the video streams also being aligned in columns.
  • the video streams 520(a)-520(i) may be arranged in multiple staggered rows 530.
  • layouts for displaying video streams for a videoconference at a site participating in the videoconference may be generated from a hybrid combination of a first arrangement of video streams from the video conference that reflects an order derived from a central layout for the video streams of the videoconference and a second arrangement of a subset of the video streams from the video conference that is displayed more prominently than the first arrangement.
  • Various different techniques may be employed to generate such hybrid layouts, and the resultant layouts may take a variety of different forms.
  • FIGS. 6A-6D are schematic diagrams of different layouts 600 for displaying the video streams 602(1 )-602(n) from a videoconference that are to be displayed at a site from which participants are participating in the videoconference.
  • the video streams 602(1 )-602(n) to be displayed at the site may include video streams representing all other remote sites from which participants are participating in the videoconference.
  • the video streams 602(1 )-602(n) to be displayed at the location may represent only some subset of less than all of the remote sites from which participants are participating in the videoconference.
  • an all hands meeting for employees of a large company with numerous, geographically dispersed offices may be conducted via videoconference, and the video streams 602(1 )-602(n) to be displayed at the site may represent only a select group of executives from the company who are scheduled to make remarks during the videoconference.
  • each layout 600 provides a continuous view 604 of video streams 602 from the videoconference arranged in a horizontal row and reflecting a particular ordering derived from an ordered list of video streams from the videoconference (e.g., a central layout of video streams for the videoconference).
  • each layout 600 also provides a featured view 606 of some subset of the video streams 602(1 )-602(n) from the videoconference to be displayed at the site that are displayed more prominently than and above the video streams 602 of the continuous view 604.
  • the subset of video streams of the featured view 606 may represent active or recent speakers in the videoconference and/or certain designated principal participants in the videoconference (e.g., presenters, facilitators, moderators, etc.), and the membership of the video streams of the featured view 606 may be dynamic, changing throughout the course of the videoconference.
  • voice-switching techniques may be employed to promote a video stream of a current speaker in the videoconference to the featured view 606.
  • video streams representing some designated number m of the most recent speakers in the videoconference also may be included in the featured view 606.
  • all of the video streams displayed in the continuous view 604 may be displayed at the same size.
  • the video streams displayed in the continuous view 604 may be displayed at different sizes and/or with different aspect ratios.
  • the video streams of the featured view 606 are displayed at larger sizes than the video streams of the continuous view.
  • each video stream displayed in the featured view may be displayed at the same size.
  • video streams displayed in the featured view 606 may be displayed at different sizes and/or with different aspect ratios.
  • the arrangement of the video streams 602 displayed in the continuous view 604 (e.g., left to right order) is derived (e.g., according to techniques described above) from an ordered list of the video streams from the videoconference (e.g., a central layout of the video streams for the videoconference as described above) and may provide the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication with participants participating in the videoconference at the remote sites corresponding to the video streams of the continuous view 604.
  • the continuous view 604 includes a first instance of each of the video streams 602(1 )-602(n) to be displayed at the site.
  • the continuous view 604 enables a participant to view each of the video streams 602(1 )-602(n) to be displayed at the site continuously throughout the videoconference (or throughout some segment of the videoconference) while also potentially providing the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication with the participants at the remote sites represented by the video streams 602(1 )-602(n); and when the video streams 602(1 )-602(n) to be displayed at the site represent each of the other sites from which participants are participating in the videoconference, the continuous view 604 enables a participant to view continuous video images from all of the other sites from which participants are participating in the videoconference while potentially providing the illusion of
  • instances of video streams included in the featured view 606 may not also be included within the continuous view 604.
  • the instance of the particular video stream included in the continuous view 604 may be removed in favor of displaying an instance of the particular video stream in the featured view 606.
  • the positioning of the particular video stream within the featured view 606 may be determined as a function of the position of the corresponding display of the particular video stream within the continuous view 604. For example, in implementations in which all of the video streams 602(1 )-602(n) to be displayed at the site are included within the continuous view 604, when a particular video stream is selected for inclusion in the featured view 606, the display of the particular video stream within the featured view 606 may be positioned as nearly as possible to the corresponding display of the particular video stream within the continuous view 604.
  • Positioning the display of the particular video stream within the featured view 606 in this fashion may preserve the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication with both participants at the remote site represented by the particular video stream as well as with participants at the other remote sites represented by the video streams to be displayed at the videoconference.
  • the display of a video stream is removed from the continuous view 604 upon its promotion to the featured view 606, the display of the particular video stream within the featured view 606 may be positioned as nearly as possible to the previous position of the corresponding display of the particular video stream within the continuous view 604.
  • multiple video streams may be displayed within the featured view 606 concurrently.
  • the positioning and consequential arrangement of the video streams within the featured view 606 may be derived from the same ordered list of the video streams from which the order of the video streams of the continuous view 604 is derived in addition to or as an alternative to positioning the video streams within the featured view 606 as a function of the positions of their corresponding displays within continuous view 604.
  • a first instance of each of the video streams 602(1 )-602(n) to be displayed at the site is displayed within the continuous view 604 in a horizontal row with a left to right order derived from an ordered list of video streams from the videoconference (e.g., a central layout of video streams for the videoconference).
  • continuous view 604 enables a participant at the site to view each of the video streams 602(1 )- 602(n) to be displayed at the location continuously throughout the videoconference (or some segment of the videoconference) while also potentially providing the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication with the participants at the remote sites represented by the video streams 602(1 )-602(n).
  • a second instance of video stream 602(1 ) is included in featured view 606 and is displayed at a greater scale than the display of the video streams 602(1 )-602(n) in the continuous view 604.
  • the location of the instance of video stream 602(1 ) displayed in the featured view 606 is based on the corresponding location of the instance of video stream 602(1 ) displayed in the continuous view 604.
  • the layout 600(b) illustrated in FIG. 6B is similar to the layout 600(a) illustrated in FIG. 6A, except that video stream 602(n-2) is featured in the featured view 606 instead of video stream 602(1 ).
  • the participant represented by video stream 602(1 ) may have been speaking. Consequently, the videoconference system may have selected video stream 602(1 ) to be featured in featured view 606.
  • the participant represented by video stream 602(n-2) may have been speaking, and the videoconference system, therefore, may have selected video stream 602(n-2) to be featured in featured view 606.
  • the videoconference system may have decided to remove the display of video stream 602(1 ) from featured view 606.
  • the scale of the instance of video stream 602(1 ) displayed in the featured view 606 is greater than the scale of the instance of video stream 602(1 ) displayed in the continuous view 604 and the location of the instance of video stream 602(1 ) displayed in the featured view 606 is near to the corresponding location of the instance of video stream 602(1 ) displayed in the continuous view 604, in layout 600(b), the scale of the instance of video stream 602(n-2) displayed in the featured view 606 is greater than the scale of the instance of video stream 602(n-2) displayed in the continuous view 604 and the location of the instance of video stream 602(n-2) displayed in the featured view 606 is based on the location of the instance of video stream 602(n-2) displayed in the continuous view 604.
  • multiple video streams may be displayed in featured view 606 concurrently.
  • a video stream corresponding to a current speaker and one or more video streams corresponding to recent speakers may be featured in featured view 606 concurrently.
  • multiple video streams corresponding to certain principal participants in the videoconference may be featured in featured view 606 concurrently.
  • a first instance of each of the video streams 602(1 )-602(n) to be displayed at the location is displayed within the continuous view 604 and second instances of both video stream 602(1 ) and video stream 602(n-2) are displayed in featured view 606.
  • the instances of video streams 602(1 ) and 602(n-2) displayed in the featured view 606 are displayed at greater scales than the instances of video streams 602(1 ) and 602(n-2) displayed in the continuous view 604.
  • the locations of the instances of video streams 602(1 ) and 602(n-2) displayed in the featured view 606 may be based on the corresponding locations of the instances of video streams 602(1 ) and 602(n-2) displayed in the continuous view 604.
  • the ordering of the instances of video streams 602(1 ) and 602(n-2) displayed in the featured view 606 may be derived from the same ordered list from which the order of the display of the video streams 602(1 )-602(n) was derived. As such, within featured view 606, video stream 602(1 ) is displayed to the left of video stream 602(n-2) just as within the continuous view 604 video stream 602(1 ) is displayed to the left of video stream 602(n-2).
  • the locations at which video streams 602(1 ) and 602(n-2) are displayed within featured view 606 may be based on the locations at which video streams 602(1 ) and 602(n-2) were displayed in continuous view 604. Additionally or alternatively, the relative ordering of video streams 602(1 ) and 602(n-2) within featured view 606 may be derived from the same ordered list from which the order of the display of the video streams 602(1 )-602(n) was derived.
  • a layout for displaying video streams for a videoconference at a site is composed from a hybrid combination of a continuous view of video streams from the videoconference that reflects a virtual spatial ordering of the video streams and a featured view of some subset of less than all of the video streams, as described above, these same techniques may be employed in order to arrange the video streams within the continuous view.
  • FIGS. 7A-7C are schematic diagrams of different examples of layouts 700 for displaying video streams 702(1 )-702(n) from a videoconference that are composed from a hybrid combination of a continuous view 704 and a featured view 706 of the video streams 702(1 )-702(n).
  • the video streams 702(1 )-702(n) displayed in the continuous view 704 are arranged in a "serpentine" fashion as described above that reflects an order derived from an ordered list for the video streams of the videoconference (e.g., a central layout of the video streams for the videoconference as described above).
  • each layout 700 also provides a featured view 706 of some subset of the video streams 702(1 )-702(n) from the videoconference to be displayed at the site that are displayed more prominently than and above the video streams 702 of the continuous view 704.
  • a first instance of each of the video streams 702(1 )-702(n) to be displayed at the site is displayed within the continuous view 704 according to a "serpentine" arrangement as described above. Consequently, continuous view 704 enables a participant at the site to view each of the video streams 702(1 )-702(n) to be displayed at the site continuously throughout the videoconference (or through some segment of the videoconference) while also potentially providing the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication with the participants at the remote sites represented by the video streams 702(1 )-702(n).
  • a second instance of video stream 702(2) is included in featured view 706 and is displayed at a greater scale than the display of the video streams 702(1 )- 702(n) in the continuous view 704. Furthermore, as illustrated in FIG. 7A, the location of the instance of video stream 702(1 ) displayed in the featured view 706 is based on the corresponding location of the instance of video stream 702(1 ) displayed in the continuous view 704.
  • the layout 700(b) illustrated in FIG. 7B is similar to the layout 700(a) illustrated in FIG. 7A, except that video stream 702(n-2) is featured in the featured view 706 instead of video stream 702(2). As in the layout 700(a) illustrated in FIG.
  • the scale of the instance of video stream 702(2) displayed in the featured view 706 is greater than the scale of the instance of video stream 702(2) displayed in the continuous view 704 and the location of the instance of video stream 702(2) displayed in the featured view 706 is near to the corresponding location of the instance of video stream 702(2) displayed in the continuous view 704, in layout 700(b), the scale of the instance of video stream 702(n-2) displayed in the featured view 706 is greater than the scale of the instance of video stream 702(n-2) displayed in the continuous view 704 and the location of the instance of video stream 702(n-2) displayed in the featured view 706 is based on the location of the instance of video stream 702(n-2) displayed in the continuous view 704.
  • multiple video streams may be displayed in featured view 706 concurrently.
  • a first instance of each of the video streams 702(1 )-702(n) to be displayed at the site is displayed within the continuous view 704 and second instances of both video stream 702(2) and video stream 702(n-2) are displayed in featured view 706.
  • the instances of video streams 702(2) and 702(n-2) displayed in the featured view 706 are displayed at greater scales than the instances of video streams 702(2) and 702(n-2) displayed in the continuous view 704.
  • the locations of the instances of video streams 702(2) and 702(n-2) displayed in the featured view 706 may be based on the corresponding locations of the instances of video streams 702(2) and 702(n-2) displayed in the continuous view 704.
  • the ordering of the instances of video streams 702(2) and 702(n-2) displayed in the featured view 706 may be derived from the same ordered list from which the order of the display of the video streams 702(1 )-702(n) was derived. As such, within featured view 706, video stream 702(2) is displayed to the left of video stream 702(n-2) just as within the continuous view 704 video stream 702(2) is displayed to the left of video stream 702(n-2).
  • FIG. 8 is a flow chart 800 of an example of a process for arranging a layout for displaying video streams for a videoconference.
  • the process illustrated in the flowchart 800 of FIG. 8 may be performed by a computing system such as, for example, computing system 1000 illustrated in FIG. 10 and described in greater detail below. More specifically, the process illustrated in the flowchart 800 of FIG. 8 may be performed by one or more of the processor(s) 1006 of the computing system 1000 as a consequence of executing the instructions 1008 illustrated in FIG. 10 and described in greater detail below.
  • an ordered list of video streams from the videoconference is accessed.
  • the ordered list may be derived from a central layout defined for the video streams of the videoconference, for example according to techniques described above.
  • positions are designated within the layout for displaying first instances of the video streams to be displayed in a manner that reflects the order of the video streams in the ordered list.
  • positions may be designated for displaying first instances of the video streams within a continuous view provided within the layout, such as, for example, continuous view 604 of FIGS. 6A-6C or continuous view 704 of FIGS. 7A-7C.
  • the layout may specify a number of equally sized positions for displaying video streams within such a continuous view, and the act of designating positions for displaying first instances of the video streams within the continuous view may involve assigning the first instances of the video streams to particular ones of the positions within the continuous view specified by the layout.
  • the first instances of the video streams are displayed according to their designated positions within the layout at 806.
  • a check is performed to determine if any of the video streams have been designated as featured video streams in the videoconference. If none of the video streams are determined to have been designated as featured video streams, this check may be repeated (e.g., periodically at some defined frequency) until one or more of the video streams are determined to have been designated as featured video streams in the videoconference. Alternatively, if the check results in a determination that one or more of the video streams have been designated as featured video streams in the videoconference, the process proceeds to 810, where the position(s) designated for displaying the first instance(s) of the featured video stream(s) are located within the layout.
  • the layout may specify positions for displaying second instances of video streams within a featured view provided within the layout, such as, for example, featured view 606 of FIGS. 6A-6D or featured view 706 of FIGS. 7A-7C, and positions within the featured view that are located relatively near to the location(s) of the first instance(s) of the featured video stream(s) may be identified for displaying the second instance(s) of the featured video stream(s).
  • the position(s) in the featured view that is/are closest to the location(s) of the first instance(s) of the featured view stream(s) is/are identified for displaying the second instance(s) of the featured video streams.
  • the identified position(s) is/are designated as being for displaying the second instance(s) of the featured video stream(s) at 814, and, at 816, the display is updated to include the second instance(s) of the featured video stream(s) at the designated position(s).
  • the process then returns to 808 and performs a check to determine if any new (or additional) video streams have been designated as featured video streams in the videoconference.
  • FIG. 9A is a flow chart 900 of another example of a process for arranging a layout for displaying video streams for a videoconference at a site from which participants are participating in the videoconference, for example, the layout 950 for displaying video streams 952(1 )-952(9) of FIG. 9B.
  • the process illustrated in the flowchart 900 of FIG. 9 may be performed by a computing system such as, for example, computing system 1000 illustrated in FIG. 10 and described in greater detail below. More specifically, the process illustrated in the flowchart 900 of FIG. 9 may be performed by one or more of the processor(s) 1006 of the computing system 1000 as a consequence of executing the instructions 1008 illustrated in FIG. 10 and described in greater detail below.
  • an ordered list of video streams from the videoconference that reflects virtual spatial relationships between the video streams relative to the site at which the video streams are to be displayed is accessed. As described above, this ordered list may be derived from a central layout defined for the videoconference, for example according to techniques described above.
  • an indication of video streams featured in the videoconference is received.
  • a first arrangement is defined for displaying at the site a first subset of the video streams in a manner that preserves the spatial relationships between the video streams of the first subset relative to the site.
  • an arrangement may be defined for displaying the first subset of video streams in a horizontal row in a manner that preserves the spatial relationships between the video streams of the first subset relative to the site, for example, according to the techniques described above in connection with FIGS. 3A and 3B.
  • an arrangement may be defined for displaying the first subset of video streams in multiple horizontal rows in a manner that preserves the spatial relationships between the video streams of the first subset relative to the site, for example, according to the techniques described above in connection with FIGS. 3A-3D.
  • a second arrangement is defined for displaying at the site a second subset of the video streams that includes the featured video streams in a manner that preserves the spatial relationships between the video streams of the second subset relative to the site.
  • an arrangement may be defined for displaying the second subset of video streams in a horizontal row in a manner that preserves the spatial relationships between the video streams of the second subset relative to the site, for example, according to the techniques described above in connection with FIGS. 3A and 3B.
  • an arrangement may be defined for displaying the second subset of video streams in multiple horizontal rows in a manner that preserves the spatial relationships between the video streams of the second subset relative to the site, for example, according to the techniques described above in connection with FIGS. 3A-3D.
  • a layout is generated for concurrently displaying at the site the video streams of the first subset according to the first arrangement and the video streams of the second subset according to the second arrangement in a manner that emphasizes the video streams of the second subset relative to the video streams of the first subset.
  • the generated layout may specify that the video streams of the second subset be displayed at a larger size than the video streams of the first subset to emphasize the video streams of the second subset relative to the video streams of the first subset.
  • FIG. 9B is a schematic diagram that illustrates an example of a layout 950 for displaying video streams 952(1 )-952(9) for a videoconference at a site defined according to the process of FIG. 9A.
  • Layout 950 provides both continuous 954 and featured 956 views of video streams 952.
  • continuous view 954 includes a first subset of video streams 952(1 )-952(9) from the videoconference, in this case all of the video streams 952(1 )-952(9) to be displayed at the site, arranged in a horizontal row in a manner that reflects virtual spatial relationships between the video streams 952(1 )-952(9) relative to the site.
  • featured view 956 includes a second subset of the video streams 952(1 )-952(9) from the videoconference, namely video streams 952(1 ), 952(2), and 952(7), which, in this case, represent a subset of the first subset of video streams 952(1 )-952(9).
  • video streams 952(1 ), 952(2) and 952(7) are arranged in a horizontal row in a manner that reflects the virtual spatial relationships between video streams 952(1 ), 952(2), and 952(7) relative to the site.
  • video streams 952(1 ), 952(2), and 952(7) are arranged in the same relative left to right order in both continuous view 954 and featured view 956. Furthermore, the video streams 952(1 ), 952(2), and 952(7) of the featured view 956 are displayed in a manner that emphasizes the display of video streams 952(1 ), 952(2), and 952(7) in the featured view 956 relative to the display of the video streams 952(1 )-952(9) in the continuous view 954. In particular, the video streams 952(1 ), 952(2), and 952(7) of the featured view 956 are displayed at a larger size (and larger scale) than the video streams 952(1 )-952(9) of the continuous view 954.
  • FIG. 10 is a block diagram of an example of a computing system 1000 for generating local layout information 1002 for displaying video streams at a site participating in a videoconference based, at least in part, on central layout information 1004 defined for the videoconference (e.g., an ordered list of video streams from the videoconference).
  • computing system 1000 includes one or more processors 1006 and computer-readable storage 1008 storing executable instructions for generating local layout information 1002 for displaying video streams at a site participating in a videoconference from central layout information 1004 defined for the videoconference as described above.
  • Processor(s) 1006 may be central processing units (CPUs), semiconductor-based microprocessors, any other hardware devices suitable for retrieval and execution of instructions stored in computer-readable storage 1008, and/or any combination thereof. Processor(s) 1006 may fetch, decode, and execute instructions stored in computer-readable storage 1008 to implement the functionality for generating local layout information 1004 for displaying video streams at a site participating in a videoconference from central layout information 1002 defined for the videoconference as described above.
  • CPUs central processing units
  • semiconductor-based microprocessors any other hardware devices suitable for retrieval and execution of instructions stored in computer-readable storage 1008, and/or any combination thereof.
  • Processor(s) 1006 may fetch, decode, and execute instructions stored in computer-readable storage 1008 to implement the functionality for generating local layout information 1004 for displaying video streams at a site participating in a videoconference from central layout information 1002 defined for the videoconference as described above.
  • layouts for displaying video streams from a videoconference at a site generally are illustrated in the figures as being displayed on a single display device, layouts arranged according to the techniques described herein may be distributed across multiple display devices at the site.
  • layouts for displaying video streams from a videoconference that are composed from a hybrid combination of a continuous view of one or more horizontal rows of video streams and a featured view of one or more horizontal rows of video streams generally are described and illustrated with row(s) of video streams of the featured view being positioned above the row(s) of video streams of the continuous view, the row(s) of video streams of the featured view instead could be displayed below the row(s) of video streams of the continuous view.
  • the video streams of a featured view generally are described as being displayed at a larger scale than the video streams of a continuous view, in some implementations, the video streams of the featured view may be displayed at substantially the same scale as the video streams of the continuous view, for example based on the screen real estate available.
  • Apparatuses implementing these techniques may include appropriate input and output devices, a computer processor, and/or a tangible computer- readable storage medium storing instructions for execution by a processor.
  • a process implementing techniques disclosed herein may be performed by a processor executing instructions stored on a tangible computer-readable storage medium for performing desired functions by operating on input data and generating appropriate output.
  • Suitable processors include, by way of example, both general and special purpose microprocessors.
  • Suitable computer-readable storage devices for storing executable instructions include all forms of non-volatile memory, including, by way of example, semiconductor memory devices, such as Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), and flash memory devices; magnetic disks such as fixed, floppy, and removable disks; other magnetic media including tape; and optical media such as Compact Discs (CDs) or Digital Video Disks (DVDs). Any of the foregoing may be supplemented by, or incorporated in, specially designed application-specific integrated circuits (ASICs).
  • ASICs application-specific integrated circuits

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

La présente invention se rapporte à des configurations d'affichage de flux vidéo pour une vidéo conférence sur un site. Dans la solution technique décrite dans la présente invention, lesdites configurations d'affichage de flux vidéo sont organisées sur la base, au moins en partie, d'une liste ordonnée de flux vidéo pour la vidéo conférence.
PCT/US2011/049928 2011-08-31 2011-08-31 Organisation de flux vidéo WO2013032461A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/US2011/049928 WO2013032461A1 (fr) 2011-08-31 2011-08-31 Organisation de flux vidéo

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2011/049928 WO2013032461A1 (fr) 2011-08-31 2011-08-31 Organisation de flux vidéo

Publications (1)

Publication Number Publication Date
WO2013032461A1 true WO2013032461A1 (fr) 2013-03-07

Family

ID=47756692

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2011/049928 WO2013032461A1 (fr) 2011-08-31 2011-08-31 Organisation de flux vidéo

Country Status (1)

Country Link
WO (1) WO2013032461A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8804321B2 (en) 2012-05-25 2014-08-12 Steelcase, Inc. Work and videoconference assembly
US9603457B2 (en) 2013-05-31 2017-03-28 Steelcase Inc. Lounge assemblies for supporting portable electronics devices
US20230276018A1 (en) * 2021-04-16 2023-08-31 Zoom Video Communications, Inc. Immersive scenes

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060098086A1 (en) * 2004-11-09 2006-05-11 Nokia Corporation Transmission control in multiparty conference
US20100073454A1 (en) * 2008-09-17 2010-03-25 Tandberg Telecom As Computer-processor based interface for telepresence system, method and computer program product
US20100157017A1 (en) * 2008-12-23 2010-06-24 Tandberg Telecom As Method, device and computer system for processing images in a conference between a plurality of video conferencing terminals
US20110025819A1 (en) * 2008-06-30 2011-02-03 Gorzynski Mark E Compositing Video Streams

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060098086A1 (en) * 2004-11-09 2006-05-11 Nokia Corporation Transmission control in multiparty conference
US20110025819A1 (en) * 2008-06-30 2011-02-03 Gorzynski Mark E Compositing Video Streams
US20100073454A1 (en) * 2008-09-17 2010-03-25 Tandberg Telecom As Computer-processor based interface for telepresence system, method and computer program product
US20100157017A1 (en) * 2008-12-23 2010-06-24 Tandberg Telecom As Method, device and computer system for processing images in a conference between a plurality of video conferencing terminals

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8804321B2 (en) 2012-05-25 2014-08-12 Steelcase, Inc. Work and videoconference assembly
US10786074B2 (en) 2012-05-25 2020-09-29 Steelcase Inc. Work and videoconference assembly
US11185158B1 (en) 2012-05-25 2021-11-30 Steelcase Inc. Work and videoconference assembly
US11612240B1 (en) 2012-05-25 2023-03-28 Steelcase Inc. Work and videoconference assembly
US9603457B2 (en) 2013-05-31 2017-03-28 Steelcase Inc. Lounge assemblies for supporting portable electronics devices
US10188215B2 (en) 2013-05-31 2019-01-29 Steelcase Inc. Lounge assemblies for supporting portable electronic devices
US20230276018A1 (en) * 2021-04-16 2023-08-31 Zoom Video Communications, Inc. Immersive scenes
US11956561B2 (en) * 2021-04-16 2024-04-09 Zoom Video Communications, Inc. Immersive scenes

Similar Documents

Publication Publication Date Title
JP6966421B2 (ja) 角度分離されたサブシーンの合成およびスケーリング
US11330021B1 (en) System and method of mirroring a display of multiple video feeds in videoconferencing systems
DE102015100911B4 (de) Verbesserte Kommunikation zwischen entfernten Teilnehmern/Teilnehmerinnen mittels erweiterter und virtueller Realität
US9438655B2 (en) Video feed layout in video conferences
US8890923B2 (en) Generating and rendering synthesized views with multiple video streams in telepresence video conference sessions
US8957940B2 (en) Utilizing a smart camera system for immersive telepresence
US9113038B2 (en) Distributed video mixing
CN109792562A (zh) 视频数据的基于内容的流分割
US20130305166A1 (en) Presenting data to electronic meeting participants
US11425334B2 (en) Dynamically controlled aspect ratios for communication session video streams
US20120249877A1 (en) Determining video stream display positioning
KR20110050595A (ko) 컴퓨터 판독 가능한 메모리를 구비하는 가상 모임을 위한 컴퓨터, 가상 모임 공통 레이아웃 서비스 제공 방법 및 호스팅 서버
US10917608B1 (en) Dynamically controlled aspect ratios based on a number of participants depicted in communication video streams
CN104822070B (zh) 多路视频流播放方法及装置
US11546385B1 (en) Method and apparatus for self-selection by participant to display a mirrored or unmirrored video feed of the participant in a videoconferencing platform
US12010153B1 (en) Method and apparatus for displaying video feeds in an online meeting user interface in a manner that visually distinguishes a first subset of participants from a second subset of participants
CN104380720B (zh) 视频会议处理方法及设备
US11637991B2 (en) Video conferencing systems featuring multiple spatial interaction modes
US11503085B2 (en) Multimedia composition in meeting spaces
WO2013032461A1 (fr) Organisation de flux vidéo
WO2013062581A1 (fr) Topologie relative à des groupes de participants
US9445052B2 (en) Defining a layout for displaying images
KR20200029062A (ko) 비디오 회의 환경을 위한 비디오 디스플레이 수정 기법
WO2012141707A1 (fr) Visioconférence
WO2013062582A1 (fr) Groupement d'un participant et d'une ressource

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11871561

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11871561

Country of ref document: EP

Kind code of ref document: A1