LU102059B1

LU102059B1 - Securely mixing a primary and secondary audiovisual conversation in an online meeting

Info

Publication number: LU102059B1
Application number: LU102059A
Authority: LU
Inventors: Karthik Mangalampalli
Original assignee: Microsoft Technology Licensing Llc
Priority date: 2020-09-14
Filing date: 2020-09-14
Publication date: 2022-03-14

Abstract

An online meeting that securely mixes a primary audiovisual conversation and a secondary audiovisual conversation. A primary audiovisual conversation is presented based on first data communicated over a primary communications channel established with a plurality of other computer systems. After establishment of the primary communications channel, a secondary communications channel is established with a subset of the plurality of other computer systems, which is less than all of the plurality of other computer systems. A mixed audiovisual conversation is presented at a common meeting user interface, which includes presenting both the primary audiovisual conversation using the first data, and simultaneously presenting the secondary audiovisual conversation using second data communicated over the secondary communications channel. While presenting the mixed audiovisual conversation, the second data of the secondary audiovisual conversation is prevented from being exposed to the primary audiovisual conversation.

Description

408838-LU-NP SECURELY MIXING A PRIMARY AND SECONDARY AUDIOVISUAL CONVERSATION IN AN L/102058

ONLINE MEETING TECHNICAL FIELD

[001] The present disclosure relates to systems, methods, and devices that improve privacy and security during online audiovisual meetings.

BACKGROUND

[002] Computer systems and related technology affect many aspects of society. Indeed, the computer system's ability to process and communicate information has transformed the way we live and work. With the widespread adoption of the Internet, and with many computing devices possessing audiovisual hardware (e.g., video camera, microphone, etc.), online meetings (i.e., videoconferencing) have proliferated, both for personal and business use. While online meetings go far to replicate the experience of in-person group conversations, they fail to provide some features of secure and/or private communications experienced during in-person group conversations.

BRIEF SUMMARY

[003] One feature lacking from contemporary online meetings is the ability to communicate with select meeting participants using visual and/or audible communications, without addressing those communications to the group as a whole. For example, within the context of an in-person group meeting, it is common for subsets of participants (i.e., less than the entirety of the participants) to have a “side conversations” via subtle, secure and/or private communications, such as whisper talk, glances, and other body language. These subtle communications are not presently possible in contemporary online meetings, since any participant communications are broadcast to the entire group.

- Page 1 -

408838-LU-NP

[004] At least some embodiments described herein provide a “whisper mode” for online L/102059 meeting software and/or services, which enables meeting participants to engage in one or more secondary audiovisual conversations within the context of a primary group audiovisual conversation. In embodiments, an online meeting client simultaneously presents a participant in a secondary audiovisual conversation with a mixture of audiovisual data from both the primary audiovisual conversation and the secondary audiovisual conversation, thereby reproducing the in-person experience. However, at least some embodiments described herein use computer technology to actually improve on the in-person experience, by fully shielding the existence and/or content of the secondary audiovisual from the entire group participating in the primary audiovisual conversation. Thus, unlike in-person meetings in which “side conversations” may be observed by unintended participants, the embodiments herein enable participants to have side conversations, while imposing technical barriers to provide privacy—by preventing the existence and/or content of these side conversations from being observed by unintended meeting participants.

[005] In some embodiments, methods, systems, and computer program products are directed to providing an online meeting that securely mixes a primary audiovisual conversation and a secondary audiovisual conversation. In these embodiments, a computer system establishes a primary communications channel that facilitates audiovisual communication between the computer system and a plurality of other computer systems. The computer system corresponds to a meeting participant, and each other computer system corresponds to a different other meeting participant of a plurality of other meeting participants. After establishing the primary communications channel, the computer system presents a primary audiovisual conversation that includes the meeting participant and the plurality of other meeting participants. The primary audiovisual conversation comprises first - Page 2 - ZI.

408838-LU-NP audiovisual data communicated over the primary communications channel. After establishing LU102059 the primary communications channel, the computer system identifies an instruction to establish a secondary audiovisual conversation between the meeting participant and a subset of the plurality of other meeting participants that is less than all of the plurality of other meeting participants. Based on identifying the instruction, the computer system establishes a secondary communications channel between the computer system and a subset of the plurality of other computer systems corresponding to the subset of the plurality of other meeting participants. After establishing the secondary communications channel, the computer system presents a mixed audiovisual conversation that includes, (i) the primary audiovisual conversation which includes the meeting participant and the plurality of other meeting participants, and (ii) the secondary audiovisual conversation which includes the meeting participant and only the subset of the plurality of other meeting participants. The secondary audiovisual conversation comprises second audiovisual data communicated over the secondary communications channel. Presenting the mixed audiovisual conversation includes presenting both a first video presentation of the primary audiovisual conversation and a second video presentation of the secondary audiovisual conversation at a common meeting user interface. While presenting the mixed audiovisual conversation, the computer system prevents exposure of the second audiovisual data to the primary audiovisual conversation.

[006] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. - Page 3 - ——————————————————————————————

408838-LU-NP BRIEF DESCRIPTION OF THE DRAWINGS LU102059

[007] In order to describe the manner in which the above-recited and other advantages and features of the invention can be obtained, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. Understanding that these drawings depict only typical embodiments of the invention and are not therefore to be considered to be limiting of its scope, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:

[008] Figure 1 illustrates an example computer architecture that facilitates providing an online meeting that securely mixes a primary audiovisual conversation and a secondary audiovisual conversation;

[009] Figure 2A illustrates an example of an online meeting user interface that enables creation of a secondary audiovisual conversation;

[010] Figure 2B illustrates an example of an online meeting user interface that securely mixes a primary audiovisual conversation and a secondary audiovisual conversation; and

[011] Figure 3 illustrates a flow chart of an example method for providing an online meeting that securely mixes a primary audiovisual conversation and a secondary audiovisual conversation.

DETAILED DESCRIPTION

[012] A “whisper mode” for online meeting software and/or services enables meeting participants to engage in one or more secondary audiovisual conversations within the context of a primary group audiovisual conversation. In embodiments, this is enabled by an online meeting client that simultaneously presents a participant in a secondary audiovisual conversation with a mixture of audiovisual data from both the primary audiovisual - Page 4 -

ES

408838-LU-NP conversation and the secondary audiovisual conversation—thereby reproducing the in- LU102059 person experience—while also providing a level of privacy and security not possible in the in- person experience.

[013] To the accomplishment of the foregoing, Figure 1 illustrates an example computer architecture 100 that facilitates providing an online meeting that securely mixes a primary audiovisual conversation and a secondary audiovisual conversation. As shown, computer architecture 100 includes a participant computer system 101 that includes a processor 110a, and that uses a communication device 111a to communicate over a network 104 (e.g., a WAN, a LAN, the Internet, etc.) with a plurality of additional participant computer systems 103 (which each includes its own processor 110b and communication device 111b). In some embodiments, the participant computer system 101 and the participant computer systems 103 communicate directly with each other over the network 104 (e.g., using peer-to-peer communications). However, in other embodiments, the participant computer system 101 and the participant computer systems 103 communicate, at least in part, via a server computer system 102 (which includes its own processor 110c and communication device 111c).

[014] Each participant computer system 101/103 enables one or more natural persons to engage in an online meeting as a single meeting “participant.” In one example, a participant computer system is a personal computing device (e.g., smartphone, tablet, personal computer, and the like) that joins to the online meeting using a particular participant account/identifier, and enables one or more natural persons to engage in the online meeting as the particular participant (e.g., using camera(s) and/or microphone(s) of the personal computing device). In another example, a participant computer system is a conference room hub computing device that joins to the online meeting using a particular participant account/identifier, and that and enables one or more natural persons to engage in the online - Page 5 -

408838-LU-NP meeting as the particular participant (e.g., using camera(s) and/or microphone(s) of a LU102059 conference room). As such, as used herein, the term “participant” refers to a single audiovisual source joined to an online meeting (e.g., as a participant account/identifier), and through which one or more natural persons engage with the online meeting.

[015] As shown, the participant computer system 101 includes a meeting client 105a, and each of the participant computer systems 103 also includes a corresponding meeting client 105b. If included, the server computer system 102 includes a meeting server 112 that facilitates operation of the meeting clients 105a/105b. While, for brevity, the embodiments | herein are described primarily in reference to the meeting clients 105a/105b, it will be appreciated that in some implementations these embodiments are actually performed, at least in part, by the meeting server 112 rather than the meeting clients 105a/105b.

[016] As shown, the meeting client 105a includes a primary communications component 106a, a secondary communications component 107a, a security component 108a, and a user interface (Ul) component 109a. Correspondingly, each meeting client 105b also includes a primary communications component 106b, a secondary communications component 107b, a security component 108b, and a Ul component 109b. The primary communications components 106a/106b facilitate creation of a primary communications channel 113, while the primary communications components 107a/107b facilitate creation of one or more secondary communications channels 114. While the primary communications channel 113 and the secondary communications channel(s) 114 are illustrated as arrows between the participant computer system 101 and the participant computer systems 103, it will be appreciated that the primary communications channel 113 and the secondary communications channel(s) 114 could be direct connections between the participant - Page 6 -

408838-LU-NP computer system 101 and the participant computer systems 103, or could be routed through LU102059 | the server computer system 102.

[017] In general, the primary communications components 106a/106b facilitate establishment of (and communications relating to) a primary audiovisual conversation that includes the participant computer system 101 and each of the participant computer systems

103. The second communications components 107a/107b, on the other hand, facilitate establishment of (and communications relating to) one or more secondary audiovisual conversations that each includes the participant computer system 101 and a subset of (i.e., less than all of) the participant computer systems 103. The security components 108a/108b enforce one or more security controls to, at least selectively, isolate any secondary audiovisual conversation(s) from the primary audiovisual conversation. The Ul components 109a/109b provide user one or more meeting client Uls that present the primary audiovisual conversation, that provide the ability to create secondary audiovisual conversations, and that present a mixed experience that simultaneously presents audiovisual data from a secondary audiovisual conversation with audiovisual data from the primary audiovisual conversation.

[018] In embodiments, during presentation of a primary audiovisual conversation conducted over the primary communications channel 113, the Ul components 109a/109b provide one or more Ul elements (e.g., a different button associated with each other participant) that are selectable for establishing a secondary audiovisual conversation over one of secondary communications channel(s) 114 with one or more other participants in the primary conversation. In an example, upon selection of one or more Ul elements presented by user interface component 109a at participant computer system 101, secondary communications component 107a initiates a secondary conversation with one or more of participant computer systems 103 over one of secondary communications channel(s) 114. In - Page 7 - me tr a

408838-LU-NP embodiments, each user interface component 109b at these participant computer systems LU102059 103 displays a prompt for a user to accept or deny the secondary conversation, or automatically accepts or denies the secondary conversation (e.g., based on participant computer system 101, or a user associated therewith, being on an allow list or a deny list). In another example, upon selection of one or more Ul elements presented by user interface component 109b at one of participant computer systems 103, that system’s secondary communications component 107b establishes a secondary conversation with participant computer system 101 over one of secondary communications channel(s) 114. In embodiments, user interface component 109a the displays a prompt for a local user to accept or deny the secondary conversation, or automatically accepts or denies the secondary conversation (e.g., based on a requesting participant computer system 103, or a participant account/identifier associated therewith, being on an allow list or a deny list).

[019] Regardless of whether the secondary conversation was initiated by the participant computer system 101 or one of the participant computer systems 103, in embodiments, the user interface components 109a/109b at each computer system participating in the secondary conversation presents a mixed audiovisual experience that simultaneously presents the primary conversation and the secondary conversation at the computer system. In embodiments, as part of this mixed audiovisual experience, each user interface component 109a/109b deemphasizes one or more of an audio or visual presentation of the primary conversation in some way, as compared to the secondary conversation. In various examples, each user interface component 109a/109b reduces a volume level of the primary conversation as compared to the secondary conversation, reduces a visual size of the primary conversation as compared to the secondary conversation, reduces a vibrancy of the primary conversation as compared to the secondary conversation, presents the primary conversation - Page 8 -

408838-LU-NP in greyscale while presenting the secondary conversation in color, blurs the primary LU102059 conversation, and the like.

[020] Regardless of whether the secondary conversation was initiated by the participant computer system 101 or one of the participant computer systems 103, in embodiments, the security component 108a/108b at each computer system participating in the secondary conversation (and/or a security component within meeting server 112) operates to ensure that the secondary conversation is not leaked to the primary conversation. In one example, each security component 108a/108b operates to ensure that any data communicated over the secondary communications channel(s) 114 is not permitted to be communicated over the primary communications channel 113. In some embodiments, avoiding communication of secondary conversation over the primary communications channel 113 is accomplished by | each security component 108a/108b muting a local microphone and/or video camera with respect to the primary conversation, while permitting the local microphone and/or video camera to be utilized in the secondary conversation. In some embodiments, in addition to muting a local video camera, a security component 108a/108b provides the primary conversation a “false” video stream showing one or more participating users, such as by looping prior captured video of the user(s), or by generating a false video (e.g., using machine learning techniques). In embodiments, even though a local video camera and/or microphone may be muted with respect to the primary conversation, a security component 108a/108b permits a user to unmute one, or both, later. In some embodiments, when a user unmutes a microphone with respect to the primary conversation, a security component 108a/108b mutes at least incoming audio from the secondary conversation to prevent the that audio from leaking to the primary conversation. Additionally, or alternatively, in some embodiments, a security component 108a/108b applies echo cancellation to incoming audio - Page 9 -

408838-LU-NP from the secondary conversation in order to prevent that audio from leaking to the primary LU102059 conversation.

[021] In some embodiments, each meeting client 105a/105b (and/or meeting server 112) permits recording of a meeting, and the security components 108a/108b (and/or a security component within meeting server 112) ensure that any recordings exclude the secondary conversation, even if those recordings are initiated by a computer system participating in the secondary conversation. In embodiments, in addition to ensuring that the secondary conversation is not leaked to the primary conversation, the security components 108a/108b ensure that the very existence of the secondary conversation is hidden from non-participating computer systems, for example by preventing the user interface components 109a/109b at non-participating computer systems from including the secondary conversation in a meeting/participants pane.

[022] In some embodiments, the security components 108a/108b (and/or a security component within meeting server 112) permit one or both of the existence of the secondary conversation, and the data communicated over the secondary communications channel(s) 114, to be exposed to the primary conversation. In some embodiments, a secondary conversation is exposed to the primary conversation from its establishment (i.e., it is created as a public secondary conversation). In other embodiments, a secondary conversation is initially established as a private conversation, and it is later converted to be a public conversation. In embodiments, the security components 108a/108b (and/or a security component within meeting server 112) only permit a conversation to be converted from private to public if all participants in the secondary conversation agree to the conversion. In embodiments, when a secondary conversation is public, the user interface components 109a/109b permit non-participating users to request to be admitted to the secondary - Page 10 -

AL

408838-LU-NP conversation. In embodiments, the security components 108a/108b (and/or a security LU102059 component within meeting server 112) permit recording of a secondary conversation during recording of the primary audiovisual conversation for any duration that the secondary conversation is public.

[023] To facilitate a further understanding of operation of the foregoing embodiments, Figure 2A illustrates an example of an online meeting user interface 200a that enables creation of a secondary audiovisual conversation. In embodiments, the online meeting user interface 200a is generated at the participant computer system 101 by the user interface component 109a during an online meeting involving the participant computer systems 103. The depicted online meeting user interface 200a includes a primary conversation pane 201 and a participants pane 202. In Figure 2A, the primary conversation pane 201 shows five participant portions 203a-203e, including a participant portion 203a presenting a video stream for participant account/identifier “Jack Richards,” a participant portion 203b presenting a video stream for participant account/identifier “Jen Hunter,” a participant portion 203c presenting a video stream for participant account/identifier “Keith Carter,” a participant portion 203d presenting a video stream for participant account/identifier “John Smith,” and a participant portion 203e presenting a video stream for participant account/identifier “Kathy Huston” (which is the participant account/identifier joined at participant computer system 101). These participants are listed a being part of the primary meeting within the participants pane 202. Each participant is associated with a microphone indicator 204 (each showing active) which indicate whether or not the participant’s microphone is muted or active. Each remote participant is also associated with a whisper button 205, which permits a local user (i.e., using the Kathy Huston participant account/identifier) to initiate a secondary conversation with the participant.

- Page 11 - ZZ.

408838-LU-NP

[024] Figure 2B illustrates an example of an online meeting user interface 200b that securely LU102059 mixes a primary audiovisual conversation and a secondary audiovisual conversation. In embodiments, the online meeting user interface 200b is generated at the participant computer system 101 by user the interface component 109a during an online meeting involving the participant computer systems 103, and shows a progression of the online meeting user interface 200b after a user of the Kathy Huston participant has utilized the whisper button 205 associated with the Jen Hunter participant to establish a secondary conversation with the Jen Hunter participant. In the depicted online meeting user interface 200b, the participants pane 202 shows that a private meeting now exists between the Kathy Huston and Jen Hunter participants. The participants pane 202 also shows that the Kathy Huston and Jen Hunter participant are still part of the primary meeting, but that these participant’s microphones have been muted in the primary conversation (but are active in the secondary conversation).

[025] In the online meeting user interface 200b, the primary conversation pane 201 has been reduced in size to make room for a new secondary conversation pane 206. For clarity, the primary conversation pane 201 is labeled in Figure 2B as being public, while the secondary conversation pane 206 is labeled in Figure 2B as being private. Depending on implementation, these labels may, or may not, actually appear in the online meeting user interface 200b. Notably, the primary conversation pane 201 still includes a participant portion 203b’ for the Jen Hunter participant and a participant portion 203e’ for the Kathy Huston participant; however the video streams within the participant portion 203b’ and the participant portion 203e’ have been muted. The secondary conversation pane 206 also includes a participant portion 203b” for the Jen Hunter participant and a participant portion 203e” for the Kathy Huston participant, but the video streams in these participant portions 203b”/203e” are - Page 12 -

408838-LU-NP active. With the configuration shown in the online meeting user interface 200b, users of the LU102059 Kathy Huston and Jen Hunter participants are free to converse, using both audio and video, without fear of that side conversation leaking to the primary conversation.

[026] Although not expressly shown, in some embodiments of the online meeting user interface 200a, the primary conversation pane 201 is deemphasized with respect to the secondary conversation pane 206, such as by making the primary conversation pane 201 visually smaller than the secondary conversation pane 206, by reducing a vibrancy of the primary conversation pane 201 as compared to the secondary conversation pane 206, by presenting at least a portion of the primary conversation pane 201 in greyscale, by blurring at least a portion of the primary conversation pane 201, and the like. In embodiments, an audio volume associated with the public conversation presented in the primary conversation pane 201 is reduced as compared to an audio volume from the private conversation presented in the secondary conversation pane 206. In one example implementation, an audio volume of the public conversation is set to 50%, while an audio volume of the private conversation is set to 100%.

[027] In embodiments, a user of the Kathy Huston participant may unmute that participant's microphone in the primary conversation (e.g., via selection of a respective microphone indicator 204 under the primary meeting in the participants pane 202). In embodiments, a user of the Kathy Huston participant may additionally, or alternatively, unmute that participant's video in the primary conversation. Thus, in some embodiments, based on the user interface component 109a receiving a user input {e.g., from a user of the Kathy Huston participant), at least one of an audio stream or a video stream of the meeting participant is automatically unmuted within the primary audiovisual conversation. In embodiments, if a user of the Kathy Huston participant unmutes that participant’s microphone in the primary - Page 13 -

LLC

408838-LU-NP conversation, the security component 108a automatically mutes and/or applies echo LV102059 cancellation to incoming audio from the Jen Hunter participant in the secondary conversation, in order to prevent that audio from being exposed to the primary conversation; in some embodiments, since the Kathy Huston participant is now actively participating in the primary conversation, the user interface component 109a ceases audible and/or visual deemphasis of the primary conversation in this circumstance.

[028] Although not expressly shown, in some embodiments, the online meeting user interface 200a permits a user of the Kath Huston participant to add one or more of the Keith Carter, John Smith, or Jack Richards participants to the secondary conversation (e.g., via selection of a respective whisper button in participants pane 202). In some embodiments, the online meeting user interface 200a permits the Kathy Huston participant to participate in one or more additional secondary conversations, either by initiating a new secondary conversation, or by accepting an incoming invitation for a secondary conversation. In these embodiments, an additional secondary conversation pane 206 is added for each new secondary conversation. In embodiments, the participants pane 202 permits interaction with secondary meeting settings, such as to change security controls relating to the secondary meeting (e.g., to make the meeting public), to rename the meeting, to add or remove participants, etc.

[029] Although not illustrated, a user interface similar to the online meeting user interface 200b (i.e., including the primary conversation pane 201 and the secondary conversation pane 206) is presented to the Jen Hunter participant, while a user interface similar to online meeting user interface 200b (i.e., including the primary conversation pane 201 but excluding the secondary conversation pane 206) is presented to the Keith Carter, John Smith, and Jack Richards participants. In some embodiments, the participants pane 202 hides the very - Page 14 -

408838-LU-NP existence of the secondary meeting from the Keith Carter, John Smith, and Jack Richards LU102059 participant, such as by presenting a participants pane 202 similar to the one shown in the online meeting user interface 200a. In other embodiments, the participants pane 202 exposes the existence of the secondary meeting to the Keith Carter, John Smith, and Jack Richards participants, but not the content, such as by presenting a participants pane 202 similar to the one shown in the online meeting user interface 200b.

[030] The following discussion new refers to a number of methods and method acts. Although the method acts may be discussed in certain orders, or may be illustrated in a flow chart as occurring in a particular order, no particular ordering is required unless specifically stated, or required because an act is dependent on another act being completed prior to the act being performed.

[031] Figure 3 illustrates a flow chart of an example method 300 for providing an online meeting that securely mixes a primary audiovisual conversation and a secondary audiovisual conversation. Method 300 will be described with respect to the components and data of computer architecture 100, and online meeting user interfaces 200a and 200b.

[032] Method 300 comprises an act 301 of establishing a primary audiovisual communications channel with meeting participants. In some embodiments, act 301 comprises establishing a primary communications channel that facilitates audiovisual communication between the computer system and a plurality of other computer systems, the computer system corresponding to a meeting participant, and each other computer system corresponding to a different other meeting participant of a plurality of other meeting participants. In an example, the primary communications component 106a at the participant computer system 101 establishes the primary communications channel 113 with primary communications components 106b at the participant computer systems 103. The primary - Page 15 - ———————————————————————

408838-LU-NP communications channel 113 is used for a “main” or primary audiovisual conversation with LU102059 participants at the participant computer systems 103. In embodiments, the primary communications channel 113 is established based on initiation of this primary audiovisual conversation.

[033] As discussed in connection with Figure 1, the primary communications channel 113 may be established with the participant computer systems 103 directly (e.g., peer-to-peer), or indirectly via the server computer system 102. Thus, in some embodiments of act 301, establishing the primary communications channel comprises communicating with at least one of the plurality of other computer systems via an intermediary server, or communicating with at least one of the plurality of other computer systems directly.

[034] Method 300 also comprises an act 302 of presenting a primary audiovisual conversation using first audiovisual data communicated over the primary channel. In some embodiments, act 302 comprises, after establishing the primary communications channel, presenting a primary audiovisual conversation that includes the meeting participant and the plurality of other meeting participants, the primary audiovisual conversation comprising first audiovisual data communicated over the primary communications channel. In an example, after the primary communications component 106a establishes the primary communications channel 113 in act 301, the user interface component 109a uses audiovisual data communicated over the primary communications channel 113 to present the primary conversation pane 201 of the online meeting user interface 200a. The primary conversation pane 201 includes a participant portion 203e for the local meeting participant (Kathy Huston), and participant portions 203a-203d of the other remote meeting participants (Keith Carter, John Smith, Jen Hunter, and Jack Richards). - Page 16 - Du as

408838-LU-NP

[035] Method 300 also comprises an act 303 of identifying an instruction to establish a LV102059 secondary audiovisual conversation. In some embodiments, act 303 comprises, after establishing the primary communications channel, identifying an instruction to establish a secondary audiovisual conversation between the meeting participant and a subset of the plurality of other meeting participants that is less than all of the plurality of other meeting participants. In one example, the user interface component 109a identifies an instruction | from a user of the Kathy Huston participant to establish a secondary audiovisual conversation with the Jen Hunter participant. In this example, the instruction is identified—at least in part—through selection of a whisper button 205 corresponding to the Jen Hunter participant within the participants pane 202 of the online meeting user interface 200a. In another example, the secondary communications component 107a identifies a request from a user of the Jen Hunter participant to establish the secondary audiovisual conversation. In this example, the instruction is identified—at least in part—through a communication from one of the participant computer systems 103 (i.e., corresponding to the Jen Hunter participant). Notably, although in either example only a single other participant (Jen Hunter) is being added to the secondary audiovisual conversation with the Kathy Huston participant, multiple other participants could have been added. As such, in some embodiments of act 303, the subset of the plurality of other meeting participants comprises more than one meeting participant.

[036] Notably, when the request to initiate the secondary audiovisual conversation is received from one of the participant computer systems 103, in embodiments, this request is accepted at the participant computer system 101. As discussed, this may be done via a Ul interaction (e.g., an accept/deny prompt). Thus, in some embodiments of act 303, identifying the instruction to establish the secondary audiovisual conversation comprises receiving a user input at the meeting user interface accepting a request to establish the secondary audiovisual - Page 17 -

408838-LU-NP conversation, the request being initiated at one of the subset of the plurality of other LU102059 computer systems. As discussed, the acceptance may alternatively be automated based on the requesting user being on an allow list. Thus, in some embodiments of act 303, identifying the instruction to establish the secondary audiovisual conversation comprises automatically accepting a request to establish the secondary audiovisual conversation based on determining that a meeting participant who initiated the request is on an allow list.

[037] Method 300 also comprises an act 304 of, based on the instruction, establishing a secondary audiovisual communications channel with a subset of the meeting participants. In some embodiments, act 304 comprises, based on identifying the instruction, establishing a secondary communications channel between the computer system and a subset of the plurality of other computer systems corresponding to the subset of the plurality of other meeting participants. In an example, the secondary communications component 107a at the participant computer system 101 establishes a secondary communications channel 114 with secondary communications components 107b at the subset of the participant computer systems 103 that are part of the secondary conversation. This secondary communications channel 114 is used for the secondary audiovisual conversation with the subset of the plurality of other meeting participants identified in act 303.

[038] As discussed in connection with Figure 1, a secondary communications channel 114 may be established with the participant computer systems 103 directly (e.g., peer-to-peer), or indirectly via the server computer system 102. Thus, in some embodiments of act 301, establishing the secondary communications channel comprises communicating with at least one of the plurality of other computer systems via an intermediary server, or communicating with at least one of the plurality of other computer systems directly. - Page 18 -

408838-LU-NP

[039] Method 300 also comprises an act 305 of presenting a mixed audiovisual conversation LU102059 that includes the primary audiovisual conversation, as well as a secondary audiovisual conversation that uses second audiovisual data communicated over the secondary channel. In some embodiments, act 305 comprises, after establishing the secondary communications channel, presenting a mixed audiovisual conversation that includes, (i) the primary audiovisual conversation which includes the meeting participant and the plurality of other meeting participants, and (ii) the secondary audiovisual conversation which includes the meeting participant and only the subset of the plurality of other meeting participants, with the secondary audiovisual conversation comprising second audiovisual data communicated over the secondary communications channel. In act 305, presenting the mixed audiovisual conversation includes presenting both a first video presentation of the primary audiovisual conversation and a second video presentation of the secondary audiovisual conversation at a common meeting user interface. In an example, the user interface component 109a presents online meeting user interface 200a, which includes the primary conversation pane 201. The primary conversation pane 201 presents the primary audiovisual conversation (communicated over primary communications channel 113), and includes all participants (i.e., participant portions 203a’-203e’). The online meeting user interface 200a also includes the secondary audiovisual pane 206, which presents the secondary audiovisual conversation (communicated over the secondary communications channel(s) 114), and which includes only a subset of participants (i.e., participant portions 203b” and 203e” corresponding to the Kathy Huston and Jen Hunter participants).

[040] As explained, in embodiments the primary audiovisual conversation is deemphasized as compared to the secondary audiovisual conversation. Thus, some embodiments of act 305 comprise, while presenting the mixed audiovisual conversation, deemphasizing the primary - Page 19 -

EE A

408838-LU-NP audiovisual conversation as compared to the secondary audiovisual conversation. In LU102059 embodiments, this deemphasis is accomplished by deemphasizing the primary audiovisual conversation’s audio stream as compared to the secondary audiovisual conversation’s audio stream. Thus, in some embodiments, deemphasizing the primary audiovisual conversation as compared to the secondary audiovisual conversation comprises reducing a first audio level of the primary audiovisual conversation as compared to a second audio level of the secondary audiovisual conversation.

[041] In additional, or alternative, embodiments, this deemphasis is accomplished by deemphasizing the primary audiovisual conversation’s visual presentation as compared to a visual presentation of the secondary audiovisual conversation. Thus, in some embodiments, deemphasizing the primary audiovisual conversation as compared to the secondary audiovisual conversation comprises at least one of (i) reducing a first visual size of the first video presentation as compared to a second visual size of the second video presentation, (ii) reducing a first vibrancy of the first video presentation as compared to a second vibrancy of the second video presentation, (iii) presenting the first video presentation in greyscale while presenting the second video presentation in color, (iv) or applying a blur to the first video presentation.

[042] In embodiments, while the online meeting user interface 200a presents the primary conversation pane 201 and the secondary conversation pane 206 within the same application window, in embodiments presenting both the first video presentation and the second video presentation at the common meeting user interface includes presenting the primary and secondary audiovisual conversations in different windows of the same application. Thus, in some embodiments, act 305 comprises presenting the first video presentation and the second video presentation in a single window of the common meeting user interface, while in other - Page 20 -

ANNE

408838-LU-NP | embodiments act 305 comprises presenting the first video presentation and the second video LU102059 presentation in different windows of the common meeting user interface.

[043] As discussed in connection with Figure 2B, in some embodiments the online meeting user interface 200b provides an indication of whether a conversation is public or private, such as the depicted “public” label in primary conversation pane 201 (associated with the primary audiovisual conversation) and the “private” label in secondary conversation pane 206 (associated with the secondary audiovisual conversation). Thus, in some embodiments of act 305, the meeting user interface provides a first indication that the primary audiovisual conversation is a public conversation, and provides a second indication that the secondary audiovisual conversation is a private conversation.

[044] Method 300 also comprises an act 306—which occurs in parallel with act 305—of preventing exposure of the second audiovisual data to the primary audiovisual conversation. In some embodiments, act 306 comprises, while presenting the mixed audiovisual conversation, preventing exposure of the second audiovisual data to the primary audiovisual conversation. In an example, the security component 108a at the participant computer system 101 operates to ensure that the secondary audiovisual conversation (presented within the secondary conversation pane 206) is not exposed to the primary audiovisual conversation (presented within the primary conversation pane 201).

[045] In some embodiments of act 306, preventing exposure of the second audiovisual data to the primary audiovisual conversation comprises preventing exposure of audio data to the primary audiovisual conversation. In these embodiments, preventing exposure of the second audiovisual data to the primary audiovisual conversation in act 306 comprises automatically muting an audio stream of the meeting participant within the primary audiovisual conversation (e.g., by muting the primary user’s microphone in the context of the primary - Page 21 -

408838-LU-NP audiovisual conversation), or preventing audio data received over the secondary LU102059 communications channel from being communicated over the primary communications channel (e.g., by muting the secondary communications audio, by applying echo cancellation to audio communicated in the second audiovisual data, and the like). Additionally, or alternatively, in some embodiments of act 306, preventing exposure of the second audiovisual data to the primary audiovisual conversation comprises preventing exposure of video data to the primary audiovisual conversation. In these embodiments, preventing exposure of the second audiovisual data to the primary audiovisual conversation in act 306 comprises automatically muting a video stream of the meeting participant within the primary audiovisual conversation, and/or preventing visual data received over the secondary communications channel from being communicated over the primary communications channel. In some embodiments of act 306, preventing exposure of the second audiovisual data to the primary audiovisual conversation comprises preventing the secondary audiovisual conversation from being recorded during recording of the primary audiovisual conversation.

[046] In embodiments, a secondary audiovisual conversation may be converted from a private secondary conversation to a public secondary conversation. When a secondary conversation is public, the existence of the secondary audiovisual conversation may be exposed to the entire group (e.g., via participants pane 202) and/or the content of the secondary audiovisual conversation may be permitted to be exposed to the primary audiovisual conversation. Thus, in some embodiments, based on receiving a user input, method 300 includes at least one of (i) permitting second audiovisual data received over the secondary communications channel to be communicated over the primary communications channel, (ii) permitting the secondary audiovisual conversation to be recorded during recording of the primary audiovisual conversation, or (iii) exposing existence of the secondary - Page 22 -

408838-LU-NP audiovisual conversation to the primary audiovisual conversation. In embodiments, when LU102059 existence of the secondary audiovisual conversation is exposed to the primary audiovisual conversation, a may request to join the secondary audiovisual conversation (e.g., via an interaction in the participants pane 202). Thus, in some embodiments, method 300 further comprises receiving a request to join the secondary audiovisual conversation, the request being received by one of the plurality of other computer systems that is not part of the subset of the plurality of other computer systems.

[047] As mentioned, in embodiments the meeting clients 105a/105b permit creation of multiple secondary audiovisual conversations (in addition to the primary audiovisual conversation). Thus, in some embodiments of method 300, the secondary audiovisual conversation comprises a first secondary audiovisual conversation, the secondary communications channel comprises a first secondary communications channel, and the subset of the plurality of other computer systems comprises a first subset of the plurality of other computer systems. In these embodiments, method 300 further comprises establishing a second secondary communications channel with a second subset of the plurality of other computer systems that is less than all of the plurality of other computer systems, the second subset being different than the first subset. In these embodiments, method 300 also comprises presenting a second secondary audiovisual conversation as part of the mixed audiovisual conversation, the second secondary audiovisual conversation including the meeting participant and only the second subset of the plurality of other meeting participants, the second secondary audiovisual conversation comprising third audiovisual data communicated over the second secondary communications channel. Then, while presenting the mixed audiovisual conversation, method 300 comprises preventing participation in the - Page 23 -

408838-LU-NP second secondary audiovisual conversation from being exposed to at least one of the primary LU102059 audiovisual conversation or the first secondary audiovisual conversation.

[048] Accordingly, at least some embodiments described herein provide a “whisper mode” for online meeting software and/or services. This “whisper mode” enables meeting participants to engage in one or more “side” or secondary audiovisual conversations within the context of a primary group audiovisual conversation. By simultaneously presenting a participant in a secondary audiovisual conversation with a mixture of audiovisual data from both the primary audiovisual conversation and the secondary audiovisual conversation, embodiments reproduce in-person “side conversation” experiences. However, embodiments go further to actually improve on the in-person experience, by fully shielding the existence and/or content of the secondary audiovisual conversation from the primary audiovisual conversation. Thus, unlike in-person meetings in which “side conversations” may be observed by unintended participants, the embodiments herein enable participants to have side conversations, while imposing technical barriers to provide privacy. In particular, these technical barriers prevent the existence and/or content of these side conversations from being observed by unintended participants that are part of the primary conversation, but not the secondary conversation.

[049] Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the described features or acts described above, or the order of the acts described above. Rather, the described features and acts are disclosed as example forms of implementing the claims.

[050] Embodiments of the present invention may comprise or utilize a special-purpose or general-purpose computer system that includes computer hardware, such as, for example, - Page 24 - ZN.

408838-LU-NP one or more processors and system memory, as discussed in greater detail below. LU102059 Embodiments within the scope of the present invention also include physical and other computer-readable media for carrying or storing computer-executable instructions and/or data structures. Such computer-readable media can be any available media that can be accessed by a general-purpose or special-purpose computer system. Computer-readable media that store computer-executable instructions and/or data structures are computer storage media. Computer-readable media that carry computer-executable instructions and/or data structures are transmission media. Thus, by way of example, and not limitation, embodiments of the invention can comprise at least two distinctly different kinds of computer-readable media: computer storage media and transmission media.

[051] Computer storage media are physical storage media that store computer-executable instructions and/or data structures. Physical storage media include computer hardware, such as RAM, ROM, EEPROM, solid state drives (“SSDs”), flash memory, phase-change memory (“PCM”), optical disk storage, magnetic disk storage or other magnetic storage devices, or any other hardware storage device(s) which can be used to store program code in the form of computer-executable instructions or data structures, which can be accessed and executed by a general-purpose or special-purpose computer system to implement the disclosed functionality of the invention.

[052] Transmission media can include a network and/or data links which can be used to carry program code in the form of computer-executable instructions or data structures, and which can be accessed by a general-purpose or special-purpose computer system. A “network” is defined as one or more data links that enable the transport of electronic data between computer systems and/or modules and/or other electronic devices. When information is transferred or provided over a network or another communications connection - Page 25 - en rt

408838-LU-NP (either hardwired, wireless, or a combination of hardwired or wireless) to a computer system, LU102059 the computer system may view the connection as transmission media. Combinations of the above should also be included within the scope of computer-readable media.

[053] Further, upon reaching various computer system components, program code in the form of computer-executable instructions or data structures can be transferred automatically from transmission media to computer storage media (or vice versa). For example, computer- executable instructions or data structures received over a network or data link can be buffered in RAM within a network interface module (e.g., a “NIC”), and then eventually transferred to computer system RAM and/or to less volatile computer storage media at a computer system. Thus, it should be understood that computer storage media can be included in computer system components that also (or even primarily) utilize transmission media.

[054] Computer-executable instructions comprise, for example, instructions and data which, when executed at one or more processors, cause a general-purpose computer system, special-purpose computer system, or special-purpose processing device to perform a certain function or group of functions. Computer-executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, or even source code.

[055] Those skilled in the art will appreciate that the invention may be practiced in network computing environments with many types of computer system configurations, including, personal computers, desktop computers, laptop computers, message processors, hand-held devices, multi-processor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile telephones, PDAs, tablets, pagers, routers, switches, and the like. The invention may also be practiced in distributed system environments where local and remote computer systems, which are linked - Page 26 -

408838-LU-NP (either by hardwired data links, wireless data links, or by a combination of hardwired and 0102059 wireless data links) through a network, both perform tasks. As such, in a distributed system environment, a computer system may include a plurality of constituent computer systems. In a distributed system environment, program modules may be located in both local and remote memory storage devices.

[056] Those skilled in the art will also appreciate that the invention may be practiced in a cloud computing environment. Cloud computing environments may be distributed, although this is not required. When distributed, cloud computing environments may be distributed internationally within an organization and/or have components possessed across multiple organizations. In this description and the following claims, “cloud computing” is defined as a model for enabling on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services). The definition of “cloud computing” is not limited to any of the other numerous advantages that can be obtained from such a model when properly deployed.

[057] A cloud computing model can be composed of various characteristics, such as on- demand self-service, broad network access, resource pooling, rapid elasticity, measured service, and so forth. À cloud computing model may also come in the form of various service models such as, for example, Software as a Service (“SaaS”), Platform as a Service (“PaaS”), and Infrastructure as a Service (“laaS”). The cloud computing model may also be deployed using different deployment models such as private cloud, community cloud, public cloud, hybrid cloud, and so forth.

[058] Some embodiments, such as a cloud computing environment, may comprise a system that includes one or more hosts that are each capable of running one or more virtual machines. During operation, virtual machines emulate an operational computing system, - Page 27 -

408838-LU-NP supporting an operating system and perhaps one or more other applications as well. In some LU102059 embodiments, each host includes a hypervisor that emulates virtual resources for the virtual machines using physical resources that are abstracted from view of the virtual machines. The hypervisor also provides proper isolation between the virtual machines. Thus, from the perspective of any given virtual machine, the hypervisor provides the illusion that the virtual machine is interfacing with a physical resource, even though the virtual machine only interfaces with the appearance (e.g., a virtual resource) of a physical resource. Examples of physical resources including processing capacity, memory, disk space, network bandwidth, media drives, and so forth.

[059] The present invention may be embodied in other specific forms without departing from its essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope. When introducing elements in the appended claims, the articles “a,” “an,” “the,” and “said” are intended to mean there are one or more of the elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that there may be additional elements other than the listed elements. Unless otherwise specified, the terms “set” and “subset” are indented to exclude an empty set, and thus “set” and is defined as a non-empty set, and “subset” is defined as a non-empty subset. Additionally, unless otherwise specified, the term “subset” excludes the entirety of its superset (i.e., the superset contains at least one item not included in the subset). - Page 28 - ER ———

Claims

408838-LU-NP CLAIMS LU102059 What is claimed:

1. A method (300), implemented at a computer system (101) that includes a processor (110a), for providing an online meeting that securely mixes a primary audiovisual conversation and a secondary audiovisual conversation, the method comprising: establishing (106a) a primary communications channel (113) that facilitates audiovisual communication between the computer system and a plurality of other computer systems (103), the computer system corresponding to a meeting participant, and each other computer system corresponding to a different other meeting participant of a plurality of other meeting participants; after establishing the primary communications channel, presenting (109a) a primary audiovisual conversation (201) that includes the meeting participant and the plurality of other meeting participants, the primary audiovisual conversation comprising first audiovisual data communicated over the primary communications channel; after establishing the primary communications channel, identifying (109a) an instruction to establish a secondary audiovisual conversation between the meeting participant and a subset of the plurality of other meeting participants that is less than all of the plurality of other meeting participants; based on identifying the instruction, establishing (107a) a secondary communications channel (114) between the computer system and a subset of the plurality of other computer systems corresponding to the subset of the plurality of other meeting participants; after establishing the secondary communications channel, presenting a mixed audiovisual conversation that includes, (i) the primary audiovisual conversation (201) which - Page 29 -

408838-LU-NP includes the meeting participant and the plurality of other meeting participants, and (ii) the LU102059 secondary audiovisual conversation (206) which includes the meeting participant and only the subset of the plurality of other meeting participants, the secondary audiovisual conversation comprising second audiovisual data communicated over the secondary communications channel, wherein presenting the mixed audiovisual conversation includes presenting both a first video presentation of the primary audiovisual conversation and a second video presentation of the secondary audiovisual conversation at a common meeting user interface; and while presenting the mixed audiovisual conversation, preventing (108a) exposure of the second audiovisual data to the primary audiovisual conversation.

2. The method of claim 1, wherein identifying the instruction to establish the secondary audiovisual conversation comprises automatically accepting a request to establish the secondary audiovisual conversation based on determining that a meeting participant who initiated the request is on an allow list.

3. The method of claim 1, wherein identifying the instruction to establish the secondary audiovisual conversation comprises receiving a user input at the meeting user interface accepting a request to establish the secondary audiovisual conversation, the request being initiated at one of the subset of the plurality of other computer systems.

4. The method of any preceding claim, further comprising, while presenting the mixed audiovisual conversation, deemphasizing the primary audiovisual conversation as compared to the secondary audiovisual conversation.

- Page 30 - ZELL.

408838-LU-NP

5. The method of claim 4, wherein deemphasizing the primary audiovisual-U102059 conversation as compared to the secondary audiovisual conversation comprises reducing a first audio level of the primary audiovisual conversation as compared to a second audio level of the secondary audiovisual conversation.

6. The method of claim 4, wherein deemphasizing the primary audiovisual conversation as compared to the secondary audiovisual conversation comprises at least one of: reducing a first visual size of the first video presentation as compared to a second visual size of the second video presentation; reducing a first vibrancy of the first video presentation as compared to a second vibrancy of the second video presentation; presenting the first video presentation in greyscale while presenting the second video presentation in color; or | applying a blur to the first video presentation.

7. The method of any preceding claim, wherein the subset of the plurality of other meeting participants comprises more than one meeting participant.

8. The method of any preceding claim, wherein preventing exposure of the second audiovisual data to the primary audiovisual conversation comprises at least one of: automatically muting at least one of an audio stream or a video stream of the meeting participant within the primary audiovisual conversation; - Page 31 - EE]

408838-LU-NP preventing audiovisual data received over the secondary communications channel LU102059 from being communicated over the primary communications channel; preventing the secondary audiovisual conversation from being recorded during recording of the primary audiovisual conversation; or ; applying an echo cancellation to audio communicated in the second audiovisual data.

9. The method of any preceding claim, further comprising, based on receiving a user input performing at least one of: automatically unmuting at least one of an audio stream or a video stream of the meeting participant within the primary audiovisual conversation; permitting second audiovisual data received over the secondary communications channel to be communicated over the primary communications channel; permitting the secondary audiovisual conversation to be recorded during recording of the primary audiovisual conversation; or exposing existence of the secondary audiovisual conversation to the primary audiovisual conversation.

10. The method of claim 9, wherein the method comprises exposing existence of the secondary audiovisual conversation to the primary audiovisual conversation.

11. The method of claim 10, further comprising receiving a request to join the secondary audiovisual conversation, the request being received by one of the plurality of other computer systems that is not part of the subset of the plurality of other computer systems.

- Page 32 - ZELL...

408838-LU-NP

12. The method of any preceding claim, wherein the secondary audiovisual LU102059 conversation comprises a first secondary audiovisual conversation, the secondary communications channel comprises a first secondary communications channel, and the subset of the plurality of other computer systems comprises a first subset of the plurality of other computer systems, the method further comprising: establishing a second secondary communications channel with a second subset of the plurality of other computer systems that is less than all of the plurality of other computer systems, the second subset being different than the first subset; presenting a second secondary audiovisual conversation as part of the mixed audiovisual conversation, the second secondary audiovisual conversation including the meeting participant and only the second subset of the plurality of other meeting participants, the second secondary audiovisual conversation comprising third audiovisual data communicated over the second secondary communications channel; and while presenting the mixed audiovisual conversation, preventing participation in the second secondary audiovisual conversation from being exposed to at least one of the primary audiovisual conversation or the first secondary audiovisual conversation.

13. The method of any preceding claim, wherein establishing at least one of the primary communications channel or the secondary communications channel comprises: communicating with at least one of the plurality of other computer systems via an intermediary server; or communicating with at least one of the plurality of other computer systems directly. | - Page 33 - ZELL.

408838-LU-NP

14. The method of any preceding claim, wherein the meeting user interface LU102059 provides a first indication that the primary audiovisual conversation is a public conversation, and provides a second indication that the secondary audiovisual conversation is a private conversation.

15. The method of any preceding claim, wherein presenting both the first video presentation the second video presentation at the common meeting user interface comprises presenting the first video presentation and the second video presentation in different windows of the common meeting user interface.

- Page 34 - To