JP2015523001A - Promote simultaneous consumption of media content by multiple users using superimposed animations - Google Patents

Abstract

An apparatus, computer-implemented method, system, device for facilitating simultaneous consumption of media content by a first user of a first computing device and a second user of a second computing device And embodiments of computer readable media are described. In various embodiments, the promotion is presented on the first computing device of the second user's animation based on the second user's captured visual data received from the second computing device. May include overlays on top of the media content being played. The animation may be visually enhanced when it is determined that the first user is interested in the second user. The promotion is based at least in part on whether the second user has assigned a trusted status and transmission of the first user's changed or unmodified visual data to the second computing device. Conditionally changing the captured visual data of one user may be included.

Description

<Cross-reference of related applications>
This application claims priority from US Patent Application No. 13 / 532,612, filed June 25, 2012, the entire contents of which are incorporated by reference for all purposes.

  Embodiments of the present invention relate generally to the technical field of data processing, and more particularly to facilitating simultaneous consumption of media content by multiple users using overlaid animations. The background description provided herein is generally for the purpose of presenting the context of the disclosure. The work of the inventors listed here in the scope described in this background art section is expressly or implicit as prior art to the present disclosure, along with other aspects not deserving prior art at the time of filing. It was not recognized. Unless otherwise indicated herein, the methods described in this section have been identified as prior art simply because they are not prior art in the claims of this disclosure and are included in this section. That's not true.

  People may wish to consume media content together. For example, a group of friends may gather and watch a movie, a television program, a sporting event, a home video, or other similar media content together. Friends may interact during the presentation to enhance the media consumption experience. However, it is also envisaged that two or more people who are physically separate from each other and cannot gather in a single place would like to share the media content consumption experience.

Embodiments will be better understood when the following detailed description is read in conjunction with the accompanying drawings. To facilitate the description, like reference numerals shall indicate like structural elements. The embodiments are shown by way of example and are not intended as limitations on the figures of the accompanying drawings.
1 schematically illustrates an example computing device configured with an applicable portion of the teachings of the present disclosure in communication with other similarly configured remote computing devices in various embodiments. FIG. 2 schematically illustrates the scenario of FIG. 1 in which a user of a computing device in various embodiments is interested in a particular superimposed animation of a remote user. FIG. 2 schematically illustrates the scenario of FIG. 1 in which a user of a computing device in various embodiments is interested in media content over a remote user's superimposed animation. 6 schematically illustrates an example method that a computing device may implement in various embodiments. 6 schematically illustrates another example method that a computing device may implement in various embodiments. 1 schematically illustrates an example computing device in which the disclosed methods and computer-readable media in various embodiments may be implemented.

  In the following detailed description, reference will be made to a part of the drawing that forms a part, where like reference numerals generally indicate like parts and are also illustrated by viable exemplary embodiments. . It should be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present disclosure. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the disclosure is defined by the appended claims and their equivalents.

  The various acts may be described as a plurality of separate actions or acts in a manner that most aids understanding of the claimed subject matter. However, the order of description should not be construed as implying that these operations are necessarily order dependent. In particular, these operations may not be performed in the order of presentation. The described operations may be performed in a different order than the described embodiments. Various further operations may be performed and / or described operations may be omitted in further embodiments.

  For purposes of this disclosure, the term “A and / or B” indicates (A), (B), or (A and B). For purposes of this disclosure, the term “A, B, and / or C” means (A), (B), (C), (A and B), (A and C), (B and C) , or (A, B and C).

  In the description, the phrases “in one embodiment” or “in an embodiment” may be utilized, which may each refer to one or more of the same or different embodiments. Further, phrases such as “comprising”, “including”, “having”, etc., that are utilized in connection with embodiments of the present disclosure are synonymous.

  The term “module” as used herein executes an application specific integrated circuit (“ASIC”), an electronic circuit, a processor (shared, dedicated, or group), and / or one or more software or firmware programs. May refer to, be part of, or include memory (shared, dedicated, or group), combinatorial logic, and / or other suitable components that provide the described functionality.

  FIG. 1 schematically illustrates an example computing device 100 configured with an applicable portion of the teachings of the present disclosure in various embodiments. Although the computing device 100 is shown as a tablet computing device, this is not meant to be limiting. The computing device 100 may include various other types of computing devices (or combinations thereof), including but not limited to laptops, netbooks, notebooks, ultrabooks, smartphones, personal digital assistants ( "PDA"), ultra mobile PC, mobile phone, desktop computer, server, printer, scanner, monitor, set-top box, entertainment control unit (eg, game console), digital camera, portable music player digital video recorder, television (eg , Plasma, liquid crystal display or “LCD”, cathode ray tube or “CRT”, projection screen).

  Computing device 100 may include a display 102. Display 102 may be various types of displays including, but not limited to, plasma, LCD, CRT, and the like. In some embodiments (not shown), the display may include a projection surface on which the projector may project graphics with superimposed animation as described herein. In various embodiments, the display 102 may be a touch screen display that may be utilized to provide input to and manipulate the computing device 100. In various embodiments, computing device 100 may include additional input controls (not shown) in addition to or instead of the touch screen display.

  In various embodiments, the computing device 100 may include a camera 104 configured to capture visual data such as one or more frames and / or digital images. As described below, the captured visual data may be transmitted to a remote computing device and utilized to facilitate overlaying animation on top of other content by the remote computing device.

  Although the camera 104 is shown as an integrated part of the computing device 100 of FIGS. 1-3, this is not intended to be limiting. In various embodiments, the camera 104 is not separate from the computing device 100. For example, the camera 104 may be an external camera (eg, a web camera) connected to the computing device 100 using one or more wired or wireless.

  In various embodiments, computing device 100 may include an eye tracking device 106. In various embodiments, such as the computing tablet shown in FIG. 1, the camera 104 also operates as the eye tracking device 106. But this is not absolutely necessary. In various embodiments, the eye tracking device 106 may be separate from the camera 104, and may be a different type of device and / or a different type of camera. For example, in embodiments where computing device 100 is a television or game console connected to a television, eye tracking device 106 is a camera or other device operably coupled to the television or game console (eg, a motion capture device). It may be. Such an example is shown in FIG. 3 and will be described later.

  In various embodiments, the visual data captured by the camera 104 and / or eye tracking device 106 is analyzed by software, hardware, or any combination of the two so that the user is looking for, if any. It may be possible to determine and / or approximate 102 parts. This determination includes, but is not limited to, determining the distance between the user's face and / or eyes and the display 102, one or more features of the user's eyes (eg, pupils in visual data, etc.). Various actions may be included such as identifying, measuring the distance between identified features, and the like. Determine which part of the display 102 the user is looking for (and therefore showed interest), and what part of the display 102 the user is not looking for (and thus did not show interest), as described below. Doing may be utilized in various ways.

  The computing device 100 may communicate with various remote computing devices via one or more networks. 1 and 2, for example, the computing device 100 is in wireless communication with a first wireless network access node 108 that is in communication with the network 110. In various embodiments, the first wireless access node 108 is an evolved NodeB, a WiMAX (IEEE802.16 family) access point, a Wi-Fi (IEEE802.11 family) access point, or the computing device 100 is wirelessly connected. It may be any other node that may be. The network 110 may include one or more personal, local, or wide area, private, and / or public networks, including but not limited to the Internet. Although the computing device 100 is shown as being wirelessly connected to the network 110, this is not intended to be limiting, and the computing device 100 may use any other method (so-called “wired” connection). May be connected to one or more networks.

  Computing device 100 may be in network communication with any number of remote computing devices. In FIG. 1 and FIG. 2, for example, computing device 100 is in network communication with a first remote computing device 112 and a second remote computing device 114. As with computing device 100, first and second remote computing devices 112, 114 may be any type of computing device, such as those previously described. For example, in FIG. 1, the first remote computing device 112 is a smartphone and the second remote computing device 114 is a laptop computer.

  The first remote computing device 112 is shown as being wirelessly connected to another wireless network access node 116. The second remote computing device 114 is shown connected to the network 110 via a wired connection. However, the type of network connection utilized by the remote computing device is not important. Any computing device may communicate with any other computing device in the manner described herein utilizing any type of network connection.

  In various embodiments, computing device 100 may include one or more remote computing devices (eg, first remote user 118 and / or second remote computing device 114 of first remote computing device 112). It may be configured to facilitate simultaneous consumption of media content 122 by a user (not shown) of computing device 100 with one or more users (such as second remote user 120). In various embodiments, the computing device 100 may be configured to superimpose one or more animations of a remote user on the media content 122 presented on the computing device 100.

  In various embodiments, these one or more superimposed animations may be rendered by computing device 100 based on visual data received from a remote computing device. In various embodiments, visual data received from a remote computing device may be based on visual data of a remote user (eg, 118, 120) captured at the remote computing device.

  As used herein, the term “animation” may refer to any moving visual representation generated from captured visual data. This includes, but is not limited to, video (eg, bitmap) playback of captured visual data, artistic presentation of visual data (eg, cartoon rendered based on user-captured visual data), etc. May be included. In other words, “animation” is used herein as a noun form of “animating”, which means “giving life”. Thus, “animation” is a description or rendering of “animated” (alive or having life), contrary to “inanimate”. “Animation” is not limited to drawing generated by an animator.

  In various embodiments, media content 122 may include audio and / or visual content such as, but not limited to, video (eg, streaming), video games, web pages, slide shows, presentations, and the like.

  By superimposing the remote user's animation over the media content, two or more users at remote locations may be able to consume the media content “together”. Each user can see other users' animations superimposed on the media content. Thus, for example, two or more friends at remote locations can share experiences such as movies, television shows, sporting events, and the like.

  In FIG. 1, a first animation 124 and a second animation 126 representing a first remote user 118 and a second remote user 120 are respectively over media content 122 on the display 102 of the computing device 100. Is superimposed on. The first animation 124 is for the first remote user 118 received by the computing device 100 from the first remote computing device 112, eg, from a camera (not shown) on the first remote computing device 112. May be based on captured visual data. For example, the first animation 124 may be a video stream depicting the first remote user 118. The second animation 126 may similarly be based on the captured visual data of the second remote user 120 received at the computing device 100 from the second remote computing device 114.

  In various embodiments, visual data on which animation is rendered may be transmitted between various forms of computing devices. In various embodiments, one computer may send captured visual data in bitmap format to another computer (eg, a video stream of other visual files with * .png or alpha mask). . In other embodiments, captured visual data may be transmitted utilizing streaming video with embedded alpha. In still other embodiments, the captured visual data may be transmitted using a stream of bitmap (eg, RGB) frames and depth frames, from which two-dimensional (“2D”) or three-dimensional Any (3D) animation may be rendered.

  1-3, the animation is rendered near the bottom of the display 102 so that the user of the computing device 100 can still view the media content 122. However, this is not intended to be limiting. Animations such as first animation 124 and second animation 126 may be rendered on any portion of display 102. In some embodiments, the animation may be displayed on multiple displays. For example, if a desktop computer user has multiple monitors, one or more of the animations may be displayed on one monitor or another monitor. In various embodiments, these animations may be overlaid on content 122 on one or both of the monitors.

  In various embodiments, the particular animation that is of interest to the user of the animation may be visually enhanced over what the computing device 100 has determined. As used herein, “visually emphasized” refers to rendering an animation that is different from other overlaid animations or media content, or to draw one animation or It may mean distinguishing from multiple other animations.

  In FIG. 1, for example, the first and second animations 124, 126 are illustrated with black and white outlines, indicating that both animations are equally visually enhanced, and the user's attention is To be less than the other. For example, both animations can show the first and second users in real time and may be rendered in a somewhat equally prominent manner. In other words, no animation is “visually emphasized”.

  “Not visually enhanced” means that the remote user's animation is rendered without any attention, or directly from other animations or media content and then visually enhanced. Refers to distracting attention to another animation or to the underlying media content. Figure 2 shows an example of not visual enhancement. The first animation 124 is all shown in black to indicate that it is not visually enhanced. The second animation 126 is shown with a black outline on white to indicate that it is visually enhanced.

  In various embodiments, the remote user's animation may not be visually enhanced in various ways. For example, instead of rendering the user's full-color or full-feature animation, a remote user's, for example, a single color silhouette (eg, gray, black, or any other color or shade) may be rendered. In various embodiments, a remote user may be rendered with a shadow. In some embodiments, animations that are not visually enhanced may not be animated at all, or may be animated at a slower frame rate than animations that are visually enhanced.

  In FIG. 3, both the first animation 124 and the second animation 126 are not visually enhanced. This may occur when the user of computing device 100 does not show interest to either user. For example, the user may have shown interest in viewing the media content 122 rather than the remote user's animation. When the user is interested in one or the other of the animations, the animation in which the user has shown interest may be visually enhanced by the computing device 100.

  A user may or may not be interested in a particular animation or other portion of display 102 in various ways. For example, the camera 104 and / or the eye tracking device 106 may be configured to collect data regarding the user's eye movements. Based on this data, computing device 100 may calculate any portion of display 102 that the user is viewing, if any.

  For example, in FIG. 2, computing device 100 may determine that the user is focused (or looking at) second animation 126 based on input from eye tracking device 106. Thus, the computing device 100 may not visually enhance the first animation 124 by visually enhancing the second animation 126.

  As another example in FIG. 3, the computing device may allow the user to focus on the media content 122 and / or the first animation 124 based on input from the eye tracking device 106. It may be determined that the second animation 126 is not focused. Thus, the computing device 100 may not visually enhance both the first animation 124 and the second animation 126, thereby facilitating distraction from the media content 122.

  Although not shown in FIGS. 1-3, the first remote computing device 112 and the second remote computing device 114, like the computing device 100, are an overlay of media content 122 and other remote user animations. And may be displayed simultaneously. For example, the first remote computing device 112 may overlay the animation of the user (not shown) of the computing device 100 and the second remote user 120 on the media content 122. Similarly, the second remote computing device 114 may overlay the animation of the user (not shown) of the computing device 100 and the first remote user 118 on the media content 122. Further, although three computing devices are shown, any number of computing devices configured with the applicable portions of the present disclosure may participate in a simultaneous media content viewing session.

  The animation shown in the drawing shows the entire body of the remote user, but this is not intended to be limiting. In various embodiments, less than the entire body of the remote user may be rendered. For example, in some embodiments, a portion (such as the upper body) of a remote user (eg, the remote user's “chest”) may be rendered. In some cases, the animation may be rendered adjacent to the bottom of the display so that the remote user's animation appears to be “popping up” from the bottom of the display. In addition, other parts of the remote user (eg, only the head, above the chest, above the knee or thigh, one half of the remote user or another half, etc.), etc. may be animated.

  In some embodiments, the computing device 100 may be configured to crop capture remote users and / or resulting animations. For example, the remote user's captured visual data may include the remote user's whole body or background. In various embodiments, the computing device 100 may be configured to automatically crop and eliminate unwanted portions (eg, a remote user's legs and / or background empty space).

  In various embodiments, the computing device 100 is configured to dynamically and / or automatically crop captured visual data of its own local or remote user based on various criteria. Good. For example, the computing device 100 may represent at least a portion of the local user's visual data or remote user's visual data of the computing device 100, and an area of the visual data representing the local or remote user may be out of the total visual data. May dynamically crop based on determining that it occupies less than a predetermined portion of. If a local or remote user is moving around, for example, close to their camera, the local or remote user may be larger in view.

  In such a case, the computing device 100 may dynamically reduce cropping as appropriate. Accordingly, the computing device 100 is suitable for the user's animation (whether local or remote) in the visual data provided to the remote computing device and in the visual data received from the remote computing device. May be guaranteed to be the correct size and ratio.

  In various embodiments, the computing device 100 may render an animation of the local user of the computing device 100 in addition to the animation of the remote user. This may allow the user to see what the remote user wants to see. This may further enhance the sense of community with the remote user's animation by placing the local user's animation in a “common area”. This further facilitates user decisions regarding their privacy, as described in more detail below.

  In various embodiments, simultaneous media content sharing sessions may be implemented utilizing peer-to-peer and / or client-server software installed on each computing device. In various embodiments, the simultaneous media content sharing session may be persisted even when one or more users sign out of the session. For example, as in FIG. 1, when the first remote user 118 signs off, the first animation 124 on the computing device 100 disappears, but the second animation 126 is displayed on the computing device 100 and the second As long as the remote computing device 114 maintains a simultaneous media content sharing session, it may be persisted.

  In various embodiments, a user may be able to join (rejoin) an existing simultaneous media content sharing session. For example, as in FIGS. 1 and 2, a second remote user 120 is participating via a laptop computer. However, in FIG. 3, the second remote user 120 has signed out of the simultaneous media content sharing session on the laptop computer and the third remote computing device 128 (using the applicable portion of the present disclosure). You can join again using

  In FIG. 3, a third remote computing device 128 is in the form of a game console connected to a television 130. In this arrangement, the television 130 may perform the same function as the display 102 of the computing device 100. Third remote computing device 128 may further be operably coupled to movement sensing device 132. In various embodiments, the movement sensing device 132 may include a camera (not shown). In various embodiments, the movement sensing device 132 may include an eye tracking device (not shown).

  In various embodiments, in addition to overlaying the animation, the computing device 100 may receive audio or other data from the remote computing device and present it to the user. For example, a remote computing device (eg, 112, 114, 128) may be equipped with a microphone (not shown), thereby recording the voice of the remote user (118, 120). The remote computing device may digitize the received audio and send it to the computing device 100.

  The computing device 100 may render the received audio data so that it can be heard, for example, with animation (eg, 124, 126).

  When multiple users are sharing media content at the same time, one user may want the audio from the remote user not to interfere with the audio component of the media content. Thus, in various embodiments, a user can disable (mute) audio from one or more remote users while allowing these remote user animations to appear on the display 102. Good. In various embodiments, the computing device 100 may be configured to overlay one or more remote user speech text manifest stations on the media content 122 on the display 102. An example of this is shown in FIG. 3 where a recalled balloon 140 is superimposed over the media content 122 to display a text manifest station for comments created by the second remote user 120.

  In various embodiments, a speech text manifest station by a remote user of computing device 100 may be based on speech-enabled text data received from a remote computing device. In various other embodiments, the audio text manifest station by the remote user may be based on audio data received by the computing device 100 from the remote computing device. In the latter case, the computing device 100 may be configured to utilize voice to text software to convert the received audio to text.

  Media may be consumed simultaneously by multiple users in a variety of ways. In various embodiments, streaming video or other media content is synchronized between multiple computing devices (eg, 100, 112, 114, 128) so that all users can view the same content at the same time. May be. Media content may be distributed in various ways. In some embodiments, the first user may have media content and may provide it to other users. For example, a user of the computing device 100 may have a streaming video account (eg, an on-demand video stream subscription) and a copy of the stream to a remote computing device (eg, 112, 114, 128). May be transferred to. In such a case, the first user's computing device may insert a delay into its playback of the video stream, which cannot be done before the video stream playback of the remote computing device. There is a case.

  In other embodiments, the media content may be centrally located (eg, to a content server) and computing devices may individually connect to the content server and receive streaming. In such cases, the computing devices may exchange synchronization signals with each other to ensure that each user watches the same content simultaneously. In some embodiments, if the user pauses playback of media content on computing device 100, playback of the content may be performed by other participating computing devices (eg, remote computing device 112, 114, 128).

  In various embodiments, a privacy mechanism may be utilized to protect user privacy. For example, the user of the computing device 100 may have visual data sufficient to render an animation of the user's silhouette or shadow for the remote computing device on the computing device 100, eg, a remote computing device (eg, 112, 114). May only be ordered to provide. In some embodiments, the user may direct the computing device 100 not to provide captured visual data. In some embodiments, the user directs the computing device 100 to capture visual data only during certain time periods and / or not to capture, modify / distort the visual data during other time periods. May be attached.

  In some embodiments, the computing device 100 utilizes one or more image processing filters to render the user's animation rendered on the remote computing device unrecognizable and / or complete. Or less than a good rendering. For example, visual data captured by the camera 104 of the computing device 100 may pass through one or more image processing filters and be blurred, pixelized, or altered in visual data. You can do it. In some embodiments, the user may direct the computing device 100 to have the resulting animation have a reduced frame rate to remove some frames from the captured visual data. In addition or alternatively, computing device 100 may reduce the sampling rate of camera 104 to capture coarser visual data.

  In some embodiments, the computing device 100 may be configured to protect the remote user's privacy, eg, in response to instructions received from a remote computing device (eg, 112, 114). For example, the computing device 100 may change the visual data representing the remote user (eg, through an image processing filter) that is otherwise fully renderable, and the resulting animation of the remote user is not recognizable. Or less than full rendering.

  In various embodiments, a user may assign a trusted status to one or more remote users. These remote users may then be considered as one of the user's contacts. When one of the user's contacts joins or rejoins the simultaneous media content viewing session, the contact's animation may appear, be reanimated, or change its appearance. If one of the user's contacts leaves the simultaneous media content viewing session, the contact's animation may disappear, appear flattened, or the appearance may change.

  In some embodiments, the computing device 100 conditionally changes the visual data transmitted to the remote computing device, depending on whether the remote user of the destination remote computing device has been assigned a trusted status. Good. For example, the computing device 100 may send “complete” or unmodified visual data to a user contact, or to a specific contact assigned a higher trusted status than other contacts (“close friends” )). The computing device 100 may have less than complete visual data (eg, visual data with frames removed or captured at a reduced sampling rate), or modified visual data (eg, blurred or pixelated). May be sent to a contact (such as “acquaintance”) that is considered to have been removed. In some embodiments, the computing device 100 may send less or no visual data to a user's remote computing device that has not been assigned a trusted status, or may send significantly changed visual data. May be sent.

  In various embodiments, the computing device 100 provides the user's captured visual data before the computing device 100 overlays the remote user's animation over the media content or to the remote computing device. Before doing so, it may be necessary to perform a handshake procedure with a remote computing device (eg, 112, 114). For example, or the user of the computing device 100 clicks or selects an icon or other graphic representing the remote user before the computing device 100 overlays the remote user's animation on the media content, or It may be required to provide visual data to the remote computing device. In some embodiments, the computing device 100 overlays an animation of a user's “closest” contact (eg, a contact to which the user has assigned a relatively high level of trust) without the need for any handshaking. Alternatively, the user's captured visual data may be provided to the closest contact.

  In some embodiments, image processing may be applied to visual data for purposes other than privacy. For example, in some embodiments, background subtraction may be implemented by computing device 100 to “cut” the user and subtract the background from the visual data. When the remote computing device uses visual data to overlay the user's animation, the user may be rendered in isolation without any background.

  In various embodiments, the superimposed animation (eg, first animation 124 and second animation 126, etc.) may be rendered in 2D and / or 3D. In embodiments that render the animation in 3D, the computing device 100 may be configured to utilize parallax correction on top of the remote user animation overlay. In some 3D embodiments, captured visual data (based on animation) may be transmitted between computing devices as a point cloud, a list of intersections, a list of triangles, and the like.

  In embodiments where the computing device 100 is configured to render animation in 3D, the computing device 100 may do this in various ways. For example, in some embodiments, computing device 100 may render a 3D shape on a 2D screen. In other 3D embodiments, the computing device 100 may render a 3D shape in 3D on a stereoscopic display and the user may wear 3D glasses.

  The overlay of the remote user's animation may be rendered on the display 102 in various ways. In various embodiments, the overlay of the remote user's animation may be rendered in a transparent window so that it is superimposed on all or a portion of other content that is displayed on the display 102. .

  FIG. 4 is implemented on a computing device such as computing device 100, first remote computing device 112, second remote computing device 114, and / or third remote computing device 128. A good exemplary method 400 is shown. At block 402, captured visual data of a remote user of the remote computing device may be received (eg, by the computing device 100 from the remote computing device). At block 404, media content (eg, video, shared web browsing session, slide show, etc.) may be presented, for example, by computing device 100 simultaneously with the presentation of the media content on the remote computing device.

  At block 406, it may be determined by computing device 100, for example, that the user of the computing device is interested or not interested in the remote user. For example, the computing device 100 may receive data from the eye tracking device (eg, 106) that may be utilized by the computing device 100 to determine where the user is looking. If the remote user's animation is at a specific distance from or within the range, it may be determined by the computing device 100 that the user is interested in the remote user, for example.

  At block 408, the remote user's animation may be overlaid on the media content (eg, 122), for example, by the computing device 100 based on the received visual data. At block 410, the animation may be visually enhanced or deemphasized, for example by computing device 100, based on the determination result that the user is interested. For example, if the user is interested in the remote user, the remote user's animation may be fully rendered. If the user is not interested in the remote user, the remote user's animation may be rendered less than complete (eg, pixelated at a lower frame rate with shadows, etc.). After block 410, if a concurrent media sharing session is still ongoing, the method 400 may return to block 402. When the session ends, the method 400 may proceed to the END block.

  FIG. 5 is implemented on a computing device such as computing device 100, first remote computing device 112, second remote computing device 114, and / or third remote computing device 128. A good exemplary method 500 is shown. At block 502, visual data may be captured (eg, by camera 104). At block 504, whether the computing device 100 includes one or more remote users that the user of the computing device 100 wants to consume media content at the same time in the list of remote users with trusted status (eg, contacts) May be determined.

  If the answer is no, at block 506, the user's captured visual data may be modified, for example, by computing device 100, to maintain the user's privacy. For example, visual data may be provided through one or more image processing filters (eg, blurring filter, pixelated filter), or render the resulting animation on a remote computing device unrecognizable, distorted, and It may be modified so that it does not appear completely. At block 508, the modified visual data may be transmitted to the remote computing device (eg, 112, 114, 128), eg, by the computing device 100. After block 508, the method 500 may return to block 502 if a concurrent media sharing session is still ongoing. If the session has ended, the method 500 may proceed to the END block.

  If the answer in block 504 is YES, block 510 may determine, for example by computing device 100, whether the user wants privacy. For example, the computing device 100 may determine whether a privacy flag is set or whether the current time is within a time period that indicates that the user wants privacy. If the user wants privacy, the method 500 may proceed to block 506 and the visual data may be changed prior to transmission to protect the user's privacy. If the answer at block 510 is NO, the unbiased visual data may be sent to one or more remote computing devices (eg, 112, 114m128) at block 508, eg, by computing device 100. .

  FIG. 6 illustrates an exemplary computing device 600 in various embodiments. The computing device 600 may include a plurality of components, a processor 604, and at least one communication chip 606. In various embodiments, the processor 604 may be a processor core. In various embodiments, at least one communication chip 606 may also be physically or electrically coupled to the processor 604. In further implementations, the communication chip 606 may be part of the processor 604. In various embodiments, the computing device 600 may be included on a printed circuit board (“PCB”) 602. In these embodiments, processor 604 and communication chip 606 may be disposed thereon. In another embodiment, the various components may be coupled without utilizing PCB 602.

  Depending on its application, computing device 600 may include other components that may or may not be physically and electrically coupled to PCB 602. These other components include, but are not limited to, volatile memory (eg, dynamic random access memory 608, also referred to as “DRAM”), non-volatile memory (eg, read only memory 610, also referred to as “ROM”). Flash memory 612, graphics processor 614, digital signal processor (not shown), crypto processor (not shown), input / output ("I / O") controller 616, antenna 618, display ( (Not shown), touch screen display 620, touch screen controller 622, battery 624, audio codec (not shown), video codec (not shown), global positioning system (“GPS”) device 628, compass 630 (not shown), acceleration Total (not shown), Jai Scope (not shown), speaker 632, camera 634, and mass storage device (eg, hard disk drive, solid state drive, compact disc (“CD”), digital versatile disc (“DVD”), etc.) (not shown), etc. Good. In various embodiments, the processor 604 may be integrated on the same die as other components to form a system on chip (“SoC”).

  In various embodiments, volatile memory (DRAM 608), non-volatile memory (ROM 610), flash memory 612, and mass storage may be added to computing device 600 in response to processor 604 executing. Programming instructions configured to cause all or selected aspects of the methods 400 and / or 500 to be performed may be included. For example, one or more of the memory components, such as volatile memory (DRAM 608), non-volatile memory (ROM 610), flash memory 612, and mass storage device may be included in the computing device 600 in the method 400 and / or the method 500. It may include a primary and / or permanent copy of instructions (illustrated as control module 636 in FIG. 6) configured to perform the disclosed techniques, such as all or selected aspects.

  Communication chip 606 may allow wired and / or wireless communication for transfer of data to / from computing device 600. The term “wireless” and its derivatives describe circuits, devices, systems, methods, techniques, communication channels, etc., through which data can be communicated utilizing electromagnetic radiation modulated over a solid medium. May be used for The language does not suggest that the associated device does not include any wireline, but may not include it in some embodiments. The communication chip 606 includes, but is not limited to, IEEE 802.1 1 ("WiFi"), IEEE 802.16 ("WiMAX"), IEEE 702.20, Long Term Evolution ("LTE"), General Packet Radio Service (general purpose Packet radio service ("GPRS"), Evolution Data Optimized (Evolved Data Optimization: "Ev-DO"), Evolved High Speed Packet Access (Evolved High Speed Packet Access: "HSPA +"), Evolved High Speed Downlink Packet Access (Evolution) High Speed Downlink Packet Access: "HSDPA +"), Evolved High Speed Uplink Packet Access (Evolved High Speed Uplink Packet Access: "HSUPA +"), Global System for Mobile Communications (GSM (registered trademark) ) "), Enhanced Data rates for GSM (TM) Evolution (" EDGE "), code division multiple access (" CDMA "), time division multiple access ( "TDMA"), Digital E Includes nhanced Cordless Telecommunications ("DECT"), Bluetooth (R), their derivatives, and any other wireless protocol designated as 3G, 4G, 5G and above Any of a plurality of wireless standards or protocols may be implemented. The computing device 600 may include a plurality of communication chips 606.

  For example, the first communication chip 606 may be dedicated to shorter-range wireless communication such as Wi-Fi and Bluetooth (registered trademark), and the second communication chip 606 may be GPS, EDGE, GPRS, CDMA, It may be dedicated to longer-distance wireless communication such as WiMAX, LTE, Ev-DO, etc.

  In various implementations, the computing device 600 may be a laptop, netbook, notebook, ultrabook, smartphone, computing tablet, personal digital assistant (“PDA”), ultra mobile PC, mobile phone, desktop computer, server. , Printers, scanners, monitors, set-top boxes, entertainment control units (eg, game consoles), digital cameras, portable music players, or digital video recorders. In further implementations, the computing device 600 may be any other electronic device that processes data.

  The apparatus, computer-implemented method, system, device, and computer-readable medium herein include simultaneous consumption of media content by a first user of a first computing device and a second user of a second computing device. Has been described to promote. In various embodiments, the promotion is presented on the first computing device of the second user's animation based on the second user's captured visual data received from the second computing device. May include overlays on top of the media content being played. In various embodiments, the animation may be visually enhanced when it is determined that the first user is interested in the second user. In various embodiments, the determination of the first user's interest may be based on data received from the eye tracking input device for the first computing device.

  In various embodiments, media content presented on the first computing device when it is determined that the first user is interested in the second user or not interested in the third user. On top of this, the superimposition of the third user's animation on the third computing device may not be visually enhanced. In various embodiments, the first computing device may render the overlay of the third user's animation with a shadow so as not to visually enhance the overlay of the third user's animation. .

  In various embodiments, the overlay of the second user's animation may include an overlay of the second user's animation adjacent to the bottom of the display of the first computing device. In various embodiments, parallax correction for the overlay of the second user's animation may be utilized.

  In various embodiments, a second user's audio text manifest station may be superimposed over the media content. In various embodiments, an audio text manifest station by a second user may be based on audio-enabled text data received from a second computing device or audio data received from a second computing device.

  In various embodiments, the overlay of the second user's animation may be rendered in a transparent window.

  In various embodiments, the first user's captured visual data may be conditionally modified based at least in part on whether the first user has assigned a trusted status to the second user. In various embodiments, the first user's captured visual data may be transmitted to the second computing device. In various embodiments, the captured visual data causes the second computing device to superimpose the first user's animation over the media content displayed on the second computing device. May be configured.

  In various embodiments, the step of conditionally changing comprises performing image processing on the first user's captured visual data, the image processing comprising blurring, pixelation, background subtraction, or Frame deletion may be included. Various embodiments may include altering the first user's captured visual data in response to a determination that the second user is not assigned a trusted status from the first user.

  In various embodiments, at least a portion of the captured visual data of the first user or the second user may be automatically cropped. In various embodiments, the area of visual data in which the first user or the second user is represented occupies less than a predetermined portion of the entire captured visual data of the first user or the second user. Based on the determination, the at least a portion of the captured visual data of the first user or the second user may be dynamically cropped.

  Although specific embodiments have been described for purposes of illustration, this application is intended to cover any adaptations or variations of the embodiments described herein. Accordingly, it is expressly intended that the embodiments described herein be limited only by the claims.

  Where the disclosure includes “a”, “first” elements, or equivalents thereof, these disclosures include one or more of these elements and include two or more of these elements Do not require or eliminate. In addition, ordinal numbers (eg, first, second, or third) are used for specific elements to distinguish between elements, but such descriptions are limited to these elements unless otherwise specified. It does not indicate or suggest that a particular number is necessary, nor does it indicate a particular position or order of these elements.

Where the disclosure includes “a”, “first” elements, or equivalents thereof, these disclosures include one or more of these elements and include two or more of these elements Do not require or eliminate. In addition, ordinal numbers (eg, first, second, or third) are used for specific elements to distinguish between elements, but such descriptions are limited to these elements unless otherwise specified. It does not indicate or suggest that a particular number is necessary, nor does it indicate a particular position or order of these elements. Here, examples of embodiments of the present invention are shown as items.
[Item 1]
Receiving captured visual data of a second user received from a second computing device by a first computing device utilized by the first user;
Presenting the media content on the first computing device simultaneously with the presentation of the media content on the second computing device;
Determining by the first computing device whether the first user is interested or not interested in the second user of the second computing device;
Overlaying the second user's animation over the media content presented on the first computing device based on the second user's captured visual data by the first computing device. When,
Visually enhancing or not enhancing the animation based on the result of the decision by the first computing device;
A computer-implemented method comprising:
[Item 2]
The step of determining whether the user of the first computing device is interested or not interested receives, by the first computing device, eye tracking data from the eye tracking device and is based on the eye tracking data. The computer-implemented method of item 1, further comprising the step of determining where the user is looking.
[Item 3]
If it is determined that the first user is not interested in the second user or not interested in the third user, the third user is presented with the third content on the media content presented on the first computing device. 2. The computer-implemented method of item 1, further comprising the step of not visually enhancing the superimposition of the third user's animation on the computing device.
[Item 4]
Rendering the third user's animation overlay with shadows by the first computing device so as not to visually enhance the third user's animation overlay, item 3 The computer-implemented method described in 1.
[Item 5]
The computer-implemented method of item 1, wherein overlaying the second user's animation includes overlaying the second user's animation adjacent to the bottom of the display of the first computing device.
[Item 6]
The computer-implemented method of item 1, further comprising utilizing a parallax correction for superimposition of the second user's animation by the first computing device.
[Item 7]
The item of claim 1, further comprising: overlaying a second user's audio text manifest station on the media content presented on the first computing device by the first computing device. Computer implemented method.
[Item 8]
8. The computer-implemented method of item 7, wherein the audio text manifest station by the second user is based on the audio-enabled text data received from the second computing device.
[Item 9]
8. The computer-implemented method of item 7, wherein the audio text manifest station by the second user is based on audio data received from the second computing device.
[Item 10]
The computer-implemented method of item 1, wherein the second user's animation is rendered in a transparent window.
[Item 11]
Conditionally modifying the captured visual data of the first user based on at least in part by the first computing device assigning the trusted status to the second user by the first computing device; ,
Transmitting visual data by a first computing device to a second computing device;
Further comprising
The computer of item 1, wherein the captured visual data causes a second computing device to superimpose a first user's animation on the media content displayed on the second computing device. Implemented method.
[Item 12]
Conditionally changing the first user's captured visual data comprises performing image processing on the captured visual data;
Item 12. The computer-implemented method of item 11, wherein the image processing includes blurring, pixelation, background subtraction, or frame deletion.
[Item 13]
Conditionally changing the first user's captured visual data comprises changing the visual data in response to a determination that the second user has not been assigned a trusted status from the first user. 12. The computer implemented method of item 11, comprising:
[Item 14]
12. The computer-implemented method of item 11, further comprising: automatically cropping at least a portion of the captured visual data of the first user or the second user by the first computing device.
[Item 15]
Based on the determination that the area of visual data representing the first user or the second user occupies less than a predetermined portion of the entire captured visual data of the first user or the second user. The computer-implemented method of claim 14, further comprising dynamically cropping at least a portion of the captured visual data of the first user or the second user.
[Item 16]
16. At least one machine-readable medium comprising a plurality of instructions that, when executed on a computing device, cause the computing device to perform the computer-implemented method of any one of items 1-15.
[Item 17]
An apparatus comprising means for performing the computer-implemented method of any one of items 1 to 15.
[Item 18]
One or more processors;
A memory operably coupled to one or more processors;
Display,
A control module included in memory and operated by one or more processors to facilitate simultaneous consumption of media content by a first user of the system and a second user of a remote computing device;
With
Facilitating superimposition of the second user's animation on the media content presented on the display based on the second user's captured visual data received from the remote computing device. Including
The animation is visually enhanced when it is determined that the first user is interested in the second user.
[Item 19]
An eye tracking device,
19. The system of item 18, wherein the determination regarding the first user's interest is based on data received from the eye tracking device.
[Item 20]
The remote computing device is a first remote computing device;
The control module further determines the second user on the media content presented on the system when it is determined that the first user is interested in the second user or not interested in the third user. 19. The system of item 18, wherein the system does not visually enhance the superimposition of the third user's animation on the computing device.
[Item 21]
21. The system of item 20, wherein the control module further renders a third user animation overlay in the shadow.
[Item 22]
Item 22. The system of any of items 18 through 21, wherein the second user animation overlay includes a second user animation overlay adjacent to the bottom of the display.
[Item 23]
22. The system according to any one of items 18 to 21, wherein the control module further utilizes a parallax correction for the overlay of the second user's animation.
[Item 24]
Item 22. The system of any of items 18-21, wherein the control module further superimposes a second user's audio text manifest station on the media content being presented on the display.
[Item 25]
22. A system according to any one of items 18 to 21, further comprising a touch screen display.

Claims (25)

Receiving captured visual data of a second user received from a second computing device by a first computing device utilized by the first user;
Presenting the media content on the first computing device simultaneously with presenting the media content on the second computing device;
Determining by the first computing device whether the first user is interested in or not interested in the second user of the second computing device;
The media being presented on the first computing device by the first computing device based on the captured visual data of the second user with the second user's animation Overlaying the content,
Visually emphasizing or not emphasizing the animation based on a result of the determination by the first computing device.   Determining whether the user of the first computing device is interested or not interested comprises receiving eye tracking data from the eye tracking device by the first computing device, and The computer-implemented method of claim 1, comprising determining where the user is viewing based on tracking data.   If it is determined that the first user is interested in the second user or not interested in the third user, the top of the media content presented on the first computing device The computer-implemented method of claim 1, further comprising: visually avoiding superimposition of the third user's animation on a third computing device.   The first computing device renders the overlay of the animation of the third user with shadows so as not to visually enhance the overlay of the animation of the third user. The computer-implemented method of claim 3, further comprising steps.   The computer-implemented method of claim 1, wherein the overlay of the second user's animation comprises the overlay of the animation of the second user adjacent to the bottom of the display of the first computing device. Method.   The computer-implemented method of claim 1, further comprising utilizing a parallax correction for the overlay of the animation of the second user by the first computing device.   The method further comprises superimposing, by the first computing device, a text manifest station of the second user's voice over the media content being presented on the first computing device. Item 2. A computer-implemented method according to Item 1.   8. The computer-implemented method of claim 7, wherein the speech text manifest station by the second user is based on speech-enabled text data received from the second computing device.   The computer-implemented method of claim 7, wherein the audio text manifest station by the second user is based on audio data received from the second computing device.   The computer-implemented method of claim 1, wherein the animation of the second user is rendered in a transparent window. The first computing device conditionally captures the first user's captured visual data based at least in part on whether the first user has assigned a trusted status to the second user. Change stage,
Sending the visual data to the second computing device by the first computing device; and
The captured visual data causes the second computing device to superimpose the first user's animation over the media content displayed on the second computing device. Item 2. A computer-implemented method according to Item 1. Conditionally changing the captured visual data of the first user comprises performing image processing on the captured visual data;
The computer-implemented method of claim 11, wherein the image processing includes blurring, pixelation, background subtraction, or frame deletion.   Conditionally changing the captured visual data of the first user is responsive to a determination that the second user has not been assigned a trusted status from the first user. The computer-implemented method of claim 11, comprising the step of modifying:   The computer-implemented method of claim 11, further comprising automatically cropping at least a portion of the captured visual data of the first user or a second user by the first computing device. Method.   The area of the visual data representing the first user or the second user occupies less than a predetermined portion of the entire captured visual data of the first user or the second user. 15. The computer-implemented method of claim 14, further comprising dynamically cropping the at least part of the captured visual data of the first user or a second user based on the determination .   16. At least one machine readable comprising a plurality of instructions that, when executed on a computing device, cause the computing device to perform the computer-implemented method of any one of claims 1-15. Medium.   An apparatus comprising means for performing the computer-implemented method according to any one of claims 1 to 15. One or more processors;
A memory operably coupled to the one or more processors;
Display,
A control module included in the memory and operated by the one or more processors to facilitate simultaneous consumption of media content by a first user of the system and a second user of a remote computing device;
The promoting the media content being presented on the display of the second user's animation based on the captured visual data of the second user received from the remote computing device. Including overlays on top of
The system, wherein the animation is visually enhanced when it is determined that the first user is interested in the second user. An eye tracking device,
The system of claim 18, wherein the determination regarding the first user's interest is based on data received from the eye tracking device. The remote computing device is a first remote computing device;
The control module further determines the media content presented on the system when it is determined that the first user is not interested in the second user or not interested in the third user. 19. The system of claim 18, wherein the system does not visually enhance the overlay of the third user's animation on a second computing device.   21. The system of claim 20, wherein the control module further renders the overlay of the animation of the third user with a shadow.   The system according to any one of claims 18 to 21, wherein the second user's animation overlay includes the second user's animation overlay adjacent to the bottom of the display.   The system according to any one of claims 18 to 21, wherein the control module further utilizes parallax correction for the superposition of the animation of the second user.   22. The control module according to any one of claims 18 to 21, wherein the control module further superimposes a text manifest station of the second user's voice on the media content presented on the display. system.   The system according to any one of claims 18 to 21, further comprising a touch screen display.

Images