WO2015035247A1 - Fenêtre virtuelle - Google Patents

Fenêtre virtuelle Download PDF

Info

Publication number
WO2015035247A1
WO2015035247A1 PCT/US2014/054409 US2014054409W WO2015035247A1 WO 2015035247 A1 WO2015035247 A1 WO 2015035247A1 US 2014054409 W US2014054409 W US 2014054409W WO 2015035247 A1 WO2015035247 A1 WO 2015035247A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
user
display device
party
view
Prior art date
Application number
PCT/US2014/054409
Other languages
English (en)
Inventor
Matthew B. Shoemake
Syed Nadeem Ahmed
Original Assignee
Biscotti Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Biscotti Inc. filed Critical Biscotti Inc.
Publication of WO2015035247A1 publication Critical patent/WO2015035247A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
    • H04N7/147—Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting

Definitions

  • the present disclosure relates, in general, to tools and techniques for implementing video communications or presenting media content, and, more particularly, to tools and techniques for sensing the presence and/or position of a user in a room, and/or for customizing displayed content (including video call content, media content, and/or the like) based on the sensed presence and/or position of the user.
  • VCDs video communication devices
  • a display such as a television, to name one example
  • a source of content such as a set-top box (“STB”), to name an example
  • STB set-top box
  • sensors such as a camera, a microphone, infrared sensors, and/or other suitable sensors.
  • Such devices present a powerful platform for various applications. Examples include, without limitation, video calling, instant messaging, presence detection, status updates, media streaming over the Internet, web content viewing, gaming, and DVR capability.
  • Another example of such value added services is the introduction of online gaming. Rather than playing a game by him- or herself, a user now can play most games in a multiplayer mode, using communication over the Internet or another network.
  • Enabling such services is a new class of user device, which generally features relatively high-end processing capability (which would have been unthinkable outside supercomputing labs just a few years ago), substantial random access memory, and relatively vast non-transient storage capabilities, including hard drives, solid state drives, and the like.
  • user devices can include, without limitation, the VCDs mentioned above, the presence detection devices ("PDDs") described in the '279 Application, various video game consoles, and the like.
  • PDDs presence detection devices
  • Such devices generally have a reliable, and relatively high-speed, connection to the Internet (to enable the value added services) and significant amounts of downtime, in which the processing and other capabilities of the devices are unused.
  • a set of embodiments provides tools and techniques to enable more lifelike audio and video communications (including, without limitation, audio/video calls, video games, media content, etc.), in which the images seen on a display device and/or the audio played through one or more speakers changes based on the position of the viewer relative to the display device/speakers.
  • certain embodiments can provide this functionality by being aware of the position or location of the viewer (or the viewer's eyes) via various means and adjusting the image (and/or audio) that is presented to the viewer in response to that position.
  • novel tools and techniques might be provided for sensing the presence and/or position of a user in a room, and/or for customizing displayed content (including video call content, media content, and/or the like) based on the sensed presence and/or position of the user.
  • a user device which might include, without limitation, a video calling device, an image capture device, a gaming console, etc.
  • a control server in communication with the user device over a network
  • adjusting an apparent view of the video or image(s) might comprise one or more of adjusting an apparent field of view of the video or image(s) and/or adjusting an apparent perspective of the video or image(s).
  • the video or image(s) displayed on the display device might comprise one of a video program, a television program, movie content, video media content, audio media content, game content, or image content, and/or the like.
  • the tools provided by various embodiments include, without limitation, methods, systems, and/or software products.
  • a method might comprise one or more procedures, any or all of which are executed by an image capture device ("ICD"), a presence detection device ("PDD”), and/or a computer system.
  • an embodiment might provide an ICD, a PDD, and/or a computer system configured with instructions to perform one or more procedures in accordance with methods provided by various other embodiments.
  • a computer program might comprise a set of instructions that are executable by an ICD, a PDD, and/or a computer system (and/or a processor therein) to perform such operations.
  • such software programs are encoded on physical, tangible, and/or non-transitory computer readable media (such as, to name but a few examples, optical media, magnetic media, and/or the like).
  • a method might comprise determining, with a user device comprising a camera, a position of a user relative to a display device in
  • the method might further comprise adjusting an apparent view of video on the display device in response to the determined position of the user relative to the display device.
  • adjusting an apparent view of video on the display device might comprise adjusting an apparent field of view of the video to correspond to the determined position of the user relative to the display device.
  • adjusting an apparent view of video on the display device might comprise adjusting an apparent perspective of the video to correspond to the determined position of the user relative to the display device.
  • the user device might comprise a video calling device, and wherein the video on the display device might comprise a video call.
  • the user device might comprise a video game console, and wherein the video on the display device might comprise a video game.
  • the video on the display device might comprise one of a video program, a television program, movie content, video media content, audio media content, game content, or image content.
  • the video on the display device might comprise a live video stream captured by a camera in a location remote from the user device.
  • the method might further comprise adjusting an audio track of the video in response to the determined position of the user relative to the display device.
  • a user device might comprise a sensor, a processor, and a computer readable medium having encoded thereon a set of instructions executable by the processor to cause the user device to perform one or more operations.
  • the set of instructions might comprise instructions for determining a position of a user relative to a display device in communication with the user device and instructions for adjusting an apparent view of video on the display device in response to the determined position of the user relative to the display device.
  • the user device might comprise the display device.
  • a method might comprise determining, with a video calling device, a position of a first party to a video call relative to a display device that displays video of a video call.
  • the method might further comprise adjusting an apparent view of the video call, based at least in part on the determined position of the first party to the video call.
  • the video calling device might comprise a video input interface to receive video input from a set-top box, an audio input interface to receive audio input from the set-top box, a video output interface to provide video output to the display device, an audio output interface to provide audio output to an audio receiver, a video capture device to capture video, an audio capture device to capture audio, a network interface, at least one processor, and a storage medium in communication with the at least one processor.
  • the storage medium might have encoded thereon a set of instructions executable by the at least one processor to control operation of the video calling device.
  • the set of instructions might comprise instructions for controlling the video capture device to capture a captured video stream, instructions for controlling the audio capture device to capture a captured audio stream, instructions for encoding the captured video stream and the captured audio stream to produce a series of data packets, and instructions for transmitting the series of data packets on the network interface for reception by a second video calling device.
  • adjusting an apparent view of the video call might comprise adjusting an apparent field of view of the video call.
  • determining a position of a first party might comprise determining a distance of the first party from the display device.
  • adjusting an apparent field of view of the video might comprise zooming the video based on the determined distance of the first party from the display device.
  • determining a position of a first party might comprises determining a horizontal position of the first party in a horizontal dimension of a plane parallel to a face of the display device.
  • adjusting an apparent field of view of the video might comprise panning the video in a horizontal direction, based on the determined horizontal position of the first party.
  • determining a position of a first party might comprise determining a vertical position of the first party in a vertical dimension of a plane parallel to a face of the display device.
  • adjusting an apparent field of view of the video might comprise panning the video in a vertical direction, based on the determined vertical position of the first party.
  • adjusting an apparent view of the video call might comprise modifying, at the video calling device, a video signal received by the video calling device.
  • the video might be received from a second video calling device.
  • Adjusting an apparent view of the video call might comprise instructing the second video calling device to adjust a view of one or more cameras of the second video calling device.
  • instructing the second video calling device to adjust a view of one or more cameras might comprise instructing the second video calling device to adjust a field of view of the one or more cameras.
  • the second video calling device might comprise an array of cameras.
  • the field of view of the one or more cameras might comprise a field of view of a composite image captured by a plurality of cameras within the array of cameras.
  • the apparent view of the video call might comprise a virtual perspective of the composite image.
  • the virtual perspective might represent a perspective of the first party to the video call relative to the display device.
  • instructing the second video calling device to adjust a view of one or more cameras might comprise instructing the second video calling device to adjust a perspective of the one or more cameras. In some cases, instructing the second video calling device to adjust a view of one or more cameras might comprise instructing the second video calling device to pan a camera in at least one of a horizontal dimension or a vertical dimension. According to some embodiments, instructing the second video calling device to adjust a view of a camera might comprise instructing the second video calling device to zoom a camera. In some instances, instructing the second video calling device to adjust a view of a camera might comprise instructing the second video calling device to crop frames of a video stream captured by the camera.
  • the method might further comprise determining, with the video calling device, that the first party has moved relative to the display device, and modifying the apparent view of the video call, in response to determined movement of the first party.
  • modifying the apparent view of the video call might comprise modifying an apparent perspective of the video call, in response to determined movement of the first party.
  • modifying the apparent view of the video call might comprise modifying the apparent view of the video call substantially in real time with the determined movement of the first party.
  • the video calling device might comprise a camera, and determining a position of a first party to a video call might comprise capturing one or more images of the first party with the camera.
  • the one or more images might comprise a video stream.
  • the method in some instances, might further comprise transmitting the video stream to a second video calling device as part of the video call.
  • determining a position of a first party to a video call might further comprise analyzing the one or more images to identify the position of the first party.
  • analyzing the one or more images might comprise identifying, in the one or more images, positions of one or more eyes of the first party to the video call.
  • an apparatus might comprise a computer readable medium having encoded thereon a set of instructions executable by one or more computers to cause the apparatus to perform one or more operations.
  • the set of instructions might comprise instructions for determining a position of a first party to a video call relative to a display device that displays video of a second party to the video call, and instructions for adjusting an apparent view of the video of the second party to the video call, based at least in part on the determined position of the first party to the video call.
  • a system might comprise a video calling device and a computer.
  • the video calling device might comprise at least one first processor and a first computer readable medium in communication with the at least one first processor.
  • the first computer readable medium might have encoded thereon a first set of instructions executable by the at least one first processor to cause the video calling device to perform one or more operations.
  • the first set of instructions might comprise instructions for determining a position of a first party to a video call relative to a display device that displays video of a second party to the video call.
  • the computer might comprise one or more second processors and a second computer readable medium in communication with the one or more second processors.
  • the second computer readable medium might have encoded thereon a second set of instructions executable by the one or more second processors to cause the computer to perform one or more operations.
  • the second set of instructions might comprise instructions for adjusting an apparent view of the video of the second party to the video call, based at least in part on the determined position of the first party to the video call.
  • the video calling device might comprise the computer.
  • the video calling device might comprise a first video calling device.
  • the system might further comprise a second video calling device that comprises a camera that records the video of the second party to the video call.
  • the instructions for adjusting an apparent field of view of the video of the second party to the video call might comprise transmitting, to the second video calling device, instructions for adjusting a field of view of the camera of the second video calling device.
  • the computer might be a control server separate from the video calling device.
  • the computer might be incorporated within a second video calling device that further comprises a camera that captures the video of the second party to the video call.
  • the video calling device might comprise a video input interface to receive video input from a set-top box, an audio input interface to receive audio input from the set-top box, a video output interface to provide video output to a display device, an audio output interface to provide audio output to an audio receiver, a video capture device to capture video, an audio capture device to capture audio, a network interface, one or more third processors, and a third storage medium in communication with the one or more third processors.
  • the third storage medium might have encoded thereon a third set of instructions executable by the one or more third processors to control operation of the video calling device.
  • the third set of instructions comprise instructions for controlling the video capture device to capture a captured video stream, instructions for controlling the audio capture device to capture a captured audio stream, instructions for encoding the captured video stream and the captured audio stream to produce a series of data packets, and instructions for transmitting the series of data packets on the network interface for reception by a second video calling device.
  • Fig. 1 is a block diagram illustrating a system for modifying an apparent view(s) of displayed content, based at least in part on sensed presence and/or determined position(s) of a user in a room, in accordance with various embodiments.
  • FIGS. 2 and 3 illustrate fields of view, in accordance with various embodiments.
  • FIGs. 4A-4F are general schematic diagrams illustrating techniques for adjusting an apparent field of view of a display device, in accordance with various embodiments.
  • FIGs. 5A and 5B are general schematic diagrams illustrating techniques for adjusting apparent fields of view of a display device for multiple users, in accordance with various embodiments.
  • Fig. 6 is a general schematic diagram illustrating a windowed field of view in relation to a sensor field of view, in accordance with various embodiments.
  • FIGs. 7A and 7B are general schematic diagrams illustrating a display device in use with one or more image capture devices, in accordance with various embodiments.
  • FIG. 8 is a block diagram illustrating another system for modifying an apparent view(s) of displayed content, based at least in part on sensed presence and/or determined position(s) of a user in a room, in accordance with various embodiments.
  • Fig. 9 is a process flow diagram illustrating a method of providing a virtual window or for modifying an apparent view(s) of displayed content, based at least in part on sensed presence and/or determined position(s) of a user in a room, in accordance with various embodiments.
  • Fig. 10 is a generalized schematic diagram illustrating a computer system, in accordance with various embodiments.
  • Fig. 11 is a block diagram illustrating a networked system of computers, which can be used in accordance with various embodiments. DETAILED DESCRIPTION OF CERTAIN EMBODIMENTS
  • PDDs Presence Detection Devices
  • Image Capture Devices
  • ICDs can contain or communicate with, inter alia, cameras, microphones, and/or other sensors (including, without limitation, infrared (“IR”) sensors). These sensors, in conjunction with the internal processing capability of the device, can allow the device to detect when a person is in the room. Additionally, through means such as facial recognition and voice detection, or the like, the devices also can automatically recognize who is in the room. More specifically, such devices can detect the presence of a particular individual.
  • ICDs might contain or communicate with, inter alia, image capture devices for capturing images or video of the person or people in the room.
  • ICDs might also contain or communicate with, inter alia, microphones, and/or other sensors (including, without limitation, infrared (“IR”) sensors). According to some embodiments, some ICDs might have similar functionality as PDDs.
  • presence detection can be local and/or cloud based.
  • the PDD or ICD itself might keep a list of all user profiles and will attempt to match an individual against its local list of all users.
  • cloud based detection the functionality of user detection can be moved into servers in the cloud.
  • a cloud based approach allows detection of a user's presence to be mobile among various devices (whether or not owned by, and/or associated with, the user). That same user can be detected on his or her device or on any other device that has the same capability and that is tied into the same cloud infrastructure.
  • a user's content, services, games, profiles e.g., contacts list(s), social media friends, viewing/listening/gaming patterns or history, etc.
  • videomail e.g., e-mail
  • content recommendations e.g., determined advertisements, preferences for advertisements, and/or preferences (e.g., content preferences, content recommendation preferences, notification preferences, and/or the like), etc.
  • preferences e.g., content preferences, content recommendation preferences, notification preferences, and/or the like
  • presence detection functionality can also allow for mobile presence detection that enables remote access and control of ICDs over a network, following automatic identification and authentication of the user by any device (e.g., PDD, ICD, or other device) so long as such device has authentication functionality that is or can be tied to the access and control of the ICDs, regardless of whether or not such device is owned or associated with the user.
  • PDD personal area network
  • ICD intranet control
  • the ability to remotely access and control one's ICDs over a network can follow the user wherever he or she goes, in a similar manner to the user' s content and profiles following the user as described in the '279 Application.
  • Such remote control of ICDs, as well as post-processing of video and/or image data captured by the ICDs is described in detail in the '263 Application (which is already incorporated by reference herein).
  • Various sensors on a PDD or an ICD can be used for user detection. Facial recognition can be used to identify a particular individual's facial characteristics, and/or voice detection can be used to uniquely identify a person.
  • PDDs, ICDs, and/or video calling devices may also have local data storage. This local data storage can be used to store a database of user profiles. The user profiles can contain the various mechanisms that can be used to identify a person, including username and password, facial characteristics, voice characteristics, etc. When sensors detect the facial features or capture the voice of a particular individual, that captured presence information can be compared against the characteristics of the users on the local storage. If a match is found, then the individual has been successfully identified by the device.
  • Presence information can be any data or information that can be used to determine the presence of a user, and/or to identify and/or authenticate such a user.
  • presence information can include raw image, video, or audio data, analyzed data (e.g., video or image data to which preliminary facial recognition procedures, such as feature extraction, have been employed, as well as verification of audio self- identification or verification of audio challenge/response information), the results of such analysis, and even the end result of the detection process - i.e., a notification that a user is present and/or an identification of the user.)
  • Detection of a user's presence can also be performed via proximity of a
  • a unique device identifier for each of a user's devices might have previously been associated with the user's profile in a cloud database or the like (i.e., making the user's devices "known devices"), and detection of such unique device identifiers might serve as a basis for identifying the user, or might streamline the identification process by verifying whether the person with the device owned by or associated with the known device is the user or simply someone in possession of the device(s) (whether lawful or unlawful).
  • Such verification might comprise one or more of facial recognition, voice recognition, audio challenge/response verification, biometric analysis, or the like.
  • audio challenge/response verification might include analysis of sub-vocal responses from the person challenged, to prevent undesired casual overhearing of audio passwords, audio keyphrases, or the like.
  • biometric analysis might include analysis of any suitable biometric (aside from facial and voice recognition) selected from a group consisting of fingerprint, iris, pupil, height, unique scar(s), other unique physical characteristics, and/or any combination of these biometrics.
  • biometric information such as fingerprints, iris, pupil, height, scar, or other unique physical characteristics
  • biometric information such as fingerprints, iris, pupil, height, scar, or other unique physical characteristics
  • the PDD, the ICD, and/or the video calling device might prompt the person being detected to position himself or herself so that his or her fingerprints, iris, pupil, full body, scar, or other unique physical characteristics, respectively, are appropriately facing the image capture device of the PDD and/or the ICD.
  • the system may be possible for the system to identify persons not normally associated with a known device being in possession of the known device. In such a case, the system might notify the original user (via e-mail or other forms of communication indicated in the user's profile, or the like) of the situation. In some instances, the user might indicate that the unknown person does have authority or permission to use, or be in possession of, the user's device.
  • the user may be given options to proceed, including, without limitation, options to lock data, options to lock device functions, options to activate location tracking (including, without limitation, global positioning system (“GPS”), global navigation satellite system (“GNSS”), etc.) of the device (in case the system loses track of the device; e.g., in the case the device moves outside the range of the system's sensor/detection/communications systems), options to contact the unknown person, options to activate speakers to emit sirens, options to activate displays or lights (e.g., light emitting diodes (“LEDs”), organic LEDs (“OLEDs”), liquid crystal displays (“LCDs”), etc.), and/or options to notify authorities (e.g., police or other law enforcement personnel) of the situation and/or the location of the device (e.g., GPS coordinates, or the like), etc.
  • GPS global positioning system
  • GNSS global navigation satellite system
  • the user may be given options to proceed, including, without limitation, options to lock data, options to lock device functions, options to
  • proximity detection can be done using GNSS location tracking functionality, which can be found in many electronic devices and authenticating the user when the secondary device is within a predefined distance of the PDD, the ICD, and/or the video calling device.
  • Proximity detection can also be done wirelessly via Bluetooth or WiFi. With respect to Bluetooth, if the secondary device pairs with the PDD, the ICD, and/or the video calling device, the user can be considered detected. With respect to WiFi, one approach could be to see if the secondary device associates with the same WiFi access point to which the PDD, the ICD, and/or the video calling device is connected.
  • NFC near-field communications
  • a NFC detector can be used to determine that the user is in the room. From these examples, a skilled reader should appreciate that many different techniques can be used to detect presence based on device proximity.
  • presence may be determined or inferred by knowing the location of the personal device (which might include, without limitation, at least one of a laptop computer, a smart phone, a mobile phone, a portable gaming device, a desktop computer, a television, a set-top box, or a wearable computing device, and/or the like).
  • the personal device is close to the display device (or the PDD, ICD, and/or video calling device), it may be determined that the personal device (and hence the user associated with the personal device) is present.
  • advertisement content (which may be determined to be relevant to the user) may be sent to the display device.
  • a highly targeted advertising may be implemented (which may be embodied, in some cases, as a highly targeted form of television advertisement, which may be thought of as being similar to what is done on web browsers today, but much more targeted).
  • recommendations of media content and/or (in some cases, automatic) presentation of recommended media content may also be based on the presence of the user and information about the user.
  • the PDD/ICD/video calling device may be one of the personal device itself, a computer/server in the cloud, and/or the personal device in conjunction with some computer/server in the cloud, or the like.
  • the recommended media content and/or advertisement may be sent to a local content source (e.g., an STB or the like) or another PDD/ICD/video calling device that has the ability to control content being played or sent to the display device (and/or, of course, to receive the recommended media content and/or advertisement from a content server).
  • a local content source e.g., an STB or the like
  • another PDD/ICD/video calling device that has the ability to control content being played or sent to the display device (and/or, of course, to receive the recommended media content and/or advertisement from a content server).
  • Such a method or apparatus may allow for the targeted presentation (or, in some cases, selling) of recommended media content and/or advertisements directly to the display device (e.g., TV or the like), based on characteristics of the user.
  • determination of recommended media content and/or advertisements to send to the display device might be based on, or might otherwise take into account, the user's Internet browsing history, the user's Internet browsing patterns, the user's Internet browser bookmarks/favorites, and/or the like.
  • detection of an individual can be fully automatic and might (in some instances) require no user interaction.
  • the system can characterize an individual's facial features (and/or unique physical characteristics or other biometrics) automatically, detect the presence of a secondary device, characterize an individual's voice print automatically, etc.
  • Several detection methods can be used in combination to reduce errors in the detection process. For example, if the system detects a person in the room and first identifies that person's facial features, it can then prompt them for voice (e.g., "Bob, is that you?"). Once the user' s voice is captured, that audio sample can be compared against the stored voice characteristics for that user, to reduce false detection.
  • Another approach for the second step may be to prompt the user to speak a PIN or password to be compared against what is stored in the user profile.
  • the characteristics of the speech e.g., user's voice, cadence, syntax, diction
  • the content of the speech e.g., a PIN or password
  • the audio capture device might be configured to capture sub-vocalizations of the passwords or PINs, for analysis.
  • the system can prompt the user to position his or her body so as to allow the image capture device to face one or more of the user's fingers (e.g., for fingerprint analysis), the user's eyes (e.g., for iris and/or pupil analysis), the user's full body (e.g., for height analysis), portions of the user's body (e.g., for analysis of scars or other unique physical characteristics, or the like), etc.
  • the user's fingers e.g., for fingerprint analysis
  • the user's eyes e.g., for iris and/or pupil analysis
  • the user's full body e.g., for height analysis
  • portions of the user's body e.g., for analysis of scars or other unique physical characteristics, or the like
  • physical geography can be used as a metric in detection to reduce the possibility of errors. For example, if a user is known to use the system in Dallas, Texas, and then is detected in Madrid, Spain, the system can weigh detection in Spain lower than detection in Dallas. Additionally, if the user is detected in Spain, a secondary authentication method may optionally be invoked to reduce false detection. According to some embodiments, in the case that the system has access to profile or other personal information of the user such as
  • the user's profiles, media content, preferences, content recommendations, determined advertisements, preferences for advertisements, or the like (or access thereto) might be sent to the friend's or relative's device in Spain or to a local data center or the like to allow the user to access the user's own content or profiles on the friend's or relative's device during the visit;
  • the user's profiles might include access and control information for remotely accessing and controlling the user's ICDs over a network
  • the user's content might include image data and/or video data captured by the user's ICDs (either in raw or processed form).
  • the user's personal information including the user's PDD, ICD, and/or video calling device, mobile devices, etc.
  • the friend's or relative's device whether the user has left the friend's or relative's location (in this example, Spain). If so determined, the content and profiles (or access thereto, as the case may be) might be removed from the friend's or relative's device (and/or from the data center or the like that is local to said device).
  • a PDD, an ICD, and/or a video calling device can also be connected to a network, such as the Internet.
  • the database of user profiles including identifiable facial and/or voice characteristics, as well as other identifying information (e.g., passwords, identifying information for other devices owned by the user, etc.), can be stored on servers located in the cloud, i.e., on the network or in a distributed computing system available over the network.
  • the distributed computing system might comprise a plurality of PDDs, a plurality of ICDs, and/or a plurality of video calling devices in communication with each other either directly or indirectly over the network.
  • the distributed computing system might comprise one or more central cloud servers linking the plurality of PDDs, the plurality of ICDs, and/or the plurality of video calling devices and controlling the distribution and redundant storage of media content, access to content, user profiles, user data, content recommendations, determined advertisements, preferences for advertisements, and/or the like.
  • a server on the network can be sent to a server on the network.
  • the server can compare the identifiable facial features against the database of user profiles. If a match is found, then the server might inform the device of the identity of the user and/or might send a user profile for the user to the device.
  • User profiles can be stored both locally on the device and on a server located in the cloud.
  • user identification can be performed by first checking the local database to see if there is a match, and if there is no local match, then checking the cloud-based database.
  • the database on the device can be configured to stay synchronized with the database in the cloud. For example, if a change is made to a user profile on the device, that change can be sent to the server and reflected on the database in the cloud. Similarly, if a change is made to the user profile in the cloud- based database, that change can be reflected on the device database.
  • Matching presence information or identifying information with an individual having a user profile can be a form of authentication in some embodiments.
  • User profiles can also contain information necessary for many authentication mechanisms. Such information may include challenge/response pairs (such as username and password combinations, security question/pass phrase combinations, or the like), facial recognition profiles, voice recognition profiles, and/or other biometric information, such as fingerprints, etc.
  • An individual may be authenticated using any combination of such techniques.
  • the system can also determine when a user is no longer present.
  • a PDD, an ICD, and/or a video calling device might continually (or periodically) monitor for the user's presence. For instance, in the case of facial recognition, the device can continually check to detect whether a captured image includes the user's face. With voice recognition, after a period of inactivity, the device might prompt the user if they are there (e.g., "Bob, are you still there?").
  • user profiles can work across heterogeneous networks. Not all user devices need to be the same. Some user devices might be PDDs, ICDs, and/or video calling devices. Other user devices might be computers, tablets, smart phones, mobile phones, etc. Each device can use any appropriate method (based on device capabilities) to determine the presence of, identify, and/or authenticate the user of the device with a user profile.
  • this automated presence detection can be used to provide user information (e.g., content, content recommendations, determined advertisements, preferences for advertisements, and/or services) to an identified user.
  • user information e.g., content, content recommendations, determined advertisements, preferences for advertisements, and/or services
  • a PDD personal area network
  • ICD intranet control
  • video calling device when a user enters the room, and the camera sensors detect that user's facial features (or other biometric features) and authenticates the individual, the content associated with that user profile (including, without limitation, profile information for handling media content, for handling content recommendations, for handling notification of content recommendations, for handling determination of advertisements, for handling presentation of advertisements, and/or the like) can automatically become available to that individual.
  • that user's content, content recommendations, determined advertisements, preferences for advertisements, and/or profiles can become available on any device. More specifically, if a user is identified by another PDD, ICD, and/or video calling device, then his or her content (e.g., media content, and/or the like), content recommendations, determined advertisements, preferences for advertisements, profiles, etc., become available to him or her even if the PDD, ICD, and/or video calling device that he or she is in front of is not the user's own device.
  • This functionality allows a new paradigm in which the user's content, content recommendations, determined advertisements, preferences for advertisements, and/or profiles follow the user automatically.
  • detection, identification, and authentication of the user on the new device can allow automatic and easy porting of the user's content, content recommendations, determined advertisements, preferences for advertisements, and/or profiles to the new device, allowing for an ultimate type of "plug-and-play" functionality, especially if the profiles include information on configurations and settings of the user devices (and interconnections with other devices).
  • PDDs, ICDs, and/or video calling devices also are capable of handling, transmitting, and/or distributing image captured content, which can include, but is not limited to, video mail and/or video mail data captured or recorded by the video calling devices.
  • the video mail and/or video mail data might be raw data, while in other cases they might be post-processed data.
  • Video mail and/or video mail data can be stored on servers in the cloud, on PDDs, ICDs, and/or video calling devices in the cloud, and/or locally on a particular user device.
  • the first PDD and/or video calling device When accessing video mail and/or video mail data from another device, the first PDD and/or video calling device that has the video mail and/or video mail data stored thereon needs to serve the video mail and/or video mail data to the new device that the user is using. In order to do this, the new PDD, ICD, and/or video calling device might need to get a list of video mail and/or video mail data that is stored on the first PDD and/or video calling device. This can, in some embodiments, be facilitated via a server that is in the cloud that all PDDs, ICDs, and/or video calling devices are always or mostly connected to.
  • the server can communicate with all PDDs, ICDs, and/or video calling devices and help send messages between PDDs, ICDs, and/or video calling devices.
  • the new device can request the list of video mail and/or video mail data from the first device. If the user requests video mail and/or video mail data from the new device, then the first PDD, ICD, and/or video calling device (or the other user device) can serve the video mail and/or video mail data to the new device. This can be done either directly in a peer-to-peer fashion and/or can be facilitated by the server.
  • peer-to-peer sessions might be initiated using a server, and after a peer-to-peer session has been initiated or established by the server, the server may be by-passed, resulting in a direct peer-to-peer connection or session.
  • this communication can be accomplished by using protocols such as XMPP, SIP, TCP/IP, RTP, UDP, etc.
  • Videomail capture, processing, and distribution is described in detail in the '499 Application, which is already incorporated herein by reference.
  • PDD, an ICD, and/or a video calling device can provide the user with remote access and control of the user's PDD(s), ICD(s), and/or video calling device(s) over a network (e.g., by porting the user's profiles associated with remote access and control of the user's device(s), and/or the like to the current PDD, ICD, and/or video calling device in front of which the user is located).
  • This functionality allows the user to remotely access media content, to remotely access and modify settings for content recommendations, to remotely access and modify settings for advertisements, and to remotely access and modify user profiles, and/or the like.
  • a master account for access to a video calling device.
  • a master account can be created on a per user basis. This master account might serve as the top-level identifier for a particular user.
  • the master account may be used to manage, control, and monitor a user's camera(s) and/or other device functionalities (whether hardware and/or software - based). Additionally, the master account can be used to control any account or device level services that are available.
  • an email account and password can be used as a master account to manage a user's settings for accessing media content, for accessing and modifying settings for content recommendations, for accessing and modifying settings for advertisements, and for accessing and modifying user profiles, and/or the like.
  • some embodiments provide the ability to reliably associate a PDD, ICD, and/or video calling device with a master account (i.e., assign the device to the master account).
  • a PDD, ICD, and/or video calling device When a PDD, ICD, and/or video calling device is associated with an account, then it can be managed and controlled from within the master account. Association ensures that a PDD, ICD, and/or video calling device is being controlled by the appropriate user and not an unauthorized user.
  • a PDD, ICD, and/or video calling device may be associated with a particular master account at the time of the device setup.
  • the user is prompted to enter a master account and password.
  • a secure communications channel may be opened up between video calling device and servers.
  • a unique and difficult to guess key can be sent from the device to the server.
  • Servers that have a master list of all keys then can associate that particular device, via its serial number, to a particular master account.
  • a feature of this approach is that a user only needs to enter a password at the time of device setup. The user never needs to enter a password again, and in fact, passwords do not need to be stored on the device at all, making them very secure.
  • a device Once a device has been associated with a master account, it may be managed from the master account via an interface such as a web interface, in accordance with some embodiments.
  • the communication link between the device and server may, in some cases, be always encrypted and authenticated. This ensures that messages between device and server are secure and ensures that the device knows it is communicating with the server on behalf of the appropriate master account. Once the secure and authenticated link is established, devices can connect to the server and are able to send and receive commands.
  • the device and server can have a common set of command codes and responses.
  • Servers can send commands down to the camera(s) to enact specific behavior.
  • the server can send remote configuration commands. These commands can be items such as changing the device address, changing the nickname that is associated with the device, changing the avatar image associated with the device.
  • the commands can be used to enact specific behavior on the device, such as running network tests, or taking a live image(s) from the video calling device. New commands and features can be added by extending the set of command codes on the device and server.
  • a set of embodiments can provide a "virtual window" that includes an apparent view of video content (or still images) that corresponds to a user's position with respect to the display device (such as a television or other display device) on which the video content (or still images) is displayed.
  • the video content might include video of parties in a video call, video of media content (e.g., movie content, television program content, gaming content, advertisement content, and/or the like), video of a live video feed, and/or the like.
  • embodiments can also adjust audio (which might be an audio track of the video content or might be a standalone audio stream with no accompanying video), using similar techniques, based on the position of a listener with respect to a video display (or any other specified point).
  • audio which might be an audio track of the video content or might be a standalone audio stream with no accompanying video
  • the effect of some embodiments is to make the displayed video appear to the user as if the user is watching the video through a virtual window, such that the apparent view of the video changes depending on the user's location relative to the virtual window (i.e., display device or the like), and can be modified in real-time (or near real-time, if the user moves with respect to the display device).
  • virtual window is used only for purposes of illustrating the concepts described herein and should not be considered limiting in any way.
  • the "apparent view” that can be adjusted by various embodiments can include an apparent field of view and/or an apparent perspective on the video.
  • an "apparent field of view,” as used herein, means the field of view (i.e., portion of the scene that is displayed) that the user perceives when watching the video (which is analogous to the field of view of a real or virtual camera that captured the scene depicted in the video).
  • An "apparent perspective” is the perspective (e.g., above, below, straight in front, on one side or the other, or any suitable combination of these perspectives) from which the user perceives that he or she is viewing the scene depicted on the video, and it is analogous to the perspective of the real or virtual camera that captured the scene displayed in the video.
  • the term “virtual camera” is used to convey an embodiment in which the displayed video is not actually live-filmed video but is generated, such as animated video or video from a video game; such generated video has a field of view and a perspective, just as live-recorded video, which is represented by a virtual camera.
  • description of movement of a user's eyes might refer to physical movement of the user's eyes relative to the display device, and not merely rotation of the user's eyes (which is merely a change in the focus of the user's visual field of view, and, in some cases, might not affect the displayed field of view through the virtual window).
  • physically moving so as to change one's eyes along x, y, or z directions relative to a virtual window might change the field of view looking through the window, but simply rotating one's eyes (without changing position of one's eyes along any of the x, y, or z directions relative to the virtual window) might not affect the field of view looking through the virtual window.
  • FIGs. 1-11 illustrate exemplary embodiments that can provide some or all of the features described above.
  • the methods, systems, and apparatuses illustrated by Figs. 1-11 may refer to examples of different embodiments that include various components and steps, which can be considered alternatives or which can be used in conjunction with one another in the various embodiments.
  • the description of the illustrated methods, systems, and apparatuses shown in Figs. 1-11 is provided for purposes of illustration and should not be considered to limit the scope of the different embodiments.
  • Fig. 1 illustrates an exemplary environment that can provide some or all of the features described herein, including, but not limited to, modifying an apparent view(s) of displayed content (including, without limitation, video call content, media content, and/or the like), based at least in part on sensed presence and/or determined position(s) of a user in a room, in accordance with various embodiments. More specifically, Fig. 1 illustrates a functional diagram of a system 100 for controlling one or more presence detection devices ("PDDs"), one or more image capture devices (“ICDs”), and/or one or more video calling devices (labeled user devices 105 in Fig.
  • PDDs presence detection devices
  • ICDs image capture devices
  • video calling devices labeled user devices 105 in Fig.
  • Fig. 1 for ease of illustration, but described herein as PDDs, ICDs, or video calling devices, each of which can be considered a type of user device).
  • PDDs personal digital assistants
  • ICDs intracellular digital converters
  • video calling devices each of which can be considered a type of user device.
  • the arrangement of the components illustrated in Fig. 1 is functional in nature, and that various embodiments can employ a variety of different structural architectures.
  • one exemplary, generalized architecture for the system 100 is described below with respect to Fig. 11, but any number of suitable hardware arrangements can be employed in accordance with different embodiments.
  • An ICD 105, a video calling device 105, or a PDD 105 can be any device that is capable of communicating with a control server 110 over a network 115 and can provide any of a variety of types of advertisement determination
  • an ICD 105, a video calling device 105, or a PDD 105 can be capable of providing pass through video/audio to a display device (and/or audio playback device) from another source (such as a local content source), and/or overlaying such video/audio with additional content generated or received by the ICD 105, the video calling device 105, or the PDD 105.
  • an ICD 105, a video calling device 105, or a PDD 105 can comprise one or more sensors (e.g., digital still cameras, video cameras, webcams, security cameras, microphones, infrared sensors, touch sensors, and/or the like), and/or can be capable, using data acquired by such sensors, of sensing the presence of a user, identifying a user, and/or receiving user input from a user; further, an ICD 105, a video calling device 105, or a PDD 105 can be capable of performing some or all of the other functions described herein and/or in any of the Related Applications.
  • sensors e.g., digital still cameras, video cameras, webcams, security cameras, microphones, infrared sensors, touch sensors, and/or the like
  • an ICD 105, a video calling device 105, or a PDD 105 can be embodied by a video calling device, such as any of the video communication devices ("VCDs") described in the '182 Patent, a video game console, a streaming media player, to name a few non-limiting examples.
  • VCDs video communication devices
  • an ICD 105, a video calling device 105, or a PDD 105 can be placed functionally inline between a local content source and a display device.
  • a local content source can be any device that provides an audio or video stream to a display device and thus can include, without limitation, a cable or satellite set-top box (“STB”), an Internet Protocol television (“IPTV”) STB, devices that generate video and/or audio, and/or acquire video and/or audio from other sources, such as the Internet, and provide that video/audio to a display device; hence, a local content source can include devices such as a video game console, a Roku ® streaming media player, an AppleTV ® , and/or the like.
  • STB cable or satellite set-top box
  • IPTV Internet Protocol television
  • the ICD, the video calling device, or the PDD can receive an audiovisual stream output from the local content source, modify that audiovisual stream in accordance with the methods described herein, in the '182 Patent, and/or in the '279 Application, and provide the (perhaps modified) audiovisual stream as input to the display device.
  • a local content source can be incorporated within an ICD, a video calling device, or a PDD, and/or the functionality of an ICD, a video calling device, or a PDD can be incorporated within a local content source; further, it should be appreciated that an ICD, a video calling device, or a PDD (which might or might not include local content source functionality) can be disposed inline with one or more other local content sources or one or more other video calling devices/PDDs. Hence, for example, an ICD, a video calling device, or a PDD with some local content source functionality (such as a video game console) might be disposed inline between one or more other local content sources or one or more other ICDs/video calling
  • devices/PDDs such as a cable STB, satellite STB, IPTV STB, and/or a streaming media player
  • a display device such as a TV, TV, and/or a TV station.
  • the system can include a software client that can be installed on a computing device (e.g., a laptop computer, wireless phone, tablet computer, etc.) that has a built-in camera and/or has a camera attached (e.g., a USB webcam).
  • a software client can be installed on a computing device (e.g., a laptop computer, wireless phone, tablet computer, etc.) that has a built-in camera and/or has a camera attached (e.g., a USB webcam).
  • This client can act as an interface to allow remote control of the built-in and/or attached camera on the computing device.
  • the computing device might have a built-in microphone(s) and/or has a microphone(s) attached (e.g., a table-top microphone, a wall-mounted microphone, and/or a microphone removably mountable on a television, on the ICD, on the video calling device, on the PDD, and/or on some other suitable user device, or the like).
  • the software client can alternatively and/or additionally act as an interface to allow remote control of the built-in and/or attached microphone on the computing device.
  • the camera and/or microphone can be automatically or autonomously controlled to obtain optimal video and/or audio input. Remote control of the video calling device and/or PDD is described in detail in the '263 Application (already incorporated herein), and may be similarly applicable to remote control of the ICD.
  • the system 100 can further include a control server 110, which can have any suitable hardware configuration, and an example of one such configuration is described below in relation to Fig. 11.
  • the control server 110 is a computer that is capable of receiving user input via a user interface 120 and/or performing operations for utilizing the ICD(s) 105, the video calling device(s) 105, and/or the PDD(s) 105 to perform one or more of receiving (and relaying) media content (either directly from a media content server or database (both not shown) via network 115, indirectly via a local content source (e.g., an STB or the like), directly from cloud storage system 130, and/or the like), monitoring the media content presented to the user(s), monitoring the user(s), sending the monitored data to the control server 110, determining content recommendations, determining at least one advertisement for the user(s) with the control server 110, receiving the at least one advertisement for the user(s) from the control server 110, presenting the at least one advertisement to the user(s),
  • control server 110 might handle all of the processes for identifying and authenticating users and for providing access to the user(s)'s profiles, content, information, recommendations, advertisements, preferences (including, without limitation, preferences for advertisements and other user preferences, etc.), as well as handling the processes involved with determining or presenting the advertisements, and/or handling processes involved with position(s) determination of the user(s) (and/or eyes of the user(s)) and handling modification/adjustment of the apparent view of content displayed on a display device based on the determined position(s) of the user(s) (and/or eyes of the user(s)).
  • the processes involved with position(s) determination of the user(s) (and/or eyes of the user(s)) and/or handling modification/adjustment of the apparent view of content displayed on a display device based on the determined position(s) of the user(s) (and/or eyes of the user(s)) might be handled by the user device 105 corresponding to the user(s) and/or to the display device.
  • control server 110 and the particular user device 105 might split the processing tasks in any suitable manner, as appropriate
  • control server controls the control server
  • control server 110 can detect user presence, identify/authenticate users, and/or enable the user to remotely access the user's master account, user preferences, media content, recommendations of media content, advertisements, preferences for advertisements, and/or the like.
  • the control server 110 can receive and/or store user input and/or user preferences that can specify whether and how presence information should be used, whether and how the user's ICD(s), video calling device(s), and/or PDD(s) may be used in a distributed infrastructure, whether and how the user's content and profiles should be handled under certain situations, and/or the like.
  • preferences might specify which account information, content, profile information, personal communications (e.g., videomail, voicemail, e-mail, etc.), media content, media content recommendations, determined advertisements, preferences for advertisements, and/or the like should be delivered to a user when present at a device not owned by the user, whether presence information should be collected for that user at all (and/or where such information should be collected); for example, a user might specify that his presence should only be monitored in selected locations or from selected devices, and the control server 110 might remove that user' s profile from the search universe when provided with presence information from a device not at the selected location or from a device other than one of the selected devices.
  • the user preference can include any types of parameters related to collecting presence information, using presence information, handling media content recommendations, handling advertisements, and/or serving content/information (including, without limitation, user account information, user content, user profile information, user's personal communications (e.g., videomail, videomail, voicemail, e-mail, etc.), media content, advertisements, and/or the).
  • content/information including, without limitation, user account information, user content, user profile information, user's personal communications (e.g., videomail, videomail, voicemail, e-mail, etc.), media content, advertisements, and/or the).
  • preferences might be stored in a user profile at the control server 110, which might also include other user-specific information, such as the user's normal location(s), identifying information (such as MAC address, etc.) of other user devices owned by or associated with the user, lists of or links to content owned by the user, lists of or links to media content recommendations, lists of or links to preferences for handling media content recommendations, lists of or links to advertisements, lists or links to products or services associated with advertisements, lists of or links to preferences for handling advertisements, and/or the like.
  • user-specific information such as the user's normal location(s), identifying information (such as MAC address, etc.) of other user devices owned by or associated with the user, lists of or links to content owned by the user, lists of or links to media content recommendations, lists of or links to preferences for handling media content recommendations, lists of or links to advertisements, lists or links to products or services associated with advertisements, lists of or links to preferences for handling advertisements, and/or the like.
  • user preferences might specify how the user would like his or her user devices to participate (or not) in a distributed infrastructure arrangement.
  • the user preferences might include, without limitation, preferences indicating whether or not to allow a user device owned by the user to be used for distributed infrastructure; preferences indicating what type of software applications, customer data, media content (of other user device users and/or subscribers of a cloud service), and/or advertisements are permitted to be hosted on a user device owned by the user; and/or preferences indicating amount of resources of a user device to dedicate to the distributed infrastructure; etc.
  • preferences indicating how a user's user device may be used in distributed
  • user preferences might allow a user to indicate how the user's own applications, data, and/or media content may be hosted on other users' user devices. For example, the user might be given the option to encrypt any and/or all personal data, any and/or all personal applications, any and/or all files or lists indicating which media content are associated with the user, any and/or all files or lists pertaining to media content recommendations and/or preferences thereof, and/or any and/or all files or lists pertaining to advertisements and/or preferences thereof.
  • Common media content (which might include popular media content, or any other media content) may remain unencrypted for common usage by any number of users on any number of user devices, subject only to any subscription, rental, or purchase restrictions on the particular media content as associated with any user and/or any user device.
  • the user's personal communications including, e.g., videomail messages and/or the like
  • preferences for media content recommendations may be encrypted.
  • the control server 110 can provide a user interface (which can be used by users of the ICDs 105, the video calling devices 105, and/or the PDDs 105, and/or the like).
  • the control server 110 might also provide machine-to-machine interfaces, such as application programming interfaces ("APIs"), data exchange protocols, and the like, which can allow for automated communications with the video calling devices 105 and/or the PDDs 105, etc.
  • the control server 110 might be in communication with a web server 125 and/or might incorporate the web server 125, which can provide the user interface, e.g., over the network to a user computer (not shown in Fig. 1) and/or a machine-to-machine interface.
  • control server 110 might provide such interfaces directly without need for a web server 125. Under either configuration, the control server 110 provides the user interface 120, as that phrase is used in this document. In some cases, some or all of the functionality of the control server 110 might be implemented by the ICD 105, the video calling device 105, and/or the PDD 105 itself.
  • the user interface 120 allows users to interact with the control server 110, and by extension, the ICDs, the video calling devices 105, and/or the PDDs 105.
  • a variety of user interfaces may be provided in accordance with various embodiments, including, without limitation, graphical user interfaces that display, for a user, display fields on display screens for providing information to the user and/or receiving user input from a user.
  • control server controls the control server
  • the user interface 110 may be configured to communicate with a user computer (not shown in Fig. 1) via a dedicated application running on the user computer; in this situation, the user interface 120 might be displayed by the user computer based on data and/or instructions provided by the control server 110. In this situation, providing the user interface might comprise providing instructions and/or data to cause the user computer to display the user interface. In other embodiments, the user interface may be provided from a web site, e.g., by providing a set of one or more web pages, which might be displayed in a web browser running on the user computer and/or might be served by the web server 125.
  • control system 110 might comprise the web server and/or be in communication with the web server 125, such that the control server 110 provides data to the web server 125 to be incorporated in web pages served by the web server 125 for reception and/or display by a browser at the user computer.
  • the network 115 can be any network, wired or wireless, that is capable of providing communication between the control server 110 and the ICDs 105, the video calling devices 105, and/or the PDDs 105, and/or of providing communication between the control server 110 (and/or the web server 125) and a user computer.
  • the network 115 can comprise the Internet, and/or any Internet service provider ("ISP") access networks that provide Internet access to the control server 110, the user computer, and/or the ICDs 105, the video calling devices 105, and/or the PDDs 105.
  • ISP Internet service provider
  • the system 100 can include a cloud storage system 130, which can be used, as described in further detail below, to store advertisements, presence information, images, video, videomail messages, media content, media content recommendations, determined advertisements, preferences for advertisements, preference information of users, past viewing/listening/playing patterns or decisions of users, and/or the like that are monitored/captured, downloaded, streamed, and/or uploaded by the ICDs 105, the video calling devices 105 and/or the PDDs 105, and/or the like.
  • the cloud storage system 130 might be a proprietary system operated by an operator of the control server 110.
  • the cloud storage system 130 might be operated by a third party provider, such as one of the many providers of commercially available cloud services.
  • the cloud storage system 130 might be implemented by using resources (e.g., compute, memory, storage network, etc.) shared by a plurality of video calling devices, and/or by a plurality of PDDs, that are distributed among various users of the system.
  • resources e.g., compute, memory, storage network, etc.
  • a plurality of user video calling devices and/or PDDs might each have some dedicated resources (such as a storage partition), which are dedicated for use by the system, and/or some ad hoc resources (such as network bandwidth, memory, compute resources, etc.) that are available to the system when not in use by a user.
  • resources can be used as cloud storage and/or can be used to provide a distributed, cloud-like platform on which a control server can run as a virtual machine, cloud container, and/or the like.
  • ICD 105, video calling device 105, and/or PDD 105 might comprise a first video input interface to receive first video input from a first local content source (which in some embodiments can include a STB and/or the like) and a first audio input interface to receive first audio input from the first local content source.
  • Video calling device 105 might further comprise a first video output interface to provide first video output to a first video display device and a first audio output interface to provide first audio output to a first audio receiver.
  • the first video display device and the first audio receiver might be embodied in the same device (e.g., a TV with built-in speaker system, or the like).
  • video calling device 105 might provide pass- through capability for video and/or audio between the first local content source and the first display device.
  • high-definition multimedia interface (“HDMI") cables or other suitable HD signal cables may be used to provide the interconnections for the pass-through.
  • Video calling device 105 may, in some cases, comprise a first image capture device to capture at least one of first image data or first video data and a first audio capture device to capture first audio data.
  • Video calling device 105 may also comprise a first network interface, at least one first processor, and a first storage medium in communication with the at least one first processor.
  • a plurality of ICDs, PDDs, or video calling devices are provided.
  • each ICD, PDD, or video calling device being located in one of a plurality of customer premises.
  • a computer might establish one or more ICDs, PDDs, or video calling devices 105 of the plurality of ICDs, PDDs, or video calling devices 105 as distributed infrastructure elements and might provide at least one of one or more software applications, customer data, and/or media content to the one or more video calling devices 105 for hosting on the one or more video calling devices 105.
  • a user can remotely access one or more ICDs, PDDs, or video calling devices 105 and/or remotely access at least one of the user's master account, the user's user preference, the user's profiles, any videomail messages addressed to the user, the user's media content, media content recommendations for the user, determined advertisements, preferences for advertisements, and/or the like over a network.
  • a user could log into the user's master account by accessing a website hosted on a web server (e.g., web server 125, which might be hosted on a cloud server, hosted on distributed PDDs, hosted on distributed video calling devices, and/or the like) and entering commands into a user interface (e.g., user interface 120) associated with remotely accessing the user's video calling device(s) 105 and/or associated with remotely accessing at least one of the user's master account, the user's user preference, the user's profiles, any videomail messages addressed to the user, the user's media content, media content recommendations for the user, determined advertisements of the user, the user's preferences for advertisements, and/or the like.
  • a web server e.g., web server 125, which might be hosted on a cloud server, hosted on distributed PDDs, hosted on distributed video calling devices, and/or the like
  • a user interface e.g., user interface 120
  • the user might access and interact with the user interface over the network (e.g., network 115) by using a user computer selected from a group consisting of a laptop computer, a desktop computer, a tablet computer, a smart phone, a mobile phone, a portable computing device, and/or the like.
  • a user computer selected from a group consisting of a laptop computer, a desktop computer, a tablet computer, a smart phone, a mobile phone, a portable computing device, and/or the like.
  • a software application or "app” running on the user's user device, which might include, without limitation, a laptop computer, a desktop computer, a tablet computer, a smart phone, a mobile phone, a portable computing device, and/or the like.
  • the app might include another user interface (similar to the web-based user interface) that might allow for access of the user's video calling device(s) (or any paired video calling device(s)) over the network (e.g., network 115) and/or that might allow for access to at least one of the user's master account, the user's user preference, the user's profiles, any videomail messages addressed to the user, the user's media content, media content recommendations for the user, determined advertisements for the user, the user's preferences for advertisements, and/or the like.
  • the network e.g., network 115
  • control server 110 which can have any suitable hardware configuration (an example of which is described below with respect to Fig. 10), might be a computer that is capable of receiving user input via a user interface 120 and/or performing operations for controlling the user device(s) 105 (which in some cases might comprise inline camera(s), which in turn might comprise cameras or other sensors, and the like).
  • the control server 110 can provide modified apparent views to be inserted in a video stream, and/or the like.
  • the control server 110 can receive and/or store user input and/or user preferences that can specify whether and how presence information should be used.
  • the user might log onto his or her master account at the control server in order to access and/or control inline cameras assigned to that account.
  • the user device 105 and/or the control server 110 might authenticate the user with a set of credentials associated with the master account (e.g., with any of several know authentication schemes, such as a userid/password challenge, a certificate exchange process, and/or the like).
  • the user interface can present the user with a variety of different information, including without limitation information about status of inline cameras (or user devices 105 comprising the inline cameras) assigned to the master account to which the user has logged on, options for controlling such inline cameras, and or the like.
  • the user device 105 and/or the control server 110 can further control and/or configure the inline camera, based at least in part on the user preferences.
  • the user might have specified that the inline camera should not be used to collect presence information at all, in which case that feature might be turned off at the inline camera.
  • the user might have specified some limitations on the collection of presence information (such as about whom such information may be collected, times at which information can be collected, and/or purposes for which information may be collected, to name a few examples).
  • these preferences can be set directly at the inline camera, e.g., through a menu system displayed on a video device. It should also be recognized that some preferences (such as with whom presence information can be shared) might not affect the inline camera and might be saved and/or operated on at the control server instead.
  • control server 110 can vary according to embodiment and implementation.
  • the control server 110 might provide fairly finegrained control over the inline camera, such as instructing the camera to capture images for purposes of determining presence, and/or the control server 110 may receive the images directly and perform the present determination procedures at the controls server.
  • the division of responsibility between the control server 110 and the inline camera or user device 105 can fall anywhere along this spectrum.
  • control server 110 might provide the user preferences to the inline camera, which then is responsible for collecting presence information in accordance with those preferences and transmitting the presence information to the control server 110, which takes the appropriate action in response to the presence information, such as, selecting an advertisement based on the presence information.
  • the inline camera itself might be responsible for taking such actions.
  • the user device or inline camera might collect presence information.
  • a variety of operations might be involved in the collection of presence information.
  • the inline camera captures one or more images of at least a portion of a room where it is located. Such images can be digital still images, a digital video stream, and/or the like.
  • Collecting presence information can further comprise analyzing one or more of the images.
  • the images might be analyzed with facial recognition software, which can be used to determine the number of people in the room with the inline camera and/or to identify any of such people (e.g., by determining a name, an age range, a gender, and/or or other identifying or demographic information about a user, based on the output of the facial recognition software).
  • analyzing the images can comprise determining that a person is watching a display device, for example using eye-tracking software to identify a focus area of the person's eyes and correlating that focus area with the location of a television. In some cases, if the number of people and the identities (or at least demographics).
  • the user device (or inline camera) 105 might determine a position(s) of a user(s) relative to a display device in communication with the user device (or inline camera) 105.
  • the user device (or inline camera) 105 and/or the control server 110 might adjust an apparent view of video and/or image(s) on the display device in response to the determined position(s) of the user(s) relative to the display device.
  • the user device (or inline camera) 105 and/or the control server 110 might adjust audio (which might be associated with the video and/or image(s), or might be stand-alone audio), in response to the determined position(s) of the user(s) relative to the display device.
  • audio which might be associated with the video and/or image(s), or might be stand-alone audio
  • This technique allows for tracking movement of the user(s), and can, in some cases, provide real-time or near- real-time adjustment of video, image, and/or audio, in response to the determined updated position(s) of the user(s).
  • server 110 might perform the methods described in detail with respect to Figs. 2-9 below, while data associated with user account(s) or preferences, data associated with monitored user(s), and/or data associated with monitored media content might be collected by the one or more user devices 105, by server 110, or by any combination of these computing devices.
  • the database 130 might store some or all of these collected data.
  • the user devices 105 and/or the server 110 might perform any functions that are described in detail in any of the Related Applications and/or in the '182 Patent, which are already incorporated herein by reference in their entirety for all purposes.
  • FIG. 2 illustrates a scenario 200 in which a camera or ICD 205 captures a scene. That camera has a fixed field of view 210, which might define an angle 215 that is rotated about a 360 degree direction about an axis that is normal to the lens of the camera or ICD 205.
  • the fixed field of view 210 generally cannot be modified unless the settings or orientation of the camera are manually modified.
  • a scene viewed on a display 320 by a user's eye 305 will have an ideal field of view 310, which is a function of the user's position (in three dimensions) and time.
  • the ideal field of view 210 might define an angle 315 that is rotated about a 360 degree direction about an axis that is normal to the lens of the user's eye 305.
  • a camera or ICD 205 might be designed to have a field of view that defines an angle 215 that matches or exceeds angle 315.
  • the field of view 310 (and the corresponding perspective) must depend on the user's position at any given time, and must change if the user's position changes.
  • position when referring to a user, can either refer generally to a user's position or can refer more specifically to the position of the user's eyes, or a proxy thereof, such as the centroid of an ellipse that encompasses the user's eyes.
  • Figs. 4A-4F are general schematic diagrams illustrating techniques for adjusting an apparent field of view of a display device, in accordance with various embodiments.
  • the display side portion shows the side on which the user 405 is located and on which the display device 410 displays content (including, without limitation, images/video captured from the capture side, and/or the like) to the user 405.
  • the position of the user 405 (and/or the user's eyes) may be tracked by camera 415a.
  • the capture side portion shown in Fig.
  • FIG. 4B shows the side on which another party to a video call is located or the side on which a live video stream is captured (or the like).
  • the other party to the video call or the objects of the live video stream may be captured by camera 415b.
  • the capture side shows the maximum field of view ("FOV") 420 (shown as a pair of solid lines in Fig. 4B) that the camera 415b captures, as well as the various FOVs 425 and 430 that the camera 415b captures in various situations.
  • FOV maximum field of view
  • position Pi which is located a distance di from the face of the display device 410
  • position P2 which is located a distance d2 from the face of the display device 410
  • position Pi the viewer is close to the display device 410. This corresponds to a wider field of view 425 as shown (as a pair of dot-dash lines) in the capture side figure (fig. 4B).
  • position P 2 the viewer is further from the display device 410. This corresponds to a narrow field of view 430 as shown (as a pair of dash lines) in the capture side figure (Fig. 4B).
  • Fig. 4C depicts the effective FOVs of the user 405, when the user 405 is located at positions Pi and P 2 , for instance.
  • display device 410 one might treat display device 410 as if it were a virtual window looking into the capture side (in a sense, through the "peephole" of camera 415b).
  • the user's effective FOV 425' might ideally extend from the display side, beyond display device 410, to the capture side.
  • objects within FOV 420 should ideally be at least on plane 435 that is parallel to a face of the camera 415b (which, from a functional perspective, might have a position that is effectively (though not actually) behind display device 410) or extend outward from camera 415b beyond plane 435. In this manner, it may be ensured that objects within the FOV 420 may be captured in images/video.
  • any objects or portions of objects between camera 415b and plane 435 may not be fully captured (or indeed captured at all), thus resulting in a somewhat unnatural image/video that is displayed on the display device, which would not effective simulate a virtual window.
  • the user device or control server might use image processing techniques to remove such objects (or partial image- captured objects) from the resultant displayed video or image(s).
  • the user's effective FOV 430' might ideally extend from the display side, beyond display device 410, to the capture side.
  • objects within FOV 420 should ideally be at least on plane 440 that is parallel to a face of the camera 415b or extend outward from camera 415b beyond plane 440.
  • Figs. 4D-4F illustrate this process for horizontal movements of the user 405.
  • camera 415a might be used for determining the user's 405 position relative to (a face of) display device 410 (and can be used to transmit video or other media content to the user 405, as well, for example, as part of a video call or the like).
  • the horizontal position is relative to the display side camera 415a.
  • position Pi indicates a horizontal offset (by distance x) from the centerline (which defines a line that is normal to a face of the camera 415a or that is normal to the face of the display device 410).
  • the FOV 425 for this offset position is shown (as a pair of dot-dash lines) in the capture side figure (Fig. 4E).
  • Position P2 corresponds to one in which the user is not horizontally offset relative to the display side camera (i.e., is aligned with the centerline).
  • the FOV 430 for this non-offset position is shown (as a pair of dash lines) in the capture side figure (Fig. 4E). In both these examples, the user 405 remains at a constant distance y from the display device 410.
  • Fig. 4F depicts the effective FOVs of the user 405, when the user 405 is located at positions Pi and P2, for example.
  • display device 410 as in Fig. 4C, one might treat display device 410 as if it were a virtual window looking into the capture side (in a sense, through the "peephole" of camera 415b).
  • the user's effective FOV 425' might ideally extend from the display side, beyond display device 410, to the capture side, with the FOV 425' shifted to the left.
  • objects within FOV 420 should ideally be at least on plane 435 that is parallel to a face of the camera 415b or extend outward from camera 415b beyond plane 435. In this manner, it may be ensured that objects within the FOV 420 may be captured in images/video. Any objects or portions of objects between camera 415b and plane 435 may not be fully captured (or indeed captured at all), thus resulting in a somewhat unnatural image/video that is displayed on the display device, which would not effective simulate a virtual window.
  • a number of techniques can be used to detect the position of the user (or, as noted above, more precisely, the user's eyes), along any combination of three dimensions.
  • location of the viewer's eyes on the display side can be detected (or estimated) by one or more of techniques including, but not necessarily limited to, (a) distance sensors (including, without limitation, lidar sensors, radar sensors, sonar sensors, and/or the like); (b) facial recognition techniques; (c) point locating device (e.g., remote control, headset, glasses, and/or similar devices), (d) silhouette detection, (e) eye tracking techniques; and/or (f) other techniques.
  • the analysis techniques to determine the user's position can be performed by a video calling device (or other user device) that captures the video of the user, by a control server, by a video calling device (or other user device) that is used to record the video to be displayed to the user, or by a combination of these devices.
  • Figs. 5A and 5B are general schematic diagrams illustrating techniques for adjusting apparent fields of view of a display device for multiple users, in accordance with various embodiments.
  • display side portion shows the side on which the users 505a and 505b (collectively, "users 505") are located and on which the display device 510 displays content (including, without limitation, images/video captured from the capture side, and/or the like) to the users 505.
  • the position of the users 505 (and/or the users' eyes) may be tracked by camera 515a.
  • the capture side portion shown in Fig.
  • FIG. 5B shows the side on which another party to a video call is located or the side on which a live video stream is captured (or the like).
  • the other party to the video call or the objects of the live video stream may be captured by camera 515b.
  • the capture side shows the maximum field of view ("FOV") 520 (shown as a pair of solid lines in Fig. 5B) that the camera 515b captures, as well as the various FOVs 525 and 530 that the camera 515b captures in various situations for each of the users 505a and 505b.
  • FOV maximum field of view
  • camera 515a might be used for determining the first user's 505a position relative to (a face of) display device 510 (and can be used to transmit video or other media content to the first user 505a, as well, for example, as part of a video call or the like).
  • the horizontal position is relative to the display side camera 515a.
  • position Pi indicates a horizontal offset (by distance x) from the centerline (which defines a line that is normal to a face of the camera 515a or that is normal to the face of the display device 510).
  • the FOV 525 for this offset position is shown (as a pair of dot-dash lines) in the capture side figure (Fig.
  • camera 515a might be used for determining the second user's 505b position relative to (a face of) display device 510 (and can be used to transmit video or other media content to the second user 505b, as well, for example, as part of a video call or the like).
  • position P 2 is shown aligned with the centerline.
  • the FOV 530 for this offset position is shown (as a pair of dash lines) in the capture side figure (Fig. 5B).
  • the users 505 remain at a constant distance y from the display device 510 (although the various embodiments are not so limited, and the users 505 may be positioned one closer to the display device 510 compared to the other).
  • various techniques may be used, including, but not limited to, techniques such as the use of active glasses that, based at least in part on time synchronization with the display device 510, can allow one pair of active glasses (worn by one user) to receive one FOV, while the other pair of active glasses (worn by the other user) blocks that particular FOV, and vice versa, such that the eyes of each user only receives images/video corresponding to one set of FOV and not the other.
  • Such a technique of using the active glasses to alternate between frames of displayed content to display different FOVs is described in detail below with respect to Fig. 8.
  • a number of techniques can be used to adjust a field of view (“FOV”) to correspond to the viewer's position.
  • One technique is the creation of a windowed field of view, as depicted by Fig. 6, which is a general schematic diagram illustrating a windowed field of view in relation to a sensor field of view, in accordance with various embodiments.
  • a sensor field of view (“FOV”) 605 is shown in relation to a windowed FOV 610.
  • the sensor FOV 605 represents the FOV that is achieved by a sensor at the capture side
  • the windowed FOV 610 represents the FOV that is displayed on a display device at the display side.
  • the video stream that is captured can be the entire FOV (referred to, in some embodiments herein, as “maximum field of view”), or can be a subset that is smaller and can be positioned arbitrarily (or to correspond to the viewer's position) within the full sensor field of view. This is denoted “windowed FOV” in Fig. 6. If the full FOV is captured, the video can be cropped to produce the desired windowed FOV.
  • one approach is to adjust the windowed FOV 610 on the capture side camera to something other than the full FOV and in a manner that corresponds to the position of the viewer's eyes on the display side.
  • One way to do this is to send the coordinates of the viewer's eyes to the capture side. This could be done in a peer-to- peer fashion and/or might be facilitated via a server.
  • peer-to-peer sessions might be initiated using a server, and after a peer-to-peer session has been initiated or established by the server, the server may be by-passed, resulting in a direct peer-to-peer connection or session.
  • the windowed FOV 610 (which in this case represents the camera's or sensor's FOV) can be adjusted accordingly, and the image that is seen on the display side would adjust based on the position of the viewer's eyes.
  • An alternative approach would be to have the capture side always send the full FOV 605 to the display side.
  • the video communications device on the display side would manipulate the video stream to display a windowed version that is a subset of the full FOV that corresponds to the position of the viewer's eyes.
  • the advantage of this approach is that no additional network communication is required, and the latency between any view movements and the image adjustment on the display side would be reduced.
  • the windowed FOV 610 is moved left when the user moves (and/or the user's eyes move) right, and/or is moved down when the user moves (and/or the user's eyes move) up.
  • the windowed FOV 610 is moved right when the user moves (and/or the user's eyes move) left, and/or is moved up when the user moves (and/or the user's eyes move) down.
  • the user (and/or the user's eyes) moving in any combination of left, right, up, and/or down relative to the display device will result in the windowed FOV 610 being moved in the corresponding combination of right, left, down, and/or up, respectively.
  • Yet another approach is to have a camera on the capture side that has a physical mechanism for the adjustment of the field of view (i.e., pan, tilt, and zoom, etc.). If the camera has such capability, then when the viewer's eyes' coordinates are sent across the network to the capture side, the camera's position can physically be adjusted (by any suitable combination of panning, tilting, zooming, and/or the like) to produce an image that is appropriate for the viewer's eyes.
  • the capture side device might feature an array of cameras (as shown, e.g., in Fig. 7B), which can expand the field of view that can be captured.
  • the images from one or more cameras can be combined and processed to produce a larger field of view than a single camera alone (as shown, e.g., in Fig. 7 A).
  • Camera arrays can be used to form a composite image using the images from one or more camera.
  • This composite image can have a virtual perspective that is different than any of the individual cameras.
  • the virtual perspective can be set to create a perspective based on the location of the viewer. For example, the perspective can be with respect to the viewer and his or her display.
  • Figs. 7A and 7B are general schematic diagrams illustrating a display device 700 in use with one or more image capture devices, in accordance with various embodiments.
  • display device 700 might comprise housing 705, display screen 705a, displayed or windowed FOV 710, image-captured object(s) 715 (which in the embodiments shown in Fig. 7 might include a call participant in a video call, or the like).
  • image-captured object(s) 715 which in the embodiments shown in Fig. 7 might include a call participant in a video call, or the like.
  • ICDs image capture devices
  • Fig. 7A a single ICD or camera 720 is shown, while, in Fig.
  • a plurality of ICDs or cameras 720 are shown (although five ICDs or cameras 720a-720e are shown, this is merely for illustration, and any suitable number of ICDs or cameras 720 may be used).
  • multiple ICDs or cameras 720 (which may be arranged in an array(s)) can be used to form a composite image using the images captured by the plurality of ICDs or cameras 720.
  • the composite image may represent one frame in a series of frames of a video (such as in a video call, movie content, television content, live video stream, etc.).
  • a plane 725 that is parallel to a plane defined by (the screen 705a or face of) the display device 700.
  • Axes x and z represent the horizontal and vertical axes, respectively.
  • determining a position of a first user might comprise determining a horizontal position of the first user in a horizontal dimension (e.g., along the x-axis) of the plane 725, which is parallel to the face of the display device.
  • adjusting an apparent or windowed FOV might comprise panning the video in a horizontal direction (i.e., along the x-axis) or moving the windowed FOV in the horizontal direction, based on the determined horizontal position of the first user.
  • determining a position of the first user might comprise determining a horizontal position of the first user in a vertical dimension (e.g., along the z-axis) of the plane 725, which is parallel to the face of the display device.
  • adjusting an apparent or windowed FOV might comprise panning the video in a vertical direction (i.e., along the z-axis; sometimes referred to as "tilting") or moving the windowed FOV in the vertical direction, based on the determined vertical position of the first user.
  • tiltting the windowed FOV in the vertical direction
  • FIG. 8 illustrates a functional diagram of a system 800 for modifying an apparent view(s) of displayed content, based at least in part on sensed presence and/or determined position(s) of a user in a room, in accordance with one set of embodiments.
  • the skilled reader should note that the arrangement of the components illustrated in Fig. 8 is functional in nature, and that various embodiments can employ a variety of different structural architectures.
  • an ICD 805 might correspond to ICD 105, video calling device 105, and/or PDD 105
  • user device 845 might correspond to non-ICD user device 105, non- video calling device user device 105, or non-PDD user device 105, as described in detail above with respect to Fig. 1.
  • Control server 810, network 815, and cloud storage system 830, in the example of Fig. 8, might correspond to control server 110, network 115, and cloud storage system 130, respectively, as described in detail above with respect to Fig. 1.
  • System 800 might further comprise a local content source 835 (e.g., a local content source as described above), a display device 840 (including, without limitation, a television ("TV"), a computer monitor, and/or the like), and high- definition (“HD") data cables 850 (or any other suitable data transmission media).
  • the HD data cables 850 might include, without limitation, high-definition multimedia interface (“HDMI”) cables.
  • One or more of the ICDs 805 e.g., the first ICD 805a and the second ICD 805b, as shown in Fig. 8) might be configured to provide pass-through audio and/or video from a local content source 835 to a display device 840 (e.g., using data cables 850).
  • an HDMI input port in the ICD 805 allows HD signals to be input from the corresponding local content source 835
  • an HDMI output port in the ICD 805 allows HD signals to be output from the PDD 805 to the corresponding display device 840 (e.g., TV, which might include, but is not limited to, an Internet Protocol TV ("IPTV"), an HDTV, a cable TV, or the like).
  • the output HD signal may, in some cases, be the input HD signal modified by the ICD 805.
  • Local content source 835 might be any suitable local content source.
  • a local content source can be any device that provides an audio or video stream to a display device and thus can include, without limitation, a cable or satellite STB, an IPTV STB, devices that generate video and/or audio, and/or acquire video and/or audio from other sources, such as the Internet, and provide that video/audio to a display device; hence a local content source can include devices such as a video game console, a Roku® streaming media player, an AppleTV®, and/or the like.
  • the ICD 805 when situated functionally inline between a local content source and a display device, the ICD 805 can receive an audiovisual stream output from the local content source, modify that audiovisual stream in accordance with the methods described in the '182 Patent, and provide the (perhaps modified) audiovisual stream as input to the display device 840.
  • first ICD 805a, local content source 835a, display device 840a, and user device 845a (if any) might be located at a first customer premises 860a
  • second ICD 805b, local content source 835b, display device 840b, and user device 845b if any
  • a user device 845 might be located at a customer premises 860 or might be a portable user device (including, without limitation, a tablet computer, a laptop computer, a smart phone, a mobile phone, a portable gaming device, and/or the like) that is not bound to any particular customer premises 860, and the like.
  • a portable user device including, without limitation, a tablet computer, a laptop computer, a smart phone, a mobile phone, a portable gaming device, and/or the like
  • system 800 might further comprise one or more access points (not shown), each of which might be located in proximity to or in the first customer premises 860a or the second customer premises 860b.
  • the access point(s) can allow wireless communication between each ICD 805 and network 815.
  • an ICD 805 might also have a wired connection to an access point, router, residential gateway, etc., such as via an Ethernet cable, which can provide similar communication functionality.
  • each ICD 805 might be communicatively coupled to network 815 (via either wired or wireless connection), without routing through any access points.
  • wired or wireless access to network 815 allows ICD 805 to obtain profiles from cloud storage system 830, media content from first content server 870 and/or database 875 that are independent of the corresponding local content source 835, which is in
  • content distribution network 865 (either via wireless connection or via wired connection).
  • content distribution network 865 (which could be, for example, a cable television distribution network, a satellite television distribution network, an Internet Protocol television (“IPTV”) distribution network, and/or the like) might be communicatively coupled with second content server 880, and thus local content source 835 might obtain media content from second content server 880 and media content database 885 independently of ICD 805.
  • IPTV Internet Protocol television
  • the content distribution network 865 might be communicatively coupled to other content servers (e.g., first content server 870 or the like) and/or other media content sources (e.g., database 875 or the like).
  • content servers e.g., first content server 870 or the like
  • media content sources e.g., database 875 or the like.
  • ICD 805 can overlay the input signal from the corresponding local content source 835 with additional media content to produce an augmented output HD signal to the corresponding display device 840 via data cables 850.
  • This functionality allows for supplemental content (which may be associated with the media content accessed by the local content source 835 for display on display device 840) to be accessed and presented using the first ICD 805, in some cases, as a combined presentation on the display device 840, which may be one of an overlay arrangement (e.g., a picture-in-picture ("PIP") display, with the supplemental content overlaid on the main content), a split screen arrangement (with the supplemental content adjacent to, but not obscuring, any portion of the main content), a passive banner stream (with non-interactive supplemental content streaming in a banner(s) along one or more of a top, bottom, left, or right edge of a display field in which the main content is displayed on display device 840), and/or an interactive banner stream (with interactive supplemental content streaming in a
  • an overlay arrangement
  • examples of interactive supplemental content might include, without limitation, content that when streamed in a banner can be caused to slow, stop, and/or replay within the banner, in response to user interaction with the content and/or the banner (as opposed to passive banner streaming, in which information is streamed in a manner uncontrollable by the user).
  • the interactive supplemental content that is streamed in the banner may, in some instances, also allow the user to invoke operations or functions by interacting therewith; for example, by the user highlighting and/or selecting the supplemental content (e.g., an icon or still photograph of a character, actor/actress, scene, etc.
  • the interactive supplemental content might include notifications or messages relating to recommendations of media content, the determination and generation of which are described in detail above.
  • the interactive supplemental content (whether related or unrelated to the media content being presented) might include advertisement content.
  • ICD 805 might detect the presence and/or proximity of one or more user devices 845 associated with the user, and might (based on user profile information associated with the user that is stored, e.g., in cloud storage system 830) automatically send supplemental media content via wireless link 855 (directly from ICD 805 or indirectly via an access point (not shown)) for display on a display screen(s) of the one or more user devices 845.
  • a user associated with first ICD 805a might have established a user profile stored in cloud storage system 830 that indicates a user preference for any and all supplemental content for movies and television programs to be compiled and displayed on one or more user devices 845a (including, but not limited to, a tablet computer, a smart phone, a laptop computer, and/or a desktop computer, etc.) concurrent to display of the movie or television program being displayed on display device 840a.
  • user devices 845a including, but not limited to, a tablet computer, a smart phone, a laptop computer, and/or a desktop computer, etc.
  • first ICD 805a accesses supplemental content (if available) from content server 870 and media content database 875 via network 815, and sends the supplemental content to the user's tablet computer and/or smart phone via wireless link(s) 855.
  • bios of actors, actresses, and/or crew might be sent to the user's smart phone for display on the screen thereof, while schematics of machines, weapons, robots, tools, etc.
  • associated with the movie or television show might be sent to and displayed on the user's tablet computer, behind the scenes videos or information, news/reviews associated with the main content, and/or music videos associated with the main content may also be sent to the user's smart phone and/or tablet computer, and so on.
  • first media content might be received by local content source 835a (in customer premises 860a) from media content database 875b via content server 870 and content distribution network 865.
  • the first ICD 805a might provide pass through capability for displaying video aspects (in some cases audio aspects as well) of the first media content from the local content source 835a.
  • the first ICD 805a might monitor the media content, and might generate or select advertisements based at least in part on the monitored media content.
  • the first ICD 805a might comprise sensors (e.g., camera, microphone, proximity sensors, user device sensors, communications links, etc.) that monitor the user(s) within the same room, e.g., to monitor or track reactions of each user (including, but not limited to, vocal expressions or outbursts, facial expressions, hand gestures, body gestures, eye movement, eye focus, shift in proximity with respect to the PDD, and/or the like), using any number or combination of techniques, including, without limitation, facial recognition techniques, facial expression recognition techniques, mood recognition techniques, emotion recognition techniques, voice recognition techniques, vocal tone recognition techniques, speech recognition techniques, eye movement tracking techniques, eye focus determination techniques, proximity detection techniques, and/or the like.
  • the first ICD 805a might determine advertisements based at least in part on the monitored reactions of each user.
  • the first ICD 805a might send the information associated with the monitored media content and/or information associated with the monitored reactions of each user to control server 810 over network 815, and control server 810 might determine or generate recommendations for media content, based at least in part on the monitored media content and/or based at least in part on the monitored reactions of each user, which is described in detail (along with other embodiments of media content recommendation, or the like) in the '435 Application (already incorporated herein by reference in its entirety).
  • control server 810 might determine (i.e., select and/or generate) advertisements based at least in part on the monitored media content and/or based at least in part on the monitored reactions of each user, which is described in detail (along with other embodiments of advertisement determination, or the like) in the '133 and
  • the detection of the presence of the user device 845 by the first ICD 805a or the second ICD 805b might allow identification of a user and thus access of profiles, content, and/or messages and notifications associated with the user's account, regardless of whether the first ICD 805 a or the second ICD 805b is owned by and/or associated with the user.
  • the user' s media content might include, without limitation, at least one of purchased video content, purchased audio content, purchased video game, purchased image content, rented video content, rented audio content, rented video game, rented image content, user-generated video content, user-generated audio content, user-generated video game content, user generated image content, and/or free media content
  • the user's profiles might include, but is not limited to, one or more of user profile information for a video game or video game console, web browser history and/or bookmarks, contact information for the user's contacts, user profile information for video or audio content, including without limitation recommended content, device preferences, messaging preferences, videomail preferences, user profile information for cloud services, and/or the like.
  • Videomail might refer to videomail messages addressed to the user or callee.
  • the user's profile might also include identifying information - including, but not limited to, the user's biometric information (e.g., facial characteristics, voice characteristics, fingerprint), etc.
  • biometric information e.g., facial characteristics, voice characteristics, fingerprint
  • iris characteristics iris characteristics, pupil characteristics, retinal characteristics, etc.
  • user's past monitored reactions e.g., vocal expressions or outbursts, facial expressions, hand gestures, body gestures, eye movement, eye focus, shift in proximity with respect to the PDD, and/or the like, or the like.
  • the user profile information for cloud services might include user log-in information (e.g., username, account number, and/or password/passphrase, etc.) or other suitable credentials for cloud services, which might include, without limitation, video calling service, videomail service, voice calling service, video broadcast/streaming service, audio broadcast/streaming service, on-line gaming service, banking/financial services, travel/accommodation/rental vehicle services, and/or dining/entertainment event reservation/ticketing services, or the like.
  • user log-in information e.g., username, account number, and/or password/passphrase, etc.
  • suitable credentials for cloud services might include, without limitation, video calling service, videomail service, voice calling service, video broadcast/streaming service, audio broadcast/streaming service, on-line gaming service, banking/financial services, travel/accommodation/rental vehicle services, and/or dining/entertainment event reservation/ticketing services, or the like.
  • a user might be associated with first ICD 805a (located in the first customer premises 860a), while her friend might be associated with second ICD 805b (located in the second customer premises 860b), and the user and the friend are both subscribers of a similar service provided by control server 810 and/or the cloud service provider associated with control server 810.
  • the friend's ICD 805b might first detect presence of the user, by querying and/or obtaining the identification information for the user's smart phone and/or tablet computer or the like, by capturing video, image, and/or voice data of the user, by infrared detection of a living person in the room, and/or by audio detection of a living person in the room, etc.
  • the friend's ICD 805b might then identify the user using the user's device(s) identification information and/or the captured video, image, and/or voice data, or might send such presence information to control server 810 for identification and authentication analysis.
  • detecting presence of, or identifying/authenticating the user might include, without limitation, analyzing captured images or video segments using one or more of facial recognition software, pupil/iris recognition software, retinal identification software, fingerprint analysis software, and/or physiology recognition software, analyzing captured audio samples using one or more of voiceprint analysis and/or comparison with stored
  • any suitable technique may be implemented including, but not limited to, at least one of detecting a Bluetooth connection of the user device, detecting that the user device is associated with a WiFi access point with which the video calling device has associated, and/or
  • NFC near field communication
  • control server 810 might send copies of the user's profiles and/or content to the second ICD 805b (either from first ICD 805a and/or from cloud storage system 830, or the like), or at least provide the user with access to her profiles, notifications of media content recommendations, notification of determined advertisements, preferences for advertisements, videomail, and/or content from her friend's ICD 805b.
  • the identification and authentication processes might include comparing the user device identification information and/or the captured video, image, and/or voice data against all similar identification data for all users/subscribers of the cloud service that are stored in cloud storage system 830.
  • the process might be facilitated where ICDs 805a and 805b might already be associated with each other (e.g., where the user has previously made a video call from first ICD 805a to her friend on second ICD 805b, where the user might have added the friend to the user's contact list, and/or where the friend might have added the user to the friend's contact list).
  • the user's first ICD 805a might have access to the user's calendar and/or communications, which might indicate that the user is visiting the friend.
  • the first ICD 805a might query control server 810 to determine whether the friend has an ICD 805b associated with the cloud service provider.
  • first ICD 805a determines that second ICD 805b is part of the same service and/or is in communication with control server 810, and based on such determination, first ICD 805a (and/or control server 810) might send the user's profiles and/or content to second ICD 805b, and/or provide second ICD 805b with access to the user's profiles, notifications of media content recommendations, notifications of determined advertisements, preferences for advertisements, videomail, and/or content.
  • the user's profiles, notifications of media content recommendations, notifications of determined advertisements, preferences for advertisements, videomail, and/or content, or access to profiles, notifications of media content recommendations, notifications of determined advertisements, preferences for advertisements, videomail, and/or content might be encrypted, and might be released/decrypted upon identification and/or authentication by second ICD 805b (and/or by control server 810) when the user is detected by second ICD 805b.
  • the user's profiles, notifications of media content recommendations, notifications of determined advertisements, preferences for advertisements, videomail, and/or content can follow the user wherever she goes, so long as there is a device (e.g., PDD or video calling device) that is associated with the same or affiliate cloud service provider at her destination, and so long as the device can recognize and authenticate the user.
  • a device e.g., PDD or video calling device
  • second ICD 805b (and/or control server 810) might determine that the user is no longer present at the location of second ICD 805b.
  • second ICD 805b and/or control server 810 might remove the user's profiles, notifications of media content recommendations, notifications of determined advertisements, preferences for advertisements, videomail, and/or media content (or access thereto) from second ICD 805b.
  • a time-out system might be utilized.
  • other suitable systems may be used for determining the user is no longer present, and removing the user's profiles, notifications of media content recommendations, notifications of determined advertisements, preferences for advertisements, videomail, and/or media content (or access thereto) from the second ICD 805b.
  • the system might either stop presenting the advertisement(s) (if currently being presented) or not present the advertisement(s) (if not yet presented).
  • system 800 might provide virtual window functionality.
  • system 800 might modify an apparent view(s) of displayed content, based at least in part on sensed presence and/or determined position(s) of a user in a room.
  • first ICD 805a might determine or collect presence and/or position information about a user with respect to the display device 840a.
  • first ICD 805a and/or control server 810 might modify an apparent view of the media content (either from first content server 870 and database 875 via network 815 or from second content server 880 and database 885 via local content source 835a and network 865, or the like) that is displayed on display device 840a, based at least in part on the position information of the user, similar to the techniques as described above with respect to Figs. 1-7.
  • the first ICD 805a might determine and/or collect the changed position of the user relative to the display device 840a, and the first ICD 805a and/or the control server 810 might modify the apparent view of the media content displayed on display device 840a by increasing the apparent field of view of the media content displayed.
  • the first ICD 805a might determine and/or collect the changed position of the user relative to the display device 840a, and the first ICD 805a and/or the control server 810 might modify the apparent view of the media content displayed on display device 840a by decreasing the apparent field of view of the media content displayed.
  • the first ICD 805a might determine and/or collect the changed position of the user relative to the display device 840a, and the first ICD 805 a and/or the control server 810 might modify the apparent view of the media content displayed on display device 840a by proportionally changing the apparent field of view of the media content displayed toward the right (in some cases, by proportionally changing an apparent perspective of the media content toward the right; herein, changing an apparent perspective of the media content might include changing the apparent field of view such that the apparent view of the media content is panned or tilted with respect to a previous apparent view of the media content, or otherwise modifying the apparent so that the image/video displayed appears to have been captured from a different angle).
  • the first ICD 805a might determine and/or collect the changed position of the user relative to the display device 840a, and the first ICD 805a and/or the control server 810 might modify the apparent view of the media content displayed on display device 840a by proportionally changing the apparent field of view of the media content displayed toward the left (in some cases, by proportionally changing an apparent perspective of the media content toward the left).
  • the user may move in any of
  • the ICD 805a can track such movements, and the ICD 805a and/or the control server 810 can modify the apparent view of the displayed media content accordingly (despite the combination of the movements), such that the resultant apparent fields of view track the movements of the user, to provide a more natural display, not unlike looking out a physical window while changing one's position relative to the window (hence, in some cases, the display device that displays modified or adjusted apparent views of content according to this technique might be referred to as a "virtual window").
  • the modification of the apparent view of the displayed media content might be performed in real-time or near real-time (i.e., with minimal, almost imperceptible lag).
  • the ICD 805 associated with the particular call participant might determine and/or collect presence information about the corresponding call participant, and the particular ICD and/or control server 810 might modify the apparent view of the corresponding video feed of the other call participant accordingly.
  • a caller at the first customer premises 860a might initiate, using first ICD 805a, a video call with a callee at the second customer premises 860b.
  • first ICD 805a might display video feeds of the callee on display device 840a
  • second ICD 805b might display video feeds of the caller on display device 840b.
  • the caller might shift position with respect to display device 840a (say, for example, moving a bit closer and to the left with respect to the display device 840a).
  • First ICD 805a might track this movement, and first ICD 805a and/or control server 810 might modify the apparent view of the callee displayed on display device 840a in one of several ways.
  • modifying the apparent view might include, but is not limited to, sending instructions to second ICD 805b to perform at least one of panning to the right, zooming in on the callee, and/or increasing the apparent field of view.
  • second ICD 805b might normally send a maximum field of view to the first ICD 805a and/or control server 810, which might normally reduce the apparent field of view prior to displaying the video feed on display device 840a.
  • modifying the apparent view might include, without limitation, changing the apparent field of view by taking the maximum field of view that is sent from second ICD 805b, by simulating the at least one of panning to the right, zooming in on the callee, and/or increasing the apparent field of view.
  • second ICD 805b might track the movement, and second ICD 805b and/or control server 810 might modify the apparent view of the caller displayed on display device 840b in a similar manner as described above with respect to the modification of the apparent view of the callee displayed on display device 840a.
  • multiple cameras or multiple image capture devices may be used, and a composite image/video with composite field of view (both maximum and displayed) may be generated (either by ICD 805 and/or by control server 810).
  • modification of the apparent view may be performed by modifying the composite image/video and/or modifying the composite field of view, or the like.
  • the composite image/video and/or the composite field of view In order for the composite image/video and/or the composite field of view to appear to be a single coherent image/video and/or composite field of view from a single image capture device, some image processing of the image or frames of the video might be necessary to ensure that stitching of the different images/frames of video is seamless. This is especially important for three-dimensional ("3-D”) images/video having been collected or captured by different image capture devices (and thus have different fields of view).
  • 3-D three-dimensional
  • each user viewing a display device 840 might wear glasses, not unlike active 3-D glasses.
  • the glasses might each be in wireless communication (e.g., infrared communication, Bluetooth communication, WiFi communication, and/or the like) with the ICD 805, and the timing of each device may be synchronized by the ICD 805.
  • a first viewer might wear a first pair of active glasses, while a second viewer might wear a second pair of active glasses, and a third viewer might wear a third pair of glasses.
  • the ICD 805 might send a first frame of video to be displayed on the display device 840, and while the first frame of video is displayed, the first pair of active glasses might be set to not block (i.e., to allow) light that is received from the frame, but each of the second and third pairs of active glasses might be set to block the light received from the frame.
  • the ICD 805 might then send a second frame of video to be displayed on the display device 840, and while the second frame of video is displayed, the second pair of active glasses might be set to not block (i.e., to allow) light that is received from the frame, but each of the first and third pairs of active glasses might be set to block the light received from the frame.
  • the ICD 805 might send a third frame of video to be displayed on the display device 840, and while the third frame of video is displayed, the third pair of active glasses might be set to not block (i.e., to allow) light that is received from the frame, but each of the first and second pairs of active glasses might be set to block the light received from the frame.
  • the fourth frame of video might be treated in the same manner as the first frame, while the fifth frame might be treated in the same manner as the second frame, and the sixth frame might be treated in the same manner as the third frame, and so on.
  • Each of the frames of video might be modified in a manner similar to the above that takes into account the relative positions of each of the first through third viewers relative to display device 840.
  • the displayed and perceived images through the display device 840 and through the first pair of glasses closely reflect an apparent field of view as if the first viewer was looking through a real window (or in this case, a virtual window) despite moving relative to the window.
  • the second and third viewers might perceive similar effects from their respectively positions relative to the display device 840.
  • n users any suitable number of viewers may be used (say, n users).
  • the first user might receive through the first active glasses first, ( ⁇ + ⁇ ) ⁇ , etc. frames of the video, while the n th user might receive through the n active glass n , 2n , etc. frames of the video.
  • the ICD 805 may also adjust the frame rate to ensure seamless display of the video.
  • 24 frames per second or 24 Hz
  • 60i or interlaced, which is effectively about 30 frames per second
  • the ICD 805 might adjust the overall frame rate to be higher, in order to account for the n viewers, such that each viewer receives an effective frame rate that is one of the same as, half of, a third of, a quarter of, or a fifth of one of these frame rates, or the like.
  • one side might have a single user, while the other side might have multiple users.
  • the single -user side might function in a manner similar to that as described above for single users, while the multiple-user side might function in a manner similar to that as described above for multiple users.
  • both sides of the video calls might have multiple, but different numbers of users (for example, one side might have n users, while the other has m users, or the like).
  • the ICD 805 might determine whether a multiple-user situation exists, by determining presence of more than one user, and in some instances determining with eye tracking techniques how many users are actually viewing the display device 840. The ICD 805 then appropriately signals active glasses of the users to appropriately delivery the appropriate frames of the video to each user to allow for individualized perceptions of the virtual window, as described in detail above.
  • each display device might be split into two panels, each showing one of the other 2 parties.
  • the apparent view of each panel might be modified accordingly. For instance, if the panels are arranged side by side, the center of each panel would be off-center with respect to the display device, and the ICD 805 and/or the control server 810 might modify the field of view of the left panel as if the viewer was shifted to the right, and might modify the field of view of the right panel as if the viewer was shifted to the left.
  • the ICD 805 and/or the control server 810 might determine the relational positions of the viewer's eyes with respect to the centers of each of the panels, and might modify the apparent views displayed in the panels accordingly.
  • the example above only discusses a 3 -party call, any number of parties may be on the video call (and any number of participants may be present at each party's location).
  • these examples are directed to adjacent and aligned panels, the various embodiments are not so limited, and the panels may be arranged in any relative position on the display screen with respect to each other. In some cases, one panel might be made smaller than another panel, or the like.
  • 3-D video content may similarly be displayed to a single viewer or to multiple viewers.
  • half of the frames might be directed to the left eye of the user, while the other half of the frames might be directed to the right eye of the user, in alternating fashion.
  • two frames would be permitted to pass through each viewer's pair of active glasses (one to only the left eye of the viewer and the other to only the right eye of the viewer).
  • the left eye view and the right eye view would be appropriately generated and/or modified such that the combined frames by the two eye views provide the desired depth information to form 3-D views.
  • Fig. 9 is a process flow diagram illustrating a method 900 of providing a virtual window or for modifying an apparent view(s) of displayed content, based at least in part on sensed presence and/or determined position(s) of a user in a room, in accordance with various embodiments. While the techniques and procedures of Fig. 9 are depicted and/or described in a certain order for purposes of illustration, it should be appreciated that certain procedures may be reordered and/or omitted within the scope of various embodiments. Moreover, while the method illustrated by Fig. 9 can be implemented by (and, in some cases, are described below with respect to) the systems 100, 1000, and/or 1100 of Figs.
  • the method 900 might comprise, at block 905, capturing (e.g., with a video calling device or other user device) an image or video of a user(s), who might be a first party to a video call or who might simply be the viewer of a video stream (or still image), such as a television program, video game, live stream of a remote scene, and/or the like. If the user is a party to a video call, this captured video can be transmitted to another video calling device in a remote location used by another party to the video call (block 910), as described in the '182 Patent, for example.
  • the method 900 can further comprise identifying one or more features within the captured image/video (block 915).
  • the method 900 might include processing video with facial recognition software, silhouette detection software, eye-tracking software, and/or the like.
  • the method 900 can include determining a position of the user(s) with respect to a display device (or speakers, or any other defined point).
  • the spatial relationship between the user device (or other camera) used to capture the image/video and the display device might be known (such as, for example, if both the camera and the display are integrated into a single device, or if the user device is designed to be placed on top of the display device).
  • the user might specify the relative positions of these devices (e.g., in a guided setup operation and/or by configuring user preferences on the user device).
  • the user device (or other camera) used to capture the image/video and/or the display device might communicate with each other or with a server computer over a local or wider network to determine relative positions (either by exchange location information, if each device has such capability, and/or by using triangulation techniques or similar techniques, or the like).
  • the location of the user device can be used as a proxy for the location of the display device itself. Hence, the user's position with respect to the user device can be used to derive or estimate the user's position with respect to the display device.
  • a known object e.g., an object packaged with one or more of the image capture device, user device, display device, video calling device, and/or the like
  • a known object might be placed within the field of view of the image capture device; because the dimensions of the object are already known, determination of the relative size of the image captured object can be used to determine distance relative to the object, and the object can be used as a point of reference for determining distance and/or position of the user(s).
  • the known object might be a wearable object (such as a pin, brooch, button, etc. that might be affixed to clothing of the user).
  • the known object need not be on the user, much less very close to the user; image analysis (e.g., lighting analysis, shadow analysis, and/or the like) might be used to determine relative positions between the user and the known object. In some cases, any object may be calibrated to serve as such a known object and point of reference. According to some embodiments, sonar, lidar, or other similar techniques might be used to determine distances and/or relative positions of the user(s), with respect to the image capture device and/or the display device.
  • image analysis e.g., lighting analysis, shadow analysis, and/or the like
  • any object may be calibrated to serve as such a known object and point of reference.
  • sonar, lidar, or other similar techniques might be used to determine distances and/or relative positions of the user(s), with respect to the image capture device and/or the display device.
  • the position of the user in three dimensions can be used to adjust the apparent view of the displayed video.
  • Two of the dimensions can be considered the horizontal and vertical dimensions in a plane parallel to the display device (and/or a plane normal to the visual axis from the user's position to the focal point of the camera on the user device.
  • Fig. 7A shows a plane 725 that is parallel to the display device, and the axes x and z represent the horizontal and vertical dimensions, respectively.
  • the third dimension i.e., dimension y, as shown, e.g., in Figs.
  • 4D, 4F, and 5A is the distance of the axis from the user to the focal point of the camera.
  • the identified features in the captured video/image of the user can be used to identify a position in both dimensions.
  • the third dimension e.g., y dimension
  • any of a number of distance estimation techniques can be used, including, without limitation, laser rangefinding, parallax focusing, and/or the like.
  • the method 900 can comprise adjusting the apparent view of the displayed video (e.g., a video call, video game, media content, etc.), based on the determined position of the viewing user (block 925). Adjusting the apparent view of the video can comprise one or more of several operations. Merely by way of example, in some cases, adjusting the apparent view can comprise adjusting the apparent FOV, that is, the field of view that the user perceives when viewing the video, to correspond to the user's position(s) relative the display device (block 930). This adjustment can be performed by creating a windowed FOV (as noted above with respect to Fig.
  • the lens 6 can include panning, tilting (or vertical panning), and/or zooming a real or virtual camera capturing the video (for example, in a live stream or video call context), and/or it can include adjusting a raw video stream to provide the appropriate apparent field of view.
  • adjusting an apparent view can comprise adjusting an apparent perspective of the displayed video, i.e., the perspective that the user perceives when viewing the display, to correspond to the user's position relative to the display device (block 935).
  • This operation can also be accomplished in a number of ways. For example, in a three-dimensional ("3-D") video feed, the 3-D aspects of the video stream can be manipulated to provide an appropriate perspective.
  • adjusting the perspective might include moving a real or virtual camera (either by pan/tilt or through translation of the camera) to capture a displayed scene that corresponds to the user's position relative to the display device.
  • the capturing device comprises an array of two or more cameras
  • the device might create a composite FOV that is a mosaic of the fields of view of a plurality of those cameras.
  • the selection of cameras that are used to create the composite FOV can be changed to adjust the perspective given to the captured (and displayed) video and the apparent perspective offered to the user.
  • adjusting the view might comprise processing the captured video to effect the adjustment (either at the capturing device, the displaying device, or a control server, or at a combination of two or more of those devices), and the method 900, accordingly, can comprise modifying a video signal (with any of such devices) to adjust the apparent view of the displayed video (block 940).
  • the position and/or behavior of cameras at the capturing device can be adjusted to effect those changes, and the method 900, therefore, can include sending instructions from a displaying device (or a control server) to the capturing device to adjust the camera(s) accordingly (block 945), receiving such instructions at the capturing device (block 950), and/or controlling one or more cameras in accordance with the received instructions (block 955).
  • certain embodiments are configured to provide real-time (or near real-time) adjustments to the apparent view of the displayed video.
  • the user device on the viewer side can be configured to continually and/or periodically monitor the position of the user relative to the display device, and if the user device determines that the user has moved (block 960), the system can modify the apparent view of the displayed video (block 965), e.g., using the techniques described above, as shown by the flow continuing back to block 930.
  • the video calling device (or other user device) might capture video of the user and transmit that video to the control server (e.g., as part of a video call), and the control server might analyze that video for user position information before forwarding it to the video calling device at the other end of the video call; the control server then might provide instructions to the video calling device capturing video to be displayed to the user to modify camera behavior and/or might modify the video signal it receives from that video calling device before forwarding it to the viewer's calling device for display.
  • the functionality described herein can be divided among system components in any appropriate manner.
  • this functionality can be provided at both ends of a video call, such that a video device capturing video first party to a call can use that video to determine the position of the first party (and adjust the first party's apparent view accordingly), while a video calling device catching video of a second party to the call can use that video to determine a position of the second party relative to a display device on the second parties and of the call (and adjust the second parties apparent view accordingly).
  • a video device capturing video first party to a call can use that video to determine the position of the first party (and adjust the first party's apparent view accordingly)
  • a video calling device catching video of a second party to the call can use that video to determine a position of the second party relative to a display device on the second parties and of the call (and adjust the second parties apparent view accordingly).
  • the video captured of each party can be adjusted for display to the other party as part of the video call, providing a much more lifelike and interesting video calling experience.
  • these techniques can have utility in a wide variety of applications and are not limited to the examples described above.
  • these techniques can be used to provide a more realistic experience in the display of video games (e.g., using cameras or camera arrays in common use with many modern video game consoles), to provide a virtual window of a picturesque scene (e.g., times square, a nature scene, a child's room, and/or the like) in a remote location, such as in a virtual picture frame in an office.
  • Similar techniques can be used to enhance the presentation of television programs, sports, and/or any other broadcast video, movies, and/or the like.
  • Fig. 10 provides a schematic illustration of one embodiment of a computer system 1000 that can perform the methods provided by various other embodiments, as described herein, and/or can function as a video calling device, ICD, PDD, user device, control server, server computer, web server, and/or the like. It should be noted that Fig. 10 is meant only to provide a generalized illustration of various components, of which one or more (or none) of each may be utilized as appropriate. Fig. 10, therefore, broadly illustrates how individual system elements may be implemented in a relatively separated or relatively more integrated manner.
  • the computer system 1000 is shown comprising hardware elements that can be electrically coupled via a bus 1005 (or may otherwise be in communication, as appropriate).
  • the hardware elements may include one or more processors 1010, including without limitation one or more general-purpose processors and/or one or more special-purpose processors (such as digital signal processing chips, graphics acceleration processors, and/or the like); one or more input devices 1015, which can include, without limitation, a mouse, a keyboard, and/or the like; and one or more output devices 1020, which can include, without limitation, a display device, a printer, and/or the like.
  • processors 1010 including without limitation one or more general-purpose processors and/or one or more special-purpose processors (such as digital signal processing chips, graphics acceleration processors, and/or the like)
  • input devices 1015 which can include, without limitation, a mouse, a keyboard, and/or the like
  • output devices 1020 which can include, without limitation, a display device, a printer, and/or the like.
  • the computer system 1000 may further include (and/or be in communication with) one or more storage devices 1025, which can comprise, without limitation, local and/or network accessible storage, and/or can include, without limitation, a disk drive, a drive array, an optical storage device, solid-state storage device such as a random access memory (“RAM”) and/or a read-only memory (“ROM”), which can be programmable, flash-updateable, and/or the like.
  • storage devices 1025 can comprise, without limitation, local and/or network accessible storage, and/or can include, without limitation, a disk drive, a drive array, an optical storage device, solid-state storage device such as a random access memory (“RAM”) and/or a read-only memory (“ROM”), which can be programmable, flash-updateable, and/or the like.
  • RAM random access memory
  • ROM read-only memory
  • Such storage devices may be configured to implement any appropriate data stores, including, without limitation, various file systems, database structures, and/or the like.
  • the computer system 1000 might also include a communications subsystem 1030, which can include, without limitation, a modem, a network card (wireless or wired), an infra-red communication device, a wireless communication device and/or chipset (such as a BluetoothTM device, an 802.11 device, a WiFi device, a WiMax device, a WW AN device, cellular communication facilities, etc.), and/or the like.
  • the communications subsystem 1030 may permit data to be exchanged with a network (such as the network described below, to name one example), with other computer systems, and/or with any other devices described herein.
  • the computer system 1000 will further comprise a working memory 1035, which can include a RAM or ROM device, as described above.
  • the computer system 1000 also may comprise software elements, shown as being currently located within the working memory 1035, including an operating system 1040, device drivers, executable libraries, and/or other code, such as one or more application programs 1045, which may comprise computer programs provided by various embodiments, and/or may be designed to implement methods, and/or configure systems, provided by other embodiments, as described herein.
  • an operating system 1040 operating system 1040
  • device drivers executable libraries
  • application programs 1045 which may comprise computer programs provided by various embodiments, and/or may be designed to implement methods, and/or configure systems, provided by other embodiments, as described herein.
  • code and/or instructions can be used to configure and/or adapt a general purpose computer (or other device) to perform one or more operations in accordance with the described methods.
  • a set of these instructions and/or code might be encoded and/or stored on a non-transitory computer readable storage medium, such as the storage device(s) 1025 described above.
  • the storage medium might be incorporated within a computer system, such as the system 1000.
  • the storage medium might be separate from a computer system (i.e., a removable medium, such as a compact disc, etc.), and/or provided in an installation package, such that the storage medium can be used to program, configure, and/or adapt a general purpose computer with the instructions/code stored thereon.
  • These instructions might take the form of executable code, which is executable by the computer system 1000 and/or might take the form of source and/or installable code, which, upon compilation and/or installation on the computer system 1000 (e.g., using any of a variety of generally available compilers, installation programs,
  • compression/decompression utilities then takes the form of executable code.
  • some embodiments may employ a computer system (such as the computer system 1000) to perform methods in accordance with various embodiments of the invention. According to a set of embodiments, some or all of the procedures of such methods are performed by the computer system 1000 in response to processor 1010 executing one or more sequences of one or more instructions (which might be incorporated into the operating system 1040 and/or other code, such as an application program 1045) contained in the working memory 1035. Such instructions may be read into the working memory 1035 from another computer readable medium, such as one or more of the storage device(s) 1025. Merely by way of example, execution of the sequences of instructions contained in the working memory 1035 might cause the processor(s) 1010 to perform one or more procedures of the methods described herein.
  • a computer system such as the computer system 1000
  • some or all of the procedures of such methods are performed by the computer system 1000 in response to processor 1010 executing one or more sequences of one or more instructions (which might be incorporated into the operating system 1040 and/or other code, such as an application
  • system 1000 might further comprise one or more sensors 1050, which might include, without limitation, one or more cameras, one or more IR sensors, and/or one or more 3D sensors, or the like.
  • the one or more sensors 1050 might be incorporated in (or might otherwise be one of) the input device(s) 1015.
  • the output device(s) 1020 might, in some embodiments, further include one or more monitors, one or more TVs, and/or one or more display screens, or the like.
  • machine readable medium and “computer readable medium,” as used herein, refer to any medium that participates in providing data that causes a machine to operate in a specific fashion.
  • various computer readable media might be involved in providing instructions/code to processor(s) 1010 for execution and/or might be used to store and/or carry such instructions/code (e.g., as signals).
  • a computer readable medium is a non-transitory, physical, and/or tangible storage medium. Such a medium may take many forms, including, but not limited to, non- volatile media, volatile media, and transmission media.
  • Non-volatile media includes, for example, optical and/or magnetic disks, such as the storage device(s) 1025.
  • Volatile media includes, without limitation, dynamic memory, such as the working memory 1035.
  • Transmission media includes, without limitation, coaxial cables, copper wire and fiber optics, including the wires that comprise the bus 1005, as well as the various components of the communication subsystem 1030 (and/or the media by which the communications subsystem 1030 provides communication with other devices).
  • transmission media can also take the form of waves (including, without limitation, radio, acoustic, and/or light waves, such as those generated during radio-wave and infra-red data communications).
  • Common forms of physical and/or tangible computer readable media include, for example, a floppy disk, a flexible disk, a hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a PROM, an EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read instructions and/or code.
  • Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to the processor(s) 1010 for execution.
  • the instructions may initially be carried on a magnetic disk and/or optical disc of a remote computer.
  • a remote computer might load the instructions into its dynamic memory and send the instructions as signals over a transmission medium to be received and/or executed by the computer system 1000.
  • These signals which might be in the form of electromagnetic signals, acoustic signals, optical signals, and/or the like, are all examples of carrier waves on which instructions can be encoded, in accordance with various embodiments of the invention.
  • the communications subsystem 1030 (and/or components thereof) generally will receive the signals, and the bus 1005 then might carry the signals (and/or the data, instructions, etc. carried by the signals) to the working memory 1035, from which the processor(s) 1005 retrieves and executes the instructions.
  • the instructions received by the working memory 1035 may optionally be stored on a storage device 1025 either before or after execution by the processor(s) 1010.
  • a set of embodiments comprises systems collecting presence information and/or enabling monitoring of media content presentation and determination (e.g., selection or generation) of advertisements, based on presence information (regardless of whether the user device detecting the presence detection is owned by and/or associated with the user).
  • Fig. 11 illustrates a schematic diagram of a system 1100 that can be used in accordance with one set of embodiments.
  • the system 1100 can include one or more user computers 1105.
  • a user computer 1105 can be a video calling device, an ICD, a PDD, and/or a user device, as described above.
  • a user computer 1105 can be a general purpose personal computer (including, merely by way of example, desktop computers, workstations, tablet computers, laptop computers, handheld computers, mobile phones, smart phones, and the like), running any appropriate operating system, several of which are available from vendors such as Apple, Microsoft Corp., as well a variety of commercially-available UNIXTM or UNIX-like operating systems.
  • a user computer 1105 can also have any of a variety of applications, including one or more applications configured to perform methods provided by various embodiments (as described above, for example), as well as one or more office applications, database client and/or server applications, and/or web browser applications.
  • a user computer 1105 can be any other electronic device, such as a thin-client computer, Internet-enabled mobile telephone, and/or personal digital assistant, capable of communicating via a network (e.g. , the network 1110 described below) and/or of displaying and navigating web pages or other types of electronic documents.
  • a network e.g. , the network 1110 described below
  • exemplary system 1100 is shown with two user computers 1105, any number of user computers can be supported.
  • Certain embodiments operate in a networked environment, which can include a network 1110.
  • the network 1110 can be any type of network familiar to those skilled in the art that can support data communications using any of a variety of commercially- available (and/or free or proprietary) protocols, including, without limitation, TCP/IP, SNATM, IPXTM, AppleTalkTM, and the like.
  • the network 1110 can include a local area network ("LAN”), including, without limitation, a fiber network, an Ethernet network, a Token-RingTM network and/or the like; a wide-area network; a wireless wide area network ("WW AN”); a virtual network, such as a virtual private network (“VPN”); the Internet; an intranet; an extranet; a public switched telephone network (“PSTN”); an infra-red network; a wireless network, including without limitation a network operating under any of the IEEE 802.11 suite of protocols, the BluetoothTM protocol known in the art, and/or any other wireless protocol; and/or any combination of these and/or other networks.
  • LAN local area network
  • WW AN wireless wide area network
  • VPN virtual private network
  • PSTN public switched telephone network
  • PSTN public switched telephone network
  • a wireless network including without limitation a network operating under any of the IEEE 802.11 suite of protocols, the BluetoothTM protocol known in the art, and/or any other wireless protocol; and/or any combination of these
  • Embodiments can also include one or more server computers 1115.
  • Each of the server computers 1115 may be configured with an operating system, including, without limitation, any of those discussed above with respect to the user computers 1105, as well as any commercially (or freely) available server operating systems.
  • Each of the servers 1115 may also be running one or more applications, which can be configured to provide services to one or more clients 1105 and/or other servers 1115.
  • one of the servers 1115 might be a control server, with the functionality described above.
  • one of the servers might be a web server, which can be used, merely by way of example, to provide communication between a user computer 1105 and a control server, for example, to process requests for web pages or other electronic documents from user computers 1105 and/or to provide user input to the control server.
  • the web server can also run a variety of server applications, including HTTP servers, FTP servers, CGI servers, database servers, Java servers, and the like.
  • the web server may be configured to serve web pages that can be operated within a web browser on one or more of the user computers 1105 to perform operations in accordance with methods provided by various embodiments.
  • the server computers 1115 might include one or more application servers, which can be configured with one or more applications accessible by a client running on one or more of the client computers 1105 and/or other servers 1115.
  • the server(s) 1115 can be one or more general purpose computers capable of executing programs or scripts in response to the user computers 1105 and/or other servers 1115, including, without limitation, web applications (which might, in some cases, be configured to perform methods provided by various embodiments).
  • a web application can be implemented as one or more scripts or programs written in any suitable programming language, such as JavaTM, C, C#TM or C++, and/or any scripting language, such as Perl, Python, or TCL, as well as combinations of any programming and/or scripting languages.
  • the application server(s) can also include database servers, including, without limitation, those commercially available from OracleTM, MicrosoftTM, SybaseTM, IBMTM, and the like, which can process requests from clients (including, depending on the configuration, dedicated database clients, API clients, web browsers, etc.) running on a user computer 1105 and/or another server 1115.
  • an application server can create web pages dynamically for displaying the information in accordance with various embodiments, such as providing a user interface for a control server, as described above.
  • Data provided by an application server may be formatted as one or more web pages (comprising HTML, JavaScript, etc., for example) and/or may be forwarded to a user computer 1105 via a web server (as described above, for example).
  • a web server might receive web page requests and/or input data from a user computer 1105 and/or forward the web page requests and/or input data to an application server.
  • a web server may be integrated with an application server.
  • one or more servers 1115 can function as a file server and/or can include one or more of the files (e.g., application code, data files, etc.) necessary to implement various disclosed methods, incorporated by an application running on a user computer 1105 and/or another server 1115.
  • files e.g., application code, data files, etc.
  • a file server can include all necessary files, allowing such an application to be invoked remotely by a user computer 1105 and/or server 1115.
  • servers 1115 can be implemented by one or more containers or virtual machines operating in a cloud environment and/or a distributed, cloud-like environment based on shared resources of a plurality of user video calling devices, a plurality of ICDs, and/or a plurality of PDDs.
  • the system can include one or more data stores 1120.
  • the nature and location of the data stores 1120 is discretionary: merely by way of example, one data store 1120 might comprise a database 1120a that stores information about master accounts, user profiles, user preferences, assigned video calling devices, viewing/listening/Internet browsing/gaming patterns,
  • a data store 1120b might be a cloud storage environment for storing master accounts, user profiles, user preferences, uploaded monitored reactions of users, and/or the like.
  • the database 1120a and the cloud storage environment 1120b might be collocated and/or separate from one another. Some or all of the data stores 1120 might reside on a storage medium local to (and/or resident in) a server 1115a. Conversely, any of the data stores 1120 (and especially the cloud storage environment 1120b) might be remote from any or all of the computers 1105, 1115, so long as it can be in communication (e.g. , via the network 1110) with one or more of these.
  • a database 1120a can reside in a storage-area network ("SAN") familiar to those skilled in the art, and/or the cloud storage environment 1120b might comprise one or more SANs.
  • SAN storage-area network
  • the database 1120a can be a relational database, such as an Oracle database, that is adapted to store, update, and retrieve data in response to SQL-formatted commands.
  • the database might be controlled and/or maintained by a database server, as described above, for example.
  • the system can also include a first ICD 1125 and a second ICD 1130.
  • the first ICD 1125 in the context of the examples described herein corresponds to a device associated with a first user (or first video call participant), while the second ICD 1130 might correspond to a device associated a second user (or second video call participant).
  • the second ICD 1130 might correspond to a device associated a second user (or second video call participant).
  • ICDs 1125-1130 may be implemented in accordance with various embodiments.
  • each of the first ICD 1125 or the second ICD 1130 can determine presence and/or positions of one or more users (or audience members, or call participants, etc.), modify the displayed view based at least in part on the determined presence and/or positioned of the one or more users, and/or the like.
  • Each of the first ICD 1125 or the second ICD 1130 may be (or may have similar functionality as) a video calling device 105, a user device 105, an ICD 105, or a PDD 105, as described in detail above; in some cases, each of the first ICD 1125 or the second ICD 1130 might be (or may have similar functionality as) a VCD as described in the ' 182 Patent.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

L'invention concerne de nouveaux outils et de nouvelles techniques permettant d'afficher une vidéo. Dans certains modes de réalisation, de nouveaux outils et de nouvelles techniques peuvent être fournis pour détecter la présence et/ou la position d'un utilisateur dans une pièce, et/ou pour personnaliser un contenu affiché (y compris un contenu d'appel vidéo, un contenu multimédia, et/ou similaire) sur la base de la présence détectée et/ou de la position détectée de l'utilisateur. En particulier, selon certains aspects, un dispositif utilisateur (qui peut comprendre, sans limitation, un dispositif d'appel vidéo, un dispositif de capture d'image, une console de jeu, etc.) peut déterminer une position d'un utilisateur par rapport à un dispositif d'affichage en communication avec le dispositif utilisateur. Le dispositif utilisateur et/ou un serveur de commande (en communication avec le dispositif utilisateur sur un réseau) peut ajuster une vue apparente de vidéo(s) ou d'image(s) affichées sur le dispositif d'affichage, sur la base, au moins en partie, de la position déterminée de l'utilisateur par rapport au dispositif d'affichage.
PCT/US2014/054409 2013-09-06 2014-09-05 Fenêtre virtuelle WO2015035247A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201361874903P 2013-09-06 2013-09-06
US61/874,903 2013-09-06

Publications (1)

Publication Number Publication Date
WO2015035247A1 true WO2015035247A1 (fr) 2015-03-12

Family

ID=52628984

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2014/054409 WO2015035247A1 (fr) 2013-09-06 2014-09-05 Fenêtre virtuelle

Country Status (1)

Country Link
WO (1) WO2015035247A1 (fr)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9253520B2 (en) 2012-12-14 2016-02-02 Biscotti Inc. Video capture, processing and distribution system
US9300910B2 (en) 2012-12-14 2016-03-29 Biscotti Inc. Video mail capture, processing and distribution
US9485459B2 (en) 2012-12-14 2016-11-01 Biscotti Inc. Virtual window
US9654563B2 (en) 2012-12-14 2017-05-16 Biscotti Inc. Virtual remote functionality
US10046234B2 (en) 2015-10-20 2018-08-14 OBE Gaming Inc. Interactive movement tracking system applicable to articles of clothing
CN114072764A (zh) * 2020-02-28 2022-02-18 乐威指南公司 用于自适应地修改媒体内容的呈现的系统和方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050007445A1 (en) * 2003-07-11 2005-01-13 Foote Jonathan T. Telepresence system and method for video teleconferencing
WO2010118202A1 (fr) * 2009-04-08 2010-10-14 Gesturetek, Inc. Pointeur de détection d'écran portatif amélioré
WO2010147281A1 (fr) * 2009-06-16 2010-12-23 (주)엘지전자 Procédé de notification du champ de vision et récepteur de télévision pour mise en œuvre de ce procédé
US20110254914A1 (en) * 2010-04-14 2011-10-20 Alcatel-Lucent Usa, Incorporated Immersive viewer, a method of providing scenes on a display and an immersive viewing system
US20130231183A1 (en) * 2007-10-09 2013-09-05 Sony Computer Entertainment America Llc Increasing the number of advertising impressions in an interactive environment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050007445A1 (en) * 2003-07-11 2005-01-13 Foote Jonathan T. Telepresence system and method for video teleconferencing
US20130231183A1 (en) * 2007-10-09 2013-09-05 Sony Computer Entertainment America Llc Increasing the number of advertising impressions in an interactive environment
WO2010118202A1 (fr) * 2009-04-08 2010-10-14 Gesturetek, Inc. Pointeur de détection d'écran portatif amélioré
WO2010147281A1 (fr) * 2009-06-16 2010-12-23 (주)엘지전자 Procédé de notification du champ de vision et récepteur de télévision pour mise en œuvre de ce procédé
US20110254914A1 (en) * 2010-04-14 2011-10-20 Alcatel-Lucent Usa, Incorporated Immersive viewer, a method of providing scenes on a display and an immersive viewing system

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9253520B2 (en) 2012-12-14 2016-02-02 Biscotti Inc. Video capture, processing and distribution system
US9300910B2 (en) 2012-12-14 2016-03-29 Biscotti Inc. Video mail capture, processing and distribution
US9310977B2 (en) 2012-12-14 2016-04-12 Biscotti Inc. Mobile presence detection
US9485459B2 (en) 2012-12-14 2016-11-01 Biscotti Inc. Virtual window
US9654563B2 (en) 2012-12-14 2017-05-16 Biscotti Inc. Virtual remote functionality
US10046234B2 (en) 2015-10-20 2018-08-14 OBE Gaming Inc. Interactive movement tracking system applicable to articles of clothing
CN114072764A (zh) * 2020-02-28 2022-02-18 乐威指南公司 用于自适应地修改媒体内容的呈现的系统和方法
US11956500B2 (en) 2020-02-28 2024-04-09 Rovi Guides, Inc. Systems and methods for adaptively modifying presentation of media content
CN114072764B (zh) * 2020-02-28 2024-04-19 乐威指南公司 用于自适应地修改媒体内容的呈现的系统和方法

Similar Documents

Publication Publication Date Title
US9485459B2 (en) Virtual window
US9253520B2 (en) Video capture, processing and distribution system
US9300910B2 (en) Video mail capture, processing and distribution
US20140359647A1 (en) Monitoring, Trend Estimation, and User Recommendations
US20150026708A1 (en) Physical Presence and Advertising
US20150070516A1 (en) Automatic Content Filtering
US10499118B2 (en) Virtual and augmented reality system and headset display
CN106105246B (zh) 直播显示方法、装置及系统
US20200128223A1 (en) System and method for presenting and viewing a spherical video segment
US9743060B1 (en) System and method for presenting and viewing a spherical video segment
WO2015035247A1 (fr) Fenêtre virtuelle
US8370878B2 (en) Mobile interface for accessing interactive television applications associated with displayed content
US9215395B2 (en) Apparatus, system, and method for providing social content
US20190188756A1 (en) Methods and devices for determining distraction level of users to select targeted advertisements
US9955204B2 (en) System and method for distributing content through a set-top box
US20160029094A1 (en) Enabling interaction between social network users during synchronous display of video channgel
US20210044779A1 (en) Communicating in a Virtual Reality Environment
US20150042771A1 (en) Methods and systems for presenting supplemental content in media assets
WO2015031671A1 (fr) Présence physique et publicité
US9549224B2 (en) Methods and systems for presenting supplemental content in media assets
US10372882B2 (en) Media distribution network, associated program products, and methods of using the same
WO2014121148A1 (fr) Capture, traitement et distribution de courriel vidéo
WO2015013592A1 (fr) Adressage d'appels vidéo et de visioconférence
US11290772B2 (en) Multi-source content displaying interface
US11546669B2 (en) Systems and methods for stream viewing with experts

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14842741

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14842741

Country of ref document: EP

Kind code of ref document: A1