US20180070115A1 - Systems And Methods For Providing Real-Time Composite Video From Multiple Source Devices - Google Patents
Systems And Methods For Providing Real-Time Composite Video From Multiple Source Devices Download PDFInfo
- Publication number
- US20180070115A1 US20180070115A1 US15/584,717 US201715584717A US2018070115A1 US 20180070115 A1 US20180070115 A1 US 20180070115A1 US 201715584717 A US201715584717 A US 201715584717A US 2018070115 A1 US2018070115 A1 US 2018070115A1
- Authority
- US
- United States
- Prior art keywords
- video
- user device
- human element
- superimposed
- computer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
- H04N7/152—Multipoint control units therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M11/00—Telephonic communication systems specially adapted for combination with other electrical systems
- H04M11/08—Telephonic communication systems specially adapted for combination with other electrical systems specially adapted for optional reception of entertainment or informative matter
- H04M11/085—Telephonic communication systems specially adapted for combination with other electrical systems specially adapted for optional reception of entertainment or informative matter using a television receiver, e.g. viewdata system
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
- H04N7/147—Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals
-
- G06K9/00275—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/169—Holistic features and representations, i.e. based on the facial image taken as a whole
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
- H04N7/142—Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
- H04N2007/145—Handheld terminals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
- H04N7/142—Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
- H04N7/144—Constructional details of the terminal equipment, e.g. arrangements of the camera and the display camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact
Abstract
Systems and methods for superimposing the human elements of video generated by computing devices, wherein a first user device and second user device capture and transmit video to a central server which analyzes the video to identify human elements, superimposes these human elements upon one another, and transmits the newly created superimposed video back to at least one of the user devices.
Description
- The present subject matter relates generally to a teleconferencing system. More specifically, the present invention relates to teleconferencing system that simulates the mutual physical presence of users in a virtual interaction.
- A phone call over almost any distance is an inherently impersonal experience. Many developments in recent technology have aimed to solve the issue of people missing the aspects of human interactions hearing someone's voice alone does not provide. From teleconferencing, to Facetime and Snapchat, the use of video calling and messages has greatly enhanced communicating over great distances, but these innovations are not without their shortcomings.
- Existing video call technology does not allow a user to feel as though they are in close proximity to the person being called. While users may be able to see and speak with a colleague or loved one over Facetime, etc. it is readily apparent both users of such technology are in separate locations.
- Accordingly, there is a need for a video communication system that simulates the mutual physical presence of users in a virtual interaction.
- To meet the needs described above and others, in one embodiment, the subject matter provided herein is embodied in a video call application that provides users the illusion of both being present in a single physical location. Specifically, the embodiment presents the users the visual experience of being able to reach out and touch the person with whom they are speaking. The experience is provided through an application that allows users to make a video call with the additional benefit of being able to superimpose the video from other user(s) devices' cameras onto the video displayed on each of the user devices' screens. This can be used to provide a real time video image of all callers on the same screen, simulating that everyone involved in the call are physically in each other's presence.
- The application may be standalone or integrated into other video calling applications. The application may run on mobile devices (e.g., smartphones, tablets, etc.) and personal computers (e.g., desktop computers, laptops, etc.).
- One way in which the application may achieve the superimposed video effect is by use of the multiple cameras of a smartphone or tablet. Most mobile devices have two cameras, one on the front face of the device and one on the back. Some newer devices (e.g., the iPhone 7) include multiple cameras on the back of the device in addition to one or more front facing cameras. In a first example, the application may allow multiple cameras on a user device to be active at the same time, with the system superimposing the human elements (e.g., face, body, hands, etc.) of the video captured by device cameras to give an illusion of physical interaction.
- In a second example, the application may utilize a first user's front facing camera and a second user's rear facing camera to enable the first user to physically reach around to the back of the first user device such that the first user's hand (a human element of a video) appears on the second user's screen when the first user's hand is in view of their device's back camera. The first user's hand could be superimposed over the face, body, or any other human (or non-human) element(s) captured by the second user's camera. This allows users of the system to carry out the illusion of physical interactions such as shaking hands, high-fiving, etc. depending on which device cameras are utilized by end users.
- The video from all cameras utilized by system users at a given time may be fed into a central server, which in turn transmits the video(s) to other user(s) involved in a given video call. The transmission and reception of the video calls may be carried out via the internet or any other functionally capable communications network with the superimposition of video carried out by user devices, the central server, or both depending on what is most functionally advantageous.
- The application may also allow users to create a user profile which features information about the user, their call preferences, contacts, etc. User profiles may be stored in the memory of the central server, on user devices, or both.
- The application may allow for many different video call modes, including: Traditional Video Call—front or rear facing camera only; One Way Touch Call—a superimposed real time video image of one user's front camera and another user's rear camera (or vice versa); Rear Touch Call—a superimposed real time video image of both users' rear cameras (typically used to show holding hands, etc.); and Front Touch Call—a superimposed real time video image of both users' front cameras (typically used to show a kiss, etc.).
- A goal of the present invention is to increase the quality and intimacy of video calls. By using the front and rear camera's on a smart phone/tablet, the video superimposing software gives the impression of reaching out and touching another person, shaking hands, kissing, etc. Such interactions are not possible with traditional video chat and could be invaluable to long distance couples, isolated elderly people, overseas business partners, etc.
- In one embodiment, a computer-implemented method of superimposing video carried out by a processor, the method includes the steps of: receiving a first live video from a first user device; receiving a second live video from a second user device; identifying a first human element in the first live video and a second human element in the second live video; combining a portion of the first live video and a portion of the second live video in real-time to create a superimposed video including a frame perimeter within which a combined portion of the first live video and second live video is contained, wherein the superimposed video includes the first human element and the second human element, wherein, within the superimposed video, the first human element and the second human element may concurrently occupy any location within the frame perimeter; and transmitting the superimposed video to at least one of the first user device and the second user device.
- In some examples, in response to real-time movement by the first human element in the first live video and the second human element in the second live video, contact is simulated between the first human element and the second human element in the superimposed video.
- In other examples, in response to real-time movement by the first human element in the first live video and the second human element in the second live video, the first human element is superimposed upon the second human element in the superimposed video such that the first human element obscures at least a portion of the second human element.
- In some examples, the first user device is a mobile computing device, in others, it is a personal computer. In some examples, the first live video is captured by a camera of the first user device. In other examples, the first live video is captured by at least two cameras of the first user device simultaneously.
- In other examples, the first live video is captured by a rear facing camera of the first user device, the second live video is captured by a front facing camera of the second user device, and the first human element obscures at least a portion of the second human element in the transmitted superimposed video. In still others, the first live video is captured by a rear facing camera of the first user device, the second live video is captured by a rear facing camera of the second user device, and the first human element obscures at least a portion of the second human element in the transmitted superimposed video. Alternatively, the first live video may be captured by a front facing camera of the first user device, the second live video is captured by a front facing camera of the second user device, and the first human element obscures at least a portion of the second human element in the transmitted superimposed video.
- Yet another embodiment may feature a computer-implemented system for superimposing video, comprising: a central server featuring a processor, memory, and networking interface; a first user device featuring a camera, processor, memory, and networking interface; a second user device featuring a camera, processor, memory, and networking interface; wherein the central server, receives a first live video from a first user device and a second live video from a second user device, identifies a first human element in the first video and a second human element in the second video, combines a portion of the first video and a portion of the second video in real-time to create a superimposed video including a frame perimeter within which a combined portion of the first video and second video is contained, wherein the superimposed video includes the first human element and the second human element, wherein, within the superimposed video, the first human element and the second human element may concurrently occupy any location within the frame perimeter, and transmits the superimposed video to at least one of the first user device and the second user device.
- This system may, in response to real-time movement by the first human element in the first live video and the second human element in the second live video, contact is simulated between the first human element and the second human element in the superimposed video. The system may also, in response to real-time movement by the first human element in the first live video and the second human element in the second live video, the first human element is superimposed upon the second human element in the superimposed video such that the first human element obscures at least a portion of the second human element.
- The system may run on a smartphone or desktop computer, wherein the first live video is captured by a rear facing camera of the first user device, the second live video is captured by a front facing camera of the second user device, and the first human element obscures at least a portion of the second human element in the transmitted superimposed video. Alternatively, the first live video may be captured by a rear facing camera of the first user device, the second live video is captured by a rear facing camera of the second user device, and the first human element obscures at least a portion of the second human element in the transmitted superimposed video. Additionally, the first live video may be captured by a front facing camera of the first user device, the second live video is captured by a front facing camera of the second user device, and the first human element obscures at least a portion of the second human element in the transmitted superimposed video.
- In some examples of the systems and methods described herein, the superimposed video may simply be the human element of both callers' live video feeds superimposed together. In another example, it may be the human element of one caller's live video feed superimposed over the full live video feed from the second caller. It is contemplated that there is a technical advantage to just overlaying one identified human element, rather than selecting two human elements. For example, just overlaying one identified human element over the full video feed of the other caller requires less computing resources and ideally results in less latency.
- An advantage of the present invention is that the application gives another dimension to traditional video calls and allows friends and families that are apart from each other to experience the sensation of being able to touch their loved ones from anywhere with an internet connection. The present invention could allow someone climbing Mt. Everest to call someone in the depths of the Amazon rainforest and both parties could simulate being beside one another and able to touch as if they were side-by-side.
- Additional objects, advantages and novel features of the examples will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following description and the accompanying drawings or may be learned by production or operation of the examples. The objects and advantages of the concepts may be realized and attained by means of the methodologies, instrumentalities and combinations particularly pointed out in the appended claims.
- The drawing figures depict one or more implementations in accord with the present concepts, by way of example only, not by way of limitations. In the figures, like reference numerals refer to the same or similar elements.
-
FIG. 1 is a schematic diagram of a multi-feed video call system. -
FIG. 2 is a schematic diagram illustrating an example of an end user device that may be used in the system shown inFIG. 1 . -
FIG. 3 is a flowchart illustrating a computer-implemented method of superimposing video carried out by a processor. -
FIG. 4 illustrates the multi-feed video call system superimposing videos. -
FIG. 5A illustrates a One Way Touch call using the multi-feed video call system. -
FIG. 5B is a hybrid schematic-flowchart showing how the components of the multi-feed video call system may cooperate to accomplish a One Way Touch call using the process shown inFIG. 3 . -
FIG. 5C illustrates a user device displaying a superimposed video in a One Way Touch call. -
FIG. 6A illustrates a Front Touch call using the multi-feed video call system. -
FIG. 6B is a hybrid schematic-flowchart showing how the components of the multi-feed video call system may cooperate to accomplish a Front Touch call using the process shown inFIG. 3 . -
FIG. 7A is an overview diagram of a Rear Touch call using the multi-feed video call system. -
FIG. 7B is a hybrid schematic-flowchart showing how the components of the multi-feed video call system may cooperate to accomplish a Rear Touch call using the process shown inFIG. 3 . -
FIG. 8A is an overview diagram of a Multi-Way Touch call using the multi-feed video call system. -
FIG. 8B is a hybrid schematic-flowchart showing how the components of the multi-feed video call system may cooperate to accomplish a Multi-Way Touch call using the process shown inFIG. 3 . -
FIG. 1 is a schematic diagram of a multi-feedvideo call system 10. As shown inFIG. 1 , thesystem 10 features multiple end users'devices 20. Each end user device 20 (e.g., a laptop computer, smartphone, tablet, etc.) sendsvideo 210 to acentral server 30 from an end userdevice camera subsystem 116 through its wireless communication subsystem(s) 120 and receivesvideo 210 from thecentral server 30 to be displayed and output through the end user device I/O subsystem 128 and the end userdevice audio subsystem 124. As shown inFIG. 2 , acamera subsystem 116 may, for example, includefront 118 and back 119 cameras of a smartphone. - As described further herein, a primary object of the
system 10 is to enable a portion of a firstlive video 212 to be superimposed upon a second live video 214 (illustrated inFIG. 4 ) to produce a superimposed video 310 (illustrated inFIG. 4 ) in which human elements (e.g., a firsthuman element 216 and secondhuman element 218—also seen inFIG. 4 ) from each of the firstlive video 212 and the secondlive video 214 may interact and be displayed in any position relative to each other to simulate the appearance of the human elements from each of the firstlive video 212 and the secondlive video 214 to be present in the same physical space. - As shown in
FIG. 1 , thecentral server 30 includes aprocessor 31 andmemory 32 for carrying out the superimposition of video 210 (e.g., combining portions of a firstlive video 212 and a secondlive video 214 into the superimposed video 310), as well as anetworking interface 33 for communication withuser devices 20, as described further herein. The superimposed video 310 (shown inFIG. 4 ) created by theserver 30 is then transmitted back to theuser devices 20. Thesuperimposed video 310 simulates users being physically in each other's presence. - It should be noted that in the example described above, the analysis, processing, and transformation of
video 210 is carried out on thecentral server 30. In alternative embodiments, some, or all, of such actions may be carried out on one or more of theend user devices 20. -
FIG. 2 is a schematic diagram illustrating an example of anend user device 20 that may be used in the system shown inFIG. 1 . In the example shown inFIG. 2 , the multi-feedvideo call system 10 runs as a video conferencing application embodied invideo conferencing software 164 on theend user device 20. As shown inFIG. 2 , theend user device 20 maybe a mobile device, such as a smartphone, runningvideo conferencing software 164 to provide the functionality described herein. A user may install thevideo conferencing software 164 on his or herend user device 20 via Apple's App Store, the Android Market, etc. Theend user device 20 may include awireless communication subsystem 120 to communicate with thecentral server 30 running thevideo conferencing software 164. - The
user device 20 may include amemory interface 102,controllers 103, such as one or more data processors, image processors and/or central processors, and aperipherals interface 106. Thememory interface 102, the one ormore controllers 103 and/or the peripherals interface 106 can be separate components or can be integrated in one or more integrated circuits. The various components in theuser device 20 can be coupled by one or more communication buses or signal lines, as will be recognized by those skilled in the art. - Sensors, devices, and additional subsystems can be coupled to the peripherals interface 106 to facilitate various functionalities. For example, a motion sensor 108 (e.g., a gyroscope), a light sensor 163, and positioning sensors 112 (e.g., GPS receiver, accelerometer) can be coupled to the peripherals interface 106 to facilitate the orientation, lighting, and positioning functions described further herein.
Other sensors 114 can also be connected to theperipherals interface 106, such as a proximity sensor, a temperature sensor, a biometric sensor, or other sensing device, to facilitate related functionalities. - A
camera subsystem 116 includes a physical camera (e.g., a charged coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) optical sensor) which can be utilized to facilitate camera functions, such as recording photographs and video clips. Modern smartphones and other devices typically feature more than one physical camera operated by thecamera subsystem 116. Such cameras may be located on the front of thedevice 20—the side of the device with a screen (e.g., front cameras 118) or rear of thedevice 20—the side opposite the screen (e.g., rear facing cameras 119). - Communication functions can be facilitated through a network interface, such as one or more
wireless communication subsystems 120, which can include radio frequency receivers and transmitters and/or optical (e.g., infrared) receivers and transmitters. The specific design and implementation of thecommunication subsystem 120 can depend on the communication network(s) over which theuser device 20 is intended to operate. For example, theuser device 20 can includecommunication subsystems 120 designed to operate over a GSM network, a GPRS network, an EDGE network, a Wi-Fi or Imax network, and a Bluetooth network. In particular, thewireless communication subsystems 120 may include hosting protocols such that theuser device 20 may be configured as a base station for other wireless devices. - An
audio subsystem 122 can be coupled to aspeaker 124 and amicrophone 126 to facilitate voice-enabled functions, such as voice recognition, voice replication, digital recording, and telephony functions. - The I/
O subsystem 128 may include atouch screen controller 130 and/or other input controller(s) 132. The touch-screen controller 130 can be coupled to atouch screen 134, such as a touch screen. Thetouch screen 134 andtouch screen controller 130 can, for example, detect contact and movement, or break thereof, using any of a plurality of touch sensitivity technologies, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with thetouch screen 134. The other input controller(s) 132 can be coupled to other input/control devices 136, such as one or more buttons, rocker switches, thumb-wheel, infrared port, USB port, and/or a pointer device such as a stylus. The one or more buttons (not shown) can include an up/down button for volume control of thespeaker 124 and/or themicrophone 126. - The
memory interface 102 may be coupled tomemory 104. Thememory 104 can include high-speed random access memory and/or non-volatile memory, such as one or more magnetic disk storage devices, one or more optical storage devices, and/or flash memory (e.g., NAND, NOR). Thememory 104 may storeoperating system instructions 140, such as Darwin, RTXC, LINUX, UNIX, OS X, iOS, ANDROID, BLACKBERRY OS,BLACKBERRY 10, WINDOWS, or an embedded operating system such as VxWorks. Theoperating system instructions 140 may include instructions for handling basic system services and for performing hardware dependent tasks. In some implementations, theoperating system instructions 140 can be a kernel (e.g., UNIX kernel). - The
memory 104 may also storecommunication instructions 142 to facilitate communicating with one or more additional devices, one or more computers and/or one or more servers. Thememory 104 may include graphicaluser interface instructions 144 to facilitate graphic user interface processing;sensor processing instructions 146 to facilitate sensor-related processing and functions; phone instructions 148 to facilitate phone-related processes and functions;electronic messaging instructions 150 to facilitate electronic-messaging related processes and functions;web browsing instructions 152 to facilitate web browsing-related processes and functions;media processing instructions 154 to facilitate media processing-related processes and functions; GPS/Navigation instructions 156 to facilitate GPS and navigation-related processes and instructions;camera instructions 158 to facilitate camera-related processes and functions; and/orother software instructions 160 to facilitate other processes and functions (e.g., access control management functions, etc.). Thememory 104 may also store other software instructions controlling other processes and functions of theuser device 20 as will be recognized by those skilled in the art. In some implementations, themedia processing instructions 154 are divided into audio processing instructions and video processing instructions to facilitate audio processing-related processes and functions and video processing-related processes and functions, respectively. An activation record and International Mobile Equipment Identity (IMEI) 162 or similar hardware identifier can also be stored inmemory 104. As described above, thevideo conferencing software 164 is also stored in thememory 104 and run by thecontrollers 103. - Each of the above identified instructions and applications can correspond to a set of instructions for performing one or more functions described herein. These instructions need not be implemented as separate software programs, procedures, or modules. The
memory 104 can include additional instructions or fewer instructions. Furthermore, various functions of theuser device 20 may be implemented in hardware and/or in software, including in one or more signal processing and/or application specific integrated circuits. Accordingly, theuser device 20, as shown inFIG. 2 , may be adapted to perform any combination of the functionality described herein. - Aspects of the systems and methods described herein are controlled by one or
more controllers 103. The one ormore controllers 103 may be adapted run a variety of application programs, access and store data, including accessing and storing data in associated databases, and enable one or more interactions via theuser device 20. Typically, the one ormore controllers 103 are implemented by one or more programmable data processing devices. The hardware elements, operating systems, and programming languages of such devices are conventional in nature, and it is presumed that those skilled in the art are adequately familiar therewith. - For example, the one or
more controllers 103 may be a PC based implementation of a central control processing system utilizing a central processing unit (CPU), memories and an interconnect bus. The CPU may contain a single microprocessor, or it may contain a plurality ofmicrocontrollers 103 for configuring the CPU as a multi-processor system. The memories include a main memory, such as a dynamic random access memory (DRAM) and cache, as well as a read only memory, such as a PROM, EPROM, FLASH-EPROM, or the like. The system may also include any form of volatile or non-volatile memory. In operation, the main memory is non-transitory and stores at least portions of instructions for execution by the CPU and data for processing in accord with the executed instructions. - The one or
more controllers 103 may further include appropriate input/output ports for interconnection with one or more output displays (e.g., monitors, printers,touchscreen 134, motion-sensinginput device 108, etc.) and one or more input mechanisms (e.g., keyboard, mouse, voice, touch, bioelectric devices, magnetic reader, RFID reader, barcode reader,touchscreen 134, motion-sensinginput device 108, etc.) serving as one or more user interfaces for the processor. For example, the one ormore controllers 103 may include a graphics subsystem to drive the output display. The links of the peripherals to the system may be wired connections or use wireless communications. - Although summarized above as a smartphone-type implementation, those skilled in the art will recognize that the one or
more controllers 103 also encompasses systems such as host computers, servers, workstations, network terminals, PCs, and the like. Further one ormore controllers 103 may be embodied in auser device 20, such as a mobile electronic device, like a smartphone or tablet computer. In fact, the use of the term controller is intended to represent a broad category of components that are well known in the art. -
FIG. 3 is a flowchart illustrating a computer-implemented method of superimposingvideo 210 carried out by aprocessor 31. As shown inFIG. 3 , the method of superimposingvideo 210 carried out by aprocessor 31 begins with theprocessor 31, at afirst step 240 receiving a firstlive video 212 from a first user'sdevice 20.Reception 240 by aprocessor 31 is illustrated inFIG. 4 , wherein theuser device 20 of a first user transmits a first live video 212 (in this case avideo 210 captured by the user's rear camera 119) to aprocessor 31 containingcentral server 30. Thesecond step 242 of superimposingvideo 210 carried out by aprocessor 31 is receiving a secondlive video 214 from a second user'sdevice 20. Again referring toFIG. 4 , reception of the secondlive video 214 from a second user'sdevice 20 by aprocessor 31 is illustrated (with the secondlive video 214 being captured by the second user's rear camera 119). Thethird step 244 of this method calls for theprocessor 31 to identify a firsthuman element 216 in thefirst video 212 and a secondhuman element 218 in asecond video 214. Suchhuman elements FIG. 4 with the firsthuman element 216 being a hand (captured by the first user's rear camera 119) and the secondhuman element 218 being a face (captured by the second user's front camera 118). - The
fourth step 246 of the computer-implemented method of superimposingvideo 210 carried out by aprocessor 31 is combing a portion of thefirst video 213 and a portion of thesecond video 215 in real-time to create asuperimposed video 310 including aframe perimeter 309 within which a combined portion of thefirst video 212 andsecond video 214 is contained, wherein the superimposedvideo 310 includes the firsthuman element 216 and the secondhuman element 218, wherein, within the superimposedvideo 310, the firsthuman element 216 and the secondhuman element 218 may concurrently occupy any location within theframe perimeter 309. - A key feature of the multi-feed
video call system 10 is that, within the superimposedvideo 310, the firsthuman element 216 and the secondhuman element 218, are each able to occupy any portion of the superimposedvideo 310. In the example shown inFIG. 4 , this feature is represented by the ability of either the firsthuman element 216 or the secondhuman element 218 to occupy any space within theframe perimeter 309 and theframe perimeter 309 is shown to occupy the entirety of the display on thedevice 20. While this is one contemplated example, it is understood that theframe perimeter 309 for the superimposedvideo 310 may otherwise occupy a smaller portion of the display on thedevice 20. The critical concept relating to the ability of either the firsthuman element 216 or the secondhuman element 218 to occupy any space within theframe perimeter 309 is that the firsthuman element 216 and the secondhuman element 218 may occupy adjacent positions on the screen, may separate from each other, and may pass in front of or behind each other, or any be represented by any combination of these relative positions. For example, some portion of the firsthuman element 216 may be shown to be touching the secondhuman element 218 while other portions of the firsthuman element 216 may be shown to be separated from the secondhuman element 218. - The portion of the
first video 213 and a portion of thesecond video 215 combined in real-time may be only the firsthuman element 216 and secondhuman element 218, or can includemore video 210 from thelive videos additional video 210, beyond thehuman elements background 219. Such abackground 219 is illustrated inFIG. 4 (in this case a plain solid color), with thehuman elements frame perimeter 309 and move independently of each other and thebackground 219. Thebackground 219 can be generated by thesystem 10 of captured by adevice 20camera 118, 119.In the example shown inFIG. 4 , the first human element 216 (a hand) is superimposed on top of the second human element 218 (a face) with bothelements human element 216 is superimposed by theprocessor 31 over the secondhuman element 218, the hand is able to obscure the face in the superimposedvideo 310. Theframe perimeter 309, also illustrated inFIG. 4 , is the defined size of the superimposed video 310 (e.g., the aspect ratio, etc.) which may be automatically determined by theprocessor 31 based off thevideos 210 provided to it. - The
final step 248 of the computer-implemented method of superimposingvideo 210 carried out by aprocessor 31 is transmitting the superimposedvideo 310 to auser device 20. Such transmission is shown inFIG. 4 , in which the superimposedvideo 310 is displayed on the first user and second user'sdevices 20. Worth noting here is that the example shown inFIG. 4 utilizes one user device's 20front camera 118 and one user device's 20back camera 119, but the present invention may also utilize multiple cameras of adevice 20 facing the same direction to capturevideo 210 concurrently. Onesuch device 20 with multiple cameras facing the same direction is the iPhone 7, which is capable of blending or stitching images captured by is multiple cameras together to provide wide angle images, greater image quality, etc. Such functionality may be utilized by the current invention to improve the quality of video calls carried out upon it. - Additionally,
FIG. 4 demonstrates twohuman elements frame perimeter 309 at the same time. This results in one of thehuman elements 216 being able to obscure the other 218. Thepresent system 10 may also be configured in a manner which prevents the identifiedhuman elements human elements element 216 being able to pass over and obscure another 218, with thebackground 219,frame perimeter 309, etc. being automatically adjusted by thesystem 10 to prevent such overlap. -
FIG. 5A is an overview diagram of a One Way Touch call using the multi-feedvideo call system 10. As shown inFIG. 5A , a superimposedreal time video 310 of one user's front camera 118 (Caller 2) and another user's rear camera 119 (Caller 1) is displayed to both users. In this example, thevideo 210 fromCaller 2's device'sfront camera 118 is shown on bothCaller 1 andCaller 2'sdevice 20 screens, with thevideo 210 fromCaller 1's device'sback camera 119 superimposed overCaller 2'sfront camera 118video 210 allowingCaller 1 to “touch” (i.e., see their hand or other human element(s) superimposed upon the face and/or body of another user)Caller 2 via an illusion created by the superimposition of thevideo 210. In this example, the firsthuman element 216 is associated withCaller 1's hand and the secondhuman element 218 is associated withCaller 2's face, neck, and upper torso. It should be noted however the labels regarding the firsthuman element 216 and secondhuman element 218 could be reversed in this example (and the examples seen inFIGS. 6A, 7A , and 8A) as which human element is labeled first and second does not automatically dictate which element will be superimposed over the other. -
FIG. 5B is a flowchart of a One Way Touch call using the multi-feedvideo call system 10. As shown inFIG. 5B , thevideo 210 which will be superimposed by thesystem 10 originate on respective caller'send user devices 20. One user's (Caller 1's)rear camera 119 and another user's (Caller 2's)front camera 118send video 210 to acentralized sever 30. In this embodiment, as well as the embodiments illustrated inFIGS. 3C, 4B, 5B, and 6B , thesystem 10 may automatically determine which user'svideo 210 is superimposed over the other (e.g., which human element (hand, face, torso, etc.) is superimposed over the other human element(s) displayed in the superimposed video 310). In other embodiments, the determination of which human element(s) ofvideo 210 are superimposed upon one another may also be manually set by the participants of a given video call or thesystem 10 may be set to not superimpose human elements. Thevideo 210 may be sent via the internet or any other functionally useful means, with thecentral server 30 receiving theseparate video 210, analyzing them, removing extraneous information from the video 210 (e.g., solid colored backgrounds, etc.), and combining the tworespective video 210 into onesuperimposed video 310. Thesuperimposed video 310 is then sent back to the user device's 20 involved in the video chat via the internet or any other functionally useful means. - In this example, the first
live video 212 is associated withCaller 1's and the secondlive video 214 is associated withCaller 2. It should be noted however the labels regarding thelive videos FIGS. 6B, 7B, and 8B ) as whichvideo -
FIG. 5C is a diagram of auser device 20 displaying a One Way Touch call. As shown inFIG. 5C , anend user device 20 may display a super imposedvideo 310 which features, in this example, thevideo 210 fromCaller 2's (as shown inFIG. 3A ) device'sfront camera 118 with thevideo 210 fromCaller 1's (as shown inFIG. 3A ) device'sback camera 119 superimposed overCaller 2'sfront camera 118video 210 allowingCaller 1 to “touch” (i.e., see their hand or other human element(s) superimposed upon the face and/or body of another user)Caller 2 via an illusion shown within the superimposedvideo 310. -
FIG. 6A is an overview diagram of a Front Touch call using the multi-feedvideo call system 10. As shown inFIG. 6A , a superimposedreal time video 310 of both users' front camera 118 (Caller 1 and Caller 2) is displayed to both users. In this example, thevideo 210 fromCaller 1's device'sfront camera 118 is shown on bothCaller 1 andCaller 2'sdevice 20 screens, with thevideo 210 fromCaller 2's device'sfront camera 118 superimposed overCaller 2'sfront camera 118video 210, allowing the users to appear to be physically side by side. -
FIG. 6B is a flowchart of a Front Touch call using the multi-feedvideo call system 10. As shown inFIG. 6B , thevideo 210 which will be superimposed by thesystem 10 originate on respective caller'send user devices 20. Both users' (Caller 1 and Caller 2)front camera 118send video 210 to acentralized server 30. Thevideo 210 may be sent via the internet or any other functionally useful means, with thecentral server 30 receiving theseparate video 210, analyzing them, removing extraneous information from the video 210 (e.g., solid colored backgrounds, etc.), and combining the tworespective video 210 into onesuperimposed video 310. Thesuperimposed video 310 is then sent back to the user device's 20 involved in the video chat via the internet or any other functionally useful means. -
FIG. 7A is an overview diagram of a Rear Touch call using the multi-feedvideo call system 10. As shown inFIG. 7A , a superimposedreal time video 310 of both users' rear cameras 119 (Caller 1 and Caller 2) is displayed to both users. In this example, thevideo 210 fromCaller 1's device's rear camera is shown on bothCaller 1 andCaller 2'sdevice 20 screens, with thevideo 210 fromCaller 2's device'srear camera 119 superimposed overCaller 1'srear camera video 210, forming thesuperimposed video 310, and allowing the users to appear to be physically holding hands, etc. -
FIG. 7B is a flowchart of a Rear Touch call using the multi-feedvideo call system 10. As shown inFIG. 7B , thevideo 210 which will be superimposed by thesystem 10 originate on respective caller'send user devices 20. Both users' (Caller 1 and Caller 2)rear camera 119send video 210 to acentralized server 30. Thevideo 210 may be sent via the internet or any other functionally useful means, with thecentral server 30 receiving theseparate video 210, analyzing them, removing extraneous information from the videos 210 (e.g., solid colored backgrounds, etc.), and combining the tworespective video 210 into onesuperimposed video 310. Thesuperimposed video 310 is then sent back to the user device's 20 involved in the video chat via the internet or any other functionally useful means. -
FIG. 8A is an overview diagram of a Multi-Way Touch call using the multi-feedvideo call system 10. As shown inFIG. 8A , a superimposedreal time video 310 of a first user's front camera 118 (Caller 1) and a second user's rear camera 119 (Caller 2) is displayed to the second user, with a superimposedreal time video 310 of the video of the second user's front camera 118 (Caller 2) and the first user's rear camera 119 (Caller 1) displayed to the first user. This allows both users to “touch” the other user simultaneously with the visual effect being enabled by the superimposition of video. -
FIG. 8B is a flowchart of a Multi-Way Touch call using the multi-feedvideo call system 10. As shown inFIG. 8B , thevideo 210 which will be superimposed by thesystem 10 originate on respective caller'send user devices 20. Both user'srear camera 119 andfront camera 118send video 210 to acentralized server 30. The video may be sent via the internet or any other functionally useful means, with thecentral server 30 receiving theseparate video 210, analyzing them, removing extraneous information from the video 210 (e.g., solid colored backgrounds, etc.), and combining the fourrespective video 210 into two superimposed video 310 (as discussed inFIG. 8A ). Thesuperimposed video 310 are then sent back to the respective user device's 20 involved in the video chat via the internet or any other functionally useful means. - Hence aspects of the systems and methods provided herein encompass hardware and software for controlling the relevant functions. Software may take the form of code or executable instructions for causing a processor or other programmable equipment to perform the relevant steps, where the code or instructions are carried by or otherwise embodied in a medium readable by the processor or other machine. Instructions or code for implementing such operations may be in the form of computer instruction in any form (e.g., source code, object code, interpreted code, etc.) stored in or carried by any tangible readable medium.
- It should be noted that various changes and modifications to the presently preferred embodiments described herein will be apparent to those skilled in the art. Such changes and modifications may be made without departing from the spirit and scope of the present invention and without diminishing its attendant advantages.
Claims (14)
1. A computer-implemented method of superimposing video carried out by a processor, the method comprising the steps of:
receiving a first live video from a first user device;
receiving a second live video from a second user device;
extracting a first human element from the first live video;
combining the first human element and a portion or all of the second live video in real-time to create a superimposed video including a frame perimeter within which the superimposed video includes the first human element and a second human element from the second live video, wherein, within the superimposed video, the first human element and the second human element may concurrently occupy any location within the frame perimeter; and
transmitting the superimposed video to at least one of the first user device and the second user device;
wherein the first live video is captured by a rear facing camera of the first user device and, in response to movement of the first human element relative to the first user device, the first human element obscures at least a portion of the second human element in the transmitted superimposed video
further wherein the location of the extracted human element within the superimposed video is directly controlled in real-time by the position of the first human element relative to the first user device.
2. The computer-implemented method of superimposing video of claim 1 , wherein in response to real-time movement by the first human element relative to the first user device and the second human element relative to the second user device, contact is simulated between the first human element and the second human element in the superimposed video.
3. The computer-implemented method of superimposing video of claim 1 , wherein the first user device is a mobile computing device.
4. The computer-implemented method of superimposing video of claim 1 , wherein the first user device is a personal computer.
5. The computer-implemented method of superimposing video of claim 1 , wherein the first live video is captured by at least two cameras of the first user device simultaneously.
6. The computer-implemented method of superimposing video of claim 1 , wherein the second live video is captured by a front facing camera of the second user device.
7. The computer-implemented method of superimposing video of claim 1 , wherein the second live video is captured by a rear facing camera of the second user device.
8. A computer-implemented system for superimposing video, comprising:
a central server featuring a processor, memory, and networking interface;
a first user device featuring a camera, processor, memory, and networking interface;
a second user device featuring a camera, processor, memory, and networking interface;
wherein the central server:
receives a first live video from a first user device and a second live video from a second user device;
extracts a first human element from the first live video;
combines the first human element with a portion or all of the second live video in real-time to create a superimposed video including a frame perimeter within which the superimposed video includes the first human element and a second human element from the second live video, wherein, within the superimposed video, the first human element and the second human element may concurrently occupy any location within the frame perimeter; and
transmits the superimposed video to at least one of the first user device and the second user device;
wherein the first live video is captured by a rear facing camera of the first user device and, in response to movement of the first human element relative to the first user device, the first human element obscures at least a portion of the second human element in the transmitted superimposed video
further wherein the location of the extracted human element within the superimposed video is directly controlled in real-time by the position of the first human element relative to the first user device.
9. The computer-implemented system for superimposing video of claim 8 , wherein in response to real-time movement by the first human element relative to the first user device and the second human element relative to the second user device, contact is simulated between the first human element and the second human element in the superimposed video.
10. The computer-implemented system for superimposing video of claim 8 , wherein the first user device is a smartphone.
11. The computer-implemented system for superimposing video of claim 8 , wherein the first user device is a desktop computer.
12. The computer-implemented system of superimposing video of claim 8 , wherein the first live video is captured by at least two cameras of the first user device simultaneously.
13. The computer-implemented system for superimposing video of claim 8 , wherein the second live video is captured by a front facing camera of the second user device.
14. The computer-implemented system for superimposing video of claim 8 , wherein the second live video is captured by a rear facing camera of the second user device.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/584,717 US9912970B1 (en) | 2016-09-02 | 2017-05-02 | Systems and methods for providing real-time composite video from multiple source devices |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201662383006P | 2016-09-02 | 2016-09-02 | |
US201662407267P | 2016-10-12 | 2016-10-12 | |
US15/360,536 US9729820B1 (en) | 2016-09-02 | 2016-11-23 | Systems and methods for providing real-time composite video from multiple source devices |
US15/584,717 US9912970B1 (en) | 2016-09-02 | 2017-05-02 | Systems and methods for providing real-time composite video from multiple source devices |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US201615359232A Continuation | 2016-09-02 | 2016-11-22 | |
US15/360,536 Continuation US9729820B1 (en) | 2016-09-02 | 2016-11-23 | Systems and methods for providing real-time composite video from multiple source devices |
Publications (2)
Publication Number | Publication Date |
---|---|
US9912970B1 US9912970B1 (en) | 2018-03-06 |
US20180070115A1 true US20180070115A1 (en) | 2018-03-08 |
Family
ID=59410788
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/360,536 Active US9729820B1 (en) | 2016-09-02 | 2016-11-23 | Systems and methods for providing real-time composite video from multiple source devices |
US15/584,717 Active US9912970B1 (en) | 2016-09-02 | 2017-05-02 | Systems and methods for providing real-time composite video from multiple source devices |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/360,536 Active US9729820B1 (en) | 2016-09-02 | 2016-11-23 | Systems and methods for providing real-time composite video from multiple source devices |
Country Status (1)
Country | Link |
---|---|
US (2) | US9729820B1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109379552A (en) * | 2018-11-09 | 2019-02-22 | 维沃移动通信有限公司 | A kind of content display method and terminal device |
US11676329B1 (en) * | 2022-01-07 | 2023-06-13 | Meta Platforms Technologies, Llc | Mobile device holographic calling with front and back camera capture |
US11676330B2 (en) | 2020-07-21 | 2023-06-13 | Meta Platforms Technologies, Llc | 3d conversations in an artificial reality environment |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10089793B2 (en) | 2016-09-02 | 2018-10-02 | Russell Holmes | Systems and methods for providing real-time composite video from multiple source devices featuring augmented reality elements |
US9979921B2 (en) | 2016-09-02 | 2018-05-22 | Russell Holmes | Systems and methods for providing real-time composite video from multiple source devices |
CN108134913B (en) * | 2017-12-21 | 2021-06-01 | 广东小天才科技有限公司 | Intelligent adjustment method for video call and wearable device |
EP3785443A4 (en) * | 2018-04-27 | 2021-12-08 | Reliance Jio Infocomm Limited | System and method for rendering stitched video media stream on a display device |
US11910034B2 (en) * | 2018-12-21 | 2024-02-20 | Koninklijke Kpn N.V. | Network-based assistance for receiver processing of video data |
US10846898B2 (en) * | 2019-03-28 | 2020-11-24 | Nanning Fugui Precision Industrial Co., Ltd. | Method and device for setting a multi-user virtual reality chat environment |
US11949527B2 (en) * | 2022-04-25 | 2024-04-02 | Snap Inc. | Shared augmented reality experience in video chat |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7855737B2 (en) * | 2008-03-26 | 2010-12-21 | Fotonation Ireland Limited | Method of making a digital camera image of a scene including the camera user |
US8502856B2 (en) * | 2010-04-07 | 2013-08-06 | Apple Inc. | In conference display adjustments |
Family Cites Families (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5572248A (en) | 1994-09-19 | 1996-11-05 | Teleport Corporation | Teleconferencing method and system for providing face-to-face, non-animated teleconference environment |
US6724417B1 (en) * | 2000-11-29 | 2004-04-20 | Applied Minds, Inc. | Method and apparatus maintaining eye contact in video delivery systems using view morphing |
KR101130413B1 (en) | 2003-06-19 | 2012-03-27 | 소니 에릭슨 모빌 커뮤니케이션즈 에이비 | Media stream mixing |
US8004555B2 (en) | 2006-05-31 | 2011-08-23 | Motorola Mobility, Inc. | Methods and devices for simultaneous dual camera video telephony |
US20090327893A1 (en) * | 2008-06-25 | 2009-12-31 | Paul Terry | Coordinated video presentation methods and apparatus |
US20110060835A1 (en) | 2009-09-06 | 2011-03-10 | Dorso Gregory | Communicating with a user device in a computer environment |
US20130083151A1 (en) * | 2011-09-30 | 2013-04-04 | Lg Electronics Inc. | Electronic device and method for controlling electronic device |
US9007427B2 (en) | 2011-12-14 | 2015-04-14 | Verizon Patent And Licensing Inc. | Method and system for providing virtual conferencing |
US20130162766A1 (en) * | 2011-12-22 | 2013-06-27 | 2Dinto3D LLC | Overlaying frames of a modified video stream produced from a source video stream onto the source video stream in a first output type format to generate a supplemental video stream used to produce an output video stream in a second output type format |
JP2017505553A (en) | 2013-11-29 | 2017-02-16 | インテル・コーポレーション | Camera control by face detection |
US9454840B2 (en) * | 2013-12-13 | 2016-09-27 | Blake Caldwell | System and method for interactive animations for enhanced and personalized video communications |
US20150188970A1 (en) | 2013-12-31 | 2015-07-02 | Personify, Inc. | Methods and Systems for Presenting Personas According to a Common Cross-Client Configuration |
US10638082B2 (en) * | 2014-08-28 | 2020-04-28 | Gregory A. Pearson, Inc. | Systems and methods for picture-in-picture video conference functionality |
US9282287B1 (en) * | 2014-09-09 | 2016-03-08 | Google Inc. | Real-time video transformations in video conferences |
US9503682B2 (en) * | 2014-12-17 | 2016-11-22 | Fuji Xerox Co., Ltd. | Systems and methods for conveying physical state of a remote device |
US9576607B2 (en) * | 2015-01-21 | 2017-02-21 | Google Inc. | Techniques for creating a composite image |
US10154247B2 (en) * | 2015-05-19 | 2018-12-11 | Hashplay Inc. | Virtual reality communication systems and methods thereof |
US9349414B1 (en) | 2015-09-18 | 2016-05-24 | Odile Aimee Furment | System and method for simultaneous capture of two video streams |
-
2016
- 2016-11-23 US US15/360,536 patent/US9729820B1/en active Active
-
2017
- 2017-05-02 US US15/584,717 patent/US9912970B1/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7855737B2 (en) * | 2008-03-26 | 2010-12-21 | Fotonation Ireland Limited | Method of making a digital camera image of a scene including the camera user |
US8502856B2 (en) * | 2010-04-07 | 2013-08-06 | Apple Inc. | In conference display adjustments |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109379552A (en) * | 2018-11-09 | 2019-02-22 | 维沃移动通信有限公司 | A kind of content display method and terminal device |
US11676330B2 (en) | 2020-07-21 | 2023-06-13 | Meta Platforms Technologies, Llc | 3d conversations in an artificial reality environment |
US11967014B2 (en) | 2020-07-21 | 2024-04-23 | Meta Platforms Technologies, Llc | 3D conversations in an artificial reality environment |
US11676329B1 (en) * | 2022-01-07 | 2023-06-13 | Meta Platforms Technologies, Llc | Mobile device holographic calling with front and back camera capture |
Also Published As
Publication number | Publication date |
---|---|
US9912970B1 (en) | 2018-03-06 |
US9729820B1 (en) | 2017-08-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9979921B2 (en) | Systems and methods for providing real-time composite video from multiple source devices | |
US9912970B1 (en) | Systems and methods for providing real-time composite video from multiple source devices | |
US10089793B2 (en) | Systems and methods for providing real-time composite video from multiple source devices featuring augmented reality elements | |
US10554921B1 (en) | Gaze-correct video conferencing systems and methods | |
US9075429B1 (en) | Distortion correction for device display | |
US11151796B2 (en) | Systems and methods for providing real-time composite video from multiple source devices featuring augmented reality elements | |
EP2701152B1 (en) | Media object browsing in a collaborative window, mobile client editing, augmented reality rendering. | |
US9317113B1 (en) | Gaze assisted object recognition | |
WO2015188614A1 (en) | Method and device for operating computer and mobile phone in virtual world, and glasses using same | |
CN112243583B (en) | Multi-endpoint mixed reality conference | |
US11423627B2 (en) | Systems and methods for providing real-time composite video from multiple source devices featuring augmented reality elements | |
US11122220B2 (en) | Augmented video reality | |
US20220398816A1 (en) | Systems And Methods For Providing Real-Time Composite Video From Multiple Source Devices Featuring Augmented Reality Elements | |
US9262689B1 (en) | Optimizing pre-processing times for faster response | |
US20230386147A1 (en) | Systems and Methods for Providing Real-Time Composite Video from Multiple Source Devices Featuring Augmented Reality Elements | |
US20140098138A1 (en) | Method and system for augmented reality based smart classroom environment | |
US9160923B1 (en) | Method and system for dynamic information display using optical data | |
CN116320721A (en) | Shooting method, shooting device, terminal and storage medium | |
CN113961133A (en) | Display control method and device for electronic equipment, electronic equipment and storage medium | |
US20170324935A1 (en) | Virtual representation of a user portion | |
KR102428438B1 (en) | Method and system for multilateral remote collaboration based on real-time coordinate sharing | |
US20240119674A1 (en) | Terminal apparatus | |
US20230289919A1 (en) | Video stream refinement for dynamic scenes | |
US20240127769A1 (en) | Terminal apparatus | |
US20230196680A1 (en) | Terminal apparatus, medium, and method of operating terminal apparatus |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: PETITION RELATED TO MAINTENANCE FEES GRANTED (ORIGINAL EVENT CODE: PTGR) |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2551); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY Year of fee payment: 4 |