JP2014532330A - Strengthen video conferencing - Google Patents

Abstract

A computer-implemented method for enhancing a video conference between a first device and a second device. The method includes receiving a virtual object at a first device, the virtual object is configured to enhance a video conference, and the virtual object is specifically associated with an event. The method also includes incorporating the virtual object into the video conference.

Description

  This application is filed on February 11, 2011 under the name "AUGMENTING A VIDEO CONFERENCE" and is pending US Patent Application No. 13 / 025,943 (Attorney Docket No.) assigned to the assignee of the present invention. TNGO-008), a co-pending application, which is hereby incorporated by reference in its entirety.

  Video conference participants communicate with each other by transmitting audio / video signals to each other. For example, participants can interact simultaneously via interactive video and audio transmissions. However, participants may not be able to fully express that they are trying to communicate with each other based solely on the captured audio captured by the microphone and the video signal captured by the video camera.

  In summary, this book provides a computer-implemented method for enhancing a video conference between a first device and a second device. The method includes receiving a virtual object at a first device, the virtual object is configured to enhance a video conference, and the virtual object is specifically associated with an event. The method also includes incorporating the virtual object into the video conference.

1 illustrates an example apparatus according to an embodiment of the present invention. 1 illustrates an example apparatus according to an embodiment of the present invention. 1 illustrates an example apparatus according to an embodiment of the present invention.

3 illustrates an embodiment of a method for providing an enhanced video conference. 3 illustrates an embodiment of a method for providing an enhanced video conference.

3 illustrates an embodiment of a method for enhancing a video conference. 3 illustrates an embodiment of a method for enhancing a video conference. 3 illustrates an embodiment of a method for enhancing a video conference. 3 illustrates an embodiment of a method for enhancing a video conference.

  It should be understood that the drawings referred to in this description are not drawn to scale except where specifically noted.

  Reference will now be made in detail to embodiments of the present technology, examples of which are illustrated in the accompanying drawings. While the technology will be described in connection with various embodiments, it will be understood that they are not intended to limit the technology to those embodiments. On the contrary, the technology is intended to cover alternatives, modifications, and equivalents, which are included within the spirit and scope of the various embodiments, as defined by the appended claims. Can be.

  Furthermore, in the following description of embodiments, numerous specific details are set forth in order to provide a thorough understanding of the present technology. However, the technology may be practiced without those specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

  FIG. 1 depicts an embodiment of an apparatus 100. Device 100 is configured to participate in a video conference. FIG. 2 depicts devices 100 and 200 participating in a video conference. In general, video conferencing allows two or more locations to interact simultaneously via interactive video and audio transmissions.

  The following description first describes the components of the apparatus 100. The description next describes the functionality of the components of device 100 during a video conference between devices 100 and 200. Devices 100 and 200 are any communication devices (eg, laptops, desktops, etc.) that can participate in a video conference. In various embodiments, the device 100 is a portable mobile device such as a smartphone or a personal digital assistant (PDA).

  Further, for clarity and brevity, the description focuses on the components and functionality of the device 100. However, device 200 operates in a manner similar to device 100. In one embodiment, device 200 is identical to device 100 and includes the same components as device 100.

  The device 100 includes a display device 110, a virtual object receiving device 120, a virtual object embedding device 130, a transmitting device 140, a camera 150, a microphone 152, and a speaker 154. The apparatus 100 optionally includes a global positioning system 160 and a virtual object generator 170.

  Display device 110 is configured to display the video captured by device 200. In another embodiment, display device 110 is further configured to display video captured on device 100.

  The virtual object receiver 120 is configured to access a virtual object. Virtual objects are configured to enhance video conferencing, which will be described in detail below.

  The virtual object embedding device 130 is configured to incorporate the virtual object into the video conference. For example, virtual object embedding device 130 is configured to embed a virtual object in the video captured by device 100 and / or device 200.

  The transmission device 140 is for transmitting data (for example, virtual object control code).

  The virtual object operation device 135 is configured to be able to operate a virtual object in a video conference.

  The camera 150 is for capturing an image in the apparatus 100. The microphone 152 is for capturing sound in the apparatus 100. The speaker 154 is for generating an audible signal in the apparatus 100.

  The global positioning system 160 is for determining the location of the device 100.

  The virtual object generation device 170 is for generating a virtual object.

  Referring now to FIG. 2, devices 100 and 200 are participating in a video conference with each other. In various embodiments, two or more devices participate in a video conference with each other.

  During the video conference, the video camera 250 captures the video on the device 200. For example, the video camera 250 captures an image of the user 205 of the device 200.

  The video camera 150 captures an image on the device 100. For example, the video camera 150 captures the video of the user 105. It should be appreciated that video cameras 150 and 250 capture any object within the respective field of view of cameras 150 and 250.

  The microphone 152 captures an audio signal corresponding to the video signal captured by the apparatus 100. Similarly, the microphone of the device 200 captures an audio signal corresponding to the video signal captured by the device 200.

  The video captured by the device 200 is transmitted to the display device 110 of the device 100 and displayed. For example, the video of the user 205 is displayed on the first view 112 of the display device 110. Further, the video of the user 205 is displayed on the second view 214 of the display device 210.

  The video captured by the device 100 is transmitted to the display device 210 of the device 200 and displayed. For example, the video of the user 105 is displayed on the first view 212 of the display device 210. Further, the video of the user 105 is displayed on the second view 114 of the display device 110.

  In one embodiment, the audio signals captured by devices 100 and 200 are incorporated into the captured video. In another embodiment, the audio signal is transmitted separately from the transmitted video.

  As depicted, the first view 112 is a primary view displayed on the display device 110 and the second view 114 is a smaller secondary view displayed on the display device 110. In various embodiments, the size of the first view 112 and the second view 114 are both adjustable. For example, the second view 114 can be enlarged to a primary view, and the size of the view 112 can be reduced to a secondary view. Further, any one of the views 112 and 114 can be closed or completely reduced to be invisible.

  The virtual object receiving device 120 receives a virtual object 190 for enhancing a video conference. The virtual object can be received from the server or device 200. Virtual objects can be received at different times. For example, the virtual object can be received when the enhanced application is downloaded to the device 100, during login, or in real time, at which time the virtual object is ordered to be incorporated into the video conference.

  The virtual objects 191 (eg, stars, palm trees, flowers, rain clouds) depicted in FIGS. 2 and 6 are just a few of the many examples of virtual objects. It should be appreciated that a virtual object can be any object that can enhance a video conference. In other words, a virtual object can be any object that can complement communication between participants in a video conference. For example, a virtual object may include, but is not limited to, kissing, heart, emoji, high touch, background (pricra-like effect), color space change, and / or image processing change (eg, lean, fat) possible.

  It should also be appreciated that virtual objects are not limited to visible virtual objects. For example, the virtual object can be one of a plurality of sounds.

  In one embodiment, virtual object 191 is displayed on display device 110 for viewing by user 105. For example, the virtual object 191 is displayed on the virtual object bar 192. In one embodiment, the virtual object bar 192 is overlaid on the first view 112. In another embodiment, the virtual object bar 192 is displayed simultaneously with the first view 112 and / or the second view 114.

  In various embodiments, the virtual object bar 192 includes, but is not limited to, keystrokes, cursor movements, detected touches on the touch screen, and user specified actions (eg, facial expressions, winks, throw kisses). , Hand gestures, etc.) in response to user input.

  The virtual object embedding device 130 makes it easy to incorporate the virtual object 190 into the video conference. In one embodiment, in the device 100, the virtual object embedding device 130 incorporates the virtual object 190 into the video captured by the device 200. For example, the virtual object 190 is incorporated on the user 205's head. Therefore, as depicted, the video captured at device 200 incorporates object 190 and the enhanced video is displayed at least at device 200. Also, the enhanced video with the embedded virtual object 190 is displayed on the device 100.

  In one embodiment, the user 105 selects the virtual object 190 in the virtual object bar 192 and places the user 205 at the location specified by the user 105 (eg, as displayed in the first view 112). Drag and place the virtual object 190 up). When placed at the designated location, the virtual object embedding device 130 incorporates the virtual object at the designated location.

  In another embodiment, virtual object embedding device 130 generates control code. This control code commands how the virtual object 190 is embedded in the video captured by the device 200.

  For example, it is possible to send a control code directly to the device 200 and to instruct the device 200 how to incorporate the virtual object 190 in the video displayed on the device 200. In such an example, the control code signals or commands device 200 that virtual object 190 is to be displayed in the video conference. In another example, the control code is transmitted to the server, and then the device 200 receives the control code from the server.

  FIG. 2 depicts a virtual object 190 embedded in a video conference. However, it should be appreciated that any number of virtual objects can be incorporated into a video conference at any point in time. For example, five different virtual objects can be simultaneously incorporated into a video conference.

  It should be appreciated that the term “embed” as used herein is used to describe that a virtual object is simply displayed with some part of a video conference. As such, the virtual object is simply displayed simultaneously with some portion of the video conference. Thus, it should be understood that the virtual object is embedded in the video and includes the virtual object. However, it should not be understood that virtual objects are integrated into or become part of a video stream.

  In one embodiment, the virtual object is overlaid on the video as an overlay. Thus, the virtual object is superimposed simultaneously as an overlay displayed on the devices 100 and 200. For example, the virtual object is overlaid simultaneously on the video displayed in view 112 and view 214 (as depicted in FIG. 2), and the virtual object is displayed on the video displayed in view 114 and view 212 (FIG. 6). Can be overlaid simultaneously).

  In another embodiment, virtual objects are integrated into a videoconference bitstream.

  In another example, the virtual object is overlaid simultaneously on the video displayed in view 112 and view 212. Also, the virtual object is displayed on a portion of the display device independent of the view in the device, and bi-directional video does not need to be active (eg, unidirectional video can be active).

  It should be noted that the various embodiments described herein can also be used in combination with each other. That is, one described embodiment can be used in combination with one or more other described embodiments.

  In one embodiment, the sending device 140 then sends the video captured at the device 200, which already contains the virtual object 190, to the second device 200, so that the video containing the virtual object 190 is on the display device 210. Is displayed. In another embodiment, the sending device 140 sends a control code to the device 200 (or server), instructing the device 200 how the virtual object 190 is incorporated into the video conference.

  The virtual object operation device 135 operates the embedded virtual object 190. In one embodiment, the virtual object 190 is manipulated on the device 100. For example, in response to a user input on the touch screen, the user 105 rotates the virtual object 190 clockwise. As a result, the video captured in device 200 (and displayed on device 100 and / or device 200) is enhanced and the virtual object rotates clockwise.

  In another embodiment, virtual object 190 is manipulated on device 200. For example, in response to the user 205 moving his head from left to right, the virtual object 190 is manipulated (via the virtual object manipulation device of the device 200) and thus from the left as the user 205 moves his head. Move to the right. Accordingly, the video captured at device 200 (and displayed on device 100 and / or device 200) is enhanced and virtual object 190 is moved from left to right.

  In a further embodiment, the virtual object 190 is operated on the device 100 and the device 200 simultaneously. For example, in response to user 205 moving his head from left to right and user 105 rotating the virtual object 190 (as described above), the virtual object 190 moves to the left as the user 205's head moves. To the right and is operated to rotate simultaneously in response to input from the user 105. Thus, the video captured at device 200 (and displayed on device 100 and / or 200) is enhanced and virtual object 190 is moved from left to right while rotating clockwise.

  In a further embodiment, the virtual object 190 is manipulated directional. For example, user 105 sends a “punch” virtual object (eg, fist, boxing glove) to user 205. In response, user 105 sees a “punch” virtual object entering display device 110, and user 205 sees a “punch” virtual object coming out of display device 210.

  It should be appreciated that virtual objects are manipulated in response to various inputs. For example, the virtual object can be manipulated through sound, gestures, facial expressions, movements, and the like. As various examples, a virtual object (eg, a star) comes out of the user's eyes in response to the user's wink, and a red lip pops out of the user's mouth in response to the user's kiss.

  In one embodiment, as described above, the virtual object 191 is not displayed on the display device 110 and / or the virtual display bar 192 until at least one of various inputs occurs. For example, the virtual object of the heart is not displayed until there are two taps on the touch screen.

  Any number of virtual objects can be accessed and / or selected for incorporation into a video conference. In one embodiment, virtual object 191 is a geo-related virtual object. For example, the virtual object 191 is based on the location of the device 100 and / or 200.

  Specifically, if the device 100 is located in Hawaii, the virtual object 191 is associated with that location. For example, a virtual object related to geography may be, but is not limited to, a surfboard, sun, palm tree, coconut, etc. based on a location in Hawaii determined from GPS 160.

  It should be appreciated that location determination can be provided by various methods. For example, determining the location of a device can be based on information provided by the user at registration, the IP address of the device, or any other method that can be used to determine the location.

  In another embodiment, the virtual object 191 is a time-related virtual object based on the video conference time. For example, if a video conference occurs on or around Christmas day, the virtual object will be related to Christmas (eg, socks, Christmas tree, candy cane, etc.). In another example, if the video conference takes place at night, the virtual object will be associated with the night (eg, moon, stars, pajamas, etc.).

  In a further embodiment, the virtual object 191 is a culture related virtual object. For example, if the users 105 and / or 205 are located in Canada, the virtual object 191 can be, but is not limited to, a Canadian flag, a hockey pack, a curling stone, and the like.

  In another embodiment, virtual object 191 is a user created virtual object. For example, the users 105 and / or 205 manually create a virtual object, and then the virtual object generation device 170 uses the creation to generate a user-generated virtual object.

  In yet another embodiment, the virtual object 191 is available and / or accessed based on the account status. For example, the user 105 has an account to pay for accessing the virtual object 191. If the user 105 provides sufficient payment for the account, the user 105 can access the virtual object 191. Conversely, if the user does not provide sufficient payment for the account, the user 105 cannot access the virtual object 191.

  Furthermore, the use and selection of virtual objects can be specifically related to events, holidays, special occasions, and so on. Holidays include, but are not limited to, religious holidays (eg, Christmas, Easter, Atonement Day, etc.), national holidays (eg, New Year, Washington President ’s Birthday, Memorial Day for Memorial of the War, etc.) or any other It may be a transfer holiday (official or informal). Events or special occasions can be, but are not limited to, birthdays, anniversaries, graduations, weddings, employment, retirement, etc.

  For example, on or before or after Thanksgiving, virtual objects such as turkeys, pumpkin pie, pilgrims, etc. are selected and / or used. In another example, clovers, gold pots, and fairy virtual objects are selected and / or used on or around St. Patrick's Day. In a further example, the Easter Rabbit and Easter Egg virtual objects are selected and / or used on or around Easter Day.

  In one embodiment, the user is prompted to utilize virtual objects that are specifically associated with events, holidays, special occasions, and the like. For example, on or around US Independence Day, the user is prompted to select and / or use a virtual object (eg, fireworks) that is specifically associated with US Independence Day. Specifically, the user is provided with a virtual object and the user is prompted to send the virtual object to another user in the video conference. In other words, the virtual object is incorporated into the video conference.

  In another embodiment, the user may be prompted to send a virtual object to the other user if the relationship between the parties is suspicious, known, or inferred. For example, a mother is talking to her son through a video conference. If the mother / son relationship is suspicious, known, or inferred, the son is prompted to use a virtual object (eg, a flower) that is specifically associated with Mother's Day.

  The relationship can be determined in various ways. For example, but not limited to these, the relationship can be determined based on the last name, the location of the user, the call record, and the like.

  In addition, the son may be prompted with a message such as "This seems to be your mother. Is it right?" In such a case, if the son responds that he is talking to his mother, the son is prompted to use a virtual object (eg, a flower) that is specifically associated with Mother's Day.

  It should also be appreciated that virtual objects can enhance revenue streams. For example, 100,000 virtual objects are used for Valentine's Day, and the fee for each virtual object is $ 0.50. As a result, a charge of $ 50,000 is accumulated on Valentine's Day.

  3-5 depict embodiments of methods 300-500, respectively. In various embodiments, the methods 300-500 are performed by processing devices and electrical components under the control of computer readable and computer executable instructions. Computer-readable and computer-executable instructions reside, for example, in data storage media such as volatile and non-volatile memory usable by the computer. However, computer readable and computer executable instructions may reside in any type of computer readable storage medium. In some embodiments, methods 300-500 are performed by apparatus 100 and / or apparatus 200 as described in FIGS.

  Referring now to FIG. 3, at 310 of the method 300, the virtual object is made accessible by the first device, the first device configured to participate in a video conference with the second device. Yes. For example, the virtual object 190 is made accessible by the device 100 and the device 100 is configured to participate in at least a video conference with the device 200.

  At 320, the virtual object can be embedded in a video conference video captured at the second device, and the video including the virtual object is configured to be displayed at the second device. For example, the virtual object 190 can be incorporated into the video captured by the user 205 on the device 200 and displayed on the device 200.

  At 330, the video including the virtual object can be transmitted from the first device to the second device. For example, transmission of a video including any one of the virtual objects 191 can be transmitted to the device 200 by the transmission device 140.

  At 340, simultaneous display of video including virtual objects is enabled on the first device and the second device. For example, an image including the object 190 can be displayed simultaneously on the devices 100 and 200.

  At 350, collaborative manipulation of embedded virtual objects in the first device and the second device is enabled. For example, the user 205 interacts with the virtual object 190 in the second view 214 and the user 105 interacts cooperatively with the virtual object in the first view 112.

  Referring now to FIG. 4, at 410 of method 400, instructions for accessing a virtual object are received. For example, instructions are received to access the virtual object 190 in response to user input (eg, key input, cursor movement, detected touch on the touch screen, etc.). In various embodiments, the virtual object (s) can be, but is not limited to, a geographic related virtual object, a time related virtual object, a culture related virtual object, and / or a user created virtual object.

  At 420, the virtual object is incorporated into a video conference, and the virtual object is configured to be accessed by the first device and displayed on the second device. For example, the virtual object 190 is accessed at the device 100 and embedded in the video captured at the device 200 at the device 100. A video including the embedded virtual object 190 is configured to be displayed on the device 200.

  In another example, the user 105 places a lip virtual object (meaning kiss) on the user's 205 cheek by specifying the location of the lips on the user's 105 cheek in the first view 112. be able to. Thus, the lip virtual object is incorporated into the video captured by the device 200 and displayed on the devices 100 and 200. The lip virtual object may be incorporated during the duration of the video conference or may be incorporated for a specified duration.

  In one embodiment, at 422, the virtual object is incorporated into the video conference in response to user input on the touch screen display. For example, in response to user input on the touch screen display device of device 100, the virtual object is incorporated into the video conference.

  At 430, video of the video conference including the embedded virtual object is transmitted to the second device. For example, the video including the virtual object is transmitted to the device 200 via the transmission device 140.

  At 440, the video conference video captured at the second device is displayed at the first device. For example, an image of the user 205 in the device 200 is captured by the device 200 and displayed on the device 100.

  At 450, the virtual object embedded in the video conference is manipulated on the second device. For example, the user 205 interacts with the virtual object 190 displayed in the second view 214 by rotating the virtual object 190.

  At 460, the virtual object embedded in the video conference is manipulated on the first device. For example, the user 105 interacts with the virtual object 190 displayed in the first view 112 by reducing the size of the virtual object 190.

  In one embodiment, at 465, a virtual object embedded in a video conference is manipulated in response to user input received at the touch screen display device of the portable device. For example, the device 100 is a mobile device (for example, a mobile phone) provided with a touch screen display device. Accordingly, in response to the user 105 touching the touch screen display device, the size of the virtual object 190 is reduced.

  At 470, the virtual object incorporated into the video conference is manipulated cooperatively on the first device and the second device. For example, the user 205 moves his head from left to right so that the virtual object 190 tracks the movement of the head. Also, the user 105 cooperatively rotates the virtual object 190 while the virtual object 190 tracks the movement of the user 205's head.

  At 480, the video conference video and virtual object captured on the second device are simultaneously displayed on the first device. For example, the video captured in the second device 200 including the embedded virtual object 190 is simultaneously displayed in the first view 112.

  At 490, the first video captured at the first device and the second video captured at the second device are simultaneously displayed on the first device. For example, the video captured by the device 200 is displayed in the first view 112 and the video captured by the device 100 is displayed in the second view 114 at the same time.

  Referring now to FIG. 5, at 510 of method 500, the video captured at the second device is displayed on the first device.

  At 515, the virtual object is received at the first device and the virtual object is configured to enhance the video conference. In various embodiments, the virtual object (s) can be, but is not limited to, a geographic related virtual object, a time related virtual object, a culture related virtual object, and / or a user generated virtual object.

  At 520, the virtual object is incorporated into the video captured at the second device. For example, the virtual object 190 is embedded in the video captured by the device 200, and the virtual object 190 is placed on the head of the user 205 and tracks the movement of the user 205 's head.

  In one embodiment, at 522, in response to user input on the touch screen display device, the virtual object is incorporated into the video captured by the second device. For example, in response to user 105 input on the touch screen display device of device 100, any number of virtual objects are incorporated into the video captured by device 200.

  In 530, the video including the virtual object can be displayed on the second device. At 535, the video including the virtual object is transmitted to the second device.

  At 540, the virtual object that is embedded in the video captured at the second device is manipulated at the second device. For example, the user 205 changes the color of the virtual device 190 displayed in the second view 214 to red.

  At 545, the virtual object that is embedded in the video captured at the second device is manipulated at the first device. For example, the user 105 changes the location of the virtual device 190 from the top of the head of the user 205 to the left hand of the user 205.

  In one embodiment, at 547, the virtual object embedded in the video captured at the second device is manipulated in response to user input received at the touch screen display device of the portable mobile device. . For example, in response to user input on the touch screen display device of device 100, user 105 changes virtual device 190 from a star (as depicted) to a light bulb (not shown).

  At 550, the virtual object embedded in the video captured at the second device is manipulated cooperatively at the first device and the second device. For example, the user 205 operates the virtual object 190 in the second view 214, and the user 105 operates the virtual object in the first view 112 cooperatively.

  At 555, the virtual object and the video captured on the second device are simultaneously displayed on the first device. At 560, the video captured at the first device and the video captured at the second device are simultaneously displayed on the first device.

  FIG. 6 depicts an embodiment of devices 100 and 200 participating in a video conference with each other. Devices 100 and 200 operate in a similar manner as described above.

  During the video conference, the video camera 150 captures video on the device 100. For example, the video camera 150 captures an image of the user 105 of the device 100.

  Video camera 250 captures video in device 200. For example, video camera 250 captures video of user 205, who is the user of device 200.

  The video captured by the device 100 is displayed on the display device 110 of the device 100. For example, the video of the user 105 is displayed in the second view 114 displayed on the display device 110. Further, the video of the user 205 is displayed on the first view 112 on the display device 110.

  The virtual object receiver 120 receives the virtual object 190 to enhance the video conference between the users 105 and 205 participating in the video conference.

  The virtual object 191 is displayed on the display device 110 for viewing by the user 105. For example, the virtual object 191 is displayed on the virtual object bar 192. In one embodiment, the virtual object bar 192 is overlaid with the first view 112. In another embodiment, the virtual object bar 192 is displayed simultaneously with the first view 112 and / or the second view 114.

  The virtual object embedding device 130 incorporates the virtual object 190 into the video conference. Specifically, in the device 100, the virtual object 190 is incorporated in the video captured by the device 100. For example, the virtual object 190 is incorporated above the user 105. Therefore, as depicted, the video captured by device 100 incorporates object 190 and the enhanced video is displayed at least on device 200. Also, the enhanced video incorporating the virtual object 190 is displayed simultaneously on the device 100.

  In one embodiment, the user 105 selects the virtual object 190 in the virtual object bar 190 and drags the virtual object 190 to a location specified by the user 105 (eg, overhead of the user 105 as depicted). And place. When placed at the designated location, the virtual object embedding device 130 incorporates the virtual object at the designated location.

  Next, the transmission device 140 transmits the video captured by the device 100 to the second device 200, which currently includes the virtual object 190, and the video including the virtual object 190 is displayed on the display device 210.

  The virtual object operation device of the device 200 operates the embedded virtual object 190. For example, in response to a user input from the user 205 on the touch screen, the user 205 rotates the virtual object 190 clockwise. Accordingly, the video captured by device 100 (displayed on device 200 and / or device 100) is enhanced and the virtual object rotates clockwise.

  In another embodiment, virtual object 190 is manipulated on device 100. For example, in response to the user 105 moving his head from left to right, the virtual object 190 is manipulated (via the virtual object manipulator 135) and moves from left to right as the user 105 moves. Thus, the image captured at device 100 (displayed on device 100 and / or device 200) is enhanced and virtual object 190 is moved from left to right.

  In a further embodiment, the virtual object 190 is operated on the device 100 and the device 200 simultaneously. For example, in response to user 105 moving his head from left to right and user 205 rotating virtual object 190, virtual object 190 is manipulated and simultaneously from left to right as user 105 moves his head. Move and rotate in response to input from user 205. Thus, the image captured on device 100 (displayed on device 100 and / or device 200) is enhanced and virtual object 190 is moved from left to right while rotating clockwise.

  FIGS. 7-9 depict embodiments of methods 700-900, respectively. In various embodiments, the methods 700-900 are performed by a processing device and electrical components under the control of computer readable and computer executable instructions. Computer-readable and computer-executable instructions reside, for example, in data storage media such as volatile and non-volatile memory usable by the computer. However, computer readable and computer executable instructions may reside in any type of computer readable storage medium. In some embodiments, methods 700-900 are performed by apparatus 100 and / or apparatus 200 as described in FIGS.

  Referring now to FIG. 7, at 710 of method 300, the virtual object is made accessible by the first device, and the first device is configured to participate in the video conference simultaneously with the second device. For example, virtual object 190 is made accessible by device 100, and device 100 is configured to participate in a video conference at least simultaneously with device 200.

  At 720, the virtual object can be incorporated into a video conference video captured at the first device, and the video including the virtual object is configured to be displayed at the second device. For example, the virtual object 190 can be embedded in the video captured by the user's 105 device 100 and displayed on the devices 100 and 200.

  At 730, the video including the virtual object can be transmitted from the first device to the second device. For example, transmission of a video including any one of the virtual objects 191 can be transmitted to the device 200 by the transmission device 140.

  At 740, simultaneous display of video including the virtual object is enabled on the first device and the second device. For example, an image including the object 190 can be displayed simultaneously on the devices 100 and 200.

  At 750, cooperative operation of the embedded virtual object on the first device and the second device is enabled. For example, the user 205 interacts with the virtual object 190 in the first view 212 and the user 105 also interacts cooperatively (or simultaneously) with the virtual object in the second view 114.

  Referring now to FIG. 8, at 810 of method 800, an instruction is received to access a virtual object. For example, a command is received to access the virtual object 190 in response to user input on the touch screen display device.

  At 820, the virtual object is incorporated into the video conference and the virtual object is manipulated by the user of the second device. For example, the virtual object 190 is accessed at the device 100 and embedded in the video captured by the device 100 at the device 100. Video including the embedded virtual object 190 is configured to be displayed and manipulated on the device 200. In one embodiment, at 822, in response to user input on the touch screen display device, the virtual object is incorporated into the video conference.

  At 830, the video conference video including the embedded virtual object is transmitted to the second device.

  At 840, the video conference captured at the first device is displayed at the second device. For example, an image of the user 105 in the device 100 is captured in the device 100 and displayed on the device 200.

  At 850, the virtual object that is to be incorporated into the video conference is manipulated on the second device. For example, the user 205 interacts with the virtual object 190 displayed in the first view 212 by rotating the virtual object 190.

  In one embodiment, at 855, in response to user input received at the touch screen display device, the virtual object embedded in the video conference is manipulated at the portable mobile device. For example, the device 200 is a portable device (eg, a mobile phone) that includes a touch screen display device. Accordingly, in response to the user 205 touching the touch screen display device, the size of the virtual object 190 is reduced.

  At 860, the virtual object embedded in the video conference is manipulated on the first device. For example, the user 105 interacts with the virtual object 190 displayed in the second view 114 by reducing the size of the virtual object 190.

  At 870, the virtual object incorporated into the video conference is manipulated cooperatively on the first device and the second device. For example, the user 105 moves his head from left to right, and the virtual object 190 tracks the movement of the head. In addition, the user 205 cooperatively rotates the virtual object 190 while the virtual object 190 is tracking the movement of the head of the user 105.

  At 880, the video conference video and virtual object captured at the second device are simultaneously displayed at the first device. For example, the video captured on the second device 200 is displayed on the first view 112, and the video captured on the device 100 including the embedded virtual object 190 is displayed on the second view 114 at the same time. Is done.

  At 890, the first video captured at the first device and the second video captured at the second device are simultaneously displayed on the first device.

  Referring now to FIG. 9, in method 910, video captured at the first device is displayed on the first device.

  At 915, a virtual object is received at the first device, and the virtual object is configured to enhance a video conference. In various embodiments, the virtual object (s) can be, but is not limited to, a geographic related virtual object, a time related virtual object, a culture related virtual object, and / or a user created virtual object.

  At 920, the virtual object is embedded in the video captured at the first device. For example, the virtual object 190 is embedded in the video captured by the device 100, the virtual object 190 is placed over the user 105 's head and tracks the movement of the user 105' s head.

  In one embodiment, at 922, in response to user input at the touch screen display device, the virtual object is incorporated into the video captured at the device. For example, in response to user 105 input on the touch screen display device of device 100, any number of virtual objects are incorporated into the video captured by device 100.

  At 930, the video including the virtual object can be displayed on the second device, and the virtual object is operated on the second device. At 935, the video including the virtual object is transmitted to the second device.

  At 940, the virtual object that is embedded in the video captured at the first device is manipulated at the second device. For example, the user 205 changes the color of the virtual device 190 displayed in the first view 212 to red.

  In one embodiment, at 942, a virtual object embedded in a video captured at the first device is manipulated in response to user input received at the touch screen display device of the portable mobile device. For example, in response to user input on the touch screen display device of device 200, user 205 changes virtual device 190 from a star (as depicted) to a light bulb (not shown).

  At 945, the virtual object embedded in the video captured at the first device is manipulated at the first device. For example, the user 105 changes the location of the virtual device 190 from the top of the head of the user 105 to the left hand of the user 105.

  At 950, the virtual object that is embedded in the video captured at the first device is operated cooperatively at the first device and the second device. For example, the user 205 operates the virtual object 190 in the first view 212, and the user 105 operates the virtual object in the second view 114 cooperatively.

  At 955, the virtual object and the video captured by the first device are simultaneously displayed on the first device. At 560, the video captured at the first device and the video captured at the second device are simultaneously displayed on the first device.

  Various embodiments of the invention are thus described. While the invention has been described in certain embodiments, it is to be understood that the invention is not to be construed as limited by such embodiments, but is to be construed according to the following claims. I want.

  All elements, portions, and steps described herein are included as desired. It will be apparent to those skilled in the art that any of these elements, portions, and steps can be replaced by other elements, portions, and steps, or can be completely deleted.

  Those skilled in the art will appreciate that the method steps described in this description include, but are not limited to, a processing device, at least an input device comprising a keyboard, mouse, scanner, camera, at least a monitoring device and a printing press. It can be executed by hardware including an output device. Method steps are carried out by suitable devices when necessary. For example, the decision step can be performed by a decision enforcement unit in the processing device by implementing a decision algorithm. Those skilled in the art will appreciate that this decision execution unit can be physically or effectively in a computer processing unit, for example, when executing the decision algorithm. The above analysis applies to the other steps described herein.

Concepts This book also discloses at least the following concepts.
Concept 1. A computer-implemented method for enhancing a video conference between a first device and a second device, comprising:
Receiving, in the first device, a virtual object that is configured to enhance the video conference and is specifically associated with an event;
Incorporating the virtual object into the video conference;
A computer-implemented method comprising:
Concept 2. The computer-implemented method of Concept 1, wherein the event is selected from the group consisting of holidays and special occasions.
Concept 3. The computer-implemented method of Concept 1 or 2, further comprising prompting a user of the first device to incorporate the virtual object into the video conference.
Concept 4 The prompt for the user of the first device to incorporate the virtual object into the video conference is:
4. The computer-implemented method of Concept 3, further comprising prompting the user of the first device to incorporate the virtual object into the video conference during the day that the event occurs.
Concept 5. The computer-implemented method of any one of Concepts 1-4, further comprising determining a possible relationship between a user of the first device and a user of the second device.
Concept 6. The computer-implemented method of Concept 5, further comprising prompting a user of the first device to confirm the determined possible relationship.
Concept 7: Encourage the user of the first device to incorporate the virtual object into the video conference based on the relationship between the first user and the second user of the second device. The computer-implemented method according to any one of concepts 1 to 6, further including:
Concept 8 The computer-implemented method according to any one of Concepts 1 to 7, further comprising: operating the virtual object embedded in the video conference in the second device.
Concept 9: The computer-implemented method according to any one of Concepts 1-8, further comprising operating the virtual object embedded in the video conference in the first device.
Concept 10 When executed on a computer processing device,
Receiving, in the first device, a virtual object that is configured to enhance the video conference and is specifically associated with an event;
Incorporating the virtual object into the video conference;
A tangible computer readable storage medium having stored instructions thereon that cause a method comprising:
Concept 11. The tangible computer readable storage medium of Concept 10, wherein the event is selected from the group consisting of holidays and special occasions.
Concept 12. The tangible computer readable storage medium of Concept 10 or 11, further comprising instructions for prompting a user of the first device to incorporate the virtual object into the video conference.
Concept 13 The prompt to the user of the first device to incorporate the virtual object into the videoconference is
13. The tangible computer readable storage medium of concept 12, further comprising prompting the user of the first device to incorporate the virtual object into the video conference during the day that the event occurs.
Concept 14. The tangible computer readable storage medium of Concept 12, further comprising instructions for determining a possible relationship between a user of the first device and a user of the second device.
Concept 15. The tangible computer readable storage medium of Concept 14, further comprising instructions for prompting a user of the first device to confirm the determined possible relationship.
Concept 16 for prompting a user of the first device to incorporate the virtual object into the video conference based on a relationship between the first user and a second user of the second device. A tangible computer readable storage medium according to any one of concepts 10 to 15, further comprising instructions.
Concept 17. The tangible computer readable storage medium of any one of concepts 10 to 16, further comprising instructions for manipulating the virtual object embedded in the video conference in the second device.
Concept 18. The tangible computer readable storage medium of any one of concepts 10-17, further comprising instructions for manipulating the virtual object embedded in the video conference in the first device.

Claims (18)

A computer-implemented method for enhancing a video conference between a first device and a second device, comprising:
Receiving, in the first device, a virtual object that is configured to enhance the video conference and is specifically associated with an event;
Incorporating the virtual object into the video conference;
A computer-implemented method comprising:   The computer-implemented method of claim 1, wherein the event is selected from the group consisting of holidays and special occasions.   The computer-implemented method of claim 1, further comprising prompting a user of the first device to incorporate the virtual object into the video conference. Prompting the user of the first device to incorporate the virtual object into the video conference;
4. The computer-implemented method of claim 3, further comprising urging the user of the first device to incorporate the virtual object into the video conference during the day that the event occurs.   The computer-implemented method of claim 1, further comprising determining a possible relationship between a user of the first device and a user of the second device.   6. The computer-implemented method of claim 5, further comprising prompting a user of the first device to confirm the determined possible relationship.   Further urging the user of the first device to incorporate the virtual object into the video conference based on the relationship between the first user and the second user of the second device. The computer-implemented method of claim 1, comprising:   The computer-implemented method according to claim 1, further comprising: operating the virtual object embedded in the video conference in the second device.   The computer-implemented method according to claim 1, further comprising: operating the virtual object embedded in the video conference in the first device. When executed, against the computer processing device
Receiving, in the first device, a virtual object that is configured to enhance the video conference and is specifically associated with an event;
Incorporating the virtual object into the video conference;
A tangible computer readable storage medium having stored instructions thereon that cause a method comprising:   The tangible computer readable storage medium of claim 10, wherein the event is selected from the group consisting of holidays and special occasions.   The tangible computer readable storage medium of claim 10, further comprising instructions for prompting a user of the first device to incorporate the virtual object into the video conference. Prompting the user of the first device to incorporate the virtual object into the video conference;
13. The tangible computer readable storage medium of claim 12, further comprising prompting the user of the first device to incorporate the virtual object into the video conference during the day that the event occurs.   The tangible computer readable storage medium of claim 12, further comprising instructions for determining a possible relationship between a user of the first device and a user of the second device.   15. The tangible computer readable storage medium of claim 14, further comprising instructions for prompting a user of the first device to confirm the determined possible relationship.   Instructions to prompt the user of the first device to incorporate the virtual object into the video conference based on the relationship between the first user and the second user of the second device The tangible computer readable storage medium of claim 10, further comprising:   The tangible computer readable storage medium of claim 10, further comprising instructions for manipulating the virtual object embedded in the video conference in the second device.   The tangible computer readable storage medium of claim 10, further comprising instructions for manipulating the virtual object embedded in the video conference in the first device.

Images