WO2012007034A1 - Envoi et réception d'informations - Google Patents

Envoi et réception d'informations Download PDF

Info

Publication number
WO2012007034A1
WO2012007034A1 PCT/EP2010/060081 EP2010060081W WO2012007034A1 WO 2012007034 A1 WO2012007034 A1 WO 2012007034A1 EP 2010060081 W EP2010060081 W EP 2010060081W WO 2012007034 A1 WO2012007034 A1 WO 2012007034A1
Authority
WO
WIPO (PCT)
Prior art keywords
region
sub
display
causing
user
Prior art date
Application number
PCT/EP2010/060081
Other languages
English (en)
Inventor
Qifen Yan
Antti Oskari Salo
Teppo Tapani Jokinen
Jarkko Tapio SAUNAMÄKI
Piers Andrew
Lotta Maria Partanen
Zoran Radivojevic
Original Assignee
Nokia Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nokia Corporation filed Critical Nokia Corporation
Priority to PCT/EP2010/060081 priority Critical patent/WO2012007034A1/fr
Publication of WO2012007034A1 publication Critical patent/WO2012007034A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04886—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/1066—Session management
    • H04L65/1083—In-session procedures
    • H04L65/1094—Inter-user-equipment sessions transfer or sharing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/1066—Session management
    • H04L65/1096—Supplementary features, e.g. call forwarding or call holding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/75—Indicating network or usage conditions on the user display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
    • H04N7/142—Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
    • H04N7/144—Constructional details of the terminal equipment, e.g. arrangements of the camera and the display camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
    • H04N7/147—Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/15—Conference systems
    • H04N7/157—Conference systems defining a virtual conference space and using avatars or agents

Definitions

  • This invention relates to sending and receiving information.
  • Modern smartphones such as the N97 from Nokia Corporation are configured for sending and receiving video information so as to allow users at different locations to communicate face-to-face in real time.
  • a first aspect of the invention provides a method comprising causing a first device to send information to and receive information from a second device, causing the first device to display graphical information representing a user of the first device or a user of the second device in a region of a display, causing the first device to define at least one sub-region of said region of the display, each of the at least one sub- region displaying a part of said graphical information and causing the first device to be responsive to a touch user input in one of said at least one sub-region to cause at least one operation to be performed by the first device and/ or the second device.
  • a second aspect of the invention provides an apparatus comprising at least one processor and at least one memory having stored therein computer program code, wherein the computer program code is configured, when executed by the at least one processor using the at least one memory, to cause the apparatus to perform a method comprising causing a first device to send information to and receive information from a second device, causing the first device to display graphical information representing a user of the first device or a user of the second device in a region of a display, causing the first device to define at least one sub-region of said region of the display, each of the at least one sub-region displaying a part of said graphical information, and causing the first device to be responsive to a touch user input in one of said at least one sub-region to cause at least one operation to be performed by the first device and/ or the second device.
  • a third aspect of the invention provides an apparatus comprising means for causing a first device to send information to and receive information from a second device, means for causing the first device to display graphical information representing a user of the first device or a user of the second device in a region of a display, means for causing the first device to define at least one sub-region of said region of the display, each of the at least one sub-region displaying a part of said graphical information and means for causing the first device to be responsive to a touch user input in one of said at least one sub-region to cause at least one operation to be performed by the first device and/ or the second device.
  • a fourth aspect of the invention provides a non-transitory computer-readable storage medium having stored thereon computer-readable code, which, when executed by computing apparatus, causes the computing apparatus to cause a first device to send information to and receive information from a second device, cause the first device to display graphical information representing a user of the first device or a user of the second device in a region of a display, cause the first device to define at least one sub-region of said region of the display, each of the at least one sub-region displaying a part of said graphical information and cause the first device to be responsive to a touch user input in one of said at least one sub-region to cause at least one operation to be performed by the first device and/ or the second device.
  • Figure 1 shows a device according to aspects of the invention
  • FIG. 2 and Figure 3 illustrate some aspects of the hardware and software
  • Figure 4 is a block diagram illustrating a configuration of communication software of the device of Figure 1.
  • Figure 5 shows the device of Figure 1 in communication with a remote device.
  • Figure 6 to 9 show display outputs of the device of Figure 1 and the remote device during a video call;
  • Figure 10 is a flow chart illustrating operation of the Figure 1 device.
  • Figure 11 is a block diagram illustrating an alternative configuration of the device communication software.
  • the device 100 embodies aspects of the invention and is not limiting on the scope of the invention.
  • the device 100 has a touch sensitive display, or touchscreen 102 for displaying content and for receiving tactile user inputs comprising one or more touch inputs.
  • the touchscreen 102 may be a resistive or a capacitive touchscreen, an optical touch touchscreen, an infra-red touchscreen or a surface acoustic wave touchscreen.
  • the device may also comprise one or more physical keys 104 for receiving inputs from a user.
  • the device 100 also has a front-facing video camera 106 configured for electronic motion picture acquisition for use in video calling and other applications.
  • the device 100 may be a mobile computer, mobile phone, PDA, internet tablet, smartphone or other portable communications device.
  • FIG. 2 is a block diagram illustrating some aspects of the hardware and software configuration of the device 100.
  • the device 100 includes a controller 200 comprising one or more processors.
  • the controller 200 communicates with other hardware components via a system bus 201.
  • Each other hardware component is connected to the system bus 201 either directly or via an interface.
  • the hardware components include the touchscreen 102, video camera 106 and also a speaker 220 and microphone 230.
  • the device 100 also contains a transceiver 210 connected to the system bus 201 via an interface for communicating over a wireless link, such as a GSM, CDMA, UMTS, LTE, WiMax or Wi-Fi link.
  • the device 100 includes both volatile memory 212 and non-volatile memory 214.
  • the volatile memory 212 is RAM, DRAM or any other type of volatile memory.
  • the volatile memory 212 is used by the controller 200 for the temporary storage of data, for instance when controlling the operation of a hardware or software component or moving data between components.
  • the non-volatile memory 214 is ROM, flash memory, a hard disk or any other type of non- volatile memory.
  • the non- volatile memory 214 stores computer readable instructions used in operating the device 100 as well as storing content and personal data such as images, videos, music and contacts.
  • the controller 200 operates under the control of computer-readable instructions stored in the non-volatile memory 214, which include an operating system 216 and additional software, middleware or firmware modules, and software applications.
  • the operating system 216 is the Maemo operating system, developed by Nokia
  • the touch sensitive display 102 comprises a display part 202 and a tactile interface part 204.
  • the tactile interface part 204 may be a multi-touch device configured to simultaneously detect multiple touch contacts, or may alternatively be a single-touch device.
  • the controller 200 receives inputs from the tactile interface 204 and controls the display shown on the display part 202.
  • the device 100 is shown as comprising a user interface 300.
  • This incorporates the touchscreen 102 and physical keys 104, but also includes outputs, particularly in the form of text and graphics provided on the display part 202.
  • the user interface 300 incorporates the user interface software 218, which is configured to operate along with user interface hardware, including the touchscreen 102 and physical keys 104.
  • the user interface software 218 may be separate from the operating system 216, in which case it interacts closely with the operating system as well as the applications. Alternatively, the user interface software 218 may be integrated with the operating system.
  • software applications include communication software 240 configured for controlling one or more of the hardware components to effect communication between a user of the device and user(s) of remote device(s). It will be appreciated that the communication software 240 receives user inputs by way of the user interface 300 and also interacts with the user interface in producing device outputs.
  • Figure 4 shows a communication software configuration according to an exemplary embodiment of the invention. As shown, the communication software 240 includes a video telephony software application 410 configured for controlling the display part 202, video camera 106, speaker 220, microphone 230, transceiver 210 and other hardware components to effect video communication with users of remote devices.
  • the communication software 240 also includes video processing software 420 for processing video data generated by the video camera 106 during a video call mediated by the video telephony software 410.
  • the video processing software 420 includes face recognition software 422 for identifying faces and facial features displayed on the display part 202.
  • the video processing software 420 also includes region identification software 424 for identifying regions of the display part in which faces or facial features such as ears, noses or eyes are displayed. The region
  • identification process may be said to be dynamic because as the video image on the display changes, for example when a call participant moves their head position or the device 100, the region identification software 424 updates the identified region accordingly to account for the change.
  • Dynamically identifying regions in which a face or facial feature is displayed is known per se to those skilled in the art. Relevant references include, inter alia:
  • the video processing software 420 further comprises video modifying software 426 for modifying the video image, for example by cropping the video image or by enlarging or reducing some areas in size, or by modifying the displayed video in other ways.
  • the video processing software 420 may further comprise gesture recognition software 430 for processing video data to identify human gestures, for example facial or hand gestures.
  • gesture recognition is known per se and described for example in US5454043 and US2008005703, and will not be described in more detail here.
  • the communication software 240 also includes tactile input processing software 440 comprising touch gesture recognition software 450 for processing tactile user inputs received by the touchscreen 102 to identify touch gestures.
  • tactile input processing software 440 comprising touch gesture recognition software 450 for processing tactile user inputs received by the touchscreen 102 to identify touch gestures.
  • touch gestures may be associated with different operations for execution on the device 100. Some gestures may involve movements such as a "pinch" gesture at a specific location, for example.
  • the gesture recognition software 430 and/ or the touch gesture recognition software 450 may have a behaviour recognition engine which can be "trained” to identify user- defined gestures by an iterative process whereby a user repeats the gesture until it can be unambiguously identified by the device 100.
  • the communication software 240 to mediate communication between a user of the device 100 and a user of a remote device 110 will now be described.
  • the remote device 110 will be described as identical to the device 100 and like reference numerals will be used to identify like components.
  • the devices may include the same communication software 240, the invention is not limited to communication between identical or even generally similar devices and the remote device 110 may be of a completely different type to the device 100 and may have substantially different features and configuration.
  • Figures 5 to 9 illustrate operations of the device 100 and the remote device 110 during a communication session between the devices 100, 110. It will be appreciated that in the operations of Figures 5 to 9, the controllers 200 of the devices 100, 110 perform the stated functions and actions, including controlling the display of content on the display parts 202, under control of instructions that form part of the communication software 240 and the user interface software 218 running on the operating systems 216 of the devices 100, 110.
  • Figure 5 shows the first and second devices 100, 110 connected via the wireless link by way of the transceivers 210.
  • the first device 100 and the second devices are exchanging information under control of the communication software 240 running on the devices 100, 110, thereby effecting a video call communication between Jane, a user of the first device, and Frank, a user of the second device.
  • the display part 202 of Jane's device 100 displays graphical information comprising video of Frank.
  • the video is derived from video information captured by the video camera 106 of Frank's device 110 and transmitted to Jane's device 100 over the wireless link.
  • the video data received from Frank's device 110 may be processed at Jane's device 100 and/or Frank's device and modified before it is displayed.
  • the region identification software 424 of Jane's device 100 may dynamically identify a region of the video in which Frank's face is displayed and the video modification software 426 may then crop the video so that Frank's face is appropriately and centrally framed by the display part 202.
  • Figure 6 shows the output 600 of the display part 202 of Jane's device 100 at one particular instant during the video call and illustrates a video 610 of Frank being displayed on the display part 202.
  • the display part 600 also displays graphical information in the form of video 620 of Jane, which is derived from video information captured by the video camera 106 of Jane's device 100. It will be appreciated that this video information may be processed and modified before it is displayed, in a similar way to that described above in relation to the video 610 of Frank.
  • Figure 7 shows the corresponding output on Frank's device 110. As shown, the display of Frank's device 110 has a similar layout to that of Jane's device but in this case the main video image 710 is of Jane and the secondary video image 720 is of Frank.
  • the devices 100, 110 may exchange signals and process the video images so as to ensure that the video image 710 of Jane on Frank's display is the same as the corresponding video image 620 on Jane's display and so that the video image 610 of Frank on Jane's display is the same as the corresponding video image 720 on Frank's display.
  • the video call allows Jane to interact with Frank by way of audio and visual communication, for example by speaking and by way of gestures.
  • Jane can also interact with Frank by way of tactile user input comprising touch contact with the region of the display part 206 where Frank's video image 610 is displayed.
  • the additional "touch" option presents an intuitive and fun option for enhancing the communication.
  • verbal communication is not possible or not wanted, the "touch" interaction allows discrete, fast and silent communication.
  • the communication software 206 is configured to execute different operations depending on the location of the touch interaction. As illustrated in the example of Figure 6, when Jane touches Frank's ears, the communication software 206 disables the audio stream from Jane's device 100 to Frank's device 110 so that Jane's voice is muted to Frank.
  • the location of Frank's ears and other facial features are determined by the region identification software 424.
  • the region identification software 424 dynamically identifies different regions of the display part 206 where Frank's facial features are displayed. For example, the region identification software 424 identifies regions where Frank's ears are displayed and regions were Frank's eyes are displayed. These display regions can be referred to as sub-regions since they are sub-regions of the larger display region in which Frank's video image is displayed. An example of the identified sub-regions 612 for Frank's eyes is shown in Figure 6.
  • the communication software 206 identifies when a touch contact on the tactile interface 204 is incident at a location of the display part within a sub-region, and in response executes a corresponding set of operations. For example, when the communication software 240 identifies that a touch contact is incident within the sub- region for Frank's ears, the communication software mutes Jane's audio stream to Frank's device, as described above. Further operations may also be executed, for example notifications may be displayed on the display part 206 to indicate that Jane has been muted to Frank, as shown in Figure 6. As shown, notification may comprise images overlayed with the video image such as a "mute" icon 640 and/ or images 660 of fingertips positioned over Frank's ears to indicate that Jane has muted her audio to Frank by way of a touch interaction.
  • the operations performed in response to a touch contact on Jane's device 100 may also include operations performed on Frank's device 110. Such operations may be performed in response to signals sent from Jane's device 100 to Frank's device 110 to indicate that particular operations should be performed on Frank's device. For example, as illustrated in Figure 7, when Jane touches Frank's ears on her device 100, notifications are displayed on the display part 204 of Frank's device 110, including fingers 702 positioned over Frank's ears in the video image 720 and a notification icon 730 to indicate that audio from Jane has been muted.
  • the communication software 260 also allows Jane to interact with the video image 620 of herself by way of tactile input.
  • the communication software 240 may be configured to identify when Jane touches a sub-region displaying her own ears and in response to mute the audio stream from Frank to Jane. It will be appreciated that interaction with the video image 620 is implemented in a similar way to interaction with the video image 610.
  • Frank's device is configured in a similar way to that of Jane's device and as a result Frank can also interact with Jane by way of touch contact with the video images of Jane and himself which are displayed on his device 110. For example, Frank may mute the audio stream from Frank's device 110 to Jane's device 100 by touch contact with a sub-region corresponding to Jane's ears, causing corresponding notifications to be displayed on Frank and Jane's display.
  • a set of operations may be performed in response to touch contact in different sub-regions. For example, as shown in Figures 8 and 9, when Frank touches a sub- region for Jane's eyes, a set of operations are performed comprising 1) reducing the brightness of the video image of Frank on Jane's device 100, 2) displaying an image of fingers covering Jane's eyes over the video of Jane displayed on Franks device 110 and 2) displaying an image of hands covering Jane's eyes over the video of Jane displayed on Jane's device 100.
  • the video data stream to Jane's device 100 may be blocked so that Jane cannot see Frank.
  • a set of operations may be performed when a specific touch gesture is recognised by the touch gesture recognition software 450, the touch gesture comprising one or more touch contacts in a sub-region displaying a facial feature.
  • the audio may mute the corresponding device 100, 110 and display notifications to both devices 100, 110 by displaying an image denoting a pinch on the mouth, along with a mute icon.
  • the devices 100, 110 may also be configured to execute particular operations in response to a combination of touch and one or more hand or other gestures. For example, if the gesture recognition software 430 identifies a raised index finger by processing video data from the camera 106, and after a short moment the touch gesture recognition software identifies a touch contact on the mouth, a "mute" operation may be executed.
  • the sub-regions may be determined to be larger than the region on which the corresponding facial feature is displayed, so that for example a tactile input representing a pinch on the lips may be identified even where the gesture is not precisely incident on the lips.
  • the sub-regions corresponding to specific facial features may not necessarily display those facial features. For example, sub-regions for Frank's lips may be determined to be above or below the region of the display part 204 where the lips are actually displayed, so that an exaggerated pinch gesture on Frank's face may be recognised as a user input for muting Frank's audio stream.
  • different communication options such as muting and adjusting the video brightness may be selected by tactile user input comprising touch contact with sub-regions of the display parts 202 of the devices 100, 110.
  • communication options may be directly adjusted by touch, without the need to navigate through menus and options of the user interface and without it being necessary to exit a full-screen video mode.
  • a video processing operation in which the appearance of a participants face is modified may be performed in response to a corresponding tactile user input.
  • the devices 100, 110 are configured to stretch or enlarge a participants face 610, 620, 710, 720 in response to a corresponding tactile user input, for example in response to a multi- touch dragging input.
  • individual facial features may be enlarged or reduced in size by way of tactile user input comprising one or more touch contacts with sub-regions for the features. In this way, users may distort participants faces for fun and to express themselves.
  • a user may enlarge a particular facial feature to make that feature easier for the other user to interact with.
  • the video processing software 420 is configured to visually enhance the participants faces or bodies, thereby to enable an avatar-like appearance.
  • FIG. 10 is a flow chart illustrating a method according to an exemplary embodiment of the invention.
  • a video call is initiated.
  • graphical information representing a participant in the video call is displayed, as shown in step 1010.
  • sub-regions are dynamically identified by identifying body features of the participant displayed on the display.
  • User input comprising at least one touch contact with a sub-region is received at step 1030 and in response, at least one operation is caused to be performed by the device 100 or the remote device 110, as shown at step 1040.
  • graphical information in the form of video images of the participants may be displayed on the devices 100, 110.
  • users may interact with the still images by tactile input, thereby to cause corresponding operations to be performed.
  • Figure 11 illustrates an exemplary communication software configuration for this type of communication.
  • the communication software 240 of Figure 11 has avatar display software 1100 for displaying graphical information comprising avatars of participants in the
  • the avatars may be two dimensional or three dimensional models or icons representing the participants, and may be generated manually by avatar generation software 1100 running on the devices 100, 110.
  • avatar generation software 1100 for manual avatar generation is well known per se and will not be described in detail here.
  • the avatars may be generated automatically by the devices 100, 110 from video or still images of the participants.
  • Jane's device 100 displays an avatar for Frank and also an avatar for Jane.
  • avatars are displayed for Jane and Frank.
  • Jane's avatar may be generated on Jane's device 100 and transmitted to Frank's device 110 over the wireless link.
  • Franks' avatar may be generated on Frank's device 100 and transmitted to Jane's device 110.
  • the devices 100, 100 are responsive to tactile input comprising touch contact with sub-regions of the display and are configured to respond to these tactile inputs by executing operations on one or both of the devices 100,110.
  • the sub-regions relate to regions of the display in which body or facial features of the avatars are displayed. Accordingly, Frank and Jane may interact with the avatars in a similar manner to the manner described above in relation to the video call images 610, 620, 710, 720 and similar or the same operations may be performed by the devices 100, 110 in response.
  • the devices 100, 110 may be configured so that in response to a multi touch dragging gesture comprising touch contact with a sub-region for Jane's avatar's mouth, the shape of the mouth is changed, for by twisting the shape of the mouth or by changing the shape from a frown to a smile.
  • the devices 100, 110 exchange data so that any such change to the appearance of an avatar is made on both devices. That is, both parties to the call get informed about the interaction in real time. It will be appreciated that many other interactions are possible.
  • the avatar's faces may be enlarged or distorted, or some facial features may be enlarged to make them easier to interact with.
  • a "pinch" on the mouth of one participant may disable the audio stream from that participant.
  • a user can pinch an avatar's mouth to reduce its size and in this way slightly lower the volume, instead of totally muting it.
  • user distortion of an avatar's mouth may cause changes in the audio stream associated with the avatar so that, for entertainment, the avatar adopts a distorted, frequency shifted or modulated voice.
  • the other avatar may fade or temporarily disappear from view.
  • Many other operations may be performed in response to tactile input, for example operations involving audio and/ or vibration on one or both devices.
  • the avatars may be animated or may be still images.
  • the avatars are animated to reflect body movements captured by the video cameras 106 of the devices 100, 110. That is, video captured by the video cameras 106 may be processed to identify gestures or body/ facial movements of the participants and the avatars may be animated accordingly. For example, in response to identifying that Jane has moved her head to the left, Jane's avatar's head also be caused to move to the left.
  • the animation need not necessarily reflect all body movements exactly and for example if a participant's face moves outside of the camera view-finder, the avatar may still be caused to remain fully on-screen for further interaction.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La présente invention se rapporte à un procédé consistant : à amener un premier dispositif à envoyer des informations à un second dispositif et à recevoir des informations de celui-ci; à amener le premier dispositif à afficher des données graphiques représentant un utilisateur du premier dispositif ou un utilisateur du second dispositif dans une région d'un affichage; à amener le premier dispositif à définir au moins une sous-région de ladite région de l'affichage, chacune de la ou des sous-régions affichant une partie desdites données graphiques; et à amener le premier dispositif à répondre à une pression tactile exercée par un utilisateur dans une sous-région parmi ladite ou lesdites sous-régions, de sorte à amener le premier dispositif et/ou le second dispositif à exécuter au moins une opération.
PCT/EP2010/060081 2010-07-13 2010-07-13 Envoi et réception d'informations WO2012007034A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/EP2010/060081 WO2012007034A1 (fr) 2010-07-13 2010-07-13 Envoi et réception d'informations

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2010/060081 WO2012007034A1 (fr) 2010-07-13 2010-07-13 Envoi et réception d'informations

Publications (1)

Publication Number Publication Date
WO2012007034A1 true WO2012007034A1 (fr) 2012-01-19

Family

ID=43827631

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2010/060081 WO2012007034A1 (fr) 2010-07-13 2010-07-13 Envoi et réception d'informations

Country Status (1)

Country Link
WO (1) WO2012007034A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014241093A (ja) * 2013-06-12 2014-12-25 株式会社リコー 情報処理システム、操作装置、情報表示方法及びプログラム
EP4220369A4 (fr) * 2020-10-27 2024-03-20 Vivo Mobile Communication Co Ltd Procédé et appareil d'interaction pour un appel vidéo

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5454043A (en) 1993-07-30 1995-09-26 Mitsubishi Electric Research Laboratories, Inc. Dynamic and static hand gesture recognition through low-level image analysis
US20080005703A1 (en) 2006-06-28 2008-01-03 Nokia Corporation Apparatus, Methods and computer program products providing finger-based and hand-based gesture commands for portable electronic device applications
US20080055272A1 (en) * 2006-09-06 2008-03-06 Freddy Allen Anzures Video Manager for Portable Multifunction Device
US20090040289A1 (en) * 2007-08-08 2009-02-12 Qnx Software Systems (Wavemakers), Inc. Video phone system
EP2098947A2 (fr) * 2008-03-04 2009-09-09 Apple Inc. Sélection de texte à l'aide de gestes
EP2114073A2 (fr) * 2008-04-30 2009-11-04 LG Electronics Inc. Terminal mobile et procédé associé de contrôle d'appel vidéo

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5454043A (en) 1993-07-30 1995-09-26 Mitsubishi Electric Research Laboratories, Inc. Dynamic and static hand gesture recognition through low-level image analysis
US20080005703A1 (en) 2006-06-28 2008-01-03 Nokia Corporation Apparatus, Methods and computer program products providing finger-based and hand-based gesture commands for portable electronic device applications
US20080055272A1 (en) * 2006-09-06 2008-03-06 Freddy Allen Anzures Video Manager for Portable Multifunction Device
US20090040289A1 (en) * 2007-08-08 2009-02-12 Qnx Software Systems (Wavemakers), Inc. Video phone system
EP2098947A2 (fr) * 2008-03-04 2009-09-09 Apple Inc. Sélection de texte à l'aide de gestes
EP2114073A2 (fr) * 2008-04-30 2009-11-04 LG Electronics Inc. Terminal mobile et procédé associé de contrôle d'appel vidéo

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party
Title
A. YUILLE; P. HAALLINAN; D.S. COHEN: "Feature extraction from faces using deformable templates", INT. J. COMPUT. VISION, vol. 8, no. 2, 1992, pages 99 - 111
M. KASS; A. WITKIN; D. TERZOPOULOS: "Snakes: active contour models", INT. J. COMPUT. VISION, vol. 1, no. 4, 1988, pages 321 - 331
S. SCLAROFF; J. ISIDORO: "Active blobs: region-based, deformable appearance models", COMPUT. VISION IMAGE UNDERSTANDING, vol. 89, no. 2-3, 2003, pages 197 - 225
T.F. COOTES; C.J. TAYLOR; D.H. COOPER; J. GRAHAM: "Active shape models-their training and application", COMPUT. VISION IMAGE UNDERSTANDING, vol. 61, no. 1, 1995, pages 38 - 59
T.F. COOTES; G.J. EDWARDS; C. TAYLOR: "Active appearance models", IEEE TRANS. PATTERN ANAL. MACH. INTELL., vol. 23, no. 6, 2001, pages 681 - 685
X.W. HOU; S.Z. LI; H.J. ZHANG; Q.S. CHENG: "Direct appearance models", PROCEEDINGS OF CVPR01, vol. 1, 2001, pages 828 - 833
ZHONG XUEA; STAN Z. LIB; EAM KHWANG TEOH; AI-EIGENSNAKE: "an affine-invariant deformable contour model for object matching", IMAGE AND VISION COMPUTING, vol. 20, no. 2, 1 February 2002 (2002-02-01), pages 77 - 84

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014241093A (ja) * 2013-06-12 2014-12-25 株式会社リコー 情報処理システム、操作装置、情報表示方法及びプログラム
EP4220369A4 (fr) * 2020-10-27 2024-03-20 Vivo Mobile Communication Co Ltd Procédé et appareil d'interaction pour un appel vidéo

Similar Documents

Publication Publication Date Title
CN109062479B (zh) 分屏应用切换方法、装置、存储介质和电子设备
US11513608B2 (en) Apparatus, method and recording medium for controlling user interface using input image
KR102062310B1 (ko) 전자 장치에서 헤드 트래킹 기술을 이용하여 제어 서비스를 제공하기 위한 장치 및 방법
CN110471596B (zh) 一种分屏切换方法、装置、存储介质及电子设备
CN105814522B (zh) 基于运动识别来显示虚拟输入设备的用户界面的设备和方法
WO2015188614A1 (fr) Procédé et dispositif de mise en œuvre d'ordinateur et de téléphone mobile dans un monde virtuel, et lunettes les utilisant
EP4002066A1 (fr) Procédé d'interaction par commande gestuelle dans l'air et dispositif électronique associé
US11546457B2 (en) Electronic device and method of operating electronic device in virtual reality
CN108920069B (zh) 一种触控操作方法、装置、移动终端和存储介质
CN111045511B (zh) 基于手势的操控方法及终端设备
US11740754B2 (en) Method for interface operation and terminal, storage medium thereof
US20180239526A1 (en) Method and systems for touch input
CN111610912B (zh) 应用显示方法、应用显示装置及存储介质
WO2017076017A1 (fr) Procédé et appareil permettant de faire fonctionner une application
EP3575917A1 (fr) Collecte d'empreintes digitales
JP2021531589A (ja) 目標対象の動作認識方法、装置及び電子機器
CN113655929A (zh) 界面显示的适配处理方法、装置和电子设备
CN111988522B (zh) 拍摄控制方法、装置、电子设备及存储介质
CN109213349A (zh) 基于触摸屏的交互方法及装置、计算机可读存储介质
CN109547696B (zh) 一种拍摄方法及终端设备
WO2012007034A1 (fr) Envoi et réception d'informations
WO2020034763A1 (fr) Procédé de reconnaissance de gestes, et procédé et appareil de traitement de gestes
KR20230168568A (ko) 정보 입력 방법 및 장치, 전자 기기 및 저장 매체
CN108877742A (zh) 亮度调整方法及装置
CN115543064A (zh) 界面显示控制方法、界面显示控制装置及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10737827

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 10737827

Country of ref document: EP

Kind code of ref document: A1