JP2020525946A - Method and system for indicating reaction of participants in a virtual conference - Google Patents

Abstract

Provides a way to show emotional reactions in virtual meetings. This method involves creating or selecting avatar data that defines one or more avatars representing one or more corresponding users in response to input from one or more corresponding users; A step of receiving one or more user selections of conference data defining one or more virtual conferences, wherein the user selection includes a receiving indicator that the user is attending the virtual conference; With the steps to generate output for viewing a virtual conference containing one or more avatars representing one or more users attending the conference using the avatar data and conference data corresponding to the virtual conference. With the step of receiving emotional input data from one or more users indicating the emotional response or body language of one or more users attending a virtual conference; using emotional input data With steps to process avatar data; virtual conferencing to render one or more avatars for one or more users to display their emotional state in response to each emotional input data. Includes steps to update the output for display of.

Description

The present invention relates to a method and system for showing participant reaction in a virtual conference.

For business and social reasons, computer users often arrange meetings such as formal business meetings or informal gatherings in virtual environments on computer network systems. Such meetings reduce the cost and travel of face-to-face meetings and save travel time. Also, such a conference is very convenient and allows immediate meetings of diverse and dispersed people.

Virtual conferences can also form the basis of a framework of social interaction between members of a user group. The interface that hosts the virtual conference can also be used as a means to provide many of the auxiliary features associated with the conference.

In meetings where people do not meet directly, it is important to try to make interactions between people in a virtual environment as natural as possible.

One aspect of the present invention provides a system for showing emotional reactions in a virtual conference. The system includes at least one processor; a memory that stores instructions, the instructions being executable by the at least one processor and responsive to input from one or more corresponding users. Creating or selecting avatar data defining one or more avatars representing a person or corresponding users; one or more user selections of meeting data defining one or more virtual meetings. The user selection includes an indication that the user is attending a virtual conference; receiving; using the avatar data and the conference data corresponding to the virtual conference, attending the conference. Generating output for display of a virtual conference that includes one or more avatars representing one or more users present; the emotional response or body of one or more users attending the virtual conference Receiving emotional input data indicating a body language from one or more users; processing avatar data using the emotional input data; depending on each emotional input data Updating the output for display of the virtual conference to render one or more avatars for one or more users to display their respective emotional states.

Another aspect of the invention provides a method of exhibiting an emotional reaction in a virtual conference, the method responding to input from one or more corresponding users. Creating or selecting avatar data that defines one or more avatars that represent; a step of receiving one or more user selections of meeting data that defines one or more virtual meetings; The selection includes receiving an indication that the user is attending a virtual conference; receiving avatar data and conference data corresponding to the virtual conference to identify one or more users attending the conference. Generating an output for display of a virtual conference that includes one or more avatars that represent; an emotional input that indicates an emotional response or body language of one or more users present in the virtual conference. Receiving data from one or more users; processing avatar data using emotional input data; displaying each emotional state according to each emotional input data Updating the output for the display of the virtual conference to render one or more avatars for one or more users accordingly.

Another aspect of the present invention provides a carrier medium or a storage medium that carries code executable by a processor to implement the lazy search method.

1 is a schematic diagram illustrating a system according to one embodiment. 2 is a flow diagram of a method of using the system of FIG. 1 according to one embodiment. FIG. 6 is a schematic diagram of a user interface for a virtual conference created according to one embodiment. FIG. 6 is a schematic diagram of a conference using an augmented reality conference display according to one embodiment. 5 is a schematic diagram of a user interface for an augmented reality conference display generated in the embodiment of FIG. 4. FIG. FIG. 6 is a schematic diagram of a user interface for a social meeting generated according to one embodiment. 1 is a schematic diagram of a basic computer device for use in one embodiment.

DETAILED DESCRIPTION The following detailed description refers to the accompanying drawings that form, by way of example, specific embodiments that may form a part of this specification and practice the subject matter of the invention. These embodiments have been described in sufficient detail to enable those skilled in the art to practice them, and other embodiments may be utilized, structurally without departing from the scope of the inventive subject matter, It should be appreciated that logical and electrical changes may be made. Such embodiments of the present subject matter are not intended to limit the scope of the present application to any single invention or inventive concept in the event that a plurality is actually disclosed, and are merely for convenience. Reference may be made herein to the term "invention" individually and/or collectively.

Therefore, the following description should not be construed in a limiting sense, and the scope of the present subject matter is defined by the appended claims.

In the following embodiments, the same components are designated by the same reference numerals.

In the following embodiments, the data will be described as being stored in at least one database. The term database refers to a relational database (eg, Oracle database, mySQL database, etc.), non-relational database (eg, NoSQL database, etc.), in-memory database, spreadsheet, comma separated value (CSV) file, extended markup language. (XML) files, text (TXT) files, flat files, spreadsheet files, and/or stores and/or stores data, including, but not limited to, widely used formats for data storage or proprietary formats. It is intended to include a data structure (and/or combination of data structures) for organization. Databases are typically stored in one or more data stores. Accordingly, each database referred to herein (eg, in the description and/or figures of the present application) should be understood as being stored in one or more data stores. "File system" (for example, disk file system such as FAT, NTFS, optical disk, flash file system, tape file system, database file system, transaction file system, network file system, etc.) controls data storage and retrieval methods. it can. For simplicity, the disclosure is described herein in terms of databases. However, the systems and techniques disclosed herein may be implemented in a file system or a combination of database and file system.

In the following embodiments, the term data store is intended to include any computer-readable storage medium and/or device (or collection of data storage media and/or devices). Examples of the data store are an optical disk (eg, CD-ROM, DVD-ROM, etc.), a magnetic disk (eg, hard disk, floppy disk, etc.), a memory circuit (eg, solid state drive, random access memory (RAM), etc.). However, the present invention is not limited to these. Another example of a data store is a hosted storage environment (commonly referred to as "cloud" storage) that contains a collection of physical data storage devices that are remotely accessible and can be provisioned quickly as needed.

The functions or algorithms described herein are implemented in hardware, software, or a combination of software and hardware in one embodiment. Software includes computer-executable instructions stored on a computer-readable carrier medium, such as memory or other type of storage device. Further, the functions described may correspond to modules that may be software, hardware, firmware, or any combination thereof. The functions may be performed in one or more modules as desired, and the described embodiments are merely examples. The software runs on a digital signal processor, ASIC, microprocessor, or other type of processor running on the system (such as a personal computer, server, router, device capable of processing data, including network interconnect devices).

Some embodiments may include two or more specific interconnect hardware modules or devices with associated control and data signals communicated between and through the modules, or as part of an application specific integrated circuit. To implement the function. Thus, the example process flow is applicable to software, firmware, and hardware implementations.

The generalized embodiments provide a method and system for showing emotional reaction in a virtual conference, where one or more corresponding responses are provided in response to input from one or more corresponding users. Avatar data defining one or more avatars representing the user is created or selected and one or more user selections of meeting data defining one or more virtual meetings are received. The user selection includes an indication that the user is attending a virtual conference. Using the avatar data corresponding to the virtual meeting and the meeting data, an output including one or more avatars representing one or more users attending the meeting is generated for display of the virtual meeting. .. Emotional input data is received from one or more users that indicates the emotional reaction or body language of one or more users attending the virtual conference. The avatar data is processed using emotional input data and renders one or more avatars for one or more users to display their respective emotional states in response to their respective emotional input data. So that the output for the display of the virtual conference is updated.

The virtual conference can be any type of conference in a virtual environment such as a business conference, a conference, a social meeting, a chat room, a virtual store, and the like. In other words, the avatar generated by the user exists in a virtual situation, and another avatar exists in this situation. By displaying the emotional state in the virtual environment, it is possible to interact with other users via an avatar that indicates the emotional state of the user. Thus, the emotional state of the avatar can be simply manipulated to reflect the emotional state of the user, with the body language easily interacting with other users without the need for any other form of indicator text. Can act. Avatar's body language is the most natural way to express emotions to other users through a virtual environment.

The virtual conference can be a "pure" virtual conference in which all of the participants' images are generated as avatars. Alternatively, the virtual conference may be an augmented reality conference in which video images of one or more participants in the conference are displayed, the augmented reality conference including a video image of the participants overlaid on the video data. It has one or more avatars that represent a person or users. In this way, participants who are not part of the "real" conference can express themselves and interact using the body language of their avatar.

Interaction input from one or more users attending the virtual conference is received, causing the avatar to perform the required interaction, and the output for displaying the virtual conference is 1 to display the required interaction. Interaction data is received from a person or users and updated to render one or more avatars for the one or more users. For example, interactions may include greeting emotional interactions, including handshakes, "high fiving," hugs, or kisses.

In one embodiment, the user interface may be provided as a conventional website with display output and pointer device and keyboard input by the user. In alternative embodiments, the interface is a keyboard, a touch screen, a pointer device (mouse, trackball, trackpad, pen device, etc.), voice recognition hardware and/or software for recognizing voice or speech from a user, gesture recognition input. It can be provided by any form of visual output and input, such as hardware and/or software.

In one embodiment, the method and system can be used with the method and system disclosed in co-pending U.S. patent application entitled "VIRTUAL OFFICE," filed on the same date as this application. The contents of are incorporated herein by reference in their entirety. Thus, the virtual conference becomes part of the virtual office, and the user can control his avatar to interact with the image of the item on the office equipment and cause the item on the office equipment to perform office functions.

In one embodiment, the method and system are disclosed in “METHOD AND APPARATUS TO TRANSFER” filed on the same date as this application.
DATA FROM A FIRST COMPUTER STATE TO A DIFFERENT COMPUTER STATE" may be used with the method and apparatus disclosed in co-pending US patent application Ser. Incorporated into the book.

In one embodiment, the method and system are disclosed in the “EVENT BASED DEFERRED SERCH METHOD” filed on the same date as the present application.
It may be used with the method and apparatus disclosed in co-pending U.S. Patent Application No. "AND SYSTEM", the contents of which are incorporated herein by reference in their entirety.

In one embodiment, the method and system are described in “USER INTERFACE METHOD AND,” filed Dec. 30, 2016.
It may be used with the method and apparatus disclosed in co-pending US patent application Ser. No. 15/395,343 entitled "APPARATUS", the contents of which are incorporated herein in their entirety. The user interface of Application No. 15/395,343 can provide a means for a user to interact with the system for input and selection.

In one embodiment, the method and system are described in “AN ELECTRONIC TRANSACTION METHOD,” filed Dec. 30, 2016.
It can be used with the electronic trading method and system disclosed in co-pending US patent application Ser. No. 15/395,487 entitled "AND APPARATUS", the contents of which are incorporated herein in their entirety. ..

Next, specific embodiments will be described with reference to the drawings.

FIG. 1 illustrates a generalized system according to one embodiment.

FIG. 1 shows two client devices 100A and 100B, each for use by a user. Any number of client devices can be used. The client devices 100A and 100B may be any type of computer or processing machine such as a personal computer, laptop, tablet computer, electronic organizer, mobile device, smartphone, mobile phone, video player, television, multimedia device, personal digital assistant, and the like. Can be included. In this embodiment, each client device can execute a web browser 101A and 101B to interact with a web page hosted on the server system 1000. In alternative embodiments, the web browsers 101A and 101B can be replaced with applications running on the client devices 100A and 100B.

The client devices 100A and 100B are connected to a network, which is the Internet 50 in this example. The network can include any suitable communication network for networking computing devices.

The server system 1000 includes any number of server computers connected to the Internet 50. Server system 1000 operates to provide services in accordance with embodiments of the present invention. The server system 1000 includes a web server 110 that hosts web pages that are accessed and rendered by browsers 101A and 101B. The application server 120 is connected to the web server 110 and provides the web server 110 with dynamic data. The application server 120 is connected to the data store 195. Data store 195 stores data in a number of different databases: user database 130, avatar database 140, virtual world data store 150, conference database 160, and emotional reaction database 170. The user database 130 stores information about users that may include identifiers, names, ages, usernames and passwords, birthdays, addresses, and so on. The avatar database 140 may store data related to avatars that the user can create to represent the user and avatars that the user generated in association with the user data. The virtual world data store 150 stores data necessary for forming a virtual conference environment. The conference database 160 includes a conference identifier, a conference name, associated users attending the conference (and thus an avatar that is indirectly rendered in the virtual conference), an identifier for the video stream rendered as part of the augmented reality virtual conference, the conference Dates can be stored for a particular conference, including conference login information and the like. The emotional response database 170 can store data indicating a set of emotional responses selected by the user and that can be used to change the rendered appearance of the avatar. Avatar data and processing for rendering in a virtual environment can be structured to apply each of the emotional responses. Emotional reactions include smiles, laughter, crying, handshake greetings, hugs or kisses, boredom, frowning, cross angry, surprise, relax, facial expressions of interest.

FIG. 2 is a flow diagram of a process for showing emotional reactions in a virtual conference using the system of FIG. 1 according to one embodiment.

In step S10, a user creates or selects an avatar to represent himself in a virtual conference. In step S11, a user selection of conference data defining a virtual conference is received. The user selection includes an indication that the user is attending a virtual conference. In step S12, the avatar data corresponding to the virtual conference and the conference data are used to generate an output for display of the virtual conference including an avatar representing a user attending the conference. In step S13, emotional input data indicating the emotional reaction or body language of the user attending the virtual conference is received from the user. In step S14, the avatar data is processed using the emotional input data, and in step S15, the output for displaying the virtual conference is displayed by the user to display the emotional state according to the emotional input data. Updated to render avatars for.

FIG. 3 is a schematic diagram of a user interface for a virtual conference created according to one embodiment.

The display 200 includes a virtual conference area 201 for displaying a virtual conference and a reaction menu area 202 for displaying user selectable menu items, and the user applies an emotional reaction or body language to the virtual conference avatar. Select an input so that you can interact with other attendees. Other attendees can see the user's emotional reaction applied to the avatar in the display area of the virtual meeting and change the emotional reaction displayed on their avatar accordingly, for example, You can react by performing other actions in the virtual conference. In this embodiment, the menu is shown as a text menu, but the menu is selected by the user to change the appearance and behavior of the avatar to display emotional reactions and body language in response to user selections. It can include icons or images that represent various possible emotional states.

In addition, a menu is displayed to the user, the user selects a sound or music, and the avatar can make “greeting”, “hey”, “what?
A selected phrase or pre-made phrase, such as "up)", or a birthday or greeting message may be output to the virtual conference. These may be selectable with various accents such as American or English, or even mimicking (voice color) of a famous person.

For example, there may be translation options that translate and play the message, such as some of what the user wants to say, such as speaking French when speaking in Roman. This could be a pre-stored record, or the system could translate what the user (avatar) said, but this could be a little late. One example is pre-recorded and saved message options that can be used by the user to record and play back via the avatar, eg, in response to another avatar in a meeting or a guest user.

Outside the virtual meeting area 201, the display 200 includes a shared message area 203 that can be used to share messages to virtual meeting attendees individually, in groups, or globally with other users. A shared display area 204 is displayed outside the virtual meeting area 201. In this example, since the shared display area 204 corresponds to the virtual whiteboard 203 of the virtual conference, everything drawn in the shared display area is displayed on the virtual whiteboard 203.

In the virtual conference area 201, avatars of attendees of the conference are displayed. 4 people are sitting. Two attendees 206 are shown shaking hands and greeting each other. To achieve this, the user corresponding to avatar 206 has selected the reaction menu item to shake hands. One avatar 207 of the user is angry. One avatar 208 is displayed with a smile.

The virtual conference can be controlled to operate as a conventional conference, and each user of the client device can speak the input voice and send it to the client devices of other attendees. In one example, a document can be entered into a conference by placing the document in a table on a virtual display. Where it is placed affects the users who can see the document. You can put a copy of the document in front of everyone for everyone to see. The document can be dragged into the virtual filing cabinet 214 for filing, the user can search or select files in the virtual filing cabinet 214, or the virtual filing cabinet 214 can be searched and the filing system searched to find the document. .. The user can move the avatar in the virtual conference and can be shown to exit the door 205 as the user exits the conference.

The perspective of the virtual conference displayed for each attendee may vary depending on the seat positions assigned around the table.

FIG. 4 is a schematic diagram of a conference using an augmented reality conference display according to one embodiment.

In the foreground, a physical real-world conference is taking place around the table with four attendees. At the end of the table is a display 300 that virtually presents the attending participants using avatars 301 and 302. Avatar 301 is controlled by emotional input by its respective user to reflect a happy face or smile. The avatar 302 is controlled by emotional input by its respective user to reflect an angry or frustrated face.

The augmented reality conference can be controlled to operate as a conventional conference, in which each user of the client device speaks the input audio and sends it to the speaker device associated with the client device and display 300 of other attendees. You can In one example, the document can be entered into the conference by placing the document in a table on the virtual display 300. Where it is placed can affect who can see the document. You need to put a copy of the document in front of everyone for everyone to see. In one example, documents can be dragged into the virtual filing cabinet 304 for filing. The user can move the avatar in a virtual conference, and it can be shown that the user exits the door 303 when the user exits the conference. As shown in FIG. 5, a video camera or webcam 305 is provided to provide the actual attendee's video feed to the remote or virtual attendee's computer.

FIG. 5 is a schematic diagram of a user interface of an augmented reality conference display generated for virtual attendees of the embodiment of FIG.

Display 350 includes an augmented reality conference area 310 that displays an augmented reality conference that includes a video stream of physical attendees and a combined virtual conference segment. The reaction menu area 380 is a user-selectable menu item that allows the user to select an input that applies emotional reaction or body language to an avatar in an augmented reality conference for interaction with other attendees. Is displayed. Other attendees see the user's emotional and physical responses applied to the avatar in the display area of the augmented reality conference and modify, for example, the emotional response displayed on their avatar accordingly. Or react by performing other actions in the augmented reality conference. In this embodiment, the menu is shown as a text menu, but the menu is selected by the user to change the appearance and behavior of the avatar to display emotional reactions and body language in response to user selections. It can include icons or images that represent various possible emotional states.

In one example, the user may choose to share music data, which may serve to show an expression of his mood or emotion, or depending on the response of other users, a song or song, say happy. You can enjoy playing, sharing, or saving different songs to share with other users (avatars). Playing stored or selected music, such as depressed sad music, loneliness of loneliness, depression, or pleasing happy music, can represent the mood of the user. Also, in one example, the use can be tailored to the radio station to find a suitable tuning at that time for the user's emotions.

Also, in one example, the user can select and apply a color, for example, a virtual paint of a different color (chronotherapy, sometimes referred to as color therapy). The user can choose to fill the virtual bedroom with a magical shimmering color or a dark color to show a friend how the user feels in the virtual space.

An augmented reality conference can be controlled to operate as a traditional conference, where each user of a client device attending a virtual conference segment can enter audio to enter the client device and the physical device of another virtual attendee. It can be sent to the speaker associated with the display 300 for the (real) attendee. In one example, the document physically presented to the actual conference can be entered into the virtual conference by placing it in a table in the virtual display segment of the augmented reality conference. The location of placement affects the users who can see those documents. A copy of the document can be placed in front of all virtual attendees to make them visible to all of the virtual segments of the augmented reality conference. Documents can be filed by dragging them into the virtual filing cabinet 304. The user can move the avatar in the virtual segment of the augmented reality conference and can be shown to exit the door 303 as the user exits the conference.

Display 350 includes a shared message area 360 that can be used to share messages with other users individually, in a group, or globally to attendees in an augmented reality conference. Also, the shared display area 370 is displayed.

FIG. 6 is a schematic diagram of a user interface for a social meeting generated according to one embodiment.

The display 400 includes a virtual conference area 410 that can display an avatar in a virtual environment. In this embodiment, the avatar 403 is controlled by the user to show a smile, the avatar 402 is controlled to smile, and the two foreground avatars 401 are controlled to greet each other by shaking hands.

The reaction menu area 404 provides user-selectable menu items that allow the user to select inputs that apply emotional reactions or body language to the virtual conference avatar for interaction with other attendees. indicate. Other attendees can see the user's emotional reaction applied to the avatar in the virtual meeting display area 410 and change, for example, the emotional reaction displayed on their avatar accordingly. Alternatively, it can react by performing other actions in the virtual conference.

Outside the virtual conference area 410, the display 400 includes a shared message area 405 that can be used to share messages with other users individually, in groups, or globally to virtual conference attendees. A shared display area 406 is displayed outside the virtual conference area 410. In this example, the shared display area 406 corresponds to a news item shared between two users represented by avatars 402 and 403. In the message area, a private message exchange between avatar 403 (David) and avatar 402 (Steve) related to the news item is displayed. The emotional response of the avatar is coordinated by the input by the relevant user to reflect the user's interaction with the news item.

The system can be controlled to allow users to join and move into conferences held in different rooms. These meeting rooms are typically displayed as a meeting room map, for example, and the user can select to move from one meeting room to another and join or leave the meeting. A room may represent various types of meetings, such as game room meetings, coffee table meetings, and so on. The user can also set up a meeting and invite other users to the meeting using the virtual location and time of the meeting set by the other user.

In the display area of the meeting, the avatar identifier may be displayed, or alternatively or additionally, a list of attendees may be displayed.

Virtual meetings using avatars may be in a store or in an environment related to the corresponding real-world environment, such as a gym.

In the embodiment described above, the user input for setting the emotional state of the avatar is based on simple menu selections. However, other forms of user input can be used. For example, a camera can be used to take a picture or video of the user's face and possibly the body and determine the user's emotional response. The user may also be provided with the ability to enter, or by recognizing voice, enter free text that describes an emotional reaction to control the avatar. The user's photo or video captures the user's current clothing and adapts the avatar to represent various clothing worn by the user, such as clothing, suits and ties, dresses, fancy dress, etc. Can also be used for. It can be used to promote the user's ability to wear smart or casual clothes in virtual meetings. The user can select clothes or suits and ties that can change from meeting to meeting, such as ties of different colors.

The generated avatar can be selected by the user to take any form. For example, the avatar may be an animal that contains the user's own traits, or other character mixed with the user, ie, an adaptable human trait.

This is suitable for different age groups, as the environment of the meeting can be selected as desired by the user or group of users. Groups of older and younger people, such as family and social groups, such as Irish grannies, can virtually meet, share stories, and laugh with young Australian grandchildren. The user can select casual clothes according to the virtual environment or in accordance with the virtual environment, or change the virtual environment according to the selected costume. The user can enjoy virtual accessories and items that meet his needs in the virtual conference, and the user can purchase these accessories and the like at the virtual shop, go to the virtual changing room, and prepare for the next virtual conference. ..

The user can select from the menu whether or not to join another virtual conference in another virtual conference room, for example.

In one example, the virtual conference is conducted at a virtual restaurant or social gathering that includes virtual food and/or drinks.

Basic Computer Device FIG. 7 is a block diagram illustrating a basic computer device 600 in which exemplary embodiments of the invention may be implemented. The computing device 600, including connections, relationships, and functions, and components thereof, are exemplary only, and are not limiting of the implementation of the exemplary embodiments. Other computing devices suitable for implementing the exemplary embodiments may have different components, including components having different connections, relationships, and functions.

The computing device 600 can include any server or user device, for example, as shown in FIG.

Computer device 600 may include a bus 602 or other communication mechanism for addressing main memory 606 and transferring data between various components of device 600.

Computing device 600 may also include one or more hardware processors 604 coupled to bus 602 for processing information. The hardware processor 604 can be a general purpose microprocessor, system on chip (SoC), or other processor.

Main memory 606, such as random access memory (RAM) or other dynamic storage, may also be coupled to bus 602 for storing information and software instructions executed by processor 604. Main memory 606 may also be used to store temporary variables or other intermediate information during execution of software instructions executed by processor 604.

The software instructions, when stored on a storage medium accessible to the processor 604, cause the computing device 600 to be a specialized computing device customized to perform the operations specified in the software instructions. The terms "software", "software instructions", "computer program", "computer executable instructions", and "processor executable instructions", whether human readable or not, refer to application software, desktop applications, scripts, binaries, Operating system, device driver, boot loader, shell, utility, system software, JAVASCRIPT (registered trademark), web page, web application, plug-in, embedded software, microcode, compiler, debugger, interpreter, virtual machine, linker, and It should be broadly construed as covering machine-readable information for instructing a computing device to perform a particular operation, including, but not limited to, a text editor.

Computing device 600 also includes read-only memory (ROM) 608 or other static storage coupled to bus 602, which memory can store static information and software instructions for processor 604.

One or more mass storage devices 610 are coupled to bus 602 and may be on fixed or erasable media such as magnetic, optical, solid state, magneto-optical, flash memory, or other available mass storage technology. Information and software instructions can be permanently stored. The mass storage may be shared on the network or may be a dedicated mass storage. Typically, at least one of the mass storage devices 610 (eg, the device's main hard disk) contains an operating system, user application programs, drivers and other support files, as well as any other type of data file. The main body of the program and data for instructing the operation of the computer device including the computer is stored.

Computing device 600 may be coupled via bus 602 to a display 612, such as a liquid crystal display (LCD) or other electronic visual display, for displaying information to a computer user. In some configurations, a touch-sensitive surface that incorporates touch-sensing technology (eg, resistive, capacitive, etc.) is overlaid on the display 612 to form a touch-sensitive display and a touch gesture (eg, finger or stylus) input. Can be communicated to the processor 604.

An input device 614, including alphanumeric characters and other keys, can be coupled to bus 602 to communicate information and command selections to processor 604. In addition to or in place of alphanumeric and other keys, input device 614 may include one or more physical buttons or buttons, such as a power (on/off) button, a “home” button, a volume control button, or the like. It may include a switch.

Another type of user input device is a cursor control, such as a mouse, trackball, cursor, touch screen, or direction key for communicating direction information and command selections to processor 604 and controlling cursor movement on display 612. 616. This input device typically has two degrees of freedom in two axes, a first axis (such as x) and a second axis (such as y), so that the device can specify a position in a plane. .. Other input device embodiments include a voice or utterance recognition input module that recognizes voice inputs such as voice, a visual input device that can recognize a gesture by a user, and a keyboard.

In some configurations, such as the configuration shown in FIG. 7, one or more of display 612, input device 614, and cursor control 616 are external components of computing device 600 (ie, peripheral devices), but display 612. , Input device 614, and some or all of cursor control 616 are integrated as part of the form factor of computing device 600 in other configurations.

In addition to or instead of display 612, any other form of user output device, such as an audio output device or a tactile (vibration) output device may be envisioned.

The functions of the disclosed systems, methods and modules may be performed by computing device 600 in response to processor 604 executing one or more programs of software instructions contained in main memory 606. Such software instructions may be read into memory 606 from storage device 610 or another storage medium such as a transmission medium. Execution of software instructions contained in main memory 606 causes processor 604 to perform the functions of the illustrative embodiments.

Although the functionality and operation of the exemplary embodiments may be fully implemented in software instructions, the hard-wired or programmable circuitry of computing device 600 (eg, ASIC, FPGA, etc.) may replace or in combination with software instructions. Used in combination in other embodiments, the functions may be performed according to the requirements of particular implementations implemented in the future.

The term "storage medium" as used herein refers to any non-transitory medium that stores data and/or software instructions that cause a computing device to operate in a particular fashion. Such storage media may include non-volatile media and/or volatile media. Non-volatile media include, for example, non-volatile random access memory (NVRAM), flash memory, optical disks, magnetic disks, or solid state drives such as storage device 610. Volatile media includes dynamic memory, such as main memory 606. Common forms of storage media include, for example, floppy disks, flexible disks, hard disks, solid state drives, magnetic tape or other magnetic data storage media, CD-ROMs, other optical data storage media, hole patterns. Included are physical media, RAM, PROM, EPROM, FLASH-EPROM, NVRAM, flash memory, other memory chips or cartridges.

A storage medium is different from a transmission medium but may be used in combination with the transmission medium. Transmission media participates in transferring information between storage media. For example, transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 602. Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave or infrared data communication. Machine-readable media that carry instructions in the form of code can include non-transitory storage media and transmission media.

Various forms of media may be involved in carrying one or more sequences of one or more software instructions to processor 604 for execution. For example, the software instructions may initially be carried on a magnetic disk or solid state drive of a remote computer. The remote computer can load the software instructions into dynamic memory and send the software instructions over a telephone line using a modem. A modem local to computing device 600 can receive the data on the telephone line and use an infrared transmitter to convert the data to an infrared signal. The infrared detector can receive the data carried in the infrared signal, and appropriate circuitry can place the data on the bus 602. The bus 602 transfers data to the main memory 606, from which the processor 604 retrieves and executes software instructions. Software instructions received by main memory 606 may optionally be stored on storage device 610 either before or after execution by processor 604.

Computing device 600 may also include one or more communication interfaces 618 coupled to bus 602. The communication interface 618 is both coupled to a wired or wireless network link 620 connected to a local network 622 (eg, Ethernet network, wireless local area network, mobile phone network, Bluetooth wireless network, etc.). Providing data communication for communication. Communication interface 618 sends and receives electrical, electromagnetic, or optical signals that carry digital data streams representing various types of information. For example, the communication interface 618 can be a wired network interface card, a wireless network interface card with integrated wireless antennas, or a modem (eg ISDN, DSL, or cable modem).

Network link 620 typically provides data communication through one or more networks to other data devices. For example, network link 620 may provide a connection via local network 622 to data equipment operated by a host computer or Internet Service Provider (ISP). ISPs currently provide data communication services via a worldwide packet data communication network commonly referred to as the "Internet". Local network 622 and the Internet use electrical, electromagnetic, or optical signals that carry digital data streams. Signals through various networks and signals on network link 620 and signals via communication interface 618, which carry digital data to and from computing device 600, are exemplary forms of transmission media.

Computing device 600 can send messages and receive data, including program code, through the network(s), network link 620 and communication interface 618. In the Internet example, the server may send the requested code to the application program via the Internet, ISP, local network 622, communication interface 618.

The received code may be executed by processor 604 as it is received, and/or stored in storage device 610 or other non-volatile storage for later execution.

One aspect conveys a non-transitory storage medium storing code executed by a processor of a machine to implement the method, or processor executable code executed by a processor of the machine to implement the method. A carrier medium such as a temporary medium is provided. Embodiments may be implemented with programmable digital logic that implements computer code. The code may be provided by a carrier medium to programmable logic such as a processor or microprocessor. One such embodiment of the carrier medium is a transitory medium, ie a signal such as an electrical signal, an electromagnetic signal, an acoustic signal, a magnetic signal, or an optical signal. Another form of carrier medium is a non-transitory storage medium that stores code, such as solid state memory, magnetic medium (hard disk drive), or optical medium (compact disc (CD) or digital versatile disc (DVD)). is there.

Various other changes in the details, materials and arrangements of parts and method steps described and illustrated to explain the nature of the subject matter of the invention are described in the dependent claims, as well as the principles and scope of the subject matter of the invention. It will be readily appreciated by those skilled in the art that what can be done without deviating from it.

Claims (12)

A system for showing an emotional reaction in a virtual conference, the system comprising:
At least one processor,
A memory for storing instructions,
The instructions are executable by the at least one processor,
Creating or selecting avatar data defining one or more avatars representing the one or more corresponding users in response to input from one or more corresponding users;
Receiving one or more user selections of meeting data defining one or more virtual meetings, the user selections including an indication that the user is attending the virtual meeting. thing,
The avatar data corresponding to the virtual conference and the conference data are used to generate an output for display of the virtual conference including one or more avatars representing one or more users attending the conference. What to do,
Receiving from one or more users emotional input data indicating an emotional reaction or body language of the one or more users attending the virtual conference;
Processing the avatar data using the emotional input data, and one for the one or more users to display respective emotional states in response to the respective emotional input data. Or updating the output for display of the virtual conference to render multiple avatars.
system. The instruction of claim 1, wherein the instruction comprises an instruction executable by the at least one processor to render the one or more avatars to display body language associated with the emotional input data. System. Instructions executable by the at least one processor to receive video data for a meeting, the video data including video images of one or more participants in the meeting, and producing output for the display. Augmented reality including one or more avatars representing one or more users, wherein the instructions executable by the at least one processor overlay the video image of the participant on the video data. 3. The system of claim 1 or 2, including instructions executable by the at least one processor to generate output for the display as a conference. Comprising instructions executable by said at least one processor to store a predefined set of emotional states, executable by said at least one processor to receive said emotional input data 4. A system according to any one of claims 1 to 3, wherein the instructions performed include instructions for receiving the emotional input data as a selection of outputs for displaying the menu of emotional states. .. Receiving interaction input from one or more users attending the virtual conference to cause the avatar to perform the required interaction, and one or more to display the required interaction. Receiving interaction data from multiple users and updating output for said display of said virtual conference to render one or more avatars for one or more users, The system according to any one of claims 1 to 4, comprising instructions executable by said at least one processor. A method of showing an emotional reaction in a virtual conference, the method comprising:
Creating or selecting avatar data defining one or more avatars representing the one or more corresponding users in response to input from one or more corresponding users;
Receiving one or more user selections of conference data defining one or more virtual conferences, wherein the user selections include an indication that the user is attending the virtual conference. Steps,
The avatar data corresponding to the virtual conference and the conference data are used to generate an output for display of the virtual conference including one or more avatars representing one or more users attending the conference. Steps to
Receiving from one or more users emotional input data indicating an emotional reaction or body language of the one or more users attending the virtual conference;
Processing the avatar data using the emotional input data,
Output for display of the virtual conference to render one or more avatars for the one or more users to display their respective emotional states in response to their respective emotional input data. Including the step of updating
Method. 7. The method of claim 6, wherein the one or more avatars are rendered to display body language associated with the emotional input data. Receiving video data of a conference, the video data comprising video images of one or more participants of the conference, the output overlaying the video images of the participants on the video data, 8. A method according to claim 6 or 7, generated for display as an augmented reality conference including one or more avatars representing a person or users. 9. A step of storing a predefined set of emotional states, said emotional input data being received as a selection of outputs for displaying said menu of emotional states. The method according to any one of 1. Receiving interaction input from one or more users attending the virtual conference to cause the avatar to perform the required interaction; and one or more to display the required interaction. Receiving interaction data from multiple users and updating an output for the display of the virtual conference to render one or more avatars for the one or more users. The method according to any one of claims 6 to 9. A carrier medium carrying processor executable code executed by a processor to perform the method of any one of claims 6-10. A non-transitory storage medium storing processor executable code executed by a processor to perform the method of any one of claims 6-10.

Images