KR20210046240A - Method and apparatus for providing teleconferences - Google Patents

Abstract

A teleconference providing method and a device thereof are provided. According to an embodiment of the present invention, the teleconference providing method comprises the following steps of: displaying a plurality of first graphic elements corresponding to a plurality of online conference participants, respectively, in a first region on a display of a computing device; obtaining an input about an intention expression from a user of the computing device; and displaying a second graphic element corresponding to the intention expression at a location of a first graphic element corresponding to the user among the plurality of participants in response to obtaining the input. The plurality of first graphic elements are arranged to be fixed to different locations in the first region.

Description

Method and apparatus for providing teleconferences{METHOD AND APPARATUS FOR PROVIDING TELECONFERENCES}

The present invention relates to a method and apparatus for providing teleconferencing. More specifically, it relates to a method and apparatus for facilitating non-verbal communication among a plurality of participants participating in a teleconference.

Due to the advancement of ICT technology, traditional offline meetings, group training, seminars and conference events are rapidly being replaced by teleconference. In particular, as companies adopting smart work systems that can work and collaborate anytime, anywhere, regardless of time and place, the importance of technology that facilitates communication and collaboration between remote locations is receiving great attention.

Types of frequent remote multi-party communication include audio conferences, video conferences using cameras, computer screens operated by some of the conference participants, or videos that collaborate or communicate with other participants while providing specific content to other participants in real time. /Web conference, etc.

Compared to traditional offline meetings, the teleconference has the advantage of significantly reducing the cost of securing a place for the meeting and the time and cost required for moving participants. However, various non-verbal communication methods used in offline face-to-face meetings are not yet easy in remote conferences based on limited audio and video.

Specifically, in offline face-to-face meetings, in addition to communication through spoken language, intentions and emotions are communicated through non-verbal means such as facial expressions, gestures, and attitudes. A lot of information is exchanged within a short time even by actions such as the speaker's pointing.

On the other hand, in teleconferencing, such non-verbal communication is very limited. Unlike offline face-to-face meetings, in teleconferences, communication is more severely restricted if multiple participants speak at the same time. Therefore, participants in multi-party teleconferences, where most of the communication is communicated primarily through voice, understands the content being discussed. It is difficult to empathize or express feelings smoothly.

On the other hand, it is difficult for a person who makes a presentation or leads a discussion on a specific topic to other participants, it is difficult to efficiently deliver the information he or she wants to deliver through a remote conference where non-verbal communication is limited.

Korean Patent Application Publication No. 10-2015-0134637

The technical problem to be solved by some embodiments of the present invention is to provide a method and apparatus for facilitating non-verbal communication among a plurality of participants participating in a teleconference.

Another technical problem to be solved by some embodiments of the present invention is to provide a method and apparatus for allowing a plurality of participants participating in a remote conference to intuitively express their intentions and emotions without interfering with the progress of the conference.

Another technical problem to be solved by some embodiments of the present invention is to allow the presenter to instruct the presenter in a way that the participants can intuitively recognize the part that the presenter wants to emphasize on the content shared to the participants in the remote conference. It is to provide a method and apparatus.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above technical problem, a method for providing a remote conference according to an embodiment of the present invention displays a plurality of first graphic elements respectively corresponding to a plurality of online conference participants in a first area on a display of a computing device. And obtaining an input regarding the expression of an opinion from a user of the computing device; and in response to obtaining the input, at a location of a first graphic element corresponding to the user among the plurality of participants, the doctor It may include displaying a second graphic element corresponding to the expression. In this case, the plurality of first graphic elements may be fixed and disposed at different positions in the first area.

In one embodiment, the first graphic elements may be arranged to be spaced apart from each other by the same distance from the virtual point in the first area. In some embodiments, in response to adding a participant to the online conference, the plurality of first graphic elements may be rearranged.

In an embodiment, the step of obtaining an input regarding the expression of the opinion from the user may include displaying an interface for selecting one of a plurality of expression elements on the second area of the display. The method for providing a remote conference according to some embodiments of the present invention may further include rearranging the plurality of first graphic elements based on inputs respectively obtained from the plurality of online conference participants.

The method for providing a teleconference according to an embodiment of the present invention may further include removing the displayed second graphic element when a preset display time elapses after the displaying of the second graphic element. According to some embodiments of the present invention, a method for providing a teleconference may include decreasing the size of the displayed second graphic element or increasing the transparency of the second graphic element as time elapses after displaying the second graphic element. It may further include the step of.

In one embodiment, the displaying of the second graphic element comprises: displaying an emoji corresponding to the pseudo-expression so as to at least partially overlap the first graphic element, and a translucent layer of a color corresponding to the pseudo-expression It may include at least one of overlapping and displaying the position of the first graphic element, and displaying the text corresponding to the expression of the opinion so as to at least partially overlap the first graphic element.

The method for providing a remote conference according to an embodiment of the present invention may further include displaying an active state determined based on a final input time point of the plurality of participants.

In order to solve the above technical problem, a method for providing a teleconference according to another embodiment of the present invention includes displaying content on a display of a computing device, obtaining a first user input from a user of the computing device, and , In response to a determination that the first user input corresponds to the first gesture, change the display attribute of the pointing element for indicating the parts of the content from the first state to the second state, and change the display attribute of the pointing element And transmitting the data indicative of the online conference to at least one of a computing device of the second participant of the online conference and a server that manages the online conference.

In one embodiment, the first gesture is a gesture of reciprocating the pointing element in a horizontal direction within a preset time within a preset number of times or a gesture of reciprocating the pointing element in a vertical direction by a preset number of times or more within a preset time to be.

In an embodiment, changing the display attribute of the pointing element may include changing at least one of a shape, a size, and a color of the pointing element. In some embodiments, the step of changing the display attribute of the pointing element includes: when the size of the pointing element is less than a threshold value, the pointing element is enlarged, and when the size of the pointing element is greater than the threshold value, the pointing element is reduced. It may include steps.

In the method for providing a remote conference according to an embodiment of the present invention, after the transmitting step, obtaining a second user input from a user of the computing device, and determining that the second user input corresponds to a second gesture In response to, the step of changing the display attribute of the pointing element from a second state to a first state may be further included. In some embodiments, the first gesture is a gesture of reciprocating the pointing element in a horizontal direction within a threshold distance, and the second gesture is a gesture of reciprocating the pointing element in a horizontal direction beyond the threshold distance. It's a gesture.

In order to solve the above technical problem, a method for providing a teleconference according to another embodiment of the present invention includes the steps of extracting text by analyzing a voice signal obtained from a user of a computing device, and Analyzing the content to determine a location in the content that matches the text, moving a pointing element to the matching location in the content, and data indicating the movement of the pointing element in the online conference 2 It may include transmitting to at least one of a computing device of the participant and a server that manages the online conference.

The method for providing a remote conference according to an embodiment of the present invention may further include moving the content so that the matched location is displayed on the display. In some embodiments, the determining may further include determining a location closest to the location in the content displayed on the display as the matched location, when two or more locations match the text. In some other embodiments, the method may further include transmitting the data instructing the movement of the content to at least one of a computing device of the second participant and a server that manages the online conference.

In one embodiment, the determining step is based on a natural language analysis result of the text included in the content.

1 is a diagram for explaining a usage environment of a conference participant terminal according to an embodiment of the present invention.
2 is a block diagram of a conference participant terminal according to an embodiment of the present invention.
3 is a flowchart of a method for providing a remote conference according to another embodiment of the present invention.
4A to 4C are diagrams for explaining an exemplary UI in which graphic elements corresponding to participants in a teleconference are arranged in the embodiment described with reference to FIG. 3.
5A to 5D are diagrams for explaining an exemplary UI in which expressions of opinions of teleconference participants are displayed in the embodiment described with reference to FIG. 3.
FIG. 6 is a diagram illustrating an exemplary UI provided to receive an expression of an opinion to participants in a teleconference in the embodiment described with reference to FIG. 3.
7 is a flowchart of a method for providing a remote conference according to another embodiment of the present invention.
FIG. 8 is a diagram for explaining an exemplary method of changing a display attribute of a pointing element according to a user's gesture in the embodiment described with reference to FIG. 7.
9 is a flowchart of a method for providing a remote conference according to another embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the technical idea of the present invention is not limited to the following embodiments, but may be implemented in various different forms, and only the following embodiments complete the technical idea of the present invention, and in the technical field to which the present invention pertains. It is provided to completely inform the scope of the present invention to those of ordinary skill in the art, and the technical idea of the present invention is only defined by the scope of the claims.

In adding reference numerals to elements of each drawing, it should be noted that the same elements are assigned the same numerals as possible, even if they are indicated on different drawings. In addition, in describing the present invention, when it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used with meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not interpreted ideally or excessively unless explicitly defined specifically. The terms used in the present specification are for describing exemplary embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase.

In addition, in describing the constituent elements of the present invention, terms such as first, second, A, B, (a) and (b) may be used. These terms are for distinguishing the constituent element from other constituent elements, and the nature, order, or order of the constituent element is not limited by the term. When a component is described as being "connected", "coupled" or "connected" to another component, the component may be directly connected or connected to that other component, but another component between each component It will be understood that elements may be “connected”, “coupled” or “connected”.

Hereinafter, some embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating a usage environment of a conference participant terminal 100 that facilitates communication in a remote conference according to an embodiment of the present invention.

One or more conference participant terminals 100 shown in FIG. 1 may be connected to another conference participant terminal 100 and a remote conference management server 10 through a network 20. The conference participant terminal 100 may directly exchange messages with other conference participant terminals 100 through the network 20. In some embodiments, the conference participant terminal 100 may exchange messages with other conference participant terminals 100 via the remote conference management server 10.

The remote conference management server 10 illustrated in FIG. 1 may be a server that provides a platform for enabling audio and/or video communication between conference participant terminals 100. In some embodiments of the present invention, a remote conference is possible only with a configuration provided in the conference participant terminal 100 without the remote conference management server 10.

The network 20 shown in FIG. 1 includes networks such as wired and wireless networks, packet switching networks, circuit switching networks, data communication networks, and voice communication networks.

2 is a block diagram of a conference participant terminal 100 according to an embodiment of the present invention. The conference participant terminal 100 includes a fixed computing device such as a desktop computer, and a mobile computing device such as a smart phone, a notebook computer, and a tablet PC. The conference participant terminal 100 is not limited to the examples listed above, and includes a dedicated device such as a multi-party audio conference phone, a video conference, or a telepresence device, and also includes a home appliance including a video display device and a sound reproduction device such as a smart TV. You may.

The conference participant terminal 100 may include a communication unit 110, an input unit 120, an output unit 130, a storage unit 140, and a control unit 150.

The communication unit 110 may transmit and receive various signals and data such as audio and video from an external device. The communication unit 110 may include a wireless communication chip and a network communication device such as a wired or wireless network adapter.

The input unit 120 receives an input from the user of the conference participant terminal 100. To this end, the input unit 120 includes a sound input device 120a, an image input device 120b, and a pointing device 120c. The sound input device 120a includes a wired/wireless microphone and a transmitter for receiving an audio signal, and the image input device 120b includes a device that receives an optical signal such as a camera or a webcam and converts it into image data. The pointing device 120c includes a mouse, a stylus, a touch pad, a touch screen, and a motion detection device based on an acceleration sensor. Although not shown, the input unit 120 may include a keyboard, a physical button, and the like.

The output unit 130 is for generating an output related to visual, auditory or tactile sense, and the output unit 130 in the present embodiment may include, in particular, an image display device 130a and a sound generating device 130b. The image display device 130a may be implemented as a touch screen by forming a layer structure or integrally with the pressure-sensitive or capacitive touch sensing module. Such a touch screen may provide an output interface to the user of the conference participant terminal 100. In addition, the touch screen may function as the input unit 120 in the range of providing an input interface to the user of the conference participant terminal 100. The sound generating device 130b includes various devices for outputting audio signals, such as a speaker, a handset, a headphone, and a wireless audio module. Although not shown, the output unit 130 may include a light emitting device that provides visual feedback.

The storage unit 140 stores various types of data, commands, and/or information. The storage unit 140 may store one or more programs for providing a method for providing a remote conference according to embodiments of the present invention. In addition, the storage unit 140 may store content and related data that can be shared among conference participants during the progress of the remote conference. The storage unit 140 may temporarily or non-temporarily store data transmitted from an external device or input by a user, or an operation result of the control unit 150. The storage unit 140 is a nonvolatile memory such as a ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, etc., a hard disk, a removable disk, an SD memory card, or the present invention. Any type of computer-readable recording medium well known in the art to which it belongs may be included. Meanwhile, in FIG. 1, the storage unit 140 is shown as a separate configuration, but may be integrated with the control unit 150 to be described later and provided in the conference participant terminal 100.

The controller 150 controls the overall operation of each component of the conference participant terminal 100. The control unit 150 includes a CPU (Central Processing Unit), MPU (Micro Processor Unit), MCU (Micro Controller Unit), Mobile Processor (Mobile Processor), or any type of processor well known in the technical field of the present invention. Can be configured. In addition, the controller 150 may perform an operation on at least one application or program for executing the method according to the embodiments of the present invention.

The control unit 150 obtains a user input from the input unit 120 of the conference participant terminal 100, and in response, displays a graphic element corresponding to the user input on the video display device 130a, etc., and transmits data related to the user input to the communication unit. It can be transmitted to the other conference participant terminal 100 through (110). In addition, the control unit changes the display attribute of the GUI element displayed on the video display device 130a, based on the result of determining whether the user input corresponds to a predetermined gesture, and transmits data related thereto to the other conference participant terminals 100. I can. Detailed operations of the conference participant terminal 100 under the control of the controller 150 will be described later with reference to FIGS. 3 to 9.

The configuration of the conference participant terminal 100 has been described above with reference to FIG. 2. Even if it is not described in the description with reference to FIG. 2, the technical idea understood through other embodiments of the present invention to be described later with reference to FIGS. 3 to 9 is related to the configuration and operation of the conference participant terminal 100 according to the present embodiment. It should be noted that it may be reflected.

3 is a flowchart illustrating a method of providing a remote conference according to another embodiment of the present invention. The method for providing a remote conference according to the present embodiment may be performed in a computing device. For example, the computing device may be the conference participant terminal 100 described with reference to FIGS. 1 and 2. In interpreting the method for providing a teleconference according to the present embodiment, the embodiments described with reference to FIGS. 1 and 2 may be reflected. Hereinafter, when the subject of each operation included in the method for providing a teleconference according to the present embodiment is omitted, the subject may be interpreted as being a computing device.

Hereinafter, a method for providing a teleconference according to the present embodiment will be described with reference to the flowchart of FIG. 3 and exemplary user interfaces illustrated in FIGS. 4A to 4C, 5A to 5D, and FIG. 6.

First, a user interface 400 for a remote conference is displayed on a display such as an image display device 130a, and a plurality of first graphic elements 410a to 410e and 420a to 420c corresponding to a plurality of conference participants are provided. It is displayed in the first area 402 in the user interface 400 (step S300). The first graphic elements 410a to 410e and 420a to 420c are profile images for identifying each meeting participant, and are captured in real time by the image input device 120b provided in the terminal 100 of each meeting participant. Includes video and the like.

Referring to FIG. 4A, in some embodiments, the first graphic elements 410a to 410e may be arranged to be spaced apart from each other by the same distance from the virtual point 406 in the first area 402. For example, the first graphic elements 410a to 410e may be arranged in a circle with equal intervals on the circumference of the virtual circle. As another example, the first graphic elements 410a to 410e may be arranged on a virtual regular polygonal vertex having as many vertices as the number of conference participants. In some embodiments, the first graphic elements 410a to 410e may have a circular border shape.

Referring to FIG. 4C, in some other exemplary embodiments, the first graphic elements 420a to 420d may each have a rectangular shape and may be arranged to have a grid shape as a whole.

In this embodiment, the first graphic elements 410a to 410e and 420a to 420d may be fixed and disposed in the first area 402 of the user interface 400 during the teleconference. For example, irrespective of transmission and reception of various data such as messages exchanged between a plurality of conference participants, the first graphic elements may maintain a displayed state in a fixed position.

In some embodiments, when a participant is added or excluded from the teleconference, the first graphic elements may be rearranged to reflect the increase or decrease of the participant. For example, when one conference participant is added while the first graphic elements 410a to 410e corresponding to five conference participants are displayed as shown in FIG. 4A, as shown in FIG. 4B The first graphic elements 410a to 410f corresponding to the six conference participants may be rearranged in the first area 402.

Referring back to FIG. 3, in step S310, a user input regarding expression of an opinion may be obtained by the input unit 120 of the conference participant terminal 100. The expression of opinion includes various expressions of opinions, judgments, facts, emotions, feelings, etc. of participants in the conference. For example, the expression of opinion includes an expression of wanting to ask a question about a content being discussed or explained at a meeting, an expression of agreeing or disagreeing with an agenda, an expression of an affirmation or denial of a fact, and the like. In addition, the above expressions are expressions of empathy or disagreement with respect to the subjective opinions of other participants, expressions about subjective feelings that are liked or disliked, expressions of wanting to applaud the opinions or facts presented, and the content being explained is difficult or easy to understand. Includes comments, etc. In addition, the expression of opinion may be an expression of an opinion of selecting one of several options.

In step S310, a user input regarding expression of an opinion may be obtained by various means provided in the input unit 120. For example, the user input is a voice input through the microphone 120a, a character, a shortcut key, or an emoji input through the keyboard, an image input through the image input device 120b, a selection or gesture input through the pointing device 120c, etc. Includes.

Referring to FIG. 6, in some embodiments, a user interface that allows a user to select one of a plurality of pre-set expression elements 610, 620, 630a, and 630b may be provided to a user. A user interface comprising selectable expression elements may be provided, for example, in the second area 404.

In some embodiments, the plurality of expression elements includes a plurality of emojis 610 or emoticons. Emoji is an image-based visual expression that implicitly conveys various emotions and intentions. An emoji is a symbol that simplifies objects, gestures, and facial expressions through an image or a combination of images and characters, and is one of the means of expression used to simply convey thoughts and emotions. In some embodiments of the present invention, a plurality of emojis 610 selectable by the user are provided on the screen as expression elements, and the user input regarding the expression is obtained by receiving an emoji selection from the user through an input means. Can be. In some other embodiments, by receiving a predefined shortcut key or an abbreviation corresponding to an emoji on a keyboard, a user input for expression of an opinion may be obtained.

In some embodiments, the plurality of expression elements includes preset text phrases 620. For example, a plurality of text phrases 620 selectable by the user such as "like", "dislike", "approve", and "disapprove" are provided on the screen as an expression element, and the text phrase is sent from the user through an input means. By being selected, a user input regarding the expression of an opinion can be obtained.

In some embodiments, the plurality of pseudo-expression elements includes a plurality of selection options 630a and 630b. For example, it may include selection options such as "YES", "NO", and/or "option 1" to "option 5", and the like.

Step S320 will be described again with reference to FIG. 3. In step S320, in response to obtaining a user input regarding the expression of intention, the position of the first graphic element corresponding to the user among the plurality of first graphic elements 410a to 410f and 420a to 420d corresponds to the expression of intention. The second graphic element to be displayed is displayed.

In some embodiments, the second graphic element may be an emoji selected by the user in step S310. Referring to FIG. 5A, an emoji 430a selected by a user is displayed to at least partially overlap with a first graphic element 420a corresponding to the user.

In some embodiments, the second graphic element may be a translucent layer having a color corresponding to an emoji, text, or a selection option selected by the user in step S310. In this case, the corresponding color may be a color that abstractly expresses the intention or emotion that the user intends to express through an image, text, or a selection option, or a color that intuitively conveys such intention or emotion. For example, if the user selects the expression “like” or “approve”, the corresponding color may be green, and if the user selects the expression “dislike” or “disapprove”, the corresponding color is red It can be color. Referring to FIG. 5B, the translucent layer is displayed so as to overlap the position of the first graphic element 420a corresponding to the user.

In some embodiments, the second graphic element may be a text phrase selected and input by a user from among the plurality of expression elements 620 in step S310. In addition, the second graphic element may be a text phrase input by a user by typing through an input device such as a keyboard. Referring to FIG. 5C, the text phrase is displayed to at least partially overlap with a first graphic element 420a corresponding to a user of the computing device. Although not shown, in some other embodiments, the text phrase may be displayed adjacent to the first graphic element 420a in the form of a speech balloon.

In an embodiment of the present invention, based on inputs respectively obtained from a plurality of conference participants, first graphic elements corresponding to conference participants may be rearranged. 5D is a diagram illustrating an example in which first graphic elements are rearranged when a plurality of conference participants inputs selecting one of two selection options. For example, if some of the participants in multiple meetings select and enter the option "YES", the rest of the participants select and enter the option "NO". The first graphic elements 410c and 410e constitute the first group 502 and the first graphic elements 410a, 410b, 410d, and 410f corresponding to the participant who input "NO" are the second group 504 ) Can be rearranged to construct.

Step S330 will be described again with reference to FIG. 3. In some embodiments, the second graphic element displayed in step S320 may be displayed for a preset time and then removed. Specifically, in step S330, it is determined whether a preset time has elapsed after the second graphic element is displayed.

If the preset time has elapsed, the process proceeds to step S340 and the second graphic element is removed. In some embodiments, the second graphic element may be immediately removed from the screen after a preset time elapses. In some other embodiments, the second graphic element gradually decreases in size or length as time passes, eventually reaches zero size or length, and may be removed from the screen. In still other embodiments, the second graphic element may be removed from the screen while gradually increasing its transparency over time and eventually becoming completely transparent.

In this embodiment, the second graphic element is removed immediately or gradually after maintaining the displayed state for a preset period of time, so that the participants leading the conference, such as the presenter, allow other conference participants to finally express their opinions. It makes it possible to intuitively recognize when a point in time was. For example, a conference participant who recently provided feedback on a content being discussed or explained in a teleconference, such as a question hope, sympathy, and dissent, etc., and a conference participant who did not, can be easily identified by the presenter.

Meanwhile, in some embodiments, an active state of each of the plurality of conference participants may be displayed based on the time point of the final expression input of each of the plurality of conference participants. Here, the active state visually indicates whether there has been an expression of an opinion for a certain time. For example, by displaying the color of the first graphic element corresponding to the meeting participant who has not expressed any intention for a certain period of time in black and white, the participant who leads the meeting, such as a presenter, can identify participants who may have lost interest in the content being discussed. It becomes easy to grasp.

So far, a method for providing a teleconference according to an embodiment of the present invention has been described with reference to FIGS. 3, 4A to 4C, 5A to 5D, and 6.

In the conventional online teleconference providing method, it is difficult to intuitively distinguish the voices of participants compared to offline in-person meetings. In addition, in the conventional teleconference providing method, unlike offline face-to-face meetings, smooth communication is difficult when multiple participants' voice utterances occur at the same time, and it is impossible to communicate intentions and emotions by non-verbal means such as facial expressions, gestures, attitudes, and hands. Do. Due to such factors, the participants who are currently leading the discussion in the teleconference or other participants who are not the presenters tend to be reluctant to express or give feedback on the contents of the discussion in order not to interfere with the flow of the discussion. On the other hand, from the perspective of the participant or presenter who is leading the discussion, it is difficult to obtain feedback from participants expressed by non-verbal means such as facial expressions, gestures, attitudes, hand gestures, etc., unlike in offline meetings.

According to the present embodiment, by providing a user interface that allows a user to select any one of a plurality of expression elements 610, 620, 630a, and 630b, and obtaining a user input (step S310), a conventional offline In face-to-face meetings, participants in the teleconference can express their thoughts, opinions, and emotions expressed through non-verbal means such as facial expressions, gestures, attitudes, and applause.

In addition, according to the present embodiment, a first graphic corresponding to each conference participant is visualized as a second graphic element such as emoji, color, and text (refer to FIGS. 5A to 5D) by visualizing the expression of the opinion input by the participant of the teleconference. By displaying the element at the location (step S320), the presenter or the like can intuitively recognize the expression of the conference participant's opinion without interrupting or interfering with the main discussion through voice in the teleconference.

In addition, according to the present embodiment, by visually rearranging the first graphic elements corresponding to the plurality of conference participants based on the response contents selected and input by each of the plurality of conference participants (see FIG. 5D), a limited number of The opinions of each participant on the option, or affirmative or disagreeable opinions, etc., can be easily identified by the participants of the conference, including the presenter. This enables the process of gathering the opinions of participants in an offline face-to-face meeting, for example, in a way that is easier and more intuitive in an online teleconference.

In addition, according to the present embodiment, by displaying the active status of each of the plurality of conference participants based on the final expression input time point of each of the plurality of conference participants, participants who lead the conference, such as the presenter, are interested in the conference. Participants who may have dropped out can be easily identified.

Hereinafter, a method for providing a remote conference according to another embodiment of the present invention will be described with reference to the flowchart of FIG. 7 and FIG. 8. In the present embodiment, in a remote conference in which presentation target content is shared online with participants, such as a web conference or online seminar, the presenter can intuitively recognize the part that the presenter wants to emphasize on the presentation target content. Provides a way to make it dictate.

First, in step S700, a user interface for a teleconference is displayed on a display such as the video display device 130a, and presentation target content shared among a plurality of teleconference participants is displayed on the display. Also, a pointing element is displayed on the display. The pointing element is a graphic element, such as an arrow-shaped mouse pointer, used by a user to interact with content or GUI objects displayed on the display. In general, the display property of the pointing element may be changed according to the type and state of the GUI object or the content of the portion where the pointing element is located on the display.

In step S710, a first user input is obtained. The user input may be obtained through the pointing device 120c or the like described with reference to FIG. 2. When the pointing device 120c is a mouse, a stylus, a touch pad, or a touch screen, the user input may be data representing a two-dimensional movement of the pointing device 120c. When the pointing device 120c is a motion detection device based on an acceleration sensor or the like, the user input may be data representing a three-dimensional movement of the pointing device 120c.

In step S720, it may be determined whether the first user input corresponds to the first gesture. In some embodiments, the first gesture is a gesture of reciprocating the pointing element horizontally or left and right within a preset distance by at least a preset number of times within a preset time. In some other embodiments, the first gesture is a gesture of reciprocating the pointing element vertically or vertically within a preset distance by at least a preset number of times within a preset time. In some other embodiments, the first gesture is a gesture of circularly moving the pointing element more than a preset number of times.

If it is determined in step S720 that the first user input does not correspond to the first gesture, the process returns to step S710 to obtain a first user input.

When it is determined in step S720 that the first user input corresponds to the first gesture, in step S730, the display attribute of the pointing element may be changed from the first state to the second state. The display properties of the pointing element include the shape, size, and color of the pointing element.

In some embodiments as shown in FIG. 8A, when the first user input corresponds to a gesture of moving a pointing element horizontally or horizontally reciprocating within a certain distance, the shape of the pointing element may be changed from an arrow to a finger. have.

In some other embodiments as shown in FIG. 8B, when the first user input corresponds to a gesture of moving the pointing element horizontally or horizontally or reciprocally within a certain distance, the size of the pointing element may increase or decrease. . In this case, when the size of the pointing element is less than the threshold value, the size of the pointing element may increase, and when the size of the pointing element reaches the threshold value, the size of the pointing element may decrease.

In some embodiments as shown in FIG. 8D, when the first user input is a gesture of moving a pointing element vertically or vertically reciprocating within a certain distance, the color of the pointing element may be changed from black to red. .

Subsequently, data indicating the change in the display attribute of the pointing element as described above is transmitted to the computing devices of the plurality of meeting participants, and the display attribute of the pointing element displayed on the display of the computing device of the plurality of meeting participants is changed from the first state. It changes to the second state.

This will be described with reference to FIG. 7 again. After the display attribute of the pointing element is changed from the first state to the second state, a second user input is obtained in step S740. Subsequently, in step S750, it may be determined whether the second user input corresponds to the second gesture. In some embodiments, the second gesture is a gesture of reciprocating the pointing element horizontally or left and right within a preset time and exceeding a preset distance by at least a preset number of times.

If it is determined in step S750 that the second user input does not correspond to the second gesture, the process returns to step S740 to obtain a second user input.

If it is determined in step S750 that the second user input corresponds to the second gesture, in step S760, the display attribute of the pointing element may be changed from the second state to the first state. As illustrated in FIG. 8C, when the second user input corresponds to a gesture of reciprocating the pointing element horizontally or left and right beyond a certain distance, the shape of the pointing element may be returned from a finger to an arrow.

Subsequently, data indicating a change in the display attribute of the pointing element as described above is transmitted to the computing devices of the plurality of conference participants, and the display attribute of the pointing element displayed on the display of the computing device of the plurality of conference participants is changed from the second state. It changes to the first state.

Until now, a method for providing a remote conference according to an embodiment of the present invention has been described with reference to FIGS. 7 and 8A to 8D.

In the conventional method of providing a remote conference, when a presenter of a web conference or online seminar wants to highlight or direct (point) a specific part of the content to be presented, it is intended to be emphasized using a drawing tool provided by the remote conference program. You can underline the part, or move the mouse pointer repeatedly and quickly around the part. However, it is difficult to attract sufficient attention from teleconference participants through such a method. In particular, compared to pointing at a specific part of the presentation screen with a physical indicator bar in an offline face-to-face meeting, the degree of attention that can be obtained from conference participants through the above-described conventional method is very low.

According to the present embodiment, when a presenter of a web conference or online seminar wants to highlight or indicate (pointing) a specific part of the content to be presented, a pointing device displayed on the screen is performed through a simple gesture input using a pointing device. By simply and quickly changing the display properties such as the shape, size, and color of an element, it is possible to attract the attention of other conference participants. In other words, in an offline face-to-face meeting, it is possible to achieve an effect similar to pointing a specific part of the presentation screen with a physical pointer.

Hereinafter, a method for providing a remote conference according to another embodiment of the present invention will be described with reference to the flowchart of FIG. 9. In the present embodiment, in a teleconference where presentation target content is shared online with participants, such as a web conference or an online seminar, a method of allowing a pointing element to automatically indicate a portion of the presentation target content that a presenter describes with an audio is provided. to provide.

First, in step S900, a user interface for a teleconference is displayed on a display such as the video display device 130a, and presentation target content shared among a plurality of teleconference participants is displayed on the display. Also, a pointing element is displayed on the display.

In step S910, a voice signal is obtained through the sound input device 120a provided in the computing device of the presenter of the teleconference.

In step S920, automatic speech recognition (ASR) is performed on the acquired speech signal, and text is extracted from the speech signal. In some embodiments, the automatic speech recognition processing may be performed through a process of extracting features from a speech signal, prediction through machine learning, classification, clustering, and association. The automatic speech recognition processing may be performed by various speech recognition techniques well known in the art.

In step S930, a portion matching the extracted text in the presentation target content is determined. More specifically, this process includes natural language analysis (NLP), context analysis, and comparative analysis with text extracted through the speech signal. It can be done through a process. In this case, if there are two or more portions of the content that match the extracted text, the portion closest to the location of the content currently displayed on the screen may be determined as the matching portion.

In step S940, the content is moved so that a part of the presentation target content determined to be matched is displayed on the screen, and in step S950, the pointing element displayed on the screen is moved so that the pointing element points to the matched position. do. Subsequently, in step S960, data indicating movement of the pointing element and movement of the content is transmitted to the computing devices of the plurality of conference participants.

According to the present embodiment, in a teleconference where presentation target content is shared online with participants, such as a web conference or online seminar, a speaker's voice is recognized and a matching part of the content of the presentation target content is automatically analyzed and The decision is made, and the pointing element is automatically moved to point to the corresponding position, so that what the presenter is describing or what the presenter is referencing in the content can be easily communicated to other conference participants.

Embodiments of the present invention described with reference to FIGS. 3 to 9 so far may be implemented as computer-readable codes on a computer-readable medium. The computer-readable recording medium is, for example, a removable recording medium (CD, DVD, Blu-ray disk, USB storage device, removable hard disk) or a fixed recording medium (ROM, RAM, computer-equipped hard disk). I can. The computer program recorded on the computer-readable recording medium may be transmitted to another computing device through a network such as the Internet and installed in the other computing device, thereby being used in the other computing device.

Hereinafter, a configuration of an exemplary computing device capable of implementing a system according to various embodiments of the present disclosure will be described.

A computing device capable of implementing a system according to various embodiments of the present disclosure may include a processor, a memory, a network interface, a storage and a processor, a memory, and a system bus connecting the network interface and the storage.

A communication module, an input module, an output module, and a control module may be stored in the storage. Each of the modules is software that operates individually, and the binary is loaded into a memory and then executed by the processor, so that the computing device performs data processing and control-related operations.

In the above, even if all the constituent elements constituting an embodiment of the present invention have been described as being combined into one or operating in combination, the technical idea of the present invention is not necessarily limited to this embodiment. That is, as long as it is within the scope of the object of the present invention, one or more of the components may be selectively combined and operated.

Although the operations are illustrated in a specific order in the drawings, it should not be understood that the operations must be executed in the specific order shown or in a sequential order, or all illustrated operations must be executed to obtain a desired result. In certain situations, multitasking and parallel processing may be advantageous. Moreover, the separation of the various components in the above-described embodiments should not be understood as necessarily requiring such separation, and the described program components and systems may generally be integrated together into a single software product or packaged into multiple software products. It should be understood that there is.

Although the embodiments of the present invention have been described with reference to the accompanying drawings, those of ordinary skill in the art to which the present invention pertains to implement the present invention in other specific forms without changing the technical idea or essential features. I can understand that it can be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not limiting. The scope of protection of the present invention should be interpreted by the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the technical ideas defined by the present invention.

Claims (23)

A method performed on a computing device of an online conference participant,
Displaying a plurality of first graphic elements respectively corresponding to a plurality of online conference participants in a first area on a display of the computing device;
Obtaining an input regarding the expression of the opinion from a user of the computing device; And
In response to obtaining the input, displaying a second graphic element corresponding to the expression of the opinion at a location of the first graphic element corresponding to the user among the plurality of participants
Including,
The plurality of first graphic elements are fixedly arranged at different positions in the first area,
Way. The method of claim 1,
The first graphic elements are arranged to be spaced apart from each other by the same distance from an imaginary point in the first area,
Way. The method of claim 2,
In response to adding a participant to the online meeting, rearranging the plurality of first graphical elements,
Way. The method of claim 1,
Obtaining an input on the expression of the opinion from the user,
Displaying an interface for selecting one of a plurality of expression elements in a second area on the display
Containing,
Way. The method of claim 4,
The step of rearranging the plurality of first graphic elements, based on the input respectively obtained from the plurality of online conference participants,
Way. The method of claim 1,
After the step of displaying the second graphic element, removing the displayed second graphic element when a preset display time elapses
Further comprising,
Way. The method of claim 6,
Decreasing the size of the displayed second graphic element or increasing the transparency of the second graphic element as time elapses after displaying the second graphic element
Further comprising,
Way. The method of claim 1,
The displaying of the second graphic element includes displaying an emoji corresponding to the expression of the intention so as to at least partially overlap the first graphic element,
Way. The method of claim 1,
The displaying of the second graphic element includes displaying a translucent layer of a color corresponding to the pseudo expression by overlapping the position of the first graphic element.
Way. The method of claim 1,
The displaying of the second graphic element includes displaying a text corresponding to the expression to be at least partially overlapped with the first graphic element,
Way. The method of claim 1,
Further comprising the step of displaying an active state determined based on a final opinion expression input time point of each of the plurality of participants,
Way. A method performed on a computing device of a first participant of an online conference, comprising:
Displaying content on a display of the computing device;
Obtaining a first user input from a user of the computing device; And
In response to determining that the first user input corresponds to a first gesture,
Changing a display attribute of a pointing element for indicating portions of the content from a first state to a second state,
Transmitting data instructing to change the display attribute of the pointing element to at least one of a computing device of the second participant of the online conference and a server managing the online conference
How to include. The method of claim 12,
The first gesture is a gesture of reciprocating the pointing element in a horizontal direction within a preset time within a preset number of times or more,
Way. The method of claim 12,
The first gesture is a gesture of reciprocating the pointing element in a vertical direction within a preset time within a preset number of times or more,
Way. The method of claim 12,
Changing the display attribute of the pointing element includes changing at least one of a shape, a size, and a color of the pointing element.
Including
Way. The method of claim 15,
The step of changing the display attribute of the pointing element,
When the size of the pointing element is less than the threshold value, the pointing element is enlarged,
When the size of the pointing element is greater than or equal to a threshold, reducing the pointing element
Containing,
Way. The method of claim 12,
After the transmitting step,
Obtaining a second user input from a user of the computing device; And
In response to determining that the second user input corresponds to a second gesture, changing the display attribute of the pointing element from a second state to a first state
How to further include. The method of claim 17,
The first gesture is a gesture of reciprocating the pointing element in a horizontal direction within a threshold distance,
The second gesture is a gesture of reciprocating the pointing element in a horizontal direction beyond the threshold distance,
Way. A method performed on a computing device of a first participant of an online conference, comprising:
Extracting text by analyzing the voice signal obtained from the user of the computing device;
Analyzing the content displayed on the display of the computing device to determine a location within the content that matches the text;
Moving a pointing element to the matching location in the content; And
Transmitting data instructing the movement of the pointing element to at least one of a computing device of a second participant of the online conference and a server managing the online conference
How to include. The method of claim 19,
Moving the content so that the matching location is displayed on the display
Further comprising,
Way. The method of claim 20,
The determining step,
When there are two or more locations that match the text, determining a location closest to the location in the content displayed on the display as a matched location
Further comprising,
Way. The method of claim 20,
Transmitting data instructing the movement of the content to at least one of a computing device of the second participant and a server managing the online conference
Further comprising,
Way. The method of claim 19,
The determining step is based on a natural language analysis result of the text included in the content,
Way.

Images