KR101879349B1 - Method, system and non-transitory computer-readable recording medium for assisting communication - Google Patents

Abstract

The present invention relates to a method, system and non-transitory computer readable recording medium for supporting communication.
According to an aspect of the present invention there is provided a method for supporting communication comprising the steps of: obtaining information about a user's attitude and information about a voice uttered by the user; And determining a field of communication through which the voice will be delivered, of at least one peripheral region in which there is at least one object capable of interacting with the user.

Description

[0001] METHOD, SYSTEM AND NON-TRANSITORY COMPUTER READABLE RECORDING MEDIUM FOR ASSISTING COMMUNICATION [0002] BACKGROUND OF THE INVENTION [0003]

The present invention relates to a method, system and non-transitory computer readable recording medium for supporting communication.

In recent years, techniques have been introduced to support a user to perform voice-based communication with another user in a real space or a virtual reality space, or to allow a user to perform voice-based control on a device, a virtual object, and the like.

In recent years, interest in Augmented Reality (AR) and Virtual Reality (VR) has increased and research and development in related technology fields have been actively carried out. As a result, There is a growing demand for technologies that enable smooth communication with possible objects.

Conventionally, a technique has been proposed in which a voice transmitted by a user is delivered only to a target object specified in advance by a user or a system, or a technique in which a user talks the name of a target object separately or explicitly indicates a target object (for example, Quot ;, "Turn off the lighting of the living room "," Please lower the air conditioner temperature of 301 ", etc.).

1 is a diagram illustrating an example of a communication support method according to the prior art. 1, the user specifies the target object 10 as an object of communication by moving the pointer 11 to indicate the target object 10 (refer to FIG. 1 (a)), The user can specify the target object 10 as an object of communication by fitting his / her line of sight to the target object 10 (see FIG. 1 (b)).

However, according to the related art as described above, it is very difficult for the user to flexibly or adaptively determine the range of the target object to which the user's voice is to be transmitted, or when the user communicates, There is a limitation in that the flexibility of communication is degraded or the convenience of the user is poor in a practical use environment.

Accordingly, the inventor of the present invention has determined that a user's voice can be transmitted to a user's intention by determining an area (also referred to as a communication area) to which a user's voice is to be transmitted with reference to the attitude of the user and the strength of the voice uttered by the user. So that it can be accurately transmitted to a region or an object.

It is an object of the present invention to solve all the problems described above.

Further, the present invention is a method for acquiring information about a user's attitude and information about a voice uttered by a user, and referring to the information obtained above, a peripheral region in which at least one object capable of interacting with the user exists A method, system and non-transitory computer readable recording medium for enabling a user's voice to be accurately communicated to an intended area or target by determining a field of communication through which a user's voice in the middle will be communicated The other purpose is to provide a medium.

In order to accomplish the above object, a representative structure of the present invention is as follows.

According to an aspect of the present invention there is provided a method for supporting communication comprising the steps of: obtaining information about a user's attitude and information about a voice uttered by the user; And determining a field of communication through which the voice will be delivered, of at least one peripheral region in which there is at least one object capable of interacting with the user.

According to another aspect of the present invention, there is provided a system for supporting communication, comprising: an information acquisition unit for acquiring information about a user's attitude and information about a voice uttered by the user; And a communication area determining unit that determines a field of communication to which the voice is to be transmitted from among peripheral areas in which at least one object capable of interacting with the user exists.

In addition, there is provided another method, system, and non-transitory computer readable recording medium for recording a computer program for carrying out the method for implementing the present invention.

According to the present invention, based on a field of communication determined by referring to a direction of a user and a strength of a voice uttered by a user, a user's voice is accurately transmitted to a region or an object intended by the user So that the user can communicate with the intended object naturally without experiencing the inconvenience of specifying the target of communication in a practical use environment.

Further, according to the present invention, by further utilizing situation information such as a filter set by a user, a gesture of a user, a voice of a user, context information about a user, and the like, The object can be specifically specified, so that the communication between the user and the object around the user can be performed more accurately and easily.

1 is a diagram illustrating an example of a communication support method according to the prior art.
2 is a diagram illustrating an exemplary internal configuration of a communication support system according to an embodiment of the present invention.
FIG. 3 through FIG. 5 illustrate a configuration for determining a communication area according to an embodiment of the present invention.
6 and 8 are diagrams illustrating various embodiments in which a configuration for determining a communication area can be utilized according to an embodiment of the present invention.
9 to 12 are views illustrating a configuration for specifying a target object among objects existing in a communication area according to an embodiment of the present invention.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention.

Composition of communication support system

Hereinafter, the internal configuration of the communication support system 200 that performs an important function for the implementation of the present invention and the functions of the respective components will be described.

2 is a diagram illustrating an internal configuration of a communication support system according to an embodiment of the present invention.

2, a communication support system 200 according to an exemplary embodiment of the present invention includes an information obtaining unit 210, a communication area determining unit 220, an object specifying unit 230, a database 240, A communication unit 250, and a control unit 260. According to an embodiment of the present invention, the information obtaining unit 210, the communication area determining unit 220, the object specifying unit 230, the database 240, the communication unit 250, Some of which may be program modules that communicate with an external system (not shown). These program modules may be included in the communication support system 200 in the form of an operating system, an application program module, and other program modules, and may be physically stored on various known storage devices. These program modules may also be stored in a remote storage device capable of communicating with the communication support system 200. [ These program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types as described below in accordance with the present invention.

Although described above with respect to the communication support system 200, the description is exemplary and at least some of the components or functions of the communication support system 200 may be carried by the user, It will be apparent to those skilled in the art that it may be realized within the user device 100 that is worn on the user device 100 (e.g., head, eyes, etc.) In some cases, all functions and all components of the communication support system 200 may be fully implemented within the user device 100 or entirely within the user device 100.

According to an embodiment of the present invention, the user device 100 may include a head mounted display (HMD) capable of implementing an Augmented Reality (AR) or a virtual reality (VR) And a near eye display, and the user device 100 may include a microphone module (not shown) for recognizing the user's voice. Also, according to one embodiment of the present invention, the user device 100 may be a wearable speech recognition device that does not involve a display module, and more specifically, a microphone module (not shown) And a microphone module (not shown) in the form of an ear plug.

Meanwhile, as will be described later, according to an embodiment of the present invention, the user device 100 can acquire information necessary for determining the communication area of the user (i.e., information about the posture of the user and the voice uttered by the user (Not shown) capable of recognizing or acquiring information (e.g.

First, according to an embodiment of the present invention, the information obtaining unit 210 may perform a function of obtaining information on the posture of the user. Specifically, the information obtaining unit 210 according to an exemplary embodiment of the present invention may obtain information on a posture of a user's body part related to voice communication, for example, It is possible to obtain information on the direction in which the mouth portion is pointing.

To this end, according to an embodiment of the present invention, a user device 100 held by a user or an external device (not shown) separated from a user and able to observe a user's posture, And at least one sensing module (not shown) that can acquire physical information about the posture or movement of the site. As an example of such a sensing module, known components include a motion sensor, an acceleration sensor, a gyroscope, a magnetic sensor, a positioning module (a GPS module, a beacon-based positioning module, etc.), a barometer, .

Also, according to an embodiment of the present invention, the user device 100 possessed or worn by the user can acquire physical information about the posture or movement of the user based on the biometric information acquired from the user's body And at least one sensing module. An example of such a sensing module is an electromyogram signal measuring device and the like.

In addition, according to an embodiment of the present invention, the information obtaining unit 210 may perform a function of obtaining information on a voice uttered by a user. More specifically, the information obtaining unit 210 according to an embodiment of the present invention can obtain information on the intensity of a voice uttered by the user, the voice recognition result on the voice uttered by the user, and the like.

Next, in accordance with an embodiment of the present invention, the communication area determination unit 220 determines, based on the information on the obtained posture of the user and the information on the voice uttered by the user, The user can perform a function of determining a communication area in which the user's voice is to be transmitted among the surrounding areas in which at least one object exists. More specifically, the communication area determination unit 220 according to an embodiment of the present invention can determine the communication area with reference to the direction in which the head or mouth part of the user is pointing and the strength of the voice uttered by the user have.

Also, according to an embodiment of the present invention, the communication area determination unit 220 may perform a function of determining at least some of the attributes of the communication area with reference to the setting information input from the user.

FIG. 3 through FIG. 5 illustrate a configuration for determining a communication area according to an embodiment of the present invention.

Referring to FIG. 3, the communication area 300 according to an exemplary embodiment of the present invention may be determined as an area having a quadrangular pyramid shape having a vertex at a reference point of the user's mouth. More specifically, according to one embodiment of the present invention, the height (or depth) direction 310 of the communicating area 300 having a quadrangular pyramid shape can be determined according to the direction in which the user's head or mouth is looking So that the communication area 300 can be oriented in the direction intended by the user. In addition, according to an embodiment of the present invention, the height 320 of the quadrangular pyramid-shaped communication area 300 can be determined according to the intensity of the voice uttered by the user, (That is, the communication area 300 is limited only to a place close to the user), whereas when the user speaks a lot, the height of the communication area 300 can be increased (300) extends far from the user).

3, the horizontal and vertical lengths of the bottom surface of the quadrangular pyramid-shaped communication area 300 may be determined according to values arbitrarily set by the user. For example, a user who wishes to widen the range of communication can set the arbitrary set value to a larger value, thereby widening the communication area 300, and a user who wants to narrow the communication range can set any of the above settings The communication area 300 can be narrowed by setting a small value.

4, the communication area 400 according to an exemplary embodiment of the present invention may be determined as a region having a conical shape having vertexes at a reference point of a user's mouth. More specifically, according to one embodiment of the present invention, the height direction 410 and height 420 of the communication area 400 are respectively the direction in which the head or mouth of the user is looking, As shown in FIG. Also, according to an embodiment of the present invention, the widths 430 and 440 of the bottom surface of the conical communication area 400 may be determined according to values arbitrarily set by the user.

5, the communication area 500 according to an exemplary embodiment of the present invention may be determined as an area having an ellipsoidal shape focusing on a reference point of a user's mouth. More specifically, according to one embodiment of the present invention, the direction 510 of the major axis and the length 520 of the major axis of the umbrella-shaped communication area 500 are respectively Direction and the strength of the speech uttered by the user. In addition, according to an embodiment of the present invention, the ratio between the length 520 of the major axis, the length 530 of the minor axis, and the length 550 of the width 550 of the umbrella- (I.e., 520: 530: 540: 550) between the length 530 of the major axis 530, the length 530 of the minor axis, the length 540 of the height, Can be determined according to a value arbitrarily set.

However, it should be noted that the method of determining a communication area according to the present invention is not necessarily limited to those listed above, but may be modified within the scope of achieving the object of the present invention. As another example, in the embodiments of Figs. 3 to 5, some attributes of the communication area (i.e., the width or length of the bottom surface, the width of the bottom surface, the ratio between the width and the length Etc.) can be used in various situations, such as information about a filter preset by the user, information about a gesture input from the user, information about the intention of the user derived from the speech recognition result of the user's voice, May be determined adaptively (i.e., automatically) based on the information.

6 and 8 are diagrams illustrating various embodiments in which a configuration for determining a communication area can be utilized according to an embodiment of the present invention.

6, a user 610 is included in a communication area 600, which is determined based on a direction of the user himself / herself and the strength of a voice uttered by the user, among a plurality of other users existing in the vicinity of the user It is possible to transmit its voice message to two different users 620 and 630. [

Next, referring to FIG. 7, each of the plurality of users 711, 721, 731, 741, and 751 includes a communication area 712 732, 733, 734, and 735 included in the respective devices 731, 722, 723, 724, and 725, respectively.

Next, referring to FIG. 8, it is assumed that, based on the direction in which the user 810 is looking in the virtual reality (VR) provided through the head-mounted display device 100 and the intensity of the speech uttered by the user, The user 810 can be determined and the user 810 can communicate his or her voice command to the virtual object included in this communication area 830. [

Next, in accordance with an embodiment of the present invention, the object specifying unit 220 may include information on a filter preset by a user, information on a gesture input from a user, A function of specifying at least one target object to which a user's voice is to be transmitted among the objects existing in the communication area with reference to various situation information including information on the user's intention and context information about the user Can be performed.

Here, according to an embodiment of the present invention, a target object may be implemented by a real object such as another user or a real device capable of receiving a voice message or voice command from the user, and a virtual reality or augmented reality, A virtual object such as an avatar or a virtual device capable of receiving a voice message or a voice command can be included.

9 to 12 are views illustrating a configuration for specifying a target object among objects existing in a communication area according to an embodiment of the present invention.

Specifically, according to one embodiment of the present invention, a filter preset by the user can be set based on various conditions that must be satisfied in order to become a target object.

For example, in the embodiment of FIG. 9, setting is made based on a relation between a user and an object (family, friend, etc.), language used by the user or object, age of the user or object, An object 940 that meets the criteria of the above filter among a plurality of objects (i.e., other users) 920 to 950 existing in the communication area 900 of the user 910, It can be determined that the voice message from the mobile terminal 910 is the target object to be delivered.

Also, according to an embodiment of the present invention, information on a user's intention derived from a result of speech recognition of a user's voice includes a keyword (e.g., a name of an object, Type, location of the object, etc.), whether or not the oblique object is used, the identity of the user recognized from the voice, and the like.

10, when the user 1110 directly talks to the location of the object he wants to specify ("Top right"), the communication area 1100 of the user 1110 It can be determined that an object 1150 existing at a position directly spoken by the user 1110 among the plurality of objects 1120 through 1150 existing in the user 1110 is a target object to which the voice message from the user 1110 is to be transmitted.

In another example, when a user calls an object name, the object corresponding to the name may be determined to be the target object, and when the user uses an aberration body, another user who is older or has a higher status than the user It can be determined to be the target object.

According to an embodiment of the present invention, a gesture input from a user may include various operations that a user can perform to communicate with a physical object or a virtual object existing around the user, Can be recognized by various sensing modules such as a sensor, an acceleration sensor, a gyroscope, a magnetic sensor, a positioning module (GPS module, beacon-based positioning module, etc.), a barometer, and a camera.

For example, in the embodiment of FIG. 11, when a user 1210 takes a gesture looking at an object he or she wants to specify, a plurality of objects 1220 (e.g., 1220) existing in the communication area 1200 of the user 1210 , 1230 may be determined to be the target object to which the voice message from the user 1210 is to be delivered.

According to an embodiment of the present invention, the context information about the user includes information on the demographic information of the user, information on the behavior pattern of the user, information on the place or time zone where the user uttered the voice, Information about the user's speech about the object, information about the user's preference for the object, information about the activities performed by the user in the social network service (SNS), information about the conversation the user has made by other users, Information about the social relationship or the ownership relationship between the user and the object, and the like.

For example, in the embodiment of FIG. 12, when the user 1310 showing a behavior pattern sleeping at 12:00 every night speaks a voice command "Turn off" at 11:30 pm one night Some household appliances 1320 and 1330 which may interfere with the sleeping of the household appliances 1320 to 1350 existing in the communication area 1300 of the user 1310 may receive the voice command Object. ≪ / RTI >

In another example, when a user utters a voice command "unlock ", a smart phone owned by a corresponding user among a plurality of smart phones existing in the communication area of the user communicates with a target object ≪ / RTI >

According to an embodiment of the present invention, the database 240 may store various information that can be used in determining the communication area according to the present invention, and a variety of information that can be used in specifying a target object according to the present invention. Information can be stored. Although the database 240 is shown in FIG. 2 as comprising and configured to be included in the communication support system 200, according to the needs of those skilled in the art implementing the present invention, the database 240 includes a communication support system 200, Or may be configured separately. On the other hand, the database 240 in the present invention is a concept including a computer-readable recording medium, and may be a broad database including not only a consultation database but also a data record based on a file system, Even if the set can retrieve the data and extract the data, it can be the database 240 in the present invention.

Next, the communication unit 250 according to an embodiment of the present invention performs a function to allow the communication support system 200 to communicate with an external device.

The control unit 260 according to an embodiment of the present invention includes data between the information obtaining unit 210, the communication area determining unit 220, the object specifying unit 230, the database 240 and the communication unit 250 And the like. That is, the control unit 260 controls the flow of data between the components of the communication support system 200 from outside, thereby controlling the flow of data between the information obtaining unit 210, the communication area determining unit 220, 230, the database 240, and the communication unit 250, respectively.

The embodiments of the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded in a non-transitory computer readable recording medium. The non-transitory computer readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the non-transitory computer-readable recording medium may be those specially designed and constructed for the present invention or may be those known to those skilled in the computer software arts. Examples of non-transitory computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs, DVDs, magneto-optical media such as floppy disks magneto-optical media), and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules for performing the processing according to the present invention, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, Those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims.

Therefore, the spirit of the present invention should not be construed as being limited to the above-described embodiments, and all of the equivalents or equivalents of the claims, as well as the following claims, I will say.

100: User device
200: Communication support system
210: Information obtaining unit
220: Communication area determining unit
230: target specific part
240: Database
250:
260:

Claims (7)

A communication support system for supporting communication, the communication support system comprising an information obtaining unit, a communication area determining unit, and a target specifying unit,
The information acquiring unit acquiring information on the posture of the user and information on the voice uttered by the user,
The communication area determination unit may determine a field of communication to which the voice is to be transmitted, from a surrounding area in which at least one object capable of interacting with the user exists, with reference to the obtained information , And
Determining at least one target object to which the speech is to be delivered, among the objects existing in the determined communication area, by referring to the target specifying unit, the obtained information and the situation information
Lt; / RTI >
Wherein the communication area is a three-dimensional area defined based on the position of the user in a real, augmented reality, or virtual reality,
Wherein the communication area determination unit determines the range of the communication area with reference to a direction in which the head or mouth part of the user is pointing and an intensity of the voice uttered by the user,
Wherein the communication area determination unit adaptively determines a distance to which the voice uttered by the user is transmitted among the attributes of the communication area according to the intensity of the voice uttered by the user,
The communication area determination unit determines the shape of the communication area by further referring to the setting information input from the user
Way. delete The method according to claim 1,
Wherein the communication area determination unit determines at least a part of the attributes of the communication area with reference to the setting information input from the user. delete The method according to claim 1,
The context information includes information on a filter previously set by the user, information on a gesture input from the user, information on the intention of the user derived from the speech recognition result on the speech, and context information on the user Lt; / RTI > is included. A non-transitory computer readable recording medium having recorded thereon a computer program for carrying out the method according to claim 1. A system for supporting communication, comprising:
An information acquisition unit for acquiring information on a posture of a user and information on a voice uttered by the user,
A communication area determination unit for determining a field of communication through which the voice is to be transmitted, from a surrounding area in which at least one object capable of interacting with the user exists, with reference to the obtained information;
Determining at least one target object to which the voice is to be transmitted among the objects existing in the determined communication area by referring to the obtained information and the situation information,
Lt; / RTI >
Wherein the communication area is a three-dimensional area defined based on the position of the user in a real, augmented reality, or virtual reality,
Wherein the communication area determination unit determines a range of the communication area with reference to a direction in which the head or mouth part of the user is pointing and an intensity of a voice uttered by the user,
Wherein the communication area determining unit adaptively determines a distance to which the voice uttered by the user is transmitted among the attributes of the communication area according to the intensity of the voice uttered by the user,
The communication area determination unit determines the shape of the communication area by further referring to the setting information input from the user
system.

Images