KR102025391B1 - Device control according to user's talk position - Google Patents

Abstract

Provides a technique for controlling the device according to the location of the user's speech. In one embodiment, a device control method of an electronic device including a voice based interface may include: receiving a command according to a voice input of a user through the voice based interface; Acquiring location information associated with the voice input when the location of the command is determined as a result of the analysis; And specifying a control target device for controlling an operation corresponding to the command among devices that may be linked to the electronic device using the location information.

Description

DEVICE CONTROL ACCORDING TO USER'S TALK POSITION}

The following description is a technology related to device control according to a user's speech position, and more specifically, a device control method and system for controlling device operation or content provision by further determining a user's speech position in a command corresponding to a voice input; And a computer program stored in a computer readable recording medium for causing the computer to execute the device control method in combination with the computer.

An interface operating based on voice, such as an AI speaker of a home network service, may receive a user's voice input through a microphone and control device operation or content provision based on the received voice input.

For example, Korean Patent Publication No. 10-2011-0139797 (published December 30, 2011) is a technology related to a home media device and a home network system and method using the same. It is possible to provide a home network service using the same second communication network, and discloses a technology that allows a user to control a plurality of multimedia devices in a home through a voice command without a separate button operation.

In the prior arts, when there are a plurality of devices to be controlled, the device can be controlled by speech using a registered device name. However, when there are several devices that play the same role, there is a difficulty in registering each device name separately and remembering all of them. Since it operates mainly on the basis of voice and does not use separate visual information, there is a problem in that a specific request and command form must be structured and expressed in words when selection among various options is required.

A device control method and system that can identify a form of speech that can be easily expressed in colloquial form and control a device that meets the user's intention, and a computer stored in a computer readable recording medium to execute the device control method in a computer in combination with a computer. Provide the program and its recording media.

A device control method and system for understanding the context of a conversation and controlling a device in accordance with a user's intention, and a computer program stored on a computer readable recording medium and the recording medium in combination with a computer for executing the device control method on a computer. to provide.

A device control method and system for controlling a device meeting user intentions based on device usage patterns and history, and a computer program stored in a computer readable recording medium in combination with a computer for executing the device control method on a computer. Provide a record carrier.

A device control method of an electronic device including a voice based interface, the method comprising: receiving a command according to a voice input of a user through the voice based interface; Acquiring location information associated with the voice input when the location of the command is determined as a result of the analysis; And specifying a control target device for controlling an operation corresponding to the command among devices that can be linked to the electronic device using the location information.

In combination with a computer, a computer program stored in a computer readable recording medium for executing the device control method on a computer is provided.

Provided is a computer-readable recording medium, in which a program for executing the device control method on a computer is recorded.

An electronic device comprising: a voice based interface; And at least one processor configured to execute a computer readable command, wherein the at least one processor receives a command according to a voice input of a user through the voice-based interface, and determines a result of analyzing the command. If necessary, the electronic device characterized in that it acquires the location information associated with the voice input, and specifies a control target device for controlling an operation corresponding to the command among the devices that can be linked with the electronic device using the location information. To provide.

According to the command corresponding to the voice input, the user's speech location may be further determined to select a device suitable for the command.

When the command includes an instruction pronoun, the device may be estimated based on the user's location.

By estimating the user's location in the context of a conversation, a device can be estimated that meets the user's intentions.

Based on the device usage pattern and history, a device that matches the user's intention can be estimated.

1 is a diagram illustrating an example of a service environment using a voice-based interface according to an embodiment of the present invention.
2 is a diagram illustrating another example of a service environment using a voice-based interface according to an embodiment of the present invention.
3 is a diagram illustrating an example of a cloud AI platform according to an embodiment of the present invention.
4 is a block diagram illustrating an internal configuration of an electronic device and a server according to an embodiment of the present invention.
5 is a flowchart illustrating an example of a device control method according to an embodiment of the present invention.
6 to 7 illustrate examples of determining a device to be controlled according to a voice input including an instruction pronoun according to an embodiment of the present invention.
8 to 9 are diagrams illustrating examples of determining a control target device according to a voice input in which a position or an instruction target is omitted according to one embodiment of the present invention.
FIG. 10 is a diagram for one example of determining a device to be controlled based on learning a user behavior pattern according to one embodiment of the present invention.

Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings.

The device control system according to embodiments of the present invention may be implemented through an electronic device that provides an interface that operates based on voice. The electronic device may further determine the location of the talker in a command according to the talker's voice input to control device operation or content provision. In this case, when the command includes an instruction pronoun or the command includes an incomplete sentence, the electronic device may estimate a device corresponding to the user's intention based on the location of the user. In addition, the electronic device may estimate a device corresponding to a user's intention by grasping a dialogue context according to a voice input of a talker or grasping a device usage pattern or a device control history.

The device control method according to embodiments of the present invention may be performed through the above-described electronic device. In this case, a computer program according to an embodiment of the present invention may be installed and driven in the electronic device, and the electronic device may perform a device control method according to an embodiment of the present invention under control of the driven computer program. . The computer program described above may be combined with an electronic device implemented as a computer and stored in a computer-readable recording medium for causing a computer to execute a device control method.

1 is a diagram illustrating an example of a service environment using a voice-based interface according to an embodiment of the present invention. In the embodiment of FIG. 1, in a technology of connecting and controlling devices in a home such as a smart home or a home network service, an electronic device 100 that provides an interface operating based on voice may be used by the user 110. An example of controlling the light power of the indoor lighting device 120 connected to the electronic device 100 through the internal network by recognizing and analyzing the voice input “turn off the light” received according to the utterance is shown.

For example, the devices in the home may consume home appliances such as televisions, personal computers (PCs), peripherals, air conditioners, refrigerators, robot cleaners, etc., as well as energy consumption such as water, electricity, and heating and heating devices, in addition to the above-described indoor lighting devices 120. It may include a variety of devices that can be connected and controlled online, such as security devices such as devices, door locks, surveillance cameras and the like. Internal networks also include wired network technologies such as Ethernet, HomePNA, and IEEE 1394, and wireless network technologies such as Bluetooth, Ultra Wide Band (UWB), ZigBee, Wireless 1394, and Home RF. Can be utilized.

The electronic device 100 may be one of devices in the home. For example, the electronic device 100 may be one of devices such as an artificial intelligence speaker or a robot cleaner provided in the home. In addition, the electronic device 100 may be a mobile device of the user 110 such as a smart phone, a mobile phone, a notebook computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a tablet PC, or the like. have. As such, the electronic device 100 is not particularly limited as long as the device includes a function capable of connecting with devices in the home to receive a voice input of the user 110 and control the devices in the home. In addition, according to an embodiment, the mobile devices of the user 110 described above may be included as devices in the home.

2 is a diagram illustrating another example of a service environment using a voice-based interface according to an embodiment of the present invention. FIG. 2 illustrates that the electronic device 100 providing an interface operating based on voice recognizes and analyzes a voice input “today's weather” received according to the user 110's speech from an external server 210 through an external network. An example of obtaining information on today's weather and outputting the obtained information in a voice such as "Weather in today is ..." is shown.

For example, the external network may be a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, or the like. It may include any one or more of the network of.

In the embodiment of FIG. 2, the electronic device 100 may be one of the devices in the home or one of the mobile devices of the user 110. The electronic device 100 may include an external network and a function for receiving and processing a voice input of the user 110. The device is not particularly limited as long as the device includes a function for providing the user 110 with a service or content provided by the external server 210 by accessing the external server 210.

As such, the electronic device 100 according to the embodiments of the present invention is not particularly limited as long as it is a device capable of processing a user command including at least a voice input received according to the speech of the user 110 through a voice-based interface. Can be. For example, the electronic device 100 may process a user command by directly recognizing and analyzing a voice input of the user and performing an operation suitable for the voice input. However, according to an embodiment, the electronic device 100 may recognize or recognize the voice input of the user. Analysis of voice input, synthesis of voice to be provided to the user, and the like may be performed through an external platform connected with the electronic device 100.

3 is a diagram illustrating an example of a cloud AI platform according to an embodiment of the present invention. 3 illustrates electronic devices 310, cloud AI platform 320, and content / service 330.

For example, the electronic devices 310 may refer to devices provided in a home, and may include at least the electronic device 100 described above. Applications (hereinafter, referred to as apps) installed and driven on the electronic devices 310 or the electronic devices 310 may be associated with the cloud AI platform 320 through the interface connect 340. Here, the interface connect 340 may provide developers with software development kits (SDKs) and / or development documents for development of apps installed and driven on the electronic devices 310 or the electronic devices 310. In addition, the interface connect 340 includes an application program interface (API) that enables applications installed and driven on the electronic devices 310 or the electronic devices 310 to utilize functions provided by the cloud AI platform 320. Can provide. As a specific example, developers may develop a device or app using a software development kit (SDK) and / or a development document provided by the interface connect 340 using a cloud AI platform (API) provided by the interface connect 340. The functions provided by 320 may be utilized.

Here, the cloud AI platform 320 may provide a function for providing a voice-based service. For example, the cloud AI platform 320 recognizes the received voice, and a voice processing module 321 for synthesizing the output voice, and a vision processing module 322 for analyzing and processing the received image or video. A dialogue processing module 323 for determining an appropriate dialogue in order to output a suitable speech according to the received speech, a recommendation module 324 for recommending a function suitable for the received speech, and a sentence unit of artificial intelligence through data learning It may include various modules for providing a voice-based service, such as neural machine translation (NMT, 325) that supports the translation of the language.

For example, in the embodiments of FIGS. 1 and 2, the electronic device 100 may transmit the voice input of the user 110 to the cloud AI platform 320 using an API provided by the interface connect 340. have. In this case, the cloud AI platform 320 may recognize and analyze the received voice input by using the above-described modules 321 to 325, and synthesize and provide an appropriate answer voice according to the received voice input. Proper action can be recommended.

In addition, the extension kit 350 may provide a development kit that allows third-party content developers or companies to implement new voice-based functions based on the cloud AI platform 320. For example, in the embodiment of FIG. 2, the electronic device 100 may transmit the received voice input of the user 110 to the external server 210, and the external server 210 may use the expansion kit 350. The voice input may be transmitted to the cloud AI platform 320 through the provided API. In this case, as described above, the cloud AI platform 320 recognizes and analyzes the received voice input, synthesizes an appropriate answer voice, and provides recommendation information about a function to be processed through the voice input. ) Can be provided. As an example, in FIG. 2, the external server 210 can send a voice input “today's weather” to the cloud AI platform 320 and through recognition of the voice input “today's weather” from the cloud AI platform 320. The extracted keywords "today" and "weather" can be received. In this case, the external server 210 generates text information such as "today's weather ..." through the keywords "today" and "weather" to send the text information generated back to the cloud AI platform 320. Can be. In this case, the cloud AI platform 320 may synthesize text information into a voice and provide it to the external server 210. The external server 210 may transmit the synthesized voice to the electronic device 100, and the electronic device 100 outputs the synthesized voice "Weather of the day ..." through the speaker, thereby outputting from the user 110. The received voice input "Weather today" can be processed.

In this case, the electronic device 100 may perform a device control method according to an embodiment of the present invention to provide a device operation or content corresponding to a voice input.

4 is a block diagram illustrating an internal configuration of an electronic device and a server according to an embodiment of the present invention. The electronic device 410 of FIG. 4 may correspond to the electronic device 100 described above, and the server 420 may correspond to one computer device implementing the external server 210 or the cloud AI platform 320 described above. Can be.

The electronic device 410 and the server 420 may include memories 411 and 421, processors 412 and 422, communication modules 413 and 423, and input / output interfaces 414 and 424. The memories 411 and 421 may be computer-readable recording media, and may include permanent mass storage devices such as random access memory (RAM), read only memory (ROM), and disk drives. The non-volatile mass storage device such as a ROM and a disk drive may be included in the electronic device 410 or the server 420 as a separate permanent storage device that is separate from the memories 411 and 421. In addition, the memory 411 and 421 may store an operating system and at least one program code (for example, a code installed in the electronic device 410 and driven by the electronic device 410 to provide a specific service). Can be. These software components may be loaded from a computer readable recording medium separate from the memories 411 and 421. Such a separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, disk, tape, DVD / CD-ROM drive, memory card, and the like. In other embodiments, software components may be loaded into the memory 411, 421 through a communication module 413, 423 rather than a computer readable recording medium. For example, the at least one program may be electronically based on a computer program (for example, the above-described application) installed by files provided through a network 430 by a file distribution system that distributes installation files of developers or applications. The memory 411 of the device 410 may be loaded.

Processors 412 and 422 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processors 412 and 422 by the memory 411 and 421 or the communication modules 413 and 423. For example, the processors 412 and 422 may be configured to execute a command received according to a program code stored in a recording device such as the memory 411 and 421.

The communication modules 413 and 423 may provide a function for the electronic device 410 and the server 420 to communicate with each other through the network 430, and the electronic device 410 and / or the server 420 may be different from each other. It may provide a function for communicating with an electronic device or another server. In one example, a request generated by the processor 412 of the electronic device 410 according to a program code stored in a recording device such as a memory 411 is controlled by the server 420 through the network 430 under the control of the communication module 213. Can be delivered. Conversely, control signals, commands, contents, files, and the like provided according to the control of the processor 422 of the server 420 pass through the communication module 223 and the network 430 to the communication module 213 of the electronic device 410. ) May be received by the electronic device 410. For example, the control signal, command, content, file, etc. of the server 420 received through the communication module 213 may be transmitted to the processor 412 or the memory 411, and the content, file, etc. may be transferred to an electronic device ( 410 may be stored as a storage medium (permanent storage device described above) that may further include.

The input / output interface 414 may be a means for interfacing with the input / output device 415. For example, the input device may include a device such as a microphone, a keyboard or a mouse, and the output device may include a device such as a display or a speaker. As another example, the input / output interface 414 may be a means for interfacing with a device in which functions for input and output are integrated into one, such as a touch screen. The input / output device 415 may be configured as one device with the electronic device 410. In addition, the input / output interface 424 of the server 420 may be a means for interfacing with an apparatus (not shown) for input or output that may be connected to or included in the server 420.

Further, in other embodiments, the electronic device 410 and server 420 may include fewer or more components than the components of FIG. 4. However, it is not necessary to clearly show most of the prior art components. For example, the electronic device 410 may be implemented to include at least some of the above-described input and output devices 415 or other components such as a transceiver, a global positioning system (GPS) module, a camera, various sensors, a database, and the like. It may also include more. More specifically, when the electronic device 410 is a smart phone, an acceleration sensor or gyro sensor, a camera module, various physical buttons, a button using a touch panel, an input / output port, a vibrator for vibration generally included in the smart phone Various components such as may be implemented to be further included in the electronic device 410.

In the present exemplary embodiment, the electronic device 410 may basically include a microphone for receiving a voice input of the user as the input / output device 415, and may provide a sound such as an answer voice or audio content corresponding to the voice input of the user. A speaker for outputting may be further included as the input / output device 415.

5 is a flowchart illustrating an example of a device control method according to an embodiment of the present invention. The device control method according to embodiments of the present invention may be performed by a computer device such as the electronic device 410 described above. In this case, the processor 412 of the electronic device 410 may be implemented to execute a control instruction according to a code of an operating system included in the memory 411 or a code of at least one program. Here, the processor 412 may perform the steps S510 to S540 included in the device control method of FIG. 5 according to a control command provided by a code stored in the electronic device 410. 410 may be controlled.

In operation S510, the electronic device 410 may receive a command according to a voice input from the talker through the voice-based interface. For example, the electronic device 410 may receive a voice input according to a user's speech through a voice input device such as a microphone included in the electronic device 410 or a microphone interlocked with the electronic device 410. Techniques for receiving a speaker's voice input will be readily understood by those skilled in the art through well known techniques with respect to speech recognition.

In operation S520, the electronic device 410 may acquire the location information of the talker associated with the voice input when it is necessary to determine the location of the analysis result of the command received by the talker's voice input. For example, the electronic device 410 may include an instruction pronoun (eg, here, there, there, here, there, there, there, here, there, there, there, there, etc.), or the instruction may include a place or an instruction object. When this omitted incomplete sentence can be expressed additional logic for position determination. Therefore, the electronic device 410 may additionally obtain the location information of the talker based on the command when it is difficult to specify the control target position or the control target device only by the command according to the voice input. When the electronic device 410 needs to estimate the location, the electronic device 410 obtains the location information of the talker associated with the voice input, wherein the location information of the talker associated with the voice input is measured at a time point or period associated with the reception of the voice input. The speaker 410 may include at least one of a relative position or direction of the talker, whether the relative position or direction changes, and a change direction of the relative position or direction.

For example, the electronic device 410 may obtain location information associated with the voice input based on a phase shift of the plurality of microphone input voice inputs included in the voice based interface. Techniques for measuring the generation position of the sound signal using the phase shift of the same sound signal input to a plurality of microphones will be easily understood by those skilled in the art through well-known techniques such as beam forming technology. In this case, since the location information is measured through the voice input of the talker, the talker does not need to be directed in a specific direction, and the location of the talker may not be limited if the talker is within a recognized distance. In addition, it becomes possible to obtain the location information of the talker without the need for a separate device other than the voice-based interface to be added to the electronic device 410.

According to an embodiment, in order to acquire the location information of the talker, the case where the electronic device 410 includes additional equipment such as a camera or a sensor may be considered, and the location information of the talker and the camera or sensor measured through the voice-based interface may be considered. All of the location information of the talker measured using additional equipment such as may be utilized. When utilizing a camera and / or a sensor, the electronic device 410 may obtain location information associated with the voice input based on output values of the camera and / or sensor when the voice input is received.

In one example, the electronic device 410 can poll the optical device to visually specify the location of the talker. Optical devices such as cameras may use recognition software (eg, face recognition, feature recognition, etc.) to identify a user. Here, polling may include requesting information or receiving information without requesting for obtaining optical information at the optical device. In addition to the optical device, it is also possible to poll the voice device to visually specify the location of the talker. The voice tool can be used to identify the talker according to the voice input based on the pre-recorded voice profile.

Furthermore, the electronic device 410 may obtain location information of the talker associated with the voice input based on a conversation context or a device control history according to the talker's voice input. In other words, the electronic device 410 grasps the talker's previous conversation context or the device control history to include at least one of the relative position or direction of the talker, whether the relative position or direction changes, and the change direction of the relative position or direction. Location information can be obtained. For example, if there was an operation such as inquiring about a recipe or setting a timer for cooking through a voice-based interface, the current position of the talker by the dialogue context of the voice input instructing the operation or the device control history. It can be estimated that the recent position is 'kitchen'.

In operation S530, the electronic device 410 may specify the device to be controlled based on the location information of the talker with respect to the command received through the talker's voice input. The electronic device 410 may specify a device corresponding to the voice input of the talker based on the location information of the talker associated with the voice input when it is difficult to specify the control target location or the control target device only by the command according to the voice input.

For example, the electronic device 410 may specify the control target device based on the location information of the talker when the instruction pronoun is included in the command received through the talker's voice input. For example, the electronic device 410 may be a device that is closest to the talker, a device that is farthest from the talker, or a device that is located in the direction of the talker among the devices capable of operating corresponding to the voice input of the talker, Alternatively, a device located at a position opposite to the direction of the talker can be specified as the device to be controlled. As an example, if a command received by a talker's voice input includes an instruction pronoun meaning 'here', 'here', 'here', etc., the device is located in the direction of the device or the speaker closest to the talker. The device can be specified as the device to be controlled. In another example, if a command received from a talker's voice input includes an indication pronoun meaning "over there," "over there," "over there," etc., the direction of the device or the speaker farthest from the talker is reversed. The device located at the location can be specified as the device to be controlled. As another example, when the command received by the talker's voice input includes an instruction pronoun meaning 'there', 'there', etc., the device at the previous location where the talker recently stayed may be designated as the controlling device. .

As another example, the electronic device 410 may specify the device to be controlled based on the location information of the talker when the command received by the talker's voice input is composed of an incomplete sentence in which a place or an instruction object is omitted. That is, the electronic device 410 may estimate a part of the indication object omitted in the dialogue context based on the location information of the talker. In this case, the electronic device 410 may specify the device to be controlled based on the previous dialogue context according to the talker's voice input or the location information of the talker estimated in the device control history. For example, in the case of a command in which a place or an instruction target is omitted, such as a voice input 'Turn on a video', a device in which the talker has recently stayed among the devices capable of responding to the voice input of the talker may be specified as the control target device. Can be.

The electronic device 410 is a device to be controlled according to a device usage pattern or a device control history when two or more devices are specified as a control target device for a control operation corresponding to a command received through a voice input of a talker. Can be selected. When it is difficult to specify the control target location or the control target device only by the command according to the voice input, the electronic device 410 specifies the control target device based on the location information of the talker associated with the voice input. If it is specified, it is possible to select a more suitable device through learning the user behavior pattern among the specified device. For example, when a command such as a voice input 'play Pororo video' is received, the device history having the most played content can be identified and the device having the most history can be selected as the control target device.

Therefore, when it is difficult to specify the device to be controlled only by the command according to the voice input, the electronic device 410 may estimate a device corresponding to the speaker's intention based on the location information of the talker associated with the voice input. In addition, the electronic device 410 may select a more suitable control target device by grasping a conversation context according to a voice input of a talker, or grasping a device usage pattern or a device control history.

In operation S540, the electronic device 410 may transmit the corresponding command to the control target device specified in operation S530 to control an operation corresponding to the command received through the voice input of the talker. The electronic device 410 may transmit a command according to the voice input of the talker to the corresponding device so that an operation suitable for the intention of the talker is performed by the controlling device. The electronic device 410 may output a voice of information for confirmation or recommendation of the control target location or the control target device before transmitting a command according to the voice input of the talker to the control target device. For example, the electronic device 410 recognizes and analyzes the voice input 'Turn off the light' to output specific information such as a confirmation or recommended voice such as 'turn off the living room light' through the speaker. In response to the voice input 'no', a command may be transmitted to the corresponding device to turn off the lighting device in the living room.

Accordingly, the electronic device 410 may further control the operation of the device or the provision of content corresponding to the intention of the talker by further determining the location of the talker with respect to the voice input for which the location or the indication target is not specified. Thus, even if a utterance form using an instruction pronoun or a location or a target is omitted, a plurality of the same functions existing in the home, such as a plurality of displays or lights, a plurality of speakers, and a room temperature control device, are based on the position of the talker. Selective control over the device is possible.

6 to 7 illustrate examples of determining a device to be controlled according to a voice input including an instruction pronoun according to an embodiment of the present invention.

FIG. 6 illustrates that the electronic device 410 including the voice-based interface recognizes and analyzes the utterance “turn on the TV here” of the user 110, so that the electronic device 410 interacts with the electronic device 410 through the internal network. An example of controlling the power source is shown. The voice input "turn on the TV here" of the user 110 includes the pronoun "here", which requires estimation of the position determination, and thus additionally obtains the location information of the user 110 and based on the user 110. The control target device can be selected.

According to the instruction pronoun, the positional relationship between the talker and the device may be determined in advance. For example, when the instruction pronoun includes meanings such as 'here', 'here', and 'here', the position is closest to the speaker. Device or device located in the direction of the talker, where the pronouns in the sense of 'there', 'there', 'there', etc. are included When the device includes an instruction pronoun meaning 'there', 'there', or the like, the device at the previous position where the talker stayed most recently may be specified as the device to be controlled.

The electronic device 410 may manage locations of a plurality of different devices linked to the electronic device 410. In this case, the location of the devices in the home may be set in various ways. For example, such a location may be entered and set in the electronic device 410 by the user 110 or the administrator, and may be already present, such as the strength of a signal transmitted and received by the electronic device 410 to communicate with other electronic devices. It may also be measured using known positional positioning techniques. In the case of a mobile device, the location can be measured dynamically using location positioning technology. In this case, the electronic device 410 may further determine the control target device using a distance between the user 110 and the plurality of other electronic devices measured based on the location of the user 110 and the positions of the plurality of other devices. . As described above, the position of the user 110 may be measured based on the speech of the user 110.

For example, as shown in FIG. 6, when the user 110 stands in the living room and utters “Turn on the TV here,” the user input is estimated to include the pronoun “here” in the voice input to estimate the location of the user 110. Among the plurality of TVs, the TV 610 in the living room closest to the location of the user 110 may be selected as the control target device.

As another example, FIG. 7 illustrates that the electronic device 410 including the voice-based interface recognizes and analyzes the ignition " turn on the room air conditioner " of the user 110, thereby interworking with the electronic device 410 through the internal network. The example which controls the power supply of the control apparatus 710 is shown. When the user 110 stands in the living room and utters "Turn the air conditioner over there", the user input is estimated by estimating the location of the user 110 according to the voice input including the pronoun "over there". The temperature control device 710 in the farthest room can be selected as the device to be controlled.

8 to 9 are diagrams illustrating examples of determining a control target device according to a voice input in which a position or an instruction target is omitted according to one embodiment of the present invention.

FIG. 8 illustrates that the electronic device 410 including the voice-based interface recognizes and analyzes the utterance “play video” of the user 110, and displays the indoor display 810 linked with the electronic device 410 through an internal network. An example of controlling is shown. For example, when the user 110 utters "play video" within a predetermined time after an operation such as inquiry about a recipe or setting a timer for cooking through a voice-based interface, the dialogue context of the previous voice input or the device control history. In FIG. 2, the user 110 may estimate that the user is located in the kitchen, and deliver the corresponding content to a display (eg, a TV, a refrigerator display, etc.) 810 installed near the kitchen. Alternatively, by analyzing the dialogue context of the utterance content itself with respect to the utterance of the user 110, “play the bulgogi recipe video”, the position of the user 110 may be estimated to be near the kitchen.

Referring to FIG. 9, when the user 110 utters “Please turn off the light” while returning to the living room after the cooking-related request from the kitchen through the voice-based interface, the user 110 according to the instruction pronoun “there” is included in the voice input. ) By estimating the location of the control unit may select the lighting 910 of the previous location where the user 110 recently stayed among the indoor lighting as the control target device.

FIG. 10 is a diagram for one example of determining a device to be controlled based on learning a user behavior pattern according to one embodiment of the present invention.

FIG. 10 illustrates an electronic device 410 including a voice-based interface that recognizes and analyzes the utterance “play a video” of the user 110, and interlocks the electronic device 410 with an indoor network 1010 through an internal network. An example of controlling is shown. The electronic device 410 may omit a position or a target to be uttered in the user 110 's utterance of the video, so that the position of the user 110 may be further determined and based on the position of the user 110. Control device can be selected. For example, when a TV 610 and a tablet 1010 exist as a device capable of playing a video in a living room in which the user 110 is located, the device history where the corresponding content is mainly played is determined to determine the TV 610 and the tablet 1010. ), The device having the most history can be selected to control content playback. Similarly, when there are a plurality of devices capable of outputting voice content in the vicinity of the user 110 with respect to the utterance "hip hip hop" of the user 110, the device genre mainly recognizes the device history where the corresponding music genre is played, and the corresponding history is the most. Many devices can be selected as the control target device to control content reproduction.

As described above, according to embodiments of the present invention, after further determining the location of the talker according to the command according to the talker's voice input, a device suitable for the command may be selected to control device operation or content provision. In particular, when a command includes an instruction pronoun or an instruction is composed of an incomplete sentence without a location or an indication object, a device corresponding to the user's intention can be estimated based on the user's location, and the dialogue context according to the voice input of the talker. The device can be estimated according to the user's intention by identifying the device usage pattern or the device control history.

The system or apparatus described above may be implemented as a hardware component, a software component or a combination of hardware components and software components. For example, the devices and components described in the embodiments are, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable gate arrays (FPGAs). Can be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. It can be embodied in. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The medium may be to continue to store a computer executable program, or to temporarily store for execution or download. In addition, the medium may be a variety of recording means or storage means in the form of a single or several hardware combined, not limited to a medium directly connected to any computer system, it may be distributed on the network. Examples of the medium include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, And ROM, RAM, flash memory, and the like, configured to store program instructions. In addition, examples of another medium may include a recording medium or a storage medium managed by an app store that distributes an application, a site that supplies or distributes various software, a server, or the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components Or even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

Claims (20)

In the device control method of an electronic device comprising a voice-based interface,
Receiving a command according to a voice input of a user through the voice-based interface;
Acquiring location information associated with the voice input when the location of the command is determined as a result of the analysis; And
Specifying a control target device for controlling an operation corresponding to the command among devices that can be linked with the electronic device using the location information;
Including,
In the case where the analysis result of the command requires position determination, it includes a case where an instruction pronoun is included in the command,
The obtaining step,
When the instruction analysis result includes an instruction pronoun included in the instruction to express the additional logic for position determination to obtain the position information,
The specifying step,
A positional relationship between the user and the device is determined according to the instruction pronoun included in the command,
Specifying, as the control target device, a device in a positional relationship determined according to the instruction pronoun based on the position information.
Device control method characterized in that. delete delete The method of claim 1,
Location information associated with the voice input,
At least one of a relative position or direction of the user with respect to the electronic device, whether the relative position or direction changes, and a change direction of the relative position or direction measured at a time point or period associated with the reception of the voice input. that
Device control method characterized in that. The method of claim 1,
The obtaining step,
Acquiring position information associated with the voice input based on a phase shift of the voice input input to a plurality of microphones included in the voice based interface;
Device control method characterized in that. The method of claim 1,
The electronic device includes at least one of a camera and a sensor,
The obtaining step,
Acquiring location information associated with the voice input based on an output value of at least one of the camera and sensor when the voice input is received;
Device control method characterized in that. The method of claim 1,
The obtaining step,
Acquiring location information associated with the voice input by identifying at least one of a dialogue context corresponding to the voice input and a device control history of the user through the voice-based interface;
Device control method characterized in that. delete The method of claim 1,
The specifying step,
Specifying the device to be controlled using the device usage pattern of the user through the voice based interface together with the location information associated with the voice input.
Device control method characterized in that. The method of claim 1,
The specifying step,
When there are a plurality of devices capable of operation corresponding to the command, specifying a device having a long history among a plurality of devices as the control target device using a device usage pattern of the user through the voice-based interface;
Device control method characterized in that. A computer program stored in a computer-readable recording medium in combination with a computer for causing the computer to execute the method of any one of claims 1, 4-7, and 9-10. A computer-readable recording medium having recorded thereon a program for causing a computer to execute the method of any one of claims 1, 4, 7, and 10. An electronic device for controlling a device,
Voice-based interface; And
At least one processor implemented to execute computer-readable instructions
Including,
The at least one processor,
Receives a command according to a voice input of a user through the voice-based interface,
If location analysis is necessary as a result of analyzing the command, obtain location information associated with the voice input,
Specifying a control target device for controlling an operation corresponding to the command among devices that can be linked with the electronic device by using the location information;
In the case where the analysis result of the command requires position determination, it includes a case where an instruction pronoun is included in the command,
The at least one processor is configured to obtain location information associated with the voice input,
When the instruction analysis result includes an instruction pronoun included in the instruction to express the additional logic for position determination to obtain the position information,
The at least one processor, in order to specify the device to be controlled,
A positional relationship between the user and the device is determined according to the instruction pronoun included in the command,
Specifying, as the control target device, a device in a positional relationship determined according to the instruction pronoun based on the position information.
Electronic device for device control, characterized in that. delete The method of claim 13,
Location information associated with the voice input,
At least one of a relative position or direction of the user with respect to the electronic device, whether the relative position or direction changes, and a change direction of the relative position or direction measured at a time point or period associated with the reception of the voice input. that
Electronic device for device control, characterized in that. The method of claim 13,
The at least one processor is configured to obtain location information associated with the voice input,
Acquiring position information associated with the voice input based on a phase shift of the voice input input to a plurality of microphones included in the voice based interface;
Electronic device for device control, characterized in that. The method of claim 13,
The electronic device includes at least one of a camera and a sensor,
The at least one processor is configured to obtain location information associated with the voice input,
Acquiring location information associated with the voice input based on an output value of at least one of the camera and sensor when the voice input is received;
Electronic device for device control, characterized in that. The method of claim 13,
The at least one processor is configured to obtain location information associated with the voice input,
Acquiring location information associated with the voice input by identifying at least one of a dialogue context corresponding to the voice input and a device control history of the user through the voice-based interface;
Electronic device for device control, characterized in that. delete The method of claim 13,
The at least one processor, in order to specify the device to be controlled,
Specifying the device to be controlled using the device usage pattern of the user through the voice based interface together with the location information associated with the voice input.
Electronic device for device control, characterized in that.

Images