JP2018194810A - Device controlling method and electronic apparatus - Google Patents

Abstract

To provide a technique for device control according to a user's talk position.SOLUTION: A device controlling method with an electronic apparatus comprising a sound board interface includes the steps of: receiving instruction words inputted by a user's voice through a sound board interface; obtaining positional information relevant to voice input when positional information is required as a result of the analysis of the instruction words; and specifying a device to be controlled among devices the electronic apparatus can control, by using the positional information, and then making the device to be controlled perform operation corresponding to the instruction words.SELECTED DRAWING: Figure 5

Description

  The following description relates to device control based on the user's utterance position, and more specifically, controls the device operation and content provision by additionally determining the user's utterance position from the command word corresponding to the voice input. BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device control method and system, a computer program stored in a computer-readable recording medium for causing the computer to execute the device control method in combination with a computer, and the recording medium.

  An interface operating based on voice, such as an artificial intelligence speaker of a home network service, can receive a user's voice input through a microphone and control device operation and content provision based on the received voice input. .

  For example, Patent Literature 1 (publication date: December 30, 2011) is a technology relating to a home media device and a home network system and method using the home media device. In home network services, Wi-Fi (in addition to a mobile communication network) is disclosed. It is possible to provide a home network service using a second communication network such as Wi-Fi, and to multiplex-control a plurality of multimedia devices in the home according to a voice command without any user button operation. The technology which can do is disclosed.

  In such a conventional technique, when there are a plurality of devices to be controlled, device control can be performed by utterance using a device name registered in advance. However, when there are a plurality of devices having the same role, there is a difficulty that the device names must be classified and registered, and further, the user must memorize all the registered devices. Since it mainly operates on the basis of sound and does not use visual information, when it is necessary to select from a plurality of options, specific requirements and instruction forms must be structured and expressed in words. There's a problem.

Korean Published Patent No. 10-2011-0139797

  A device control method and system capable of grasping an utterance form (for example, a simple conversational tone) that can be easily expressed in a colloquial form and controlling a device suitable for the user's intention, and a device control method combined with a computer A computer program stored in a computer-readable recording medium and the recording medium are provided to be executed by a computer.

  A device control method and system capable of grasping a conversational strategy and controlling a device suitable for the user's intention, and a computer-readable recording medium for causing the computer to execute the device control method in combination with a computer A stored computer program and its recording medium are provided.

  Device control method and system capable of controlling device according to user intention based on device usage pattern and history, and stored in computer-readable recording medium for causing computer to execute device control method in combination with computer Computer program and its recording medium are provided.

  A device control method for an electronic device including a voice infrastructure interface, wherein a command word by a user's voice input is received through the voice infrastructure interface, and if the position information is required as a result of the analysis of the command word, the voice input Obtaining position information related to the device, and using the position information to identify a control target device from devices that can be controlled by the electronic device, and to control an operation corresponding to the command word A device control method is provided that includes causing a device to execute.

  A computer program stored in a computer-readable recording medium for causing the computer to execute the device control method in combination with a computer is provided.

  Provided is a computer-readable recording medium on which a program for causing a computer to execute the device control method is recorded.

  An electronic device comprising a voice based interface and at least one processor implemented to execute computer readable instructions, wherein the at least one processor is a command by user voice input through the voice based interface When the position information is required as a result of analyzing the command word, the position information related to the voice input can be acquired and the electronic device can be controlled using the position information. Provided is an electronic device that identifies a control target device from among the devices and causes the control target device to execute an operation corresponding to the command word.

  The user's utterance position can be additionally determined according to the command word corresponding to the voice input, and a device that matches the command word can be selected.

  When a command pronoun is included in the command word, a device that matches the user's intention can be estimated based on the user's position.

  A user's position can be estimated from the conversational abbreviation, and a device suitable for the user's intention can be estimated.

  Based on the device usage pattern and history, it is possible to estimate a device that matches the user's intention.

It is the figure which showed the example of the service environment using the audio | voice infrastructure interface in one Embodiment of this invention. It is the figure which showed the other example of the service environment using the voice infrastructure interface in one Embodiment of this invention. It is the figure which showed the example of the cloud artificial intelligence platform in one Embodiment of this invention. It is a block diagram for demonstrating the internal structure of the electronic device and server in one Embodiment of this invention. It is the flowchart which showed the example of the device control method in one Embodiment of this invention. It is the figure which showed the example which determines a control object device according to the audio | voice input in which the indication pronoun was included in one Embodiment of this invention. It is the figure which showed the example which determines a control object device according to the audio | voice input in which the indication pronoun was included in one Embodiment of this invention. It is the figure which showed the example which determines a control object device according to the audio | voice input by which position and the instruction | indication object were abbreviate | omitted in one Embodiment of this invention. It is the figure which showed the example which determines a control object device according to the audio | voice input by which position and the instruction | indication object were abbreviate | omitted in one Embodiment of this invention. It is the figure which showed the example which determines a control object device based on user action pattern learning in one Embodiment of this invention.

  Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

  The device control system according to the embodiment of the present invention may be realized by an electronic device that provides an interface that operates based on audio. The electronic device may additionally determine the position of the speaker from the command word by the voice input of the speaker and control device operation and content provision. Here, the electronic device may estimate a device that matches the user's intention based on the position of the user when the command word includes an instructional pronoun or the command word includes an incomplete sentence. Furthermore, the electronic device may estimate a device that matches the user's intention by grasping a conversational strategy by a voice input of a speaker or grasping a device usage pattern and a device control history.

  The device control method according to the embodiment of the present invention may be executed by the electronic apparatus described above. Here, the electronic apparatus may be installed and driven with the computer program according to the embodiment of the present invention, and the electronic apparatus performs the device control method according to the embodiment of the present invention according to the control of the computer program to be driven. May be executed. The computer program described above may be stored in a computer-readable recording medium in order to cause the computer to execute the device control method in combination with an electronic device realized by the computer.

  FIG. 1 is a diagram illustrating an example of a service environment using a voice infrastructure interface according to an embodiment of the present invention. In the embodiment of FIG. 1, in a technology for connecting and controlling devices in a home such as a smart home or a home network service, an electronic device 100 that provides an interface that operates based on voice is uttered by a user 110. An example of receiving and recognizing and analyzing a voice input “turn off electricity” and controlling a light power source of a home lighting device 120 that cooperates with the electronic device 100 via an internal network in the home is shown.

  For example, in addition to the above-described home lighting device 120, home devices include not only home appliances such as televisions, PCs (Personal Computers), peripheral devices, air conditioners, refrigerators, robot cleaners, but also water, electricity, and air conditioning. It may include various devices that can be connected and controlled on-line, such as energy consuming devices such as devices, security devices such as door locks and surveillance cameras. The internal network includes wired network technologies such as Ethernet (registered trademark (Ethernet)), HomePNA, and IEEE 1394, Bluetooth (registered trademark (Bluetooth)), UWB (Ultra Wide Band), ZigBee, and Wireless 1394. Wireless network technology such as Home RF may be used.

  The electronic device 100 may be one of devices in the home. For example, the electronic device 100 may be one of devices such as an artificial intelligence speaker or a robot cleaner provided in a home. The electronic device 100 is a mobile device of the user 110 such as a smart phone, a mobile phone, a notebook type pancon, a digital broadcasting terminal, a PDA (Personal Digital Assistant), a PMP (Portable Multimedia Player), and a tablet. There may be. As described above, the electronic device 100 is not particularly limited as long as the electronic device 100 includes a function that can be connected to a home device in order to receive the voice input of the user 110 and control the home device. In addition, depending on the embodiment, the mobile device of the user 110 described above may be included as a home device.

  FIG. 2 is a diagram illustrating another example of a service environment using a voice infrastructure interface according to an embodiment of the present invention. FIG. 2 shows that an electronic device 100 that provides a voice-based interface receives and recognizes and analyzes the speech input of the user 110 utterance “Today's Weather” from an external server 210 via an external network. An example is shown in which information related to the weather is acquired and the acquired information is output by voice such as “Today's weather is ...”.

  For example, the external network includes a PAN (personal area network), a LAN (local area network), a MAN (metropolitan area network), a WAN (wide area network, etc.), a WAN (wide area network, etc.), and a WAN (wide area network, etc.). One or more of any of the networks may be included.

  Also in the embodiment of FIG. 2, the electronic device 100 may be one of the devices in the home or one of the mobile devices of the user 110, and the function for receiving and processing the voice input of the user 110 and Any device including a function for connecting to the external server 210 via an external network and providing the user 110 with services and contents provided by the external server 210 is not particularly limited.

  As described above, the electronic device 100 according to the embodiment of the present invention is particularly limited as long as it is a device that can process a user command including at least a voice input received by the speech of the user 110 through the voice based interface. There is no. For example, the electronic device 100 may process user instructions by directly recognizing and analyzing the user's voice input and performing operations suitable for the voice input, but in some embodiments, Processing such as recognition, analysis of recognized speech input, and speech synthesis provided to the user can be executed on an external platform linked to the electronic device 100.

  FIG. 3 is a diagram illustrating an example of a cloud artificial intelligence platform according to an embodiment of the present invention. FIG. 3 shows an electronic device 310, a cloud artificial intelligence platform 320, and a content service 330.

  As an example, the electronic device 310 may mean a device provided in a home, and may include at least the electronic device 100 described above. Such an electronic device 310 and an application installed and driven on the electronic device 310 (hereinafter referred to as an application) may be linked to the cloud artificial intelligence platform 320 through the interface connect 340. Here, the interface connect 340 may provide the developer with an SDK (Software Development Kit) and / or development document for developing the electronic device 310 or an application installed and driven in the electronic device 310. Further, the interface connect 340 may provide an application program interface (API) that allows the electronic device 310 or an application installed and driven to the electronic device 310 to utilize the function provided by the cloud artificial intelligence platform 320. As a specific example, a developer can use a device and application developed using SDK (Software Development Kit) provided by the interface connect 340 and / or a development document in the cloud using an API provided by the interface connect 340. The functions provided by the artificial intelligence platform 320 can be utilized.

  Here, the cloud artificial intelligence platform 320 may provide a function for providing a voice-based service. For example, the cloud artificial intelligence platform 320 recognizes the received voice and synthesizes the voice to be output, the voice processing module 321 for analyzing the received video and video, and the received voice. Conversation processing module 323 for determining an appropriate conversation for outputting a sound that conforms to the language, a recommendation module 324 for recommending a function suitable for the received speech, and a language in which artificial intelligence is based on data learning in sentence units Various modules for providing voice-based services may be included, such as an artificial neural network-based machine translation (NMT) 325 that assists in translating.

  For example, in the embodiment of FIGS. 1 and 2, the electronic device 100 may transmit the voice input of the user 110 to the cloud artificial intelligence platform 320 using an API provided from the interface connect 340. In this case, the cloud artificial intelligence platform 320 may recognize and analyze the received voice input using the above-described modules 321 to 325, synthesize and provide an appropriate response voice according to the received voice input, Appropriate action may be recommended.

  The extension kit 350 may provide a development kit that allows a third-party content developer or company to realize a new voice infrastructure function based on the cloud artificial intelligence platform 320. For example, in the embodiment of FIG. 2, the electronic device 100 may transmit the received voice input of the user 110 to the external server 210, and the external server 210 transmits the cloud artificial intelligence platform 320 through an API provided from the extension kit 350. Voice input may be sent to In this case, as described above, the cloud artificial intelligence platform 320 recognizes and analyzes the received voice input, synthesizes and provides an appropriate response voice, or relates to a function to be processed according to the voice input. Recommendation information may be provided to the external server 210. As an example, in FIG. 2, the external server 210 may send a speech input “Today's weather” to the cloud artificial intelligence platform 320, which is extracted by recognition of the speech input “Today's weather”. The keywords “today” and “weather” may be received. In this case, the external server 210 generates text information such as “Today's weather is ...” based on the keywords “Today” and “Weather”, and the text information generated in the cloud artificial intelligence platform 320 again. May be sent. Here, the cloud artificial intelligence platform 320 may synthesize text information by voice and provide it to the external server 210. The external server 210 may transmit the synthesized voice to the electronic device 100, and the electronic device 100 receives the synthesized voice “Today's weather ...” from the user 110 by outputting from the speaker. The voice input “Today's weather” may be processed.

  At this time, the electronic device 100 may execute the device control method according to the embodiment of the present invention in order to provide device operation corresponding to voice input and content provision.

  FIG. 4 is a block diagram for explaining the internal configuration of the electronic device and the server in one embodiment of the present invention. The electronic device 410 of FIG. 4 may correspond to the electronic device 100 described above, and the server 420 may correspond to one computer device that implements the external server 210 and the cloud artificial intelligence platform 320 described above.

  The electronic device 410 and the server 420 may include memories 411 and 421, processors 412 and 422, communication modules 413 and 423, and input / output interfaces 414 and 424. The memories 411 and 421 are computer-readable recording media and include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. It's okay. Here, a permanent mass storage device such as a ROM and a disk drive may be included in the electronic device 410 or the server 420 as another permanent storage device separated from the memories 411 and 421. The memories 411 and 421 include an operating system (OS) and at least one program code (for example, an application installed in the electric device 410 and driven by the electronic device 410 to provide a specific service). May be stored. Such software components may be loaded from a computer-readable recording medium different from the memories 411 and 421. Such another computer-readable recording medium may include a computer-readable recording medium such as a floppy (registered trademark) drive, a disk, a tape, a DVD / CD-ROM drive, and a memory card. In other embodiments, the software components may be loaded into the memories 411 and 421 through the communication modules 413 and 423 that are not computer-readable recording media. For example, the at least one program is based on a program (for example, the application described above) installed by a file distribution system that distributes an installation file of a developer or an application via a network 430. It may be loaded into the memory 411.

  Processors 412, 422 may be configured to process computer program instructions by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to the processors 412, 422 by the memories 411, 421 or the communication modules 413, 423. For example, the processors 412, 422 may be configured to execute instructions received according to program code stored in a recording device such as the memory 411, 421.

  The communication modules 413 and 423 may provide a function for the electronic device 410 and the server 420 to communicate with each other via the network 430, and the electronic device 410 and / or the server 420 may communicate with other electronic devices or other servers. A function for communication may be provided. As an example, a request generated by the processor 412 of the electronic device 410 according to a program code stored in a recording device such as the memory 411 may be transmitted to the server 420 via the network 430 under the control of the communication module 213. On the contrary, control signals, commands, contents, files, etc. provided in accordance with the control of the processor 422 of the server 420 are received by the electronic device 410 through the communication module 223 and the network 430 and the communication module 213 of the electronic device 410. May be. For example, the control signal, command, content, file, and the like of the server 420 received through the communication module 213 may be transmitted to the processor 412 and the memory 411, and the storage medium that the electronic device 410 can further include the content, the file, and the like. (The permanent storage device described above).

  Input / output interface 414 may be a means for interfacing with input / output device 415. For example, the input device may include a device such as a microphone, a keyboard, or a mouse, and the output device may include a device such as a display or a speaker. As another example, the input / output interface 414 may be a means for interfacing with a device that integrates functions for input and output, such as a touch screen. The input / output device 415 may be configured by the electronic device 410 and one device. The input / output interface 424 of the server 420 may also be a means for interfacing with an apparatus (not shown) for input or output that may be coupled to or included in the server 420.

  In other embodiments, electronic device 410 and server 420 may include more components than the components of FIG. However, most prior art components need not be clearly illustrated in the figure. For example, the electronic device 410 may be realized to include at least a part of the input / output device 415 described above, or may be a transceiver, a GPS (Global Positioning System) module, a camera, various sensors, a database, or the like. Other components may also be included. As a more specific example, when the electronic device 410 is a smartphone, generally an acceleration sensor or gyro sensor included in the smartphone, a camera module, various physical buttons, buttons using a touch panel, input / output ports, Various components such as a vibrator for vibration may be further included in the electronic device 410.

  In the present embodiment, the electronic device 410 may basically include a microphone for receiving the user's voice input as the input / output device 415, and a sound such as a reply voice or audio content corresponding to the user's voice input. May be further included as an input / output device 415.

  FIG. 5 is a flowchart illustrating an example of a device control method according to an embodiment of the present invention. The device control method according to the embodiment of the present invention may be executed by a computer apparatus such as the electronic apparatus 410 described above. At this time, the processor 412 of the electronic device 410 may be implemented so as to execute a control instruction (instruction) based on an operating system code included in the memory 411 or a code of at least one program. Here, the processor 412 controls the electronic device 410 so that the electronic device 410 executes steps 510 to 540 included in the device control method of FIG. 5 according to the control command provided by the code stored in the electronic device 410. You can do it.

  In operation 510, the electronic device 410 may receive a command word by voice input from a speaker through a voice based interface. For example, the electronic device 410 may receive a voice input by a user's utterance through a voice input device such as a microphone included in the electronic device 410 or a microphone linked to the electronic device 410. Techniques for receiving a speaker's voice input will be readily apparent to those skilled in the art from well-known techniques for voice recognition.

  In operation 520, the electronic device 410 may obtain the position information of the speaker related to the voice input when the result of analyzing the command word received by the voice input of the speaker and the position determination is necessary. As an example, the electronic device 410 may include a command pronoun (eg, here, there, over there, here, over there, over there, over there, over there, over there, the location, etc.) In the case where an incomplete sentence is omitted, additional processing for position determination may be performed. Therefore, when it is difficult to specify the position to be controlled and the device to be controlled only by the command word by voice input, the electronic device 410 may additionally acquire the position information of the speaker based on the command word. The electronic device 410 acquires the position information of the speaker related to the voice input when it is necessary to estimate the position determination. At this time, the position information of the speaker related to the voice input is related to the reception of the voice input. At least one of the relative position and direction of the speaker with respect to the electronic device 410 measured at the time or period, the presence / absence of a change in the relative position and direction, and the direction of change in the relative position and direction. May include.

  For example, the electronic device 410 may acquire position information related to voice input based on a phase shift of voice input input to a plurality of microphones included in the voice-based interface. A person skilled in the art can easily understand a technique for measuring a generation position of a sound signal by using a phase shift of the same sound signal inputted to a plurality of microphones from a known technique such as a beam forming technique. Will. In this case, since the position information is measured by the voice input of the speaker, the speaker does not need to face a specific direction, and the speaker's position is within a distance that allows the speaker's speech to be recognized. There is no limit. Further, it is not necessary to add another device other than the voice-based interface to the electronic device 410, and it is possible to acquire the position information of the speaker.

  In some embodiments, it may be considered that the electronic device 410 includes additional equipment such as a camera or a sensor in order to acquire the speaker's position information. The position information of the speaker measured using additional equipment such as a sensor or a sensor may be used. When utilizing a camera and / or sensor, the electronic device 410 may obtain position information associated with the audio input based on the output value of the camera and / or sensor when the audio input is received.

  As an example, the electronic device 410 may poll the optical device to visually locate the speaker. Optical devices such as cameras may utilize recognition software (eg, face recognition, feature recognition, etc.) to identify the user. Here, polling may include receiving information without or requesting information for obtaining optical information with an optical device. In addition to optical devices, it is also possible to poll the audio device to visually locate the speaker. The voice tool may be used to identify a speaker by voice input based on a pre-recorded voice profile.

  Further, the electronic device 410 may acquire the position information of the speaker related to the voice input based on the conversational strategy and the device control history by the voice input of the speaker. In other words, the electronic device 410 grasps the previous conversation of the speaker and the device control history to determine the relative position and direction of the speaker, whether there is a change in the relative position and direction, and the relative position. Alternatively, position information including at least one of the change directions of the direction may be acquired. For example, recently, if there is an operation history such as a question about a recipe for cooking or a timer setting for cooking through an interface that operates based on voice, it will be included in the conversation strategy and device control history of the voice input instructing the corresponding operation Based on this, it may be estimated that the current position or the latest position of the speaker is “kitchen”.

  In operation 530, the electronic apparatus 410 may identify a control target device based on the position information of the speaker with respect to the command word received by the voice input of the speaker. In the case where it is difficult to specify the control target position and the control target device only by the command word by voice input, the electronic device 410 is a device suitable for the voice input of the speaker based on the position information of the speaker related to the voice input. May be specified.

  As an example, when a command pronoun is included in the command word received by the voice input of the speaker, the electronic device 410 may specify the control target device based on the position information of the speaker. For example, the electronic device 410 may be a device that can operate in response to the voice input of a speaker, a device that is closest to the speaker, a device that is farthest from the speaker, or in the direction of the speaker. A device that is located or a device that is located at a location opposite to the direction of the speaker may be specified as the device to be controlled. As an example, if a command word received by voice input of a speaker includes a pronoun meaning such as “here”, “here”, “here”, or the like, the device closest to the speaker or A device located in the direction of the speaker may be specified as a control target device. As another example, when the command word received by the voice input of the speaker includes a pronoun meaning such as “over”, “over”, “over”, etc., it is farthest from the speaker A device located at a location opposite to the direction of the device or the speaker may be specified as the control target device. As another example, if the command word received by the voice input of the speaker includes a pronoun that means “there”, “the place”, etc. A certain device may be specified as a control target device.

  As another example, when the command word received by the voice input of the speaker is composed of an incomplete sentence in which the location or the target object is omitted, the electronic device 410 is controlled based on the position information of the speaker. You may identify the device. That is, the electronic device 410 may estimate a portion corresponding to the instruction target omitted from the conversation pulse based on the position information of the speaker. At this time, the electronic device 410 may specify the control target device based on the location information of the speaker estimated from the previous conversational strategy or the device control history by the voice input of the speaker. For example, in the case of a command word in which the location and the instruction target are omitted, such as voice input “with video”, the speaker has been in the list of devices that can handle the voice input of the speaker until recently. The device at the position may be specified as the control target device.

  When two or more devices are identified as control target devices for one control operation corresponding to the command word received by the voice input of the speaker, the electronic apparatus 410 is based on the device usage pattern and the device control history. You may select a device that suits your intention. When it is difficult to specify a control target position or a control target device only with a command word by voice input (electronic device 410) (when there are a plurality of target candidates), control target device based on the position information of the speaker related to the voice input Is identified. At this time, when a plurality of devices to be controlled are specified, a more suitable device may be selected from the specified devices based on user behavior pattern learning. For example, when a command such as voice input “with animation of animation character A” is received, a device history indicating that the corresponding content is mainly played is grasped, and a device having the largest history is controlled. You may select as a target device.

  Therefore, when it is difficult to specify a device to be controlled only by a command word by voice input, the electronic device 410 estimates a device that matches the speaker's intention based on the position information of the speaker related to the voice input. Can do. Furthermore, the electronic device 410 can select a more suitable control target device by grasping the conversational strategy by the voice input of the speaker, or grasping the device usage pattern or the device control history.

  In operation 540, the electronic apparatus 410 may transmit the corresponding instruction word to the control target device identified in operation 530 in order to control an operation corresponding to the instruction word received by the voice input of the speaker. The electronic device 410 may transmit a command word based on the voice input of the speaker to the corresponding device so that an operation suitable for the intention of the speaker is performed in the controlled device. The electronic device 410 may output information for confirming and recommending the position of the control target and the control target device before transmitting the command word by the voice input of the speaker to the control target device. For example, the electronic device 410 recognizes and analyzes a voice input “turn off electricity”, and confirms or recommends a specific information such as “Let's turn off the living room?” From a speaker. When the voice input “Yes” is received, an instruction may be transmitted to the corresponding device to turn off the lighting device in the living room.

  Therefore, the electronic device 410 can control device operation and content provision that match the intention of the speaker by additionally determining the position of the speaker with respect to the voice input in which the position and the instruction target are not specified. it can. As a result, even if the utterance form uses a pronoun, or the utterance form omits the position and the target, etc., the temperature control for each display, lighting, many speakers, and rooms is based on the position of the speaker. It is possible to selectively control a plurality of devices having the same function existing in the house such as an apparatus.

  6 to 7 are diagrams illustrating an example in which a control target device is determined according to a voice input including a pronoun in one embodiment of the present invention.

  In FIG. 6, the electronic device 410 including the voice-based interface recognizes and analyzes the user 110's utterance “Turn on the TV here” and controls the power supply of the home TV 610 linked with the electronic device 410 via the internal network ( That is, an example is shown in which the power is turned on. Since the pronoun pronoun “here” is included in the voice input “turn on TV here” of the user 110, it is necessary to estimate the position determination, thereby additionally acquiring the position information of the user 110. The control target device intended by the user 110 may be selected based on the above.

  The positional relationship between the speaker and the device may be determined in advance by the demonstrative pronoun. For example, if a demonstrative pronoun with a meaning such as "here", "here", or "here" is included, the speaker If the device located closest to or in the direction of the speaker contains a pronoun that means “over”, “over”, “over”, etc., the position farthest from the speaker If the device is located in a location opposite to the speaker's direction or contains a pronoun that means something like `` There '', `` That location '', etc., the previous location where the speaker was most recently May be specified as a device to be controlled.

  The electronic device 410 may manage the positions of a plurality of different devices that are linked to the electronic device 410. Here, the position of the device in the home may be set in various ways. For example, such a position may be input and set in the electronic device 410 by the user 110 or an administrator, or is well-known such as the strength of a signal that the electronic device 410 transmits / receives to communicate with another electronic device. It may be measured by utilizing the position positioning technique. In the case of a mobile device, the position may be measured dynamically using a positioning technique. At this time, the electronic device 410 further determines the control target device by further using the distance between the user 110 and the plurality of other electronic devices measured based on the position of the user 110 and the positions of the plurality of other devices. Good. Here, as described above, the position of the user 110 is measured based on the utterance of the user 110.

  For example, as shown in FIG. 6, when the user 110 stands in the living room and utters “Turn on TV here”, the pronoun “here” is included in the voice input to estimate the position of the user 110, The TV 610 in the living room closest to the position of the user 110 may be selected as a control target device from among a plurality of TVs in the house.

  As another example, FIG. 7 shows an example in which an electronic device 410 including a voice-based interface recognizes and analyzes the user 110's utterance “turn on an air conditioner in that room” and links the electronic device 410 via an internal network. The example which controls the power supply of the temperature control apparatus 710 is shown. When the user 110 stands in the living room and utters “turn on the air conditioner in that room”, the position of the user 110 is estimated by including the instruction pronoun “that” in the voice input, and the temperature control device in the house 110 The temperature control device 710 in the room farthest from the position of the user 110 may be selected as a control target device.

  8 to 9 are diagrams illustrating an example in which a control target device is determined according to a voice input in which a position and an instruction target are omitted according to an embodiment of the present invention.

  FIG. 8 shows that the electronic device 410 including the voice-based interface recognizes and analyzes the utterance “add video” of the user 110, and controls (turns on) the in-home display 810 linked to the electronic device 410 via the internal network. Or, a moving image is reproduced). For example, when the user 110 utters “with video” within a predetermined time after performing an operation such as a recipe question or setting a timer for cooking through the voice-based interface, the conversational context of the previous voice input Alternatively, it may be estimated that the user 110 is located in the kitchen from the device control history, and the corresponding content may be transmitted to a display (for example, a TV or a refrigerator display) 810 installed near the kitchen. Alternatively, it is also possible to estimate that the position of the user 110 is in the vicinity of the kitchen by analyzing the conversation of the utterance content itself with respect to the utterance of the user 110 “with a recipe video of bulgogi”.

  Referring to FIG. 9, when the user 110 makes a request for cooking in the kitchen through the voice-based interface and then returns to the living room and utters “Turn off the electricity there”, the voice pronoun “There” is included. Thus, the position of the user 110 may be estimated, and the illumination 910 at the previous position from which the user 110 has been located recently may be selected as the control target device.

  FIG. 10 is a diagram illustrating an example of determining a control target device based on user behavior pattern learning in an embodiment of the present invention.

  FIG. 10 shows that the electronic device 410 including the voice-based interface recognizes and analyzes the user 110's utterance “turn on the animation of the animation character A” and controls the home display 1010 linked to the electronic device 410 via the internal network. An example is shown. The electronic device 410 may additionally determine the position of the user 110 because the position, the instruction target, and the like are omitted from the user 110's utterance “add the animation of the animation character A”. The control target device may be selected based on the above. For example, when a TV 610 and a tablet 1010 exist as devices capable of playing a video in the living room where the user 110 is located, the device history in which the corresponding content is mainly played back is grasped, and the corresponding history from the TV 610 and the tablet 1010 is obtained. The device with the largest number may be selected to control content playback. Similarly, when there are a plurality of devices capable of outputting audio content in the vicinity of the user 110 with respect to the utterance “hip hop” of the user 110, the device in which the corresponding music genre is mainly played back A history may be grasped, and a device with the largest number of corresponding history may be selected as a control target device to control content reproduction.

  As described above, according to the embodiment of the present invention, after additionally determining the position of the speaker according to the command word by the voice input of the speaker, a device suitable for the command word is selected to provide device operation and content. Can be controlled. In particular, when a command word includes a command pronoun or is composed of an incomplete sentence in which a position and a command target are omitted from the command word, a device suitable for the user's intention can be estimated based on the user's position. At the same time, it is possible to estimate a device that meets the user's intention by grasping the conversational tactics by the voice input of the speaker or grasping the device usage pattern and the device control history.

  The system or apparatus described above may be realized by a hardware component, a software component, or a combination of a hardware component and a software component. For example, the devices and components described in the embodiments include, for example, a processor, a controller, an ALU (arithmic logic unit), a digital signal processor, a microcomputer, an FPGA (field programmable gate array), a PLU (programmable logic unit), a micro It may be implemented using one or more general purpose or special purpose computers, such as a processor or various devices capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the OS. The processing device may also respond to software execution, access data, and store, manipulate, process, and generate data. For convenience of understanding, one processing device may be described as being used, but those skilled in the art may include a plurality of processing elements and / or multiple types of processing elements. You can understand. For example, the processing device may include a plurality of processors or a processor and a controller. Other processing configurations such as parallel processors are also possible.

  The software may include computer programs, code, instructions, or a combination of one or more of these, configuring the processor to operate as desired, or instructing the processor independently or collectively. You may do it. Software and / or data may be embodied on any type of machine, component, physical device, virtual device, computer storage medium or device to be interpreted based on the processing device or to provide instructions or data to the processing device. May be used. The software may be distributed over computer systems connected by a network and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

  The method according to the embodiment may be realized in the form of program instructions executable by various computer means and recorded on a computer-readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The medium may continuously store a program that can be executed by a computer, or may be temporarily stored for execution or download. The medium may be a variety of recording means or storage means in which a single piece or a plurality of pieces of hardware are combined. However, the medium is not limited to a medium that is directly connected to any computer system. It may be dispersed on the surface. Examples of the medium include a magnetic medium such as a hard disk, a floppy (registered trademark) disk, and a magnetic tape, an optical medium such as a CD-ROM and a DVD, a magneto-optical medium such as a floppy disk, The program instruction word may be stored including ROM, RAM, flash memory and the like. Examples of other media include an application store that distributes applications, a site that supplies or distributes various software, a recording medium that is managed by a server, or a storage medium. Examples of program instructions include not only machine language code such as that generated by a compiler, but also high-level language code executed by a computer using an interpreter or the like.

  As mentioned above, although embodiment was described based on limited embodiment and drawing, those skilled in the art will be able to perform various correction and deformation | transformation from the above-mentioned description. For example, the described techniques may be performed in a different order than the described method and / or components of the described system, structure, apparatus, circuit, etc. may be different from the described method. Appropriate results can be achieved even when combined or combined, or opposed or replaced by other components or equivalents.

  Accordingly, even different embodiments belong to the appended claims as long as they are equivalent to the claims.

310: Electronic device 320: Cloud artificial intelligence platform 330: Content service 340: Interface connect 350: Expansion kit

Claims (20)

A device control method executed by an electronic device including an audio infrastructure interface,
Receiving a command word by a user's voice input through the voice infrastructure interface;
If position information is required as a result of analysis of the command word, obtaining position information related to the voice input; and using the position information, a control target among devices that can be controlled by the electronic device A device control method comprising: specifying a device and causing the control target device to execute an operation corresponding to the command word. The obtaining step includes
If the command word includes a pronoun, a position information associated with the voice input is acquired.
The device control method according to claim 1. The obtaining step includes
When the command word is composed of an incomplete sentence in which the location and the instruction target are omitted, the position information related to the voice input is acquired.
The device control method according to claim 1. The positional information related to the voice input is
The relative position and direction of the user relative to the electronic device measured at the time or period associated with the reception of the audio input, the presence or absence of a change in the relative position and direction, and the relative position and direction. Including at least one of the changing directions,
The device control method according to claim 1. The obtaining step includes
Obtaining positional information related to the voice input based on a phase shift of the voice input inputted to a plurality of microphones included in the voice base interface;
The device control method according to claim 1. The electronic device includes at least one of a camera and a sensor,
The obtaining step includes
Obtaining positional information associated with the audio input based on an output value of at least one of the camera and sensor when the audio input is received;
The device control method according to claim 1. The obtaining step includes
Grasping at least one of the conversational abbreviation corresponding to the voice input and the device control history of the user by the voice-based interface to acquire position information related to the voice input;
The device control method according to claim 1. The identifying step includes:
The positional relationship between the user and the device is determined by the indicating pronoun, and the device to be controlled is specified using the positional information related to the voice input according to the positional relationship.
The device control method according to claim 2. The identifying step includes:
Along with the position information related to the voice input, the device to be controlled is specified using the device usage pattern of the user based on the voice-based interface.
The device control method according to claim 1. The identifying step includes:
When there are a plurality of devices capable of executing an operation corresponding to the command word, a device having a relatively high frequency of use among a plurality of devices using the user device usage pattern by the voice-based interface Identify the device to be controlled,
The device control method according to claim 1.   The computer program which makes a computer perform the method as described in any one of Claims 1-10.   The computer-readable recording medium with which the program for making a computer perform the method as described in any one of Claims 1-10 is recorded. Electronic equipment,
A voice based interface and at least one processor implemented to execute computer readable instructions;
The at least one processor comprises:
Receiving a command word by a user's voice input through the voice base interface;
As a result of the analysis of the command word, if position information is required, position information related to the voice input is acquired,
Using the position information, specify a control target device from devices that can be controlled by the electronic device, and cause the control target device to perform an operation corresponding to the command word.
Electronics. The at least one processor is configured to obtain location information associated with the voice input.
When the command word includes a pronoun or when the command word is composed of an incomplete sentence in which a location or a command object is omitted from the command word, the position information related to the voice input is acquired.
The electronic device according to claim 13. The positional information related to the voice input is
The relative position and direction of the user relative to the electronic device measured at the time or period associated with the reception of the audio input, the presence or absence of a change in the relative position and direction, and the relative position and direction. Including at least one of the changing directions,
The electronic device according to claim 13. The at least one processor is configured to obtain location information associated with the voice input.
Obtaining positional information related to the voice input based on a phase shift of the voice input inputted to a plurality of microphones included in the voice base interface;
The electronic device according to claim 13. The electronic device includes at least one of a camera and a sensor,
The at least one processor is configured to obtain location information associated with the voice input.
Obtaining positional information associated with the audio input based on an output value of at least one of the camera and sensor when the audio input is received;
The electronic device according to claim 13. The at least one processor is configured to obtain location information associated with the voice input.
Grasping at least one of the conversational abbreviation corresponding to the voice input and the device control history of the user by the voice-based interface to acquire position information related to the voice input;
The electronic device according to claim 13. The at least one processor is configured to obtain location information associated with the voice input.
If the command word includes a pronoun, the position information associated with the voice input is acquired,
The at least one processor specifies the device to be controlled,
The positional relationship between the user and the device is determined by the indicating pronoun, and the device to be controlled is specified using the positional information related to the voice input according to the positional relationship.
The electronic device according to claim 13. The at least one processor specifies the device to be controlled,
Along with the position information related to the voice input, the device to be controlled is specified using the device usage pattern of the user based on the voice-based interface.
The electronic device according to claim 13.

Images