JP2018206385A - Method and device for actively providing information suitable for position and space for user - Google Patents

Abstract

To provide a method and a device for actively providing information suitable for the position and the space of a user.SOLUTION: An information providing device 420 includes at least one processor 422, which realizes execution of a command that a sound-based input/output interface and a computer can read. The at least one processor 422 determines the space where a user is positioned from a plurality of spaces and interprets a sound request of the user received by the sound-based input/output interface on the basis of the determined space. Alternatively, the processor provides the user with the information actively determined by the at least one processor 422 on the basis of the determined space via the sound-based input/output interface 424.SELECTED DRAWING: Figure 4

Description

  The following description is directed to a method and apparatus for actively providing information suitable for the position and space of a user, a computer program that causes a computer to execute an information providing method according to an embodiment of the present invention in combination with a computer, and the computer The present invention relates to a recording medium for storing a program.

  An interface that operates based on voice, such as an artificial intelligence speaker of a home network service, receives a user's voice request through a microphone and synthesizes a reply voice to provide response information corresponding to the voice request. Audio content such as music content can be provided from a speaker or as response information. For example, Patent Document 1 is a technology related to a home media device and a home network system and method using the same, and provides a home network service using a second communication network such as Wi-Fi in addition to a mobile communication network. A technique is disclosed in which a plurality of multimedia devices in a home can be multi-controlled based on a voice command without a user operating a button.

  In such a home network service, in order to provide optimal information as a response to the user's voice request, information on the indoor structure and the space in which a device such as an artificial intelligence speaker is located, information on the space in which the user is located, etc. Need to figure out. However, in the prior art, the image taken by the camera must be analyzed to grasp the indoor structure, and the user must manually input information and position regarding the space where the device is located. Furthermore, even if the image captured by the camera is used to automatically grasp the space, the camera can only obtain an image of the surrounding space where the device is placed. , Kitchen, bedroom, entrance, etc.), the user has to specify the information. In addition to this, when the position of the device is changed, there is a problem that the user has to manually reset the information regarding the set space and position.

Korean Published Patent No. 10-2011-0139797

  The present invention can be combined with an information providing method and apparatus capable of determining a space where a user is located from among partitioned spaces in a room and providing information suitable for the space where the user is located, and a computer. In order to cause a computer to execute the information providing method according to the embodiment, a computer program stored in a computer-readable recording medium and the recording medium are provided.

  Even if there is no setting input from the user, it is possible to position the partitioned space in the room and determine the type of each space, and to determine the type of each space based on the type of the space, the user's situation, and / or the action pattern. Information providing method and apparatus capable of improving usefulness of information provision by providing information / content highly relevant to context, and information providing method according to an embodiment of the present invention combined with a computer There are provided a computer program to be executed and a recording medium for storing the computer program.

  One aspect of the invention includes a voice-based input / output interface and at least one processor implemented to execute computer readable instructions, the at least one processor being a user from among a plurality of spaces. Determines the space in which the user is located and interprets the user's voice request received at the voice based input / output interface based on the determined space or is active in at least one processor based on the determined space The information providing apparatus is characterized in that information determined by the user is provided to a user through an audio-based input / output interface.

  In another aspect of the present invention, there is provided an information providing method for an information providing apparatus, comprising: determining a space where a user is located from a plurality of spaces; and an audio-based input / output interface included in the information providing apparatus. Interpreting a received voice request of a user or including providing information to a user through a voice-based input / output interface, which is actively determined based on the determined space, Provide information provision methods.

  In another aspect of the present invention, there is provided a computer-readable recording medium in which a computer program for causing a computer to execute the information providing method is recorded.

  In another aspect of the present invention, a computer program is provided that causes a computer to execute an information providing method in combination with a computer.

  ADVANTAGE OF THE INVENTION According to this invention, the space where a user is located can be determined from the indoor partitioned space, and the information suitable for the space where a user is located can be provided.

  According to the present invention, even if there is no setting input from the user, it is possible to position the partitioned space in the room and determine the type of each space, and to determine the type of space and the user's situation and / or behavior pattern. By providing information / content that is highly relevant to the user context for each space, it is possible to improve the usefulness of information provision.

It is the figure which showed the example of the service environment using the voice-based interface in one Embodiment of this invention. It is the figure which showed the other example of the service environment using the voice-based interface in one Embodiment of this invention. It is the figure which showed the example of the cloud artificial intelligence platform in one Embodiment of this invention. It is a block diagram for demonstrating the internal structure of the electronic device and server in one Embodiment of this invention. It is the figure which showed the example of the information regarding the indoor partitioned space and position in one Embodiment of this invention. It is the figure which showed the example which grasps | ascertains physical space with the single apparatus in one Embodiment of this invention. It is the figure which showed the example which grasps | ascertains physical space using several sub apparatus in one Embodiment of this invention. It is the figure which showed the example which provides information from several output media in one Embodiment of this invention. It is the figure which showed the example of the information regarding an output medium in one Embodiment of this invention. It is the flowchart which showed the example of the information provision method of the electronic device in one Embodiment of this invention. 6 is a flowchart illustrating an example of a method for determining a space in which a user is located according to an exemplary embodiment of the present invention. It is the flowchart which showed the other example of the method of determining the space where a user is located in one Embodiment of this invention. 6 is a flowchart illustrating an example of a method for predicting a user's situation and / or behavior and providing information based on the predicted situation and / or behavior in an embodiment of the present invention.

  Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

  The information providing apparatus according to the embodiment of the present invention may be realized by an electronic device that provides an interface that operates based on voice. Such an electronic device may directly process the voice request to generate response information, or may recognize and analyze the voice request by an external platform connected through a separate interface connect. For example, the external platform includes a cloud artificial intelligence platform that receives a user's voice request from an electronic device, analyzes the received voice request, and generates response information corresponding to the user's intention or corresponding query. Good. At this time, the electronic device may determine a space where the user is located from a plurality of spaces partitioned in the room, and actively provides information to the user according to the space where the user is located. Good. For example, the electronic device provides information suitable for the space in which the user is located regardless of the user's voice request, and adds information suitable for the space in which the user is located in the process of providing a response to the user's voice request. Further may be provided. As another example, the electronic device may interpret the same voice request of the user differently according to the space where the user is located, and may correct a recognition error for the voice request of the user.

  The information providing method according to the embodiment of the present invention may be executed by the electronic device described above. At this time, the computer program according to the embodiment of the present invention may be installed and driven in the electronic device, and the electronic device executes the information providing method according to the embodiment of the present invention in accordance with the control of the computer program to be driven. You can do it. The computer program described above may be stored in a computer-readable recording medium so that the computer can execute an information providing method in combination with an electronic device realized by the computer.

  FIG. 1 is a diagram showing an example of a service environment using a voice-based interface in an embodiment of the present invention. In the embodiment of FIG. 1, in a technology for connecting and controlling home devices such as a smart home and a home network service, an electronic device 100 including an audio-based interface (for example, a microphone and a speaker) is provided. An example of recognizing and analyzing a voice request by receiving the user 110's utterance “turn off electricity” 121 with a microphone and outputting response information such as “let's turn off the living room electricity?” 122 from the speaker. Show. Further, in the embodiment of FIG. 1, the home lighting device 130 that recognizes and analyzes the voice request received according to the utterance “Yes” 123 of the user 110 corresponding to the response information and is linked to the electronic device 100 via the internal network. The example which controls a light power supply is shown.

  Here, in addition to the above-described home lighting device 130, home appliances include home appliances such as IPTV, PC (Personal Computer), peripheral devices, air conditioners, refrigerators, robot cleaners, water supply, electricity, It may include various devices that can be connected and controlled on-line, such as energy consuming devices such as air-conditioning devices, and security devices such as door locks and surveillance cameras. In addition, the internal network includes wired network technologies such as Ethernet (registered trademark), HomePNA, IEEE 1394, Bluetooth (registered trademark), UWB (ultra Wide Band), and ZigBee (registered trademark). Wireless network technologies such as Wireless 1394 and Home RF may be used.

  The electronic device 100 may be one of home devices. For example, the electronic device 100 may be one of devices such as an artificial intelligence speaker and a robot cleaner provided in the house. The electronic device 100 is a mobile device of the user 110 such as a smart phone, a mobile phone, a notebook type pancon, a digital broadcasting terminal, a PDA (Personal Digital Assistant), a PMP (Portable Multimedia Player), and a tablet. There may be. As described above, the electronic device 100 is not particularly limited as long as the electronic device 100 includes a function that can be connected to a home device in order to receive the voice request of the user 110 and control the home device. In addition, depending on the embodiment, the mobile device of the user 110 described above may be included as a home device.

  The response information may be output from a main medium (as an example, a speaker included in the electronic device 100) basically corresponding to the voice-based interface as in the embodiment of FIG. 1, but as described above, In the embodiment, the response information may be output from various media, and the electronic device 100 may determine a medium on which the response information is output according to the voice request and / or the response information. For example, FIG. 1 further shows the IPTV 140 and the mobile device 150 of the user 110. In this case, the response information may be visually output to the user 110 from the display of the IPTV 140 or the mobile device 150. At this time, the electronic device 100 may determine the medium for outputting the response information based on the efficiency of the audio output and the visual output as well as the medium for outputting the response information.

  FIG. 2 is a diagram illustrating another example of a service environment using a voice-based interface according to an embodiment of the present invention. FIG. 2 shows that the electronic device 100 including the voice-based interface recognizes and analyzes the utterance “Today's weather” 210 of the user 110, acquires information about today's weather from the external server 220 via the external network, and acquires the acquired information. Shows an example of synthesizing with voice and outputting, such as “Today's weather is ...” 230. For example, the external network includes a PAN (personal area network), a LAN (local area network), a MAN (metropolitan area network, etc.), a WAN (wide area network, etc.), a WAN (wide area network, etc.), and a WAN (wide area network, etc.). Any one or more of the networks may be included.

  Also in the embodiment of FIG. 2, the electronic device 100 may be one of the devices in the home or one of the mobile devices of the user 110, and receives and processes the voice request of the user 110. And a function including a function for communicating with the external server 220 via an external network and a service or content provided by the external server 220 to the user 110 is not particularly limited. As described above, the electronic device 100 according to the embodiment of the present invention is not particularly limited as long as the electronic device 100 can process a voice request received according to the utterance of the user 110 using a voice-based interface. For example, the electronic device 100 may directly recognize and analyze the user's voice request and process the voice request. However, depending on the embodiment, the electronic device 100 may include an external platform linked to the electronic device 100 (for example, the above-described cloud artificial Voice requests may be processed by an intelligence platform.

  At this time, the electronic device 100 may automatically determine a medium for outputting response information corresponding to the user's voice request based on the voice request and / or response information. FIG. 2 shows an example in which the electronic device 100 outputs the response information audibly such as “Today's weather is ...” 230 and at the same time visually output from the display of the IPTV 140. To this end, the electronic device 100 may manage information on a plurality of media that can output response information, and output response information from the plurality of media based on the voice request and / or response information. At least one medium may be selected and response information communicated.

  FIG. 3 is a diagram illustrating an example of a cloud artificial intelligence platform according to an embodiment of the present invention. FIG. 3 shows an electronic device 310, a cloud artificial intelligence platform 320, and a content service 330.

  As an example, the electronic device 310 may mean a device provided in a home, and may include at least the electronic device 100 described above. Such an electronic device 310 or an application installed and driven on the electronic device 310 (hereinafter referred to as an application) may be linked to the cloud artificial intelligence platform 320 via the interface connect 340. Here, the interface connect 340 may provide the developer with a software development kit (SDK) and / or development document for developing the electronic device 310 or an application installed and driven in the electronic device 310. . In addition, the interface connect 340 provides an application program interface (API) that allows the electronic device 310 or an application installed and driven to the electronic device 310 to utilize the functions provided by the cloud artificial intelligence platform 320. It's okay. As a specific example, a device or application developed by a developer using an SDK and / or development document provided by the interface connect 340 may be used for the cloud artificial intelligence platform 320 using the API provided by the interface connect 340. Allows you to take advantage of the functionality provided by.

  Here, the cloud artificial intelligence platform 320 may provide a function for providing a voice-based service. For example, the cloud artificial intelligence platform 320 recognizes received speech and synthesizes output speech, an audio processing module 321, a vision processing module 322 for analyzing and processing received video and moving images, and receiving A dialogue processing module 323 for determining an appropriate dialogue in order to output a suitable voice according to the received voice, a recommendation module 324 for recommending a function suitable for the received voice, and artificial intelligence by sentence learning Various modules for providing voice-based services may be included, such as Neural Machine Translation (NMT) 325 that assists in translating languages.

  For example, in the embodiment of FIGS. 1 and 2, it is assumed that the electronic device 100 transmits the voice request of the user 110 to the cloud artificial intelligence platform 320 using the API provided by the interface connect 340. In this case, the cloud artificial intelligence platform 320 may recognize and analyze the received voice request using the modules 321 to 325 described above, and synthesize and provide an appropriate response voice according to the received voice request. Appropriate actions may be recommended.

  Further, the extension kit 350 may provide a development kit that allows a third-party content developer or company to realize a new voice infrastructure function based on the cloud artificial intelligence platform 320. For example, in the embodiment of FIG. 2, the electronic device 100 may transmit the received voice request of the user 110 to the external server 220, and the external server 220 uses the API provided by the extension kit 350 to perform cloud artificial intelligence. A voice request may be sent to the platform 320. In this case, as described above, the cloud artificial intelligence platform 320 recognizes and analyzes the received voice request and synthesizes and provides an appropriate response voice, or recommendation information regarding a function to be processed by the voice request. May be provided to the external server 220. As an example, in FIG. 2, when the external server 220 transmits a voice request “Today's weather” to the cloud artificial intelligence platform 320, a keyword extracted from the cloud artificial intelligence platform 320 by recognizing the voice request “Today's weather”. It is assumed that “today” and “weather” are received. In this case, the external server 220 generates text information such as “Today's weather is ...” based on the keywords “Today” and “Weather”, and sends the generated text information to the cloud artificial intelligence platform 320. You may send it again. At this time, the cloud artificial intelligence platform 320 may synthesize the text information by voice and provide it to the external server 220. The external server 220 may transmit the synthesized voice to the electronic device 100, and the voice received from the user 110 by outputting the synthesized voice “Today's weather is ...” from the speaker. The request “Today's Weather” may be processed.

  FIG. 4 is a block diagram for explaining the internal configuration of the electronic device and the server in one embodiment of the present invention. The electronic device 410 in FIG. 4 may correspond to the electronic device 100 described above, and may correspond to the information providing apparatus according to the embodiment of the present invention. The server 420 may correspond to one computer device that implements the external server 220 or the cloud artificial intelligence platform 320 described above.

  The electronic device 410 and the server 420 may include memories 411 and 421, processors 412 and 422, communication interfaces 413 and 423, and input / output interfaces 414 and 424, respectively. The memories 411 and 421 are computer-readable recording media and include a RAM (Random Access Memory), a ROM (Read Only Memory), and a permanent mass storage device such as a disk drive. It's okay. Here, a permanent mass storage device such as a ROM and a disk drive may be included in the electronic apparatus 410 or the server 420 as another permanent storage device that is distinguished from the memories 411 and 421. The memories 411 and 421 include an operating system and at least one program code (for example, a code for an application installed in the electronic device 410 and driven by the electronic device 410 to provide a specific service). May be stored. Such software components may be loaded from a computer-readable recording medium different from the memories 411 and 421. Such another computer-readable recording medium may include a computer-readable recording medium such as a floppy (registered trademark) drive, a disk, a tape, a DVD / CD-ROM drive, and a memory card. In other embodiments, the software components may be loaded into the memories 411 and 421 through the communication interfaces 413 and 423 that are not computer-readable recording media. For example, the at least one program is based on a computer program (for example, the application described above) installed by a file distribution system that distributes an installation file of a developer or an application via a network 430. It may be loaded into the memory 411.

  The processors 412, 422 may be configured to process computer program instructions by performing basic arithmetic, logic and input / output operations. The instructions may be provided to the processors 412 and 422 by the memories 411 and 421 or the communication interfaces 413 and 423. For example, the processors 412, 422 may be configured to execute instructions received in accordance with program codes stored in a recording device such as the memory 411, 421.

  The communication interfaces 413 and 423 may provide a function for the electronic device 410 and the server 420 to communicate with each other via the network 430, and the electronic device 410 and / or the server 420 may be other electronic devices or other devices. A function for communicating with the server may be provided. As an example, a request generated by the processor 412 of the electronic device 410 according to a program code stored in a recording device such as the memory 411 may be transmitted to the server 420 via the network 430 according to the control of the communication interface 413. On the contrary, control signals, commands, contents, files, etc. provided in accordance with the control of the processor 422 of the server 420 are received by the electronic device 410 via the communication interface 423 and the network 430 and the communication interface 413 of the electronic device 410. May be. For example, a control signal, a command, a content, a file, and the like of the server 420 received through the communication interface 413 may be transmitted to the processor 412 and the memory 411, and the content and the file may be further stored in the electronic device 410 ( It may be stored in the permanent storage device described above.

  Input / output interface 414 may be a means for interfacing with input / output device 415. For example, the input device may include a device such as a microphone, a keyboard, or a mouse, and the output device may include a device such as a display or a speaker. As another example, the input / output interface 414 may be a means for interfacing with a device that integrates functions for input and output, such as a touch screen. The input / output device 415 may be configured by the electronic device 410 and one device. Also, the input / output interface 424 of the server 420 may be a means for interfacing with the server 420 or for interfacing with an input or output device (not shown) that the server 420 may include. .

  In other embodiments, the electronic device 410 and the server 420 may include fewer or more components than the components of FIG. However, most prior art components need not be clearly illustrated in the figure. For example, the electronic device 410 may be realized to include at least a part of the input / output device 415 described above, or may be a transceiver, a GPS (Global Positioning System) module, a camera, various sensors, a database, or the like. Other components may be further included. As a more specific example, when the electronic device 410 is a smartphone, an acceleration sensor, a gyro sensor, a camera module, various physical buttons, buttons using a touch panel, an input / output port, and vibration are generally included in the smartphone. Various components such as a vibrator for the electronic device 410 may be further included in the electronic device 410.

  In the present embodiment, the electronic device 410 includes a microphone (for example, the multi-channel microphone 417 in FIG. 4) for receiving a user's voice request and / or a speaker (for example, for outputting response information audibly). The speaker 416) of FIG. 4 may be included as an input / output device 415.

  At this time, the embodiment of the present invention may further consider a partitioned space in the room.

  FIG. 5 is a diagram illustrating an example of information related to a partitioned space and position in the room according to an embodiment of the present invention. A plan view 500 in FIG. 5 shows examples of types of indoor partitioned spaces (living room, kitchen, entrance, toilet, bedroom, etc.), positions of electronic devices 410 in the room, and positions 510, 520, and 530 of users. ing. For example, the space by the first position 510 of the user may be “living room”, the space by the second position 520 may be “kitchen”, and the space by the third position 530 may be “entrance”.

  The electronic device 410 may determine a space where the user is located from among the partitioned spaces in the room, and based on the determined space, an audio-based input / output interface included in the electronic device 410 (for example, 4 dynamically adjusts the interpretation of the user's voice request received at the input / output interface 414) of FIG. 4, or the information actively determined by the electronic device 410 based on the determined space It may be provided to the user through a platform input / output interface. For example, the electronic device 410 may provide different information to the user when the user is located in the “living room” that is the first position 510 and when the user is located in the “kitchen” that is the second position 520. As another example, in the case where the user is located in the “living room” which is the first position 510 and the case where the user is located in the “kitchen” which is the second position 520, the electronic device 410 has the same user utterance. May be interpreted differently.

  As a more specific example, electronic device 410 outputs “singer onion song” or “TV program related to onion” in response to the user's utterance “onion” located in “living room” that is first position 510. The electronic device 410 may provide the user with information related to “onion purchase” or “recipe using onion” for the user's utterance “onion” located in the “kitchen” that is the second position 520.

  As described above, in order to provide information suitable for each space, the electronic apparatus 410 may grasp a plurality of physical spaces partitioned in the room, and grasp such physical spaces and / or a user. The position (relative direction and distance from the electronic device 410) may be grasped periodically even when the user is not using the device.

  FIG. 6 is a diagram showing an example of grasping a physical space by a single device in an embodiment of the present invention. The electronic device 410 may output a sound signal from a speaker included in an audio-based input / output interface. Such a sound signal may be a sound band that the user can recognize or a sound band that the user cannot recognize. In addition, the electronic device 410 may receive an input of a reflected signal received when the output sound signal is reflected indoors by a multi-channel microphone for each channel. A sound signal output from one speaker spreads around the electronic device 410, is reflected by a wall surface of a partitioned boundary in the room, and is transmitted again to the electronic device 410. Such a reflection signal may be input for each channel of the multi-channel microphone of the electronic device 410. At this time, the relative direction of the reflected signal input to the multi-channel microphone with respect to the electronic device 410 is determined based on the phase shift of the specific reflected signal input for each channel of the multi-channel microphone. May be. A technique for measuring the generation position of a sound signal using the phase shift of the same sound signal input to a plurality of microphones is a well-known technique such as a beam forming technique. Would be easy to understand. In the present embodiment, the electronic device 410 receives a specific reflected signal for each channel of the multi-channel microphone, and determines the direction of the corresponding reflected signal based on the phase displacement of the specific reflected signal input for each channel. You can do it. In addition, the electronic device 410 may calculate information (for example, distance) related to the boundary between the electronic device 410 and the space partitioned based on the output time of the sound signal and the input time of the corresponding reflected signal for each channel. In other words, the electronic device 410 can determine a direction and a distance with respect to a specific reflected signal, which means that it is possible to measure a boundary of a space partitioned with the electronic device 410 as a reference. May mean. For example, the electronic device 410 can distinguish between the direction of a spatially closed place and the direction of an open place within a certain distance.

  Referring to FIG. 5 again, since the electronic device 410 is located in the “living room” which is the first position 510, the direction of the user's utterance is a direction to a spatially closed place. Recognize that they exist in the same space. In this case, the direction of the user's utterance may also be determined by the beamforming technique described above.

  On the other hand, in the electronic device 410, the direction of the user's utterance located at the “kitchen” at the second position 520 or the “entrance” at the third position 530 is a direction in which the user speaks electronically. You may recognize that it exists in the space different from the apparatus 410. FIG.

  FIG. 7 is a diagram illustrating an example of grasping a physical space using a plurality of sub devices in an embodiment of the present invention. The plan view 700 in FIG. 7 is similar to the plan view 500 in FIG. 5, the types of partitioned spaces in the room (living room, kitchen, entrance, toilet, bedroom, etc.), the position of the electronic device 410 in the room, and the user's Examples of positions 510, 520, and 530 are shown, and an example in which a plurality of sub devices 710, 720, and 730 are added is shown. The plurality of sub-devices 710, 720, and 730 may be a plurality of sub-speakers that can communicate with the electronic device 410 as an example, and include sensors (such as proximity sensors and IR sensors) that detect the approach of the user. It's okay. In this case, the electronic device 410 may manage information regarding the space in which each of the plurality of sub devices 710, 720, and 730 is located. The information regarding the space and the sensor information of the plurality of sub devices 710, 720, and 730 may be included. Based on this, the space where the user is located may be determined. For example, the electronic device 410 may determine the distance and direction of each of the plurality of sub-speakers based on the sound signal for each channel output from each of the plurality of sub-speakers and input to the multi-channel microphone. Further, based on the direction, information on a space where each of the plurality of sub devices is located may be generated.

  As a more specific example, when the sound signal output from the first sub device 710 is input to the multi-channel microphone of the electronic device 410, the electronic device 410 is relative to the direction in which the first sub device 710 is located. The correct distance may be determined. At this time, in the electronic device 410, the direction of the first sub device 710 is the direction to the closed place, and the relative distance from the direction in which the first sub device 710 is located is relative to the boundary of the space. As an example, the electronic device 410 grasps that the first sub device 710 is located in “a space different from the space in which the electronic device 410 is located” in the “clock 1 o'clock direction”. You can do it. When the first sub device 710 notifies the approach of the user by the sensor, the electronic device 410 is located in “a space different from the space in which the electronic device 410 is located” in the “clock 1 o'clock direction”. You may decide.

  Hereinafter, an embodiment for determining the type of “a space different from the space in which the electronic device 410 is located” grasped by the electronic device 410 will be described. For example, in the embodiment of FIG. 7 described above, the electronic device 410 has grasped that the user is located in the space where the first sub device 710 is located, but it is also necessary to grasp the type of such space. For this reason, the electronic device 410 may pattern the user information about at least one of the user's utterance content and the user activity related to the utterance content according to the time zone in which the user is located. For example, the electronic device 410 may store and accumulate user information in a database for each space and for each time zone, and may analyze the accumulated information for each space and for each time zone. When the user frequently requests utterances regarding “recipe”, “calorie”, “cooking ingredients”, etc. in “different space” in “clock 1 o'clock direction”, the electronic device 410 may “ “Different space” in “Direction” may be recognized as “kitchen”. As another example, when the user goes out of the room or the activity of entering from outside is detected in “different space” of “4 o'clock direction of the clock”, the electronic device 410 may read “4 o'clock of the clock”. “Different space” in “Direction” may be recognized as “entrance”. Such user activity may be determined based on external information such as the user's speech meaning the user's going out, presence or absence of the user's location and / or the user's schedule. For example, the electronic device 410 has a promise at “7 pm” in the user's schedule, and the user's utterance “whole light off” is recognized in “different space” of “4 o'clock direction of the clock” at “6:30 pm”. After that, if the position of the user is not detected any more, it may be understood that the user has gone out of the room from a “different space” in the “clockwise 4 o'clock direction”. By such user behavior, the electronic device 410 can recognize that the “different space” in the “clockwise 4 o'clock direction” is the “entrance”. In addition, it can be understood by those skilled in the art by referring to the above-described example that the electronic device 410 can grasp the type of the corresponding space based on the speciality of each space. Will.

  Also, the electronic device 410 may determine the user's predicted situation or predicted behavior in the current time zone and the space in which the current user is located. The electronic device 410 may actively provide content suitable for the user based on such prediction. Such a predicted situation or predicted behavior may be based on the type of space in which the current user is located, and user information patterned by space and time zone. For example, in the embodiment of FIG. 5 and / or FIG. 7, based on the patterned user information of the user located in the “kitchen” that is the second location 520, the electronic device 410 is mainly responsible for the afternoon. You may know that you are requesting a recipe for dinner preparation around 4pm. If it is determined that the current user's position is “kitchen” around 4:00 pm, the electronic device 410 may automatically recommend the type of recipe for dinner preparation to the user regardless of the user's voice request. . As another example, when it is determined that the user located at the “entrance” is going out, the electronic device 410 uses information registered by the user (for example, user schedule information), You can advise them to bring a car key or umbrella. Furthermore, when the user is going out of the entrance and the room is turned on, after actively asking “Do you want to turn off the electricity?”, Add device control according to the user's response. The function may be executed.

  For this purpose, the electronic device 410 may further utilize external information such as schedule information and weather information. The schedule information may be acquired from a separate external service associated with the user, or the electronic device 410 may receive and manage schedule registration directly from the user. As another example, the electronic device 410 informs the user located in the “balcony” that it is “time to water the plant”, and informs the user located in the “kitchen” together with the remaining number of “eggs”. It is also possible to notify information related to “egg purchase”.

  As another example, the electronic device 410 may dynamically adjust the probability of interpreting the user's utterance content as a specific word based on the determined predicted situation or behavior of the user. For example, in the case of utterances with similar pronunciations, such as “dog and (Keran in Korean pronunciation)” and “metric (Keryang in Korean pronunciation)” It is expected that the probability of uttering the word "is relatively high. As a result, when an utterance with a pronunciation similar to “Kelan” is received, a user located in “Kitchen” is recognized as “Weighing”, whereas in a case of a user located in “Living” Can adjust the interpretation of the user's voice request to recognize as “dog”. As described above, the electronic device 410 according to the present embodiment may determine the type of each space by positioning the indoor space without any setting input from the user. By providing information / content that is highly relevant to the user context for each space based on the situation and / or behavior pattern, the usefulness of information provision can be improved. Further, even if there is no specific request from the user, the electronic device 410 may actively determine specific information based on the space in which the user is located and the user's situation and / or behavior pattern, and provide the information to the user. .

  The electronic device 410 may determine an output medium for information based on the position of the user. For example, the electronic device 410 may include a communication interface that communicates with a plurality of output media located in a room, and based on the positions of the plurality of output media and the user, at least one of the plurality of output media. The output medium may be determined. In this case, the electronic device 410 outputs at least one output determined through the communication interface to provide information to the user so that the information to be provided to the user is output from the determined at least one output medium. May be sent to media.

  FIG. 8 is a diagram showing an example in which information is provided from a plurality of output media in an embodiment of the present invention. The electronic device 410 may communicate with a plurality of output media. The plurality of output media may be devices capable of outputting information visually and / or audibly among the above-described devices in the home (including the user's mobile device). The plurality of output media may be applications installed in a device capable of outputting information visually and / or audibly. For example, the electronic device 410 may provide information to a user located in the “kitchen” that is the second location 520 through the first sub device 710 located in the “kitchen”. Such information may be a response to the user's voice request or information that the electronic device 410 actively provides to the user. As another example, the electronic device 410 may provide information to the user located in the “living room” that is the first position 510 through the IPTV 810. As described above, the electronic device 410 may select an output medium located in a space where the user is located, and may output information to be provided to the user from the selected output medium.

  In other embodiments, the electronic device 410 may provide information to the user's mobile device. For example, if the user is currently operating a mobile device, the electronic device 410 may receive information regarding the operation from the mobile device and may determine the output media as the mobile device that the user is operating. .

  If a change in the space where the user is located is recognized in the middle of providing information, the output media may also be changed. For example, when the space in which the user is located is changed from “kitchen” to “bedroom 1”, the electronic device 410 may change the output medium from the first sub device 710 to the second sub device 720.

  In addition, the electronic device 410 may manage information related to such a device in order to use the above-described home device (including the user's mobile device) as an output medium.

  FIG. 9 is a diagram illustrating an example of information regarding an output medium according to an embodiment of the present invention. The table 900 of FIG. 9 includes a device item 910 for identifying an electronic device that can communicate with the electronic device 410, a location item 920 for identifying the home position of another electronic device, and the other electronic device. An example including an output medium item 930 for identifying an output medium, a detailed item 940 including information on the output medium, and an information type item 950 for identifying what type of information the output medium can be associated with. Is shown. As shown in Table 900, the electronic device 410 may manage information related to output media of the corresponding other electronic device together with information related to other electronic devices linked in the house. In addition, such information may be linked in advance to an information type of voice request and / or response information to the voice request. For example, when a user's voice request or corresponding response information is analyzed as an information type related to “external specific location”, the electronic device 410 uses the map application of the electronic device A as an output medium of response information based on the table 900. May be determined as In this case, the response information may be transmitted to the electronic device A, and the electronic device A may provide the user with a map service corresponding to the response information through the map application of the electronic device A. As another example, the response information may be transmitted to a service server that provides a map service through a map application. In this case, the service server may provide a map service through a map application installed in the electronic device A. To this end, the electronic device 410 may further manage information such as information for accessing the service server and a user account for allowing the service server to identify the user's electronic device A. . For example, the electronic device 410 may access the service server using the URL that the table 900 further stores in association with the map application, and the service server transmits the response information together with the user account so that the service server After referring to the account, a service linked to the response information may be provided through a map application installed in the electronic device A.

  In addition, based on the table 900, the electronic device 410 transmits corresponding information to the electronic device D so that the information is visually output from the display 4 of the electronic device D when the user is located in the “kitchen”. It's okay. As another example, when there are a plurality of output media in one space, the electronic device 410 may select a device suitable for the type of information to be output or a device closer to the user. . At this time, in order to select an output medium of a device closer to the user in one space, a more specific device position may be managed in the table 900. The position of such an apparatus may be a relative position with respect to the electronic apparatus 410 as in the case of the user.

  FIG. 10 is a flowchart illustrating an example of an information providing method for an electronic device according to an embodiment of the present invention. The information providing method according to the present embodiment may be executed by a computer device such as the electronic device 410 described above. At this time, the processor 412 of the electronic apparatus 410 may be implemented to execute a control instruction (instruction) based on an operating system code included in the memory 411 or at least one program code. Here, the processor 412 controls the electronic device 410 so that the electronic device 410 executes steps 1010 and 1020 included in the information providing method of FIG. 10 according to the control command provided by the code stored in the electronic device 410. You can do it.

  In step 1010, the electronic device 410 may determine a space where the user is located from among a plurality of spaces. As an example, the plurality of spaces may include a partitioned space in a room such as a home to which a home network service is applied.

  In operation 1020, the electronic device 410 interprets the user's voice request received at the voice-based input / output interface included in the electronic device 410 based on the determined space, or based on the determined space. Actively determined information may be provided to the user through a voice based input / output interface. For example, the electronic device 410 interprets the voice request of the same user dynamically adjusted according to the determined space, or actively transmits information related to the space to the user without a request from the user. May be provided.

  As described above, in this embodiment, the user's position and space are not used in order to provide the already determined information, but the request from the user is actively adjusted according to the user's position and space. Or the electronic device 410 preferentially provides information that is preferentially related to a user context according to space, thereby improving the usefulness of information provision.

  FIG. 11 is a flowchart illustrating an example of a method for determining a space where a user is located according to an embodiment of the present invention. The steps 1110 and 1120 shown in FIG. 11 are included in the step 1010 of FIG. 10 and may be performed by the electronic device 410.

  In step 1110, the electronic device 410 obtains information on the partitioned space based on the sound signal output from the speaker and the reflected signal input to the multi-channel microphone by reflecting the output sound signal in the room. You may get it. For this reason, as described above, the electronic device 410 determines the direction of the specific reflected signal based on the phase displacement of the specific reflected signal input to the multi-channel microphone for each channel, and the output time of the sound signal, Based on the input time for each channel when a specific reflected signal is input to the multi-channel microphone, the relative distance from the boundary of the space in the direction of the specific reflected signal is determined, and the reflected signal corresponding to the sound signal A partitioned space may be recognized based on the direction and relative distance determined for each.

  In step 1120, the electronic device 410 may match the user's position with a specific space according to the acquired information regarding the space and the user's position measured based on the user's utterance. The position of the user may also be obtained by measuring a relative direction and / or distance from the user by receiving sound signals from the user's utterance by the multi-channel microphone by the electronic device 410 for each channel.

  As described above, according to the present embodiment, the electronic device 410 automatically obtains information about the partitioned space in the room without the user setting information on the space (such as the type, structure, and position of the space). It becomes possible to grasp what kind of space the user is located in.

  FIG. 12 is a flowchart illustrating another example of a method for determining a space in which a user is located according to an embodiment of the present invention. The steps 1210 and 1220 shown in FIG. 12 are included in the step 1010 of FIG. 10 and may be performed by the electronic device 410.

  In step 1210, the electronic device 410 may manage information regarding a space in which each of the plurality of sub devices is located. Here, the plurality of sub devices may include a sensor that senses the approach of the user, and may communicate with the electronic device 410 via the communication interface 413 included in the electronic device 410.

  In step 1220, the electronic device 410 may determine the space where the user is located based on the sensor information received from the plurality of sub devices and the information regarding the space where each of the plurality of sub devices is located by the communication interface 413. As an example, the plurality of sub devices may include a plurality of sub speakers. In this case, the electronic device 410 determines the distance and direction from each of the plurality of sub-speakers based on the channel-specific sound signal output from each of the plurality of sub-speakers and input to the multi-channel microphone. Based on the distance and direction, information regarding a space in which each of the plurality of sub devices is located may be generated. Depending on the embodiment, as described with reference to FIG. 9, position information regarding various home devices including a plurality of sub devices may be further utilized.

  According to the present embodiment, even if the user does not set information regarding the space (such as the type, structure, and position of the space), the electronic device 410 is provided with the sensor information provided from the plurality of sub devices and the plurality of sub devices. Based on the information on the space to be played, information on the partitioned space in the room can be acquired. The embodiment of FIG. 12 may be used in combination with the embodiment of FIG. 11 to obtain more accurate information about the space, which also makes the information about the space where the user is located more accurate.

  FIG. 13 is a flowchart illustrating an example of a method for predicting a user's situation and / or behavior and providing information based on the predicted situation and / or behavior in an embodiment of the present invention.

  In step 1310, the electronic device 410 may pattern user information of at least one of user's utterance contents and user activities by space in which the user is located according to time zones.

  In operation 1320, the electronic apparatus 410 may determine the type of each partitioned space based on the user information patterned by space and time zone.

  In operation 1330, the electronic apparatus 410 displays the user information in the current time zone and the space where the current user is located, or the user information patterned for each space and time zone, and the space where the current user is located. You may decide based on the kind of.

  In operation 1340, the electronic device 410 dynamically adjusts the probability of interpreting the user's utterance content into a specific word based on the determined user's predicted situation or behavior, or provides the user's utterance content according to the user's utterance content. Information to be determined or information to be provided to the user independently of the user's utterance may be determined.

  As described above, according to the present embodiment, the user's situation or behavior is predicted by utilizing the user information patterned according to the type of space in which the user is currently located and the space and / or the time zone. The electronic device 410 actively adjusts the interpretation of the user's utterance according to the predicted situation and behavior by space / time zone, or actively transmits information regardless of the user's voice request. Or can be provided.

  The system or apparatus described above may be realized by a hardware component, a software component, or a combination of a hardware component and a software component. For example, the apparatus and components described in the embodiments include, for example, a processor, a controller, an ALU (arithmetic logic unit), a digital signal processor, a microcomputer, an FPGA (field programmable gate array), a PLU (programmable logic unit), a microprocessor, Or it may be implemented using one or more general purpose or special purpose computers, such as various devices capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the OS. The processing device may also access data and store, manipulate, process and generate data in response to software execution. For convenience of understanding, a single processing device may be described as being used, but those skilled in the art may include a plurality of processing elements and / or multiple types of processing elements. You can understand. For example, the processing device may include a plurality of processors or a processor and a controller. Other processing configurations such as parallel processors are also possible.

  The software may include computer programs, code, instructions, or a combination of one or more of these, configuring the processing device to operate as desired, or instructing the processing device independently or collectively. You may do it. Software and / or data may be embodied on any type of machine, component, physical device, virtual device, computer storage medium or device to be interpreted based on the processing device or to provide instructions or data to the processing device. May be used. The software may be distributed over computer systems connected by a network and may be stored or executed in a distributed state. Software and data may be stored on one or more computer-readable recording media.

  The method according to the embodiment may be realized in the form of program instructions executable by various computer means and recorded on a computer-readable medium. A computer readable medium may include program instructions, data files, data structures, etc., alone or in combination. The medium may continuously store a program that can be executed by a computer, or may be temporarily stored for execution or download. The medium may be a variety of recording means or storage means in a form in which a single piece or a plurality of pieces of hardware are combined. However, the medium should not be limited to a medium directly connected to a computer system, It may exist in a dispersed manner. Examples of media include magnetic media such as hard disks, floppy disks and magnetic tape, optical media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, and It may be configured to store program instruction words including ROM, RAM, flash memory, and the like. Examples of other media may include an application store that distributes applications, a site that supplies or distributes various other software, a recording medium that is managed by a server, or a storage medium. Examples of program instructions include not only machine language code generated by a compiler but also high-level languages that can be executed by a computer using an interpreter or the like.

  As described above, the embodiments have been described based on the limited embodiments and the drawings, but those skilled in the art will be able to make various modifications and variations from the above description. For example, the described techniques may be performed in a different order than the described method and / or components of the described system, structure, apparatus, circuit, etc. may be combined in a different manner than the described method. Appropriate results can be achieved even if combined or combined, replaced or replaced by other components or equivalents.

  Accordingly, even different embodiments belong to the appended claims as long as they are equivalent to the claims.

100: Electronic device 110: User 140: IPTV
150: Mobile device 220: External server

Claims (17)

An audio based input / output interface and at least one processor implemented to execute computer readable instructions;
The at least one processor comprises:
Determine the space where the user is located from multiple spaces,
Interpret the user's voice request received at the voice based input / output interface based on the determined space, or actively determined by the at least one processor based on the determined space Information is provided to a user through the voice-based input / output interface. The voice-based input / output interface is:
Including a speaker that outputs sound signals and a multi-channel microphone,
The at least one processor comprises:
Based on the sound signal output from the speaker and the reflected signal input for each channel to the multi-channel microphone by reflecting the output sound signal in space, information on the plurality of spaces is obtained,
2. The information providing apparatus according to claim 1, wherein the user's position is matched with a specific space based on the acquired information on the plurality of spaces and the user's position measured based on the user's utterance. A communication interface for communicating with a plurality of sub-devices including a sensor that senses a user's approach
The at least one processor comprises:
Managing information about the space in which each of the plurality of sub-devices is located;
The space where the user is located is determined based on sensor information received from the plurality of sub-devices and information on a space where each of the plurality of sub-devices is located by the communication interface. The information providing device described in 1. The at least one processor comprises:
The user information of at least one of the user's utterance content according to the space where the user is located and the user's activity related to the utterance content is patterned by time zone,
The information providing apparatus according to claim 1, wherein a type of each of the plurality of spaces is determined based on user information patterned for each space and each time zone. The at least one processor comprises:
Determining a user's predicted situation or behavior in the current time zone and the space in which the current user is located based on the user information patterned for each space and time zone and the type of space in which the current user is located The information providing apparatus according to claim 4, wherein: The at least one processor comprises:
Based on the predicted state or predicted behavior of the determined user, dynamically adjusting the probability of interpreting the user's utterance content into a specific word, or determining information to be provided according to the user's utterance content. The information providing device according to claim 5, wherein The at least one processor comprises:
The information providing apparatus according to claim 5, wherein information to be provided to a user is actively determined based on the determined predicted situation or predicted behavior of the user without a request from the user. A communication interface for communicating with a plurality of output media;
The at least one processor comprises:
Determining at least one output medium from the plurality of output media based on a position of the plurality of output media and a position of a user;
Transmitting information for providing to the user by the communication interface to the determined at least one output medium such that information for providing to the user is output from the determined at least one output medium. The information providing apparatus according to claim 1, wherein: The at least one processor comprises:
When the user position is changed, the output medium is changed according to the changed user position,
The information to be provided to the user is transmitted to the changed output medium through the communication interface so that the information to be provided to the user is continuously output from the changed output medium. The information providing apparatus according to claim 8. An information providing method for an information providing apparatus, comprising:
Determining a space in which a user is located from a plurality of spaces, and interpreting a voice request of a user received at a voice-based input / output interface included in the information providing apparatus, or determining the determined space Providing the information actively determined based on the information to the user through the voice-based input / output interface. Determining the space in which the user is located,
A sound signal output from a speaker included in the audio-based input / output interface, and the output sound signal are reflected by a space and input to a multi-channel microphone included in the audio-based input / output interface for each channel. Acquiring the information about the plurality of spaces based on the reflected signal, and matching the user's position with the space based on the acquired information about the plurality of spaces and the user's position measured based on the user's utterance. The method of providing information according to claim 10, further comprising: Determining the space in which the user is located,
Managing information about a space in which each of a plurality of sub-devices including a sensor that senses the approach of a user is located; and sensor information received from the plurality of sub-devices by a communication interface of the information providing device; The information providing method according to claim 10, further comprising: determining a space where the user is located based on information regarding a space where each of the sub devices is located. Patterning the user information of at least one of the user's utterance content by space where the user is located and the user's activity related to the utterance content by time zone; and patterning by space and time zone The method according to claim 10, further comprising: determining a type of each of the plurality of spaces based on the user information. Determining a user's predicted situation or behavior in the current time zone and the space in which the current user is located based on the user information patterned for each space and time zone and the type of space in which the current user is located The information providing method according to claim 13, further comprising: The probability of interpreting the user's utterance content into a specific word is dynamically adjusted based on the determined user's predicted situation or prediction behavior, or information provided according to the user's utterance content is determined, or The information providing method according to claim 14, further comprising: actively determining information to be provided to the user without a request from the user.   A computer program that, when combined with a computer, causes the computer to execute the method according to any one of claims 10 to 15.   A computer-readable recording medium in which a program for causing a computer to execute the method according to any one of claims 10 to 15 is recorded.