KR101873538B1 - Method and computer program for controlling IoT devices using virtual keyboard - Google Patents

Abstract

A method for a mobile device to control an Internet of Things (IoT) device using a virtual keyboard is disclosed. The IoT device control method includes: displaying a dialog for a user of the mobile device and a user of the other party to communicate; Analyzing conversation contents entered in a dialog window and selecting one or more IoT devices that need to be controlled according to the analysis result; Displaying a virtual keyboard including a list of icons corresponding to the selected one or more IoT devices; And displaying the remote control image for controlling the corresponding IoT device when the user selects a desired icon on the virtual keyboard.

Description

TECHNICAL FIELD [0001] The present invention relates to a method and a computer program for controlling an IoT device using a virtual keyboard,

The present invention relates to control of an Internet of Things (IoT) device, and more particularly, to a method and a computer program for controlling an IoT device using a virtual keyboard of a social network service (SNS) service.

IoT (Internet of Things) is also called Internet of Objects or Internet, and refers to the concept that all objects that can communicate can be connected to the network to perform mutual communication. Every systemically recognizable entity is classified as Things or Objects, which can include objects or persons that are equipped with local and remote communication capabilities and can produce and use data, such as sensors.

In order to operate or monitor each IoT device in the IoT environment, various methods such as a dedicated application, a web access, and a SNS (Social Network Service) control are used.

In particular, a method for controlling an IoT device through an instant messenger service, which is one of the SNS services, has been actively researched. According to a method of giving an SNS account to an IoT device and exchanging status information in the form of an SNS message, There is a problem that it is cumbersome and difficult to use because a message is required to be input.

Korean Patent Laid-Open Publication No. 10-2015-0028175 (Publication date 2013.03.13), "Control device"

SUMMARY OF THE INVENTION The present invention has been made to solve the problems of the related art as described above, and it is an object of the present invention to provide an instant messenger service in which a user directly selects an IoT device and inputs a control message through a character input, It is intended to provide a way to easily control the IoT device through the graphical interface by automatically recommending the IoT device and changing the virtual keyboard according to the situation.

The present invention further provides an IoT device control method for allowing a specific IoT device to be shared with a specific party for a predetermined period by using an instant messenger service or allowing a plurality of users such as a family member to commonly use a specific IoT device.

According to an aspect of the present invention, there is provided a method for controlling an Internet of Things (IoT) device using a virtual keyboard, the method comprising: Displaying a dialog window; Analyzing conversation contents inputted in the dialog window and selecting one or more IoT devices that need to be controlled according to the analysis result; Displaying a virtual keyboard including a list of icons corresponding to the selected one or more IoT devices; And displaying the remote control image for controlling the corresponding IoT device when the user selects a desired icon on the virtual keyboard.

The IoT device control method includes: performing control of a corresponding IoT device according to a user's input through the remote control image; And inputting information on the control of the performed IoT device into the dialog of the dialog window.

The control method for the IoT device may include receiving a control message for giving a control right of the IoT device corresponding to the selected icon to the conversation window, Waiting to be received from a user; And controlling the IoT device according to a control message received from the counterpart user.

The control phrase granting the control authority may include information on at least one of a controllable range and a controllable time for the corresponding IoT device.

The step of displaying the remote controller image may include receiving information on a current state from the corresponding IoT device; Generating a remote control image including the received status information and a graphical interface for controlling the corresponding IoT device; And displaying the remote control image on the virtual keyboard.

The other user may be a registered virtual chat buddy for controlling the IoT device.

The IoT device control method includes: searching IoT devices controllable through the mobile device when opening a dialog for chat with a virtual chat buddy; And displaying a virtual keyboard including a list of icons corresponding to the retrieved IoT devices.

According to another embodiment of the present invention, a method for a mobile device to control an Internet of Things (IoT) device using a virtual keyboard includes the steps of: displaying a dialog for a user of the mobile device to communicate with an opposite user; Receiving a control statement for granting a control right of a predetermined IoT device from the counterpart user via the dialog window; When the control phrase is received, displaying a virtual keyboard including a remote control image for controlling the IoT device; Generating a control message for the IoT device according to the input of the user via the remote control image; And transmitting the generated IoT device control message to the counterpart user through the dialog window.

The control phrase giving the control authority includes information on the controllable range for the IoT device, and the remote control image can be generated reflecting the controllable range.

The control phrase granting the control authority includes information on the controllable time for the IoT device, and the remote control image may be deactivated when the controllable time elapses.

According to the present invention, since the IoT device requiring control according to the dialog context in the dialog window is automatically recommended, it is possible to perform control on the IoT device while conversing, analyze specific words or commands input in the dialog window, IoT devices that need to be controlled are grouped so that they can be conveniently used.

In addition, according to the present invention, an IoT device can be easily selected through a graphic interface of a virtual keyboard connected to a dialog, and a control command can be issued, so that the IoT device can be controlled in an intuitive manner. It is possible to perform efficient control even when the input is impossible or the text input does not allow accurate control.

In addition, according to the present invention, it is possible to share a specific IoT device among a recommended IoT device with a friend or share it with a specific large number of family members and use it together, and to easily control sharing and sharing time of the IoT device.

1 is a diagram for explaining a network configuration of an IoT device control system according to an embodiment of the present invention.
2 is a flowchart illustrating a method of controlling an IoT device using a virtual keyboard according to an embodiment of the present invention.
FIG. 3 illustrates a screen in which a virtual keyboard for controlling the IoT device is generated according to an embodiment of the present invention.
4 is a flowchart illustrating a process of controlling an IoT device using a virtual keyboard according to an embodiment of the present invention.
FIG. 5 illustrates a remote control image of a virtual keyboard according to an embodiment of the present invention.
FIG. 6 illustrates a screen in which a dialog window provides IoT control information using a virtual keyboard according to an embodiment of the present invention.
FIGS. 7 to 9 are exemplary screens for explaining a process of granting a control right of an IoT device to a counterpart user according to an embodiment of the present invention.
FIG. 10 illustrates a screen for selecting and controlling an intra-home IoT device using a virtual keyboard according to an embodiment of the present invention.
11 to 13 illustrate a screen for selecting and controlling IoT devices in a vehicle using a virtual keyboard according to an embodiment of the present invention.
14 is a diagram for explaining an example of recommending IoT devices requiring control by analyzing conversation contents according to an embodiment of the present invention.
15 to 17 illustrate a screen for selecting and controlling an intra-home IoT device using a virtual keyboard according to an embodiment of the present invention.

The terms used in this specification will be briefly described, and the present invention will be described in detail.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. Also, in certain cases, there may be a term selected arbitrarily by the applicant, in which case the meaning thereof will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term, not on the name of a simple term, but on the entire contents of the present invention.

When an element is referred to as "including" an element throughout the specification, it is to be understood that the element may include other elements as well, without departing from the spirit or scope of the present invention. The term " means ", "part "," module ", etc. in the specification means units for processing at least one function or operation, Lt; / RTI >

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

1 is a diagram for explaining a network configuration of an IoT device control system according to an embodiment of the present invention.

1, the IoT device control system includes at least one IoT device 12, a user terminal 10 and a user of the user terminal 10 who can transmit and receive data using a wired / wireless network such as the Internet And includes the terminal 11 of the recipient user.

The IoT device 12 includes a smart grid system for managing energy such as electricity, gas, and water supply, a home network system for integrated management of home cooling and heating, security, and home appliances, a door lock and remote control device for a car, A car system that can control the IoT technology, and the like.

The user terminal 10 and the counterpart user terminal 11 are mobile devices such as smart phones, tablets and wearable computers that can control the IoT device 12 by a wireless communication method. But the present invention is not limited thereto, and the IoT device control method of the present invention may be applied to a PC (Personal Computer).

The user terminal 10 and the counterpart user terminal 11 are terminals capable of using an instant messenger service capable of exchanging conversations with other users and installing and executing a dedicated application for controlling the IoT device . The user terminal 10 and the counterpart user terminal 11 can perform a service-wide operation such as configuration of a service screen, data input, data transmission / reception, and data storage under the control of a dedicated application.

The user terminal 10 is a user terminal having control authority of the IoT device 12, and the user registers IoT devices to be controlled in a dialog window or a setting window of the instant messenger service.

The IoT device 12 is controlled by a wireless communication method and performs a corresponding function when receiving a control signal from the user terminal 10. [

1, an IoT control platform may be included in the IoT device control system according to an embodiment. In this case, when the user terminal 10 transmits a control signal for controlling a specific IoT device to the IoT control platform , The IoT control platform will send a control signal to the corresponding IoT device 12 to perform control. Data transmission / reception between the user terminal 10 and the IoT device 12 in addition to the control signal will also be performed through the IoT control platform.

The user executes a dedicated application in the user terminal 10, opens a dialog window to communicate with the other user, and the dedicated application analyzes the conversation contents of the two users and selects which IoT device to control according to the analysis result. The selected IoT device includes at least a part of all IoT devices that the user terminal 10 can control, and a list of icons representing the selected one or more IoT devices is displayed on the screen.

A virtual keyboard is a virtual input device for inputting characters in a smart device without a keyboard or keyboard, and a virtual keyboard connected to input characters in a dialogue when using an instant message service.

The icon list is displayed through a virtual keyboard for inputting a dialog in a dialogue box, and the icon list can be displayed in the form of being added to the virtual keyboard or overlaid on the virtual keyboard while the virtual keyboard is displayed.

Meanwhile, the virtual keyboard may include a user interface such as a button for listing controllable IoT devices. When the user selects the corresponding button, a list of icons of all controllable IoT devices will be displayed on the virtual keyboard. This is to allow the user to manually invoke the desired IoT device control.

When the user selects a specific icon from the icon list of the virtual keyboard, a remote controller image for controlling the corresponding IoT device, i.e., a virtual remote controller is displayed as a virtual keyboard. When the user manipulates the remote control image, a corresponding control signal is generated and transmitted to the IoT device. Information on which control has been performed is automatically input into the dialog in the dialog window so that the user and the opponent user can check the control contents.

The control of the IoT device 12 using the virtual keyboard will be described in more detail below with reference to FIG. 2 to FIG.

2 is a flowchart illustrating a method of controlling an IoT device using a virtual keyboard according to an embodiment of the present invention.

Referring to FIG. 2, when a user executes a dedicated application and opens a dialog window, a dialog window for communicating with the opposite user is displayed on the screen (S42). The opening of the dialog for the start of the conversation can also be done by the other user.

Here, 'user' is a person having control authority of the IoT device, and the corresponding IoT device will operate according to a control signal outputted from the user terminal 10.

The 'other user' does not have the control authority of the IoT device as the user of the user but can induce the control of the IoT device by talking with the user.

The control unit 10 analyzes the conversation contents of the user and the user of the other party, that is, the sentence input by the user and / or the sentence entered by the user of the other party, to determine whether there is content related to the control of the IoT device. Lt; RTI ID = 0.0 > IoT < / RTI > The analysis of the conversation contents is for automatically determining which of the IoT devices the user or the user wants to control.

Next, a virtual keyboard including the icon list corresponding to the one or more IoT devices selected in the above is displayed (S22). It is to recommend the appropriate IoT device (s) according to the context of the conversation.

FIG. 3 illustrates a screen in which a virtual keyboard for controlling the IoT device is generated according to an embodiment of the present invention.

Referring to FIG. 3, a dialog window for conversation with the conversation partner 'Kim OO' is opened, and the text of the sentence is analyzed when 'Kim OO' inputs a sentence '35? Various methods for analyzing the context can be used for analysis of the conversation contents. A list of icons representing the selected IoT devices is displayed at the bottom of the virtual keyboard 36 according to the results of the analysis.

3, a home icon 31 representing IoT devices on the home network, an icon 32 representing a light control device, an icon 33 representing a door lock as well as an icon 30 corresponding to a heating device, Are also displayed. If users are talking about whether or not the boiler has been turned off, the user will also want to check the status of the lamp or door lock.

As another example, if the user inputs "I am leaving now" in the chat room, the dedicated application analyzes the conversation contents, recognizes the context, selects IoT devices that need to be controlled at work, displays a corresponding icon list will be.

IoT devices that need to be controlled at work may be set in advance, but the same devices may be selected after storing information on IoT devices that the user has previously controlled at the time of work.

Referring again to FIG. 2, when the user selects a desired icon on the virtual keyboard, a remote control image for controlling the corresponding IoT device is displayed (S23). In order to display the remote control image, first, the information about the current state or the last used information is received from the corresponding IoT device, and a remote control image including the received state information and a graphic interface for controlling the corresponding IoT device is generated And display the generated remote control image on the virtual keyboard.

In the example of Fig. 3, when the user touches and selects the heating device icon 30, the remote control image for the control operation of the heating device will be displayed. When the user selects the home icon 31, the IoT device (s) belonging to the home network will be listed, and if one is selected from this list, the remote control image for the control operation of the corresponding IoT device will be displayed. When the user selects the light control device icon 32, a remote control image for light control of each room will be displayed. Also, when the user selects the door lock icon 33, a remote control image capable of controlling opening and closing of the door lock will be displayed.

At this time, detailed information of the selected icon (current status information, past last information or previously set information) is displayed on the dialog window displayed together with the virtual keyboard, and detailed information is deleted using the remote controller image of the virtual keyboard Or modify the control command.

The remote control image for controlling the IoT device is a virtual remote controller that forms an actual control device, and will be generated and displayed differently according to the type of the device. The remote control image may be displayed in place of the keyboard for inputting characters of the virtual keyboard 36 or may be generated and displayed as a virtual keyboard separate from the existing virtual keyboard 36. [

4 is a flowchart illustrating a process of controlling an IoT device using a virtual keyboard according to an embodiment of the present invention.

Referring to FIG. 4, a user can input an input for controlling a desired device through a remote controller image of a virtual keyboard (S40). The user terminal controls the IoT device according to the user's input received through the remote controller image (S41). To this end, the user's input is converted into a control signal for the corresponding IoT device and transmitted to the IoT device, and the IoT device will operate according to the received control signal.

FIG. 5 illustrates a remote control image of a virtual keyboard according to an embodiment of the present invention. In the example of FIG. 3, when the user having the control right selects the heating device icon 30 from his / her user terminal, the remote control image 50 as shown in FIG. 5 may be displayed.

The remote control image may include one or more user interfaces for control and status information of the corresponding IoT device. A user interface button for control, a switch, a jog shuttle, and the like.

The user can generate a control command by touching or dragging a predetermined portion of the remote control image.

At this time, when a touch or a drag is applied, a vibration and a sound effect can be given to check the operation state of the button on the remote controller image with visual, auditory, or tactile sense, thereby preventing erroneous control commands due to malfunction of the buttons existing on the image.

5, the remote controller image 50 corresponding to the heating device includes status information indicating the current state of the heating operation, current temperature, desired temperature, etc., temperature control buttons 51 and 52, and power button 53 As shown in FIG.

When the user touches and selects the power button 53, a control signal to turn off the heating device is generated and transmitted to the corresponding IoT device. The IoT device can perform a control operation according to the control signal and return the result of the operation.

Referring again to FIG. 4, information on control of the IoT device performed through the remote controller image is input into the dialog window dialog (S42).

Instead of typing control commands or dialogs through the keyboard, users can easily turn on the lights in the home, adjust the temperature, add destinations to the vehicle's navigation, etc. via the remote control image, The control contents are displayed in the dialogue contents. Since the content of the control is converted into text and entered as a dialog, the other user can also see which device is controlled.

In addition, since the control and execution histories are recorded in text, they can be browsed later or stored and utilized as device control information.

FIG. 6 illustrates a screen in which a dialog window provides IoT control information using a virtual keyboard according to an embodiment of the present invention.

In the remote controller image shown in FIG. 5, when the power button 53 is touched to turn off the heating device, the corresponding information 60 will be input into the dialog window in dialog with the other user as shown in FIG. The user who has cut off the power of the heating device automatically inputs the information into the conversation even if the user does not directly input the information into the chat room.

As described above, in the IoT device control method according to the present invention, a user interface (IoT device selection button, IoT device control remote controller, etc.) for controlling the IoT device is coupled to the virtual keyboard displayed for dialog input in the dialog window. You can view and control the current status of all IoT devices, control histories, etc. from the virtual keyboard.

In addition, by analyzing the dialog between the users, it is possible to estimate the IoT device requiring control, and to show the icon of the corresponding IoT device to the user having the control right, thereby inducing device control, and the touch, drag and / It is easy to control the IoT device by pinging. When the use of the hand is not free, it is possible to input a control command by voice. In this case, the input speech is recognized and replaced with text, and then a control signal can be generated using it and a conversation to be input to the dialog window can be generated.

FIGS. 7 to 9 are exemplary screens for explaining a process of granting a control right of an IoT device to a counterpart user according to an embodiment of the present invention.

FIG. 7 shows a mobile device of a child who is a user of the other user. In the dialog box with the user 'Dad', the other user 'son' &Quot;, " I will watch TV ".

The IoT device control application running on the terminal of the user 'Pa' analyzes the context of the input dialog, determines that it is necessary to control the IoT device called TV, and sets the TV control icon 74 as virtual It will display on the keyboard.

When the user selects the TV control icon 74, the TV remote control image will be displayed on the virtual keyboard, and a control signal will be generated and transmitted to the TV according to user manipulation through the TV remote control image. However, in the example of FIG. 8, the user 'Dad' gives the control authority to the opponent user 'son' instead of directly controlling the TV.

As shown in FIG. 8, when the user clicks' OK. Quot; one hour only " 72, the dedicated application can grasp the context, grant control authority to the other user, and further determine that the other user is allowed to control the TV for one hour.

At this time, it is recognized that the dialog input by the user includes a control phrase for giving the control authority of the corresponding IoT device to the counterpart, and the application of the user terminal waits for the control message from the counterpart user via the dialog window.

The control phrase granting the control authority may include information on the controllable time for the corresponding IoT device, and may also include information on the controllable range.

The user terminal of the other party uses the dialog window to say 'Okay. Quot; one hour only ", it can be judged that the control phrase for granting authority is received through the context analysis. When the remote control user touches and selects the received control phrase, a remote controller image for controlling the corresponding IoT device is displayed on the virtual keyboard connected to the dialog window of the remote user terminal.

If the control phrase includes information on the controllable range, a remote control image capable of operating only the controllable functions will be generated by reflecting the information. If the control phrase includes information on the controllable time, the device controllable time through the remote control image is limited.

As shown in FIG. 9, the dedicated application of the counterpart user terminal analyzes the conversation of the user entered in the chat room, recognizes that the TV control authority is temporarily granted to the counterpart user for one hour, and displays the remote control image 90 ) On the virtual keyboard.

The counterpart user 'Son' can perform control operations such as turning on the power of the TV and selecting a desired channel through the virtual keyboard, and a control message corresponding to the control operation performed at this time is generated and automatically inputted into the dialog window. The user terminal 'Dad' performs control according to the received control message. That is, the user terminal converts the control message received through the dialog window into a control signal, and transmits the control message to the corresponding IoT device, thereby performing the control requested by the counterpart user. Also, the user 'dad' can know the control contents by the user of the other party through the dialog window.

In this embodiment, since there is a time limit for authorization, when the time limit of 1 hour has elapsed, the power of the corresponding IoT device is automatically shut off, the TV control remote controller 90 is erased from the terminal of the opposite user 'son' The virtual keyboard will reappear.

8, instead of inputting a dialog for authorization, an authorization icon may be given to the user terminal so that a remote control image capable of controlling the corresponding IoT device is displayed on the terminal of the other user .

For example, it is possible to provide a TV-shaped TV control authorization icon on the virtual keyboard displayed on the user terminal of FIG. 8, to control the remote control image for TV control to be displayed on the terminal of the other user instead of the user terminal, It is possible to control the TV remote controller to be displayed on both the user terminal and the terminal of the other user. In this way, the user can take the TV remote control out of the dialog window and give it to the other party so that the control right can be handed down in a more intuitive way.

In this case, a control phrase for granting control authority is generated and input to the dialog window, and the application of the user terminal waits for the control message to be received from the user of the opposite user through the dialog window.

The user of the other party can control the TV using the TV control remote controller while the corresponding window is open. If there is a time limit, the TV can be controlled within a limited time even if the dialog window is opened. The TV control remote controller image 90 assigned to the terminal will be deactivated or erased.

According to another embodiment, a user interface can be provided so that an IoT control command can be issued through an artificial intelligence (AI) user who is not an actual user, that is, a chat window with a virtual chat buddy.

In this embodiment, when an AI user for controlling an IoT device is registered as a buddy (chat buddy) and starts a conversation with an AI user, the conversation contents are analyzed, an IoT device to be controlled according to a context is recommended, It is possible to support control of the IoT device by displaying the corresponding remote control image on the virtual keyboard.

When opening a dialog for chatting with a virtual chat buddy registered for control of the IoT device, it is possible to search IoT devices controllable through the user terminal and display a virtual keyboard including an icon list corresponding to the retrieved IoT devices have. When the user selects the desired IoT device from this list, the corresponding remote-controller image is displayed on the virtual keyboard, thereby allowing the device to be controlled.

FIG. 10 illustrates a screen for selecting and controlling an intra-home IoT device using a virtual keyboard according to an embodiment of the present invention.

Referring to FIG. 10, IoT devices that are recommended according to an analysis of conversation contents in a virtual keyboard 101 associated with a chat window of an instant messenger service can be identified 100. If a user selects one of them, The remote control image 106 of FIG.

A plurality of registered IoT devices can be classified according to the category 104. 10 is a screen when the user selects the home icon 105. Detailed information about the IoT devices connected to the home network is displayed in the dialog window 103. When the audio 107 is selected, An image 106 is displayed.

The user can input a control command by touching or dragging from the remote control image 106 as if the user operated the real controller, the remote controller, or the switch in off-line. The input control command is converted into a control signal by the application and transmitted to the corresponding IoT device.

When the user succeeds in powering off the audio device through the remote control image 106, the control history 108 is automatically input in the dialog window, so that the user and the user of the other party can confirm this.

11 to 13 illustrate a screen for selecting and controlling IoT devices in a vehicle using a virtual keyboard according to an embodiment of the present invention.

Referring to FIG. 11, when the vehicle icon 111 is selected in the virtual keyboard 110 in which icons of the registered IoT devices are displayed, a remote control image 112 for controlling the vehicle navigation apparatus is displayed. When the user touches the waypoint in the remote control image 112, the corresponding control signal is generated and transmitted to the vehicle navigation apparatus, and information on the control signal is displayed in the dialog 113 of the dialog window.

FIG. 12 shows an example in which a waypoint is selected by a text input 121 in a virtual keyboard, and FIG. 13 shows an example of inputting waypoint information through a remote controller image 112 including a map image.

14 is a diagram for explaining an example of recommending IoT devices requiring control by analyzing conversation contents according to an embodiment of the present invention.

Referring to FIG. 14, the virtual keyboard connected to the dialog window 144 includes an IoT icon 140, and if it is selected, the list of registered IoT devices and the current status of each IoT device can be confirmed.

Also, if the user inputs a conversation 141 of " leaving work " to the virtual keyboard, it refers to at least one of the contents of the input character, current time information, current position information and previous IoT device control history, Selects IoT devices suitable for the current situation among the IoT devices and displays corresponding icons on the virtual keyboard 142. [

According to an embodiment of the present invention, the grouping of IoT devices is not performed manually but the input sentences are analyzed to select and sort the related IoT devices, and then the current status, the previous history, Etc.). In the example of FIG. 14, information about the selected IoT devices is displayed in the dialog window area 144.

In addition, the details controlled by the user through the virtual keyboard are converted into conversations and automatically input into the dialog window 144. [ The details of the control may include what the IoT device selected by the user is, and what control is indicated for this IoT device.

Specifically, when the user selects the vehicle icon 143 from the virtual keyboard 142, the selection information is automatically input into the dialog 146, and the corresponding remote-controller image 148 is displayed as a virtual keyboard. When a specific position 147 is selected as the stop position on the remote control image 148, a corresponding control signal is generated and transmitted to the vehicle navigation apparatus, and the control details are input to the dialog 149.

Referring to FIG. 14, two routes are displayed in the original travel route. When the route 147 is selected by the user's touch, an appropriate one of the two routes is recommended. Thereafter, the IoT devices associated with " off work " can be preset to operate sequentially in time zones. For example, it is possible to set the temperature controller to be operated so that the room temperature becomes the predetermined value at the arrival time, to prepare for unlocking the door lock, and then to control the ON / OFF time of the TV apparatus. That is, it can be implemented not only by analyzing the conversation contents to recommend the associated IoT devices, but also by the application itself to judge the progress schedule of the IoT devices in conversation form and notify the user.

The user can add and / or delete the IoT device to be controlled in the interface as shown in FIG. 14 or modify the reserved control operation. Specifically, control such as changing the desired indoor temperature or changing the reserved TV channel by touching, dragging and / or swapping can be performed on the remote control image, similar to changing the traveling route by adding the waypoint in the previous example have.

15 to 17 illustrate a screen for selecting and controlling an intra-home IoT device using a virtual keyboard according to an embodiment of the present invention.

15, when the home icon 151 of the virtual keyboard is selected, a list of the IoT devices connected to the home network and the current status thereof are displayed in the dialog window area 150, and the IoT device icon list and the control of the selected IoT device 153 A virtual keyboard 152 including a remote control image for the remote controller appears instead.

When the virtual keyboard 152 touches and selects each room, the A / V device existing at the corresponding location can be controlled, and the control details will be displayed in the dialogue 154.

Fig. 16 shows an example of controlling the light devices in each room in the home. By touching each room on the virtual keyboard, you can reserve the time to turn on / off the light at that location.

17, when the home icon 151 is selected in the virtual keyboard, the selection details are automatically input into the conversation 171, and the virtual keyboard 170 including the icon list of IoT devices belonging to the home category and the remote control area, Was displayed. The remote control image corresponding to the IoT device selected in the virtual keyboard 170 will be displayed in the remote control area and the user can control the desired IoT device through remote control operation. The details of the control will also be automatically entered into the conversation dialog.

The method according to an embodiment of the present invention can be implemented in the form of a program command which can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

10: User terminal
11: terminal of the other user
20: IoT (Internet of Things) device

Claims (12)

A method for a mobile device to control an Internet of Things (IoT) device using a virtual keyboard,
Displaying a dialog for the user of the mobile device and the user of the other party to communicate;
Analyzing conversation contents inputted in the dialog window and selecting one or more IoT devices that need to be controlled according to the analysis result;
Displaying a virtual keyboard including a list of icons corresponding to the selected one or more IoT devices;
Displaying a remote control image for controlling the IoT device corresponding to an icon selected on the virtual keyboard,
Wherein the selection of the IoT device is performed by a word set as an association word in the IoT device among the conversation contents,
The remote control image has an operation image for each IoT device for easy operation by a user,
Wherein information about control of the IoT device is output as a dialog of a dialog corresponding to user control through the remote control image. delete The method according to claim 1,
Waiting for the control message for the IoT device to be received from the counterpart user via the dialog window when a control phrase for giving control authority of the IoT device corresponding to the selected icon to the counterpart user is input into the dialog of the dialog window, Wow;
And controlling the IoT device according to a control message received from the counterpart user. The method of claim 3,
Wherein the control phrase giving the control right includes information on at least one of a controllable range and a controllable time for the corresponding IoT device. The method according to claim 1,
Wherein the step of displaying the remote-
Receiving information on a current state from the corresponding IoT device;
Generating a remote control image including the received status information and a graphical interface for controlling the corresponding IoT device;
And displaying the remote control image on the virtual keyboard. The method according to claim 1,
Wherein the counterpart user is a virtual chat buddy registered to control the IoT device. The method according to claim 6,
Retrieving controllable IoT devices upon outputting a dialog for chat with the virtual chat buddies;
Further comprising the step of displaying a virtual keyboard including a list of icons corresponding to the searched IoT devices. A computer program recorded on a recording medium for performing an IoT device control method using the virtual keyboard according to any one of claims 1 to 7. A method for a mobile device to control an Internet of Things (IoT) device using a virtual keyboard,
Displaying a dialog for the user of the mobile device to communicate with the other user;
Receiving a control statement for granting a control right of a predetermined IoT device from the counterpart user via the dialog window;
When the control phrase is received, displaying a virtual keyboard including a remote control image for controlling the IoT device;
Generating a control message for the IoT device according to the input of the user via the remote control image;
And transmitting the generated IoT device control message to the counterpart user through the dialog window,
The IoT device is selected from the conversation contents by a word set as an association word in the IoT device,
The remote control image has an operation image for each IoT device for easy operation by a user,
Wherein information about control of the IoT device is output as a dialog of a dialog corresponding to user control through the remote control image. 10. The method of claim 9,
Wherein the control phrase for granting the control right includes information on a controllable range for the IoT device, and the remote control image is generated reflecting the controllable range. 10. The method of claim 9,
Wherein the control statement for granting the control authority includes information on a controllable time for the IoT device, and the remote control image is inactivated when the controllable time elapses. A computer program recorded on a recording medium for performing an IoT device control method using the virtual keyboard according to any one of claims 9 to 11.

Images