KR102514435B1 - An electric device and a server for providing service related to an IoT(Internet of things) device - Google Patents

Abstract

An electronic device providing a service related to an Internet of Things (IoT) device is disclosed. The electronic device determines an IoT device list based on the display unit and the location of the electronic device, controls the display unit to display a marker in an area corresponding to the location of the IoT device included in the IoT device list, and inputs selecting the marker. Based on, it may include a control unit that provides a service result corresponding to the service type of the IoT device.

Description

An electric device and a server for providing service related to an IoT (Internet of things) device}

The disclosed embodiments relate to an electronic device and server that provide services related to Internet of Things (IoT) devices.

As IoT technology develops, the need to control IoT devices providing various services is increasing. Services provided by IoT devices may be different types of services. Accordingly, not only are the types of data obtained from IoT devices different, but also applications for each service are required to use services provided by IoT devices.

In addition, in existing IoT technologies, protocols such as CoAP, MQTT, and XMPP can receive services from IoT devices when there is prior information including IP address, identification information (ID), IoT device name, and key. could Therefore, in the past, the amount of prior information required to use IoT technology was enormous. In addition, when there is no prior information about the IoT device, there is a problem in that it is difficult to receive a service from the IoT device.

The disclosed embodiments are intended to provide various services by Internet of Things (IoT) devices even when service types provided by IoT devices are different.

The disclosed embodiments are intended to provide a service related to an IoT device located around an electronic device even when there is no prior information about the IoT device.

The disclosed embodiments are intended to unify control of IoT devices and information processing for IoT devices in electronic devices.

An electronic device providing a service related to an Internet of Things (IoT) device according to an embodiment determines an IoT device list based on the locations of the display unit and the electronic device, and corresponds to the location of the IoT device included in the IoT device list. and a control unit controlling the display unit to display a marker in the selected area and providing a service result corresponding to the service type of the IoT device based on an input for selecting the marker.

The controller according to an embodiment may transmit the location of the electronic device to the server, and may receive the IoT device list determined based on IoT device information stored in the server and the location of the electronic device from the server.

IoT device information according to an embodiment may include a location of the IoT device and a service type of the IoT device.

The controller according to an embodiment may control the display unit to display information generated or provided by the IoT device when the service type includes an information production service or an information consumption service.

If the service type includes a control service, the controller according to an embodiment may control the display unit to display at least one of information on a current state of the IoT device and a button for receiving a request for the control service.

The control unit according to an embodiment transmits identification information of the IoT device to the server, receives a service result of the IoT device from the server, and the service result is information generated by the IoT device, information provided by the IoT device, and IoT device. At least one of status information may be included.

The IoT device list according to an embodiment includes at least one of a location of an IoT device located near an electronic device, identification information of the IoT device, a type of service of the IoT device, information used by the IoT device, and state information of the IoT device. can include

A server providing a service related to an Internet of Things (IoT) device according to an embodiment includes a memory for storing IoT device information including a location of the IoT device and a service type of the IoT device, and a location of the electronic device from the electronic device. In response to the reception, the IoT device list determined based on the location of the electronic device and the IoT device information is transmitted to the electronic device, and in response to receiving the identification information of the IoT device from the electronic device, it responds to the service type of the IoT device. It may include a control unit for transmitting the service result to the electronic device.

The controller according to an embodiment may request a service from the IoT device based on identification information of the IoT device and update IoT device information based on a response to the service request received from the IoT device.

The control unit according to an embodiment transmits a service result of the IoT device to the electronic device based on the updated IoT device information, and the service result is information generated by the IoT device, information provided by the IoT device, and state information of the IoT device. may contain at least one.

The controller according to an embodiment may generate identification information of the IoT device and register IoT device information including a location of the IoT device received from the IoT device, a service type of the IoT device, and identification information.

A method of providing a service related to an Internet of Things (IoT) device, comprising determining an IoT device list based on a location of an electronic device, displaying a marker in an area corresponding to the location of an IoT device included in the IoT device list. The method may include controlling a display unit and providing a service result corresponding to a service type of an IoT device based on an input for selecting a marker.

A method for providing a service related to an Internet of Things (IoT) device in a server, comprising: storing IoT device information including a location of an IoT device and a service type of the IoT device; receiving the location of an electronic device from an electronic device In response to the step of transmitting the IoT device list determined based on the location of the electronic device and the IoT device information to the electronic device, and in response to receiving the identification information of the IoT device from the electronic device, corresponding to the service type of the IoT device. and transmitting the service result to the electronic device.

According to the disclosed embodiments, various services can be provided by Internet of Things (IoT) devices even when service types provided by IoT devices are different.

In addition, according to the disclosed embodiments, even when there is no prior information about the IoT device, a service related to an IoT device located around an electronic device can be provided.

In addition, according to the disclosed embodiments, it is possible to unify the control of the IoT device and the processing of information about the IoT device in the electronic device.

1 is a block diagram of an electronic device 100 according to one embodiment.
2 is a block diagram of a server 200 that is operable in connection with the electronic device 100 according to an embodiment.
3 is a diagram illustrating a system for providing a service related to an Internet of Things (IoT) device according to an embodiment.
4 is another diagram illustrating a system for providing a service related to an Internet of Things (IoT) device according to an embodiment.
5 is a flowchart of a system for providing a service related to an Internet of Things (IoT) device according to an embodiment.
6 is a flowchart of a method performed in the electronic device 100 according to one embodiment.
7 is another flowchart of a method performed in electronic device 100 according to an embodiment.
8 is a flowchart of a method performed in server 200 according to one embodiment.
9 is another flowchart of a method performed in server 200 according to an embodiment.
10A illustrates displaying a marker in an area corresponding to a location of an IoT device according to an embodiment.
10B illustrates providing a service related to an Internet of Things (IoT) device according to an embodiment.
11 is a detailed configuration diagram of an electronic device 100a according to an embodiment.

Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the contents described in the accompanying drawings. In addition, a method of configuring and using an electronic device according to an embodiment of the present invention will be described in detail with reference to the contents described in the accompanying drawings. Like reference numerals or numerals in each drawing indicate parts or components that perform substantially the same function.

Terms including ordinal numbers, such as first and second, may be used to describe various elements, but elements are not limited by the terms. Terms are only used to distinguish one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include a combination of a plurality of related items or any one of a plurality of related items.

Terms used in this specification are used to describe embodiments, and are not intended to limit and/or limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms include or have are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features, numbers, or steps However, it should be understood that it does not preclude the possibility of existence or addition of operations, components, parts, or combinations thereof.

1 is a block diagram of an electronic device 100 according to one embodiment.

The electronic device 100 is a device capable of providing a service related to an Internet of Things (IoT) device. The electronic device 100 may include, for example, a smartphone, a head mound display (HMD), a head up display (HUD), a wearable device, a tablet PC, a PC, a smart TV, a laptop, a global positioning system (GPS) device, and a digital device. It may be, but is not limited to, a broadcasting terminal, navigation, kiosk, digital camera, and other mobile or non-mobile computing device. In addition, the electronic device 100 may include a device equipped with an optical system capable of generating an image by photographing a subject, including a lens and an imaging device. In addition, the electronic device 100 includes a VR device providing a virtual reality (VR) image having a communication function and a data processing function, an AR device providing an augmented reality (AR) image, a watch, glasses, a hair band, and a ring. can be

Referring to FIG. 1 , the electronic device 100 includes a control unit 110 and a display unit 120.

The controller 110 may determine the IoT device list based on the location of the electronic device 100 .

The location of the electronic device 100 may be determined based on a gyroscope sensor (not shown) and a location sensor (eg, GPS) (not shown) included in the electronic device 100 . Also, the location of the electronic device 100 may include altitude, latitude, and longitude. Also, the location of the electronic device 100 may include the orientation of the electronic device 100 . That is, the location of the electronic device 100 may include direction information toward which the electronic device 100 is heading.

The IoT device list may be a list of IoT devices located in the vicinity of the electronic device 100 . The vicinity of the electronic device 100 may refer to an area currently capable of being photographed by the electronic device 100 . The IoT device list may include at least one of the location of the IoT device, identification information of the IoT device, service type of the IoT device, information used by the IoT device, and state information of the IoT device.

The electronic device 100 may determine an IoT device list based on IoT device information. Also, the electronic device 100 may determine the IoT device list based on the location of the electronic device 100 and the location of the IoT device included in the IoT device information.

The IoT device information may include the location of the IoT device and the type of service of the IoT device. Also, the IoT device information may be information based on an ontology schema. As an ontology schema, SSN (Semantic Sensor Network) ontology, OWL-S Service Profile, and OWL-S Service Grounding ontology, which are ontology models based on service modeling for the Internet of Things announced by the IoT-A Project, can be used. IoT device information of an IoT device may correspond to an instance of an ontology. An instance of Ontology can be created and updated through a server.

The electronic device 100 may determine the IoT device list based on IoT device information stored in a memory (not shown) of the electronic device 100 .

Also, the electronic device 100 may receive an IoT device list from a server (not shown). Specifically, the electronic device 100 may transmit the location of the electronic device 100 to the server. In addition, the electronic device 100 may receive the IoT device list determined based on the IoT device information stored in the server and the location of the electronic device 100 from the server.

The controller 110 may control a marker to be displayed in an area corresponding to the location of an IoT device included in the IoT device list. The marker displayed on the area corresponding to the location of the IoT device is for receiving an input for selecting the IoT device. For example, it may be defined as an input for selecting an IoT device when a marker stays within a predetermined area for a predetermined period of time or more.

The controller 110 may provide a service result corresponding to a service type of an IoT device based on an input for selecting a marker.

The service type may include, for example, an information production service, an information consumption service, and a control service.

The information production service may mean generating information in an IoT device. For example, if the IoT device is a temperature sensor, the service type of the temperature sensor may include an information production service. Information generated through the information production service of the temperature sensor may be ambient temperature.

The information consumption service may refer to acquiring information based on data currently being provided by the IoT device. For example, when the IoT device is a bus stop, the service type of the bus stop may include an information consumption service. Information obtained through an information consumption service at a bus stop may be a bus arrival time.

The control service may refer to controlling the current state of the IoT device based on information about the current state. For example, if the IoT device is an LED, the service type of the LED may include a control service. The current state information obtained from the LED through the control service may be the current on/off state of the LED. In addition, the on/off state can be changed through the control service in the LED.

Also, the service result may be information generated or provided by the IoT device. Also, the service result may be status information of the IoT device.

The controller 110 may directly receive a service result from the IoT device based on IoT device information stored in a memory (not shown) of the electronic device 100 . That is, the controller 110 may directly transmit a service request to the IoT device and receive a response to the service request.

When there is no IoT device information stored in the memory (not shown) of the electronic device 100, the controller 110 may receive a service result from the server using identification information of the IoT device. At this time, the controller 110 may transmit identification information of the IoT device to the server and receive a service result through the server.

The display unit 120 may display a marker in an area corresponding to the location of the IoT device.

Also, the display unit 120 may display a marker representing the location of the IoT device together with the image. An image displayed on the display unit 120 may be an image captured using the electronic device 100 . For example, the display unit 120 may display a still image or a live view image of a moving image being captured using the electronic device 100 . Also, the image displayed on the display unit 120 may be at least one of an image stored in the electronic device 100 and an image received from an external device.

Also, the display unit 120 may display a marker representing the location of the IoT device together with an image of the vicinity of the electronic device 100 captured. An image of the vicinity of the electronic device 100 may be an augmented reality image. However, it is not limited thereto.

2 is a block diagram of a server 200 that is operable in connection with the electronic device 100 according to an embodiment.

Referring to FIG. 2 , the server 200 is a device capable of providing services related to Internet of Things (IoT) devices. The server 200 includes a memory 210 and a controller 220 .

The memory 210 may store IoT device information including a location of the IoT device and a service type of the IoT device.

In addition, the IoT device information may include a location of the IoT device received from the IoT device, a service type of the IoT device, and IoT device identification information.

In response to receiving the location of the electronic device 100 from the electronic device 100, the control unit 220 transmits the IoT device list determined based on the location of the electronic device 100 and the IoT device information to the electronic device 100. can be sent

The controller 220 may transmit a service result corresponding to the service type of the IoT device to the electronic device 100 in response to receiving the identification information of the IoT device from the electronic device 100 .

The controller 220 may request a service from the IoT device based on identification information of the IoT device. The controller 220 may update IoT device information based on a response to a service request received from the IoT device.

The controller 220 may transmit a service result of the IoT device to the electronic device 100 based on the updated IoT device information. The service result may include at least one of information generated by the IoT device, information provided by the IoT device, and state information of the IoT device.

Meanwhile, the control unit 220 may generate identification information of the IoT device. For example, when there is no IoT device information stored in the memory 210, the controller 220 may generate IoT device identification information. That is, when the IoT device is a new device, the control unit 220 may generate identification information of the IoT device. Also, the control unit 220 may generate IoT device information including identification information of the IoT device. The control unit 220 may register the generated IoT device information.

Also, if there is IoT device information stored in the memory 210, the controller 220 may update the IoT device information based on the location of the IoT device received from the IoT device.

3 is a diagram illustrating a system for providing a service related to an Internet of Things (IoT) device according to an embodiment.

Referring to FIG. 3 , a system for providing a service related to an Internet of Things (IoT) device may include an electronic device 310, a server 320, an IoT device 330, and an OPEN API storage unit 340. .

The electronic device 310 may transmit and receive data to and from the server 320 . In addition, the electronic device 310 may include a control unit 312 and a display unit 314. The control unit 312 and the display unit 314 of FIG. 3 may include functions corresponding to the control unit 110 and the display unit 120 of FIG. 1 , respectively. Descriptions of the control unit 312 and the display unit 314 are omitted because they overlap with those described in FIG. 1 .

The server 320 may include a controller 322 and a memory 324 . The controller 322 and the memory 324 of FIG. 3 may include functions corresponding to the controller 220 and the memory 210 of FIG. 2 , respectively. Descriptions of the control unit 322 and the memory 324 are omitted because they overlap with those described in FIG. 2 .

The IoT device 330 may provide a REST service to allow an external device to access the IoT device 330 . In addition, the IoT device 330 may provide IoT device information of the IoT device 330 in RDF format. For example, the IoT device 330 may provide IoT device information in RDF format based on the ontology model provided by the server 320 .

The OPEN API storage unit 340 may transmit data to the IoT device 330.

The OPEN API storage unit 340 may transmit data to the IoT device 330 using a predetermined communication protocol. For example, communication protocols may include CoAP, MQTT, REST, and SOAP. The OPEN API storage unit 340 may determine whether the IoT device 330 accesses data based on a pre-registered API key.

4 is a diagram illustrating a system for providing a service related to an Internet of Things (IoT) device according to another embodiment.

Referring to FIG. 4, a system for providing a service related to an Internet of Things (IoT) device includes an electronic device 410, a server 420, an Internet of Things (IoT) device 430, and an OPEN API storage unit 440. can include

First, the electronic device 410 may include a user interface unit 411, an API management unit 413, a data management unit 415, an augmented reality engine 417, and a display unit 419.

The user interface unit 411 may control a marker to be displayed in an area corresponding to the location of the IoT device 430 . Also, the user interface unit 411 may display an area for selecting a marker. For example, an area for selecting a marker may be displayed in the center of the display unit 419 of the electronic device 410 . Also, the user interface unit 411 may display an area for marker selection in a rectangular shape in the center of the display unit 419 . The input for selecting the marker may be that the marker stays inside the area for selecting the marker for a certain period of time or more.

When the service type of the IoT device 430 includes an information producing service or an information consuming service, the user interface unit 411 may control information generated by the IoT device 430 to be displayed. In addition, when the service type of the IoT device 430 includes a control service, the user interface unit 411 includes at least one of buttons for receiving information about the current state of the IoT device 430 and a request for the control service. can be displayed.

The API management unit 413 may request the server 420 to call the OPEN API. The API management unit 413 may request the server 420 when an OPEN API is required among services provided by the IoT device 430 .

The data management unit 415 may manage data displayed together with the marker of the IoT device 430 . For example, data on the current state of the IoT device 430, data on the current location of the IoT device 430, identification information on the IoT device 430, service type of the IoT device 430, IoT device 430 ), you can manage whether to display the data for the name.

Also, the data management unit 415 may manage the location of the electronic device 410 and data stored in the electronic device 410 .

The augmented reality engine 417 may implement augmented reality based on the location of the electronic device 410 . The augmented reality engine 417 may display a marker corresponding to a location of a nearby IoT device 430 based on the location of the electronic device 410 .

Also, the augmented reality engine 417 may overlap and display an area for selecting a marker on an image. For example, an area for selecting a marker may be displayed in the center of the display unit 419 of the electronic device 410 . The augmented reality engine 417 may determine whether the marker has entered an area for marker selection.

According to one embodiment, the augmented reality engine 417 may include the Mixare open source augmented reality engine. However, it is not limited thereto. Mixare is a GPLv3 license and can be implemented in Android and iOS environments.

Since the display unit 419 corresponds to the display unit 120 described in FIG. 1, the description of the display unit 419 will be omitted.

Meanwhile, when the service type of the IoT device 430 is an information production service, the electronic device 410 according to an embodiment may receive information generated by the IoT device 430 from the server 420 . Also, when the service type of the IoT device 430 is an information consumption service, the electronic device 410 may receive information provided from the IoT device 430 from the server 420 . For example, information received from the server 420 may be provided in an Android Toast form.

When the service type of the IoT device 430 is a control service, the electronic device 410 may display a button corresponding to the control service. When the button is selected, the electronic device 410 may expose a service result as a Toast.

According to an embodiment, when the electronic device 410 operates in an Android environment, a delay may occur in the operation of the electronic device 410 when a REST service request is made to the server 420 . In this case, the electronic device 410 may transmit and receive data with the server 420 in a multi-threaded background through the AsyncTask function. That is, the electronic device 410 may not affect the UI of the main screen by allowing the server 420 to operate in the background.

The server 420 may control the entire system and manage all data.

The server 420 may include a service management unit 421, an IoT management unit 422, an API management unit 423, a semantic information management unit 425, a common management unit 427, and a memory 429.

The service management unit 421 may provide an IoT device list. In addition, the service management unit 421 may provide a service provided by the IoT device when there is a request from the electronic device 410 .

The IoT manager 422 may register IoT device information of the first IoT device 430 . In addition, the IoT manager 422 may manage the service of the IoT device 430 for each service type.

The API management unit 423 may make an API request and reply to the IoT device 430 . According to an embodiment, when the IoT device 430 uses the OPEN API, public information may be linked.

The semantic information management unit 425 may create and update IoT device information. The semantic information management unit 425 may manage IoT device information and information included in the IoT device information. For example, IoT device information may use an ontology. Also, the ontology schema can be created using the Protege program. The IoT device information may include location information of the IoT device, service information, user information, and service type. In addition, the IoT device information includes information generated and consumed by the IoT device and information about the current state of the IoT device. The semantic information management unit 425 may continuously update IoT device information using the ontology.

The common management unit 427 may perform a series of calculations and additional processing.

Since the memory 429 corresponds to the memory 210 of FIG. 2, the description of the memory 429 will be omitted.

Meanwhile, the server 420 according to an embodiment may be implemented using the MVC model of the Spring Framework. Also, the server 420 may use Tomcat Web Server 8.0. However, it is not limited thereto.

The IoT device 430 may include a web service management unit 431 , a semantic information management unit 433 and a hardware management unit 435 .

The web service manager 431 may provide a REST service to allow an external device to access the IoT device 430 .

The semantic information management unit 433 may provide IoT device information of the IoT device 430 in RDF format. For example, the semantic information management unit 433 may provide IoT device information in RDF format based on the ontology model provided by the server 420 . The semantic information management unit 433 may provide IoT device information using a format other than the RDF format.

The hardware management unit 435 may provide a service result or transmit information received from the outside. Also, the hardware management unit 435 may control the IoT device 430 .

The OPEN API storage unit 440 may transmit data to the IoT device 430.

The OPEN API storage unit 440 may include an API management unit 441 and a user authority management unit 443 . The API management unit 441 may transmit data to the IoT device 430 using a predetermined communication protocol. For example, communication protocols may include CoAP, MQTT, REST, and SOAP. The user authority management unit 443 may determine whether the IoT device 430 accesses data based on a pre-registered API key.

5 is a flowchart of a system for providing a service related to an Internet of Things (IoT) device according to an embodiment.

In step S511, the IoT device may transmit the location and service type of the IoT device to the server (S511).

In step S513, the server may update the ontology based on the location and service type of the IoT device (S513). Specifically, the server may update IoT device information including the location and service type of the IoT device.

Then, in step S515, the electronic device may receive an input for executing an application for providing a service related to the IoT device from the user (S515).

In step S517, the electronic device may update the location of the electronic device (S517). The location of the electronic device may include altitude, latitude, and longitude. Also, the location of the electronic device may include the orientation of the electronic device.

In step S519, the electronic device may transmit the location of the electronic device to the server (S519).

In step S521, the server may transmit the IoT device list to the electronic device (S521). For example, the server may transmit an IoT device list determined based on the location of the electronic device and IoT device information to the electronic device in response to receiving the location of the electronic device.

In step S523, the electronic device may display a marker (S523). For example, in response to receiving the IoT device list, the electronic device may display a marker in an area corresponding to a location of an IoT device included in the IoT device list.

In step S525, the electronic device may receive an input for selecting a marker from the user (S525).

In step S527, the electronic device may transmit identification information of the IoT device to the server (S527).

In step S529, the server may check the service type of the IoT device in response to the received identification information of the IoT device (S529).

In step S531, the server may transmit a service request to the IoT device (S531).

In step S533, the server may transmit a response to the service request to the IoT device (S533).

In step S535, the server may call OPEN API (S535). Also, in step S537, the OPEN API storage unit may provide the OPEN API in response to the OPEN API call (S537).

In step S539, the server may update the ontology (S539). Specifically, IoT device information may be updated based on a response to a service request received from the IoT device.

In step S541, the server may transmit a service result to the electronic device (S541). For example, the server may transmit a service result based on a response to a service request received from the IoT device.

6 is a flowchart of a method performed in the electronic device 100 according to one embodiment.

In step S110, the electronic device 100 may transmit the location of the electronic device 100 to the server 200 (S110). Also, the electronic device 100 may transmit the location of the electronic device to the IoT device.

In step S120, the electronic device 100 may receive an IoT device list from the server 200 (S120). The IoT device list may be determined based on the location of the electronic device 100 .

In step S130, the electronic device 100 may display a marker in an area corresponding to the location of the IoT device (S130). The electronic device 100 may display a marker in an area corresponding to the location of an IoT device included in the IoT device list.

In step S140, the electronic device 100 may provide a service result corresponding to the service type of the IoT device (S140). The electronic device 100 may provide a service result corresponding to the service type of the IoT device based on the marker selection input.

7 is another flowchart of a method performed in electronic device 100 according to an embodiment.

Steps S210, S220, and S230 correspond to steps S110, S120, and S130 described in FIG. 6, respectively, and thus descriptions thereof will be omitted.

In step S240, the electronic device 100 may receive an input for selecting a marker (S240).

In step S250, the electronic device 100 may transmit identification information of the IoT device to the server 200 (S250). For example, the electronic device 100 may transmit identification information of an IoT device corresponding to the selected marker to the server 200 based on an input for selecting a marker.

In step S260, the electronic device 100 may receive a service result of the IoT device from the server 200 (S260).

In step S270, the electronic device 100 may provide a service result corresponding to the service type of the IoT device (S270).

8 is a flowchart of a method performed in server 200 according to one embodiment.

In step S310, the server 200 may receive the location of the electronic device 100 (S310).

In step S330, the server 200 may transmit the IoT device list to the electronic device 100 (S320). In response to receiving the location of the electronic device 100 from the electronic device 100, the server 200 provides an IoT device list determined based on the location of the electronic device 100 and the IoT device information to the electronic device 100. can be sent to

In step S330, the server 200 may receive identification information of the IoT device based on the input of the electronic device 100 (S330). The server 200 may receive identification information of the IoT device based on an input for selecting a marker corresponding to the IoT device of the electronic device 100 .

In step S340, the server 200 may transmit the service result of the IoT device (S340). The server 200 may transmit a service result corresponding to the service type of the IoT device to the electronic device 100 in response to receiving the identification information of the IoT device from the electronic device 100 .

9 is another flowchart of a method performed in server 200 according to an embodiment.

In step S410, the server 200 may receive the location of the IoT device and the type of service of the IoT device (S410).

In step S420, the server 200 may generate identification information of the IoT device (S420).

In step S430, the server 200 may update the ontology (S430). The ontology may correspond to IoT device information including generated identification information, a location of the IoT device received from the IoT device, and a service type of the IoT device. The server 200 may store IoT device information including location and service type in order to provide a service suitable for the service type of the IoT device.

Steps S440, S450, and S460 correspond to steps S310, S320, and S330 described in FIG. 8, respectively, and thus descriptions thereof will be omitted.

In step S470, the server 200 may transmit a service request to the IoT device (S470).

In step S480, the server 200 may update the ontology based on the response to the service request of the IoT device (S470). For example, updating the ontology may be updating IoT device information stored in the server 200 .

In step S490, the server 200 may transmit the service result of the IoT device (S480). The server 200 may transmit a service result of the IoT device to the electronic device 100 based on the updated IoT device information.

10A illustrates displaying a marker in an area corresponding to a location of an IoT device according to an embodiment.

Referring to FIG. 10 , the electronic device 100 may display markers 1010, 1020, and 1030 in regions corresponding to locations of IoT devices included in the IoT device list. For example, IoT devices included in the IoT device list may be a temperature sensor, a humidity sensor, and a bus stop. The electronic device 100 may display markers corresponding to all IoT devices included in the IoT device list. Also, the electronic device 100 may display only markers for some of the IoT devices included in the IoT device list. The electronic device 100 may display markers for some of the IoT devices according to a service type selected from among service types.

In addition, the electronic device 100 may provide data 1012 about the IoT device along with the marker. For example, the data 1012 for the IoT device may indicate that the IoT device is a temperature sensor and that the IoT device's distance to the electronic device 100 is 560 m. For example, the electronic device 100 may provide data 1012 about IoT devices based on an IoT device list.

10B illustrates providing a service related to an Internet of Things (IoT) device according to an embodiment.

The electronic device 100 may provide a service result 1044 of the IoT device. Referring to FIG. 10B , an IoT device may be a temperature sensor, and a service type provided by the IoT device may be an information production service.

The service result 1044 may include at least one of information generated by the IoT device, information provided by the IoT device, and state information of the IoT device. Referring to FIG. 10B , the electronic device 100 may provide a service result 1044 including the ambient temperature measured by the temperature sensor, which is information generated by the IoT device.

11 is a block diagram of an electronic device 100a according to one embodiment.

For example, as shown in FIG. 11 , the electronic device 100a according to an embodiment includes a sensing unit ( 1400), an A/V input unit 1600, and a memory 1700 may be further included.

The user input unit 1100 means a means through which a user inputs data for controlling the electronic device 100a. For example, the user input unit 1100 includes a key pad, a dome switch, a touch pad (contact capacitance method, pressure resistive film method, infrared sensing method, surface ultrasonic conduction method, integral type tension measuring method, piezo effect method, etc.), a jog wheel, a jog switch, and the like, but are not limited thereto.

The user input unit 1100 may receive a password from the user. The user input unit 1100 may also receive an input about the type of password to be input from the user.

The output unit 1200 may output an audio signal, a video signal, or a vibration signal, and the output unit 1200 may include a display unit 1210, a sound output unit 1220, and a vibration motor 1230. there is.

The display unit 1210 displays and outputs information processed by the electronic device 100a.

Meanwhile, when the display unit 1210 and the touch pad form a layer structure to form a touch screen, the display unit 1210 may be used as an input device as well as an output device. The display unit 1210 includes a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a 3D display ( 3D display) and at least one of electrophoretic display. Also, depending on the implementation form of the electronic device 100a, the electronic device 100a may include two or more display units 1210. In this case, two or more display units 1210 may be disposed facing each other using a hinge.

The display unit 1210 of FIG. 11 may include the display unit 120 described in FIG. 1 .

The audio output unit 1220 outputs audio data received from the communication unit 1500 or stored in the memory 1700 . Also, the sound output unit 1220 outputs sound signals related to functions performed by the electronic device 100a (eg, a call signal reception sound, a message reception sound, and a notification sound). The sound output unit 1220 may include a speaker, a buzzer, and the like.

The vibration motor 1230 may output a vibration signal. For example, the vibration motor 1230 may output a vibration signal corresponding to an output of audio data or video data (eg, call signal reception sound, message reception sound, etc.). Also, the vibration motor 1230 may output a vibration signal when a touch is input to the touch screen.

The controller 1300 typically controls overall operations of the electronic device 100a. For example, the control unit 1300 executes programs stored in the memory 1700, thereby enabling the user input unit 1100, the output unit 1200, the sensing unit 1400, the communication unit 1500, and the A/V input unit 1600. ) can be controlled overall.

The controller 1300 of FIG. 11 may include the controller 110 described in FIG. 1

The sensing unit 1400 may detect a state of the electronic device 100a or a state around the electronic device 100a and transmit the sensed information to the controller 1300 .

The sensing unit 1400 includes a magnetic sensor 1410, an acceleration sensor 1420, a temperature/humidity sensor 1430, an infrared sensor 1440, a gyroscope sensor 1450, and a position sensor. (eg, GPS) 1460, an air pressure sensor 1470, a proximity sensor 1480, and an RGB sensor (illuminance sensor) 1490, but may include at least one, but is not limited thereto. Since a person skilled in the art can intuitively infer the function of each sensor from its name, a detailed description thereof will be omitted.

The communication unit 1500 may include one or more components that enable communication between the electronic device 100a and an external device (not shown) or between the electronic device 100a and a server (not shown). For example, the communication unit 1500 may include a short range communication unit 1510, a mobile communication unit 1520, and a broadcast reception unit 1530.

The short-range wireless communication unit 151 includes a Bluetooth communication unit, a Bluetooth Low Energy (BLE) communication unit, a Near Field Communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, and an infrared ( It may include an infrared data association (IrDA) communication unit, a Wi-Fi Direct (WFD) communication unit, an ultra wideband (UWB) communication unit, an Ant+ communication unit, etc., but is not limited thereto.

The mobile communication unit 1520 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the radio signal may include a voice call signal, a video call signal, or various types of data according to text/multimedia message transmission/reception.

The broadcast receiver 1530 receives a broadcast signal and/or broadcast-related information from the outside through a broadcast channel. Broadcast channels may include satellite channels and terrestrial channels. According to an implementation example, the electronic device 100a may not include the broadcast receiver 1530.

Also, the communication unit 1500 may transmit/receive data necessary for providing a hint using the user's biometric information to and from the user's external device (not shown) or server (not shown).

An audio/video (A/V) input unit 1600 is for inputting an audio signal or a video signal, and may include a camera 1610 and a microphone 1620. The camera 1610 may obtain an image frame such as a still image or a moving image through an imaging device in a video call mode or a photographing mode. An image captured by the imaging device may be processed through the controller 1300 or a separate image processing unit (not shown). The camera 1610 according to an embodiment may include the photographing unit 110 of FIG. 3 . In addition, the camera 1610 may include the optical system 112 and the infrared imaging unit 114 of FIG. 3 .

An image frame processed by the camera 1610 may be stored in the memory 1700 or transmitted to the outside through the communication unit 1500 . Two or more cameras 1610 may be provided according to the configuration of the terminal.

The microphone 1620 receives external sound signals and processes them into electrical voice data. For example, the microphone 1620 may receive a sound signal from an external device or a speaker. The microphone 1620 may use various noise cancellation algorithms to remove noise generated in the process of receiving an external sound signal.

The memory 1700 may store programs for processing and control of the controller 1300 and may store data input to or output from the electronic device 100a.

The memory 1700 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg SD or XD memory, etc.), RAM (RAM, Random Access Memory) SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk , an optical disk, and at least one type of storage medium.

Programs stored in the memory 1700 may be classified into a plurality of modules according to their functions, such as a UI module 1710, a touch screen module 1720, and a notification module 1730. .

The UI module 1710 may provide a specialized UI, GUI, and the like that are linked with the electronic device 100a for each application. The touch screen module 1720 may detect a user's touch gesture on the touch screen and transmit information about the touch gesture to the controller 1300 . The touch screen module 1720 according to an embodiment may recognize and analyze a touch code. The touch screen module 1720 may be configured as separate hardware including a controller.

Various sensors may be provided inside or near the touch screen to detect a touch or a proximity touch of the touch screen. There is a tactile sensor as an example of a sensor for detecting a touch of a touch screen. A tactile sensor refers to a sensor that detects a contact with a specific object to a degree or more than a human can feel. The tactile sensor may detect various information such as the roughness of a contact surface, the hardness of a contact object, and the temperature of a contact point.

In addition, there is a proximity sensor as an example of a sensor for sensing a touch of a touch screen. A proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object existing nearby without mechanical contact by using the force of an electromagnetic field or infrared rays. Examples of the proximity sensor include a transmissive photoelectric sensor, a direct reflection photoelectric sensor, a mirror reflection photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, an infrared proximity sensor, and the like. The user's touch gestures may include tap, touch & hold, double tap, drag, panning, flick, drag and drop, and swipe.

The notification module 1730 may generate a signal for notifying occurrence of an event of the electronic device 100a. Examples of events occurring in the electronic device 100a include reception of a call signal, reception of a message, input of a key signal, and notification of a schedule. The notification module 1730 may output a notification signal in the form of a video signal through the display unit 1210, output a notification signal in the form of an audio signal through the sound output unit 1220, and may output a notification signal in the form of a vibration motor 1230. A notification signal may be output in the form of a vibration signal through

A display method according to an embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the medium may be those specially designed and configured for the present invention or those known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

In the foregoing specification, the present disclosure and advantages have been described with reference to specific embodiments. However, those skilled in the art can easily accomplish various changes and changes without departing from the scope of the present disclosure as set forth in the claims below. Accordingly, the detailed description and drawings are to be regarded as illustrative examples of the present disclosure and not in a limiting sense. All such possible modifications are intended to be included within the scope of this disclosure.

Claims (20)

In an electronic device that provides a service related to an Internet of Things (IoT) device,
display unit; and
Including a control unit,
The control unit,
transmit the location of the electronic device to a server;
Receiving the IoT device list determined based on the IoT device information stored in the server and the location of the electronic device from the server;
Controlling the display unit to display a live view image currently acquired by a camera of the electronic device;
Displaying a marker indicating the IoT device included in the IoT device list, which is different from the electronic device, in an area corresponding to the location of the IoT device in the live view image;
Displaying a marker selection area on the live view image;
Based on the input for selecting the marker, providing a service result corresponding to the service type of the IoT device;
The electronic device of claim 1 , wherein the input to select the marker includes moving the electronic device so that the marker is positioned within the marker selection area.
According to claim 1,
The IoT device information includes a location of the IoT device and a service type of the IoT device.
According to claim 1,
The control unit,
If the service type includes information production service or information consumption service,
An electronic device that controls the display unit to display information generated or provided by the IoT device.
According to claim 1,
The control unit,
If the service type includes a control service,
Controlling the display unit to display at least one of information on the current state of the IoT device and a button for receiving a request for the control service, the electronic device.
According to claim 1,
The control unit
Transmitting the identification information of the IoT device to the server, receiving the service result of the IoT device from the server,
The service result includes at least one of information generated by the IoT device, information provided by the IoT device, and state information of the IoT device.
According to claim 1,
The IoT device list is
At least one of the location of the IoT device located near the electronic device, identification information of the IoT device, type of service of the IoT device, information used in the IoT device, and state information of the IoT device, electronic device.
delete delete delete delete A method for providing a service related to an Internet of Things (IoT) device,
transmitting the location of the electronic device to the server;
Receiving the IoT device list determined based on the IoT device information stored in the server and the location of the electronic device from the server; and
Displaying a live view image currently acquired by a camera of the electronic device;
displaying a marker indicating the IoT device included in the IoT device list, which is different from the electronic device, in an area corresponding to the location of the IoT device in the live view image;
displaying a marker selection area on the live view image; and
Providing a service result corresponding to a service type of the IoT device based on an input for selecting the marker;
Wherein the input for selecting the marker includes moving the electronic device so that the marker is positioned within the marker selection area.
According to claim 11,
Wherein the IoT device information includes a location of the IoT device and a service type of the IoT device.
According to claim 11,
The step of providing the service result is
If the service type includes information production service or information consumption service,
A method of providing a service related to an IoT device comprising the step of displaying information generated or provided by the IoT device.
According to claim 11,
The step of providing the service result is
If the service type includes a control service,
And displaying at least one of information about the current state of the IoT device and a button for receiving a request for the control service.
◈Claim 15 was abandoned when the registration fee was paid.◈ 12. The method of claim 11, wherein providing the service result
Transmitting identification information of the IoT device to the server; and
Receiving the service result of the IoT device from the server;
The service result includes at least one of information generated by the IoT device, information provided by the IoT device, and state information of the IoT device.
◈Claim 16 was abandoned when the registration fee was paid.◈ The method of claim 11, wherein the IoT device list
At least one of the location of the IoT device located near the electronic device, identification information of the IoT device, type of service of the IoT device, information used in the IoT device, and state information of the IoT device, How to provide services related to IoT devices.
delete delete delete ◈Claim 20 was abandoned when the registration fee was paid.◈ A computer-readable storage medium storing a program for executing the method according to any one of claims 11 to 16.

Images