KR20230159207A - Electronic device for managing controlled device and method of operating the same - Google Patents

Abstract

An electronic device including a communication circuit and at least one processor is disclosed. At least one processor receives device information of an external electronic device from a server, displays a guide screen guiding a communication connection with the external electronic device based on the device information, and displays a guide screen for guiding a communication connection with the external electronic device based on a user input to the guide screen. Establishing the communication connection with the external electronic device through the communication circuit, and receiving connection information and device log data including an error code related to an offline state of the external electronic device from the external electronic device through the communication connection. and transmit a recovery command to the external electronic device through the communication connection based on the error code and the device log data, and receive a message from the server notifying that the external electronic device is connected to the server. there is.

Description

Electronic device for managing a controlled device and its operating method {ELECTRONIC DEVICE FOR MANAGING CONTROLLED DEVICE AND METHOD OF OPERATING THE SAME}

Embodiments of the present invention relate to an electronic device that manages a controlled device and a method of operating the same.

The variety of services and additional functions provided through user terminals, for example, electronic devices such as smart phones, is gradually increasing. In order to increase the utility value of these electronic devices and satisfy the needs of various users, communication service providers or electronic device manufacturers are competitively developing electronic devices that provide various functions. Accordingly, various functions provided through electronic devices are becoming increasingly sophisticated.

As wireless communication technology develops, devices using artificial intelligence (AI) are being widely introduced. For example, home appliances that are connected to a network using Internet of Things (IoT) technology can use artificial intelligence. IoT technology can provide intelligent Internet technology services that create new value in human life by collecting and analyzing data generated from devices. Through the convergence and combination of existing Internet technology and various industries, IoT technology can be applied to fields such as smart homes, smart buildings, smart cities, smart cars, and smart home appliances.

Meanwhile, homes are equipped with various home appliances for user convenience. Various services are being proposed to utilize IoT technology to make the operation or control of home appliances more convenient. Home network technology can provide various services to users at home through a home network. For example, a user can use a personal electronic device (e.g., a smart phone) to control various controlled devices (e.g., home appliances with IoT technology) that make up a home network. . Users may wish to receive more diverse services to control controlled devices. Accordingly, there is a demand for the development of various technologies to manage controlled devices by reflecting the user's intention.

The user uses an electronic device (e.g. a smartphone or wearable device) owned by the user to control the controlled device (e.g. a television (TV), air conditioner, washing machine, security camera, lighting device, or switch). Thus, a procedure (e.g., onboarding) for registering the controlled device to a network (e.g., cloud server) can be performed. The electronic device can connect the controlled device to the user account by controlling the controlled device to register with the server. The electronic device can access the server and control the controlled device through a client application using a user account.

The electronic device can check and control the status of controlled devices registered for the user account through a client application. If the controlled device cannot properly connect to the server, the controlled device may be displayed as offline in the client application. A controlled device can go offline for a number of reasons, such as device issues, network failures, cloud failures, or application failures. If the controlled device is displayed as offline, the electronic device may provide an offline guide pop-up using a plug-in application. In conventional technology, the offline guide pop-up only provides general action guides (help guides) such as checking the in-house network environment and checking the power plug, so it may not be of practical help in resolving the connection failure of the controlled device.

Additionally, if an offline problem occurs due to a software failure in a controlled device that uses constant power (e.g. a refrigerator), the connection cannot be restored by turning the power on/off and the power plug must be unplugged and plugged in again. However, in the case of built appliances such as refrigerators, it may be difficult for users to unplug the power plug and plug it back in. In this case, in order to resolve the offline state of the controlled device, it is necessary to delete the controlled device and then register it on the server again, but it was difficult for the user to know the exact action guide. In order to re-register a controlled device, existing registration information must be deleted and then the registration process must be performed again, which causes inconvenience to the user.

Embodiments of the present disclosure can directly connect an electronic device to an offline controlled device and provide easy connectivity for diagnosis.

Embodiments of the present disclosure relate to an electronic device and a method of operating the same that determine the cause of an offline controlled device through direct connection diagnosis and provide a suitable solution to the controlled device.

Embodiments of the present disclosure relate to an electronic device and a method of operating the same that improve service stability and user satisfaction of a controlled device through log information when diagnosing a direct connection of an offline controlled device.

The technical problem to be achieved in the present disclosure is not limited to the technical problem mentioned above, and other technical problems not mentioned can be clearly understood by those skilled in the art from the description below. There will be.

An electronic device according to an embodiment may include a communication circuit and at least one processor operatively connected to the communication circuit. The at least one processor may be configured to receive device information of an external electronic device from a server. The at least one processor may be configured to display a guide screen that guides communication connection with the external electronic device based on the device information. The at least one processor may be configured to establish the communication connection with the external electronic device through the communication circuit based on a user input on the guide screen. The at least one processor may be configured to receive connection information and device log data including an error code related to an offline state of the external electronic device from the external electronic device through the communication connection. The at least one processor may be configured to transmit a recovery command to the external electronic device through the communication connection based on the error code and the device log data. The at least one processor may be configured to receive a message from the server notifying that the external electronic device is connected to the server.

A method of operating an electronic device according to an embodiment may include receiving device information of an external electronic device from a server. The method may include displaying a guide screen that guides communication connection with the external electronic device based on the device information. The method may include establishing the communication connection with the external electronic device through the communication circuit based on a user input to the guide screen. The method may include receiving connection information and device log data including an error code related to an offline state of the external electronic device from the external electronic device through the communication connection. The method may include transmitting a recovery command to the external electronic device through the communication connection based on the error code and the device log data. The method may include receiving a message from the server indicating that the external electronic device is connected to the server.

According to one embodiment, a non-transitory computer-readable storage medium storing one or more programs, wherein the one or more programs, when executed by at least one processor of an electronic device, cause the electronic device to: retrieve device information of an external electronic device from a server; Receive, display a guide screen guiding a communication connection with the external electronic device based on the device information, and establish a communication connection with the external electronic device through the communication circuit based on a user input on the guide screen. Establishing and receiving connection information and device log data including an error code related to an offline state of the external electronic device from the external electronic device through the communication connection, and performing the communication based on the error code and the device log data It may include commands configured to transmit a recovery command to the external electronic device through connection and receive a message from the server notifying that the external electronic device is connected to the server.

1 illustrates an internet of things (IoT) system according to various embodiments.
2 is a block diagram of an electronic device in a network environment, according to various embodiments.
FIG. 3 is a diagram illustrating a network including controlled devices according to an embodiment.
FIG. 4 is a diagram for explaining the status of a controlled device displayed on an electronic device, according to an embodiment.
FIG. 5 is a diagram for explaining a system architecture including controlled devices according to an embodiment.
Figure 6 is a flowchart illustrating a procedure for providing direct connection diagnosis of a controlled device according to an embodiment.
Figure 7 is a diagram for explaining a diagnostic procedure according to an embodiment.
FIG. 8 is a diagram illustrating a procedure for a controlled device to update device information on a server, according to an embodiment.
FIG. 9 is a diagram illustrating a procedure in which a controlled device registers device information in a server according to an embodiment.
Figure 10 shows a signal flow diagram (sequence diagram) to explain a procedure for executing direct connection diagnosis of a controlled device through a secure connection according to an embodiment.
Figure 11 is a signal flow diagram to explain a procedure for confirming ownership of a controlled device according to an embodiment.
FIG. 12 is a signal flow diagram illustrating a procedure for receiving connection information and device log data according to an embodiment.
Figure 13 is a signal flow diagram to explain error handling and error reporting according to an embodiment.
FIG. 14 is a signal flow diagram illustrating a procedure for executing direct connection diagnosis of a controlled device through a non-secure connection according to an embodiment.
Figure 15 shows a signal flow diagram to explain a procedure for executing direct connection diagnosis of a controlled device using soft AP mode according to an embodiment.
FIG. 16 is a signal flow diagram illustrating a procedure for receiving connection information and device log data according to an embodiment.
Figure 17 is a diagram illustrating a user interface screen for executing general diagnosis according to one embodiment.
FIGS. 18A, 18B, and 18C are diagrams illustrating user interface screens for executing direct connection diagnosis according to an embodiment.
19A and 19B are diagrams illustrating user interface screens for error handling and error reporting according to an embodiment.
FIG. 20 is a diagram illustrating user interface screens for executing error handling through a non-secure connection, according to an embodiment.
FIG. 21 is a diagram illustrating user interface screens for executing error handling through a secure connection according to an embodiment.
FIG. 22 is a diagram illustrating user interface screens that guide network reselection through error handling according to an embodiment.

1 illustrates an internet of things (IoT) system 100 according to various embodiments. Meanwhile, at least some of the components in FIG. 1 may be omitted, and may be implemented to include additional components not shown.

Referring to FIG. 1, the IoT system 100 according to one embodiment includes a plurality of electronic devices connectable to the data network 116 or 146. For example, the IoT system 100 includes a first IoT server 110, a first node 120, a voice assistance server 130, a second IoT server 140, and a second node. 150, or may include at least one of the devices 121, 122, 123, 124, 125, 136, 137, 151, 152, and 153.

According to one embodiment, the first IoT server 110 may include at least one of a communication interface 111, a processor 112, or a storage unit 113. The second IoT server 140 may include at least one of a communication interface 141, a processor 142, or a storage unit 143. “IoT server” in this document refers to a relay device (e.g., first node 120 or second node (120), for example, based on a data network (e.g., data network 116 or data network 146). One or more devices (e.g., devices 121, 122, 123, 124, 125, 151, 152, 153) can be remotely controlled and/or monitored via 150)) or directly without a relay device. “Device” herein refers to a sensor, appliance, office electronic device, or It is a device for performing processes, and there are no restrictions on its type. A device that receives a control command and performs an operation corresponding to the control command may be named a “target device.” The IoT server may be called a central server in that it selects a target device among a plurality of devices and provides control commands.

According to one embodiment, the first IoT server 110 may communicate with the devices 121, 122, and 123 through the data network 116. Data network 116 may refer to a network for long-distance communication, such as the Internet or a computer network (e.g., LAN or WAN), or may include a cellular network.

According to one embodiment, the first IoT server 110 may be connected to the data network 116 through the communication interface 111. The communication interface 111 may include a communication device (or communication module) to support communication of the data network 116, and may be integrated into one component (e.g., a single chip), or may be integrated into a plurality of separate components. It can be implemented with components (e.g., multiple chips). The first IoT server 110 may communicate with the devices 121, 122, and 123 through the first node 120. The first node 120 may receive data from the first IoT server 110 through the data network 116 and transmit the received data to at least some of the devices 121, 122, and 123. Alternatively, the first node 120 may receive data from at least some of the devices 121, 122, and 123, and transmit the received data to the first IoT server 110 through the data network 116. The first node 120 may function as a bridge between the data network 116 and the devices 121, 122, and 123. Meanwhile, in FIG. 1, it is shown as if there is only one first node 120, but this is simply an example and there is no limit to the number.

A “node” in this document may be an edge computing system, or may be a hub device. According to one embodiment, the first node 120 supports wired and/or wireless communication of the data network 116, and may also support wired and/or wireless communication with the devices 121, 122, and 123. . For example, the first node 120 may be configured to communicate via a short-range communication network such as at least one of Bluetooth, Wi-Fi, Wi-Fi direct, Z-wave, Zig-bee, INSETEON, X10, or IrDA (infrared data association). It can be connected to devices 121, 122, and 123, but there is no limitation on the type of communication. The first node 120 is placed in an environment such as a home, office, factory, building, external location, or other types of premises. (or, location). Accordingly, the devices 121, 122, and 123 may be monitored and/or controlled by the service provided by the first IoT server 110, and the devices 121, 122, and 123 may be connected to the first IoT server 110. It may not be required to have the capability of complete network communication (e.g., Internet communication) for direct connection to the IoT server 110. Devices 121, 122, and 123 may include, for example, a light switch, a proximity sensor, Although it is shown as being implemented as an electronic device in a home environment, such as a temperature sensor, this is illustrative and is not limiting.

According to one embodiment, the first IoT server 110 may support direct communication with the devices 124 and 125. Here, “direct communication” may mean communication that does not go through a relay device such as the first node 120, for example, communication through a cellular communication network and/or a data network.

According to one embodiment, the first IoT server 110 may transmit a control command to at least some of the devices 121, 122, 123, 124, and 125. Here, “control command” may mean data that causes a controllable device to perform a specific operation, and the specific operation is an operation performed by the device, such as outputting information, sensing information, reporting information, It may include management of information (e.g. deletion or creation), and there is no limit to the type. For example, the processor 112 generates a control command from an external source (e.g., the voice assistant server 130, the second IoT server 140, the external system 160, or at least some of the devices 121, 122, 123, 124, and 125). Information (or request) to do so may be obtained, and a control command may be generated based on the obtained information. Alternatively, the processor 112 may generate a control command based on the monitoring results of at least some of the devices 121, 122, 123, 124, and 125 satisfying specified conditions. The processor 112 may control the communication interface 111 to transmit control commands to the target device.

According to one embodiment, the processor 112, or the processor 132, or the processor 142 includes a central processing unit (CPU), a digital signal processor (DSP), an application processor (AP), or a communication processor (CP). It may be implemented as a combination of one or more of a general-purpose processor, a graphics-specific processor such as a GPU (graphical processing unit), a VPU (vision processing unit), or an artificial intelligence-specific processor such as an NPU (neural processing unit). Those skilled in the art will understand that the above-described processing unit is merely an example, and that the processor 112 is not limited to any computational means that can, for example, execute instructions stored in the memory 113 and output the executed result.

According to one embodiment, the processor 112 may configure a web-based interface based on the API 114 or expose resources managed by the first IoT server 110 to the outside. . The web-based interface may support communication between the first IoT server 110 and an external web service, for example. The processor 112 may, for example, allow the external system 160 to control and/or access the devices 121, 122, and 123. External system 160 may be, for example, an independent system that is not related to or part of system 100. External system 160 may be, for example, an external server or a website. However, security is required for access to the devices 121, 122, and 123 from the external system 160 or the resources of the first IoT server 110. According to one embodiment, the processor 112 and the automation application may expose an API endpoint (eg, a universal resource locator (URL)) based on the API 114 to the outside. According to the above description, the first IoT server 110 may transmit a control command to the target device among the devices 121, 122, and 123. Meanwhile, the description of the communication interface 141, the processor 142, the API 144 of the storage unit 143, and the database 145 of the second IoT server 140 are described in detail in the communication of the first IoT server 110. It may be substantially the same as the description of the interface 111, the processor 112, the API 114 of the storage unit 113, and the database 115. In addition, the description of the second node 150 may be substantially the same as the description of the first node 120. The second IoT server 140 may transmit a control command to a target device among the devices 151, 152, and 153. The first IoT server 110 and the second IoT server 140 may be operated by the same service provider in one embodiment, but may be operated by different service providers in another embodiment.

According to one embodiment, the voice assistant server 130 may transmit and receive data with the first IoT server 110 through the data network 116. The voice assistant server 130 according to one embodiment may include at least one of a communication interface 131, a processor 132, and a storage unit 133. The communication interface 131 may communicate with the smart phone 136 or the AI speaker 137 through a data network (not shown) and/or a cellular network (not shown). The smart phone 136 or the AI speaker 137 may include a microphone, acquire a user voice, convert it into a voice signal, and transmit the voice signal to the voice assistant server 130. The processor 132 may receive a voice signal from the smart phone 136 or the AI speaker 137 through the communication interface 131. The processor 132 may process the received voice signal based on the stored model 134. The processor 132 may generate (or confirm) a control command using the processing result based on information stored in the database 135. According to one embodiment, the storage units 113, 133, and 143 include flash memory type, hard disk type, multimedia card micro type, and card type memory (e.g. SD or memory), magnetic memory, magnetic disk, or optical disk, and may include at least one type of non-transitory storage medium, and there is no limitation on the type.

In various embodiments, at least one device (e.g., device 124) communicating with the first IoT server 110 may be a smartphone (e.g., electronic device 201 of FIG. 2) in a network environment. .

FIG. 2 is a block diagram of an electronic device 201 in a network environment 200, according to various embodiments.

Referring to FIG. 2, in the network environment 200, the electronic device 201 communicates with the electronic device 202 through the first network 298 (e.g., a short-range wireless communication network) or through the second network 299. It is possible to communicate with at least one of the electronic device 204 or the server 208 through (e.g., a long-distance wireless communication network). According to one embodiment, the electronic device 201 may communicate with the electronic device 204 through the server 208. According to one embodiment, the electronic device 201 includes a processor 220, a memory 230, an input module 250, an audio output module 255, a display module 260, an audio module 270, and a sensor module ( 276), interface 277, connection terminal 278, haptic module 279, camera module 280, power management module 288, battery 289, communication module 290, subscriber identification module 296 , or may include an antenna module 297. In some embodiments, at least one of these components (eg, the connection terminal 278) may be omitted, or one or more other components may be added to the electronic device 201. In some embodiments, some of these components (e.g., sensor module 276, camera module 280, or antenna module 297) are integrated into one component (e.g., display module 260). It can be.

Processor 220, for example, executes software (e.g., program 240) to operate at least one other component (e.g., hardware or software component) of electronic device 201 connected to processor 220. It can be controlled and various data processing or calculations can be performed. According to one embodiment, as at least part of the data processing or computation, the processor 220 stores instructions or data received from another component (e.g., the sensor module 276 or the communication module 290) in the volatile memory 232. The commands or data stored in the volatile memory 232 can be processed, and the resulting data can be stored in the non-volatile memory 234. According to one embodiment, the processor 220 includes a main processor 221 (e.g., a central processing unit or an application processor) or an auxiliary processor 223 that can operate independently or together (e.g., a graphics processing unit, a neural network processing unit ( It may include a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device 201 includes a main processor 221 and a auxiliary processor 223, the auxiliary processor 223 may be set to use lower power than the main processor 221 or be specialized for a designated function. You can. The auxiliary processor 223 may be implemented separately from the main processor 221 or as part of it.

The auxiliary processor 223 may, for example, act on behalf of the main processor 221 while the main processor 221 is in an inactive (e.g., sleep) state, or while the main processor 221 is in an active (e.g., application execution) state. ), together with the main processor 221, at least one of the components of the electronic device 201 (e.g., the display module 260, the sensor module 276, or the communication module 290) At least some of the functions or states related to can be controlled. According to one embodiment, coprocessor 223 (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module 280 or communication module 290). there is. According to one embodiment, the auxiliary processor 223 (eg, neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. Artificial intelligence models can be created through machine learning. For example, such learning may be performed in the electronic device 201 itself on which the artificial intelligence model is performed, or may be performed through a separate server (e.g., server 208). Learning algorithms may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but It is not limited. An artificial intelligence model may include multiple artificial neural network layers. Artificial neural networks include deep neural network (DNN), convolutional neural network (CNN), recurrent neural network (RNN), restricted boltzmann machine (RBM), belief deep network (DBN), bidirectional recurrent deep neural network (BRDNN), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the examples described above. In addition to hardware structures, artificial intelligence models may additionally or alternatively include software structures.

The memory 230 may store various data used by at least one component (eg, the processor 220 or the sensor module 276) of the electronic device 201. Data may include, for example, input data or output data for software (e.g., program 240) and instructions related thereto. Memory 230 may include volatile memory 232 or non-volatile memory 234.

The program 240 may be stored as software in the memory 230 and may include, for example, an operating system 242, middleware 244, or application 246.

The input module 250 may receive commands or data to be used in a component of the electronic device 201 (e.g., the processor 220) from outside the electronic device 201 (e.g., a user). The input module 250 may include, for example, a microphone, mouse, keyboard, keys (eg, buttons), or digital pen (eg, stylus pen).

The sound output module 255 may output sound signals to the outside of the electronic device 201. The sound output module 255 may include, for example, a speaker or a receiver. Speakers can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.

The display module 260 can visually provide information to the outside of the electronic device 201 (eg, a user). The display module 260 may include, for example, a display, a hologram device, or a projector, and a control circuit for controlling the device. According to one embodiment, the display module 260 may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of force generated by the touch.

The audio module 270 can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module 270 acquires sound through the input module 250, the sound output module 255, or an external electronic device (e.g., directly or wirelessly connected to the electronic device 201). Sound may be output through an electronic device 202 (e.g., speaker or headphone).

The sensor module 276 detects the operating state (e.g., power or temperature) of the electronic device 201 or the external environmental state (e.g., user state) and generates an electrical signal or data value corresponding to the detected state. can do. According to one embodiment, the sensor module 276 includes, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, humidity sensor, or light sensor.

The interface 277 may support one or more designated protocols that can be used to connect the electronic device 201 directly or wirelessly with an external electronic device (eg, the electronic device 202). According to one embodiment, the interface 277 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 278 may include a connector through which the electronic device 201 can be physically connected to an external electronic device (eg, the electronic device 202). According to one embodiment, the connection terminal 278 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 279 can convert electrical signals into mechanical stimulation (e.g., vibration or movement) or electrical stimulation that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module 279 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 280 can capture still images and moving images. According to one embodiment, the camera module 280 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 288 can manage power supplied to the electronic device 201. According to one embodiment, the power management module 288 may be implemented as at least a part of, for example, a power management integrated circuit (PMIC).

Battery 289 may supply power to at least one component of electronic device 201. According to one embodiment, the battery 289 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

Communication module 290 provides a direct (e.g., wired) communication channel or wireless communication channel between electronic device 201 and an external electronic device (e.g., electronic device 202, electronic device 204, or server 208). It can support establishment and communication through established communication channels. Communication module 290 operates independently of processor 220 (e.g., an application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module 290 is a wireless communication module 292 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 294 (e.g., : LAN (local area network) communication module, or power line communication module) may be included. Among these communication modules, the corresponding communication module is a first network 298 (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 299 (e.g., legacy It may communicate with an external electronic device 204 through a telecommunication network such as a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., LAN or WAN). These various types of communication modules may be integrated into one component (e.g., a single chip) or may be implemented as a plurality of separate components (e.g., multiple chips). The wireless communication module 292 uses subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 296 within a communication network such as the first network 298 or the second network 299. The electronic device 201 can be confirmed or authenticated.

The wireless communication module 292 may support 5G networks after 4G networks and next-generation communication technologies, for example, NR access technology (new radio access technology). NR access technology provides high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low latency). -latency communications)) can be supported. The wireless communication module 292 may support high frequency bands (e.g., mmWave bands), for example, to achieve high data rates. The wireless communication module 292 uses various technologies to secure performance in high frequency bands, for example, beamforming, massive array multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. It can support technologies such as input/output (FD-MIMO: full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module 292 may support various requirements specified in the electronic device 201, an external electronic device (e.g., electronic device 204), or a network system (e.g., second network 299). According to one embodiment, the wireless communication module 292 supports Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mmTC, or U-plane latency (e.g., 164 dB or less) for realizing URLLC. Example: Downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) can be supported.

The antenna module 297 may transmit or receive signals or power to or from the outside (e.g., an external electronic device). According to one embodiment, the antenna module 297 may include an antenna including a radiator made of a conductor or a conductive pattern formed on a substrate (eg, PCB). According to one embodiment, the antenna module 297 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 298 or the second network 299 is, for example, connected to the plurality of antennas by the communication module 290. can be selected Signals or power may be transmitted or received between the communication module 290 and an external electronic device through the at least one selected antenna. According to some embodiments, in addition to the radiator, other components (eg, radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module 297.

According to various embodiments, the antenna module 297 may form a mmWave antenna module. According to one embodiment, a mmWave antenna module includes: a printed circuit board, an RFIC disposed on or adjacent to a first side (e.g., bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., mmWave band); And a plurality of antennas (e.g., array antennas) disposed on or adjacent to the second side (e.g., top or side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (e.g., bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and signal ( (e.g. commands or data) can be exchanged with each other.

According to one embodiment, commands or data may be transmitted or received between the electronic device 201 and the external electronic device 204 through the server 208 connected to the second network 299. Each of the external electronic devices 202 or 204 may be of the same or different type as the electronic device 201. According to one embodiment, all or part of the operations performed in the electronic device 201 may be executed in one or more of the external electronic devices 202, 204, or 208. For example, when the electronic device 201 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 201 may perform the function or service instead of executing the function or service on its own. Alternatively, or additionally, one or more external electronic devices may be requested to perform at least part of the function or service. One or more external electronic devices that have received the request may execute at least part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device 201. The electronic device 201 may process the result as is or additionally and provide it as at least part of a response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology can be used. The electronic device 201 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 204 may include an Internet of Things (IoT) device. Server 208 may be an intelligent server using machine learning and/or neural networks. According to one embodiment, the external electronic device 204 or server 208 may be included in the second network 299. The electronic device 201 may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

FIG. 3 is a diagram illustrating a network including controlled devices according to an embodiment.

Referring to FIG. 3, the network 300 includes a server 310 operating as an IoT cloud, an electronic device 201 capable of communicating with the server 310 through network communication (e.g., the Internet), and IoT technology. At least one external electronic device 320 (e.g., controlled devices 320a, 320b, 320c, 320d) that supports and can communicate with the server 310 through network communication (e.g., the Internet) , 320e)). In one embodiment, a hub device 330 that supports a connection between the controlled device 320 and the electronic device 201 and/or a connection between the controlled device 320 and the server 310 may be further included in the network 300. You can. In one embodiment, the electronic device 201 is connected via hub device 330, via server 310, via long-range wireless communication (e.g., second network 299), or short-range wireless communication (e.g., It is possible to communicate with the controlled device 320 through the first network 298).

In one embodiment, the controlled device 320 may be controlled (e.g., report status and/or execute a specific function) by a remote command (e.g., a control command of the electronic device 201), for example It may include at least one of a television, air conditioner, refrigerator, washing machine, lighting device, security camera, sensor, or window treatment. The controlled device 320 communicates with the electronic device 201 through the hub device 330, communicates with the electronic device 201 through the server 310, and/or directly with the electronic device 201 (e.g. For example, communication can be performed (without going through the server 310 or the hub device 330). In one embodiment, the controlled device 320 is connected to the electronic device 201 via long-range wireless communication (e.g., the second network 299) or short-range wireless communication (e.g., the first network 298). It may be configured to communicate with. In one embodiment, the controlled device 320 is connected to the server 310 through long-distance wireless communication (e.g., the second network 299) or short-range wireless communication (e.g., the first network 298). Can be configured to communicate.

In one embodiment, the electronic device 201 may be a personal electronic device, such as a smart phone, tablet, or wearable device, or an electronic device that includes a display and a user interface, such as a television or control console. The electronic device 201 can discover the controlled device 320 and execute a registration procedure to register the discovered controlled device 320 with the server 310. The controlled device 320 may be registered with the server 310 to be associated with a user account. The electronic device 201 can monitor and control the controlled device 320 registered with the server 310 using the user account.

In one embodiment, the electronic device 201 checks the status of the controlled device 320 that the user will use for the IoT control service, or controls the controlled device 320 (for example, a control command instructing to execute a specific function) can be transmitted). In one embodiment, the electronic device 201 may be an owner device included in the network 300. In one embodiment, at least one member device that includes at least some functions and/or permissions of the electronic device 201, although not shown, may be included in the network 300. In one embodiment, the member device cannot perform a registration procedure for the controlled device 320, but can perform a function to check or control the status of the controlled device 320 registered in the server 310.

In one embodiment, the hub device 330 is an electronic device that operates an IoT control service and may be a server or gateway placed within a building (home or hotel), or a remote server placed outside the building. In one embodiment, the hub device 330 may be a home appliance such as a smartphone, tablet, personal computer (PC), or TV that has a hub function. The hub device 330 may be registered with the server 310 through the electronic device 201 similarly to the controlled device 320 according to at least one of the embodiments described later.

FIG. 4 is a diagram for explaining the status of a controlled device displayed on an electronic device, according to an embodiment.

Referring to FIG. 4, the electronic device 201 executes a client application for an IoT control service and displays an execution screen 400 provided by the client application through a display module (e.g., display module 260). can do. The execution screen 400 displays the states (e.g., at least one of image, location, name, or connection status) of at least one external electronic device (e.g., controlled device 320) registered for the user account. It can be included.

In one embodiment, when the controlled device 320 is disconnected from the Internet, the execution screen 400 may include a status item (e.g., status item 410) indicating that the controlled device 320 is offline. You can. For example, based on receiving information about the controlled device 320 being offline from the server 310, the electronic device 101 displays a status item 410 indicating that the controlled device 320 is offline. ) can be displayed. In one embodiment, the status item 410 of the controlled device 320 that is offline may be displayed in black and white. Although not shown, status items of controlled devices that are online may be displayed in color. Although not shown, when a user input (for example, a touch) for the status item 410 is received, the electronic device 201 may display an offline guide pop-up indicating that the controlled device 320 is offline. For example, the offline guide pop-up might say, "Make sure your device's power cord is plugged in properly," "Make sure your device powers on properly," "Make sure your phone is connected to the network," or "Make sure your device is connected to the Wi-Fi network." You can include general action guides such as "Make sure your Fi router is working properly."

FIG. 5 is a diagram for explaining a system architecture including controlled devices according to an embodiment.

Referring to FIG. 5, the electronic device 201 registers at least one external electronic device (e.g., the controlled device 320) with the server 310 and remotely controls the registered controlled device 320. It may include a client application configured to do so. The controlled device 320 is, for example, at least one of hub-connected (502), direct-connected (504), or cloud-to-cloud (506). You can connect to the server 310 using . The hub connection method 502 may connect the controlled device 320 to the server 310 through a hub having a communication function such as Zig-bee, Z-wave, or LAN. In the direct connection method 504, the controlled device 320 can connect directly to a Wi-Fi access point (AP) and connect to the server 310 through the AP. In the inter-cloud connection method 506, the controlled device 320 may be registered in a third-party cloud, and the server 310 may be connected to the server 310 using an API (application programming interface) through the third-party cloud. You can connect with.

In one embodiment, the server 310 may include a front-end server (not shown) that manages device information registered by the manufacturer of the controlled device 320, and the controlled device 320 By confirming that it is registered as a support device on the end server, the controlled device 320 can be authenticated, and the authenticated controlled device 320 can be included in the list of supported devices and managed. The electronic device 201 may register (eg, onboard) the controlled device 320 included in the support management list of the server 310 with the server 310 using a client application. Registration (or onboarding) may refer to a process of storing device information related to the controlled device 320 on the server 310 through a client application of the electronic device 201.

In one embodiment, when the controlled device 320 is connected to the server 410 using the direct connection method 504, it may go offline due to causes such as, for example, device problems, network failures, cloud failures, or application failures. You can. When the connection between the controlled device 320 and the server 310 is lost, the electronic device 201 may display the status item (for example, the status item 410) of the controlled device 320 as offline.

Figure 6 is a flowchart illustrating a procedure for providing direct connection diagnosis of a controlled device according to an embodiment. At least one of the operations described later may be executed by a processor (eg, processor 220) of the electronic device 201. At least some of the operations described below may be omitted, modified, or changed in order.

Referring to FIG. 6, in operation 605, the electronic device 201 (e.g., processor 220) receives device information of an offline controlled device (e.g., controlled device 320) from the server 310. You can receive it. In one embodiment, the device information may include diagnostics capability information indicating whether the controlled device 320 supports direct connection diagnostics (DCD). In one embodiment, the electronic device 201 (e.g., the processor 220) receives from the server 310 that the controlled device (e.g., the controlled device 320) is offline, and the controlled device 320 ) can be received by requesting the device information from the server 310.

In operation 610, the electronic device 201 (eg, processor 220) may determine whether direct connection diagnosis for the controlled device 320 is supported based on the device information. If direct connection diagnosis is not supported, in operation 650, the electronic device 201 (e.g., processor 220) may execute general diagnostic procedures (basic diagnostics or basic analysis) (e.g., operation 720 of FIG. 7). . On the other hand, if direct connection diagnostics (or detailed analysis) is supported, the electronic device 201 (eg, processor 220) may proceed to operation 615. In one embodiment, the electronic device 201 (e.g., the processor 220) may already know that the controlled device 320 supports DCD, and operation 610 may be omitted.

In operation 615, the electronic device 201 (e.g., processor 220) uses the device information to establish a communication connection (e.g., D2D connection) with the controlled device 320 to display a guide screen (e.g., A guide screen 1804 in FIG. 18B may be provided. In one embodiment, the guide screen is information for guiding the D2D connection between the electronic device 201 and the controlled device 320, for example, a device image of the controlled device 320 and a guide text (e.g. “Power on the device and press the button for at least 3 seconds until the AP appears”). In one embodiment, the guide screen may further include an object (eg, an input button) for starting the D2D connection. In one embodiment, the electronic device 201 (e.g., processor 220) uses the information included in the device information (e.g., mnID and/or setupId) to generate setup data (e.g., setup data necessary for configuring the guide screen). For example, a device image and guide text) may be received from the server 310. In one embodiment, the server 310 may identify that the controlled device 320 is registered with the server 310 based on the above information and provide the setup data to the electronic device 201.

In operation 620, the electronic device 201 (eg, processor 220) may establish a D2D connection with the controlled device 320. In one embodiment, the D2D connection uses soft AP, Bluetooth (e.g., Bluetooth classic, Bluetooth legacy, or Bluetooth low energy (BLE)), or hypertext transfer protocol (HTTP) of an internet protocol (IP) network. It can be established based on

In operation 625, the electronic device 201 (eg, processor 220) may execute a procedure to confirm ownership of the controlled device 320 through the D2D connection. In one embodiment, the electronic device 201 (eg, the processor 220) may confirm that the user owns the controlled device 320 based on input information related to the controlled device 320. In one embodiment, the electronic device 201 (e.g., the processor 220) determines whether the personal identification number (PIN) provided by the controlled device 320 matches the PIN entered by the user, thereby It can be confirmed that the controlled device 320 is owned. In one embodiment, the electronic device 201 (e.g., processor 220) displays a verification indicator (e.g., a quick response (QR) code or bar) attached or printed on the surface of the controlled device 320. code), receiving a signal from the controlled device 320 indicating that a specific button of the controlled device 320 is input, or receiving biometric information or a password through the controlled device 320, so that the user can It can be confirmed that the control device 320 is owned.

In one embodiment, the electronic device 201 (e.g., processor 220) provides credential information (e.g., device ID of the controlled device 320) of the controlled device 320 through ownership verification. can be saved. In one embodiment, if the electronic device 201 (eg, processor 220) already has credential information of the controlled device 320, operation 625 may be omitted.

In one embodiment, the electronic device 201 may be an owner device that has performed the registration procedure of the controlled device 320, or may be a member device whose ownership has been permitted by the owner device. When the electronic device 201 is a member device, the electronic device 201 can confirm ownership of the controlled device 320 using credential information received from the owner device or the controlled device 320.

In operation 630, the electronic device 201 (eg, processor 220) may receive connection information including an offline error code from the controlled device 320. In one embodiment, the offline error code may include a code for at least one of a cloud error, a network error, or an unknown error.

In operation 635, the electronic device 201 (eg, processor 220) may receive device log data from the controlled device 320. In one embodiment, the device log data may include the operation history of the controlled device 320 (for example, at least one of power on, function execution, status change, error occurrence, or power off). In one embodiment, operation 635 may be omitted.

In operation 640, the electronic device 201 (e.g., the processor 220) may transmit a recovery command to the controlled device 320 based on the offline error code and/or the device log data, and the server 310 You can check the status of the controlled device 320. In one embodiment, the recovery command may be delivered to the controlled device 320 through the D2D connection established in operation 620. In one embodiment, the electronic device 201 (eg, the processor 220) may display an action guide screen related to the offline state of the controlled device 320 processed based on the offline error code. In one embodiment, the electronic device 201 (e.g., the processor 220) receives a response message from the server 310 indicating that the controlled device 320 is connected to the server 310, thereby 320) can be judged to have been restored normally.

In one embodiment, the recovery command was not received by the controlled device 320, the controlled device 320 failed to properly perform self-diagnosis according to the recovery command, or the controlled device 320 failed to perform the self-diagnosis according to the recovery command. If connection to the server 310 is not possible despite performing self-diagnosis, the server 310 may still manage the controlled device 320 in an offline state, and the electronic device 201 (e.g., processor 220 )) may receive a response message from the server 310 indicating that the controlled device 320 is not connected. In one embodiment, the electronic device 201 (e.g., the processor 220) outputs a message notifying that the controlled device 320 is offline or that the self-diagnosis of the controlled device 320 has failed. You can.

In operation 645, the electronic device 201 (eg, the processor 220) may transmit an error reporting message to the server 310 to notify that error handling of the controlled device 320 is complete. In one embodiment, the error reporting message may include the connection information (eg, offline error code) and/or the device log data. In one embodiment, the electronic device 201 (e.g., processor 220) may display an error reporting guide screen including an object for error reporting (e.g., error reporting button 1906), and The error reporting message may be transmitted to the server 310 in response to receiving a user input to the report button. In one embodiment, when a user input for the error reporting button is not received, or when error reporting is determined to be unnecessary based on an offline error code, the electronic device 201 may omit operation 645.

Figure 7 is a diagram for explaining a diagnostic procedure according to an embodiment.

Referring to FIG. 7 , the electronic device 201 may display an offline diagnosis screen 700 including information on at least one controlled device 320 that is offline. In one embodiment, the offline diagnosis screen 700 may include a status item 705 corresponding to the controlled device 320 (e.g., TV). Upon receiving the user input 710 for the status item 705, the electronic device 201 may determine whether the controlled device 320 supports DCD in operation 715.

In one embodiment, the electronic device 201 receives device information of the controlled device 320 from the server 310 to check whether the offline controlled device 320 supports DCD, and the device If the information includes a DCD support field of the controlled device 320, it can be determined that the controlled device 320 supports DCD. The device information may include a device profile representing basic information for device registration and may be defined by the manufacturer. The device profile may be received from the server 310 through, for example, a firmware update in the electronic device 201. In one embodiment, the device information may include diagnostic performance information indicating whether the controlled device 320 supports DCD.

If the controlled device 320 does not support DCD, the electronic device 201 may execute a basic diagnostics procedure 720. In one embodiment, the general diagnostic procedure 720 includes an operation 722 for checking a client application (e.g., software version) for the controlled device 320 and an operation for checking a Wi-Fi connection in the electronic device 201. It may include (724), an operation (726) of executing a predefined general diagnosis, and an operation (728) of displaying the results.

If the controlled device 320 supports DCD, the electronic device 201 may execute a direct connection diagnosis (DCD) procedure 730. In one embodiment, the DCD procedure 730 includes an operation 732 for executing a general diagnosis, an operation 734 for directly connecting to an offline controlled device 320, and an operation 736 for executing a predefined direct connection diagnosis. ), and an operation 738 for displaying the results. Compared to the general diagnosis procedure 720, the DCD procedure 730 allows the electronic device 201 to directly connect to the controlled device 320 and obtain information related to diagnosis (e.g., connection information and/or device log data). Additional actions may be included. In order for the electronic device 201 to connect to the controlled device 320, the controlled device 320 may perform an operation to prepare for connection (for example, pressing a specific button on a washing machine).

Table 1 below shows an example of diagnostic performance information according to an embodiment.

Attributes Value Description logType errCode, dump Logging method endpoint PIPER, SSM Logging system endpoint minVersion 1.0 Offline Diagnostic Service Version setupId XXX Onboarding ID protocolType ble_ocf, wifi_https, ble_stdk Offline Diagnostic Service Protocol mnId SSS Manufacturer ID dumpType File, id Device log delivery method

In one embodiment, the diagnostic performance information may include at least one of the 'logType' attribute, the 'endpoint' attribute, the 'minVersion' attribute, the 'setupId' attribute, the 'protocolType' attribute, the 'mnId' attribute, or the 'dumpType' attribute. there is. In one embodiment, mnId may represent a manufacturer ID, setupId may represent an onboarding ID given when registering with the server 310, and protocolType may represent a service protocol to be used for offline diagnosis (e.g., BLE or Wi-Fi). Fi HTTPS).

In one embodiment, the electronic device 201 uses the mnId and setupId of the controlled device 320 to send setup data including a device image (e.g., external image) and a guide phrase of the controlled device 320 to the server ( 310), and a guide screen for guiding a communication connection (for example, D2D connection) between the electronic device 201 and the controlled device 320 can be configured based on the setup data. The D2D connection may refer to an operation of connecting between the electronic device 201 and the controlled device 320 according to a discovery method (for example, based on Soft-AP, BLE, or On IP network). In one embodiment, when using the Soft-AP method, the guide screen may guide the controlled device 320 to operate in Soft-AP mode by pressing a specific button or combination of buttons on the controlled device 320. In one embodiment, when using the BLE method, the guide screen may guide the controlled device 320 to broadcast advertising signals at regular intervals. BLE-based D2D connections may generally include GATT (generic attribute profile) connections.

In one embodiment, the electronic device 201 discovers the controlled device 320 through a BLE scan or Wi-Fi scan according to the protocolType included in the diagnostic performance information, and includes the device information of the controlled device 320. It can be determined whether the controlled device 320 is registered with the server 310 using the device identification information. When the controlled device 320 is registered with the server 310, the server 310 can manage the device information and status (for example, online/offline status) of the controlled device 320. In one embodiment, the device identification information may include a media access control (MAC) address. In one embodiment, the device information of the controlled device 320 is directly transferred from the controlled device 320 to the server 310 during the process of registering the controlled device 320 with the server 310, or the controlled device ( After 320 is registered with the server 310, it can be transmitted from the controlled device 320 to the server 310.

FIG. 8 is a diagram illustrating a procedure in which a controlled device uploads device information to a server according to an embodiment.

Referring to FIG. 8 , in operation 802, the electronic device 201 may transmit provisioning data to instruct the controlled device 320 to proceed with a registration procedure for the server 310. In one embodiment, provisioning data may include terminal information related to the electronic device 201 and information about a user account. In operation 804, the controlled device 320 may proceed with a registration procedure for the server 310. After or during the registration process, the controlled device 320 may transmit device information to the server 310 in operation 806. In one embodiment, the device information may include terminal information and user account information related to the electronic device 201, or may be transmitted to the server 310 together with terminal information and user account information.

In operation 808, the server 310 may store the device information. In one embodiment, the server 310 is a device backend server in charge of device onboarding, a metadata server that manages metadata information of the controlled device 320, or an inception ( inception) server. The controlled device 320 may transmit device information including device identification information to the device backend server during the registration process. The device backend server may transmit metadata information excluding device identification information to the metadata server, and the device identification information may be transmitted to the Inception server. In operation 810, the controlled device 320 may transmit a new device ID to the electronic device 201.

FIG. 9 is a diagram illustrating a procedure in which an electronic device uploads device information of a controlled device to a server, according to an embodiment.

Referring to FIG. 9, in operation 902, the electronic device 201 establishes a D2D connection (for example, a Bluetooth connection or Wi-Fi connection) with the controlled device 320, and controls the controlled device 320 through the D2D connection. ) device identification information (for example, MAC address) can be obtained. In operation 904, the controlled device 320 may register (for example, sign up/sign in) with the server 310. If the controlled device 320 does not have the ability to directly transmit device information to the server 310 (for example, if it contains an old version of software), in operation 906, the server 310 A notification indicating that registration is complete may be transmitted to the electronic device 201. In operation 908, the electronic device 201 may transmit device information (for example, including at least device identification information) of the controlled device 320 to the server 310 in response to the notification. In operation 910, the server 310 may store device information of the controlled device 320 in association with the user account of the electronic device 201. In operation 912, the server 310 may transmit a response message (eg, “200 OK”) to the electronic device 201 indicating that storage of device information has been completed.

Table 2 below shows an example of device identification information according to an embodiment.

Attributes Value mnId "some-mnid" setupId "some-setupId" modelCode "some-code" serialNumber "Some-serial" ssid "some-ssid" macAddressHash "wifi":"hash(some-address)", "ble":"hash(some-address)", or
"p2p":"hash(some-address)"

In one embodiment, the device identification information may include at least one of mnId, setupId, modelCode, serialNumber, service set identifier (SSID), or macAddressHash. In one embodiment, mnId may represent a manufacturer ID, setupId may represent an onboarding ID, modelCode may represent a device model of the controlled device 320, and serialNumber may represent the serial number of the controlled device 320. It can represent (serial number: SN). macAddressHash is at least one MAC address (e.g., Wi-Fi MAC address, It may include a hash value for at least one of a BLE MAC address or a point-to-point (P2P) MAC address.

In one embodiment, the electronic device 201 obtains the MAC from the controlled device 320 discovered through a BLE scan or Wi-Fi scan to determine whether the controlled device 320 is registered with the server 310. The hash value of the address is compared with the 'macAddressHash' of the device identification information, and if they are the same, it is determined that the controlled device 320 is registered with the server 310 and establishment of a D2D connection with the controlled device 320 is executed. You can.

Figure 10 shows a signal flow diagram (sequence diagram) to explain a procedure for executing direct connection diagnosis of a controlled device through a secure connection according to an embodiment. At least one of the operations described below may be omitted, modified, or changed in order.

Referring to FIG. 10, operation 1000 may refer to an operation in which the electronic device 201 establishes a communication connection (eg, D2D connection) with the controlled device 320. In one embodiment, operation 1000 may include at least one of operation 1002, operation 1004, operation 1006, operation 1008, operation 1010, operation 1012, operation 1014, operation 1016, operation 1018, or operation 1020.

In operation 1002, the electronic device 201 may request device information of the controlled device 320 from the server 310. In one embodiment, the electronic device 201 executes a client application and receives a list of controlled devices (for example, including the controlled device 320) and the controlled device 320 from the server 310 through the client application. Status information (e.g. online or offline) may be received, for example periodically or upon request. As the controlled device 320 is confirmed to be offline based on the status information, the electronic device 201 may execute operation 1002. In one embodiment, the electronic device 201 may execute operation 1002 based on receiving a user input (eg, user input 710) executing DCD through a client application. In one embodiment, the electronic device 201 may receive device information and status information of the controlled device 320 from the server 310 periodically or upon request.

In operation 1004, the electronic device 201 may receive device information including diagnostic performance information and/or device identification information from the server 310. In operation 1006, the electronic device 201 may determine whether the controlled device 320 supports DCD based on device information. In one embodiment, the electronic device 201 may execute operation 1006 when it recognizes that the controlled device 320 is offline based on status information received along with device information from the server 310. In one embodiment, the device information may include a DCD support field indicating whether the controlled device 320 supports DCD, and the electronic device 201 controls the controlled device 320 according to the value of the DCD support field. ) may be determined to support DCD, or it may be determined that the controlled device 320 supports DCD by identifying that the device information includes the DCD support field. In one embodiment, the diagnostic performance information may be configured as shown in <Table 1>, and the controlled device 320 supports DCD based on the electronic device 201 confirming that the device information includes diagnostic performance information. It can be judged that

In operation 1008, the electronic device 201 displays an object for DCD execution (e.g., a DCD execution object or a DCD button) and, if the controlled device 320 supports DCD, activates the D2D execution object (e.g. enable) can be done. If the controlled device 320 does not support DCD, the DCD execution object may be deactivated (eg, disabled). In one embodiment, the electronic device 201 displays a DCD execution object for the controlled device 320 if the controlled device 320 supports DCD, and if the controlled device 320 does not support DCD, the electronic device 201 displays a DCD execution object for the controlled device 320. The DCD execution object for the controlled device 320 may not be displayed. In operation 1010, the electronic device 201 may receive a user input (eg, touch) on the (activated or displayed) DCD execution object.

In operation 1012, the electronic device 201 may transmit a request message for setup data (eg, device image and guide text) required to configure the guide screen to the server 310. In one embodiment, the request message may include mnID and/or setupId obtained from device information of the controlled device 320. In operation 1014, the electronic device 201 may receive setup data from the server 310. In operation 1016, the electronic device 201 may display a guide screen (for example, the guide screen 1804 in FIG. 18B). In one embodiment, the guide screen may include a device image of the controlled device 320 and guide text. The guide text is intended to guide the user in D2D connection according to the device type of the controlled device 320, and includes, for example, "Turn on the device and press the button for more than 3 seconds until the AP appears." can do. In one embodiment, the guide screen may include an object (eg, an input button) for executing establishment of the D2D connection.

In operation 1018, the electronic device 201 may discover the controlled device 320 through device scanning. In one embodiment, the user can turn on the controlled device 320 according to the guide screen, and the controlled device 320 can prepare to be discovered by the electronic device 201 according to the protocol type. . In one embodiment, the controlled device 320 may broadcast a signal containing its MAC address (e.g., a BLE advertising signal or a Wi-Fi broadcast signal (i.e., a beacon signal)) at regular intervals. . In one embodiment, the electronic device 201 may discover the controlled device 320 among the devices in the scan list using the MAC address (eg, 'macAddressHash') included in the device identification information. In operation 1020, the electronic device 201 may establish a D2D connection (for example, a BLE connection or a Wi-Fi connection) with the controlled device 320.

In operation 1022, the electronic device 201 can confirm ownership of the controlled device 320. In one embodiment, the electronic device 201 confirms that the information (e.g., PIN) received from the controlled device 320 matches the information (e.g., PIN) entered by the user, thereby ) can be confirmed that it is actually owned by the user. The electronic device 201 may confirm ownership of the controlled device 320 and consider the D2D connection as a secure connection. In one embodiment, the electronic device 201 may confirm that it already has the credential information of the controlled device 320 and skip operation 1022. In one embodiment, the electronic device 201 is a member device invited by the owner device that performed registration of the controlled device 320, and may execute operation 1022 based on the credential information received from the owner device. In one embodiment, the electronic device 201 may request and receive the credential information from the owner device after operation 1010.

In operation 1024, the electronic device 201 may receive connection information including an offline error code from the controlled device 320 through the D2D connection. In one embodiment, the offline error code may indicate the cause of the controlled device 320 being offline, and may indicate, for example, at least one of a cloud error, a network error, a device error, or an unidentified error. In one embodiment, connection information may include at least one of the fields in <Table 3>.

In operation 1026, the electronic device 201 may receive device log data from the controlled device 320 through the D2D connection. The device log data may include the operation history of the controlled device 320 (for example, at least one of power on, function execution, status change, error occurrence, or power off). In one embodiment, the electronic device 201 requests the controlled device 320 to generate device log data, and receives the generated device log data (e.g., a dump file of log data) from the controlled device 320. can receive.

In operation 1028, the electronic device 201 may perform error handling for the controlled device 320. In one embodiment, error handling includes transmitting the offline error code and a recovery command corresponding to the device type of the controlled device 320 to the controlled device 320, and sending the controlled device 320 to the server 310. ) may include an operation to check the connection status. In one embodiment, the electronic device 201 may display an action guide to the user based on the offline error code and transmit the recovery command to the controlled device 320 in response to a user input. In one embodiment, the controlled device 320 may perform self-diagnosis in response to a recovery command received from the electronic device 201 and then attempt to connect to the server 310. The server 310 may confirm that it is connected to the controlled device 320 and transmit a message notifying that the controlled device 320 is connected to the electronic device 201. In one embodiment, in operation 1028, the electronic device 201 may display an action guide according to the value of the offline error code included in the connection information, and send a recovery command corresponding to the offline error code to the controlled device 320. Can be transmitted.

In operation 1030, the electronic device 201 may transmit an error reporting message indicating the result of the error handling to the server 310. In one embodiment, the error reporting message may include the offline error code and/or the device log data. In one embodiment, the error reporting message may be transmitted in response to user input to the electronic device 201. In one embodiment, the electronic device 201 may disconnect the D2D connection with the controlled device 320 after the error handling is completed.

<Table 3> below shows an example of connection information according to an embodiment.

Field Description mnmo Product identifier including model id network_type Wired/wireless Internet connection status ap_mac homeAP MAC OUI (organizationally unique identifier) ap_freq homeAP frequency band ap_protocol homeAP wifi protocol ap_rssi homeAP RSSI (receive signal strength indicator) error_code Error details online_status Cloud online/offline status of the controlled device at the time of connection information call ap_security homeAP security type disconnection_time Time elapsed since the controlled device went offline

In one embodiment, the connection information may include at least one of mnmo, network_type, ap_mac, ap_freq, ap_protocol, ap_rssi, error_code, online-status, ap_security, or disconnection_time. In one embodiment, error_code may include one of the values of an error code (eg, offline error code) for offline diagnosis of the controlled device 320.

Table 4 below shows an example of an offline error code according to an embodiment.

classification Main value cause Sub value detailed cause Cloud Error CE01 DNS failure with SmartThings Cloud address One Private IP response CE80 Failed to connect session with SmartThings Cloud 2 CE20 Sign-in failed One send fail 2 failure response value 3 timeout CE82 Sign-out failed One send fail 2 failure response value 3 timeout CE83 Keep alive resource discovery failed One send fail 2 failure response value 3 timeout CE84 Keep alive request failed One send fail 2 failure response value 3 timeout CE60 Access token refresh failed One send fail 2 failure response 3 timeout CE90 Reset device CE70 Rate Limit Network Error NE11 Failed to connect to the router One AP not found 2 Authentication failed 3 Association failed 4 password mismatch
(same as EAPOL (EAP encapsulation over LAN) failure) 5 OPEN-password exists 6 Wi-Fi module detached NE30 Router disconnected One beacon loss 2 De-authenticated NE12 Router DHCP failure One discovery + request timeout NE50 Failed to connect to Ethernet router One ethernet cable detach 2 dhcp failed 3 ethernet router check failed (arping) Device-specific Error DS01 Long term unused One Long term unused DS02 Power-on not supported One Power On not supported 2 “Power on via mobile” setting Off 3 When an old TV is connected to a wired LAN DS03 Device error One Wakeup timer error 2 cold power off 3 SES Disconnection Unknown Error UE01 or Unknown Error / Default Error code Unknown Error

Figure 11 is a signal flow diagram illustrating a procedure for confirming ownership of a controlled device according to an embodiment. In embodiments, at least one of the operations described below may be omitted, modified, or changed in order.

Referring to FIG. 11 , in operation 1022 (eg, operation 1022 of FIG. 10 ), the electronic device 201 may confirm ownership of the controlled device 320 . In one embodiment, operation 1022 may include at least one of operation 1102, operation 1104, operation 1106, or operation 1108. In one embodiment, the electronic device 201 may confirm ownership through operation 1022 to receive connection information and device log data through a D2D connection.

In operation 1102, the electronic device 201 may transmit an ownership transfer method (OTM) request message to the controlled device through a D2D connection (for example, the D2D connection established in operation 1020 of FIG. 10). OTM is a procedure to prove whether the controlled device 320 is actually owned by the user, for example, user confirmation, QR (quick response) confirmation, random PIN, serial number confirmation, or ultrasonic (Ultrasound) confirmation. ) At least one of the following confirmations can be used. In one embodiment, when the random PIN method is used, in operation 1104, the controlled device 320 may transmit an OTM response message including a randomly generated PIN (for example, a first PIN) to the electronic device 201. there is. In one embodiment, the controlled device 320 may display the first PIN on its display module (not shown) so that the user can check it.

In operation 1106, the electronic device 201 may display an input screen on the display module 260 and receive a second PIN for confirming ownership of the controlled device 320 from the user through the input screen. The electronic device 201 may compare the first PIN received through the OTM response message with the second PIN and, if they match, determine that the controlled device 320 is actually owned by the user. In operation 1108, the electronic device 201 may complete ownership verification by sharing credential information with the controlled device 320 through an OTM success message.

In one embodiment, during the process of registering the controlled device 320 with the server 310, the electronic device 201 and the controlled device 320 may each store credential information. The credential information may be transmitted to the controlled device 320 through the OTM request message in operation 1102, and the controlled device 320 confirms that the credential information is the same as the credential information it has, and In operation 1104, an OTM response message may be transmitted. In one embodiment, if credential information is used, user verification may be omitted and the OTM response message may not include a PIN. The electronic device 201 may skip the PIN check in operation 1106 and determine that ownership of the controlled device 320 has been confirmed based on reception of the OTM response message.

In one embodiment, the electronic device 201 may be an owner device that has performed a procedure to register the controlled device 320 with the server 310, and verifies ownership in operation 1022 using the credential information generated during the registration procedure. You can run . In one embodiment, the electronic device 201 may be a member device invited by or permitted by the owner device, and may receive credential information from the owner device and perform ownership verification in operation 1022 using the credential information. You can. In one embodiment, the controlled device 320 may store credential information for the owner device and at least one member device, respectively. In one embodiment, the electronic device 201 may be a member device and may perform ownership verification in operation 1022 using a PIN entered by the user. In one embodiment, if the electronic device 201 has already confirmed ownership of the controlled device 320, operation 1022 may be omitted.

In one embodiment, when ownership verification is completed through operation 1022, the electronic device 201 may store the device ID of the controlled device 320, and the controlled device 320 may store the terminal ID (mobile) of the electronic device 201. ID) can be saved. In one embodiment, the electronic device 201 checks the device ID of the controlled device 320 while establishing the D2D connection in operation 1020 and, if the device ID of the controlled device 320 is already stored, performs operation 1022. You may decide to omit it.

FIG. 12 is a signal flow diagram illustrating a procedure for receiving connection information and device log data according to an embodiment. In embodiments, at least one of the operations described below may be omitted, modified, or changed in order.

Referring to FIG. 12, in operation 1024 (e.g., operation 1024 of FIG. 10), the electronic device 201 establishes a D2D connection (e.g., a D2D connection established in operation 1020 of FIG. 10) (e.g., a BLE connection). ), connection information can be received from the controlled device 320 and proceed to operation 1026. In one embodiment, operation 1024 may include at least one of operation 1202, operation 1204, operation 1206, operation 1208, or operation 1210.

In operation 1202, the electronic device 201 may transmit a Discovery resource message to inquire whether the controlled device 320 has connection information. In operation 1204, the controlled device 320 may transmit a Found resource message indicating that it has connection information to the electronic device 201. In operation 1206, the electronic device 201 may transmit a Get resource message requesting connection information to the controlled device 320. In operation 1208, the controlled device 320 may transmit a Response message including the connection information to the electronic device 201. In operation 1210, the electronic device 201 may generate log data including the connection information. In one embodiment, the log data may include a mobile log of the electronic device 201.

In operation 1026 (e.g., operation 1026 of FIG. 10), the electronic device 201 is connected to the controlled device 320 via a D2D connection (e.g., the D2D connection established in operation 1020) (e.g., a BLE connection). Device log data can be received. In one embodiment, operation 1026 may include at least one of operation 1212, operation 1214, operation 1216, operation 1218, operation 1220, or operation 1022.

In operation 1212, the electronic device 201 may transmit a Discovery resource message to inquire whether the controlled device 320 has device log data. In operation 1214, the controlled device 320 may transmit a Found resource message indicating that it has device log data to the electronic device 201. In operation 1216, the electronic device 201 may transmit a Post resource message requesting the controlled device 320 to generate device log data (for example, a dump file of the device log) including a time stamp. In operation 1218, the controlled device 320 may generate the device log data and transmit a Response Ok message to the electronic device 201.

In operation 1220, the electronic device 201 may transmit a Get resource message requesting the device log data to the controlled device 320. In operation 1222, the controlled device 320 may transmit a Response message including the device log data to the electronic device 201.

Figure 13 is a signal flow diagram to explain error handling and error reporting according to an embodiment. In embodiments, at least one of the operations described below may be omitted, modified, or changed in order.

Referring to FIG. 13, in operation 1028 (e.g., operation 1028 of FIG. 10), the electronic device 201 controls the controlled device 320 according to the offline error code obtained in operation 1024 (e.g., operation 1024 of FIG. 10). ) offline errors can be handled. In one embodiment, operation 1028 may include at least one of operation 1302, operation 1304, operation 1306, operation 1306a, operation 1308, or operation 1310.

In operation 1302, the electronic device 201 displays an action guide screen (e.g., screen 1904 in FIG. 19B, screen 2002 in FIG. 20, or screen 2102 in FIG. 21) including an action guide corresponding to the offline error code. )) can be displayed. In one embodiment, the action guide screen may display, for example, “Wi-Fi password is incorrect,” “Wi-Fi router not found,” or “Turn off the device,” depending on the offline error code illustrated in Table 4. It can include guide phrases such as “Please turn it on again.” In one embodiment, the action guide screen is an object instructing to transmit a recovery command to the controlled device 320 (e.g., an input button for “Continue on TV” in FIG. 20 or an input button for “Reboot” in FIG. 21). may include.

Based on receiving a user input for the input button on the action guide screen, in operation 1304, the electronic device 201 avoids sending a message (e.g., “Request command”) containing a recovery command corresponding to the offline error code. It can be transmitted to the control device 320. The recovery command may instruct a specific function to be performed in the controlled device 320 to restore the offline state, for example, Wi-Fi reconnection or rebooting. In operation 1306, the controlled device 320 may execute a function (eg, Wi-Fi reconnection or reboot) corresponding to the recovery command. In one embodiment, when the controlled device 320 has a large display screen, such as a TV, the recovery command may indicate “Continue on TV,” and in operation 1306, the controlled device 320 responds to the recovery command. By responding and linking with the user, error handling such as connection checking can be performed directly. In operation 1306a, the controlled device 320 may reconnect with the server 310 after resolving the offline error through the error handling. If the controlled device 320 is normally connected to the server 310, the server 310 may update the status of the controlled device 320 to connected ('Connected') (or online).

In operation 1308, after transmitting the recovery command, the electronic device 201 may transmit a message requesting the device status (for example, “Request device status”) to the server 310. In operation 1310, the server 310 determines whether the controlled device 320 is connected to the server 310 and sends a status message including the result (e.g., Connected or Not Connected). (For example, “Response (Connected)”) may be transmitted to the electronic device 201.

In operation 1030 (eg, operation 1030 of FIG. 10 ), the electronic device 201 may report the result of the error handling of operation 1028 to the server 310 . In one embodiment, operation 1030 may include at least one of operation 1312, operation 1314, operation 1316, or operation 1318.

In operation 1312, the electronic device 201 may disconnect from the controlled device 320 as it confirms that the controlled device 320 is connected to the server 310 through the status message of operation 1310. In operation 1314, the electronic device 201 may display an error reporting guide screen to guide error reporting. In one embodiment, the error reporting guide screen may include an object (for example, an error reporting button 1906) that requests the electronic device 201 to transmit an error report to the server 310. In operation 1316, the electronic device 201 may receive a user input through the error reporting button on the error reporting guide screen. In operation 1318, the electronic device 201 may transmit an error reporting message (eg, “Send log data”) to the server 310 in response to the user input. In one embodiment, the error reporting message includes connection information obtained in operation 1024 (e.g., operation 1024 of FIG. 10), device log data obtained in operation 1026 (e.g., operation 1026 of FIG. 10), or operation 1028. It may include at least one piece of information related to error handling results. Server 310 may store the connection information and the device log data.

FIG. 14 is a signal flow diagram illustrating a procedure for executing direct connection diagnosis of a controlled device through a non-secure connection according to an embodiment. In embodiments, at least one of the operations described below may be omitted, modified, or changed in order.

Referring to FIG. 14, operation 1400 may refer to an operation in which the electronic device 201 establishes a communication connection (eg, D2D connection) with the controlled device 320. In one embodiment, operation 1400 may include at least one of operation 1002, operation 1004, operation 1006, operation 1008, operation 1010, operation 1012, operation 1014, operation 1016, operation 1018, or operation 1020 shown in FIG. 10. . In one embodiment, the electronic device 201 may not be able to perform ownership verification because it does not have credential information for the controlled device 320. In one embodiment, the electronic device 201 does not support multi ownership, and because credential information related to existing ownership verification has been reset in the server 310, the electronic device 201 may not perform ownership verification and proceed to operation 1402. .

In operation 1402, the electronic device 201 may receive connection information including an offline error code from the controlled device 320 through the D2D connection established in operation 1400. In one embodiment, connection information may include at least one of the fields in <Table 3>. In one embodiment, the connection information may not include ap_mac because the D2D connection is considered an insecure connection for which ownership has not been confirmed. In one embodiment, the offline error code may include one of the error code values in <Table 4>.

In operation 1404, the electronic device 201 may receive device log data from the controlled device 320 through the D2D connection. In one embodiment, the electronic device 201 requests the controlled device 320 to generate device log data, and receives the generated device log data (e.g., a dump file of log data) from the controlled device 320. can receive.

In operation 1406, the electronic device 201 may perform error handling for the controlled device 320. In one embodiment, error handling includes transmitting the offline error code and a recovery command corresponding to the device type of the controlled device 320 to the controlled device 320, and sending the controlled device 320 to the server 310. ) may include an operation to check the connection status. In one embodiment, the electronic device 201 may display an action guide to the user based on the offline error code and transmit the recovery command to the controlled device 320 in response to a user input.

In one embodiment, because the D2D connection established in operation 1400 does not verify ownership and may be considered an insecure connection, in operation 1406 the controlled device 320 performs user confirmation in response to a recovery command from the electronic device 201. It can be run. In one embodiment, the controlled device 320 may display a pop-up screen for user confirmation before executing the function corresponding to the recovery command, and may receive a user input approving execution of the function through the pop-up screen. Once received, the above function can be executed. In one embodiment, operation 1406 may be the same or similar to operation 1028 of FIG. 13 .

In operation 1408, the electronic device 201 may transmit an error reporting message indicating the result of the error handling to the server 310. In one embodiment, the error reporting message may include the offline error code and/or the device log data. In one embodiment, the error reporting message may be transmitted in response to a user input to an error reporting button (eg, error reporting button 1906) displayed on the electronic device 201. In one embodiment, the electronic device 201 may terminate the D2D connection with the controlled device 320 after the error handling is completed. In one embodiment, operation 1408 may be the same or similar to operation 1030 of FIG. 13 .

Figure 15 shows a signal flow diagram to explain a procedure for executing direct connection diagnosis of a controlled device using soft AP mode according to an embodiment. In embodiments, at least one of the operations described below may be omitted, modified, or changed in order.

Referring to FIG. 15, operation 1500 may refer to an operation of the electronic device 201 establishing a communication connection (eg, D2D connection) with the controlled device 320. In one embodiment, operation 1400 includes at least one of operation 1002, operation 1004, operation 1006, operation 1008, operation 1010, operation 1012, operation 1014, or operation 1016, and operations 1502, 1504, and 1506 shown in FIG. 10. Alternatively, it may include at least one of operation 1508. In one embodiment, the guide screen of operation 1016 may include guide text explaining how to change the controlled device 320 to the soft AP mode. According to the above guide text, the user can set the controlled device 320 to soft AP mode through manual operation.

In operation 1502, the controlled device 320 may enter the soft AP mode and broadcast a Wi-Fi broadcast signal (for example, a beacon signal) at regular intervals. In one embodiment, the controlled device 320 may start operating as an HTTP server to deliver device log data.

In operation 1504, the electronic device 201 may discover the controlled device 320 through device scanning. In one embodiment, the electronic device 201 includes identification information (e.g., MAC address and/or serial number) obtained from a beacon signal broadcast from the controlled device 320 in the device identification information obtained in operation 1004. The controlled device 320 can be targeted by checking whether it matches the identification information. In operation 1506, the electronic device 201 may establish a D2D connection (eg, Wi-Fi connection) with the controlled device 320. In operation 1508, the electronic device 201 may establish a secure session with a certificate based on the D2D connection with the controlled device 320.

In operation 1510, the electronic device 201 may receive connection information including an offline error code from the controlled device 320 through the D2D connection established in operation 1500. In one embodiment, connection information may include at least one of the fields in <Table 3>. In one embodiment, the offline error code may include one of the error code values in <Table 4>.

In operation 1512, the electronic device 201 may receive device log data from the controlled device 320 through the D2D connection. In one embodiment, the electronic device 201 requests the controlled device 320 to generate device log data, and receives the generated device log data (e.g., a dump file of log data) from the controlled device 320. can receive.

In operation 1514, the electronic device 201 may perform error handling for the controlled device 320. In one embodiment, operation 1514 may be the same or similar to operation 1028 of FIG. 13 . In operation 1516, the electronic device 201 may transmit an error reporting message indicating the result of the error handling to the server 310. In one embodiment, operation 1516 may be the same or similar to operation 1030 of FIG. 13 .

FIG. 16 is a signal flow diagram illustrating a procedure for receiving connection information and device log data according to an embodiment. In embodiments, at least one of the operations described below may be omitted, modified, or changed in order.

Referring to FIG. 16, in operation 1510, the electronic device 201 connects the controlled device through a D2D connection (e.g., the D2D connection established in operation 1020 of FIG. 10) (e.g., an HTTP security session on a Wi-Fi connection). Connection information can be received from (320). In one embodiment, operation 1510 may include at least one of operation 1602, operation 1604, or operation 1606.

In operation 1602, the electronic device 201 may transmit a request message (eg, “Request (connectioninfo)”) requesting connection information to the controlled device 320. In one embodiment, the electronic device 201 may include an API (application programming interface) key and/or a device ID in the HTTPS header of the request message for permission check. The controlled device 320 may compare the API key and/or device ID with the device information it stores, and if they are the same, proceed to operation 1604.

In operation 1604, the controlled device 320 may transmit a response message (eg, “Response (connectioninfo)”) including connection information to the electronic device 201. In operation 1606, the electronic device 201 may generate log data including the connection information. In one embodiment, the log data may include a mobile log of the electronic device 201.

In operation 1512, the electronic device 201 receives a device from the controlled device 320 through the D2D connection (e.g., HTTP security session over a Wi-Fi connection) established in operation 1500 (e.g., operation 1500 of FIG. 15). Log data can be received. In one embodiment, operation 1512 may include at least one of operation 1608, operation 1610, operation 1612, or operation 1614.

In operation 1608, the electronic device 201 may transmit a request message (for example, “Request create log (devicelog.dump)”) to request the controlled device 320 to create device log data. In operation 1610, the controlled device 320 may generate device log data and transmit a response message (eg, “Response ok”) to the electronic device 201. In operation 1612, the electronic device 201 sends a request message (e.g., “Request get log (devicelog. dump)") can be sent. In operation 1614, the controlled device 320 may transmit a response message (eg, “Response (devicelog.dump)”) including the device log data to the electronic device 201. In one embodiment, the request message of operations 1608 and/or 1612 may include an API key and/or a device ID known to the electronic device 201, and the controlled device 320 may use the API key and/or the device. The permission of the electronic device 201 may be checked based on the ID, and if the permission check passes, the response message in operation 1610 and/or operation 1614 may be transmitted.

In one embodiment, the recovery command transmitted by the electronic device 201 to the controlled device 320 in at least one of operation 1028, operation 1406, or operation 1514 may include a function for performing an action corresponding to an offline error code. You can. In one embodiment, the recovery command may indicate restart of the controlled device 320.

In one embodiment, when the offline error code is NE11-1, NE11-4, or NE11-5 in <Table 4>, the electronic device 201 avoids the Wi-Fi scan list to which the controlled device 320 can connect. A request may be made to the control device 320 and a Wi-Fi scan list including at least one Wi-Fi network may be received from the controlled device 320. The Wi-Fi scan list includes at least one of supported frequency ('supported frequency') (e.g., at least one of 2.4G, 5G, or 6G), supported authentication type ('supported Auth Type'), or Wi-Fi scan information. may include. In one embodiment, the Wi-Fi scan information may include at least one of SSID, authentication type, encryption type, MAC address, frequency, or RSSI.

In one embodiment, the electronic device 320 displays the Wi-Fi scan list, and when the user selects at least one AP from the Wi-Fi scan list, the electronic device 320 sends Wi-Fi information related to the selected AP to the recovery command. It can be transmitted to the controlled device 320. The Wi-Fi information may include, for example, at least one of SSID, password, authentication type, or encryption type. In one embodiment, the recovery command may direct Wi-Fi update using the Wi-Fi information.

An electronic device for managing a controlled device and its operating method according to embodiments of the present disclosure provide the user with the cause of the offline controlled device through direct connection diagnosis of the controlled device that is offline and enable the user to take self-action. Therefore, a solution suitable for the error situation of the controlled device can be provided.

Figure 17 is a diagram illustrating a user interface screen for executing general diagnosis according to one embodiment.

Referring to FIG. 17, when the controlled device 320 does not support direct connection diagnosis, the electronic device 201 executes a general diagnosis procedure (for example, operation 720 of FIG. 7) while displaying a general diagnosis execution screen 1702. can be displayed on the display module 260. Through a general diagnostic procedure, the electronic device 201 may check at least one of the software version of the client application, cloud status, device information, or device operation status.

FIGS. 18A, 18B, and 18C are diagrams illustrating user interface screens for executing direct connection diagnosis according to an embodiment.

Referring to FIG. 18A, when the controlled device 320 supports direct connection diagnosis, the electronic device 201 displays the screen 1802 while executing a direct connection diagnosis procedure (for example, operation 730 of FIG. 7). It can be displayed at (260). In one embodiment, the electronic device 201 executes a general diagnosis (e.g., operation 732 of FIG. 7) included in the direct connection diagnosis procedure 730, and if no problem is found as a result of executing the general diagnosis 732, In this case, the screen 1802 may be displayed. For example, the screen 1802 may include a phrase indicating that no problems were found as a result of executing the general diagnosis 732, as well as a phrase guiding connection to the controlled device 320 for direct connection diagnosis. there is. In one embodiment, the first screen 1802 may be displayed by the electronic device 201 after operation 1010.

Referring to FIG. 18B , the electronic device 201 may display a guide screen 1804 generated based on setup data received from the server 310 on the display module 260. In one embodiment, the guide screen 1804 may include a device image of the controlled device 320 and guide phrases (e.g., “Turn on the device and press the button for more than 3 seconds until the AP appears”). there is. In one embodiment, the guide screen 1804 may be displayed by the electronic device 201 in operation 1016.

Referring to FIG. 18C, the electronic device 201 may receive a user input requesting connection to the controlled device 320 through the guide screen 1804 and display the screen 1806 on the display module 260. there is. In one embodiment, the screen 1806 may include a phrase indicating that a connection is being made with the controlled device 320. In one embodiment, screen 1806 may be displayed during operation 1020 or operation 1506.

19A and 19B are diagrams illustrating screens for performing error handling and error reporting according to various embodiments.

Referring to FIG. 19A, the electronic device 201 displays the screen 1902 on the display module 260 while analyzing the cause of the offline state based on the connection information and device log data received from the controlled device 320. You can. In one embodiment, screen 1902 may include text announcing diagnostic data collection, device information analysis, and/or failure assessment.

Referring to FIG. 19B, the electronic device 201 may display a screen 1904 including an action guide generated according to the results of analyzing the cause of the offline state of the controlled device 320 on the display module 260. there is. In one embodiment, the screen 1904 may include text indicating a test result for at least one of the software version of the client application, cloud status, device information, or device operating status. In one embodiment, if a fault is not found, screen 1904 may not provide action guidance. In one embodiment, when a failure is detected, the screen 1904 may include an object 1906 (for example, an error report button) for sending an error report. In one embodiment, screen 1904 may be displayed in operation 1302.

FIG. 20 is a diagram illustrating screens for executing error handling through a non-secure connection according to various embodiments.

Referring to FIG. 20, the electronic device 201 displays an action guide screen 2002 on the display module 260, including an action guide generated according to the results of analyzing the cause of the offline state of the controlled device 320. can do. In one embodiment, the action guide screen 2002 includes a phrase indicating the cause of the failure (e.g., “Wi-Fi password is incorrect”) and an action guide (e.g., “Run a connection test on the device and enter the correct password.”) “Please do this” and/or “Click the button below to further troubleshoot your device”). In one embodiment, the controlled device 320 may be a TV, and the action guide screen 2002 includes a button (e.g., “Continue on TV”) to confirm that the controlled device 320 will continue error handling. can do. In one embodiment, action guide screen 2002 may be displayed in operation 1302.

In one embodiment, the electronic device 201 may receive a user input through the button and transmit a recovery command instructing the controlled device 320 (eg, TV) to perform additional error handling operations. The controlled device 320 may display a screen 2004 for a connection test in response to the recovery command. The connection test screen 2004 may include, for example, a button for executing a connection test, and when a user input is received from the button, the controlled device 320 may execute a hub connection test according to its own function. For example, the controlled device 320 may display a screen 2006 based on a hub connection test. In one embodiment, the controlled device 320 may transmit the results of the hub connection test to the electronic device 201.

In one embodiment, the electronic device 201 may display a result screen 2008 on the display module 260 based on receiving the result of the hub connection test. In one embodiment, the result screen 2008 may include a phrase notifying that the controlled device 320 is online. In one embodiment, the result screen 2008 may include an error reporting button instructing to transmit an error report related to the controlled device 320, and in response to receiving a user input from the error reporting button, the electronic device (201) may transmit an error reporting message including the result of error handling of the controlled device (320) to the server (310). In one embodiment, the results screen 2008 may be displayed in operation 1314.

FIG. 21 is a diagram illustrating screens for executing error handling through a secure connection according to various embodiments.

Referring to FIG. 21, the electronic device 201 displays an action guide screen 2102 including an action guide generated according to the results of analyzing the cause of the offline state of the controlled device 320 on the display module 260. can do. In one embodiment, the action guide screen 2102 displays a guide phrase (e.g., "Please check if the router is turned on) according to the results of the electronic device 201 analyzing the offline state of the controlled device 320 according to the offline error code. ", "Please check the network name of the router", "If the device supports only 2.4GHz Wi-Fi networks, connect to a 2.4GHz network", "If the SSID has changed, delete the device and reconnect", "Device “Please remove any other wireless devices that are interfering with the Wi-Fi connection,” “Turn off the router and turn it back on, and then try to connect,” or “Update your router’s firmware to the latest.”) In one embodiment, the action guide screen 2102 may include a button 2104 (eg, “Reboot”) that instructs to transmit a recovery command to the controlled device 320. In response to receiving a user input through the button 2104, the electronic device 201 may transmit a recovery command (eg, reboot command) to the controlled device 320.

In one embodiment, the electronic device 201 may transmit a recovery command and display the screen 2104 on the display module 260. Screen 2104 may include text indicating that a connection is being attempted to the device. In one embodiment, screen 2104 may be displayed in at least one of operation 1304, operation 1308, or operation 1310.

In one embodiment, the electronic device 201 receives a message notifying that the controlled device 320 is connected to the server 310 (e.g., a status message of operation 1310) from the server 310 and displays a result screen 2106. It can be displayed on the display module 260. The result screen 2106 may include a phrase indicating that the controlled device 320 has come online. In one embodiment, the result screen 2106 may include an error reporting button instructing to transmit an error report related to the controlled device 320, and in response to receiving a user input from the error reporting button, the electronic device (201) may transmit an error reporting message including the result of error handling of the controlled device (320) to the server (310). In one embodiment, the results screen 2106 may be displayed in operation 1314.

FIG. 22 is a diagram illustrating screens that guide network reselection through error handling according to various embodiments.

Referring to FIG. 22, the electronic device 201 displays an action guide screen 2202 on the display module 260, including an action guide generated according to the results of analyzing the cause of the offline state of the controlled device 320. can do. In one embodiment, the action guide screen 2202 displays a guide phrase (e.g., "Wi-Fi set on the device It may include a button 2204 (e.g., “Connect to Wi-Fi”) that instructs Wi-Fi reconnection along with “the password is incorrect.” In response to receiving a user input through the button 2204, the electronic device 201 may request a Wi-Fi scan list from the controlled device 320.

In one embodiment, the electronic device 201 may display a screen 2206 including a Wi-Fi scan list received from the controlled device 320 on the display module 260. In one embodiment, the screen 2206 may include the name of at least one network (eg, Wi-Fi AP) to which the controlled device 320 can connect. When one network is selected on the screen 2206, the electronic device 201 transmits a recovery command containing Wi-Fi information related to the selected network to the controlled device 320 and displays the screen 2208 on the display module ( 260). Screen 2208 may include text indicating that a connection is being attempted to the device. In one embodiment, screen 2208 may be displayed in at least one of operation 1304, operation 1308, or operation 1310.

In one embodiment, the electronic device 201 receives a message notifying that the controlled device 320 is connected to the server 310 (for example, a status message of operation 1310) from the server 310 and displays the result screen 2210. It can be displayed on the display module 260. The result screen 2210 may include a phrase indicating that the controlled device 320 is online. In one embodiment, the result screen 2210 may include an error reporting button instructing to transmit an error report related to the controlled device 320, and in response to receiving a user input from the error reporting button, the electronic device (201) may transmit an error reporting message including the result of error handling of the controlled device (320) to the server (310). In one embodiment, the results screen 2210 may be displayed in operation 1314.

The electronic device 201 according to one embodiment may include a communication circuit 290 and at least one processor 220 operatively connected to the communication circuit. The at least one processor may be configured to receive device information of the external electronic device 320 from the server 310. The at least one processor may be configured to display a guide screen that guides communication connection with the external electronic device based on the device information. The at least one processor may be configured to establish the communication connection with the external electronic device through the communication circuit based on a user input on the guide screen. The at least one processor may be configured to receive connection information and device log data including an error code related to an offline state of the external electronic device from the external electronic device through the communication connection. The at least one processor may be configured to transmit a recovery command to the external electronic device through the communication connection based on the error code and the device log data. The at least one processor may be configured to receive a message from the server notifying that the external electronic device is connected to the server.

In one embodiment, the guide screen may include at least one of a phrase guiding the communication connection or an object for establishing the communication connection.

In one embodiment, the at least one processor may be configured to display an action guide screen related to the offline state processed based on the error code and transmit the recovery command based on a user input to the action guide screen. You can.

In one embodiment, the at least one processor may be configured to release the communication connection with the external electronic device based on receiving the message.

In one embodiment, the at least one processor transmits a request message to confirm ownership to the external electronic device through the communication connection, and includes a first personal identification number (PIN) from the external electronic device. It may be configured to receive a response message, receive a second PIN from the user, and complete ownership verification of the external electronic device when the first PIN and the second PIN match.

In one embodiment, the at least one processor may be configured to confirm ownership of the external electronic device using credential information obtained in the process of registering the external electronic device with the server. In one embodiment, the at least one processor may be configured to confirm ownership of the external electronic device using credential information received from the owner device of the external electronic device.

In one embodiment, the at least one processor determines that the device information includes information indicating that the external electronic device supports direct connection diagnosis, and activates an object that executes the direct connection diagnosis in the guide screen. and may be configured to establish the communication connection based on receiving the user input through an object executing the direct connection diagnosis.

In one embodiment, the recovery command may include at least one of a reboot command for the external electronic device or Wi-Fi information about a Wi-Fi network to which the external electronic device can connect.

In one embodiment, the at least one processor, when the offline error code indicates a network error, sends a Wi-Fi scan list to the external electronic device, including at least one Wi-Fi network to which the external electronic device can connect. Request a, receive the Wi-Fi scan list from the external electronic device, select a first Wi-Fi network from the Wi-Fi scan list based on user input, and connect to the selected first Wi-Fi network. It may be configured to transmit the recovery command including Wi-Fi information to the external electronic device.

In one embodiment, the at least one processor may be configured to transmit an error reporting message including the error code to the server based on receiving the message.

A method of operating the electronic device 201 according to an embodiment may include an operation 605 of receiving device information of the external electronic device 320 from the server 310. The method may include an operation 615 of displaying a guide screen guiding a communication connection with the external electronic device based on the device information. The method may include an operation 620 of establishing the communication connection with the external electronic device through the communication circuit based on a user input on the guide screen. The method may include operations 630 and 635 of receiving connection information and device log data including an error code related to the offline state of the external electronic device from the external electronic device through the communication connection. The method may include an operation 640 of transmitting a recovery command to the external electronic device through the communication connection based on the error code and the device log data. The method may include an operation 640 of receiving a message from the server indicating that the external electronic device is connected to the server.

In one embodiment, the guide screen may include at least one of a phrase guiding the communication connection or an object for establishing the communication connection.

In one embodiment, the method includes displaying an action guide screen related to the offline state processed based on the error code, and transmitting the recovery command based on the user input to the action guide screen. More may be included.

In one embodiment, the method may further include releasing the communication connection with the external electronic device based on receiving the message.

In one embodiment, the method includes transmitting a request message to confirm ownership to the external electronic device through the communication connection, and receiving a first personal identification number (PIN) from the external electronic device. The method may further include receiving a response message, receiving a second PIN from the user, and completing ownership verification of the external electronic device when the first PIN and the second PIN match.

In one embodiment, the method may further include confirming ownership of the external electronic device using credential information obtained in the process of registering the external electronic device with the server. In one embodiment, the method may further include confirming ownership of the external electronic device using credential information received from the owner device of the external electronic device.

In one embodiment, the operation of establishing the communication connection includes confirming that the device information includes information indicating that the external electronic device supports direct connection diagnosis, and executing the direct connection diagnosis in the guide screen. activating an object that performs the direct connection diagnosis, and establishing the communication connection based on receiving the user input through the object that performs the direct connection diagnosis.

In one embodiment, the recovery command may include at least one of a reboot command for the external electronic device or Wi-Fi information about a Wi-Fi network to which the external electronic device can connect.

In one embodiment, the method includes requesting, from the external electronic device, a Wi-Fi scan list including at least one Wi-Fi network to which the external electronic device can connect when the error code indicates a network error. An operation of receiving the Wi-Fi scan list from the external electronic device, an operation of selecting a first Wi-Fi network from the Wi-Fi scan list based on a user input, and the selected first Wi-Fi The method may further include transmitting the recovery command including Wi-Fi information about the network to the external electronic device.

In one embodiment, the method may further include transmitting an error reporting message including the error code to the server based on receiving the message.

Electronic devices according to various embodiments disclosed in this document may be of various types. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of this document are not limited to the above-described devices.

The various embodiments of this document and the terms used herein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various changes, equivalents, or replacements of the embodiments. In connection with the description of the drawings, similar reference numbers may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the above items, unless the relevant context clearly indicates otherwise. As used herein, “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “A Each of phrases such as “at least one of , B, or C” may include any one of the items listed together in the corresponding phrase, or any possible combination thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish one component from another, and to refer to that component in other respects (e.g., importance or order) is not limited. One (e.g., first) component is said to be “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicatively.” When mentioned, it means that any of the components can be connected to the other components directly (e.g. wired), wirelessly, or through a third component.

The term “module” used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as logic, logic block, component, or circuit, for example. It can be used as A module may be an integrated part or a minimum unit of the parts or a part thereof that performs one or more functions. For example, according to one embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document are one or more instructions stored in a storage medium (e.g., built-in memory 236 or external memory 238) that can be read by a machine (e.g., electronic device 201). It may be implemented as software (e.g., program 240) including these. For example, a processor (e.g., processor 220) of a device (e.g., electronic device 201) may call at least one command among one or more commands stored from a storage medium and execute it. This allows the device to be operated to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves), and this term refers to cases where data is semi-permanently stored in the storage medium. There is no distinction between temporary storage cases.

According to one embodiment, methods according to various embodiments disclosed in this document may be provided and included in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or through an application store (e.g. Play StoreTM) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smart phones) or online. In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

According to various embodiments, each component (e.g., module or program) of the above-described components may include a single or plural entity, and some of the plurality of entities may be separately placed in other components. there is. According to various embodiments, one or more of the components or operations described above may be omitted, or one or more other components or operations may be added. Alternatively or additionally, multiple components (eg, modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each component of the plurality of components in the same or similar manner as those performed by the corresponding component of the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, or omitted. Alternatively, one or more other operations may be added.

Claims (20)

In the electronic device 201,
communication circuit 290; and
At least one processor (220) operatively connected to the communication circuit, wherein the at least one processor comprises:
Receive device information of the external electronic device 320 from the server 310,
Displaying a guide screen to guide communication connection with the external electronic device based on the device information,
Establishing the communication connection with the external electronic device through the communication circuit based on user input on the guide screen,
Receiving connection information and device log data including an error code related to an offline state of the external electronic device from the external electronic device through the communication connection,
Transmitting a recovery command to the external electronic device through the communication connection based on the error code and the device log data,
An electronic device configured to receive a message from the server indicating that the external electronic device is connected to the server. The method of claim 1, wherein the guide screen is:
An electronic device comprising at least one of a phrase guiding the communication connection or an object for establishing the communication connection. The method of claim 1 or 2, wherein the at least one processor:
Displaying an action guide screen related to the offline status processed based on the error code,
An electronic device configured to transmit the recovery command based on a user input to the action guide screen. The method of any one of claims 1 to 3, wherein the at least one processor:
An electronic device, configured to release the communication connection with the external electronic device based on receiving the message. The method of any one of claims 1 to 4, wherein the at least one processor:
Transmitting a request message to confirm ownership to the external electronic device through the communication connection,
Receive a response message including a first personal identification number (PIN) from the external electronic device,
Receive a second PIN from the user,
An electronic device configured to complete ownership verification of the external electronic device when the first PIN and the second PIN match. The method of any one of claims 1 to 5, wherein the at least one processor:
Verify ownership of the external electronic device using credential information obtained in the process of registering the external electronic device with the server, or
An electronic device configured to confirm ownership of the external electronic device using credential information received from an owner device of the external electronic device. The method of any one of claims 1 to 6, wherein the at least one processor:
Confirm that the device information includes information indicating that the external electronic device supports direct connection diagnosis,
Activating an object that executes the direct connection diagnosis in the guide screen,
The electronic device, characterized in that it is configured to establish the communication connection based on receiving the user input through an object executing the direct connection diagnosis. The method according to any one of claims 1 to 7, wherein the recovery order is:
A reboot command for the external electronic device or
An electronic device comprising at least one of Wi-Fi information about a Wi-Fi network to which the external electronic device can connect. The method of any one of claims 1 to 8, wherein the at least one processor:
If the offline error code indicates a network error, request a Wi-Fi scan list from the external electronic device including at least one Wi-Fi network to which the external electronic device can connect,
Receive the Wi-Fi scan list from the external electronic device,
Selecting a first Wi-Fi network from the Wi-Fi scan list based on user input,
An electronic device configured to transmit the recovery command including Wi-Fi information about the selected first Wi-Fi network to the external electronic device. The method of any one of claims 1 to 9, wherein the at least one processor:
An electronic device configured to transmit an error reporting message including the error code to the server based on receiving the message. In a method of operating an electronic device,
An operation 605 of receiving device information of an external electronic device from a server;
An operation 615 of displaying a guide screen guiding a communication connection with the external electronic device based on the device information;
An operation 620 of establishing the communication connection with the external electronic device through the communication circuit based on a user input to the guide screen;
Operations 630 and 635 of receiving connection information and device log data including an error code related to an offline state of the external electronic device from the external electronic device through the communication connection;
An operation 640 of transmitting a recovery command to the external electronic device through the communication connection based on the error code and the device log data;
A method comprising an operation (640) of receiving a message from the server indicating that the external electronic device is connected to the server. The method of claim 11, wherein the guide screen is:
A method comprising at least one of a phrase guiding the communication connection or an object for establishing the communication connection. The method of claim 11 or 12,
An operation of displaying an action guide screen related to the offline state processed based on the error code;
The method further comprising transmitting the recovery command based on the user input to the action guide screen. The method according to any one of claims 11 to 13,
The method further comprising releasing the communication connection with the external electronic device based on receiving the message. The method according to any one of claims 11 to 14,
An operation of transmitting a request message to confirm ownership to the external electronic device through the communication connection;
Receiving a response message including a first personal identification number (PIN) from the external electronic device;
An operation of receiving a second PIN from a user;
The method further includes completing ownership verification of the external electronic device when the first PIN and the second PIN match. The method according to any one of claims 11 to 15,
An operation of confirming ownership of the external electronic device using credential information obtained in the process of registering the external electronic device with the server, or
The method further includes confirming ownership of the external electronic device using credential information received from the owner device of the external electronic device. The method of any one of claims 11 to 16, wherein the operation of establishing the communication connection comprises:
Confirming that the device information includes information indicating that the external electronic device supports direct connection diagnosis;
An operation of activating an object that executes the direct connection diagnosis within the guide screen;
Establishing the communication connection based on receiving the user input through an object executing the direct connection diagnosis. The method according to any one of claims 11 to 17, wherein the recovery order is:
A reboot command for the external electronic device or
A method comprising at least one of Wi-Fi information about a Wi-Fi network to which the external electronic device can connect. The method according to any one of claims 11 to 18,
If the error code indicates a network error, requesting, from the external electronic device, a Wi-Fi scan list including at least one Wi-Fi network to which the external electronic device can connect;
Receiving the Wi-Fi scan list from the external electronic device;
Selecting a first Wi-Fi network from the Wi-Fi scan list based on user input;
The method further comprising transmitting the recovery command including Wi-Fi information about the selected first Wi-Fi network to the external electronic device. The method according to any one of claims 11 to 19,
The method further comprising transmitting an error reporting message including the error code to the server based on receiving the message.

Images