KR20240057296A - Electronic device for controlling a display device and method of operating the same - Google Patents

Abstract

An electronic device according to an embodiment may include a GPS module, a communication module, memory, and a processor. The processor according to one embodiment may be set to confirm that the electronic device enters a designated space including a plurality of areas through the GPS module. The processor according to an embodiment may be set to receive first communication signal information about a communication signal received by the wearable electronic device from the wearable electronic device through the communication module. The processor according to one embodiment may be set to perform clustering on the first communication signal information and confirm a clustered area corresponding to the first communication signal information. The processor according to an embodiment may, through the communication module, send information about the clustered areas to the wearable electronic device so that the wearable electronic device can identify a first area in which the wearable electronic device is located among the plurality of areas. It can be set to transmit to the device. The processor according to one embodiment may be set to obtain information about the user's first activity and information about the first area from the wearable electronic device through the communication module. The processor according to one embodiment is set to obtain a first command for augmented reality information of a head mounted display (HMD) device based on the first activity information and information about the first area. It can be. The processor according to one embodiment may be set to transmit the first command to the HMD device through the communication module so that the HMD device displays the augmented reality information through a display of the HMD device. In addition, various embodiments may be applied.

Description

Electronic device for controlling a display device and method of operating the same {ELECTRONIC DEVICE FOR CONTROLLING A DISPLAY DEVICE AND METHOD OF OPERATING THE SAME}

One embodiment of the present invention relates to an electronic device that controls a display device and a method of operating the same.

Recently, electronic devices can be equipped with various functions. For example, in addition to communication functions, entertainment functions such as games, multimedia functions such as music/video playback, communication and security functions for mobile banking, or various functions such as schedule management or electronic wallet are integrated into one electronic device. there is. These electronic devices are being miniaturized so that users can conveniently carry and wear them. As electronic and communication technologies develop, these electronic devices are becoming smaller and lighter to the point where they can be used without significant inconvenience even when worn on the body.

The use of wearable electronic devices is increasing as they are portable or wearable electronic devices that are convenient to use in daily life. For example, wearable electronic devices can be implemented in various forms such as accessories such as glasses, watches, rings, clothing, or body implants, and can collect and provide detailed information about the surrounding environment or changes in an individual's body in real time. You can.

An electronic device according to an embodiment may include a GPS module, a communication module, memory, and a processor. The processor according to one embodiment may be set to confirm that the electronic device enters a designated space including a plurality of areas through the GPS module. The processor according to an embodiment may be set to receive first communication signal information about a communication signal received by the wearable electronic device from the wearable electronic device through the communication module. The processor according to one embodiment may be set to perform clustering on the first communication signal information and confirm a clustered area corresponding to the first communication signal information. The processor according to an embodiment may, through the communication module, send information about the clustered areas to the wearable electronic device so that the wearable electronic device can identify a first area in which the wearable electronic device is located among the plurality of areas. It can be set to transmit to the device. The processor according to one embodiment may be set to obtain information about the user's first activity and information about the first area from the wearable electronic device through the communication module. The processor according to one embodiment is set to obtain a first command for augmented reality information of a head mounted display (HMD) device based on the first activity information and information about the first area. It can be. The processor according to one embodiment may be set to transmit the first command to the HMD device through the communication module so that the HMD device displays the augmented reality information through a display of the HMD device.

A method of operating an electronic device according to an embodiment may include confirming that the electronic device enters a designated space including a plurality of areas through a GPS module included in the electronic device. A method of operating an electronic device according to an embodiment may include receiving first communication signal information about a communication signal received by the wearable electronic device from a wearable electronic device through a communication module included in the electronic device. You can. A method of operating an electronic device according to an embodiment may include performing clustering on the first communication signal information and confirming a clustered area corresponding to the first communication signal information. A method of operating an electronic device according to an embodiment includes, through the communication module, providing information about the clustered areas so that the wearable electronic device identifies a first area in which the wearable electronic device is located among the plurality of areas. It may include transmitting to the wearable electronic device. A method of operating an electronic device according to an embodiment may include obtaining information about a user's first activity and information about the first area from the wearable electronic device through the communication module. A method of operating an electronic device according to an embodiment includes, based on the first activity information and information about the first area, a first command for augmented reality information of a head mounted display (HMD) device. It may include acquisition operations. A method of operating an electronic device according to an embodiment may include transmitting the first command to the HMD device through the communication module so that the HMD device displays the augmented reality information through a display of the HMD device. You can.

A wearable electronic device according to an embodiment may include a sensor, a GPS module, a communication module, and a processor. The processor according to one embodiment may be set to confirm that the electronic device enters a designated space including a plurality of areas through the GPS module. The processor according to one embodiment may be set to check the user's activity information through the sensor. The processor according to one embodiment may be set to obtain first communication signal information about the communication signal received through the communication module when the activity information is confirmed. The processor according to one embodiment may be set to perform clustering on the first communication signal information and confirm a clustered area corresponding to the first communication signal information. The processor according to an embodiment may be set to identify a first area in which the wearable electronic device is located among the plurality of areas based on the clustered area. The processor according to one embodiment is set to obtain a first command for augmented reality information of a head mounted display (HMD) device based on the first activity information and information about the first area. It can be. The processor according to one embodiment may be set to transmit the first command to the HMD device through the communication module so that the HMD device displays the augmented reality information through a display of the HMD device.

1 is a block diagram of an electronic device in a network environment, according to various embodiments.
2A is a schematic diagram of an electronic system, according to one embodiment.
FIG. 2B is a block diagram showing a schematic configuration of an electronic system according to an embodiment.
FIG. 3 is a diagram for explaining operations of a wearable electronic device, an electronic device, and a VST device, according to an embodiment.
FIG. 4A is a flow chart for explaining an operation in which a wearable electronic device checks a user's activity, according to an embodiment.
FIG. 4B is a diagram of a machine learning-based classifier model used by a wearable electronic device to identify a user's activity, according to an embodiment.
FIG. 5A is a block diagram for explaining an operation in which an electronic device controls a VST device, according to an embodiment.
FIG. 5B is a flow chart to explain an operation in which an electronic device learns a usage pattern for a user's VST device, according to an embodiment.
FIG. 6 is a flow chart to explain an operation in which an electronic device performs clustering, according to an embodiment.
FIG. 7A is a diagram for explaining an operation in which an electronic device performs clustering, according to an embodiment.
FIG. 7B is a diagram showing clustered areas in a designated space, according to one embodiment.
FIG. 8 is a diagram illustrating an operation of labeling at least one area included in a designated space by an electronic device, according to an embodiment.
FIG. 9A is a block diagram of an inference model in which an electronic device labels at least one area included in a designated space, according to an embodiment.
FIG. 9B is a table showing the results of labeling at least one area included in a designated space by an electronic device, according to one embodiment.
FIG. 10 is a flow chart to explain an operation in which an electronic device learns a usage pattern for a user's VST device, according to an embodiment.
FIG. 11 is a diagram illustrating an operation in which a wearable electronic device controls a VST device, according to an embodiment.
FIGS. 12A to 12D are diagrams for explaining an operation of a VST device to display augmented reality information, according to an embodiment.

1 is a block diagram of an electronic device 101 in a network environment 100, according to various embodiments. Referring to FIG. 1, in the network environment 100, the electronic device 101 communicates with the electronic device 102 through a first network 198 (e.g., a short-range wireless communication network) or a second network 199. It is possible to communicate with at least one of the electronic device 104 or the server 108 through (e.g., a long-distance wireless communication network). According to one embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108. According to one embodiment, the electronic device 101 includes a processor 120, a memory 130, an input module 150, an audio output module 155, a display module 160, an audio module 170, and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or may include an antenna module 197. In some embodiments, at least one of these components (eg, the connection terminal 178) may be omitted or one or more other components may be added to the electronic device 101. In some embodiments, some of these components (e.g., sensor module 176, camera module 180, or antenna module 197) are integrated into one component (e.g., display module 160). It can be.

The processor 120, for example, executes software (e.g., program 140) to operate at least one other component (e.g., hardware or software component) of the electronic device 101 connected to the processor 120. It can be controlled and various data processing or calculations can be performed. According to one embodiment, as at least part of data processing or computation, the processor 120 stores commands or data received from another component (e.g., sensor module 176 or communication module 190) in volatile memory 132. The commands or data stored in the volatile memory 132 can be processed, and the resulting data can be stored in the non-volatile memory 134. According to one embodiment, the processor 120 includes a main processor 121 (e.g., a central processing unit or an application processor) or an auxiliary processor 123 that can operate independently or together (e.g., a graphics processing unit, a neural network processing unit ( It may include a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device 101 includes a main processor 121 and a secondary processor 123, the secondary processor 123 may be set to use lower power than the main processor 121 or be specialized for a designated function. You can. The auxiliary processor 123 may be implemented separately from the main processor 121 or as part of it.

The auxiliary processor 123 may, for example, act on behalf of the main processor 121 while the main processor 121 is in an inactive (e.g., sleep) state, or while the main processor 121 is in an active (e.g., application execution) state. ), together with the main processor 121, at least one of the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190) At least some of the functions or states related to can be controlled. According to one embodiment, co-processor 123 (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module 180 or communication module 190). there is. According to one embodiment, the auxiliary processor 123 (eg, neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. Artificial intelligence models can be created through machine learning. For example, such learning may be performed in the electronic device 101 itself on which the artificial intelligence model is performed, or may be performed through a separate server (e.g., server 108). Learning algorithms may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but It is not limited. An artificial intelligence model may include multiple artificial neural network layers. Artificial neural networks include deep neural network (DNN), convolutional neural network (CNN), recurrent neural network (RNN), restricted boltzmann machine (RBM), belief deep network (DBN), bidirectional recurrent deep neural network (BRDNN), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the examples described above. In addition to hardware structures, artificial intelligence models may additionally or alternatively include software structures.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176) of the electronic device 101. Data may include, for example, input data or output data for software (e.g., program 140) and instructions related thereto. Memory 130 may include volatile memory 132 or non-volatile memory 134.

The program 140 may be stored as software in the memory 130 and may include, for example, an operating system 142, middleware 144, or application 146.

The input module 150 may receive commands or data to be used in a component of the electronic device 101 (e.g., the processor 120) from outside the electronic device 101 (e.g., a user). The input module 150 may include, for example, a microphone, mouse, keyboard, keys (eg, buttons), or digital pen (eg, stylus pen).

The sound output module 155 may output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. Speakers can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.

The display module 160 can visually provide information to the outside of the electronic device 101 (eg, a user). The display module 160 may include, for example, a display, a hologram device, or a projector, and a control circuit for controlling the device. According to one embodiment, the display module 160 may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of force generated by the touch.

The audio module 170 can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device (e.g., directly or wirelessly connected to the electronic device 101). Sound may be output through the electronic device 102 (e.g., speaker or headphone).

The sensor module 176 detects the operating state (e.g., power or temperature) of the electronic device 101 or the external environmental state (e.g., user state) and generates an electrical signal or data value corresponding to the detected state. can do. According to one embodiment, the sensor module 176 includes, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, humidity sensor, or light sensor.

The interface 177 may support one or more designated protocols that can be used to connect the electronic device 101 directly or wirelessly with an external electronic device (eg, the electronic device 102). According to one embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102). According to one embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 can convert electrical signals into mechanical stimulation (e.g., vibration or movement) or electrical stimulation that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 can capture still images and moving images. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 can manage power supplied to the electronic device 101. According to one embodiment, the power management module 188 may be implemented as at least a part of, for example, a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101. According to one embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

Communication module 190 is configured to provide a direct (e.g., wired) communication channel or wireless communication channel between electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108). It can support establishment and communication through established communication channels. Communication module 190 operates independently of processor 120 (e.g., an application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., : LAN (local area network) communication module, or power line communication module) may be included. Among these communication modules, the corresponding communication module is a first network 198 (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (e.g., legacy It may communicate with an external electronic device 104 through a telecommunication network such as a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., LAN or WAN). These various types of communication modules may be integrated into one component (e.g., a single chip) or may be implemented as a plurality of separate components (e.g., multiple chips). The wireless communication module 192 uses subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199. The electronic device 101 can be confirmed or authenticated.

The wireless communication module 192 may support 5G networks after 4G networks and next-generation communication technologies, for example, NR access technology (new radio access technology). NR access technology provides high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and access to multiple terminals (massive machine type communications (mMTC)), or ultra-reliable and low-latency (URLLC). -latency communications)) can be supported. The wireless communication module 192 may support high frequency bands (eg, mmWave bands), for example, to achieve high data rates. The wireless communication module 192 uses various technologies to secure performance in high frequency bands, for example, beamforming, massive array multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. It can support technologies such as input/output (FD-MIMO: full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module 192 may support various requirements specified in the electronic device 101, an external electronic device (e.g., electronic device 104), or a network system (e.g., second network 199). According to one embodiment, the wireless communication module 192 supports Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mmTC, or U-plane latency (e.g., 164 dB or less) for realizing URLLC. Example: Downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) can be supported.

The antenna module 197 may transmit or receive signals or power to or from the outside (eg, an external electronic device). According to one embodiment, the antenna module 197 may include an antenna including a radiator made of a conductor or a conductive pattern formed on a substrate (eg, PCB). According to one embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected to the plurality of antennas by, for example, the communication module 190. can be selected. Signals or power may be transmitted or received between the communication module 190 and an external electronic device through the at least one selected antenna. According to some embodiments, in addition to the radiator, other components (eg, radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module 197.

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, a mmWave antenna module includes a printed circuit board, an RFIC disposed on or adjacent to a first side (e.g., bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., mmWave band); And a plurality of antennas (e.g., array antennas) disposed on or adjacent to the second side (e.g., top or side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (e.g., bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and signal ( (e.g. commands or data) can be exchanged with each other.

According to one embodiment, commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199. Each of the external electronic devices 102 or 104 may be of the same or different type as the electronic device 101. According to one embodiment, all or part of the operations performed in the electronic device 101 may be executed in one or more of the external electronic devices 102, 104, or 108. For example, when the electronic device 101 must perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 may perform the function or service instead of executing the function or service on its own. Alternatively, or additionally, one or more external electronic devices may be requested to perform at least part of the function or service. One or more external electronic devices that have received the request may execute at least part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may process the result as is or additionally and provide it as at least part of a response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology can be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to one embodiment, the external electronic device 104 or server 108 may be included in the second network 199. The electronic device 101 may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

2A is a schematic diagram of an electronic system, according to one embodiment.

Referring to FIG. 2A , according to one embodiment, the electronic system may include a wearable electronic device 201, an electronic device 202, and a video see-through (VST) device 204. The wearable electronic device 201, the electronic device 202, and the video see-through (VST) device 204 may transmit and receive data (or information) with each other. For example, each of the wearable electronic device 201, the electronic device 202, and the video see-through (VST) device 204 may be implemented the same or similar to the electronic devices 101, 102, and 104 of FIG. 1. there is.

According to one embodiment, the electronic system may be implemented in a designated space. For example, the designated space may be a space corresponding to a geo-fence established using GPS signals. For example, a designated space may include multiple zones. For example, the designated space may be a home space, an office space, or a space inside a building.

According to one embodiment, the wearable electronic device 201 may sense or check the user's activity. For example, the wearable electronic device 201 may be implemented as a smart watch. For example, the wearable electronic device 201 may acquire a sensing signal corresponding to the user's motion through a sensor (e.g., the sensor 211 in FIG. 2B) while being worn by the user. The wearable electronic device 201 ) can analyze the sensing signal to check the activities performed by the user, such as walking, running, stretching, sleeping, brushing teeth, washing hands, flushing the toilet, eating, typing on the keyboard, studying or working. This may include reading, etc.

According to one embodiment, the wearable electronic device 201 may receive a communication signal (eg, a Wi-Fi signal) from an access point (or a Wi-Fi router) located in a designated space. The wearable electronic device 201 may acquire communication signal information (e.g., basic service set identifier (BSSID), service set identifier (SSID), receiver signal strength indicator (RSSI)) by analyzing the communication signal.

According to one embodiment, the wearable electronic device 201 may cluster communication signal information to determine which area (or specific area) it is located in in a designated space. Alternatively, the wearable electronic device 201 may check, through the electronic device 202, which area the wearable electronic device 201 is located in the designated space. At this time, the electronic device 202 may cluster communication signal information received from the wearable electronic device 201 and transmit information about the clustered area to the wearable electronic device 201.

According to one embodiment, the wearable electronic device 201 may transmit activity information and information about the area in which the wearable electronic device 201 is located to the electronic device 202. For example, information about the area may mean information indicating in which area the wearable electronic device 201 is located within the designated space, based on information about the clustered area. For example, information about the area may indicate that the wearable electronic device 201 is located in a specific area (eg, a bathroom, living room, study, or hallway) among a plurality of areas included in the home.

Meanwhile, in FIG. 2A, the wearable electronic device 201 is shown as a smart watch, but embodiments of the present invention may not be limited thereto. For example, the wearable electronic device 201 may be implemented as another type of wearable electronic device (eg, smart ring).

According to one embodiment, the electronic device 202 may perform a clustering operation on communication signal information received from the wearable electronic device 201. The electronic device 202 may check whether the communication signal information is included in one of the clusters for the designated space and check the area corresponding to the confirmed cluster. The electronic device 202 may transmit information about the clustered area to the wearable electronic device 201.

According to one embodiment, the electronic device 202 controls the VST device 204 based on the activity information received from the wearable electronic device 201 and information about the area in which the wearable electronic device 201 is located. You can. For example, the electronic device 202, based on the activity information received from the wearable electronic device 201 and information about the area in which the wearable electronic device 201 is located, determines the pre-learned user's VST device 204. You can check usage patterns (or behavior patterns). The electronic device 202 may control the VST device 204 based on the confirmed usage pattern. For example, the electronic device 202 may control the VST device 204 based on a usage pattern in which the user used the VST device 204 when located in the corresponding area while performing the corresponding activity. For example, the electronic device 202 may transmit a command to the VST device 204 to execute a specific application or display specific augmented reality information. For example, the electronic device 202 may be implemented as a smartphone.

According to one embodiment, the VST device 204 may perform a specific function or change to a specific state based on a command received from the electronic device 202. For example, the VST device 204 may execute a specific application or display specific augmented reality information on a display (eg, the display 290 of FIG. 2B) based on the command received from the electronic device 202. For example, the VST device 204 may be implemented as a head mounted display (HMD) device. For example, the VST device 204 may be implemented as a virtual reality (VR) device.

FIG. 2B is a block diagram showing a schematic configuration of an electronic system, according to an embodiment.

Referring to FIG. 2B, according to one embodiment, the wearable electronic device 201 may include a sensor 211, a processor 212, a memory 213, and a communication module 218.

According to one embodiment, the processor 212 may control the overall operation of the wearable electronic device 201. For example, the processor 212 may be implemented identically or similarly to the processor 120 of FIG. 1 .

According to one embodiment, the processor 212 may sense or check the user's activity through the sensor 211 (eg, sensor module 176 in FIG. 1). For example, when the processor 212 confirms that the user enters a designated space (e.g., home, office, building), it can check the user's activities through the sensor 211. For example, the processor 212 may confirm that the user enters the designated space based on a notification received from the electronic device 202.

According to one embodiment, the processor 212 may check the activity performed by the user by analyzing the signal obtained through the sensor 211 while the wearable electronic device 201 is worn by the user. For example, the processor 212 may analyze the user's motion, the user's heart rate, the user's voice, surrounding sounds, surrounding atmospheric pressure or temperature, etc. through the sensor 211. For example, the processor 212 may run a machine learning-based classifier model to analyze the signal obtained through the sensor 211 and check the user's activity. The processor 212 may store information about the confirmed activity (hereinafter, activity information) in the memory 213 (eg, memory 130 in FIG. 1).

According to one embodiment, when the user's activity is confirmed, the processor 212 receives a communication signal (or WIFI signal) from an access point (or WIFI router) located in a designated space through the communication module 218. You can. For example, the processor 212 may disable the communication module 218 or run it at low power until the user's activity is confirmed. When the user's activity is confirmed, the processor 212 may activate the communication module 218 to receive a communication signal from an access point located in a designated space.

According to one embodiment, the processor 212 may obtain communication signal information about the communication signal. For example, communication signal information may include address information (eg, BSSID or MAC address) of the access point that transmitted the communication signal. Additionally, communication signal information may include information (eg, RSSI) indicating the strength of the communication signal.

For example, the processor 212 may identify an access point located within a designated space based on a basic service set identifier (BSSID) included in a communication signal. For example, the location of the corresponding access point within the designated space can be confirmed in advance. The processor 212 may acquire location information indicating the location where the communication signal was received (or the current location of the wearable electronic device 201) by analyzing the strength of the communication signal (e.g., RSSI).

According to another embodiment, the processor 212 may receive a communication signal based on ultra wideband (UWB) technology and obtain information indicating the location of the wearable electronic device 201 based on the received communication signal. For example, the processor 212 may obtain a communication signal based on UWB technology from an electronic device installed or located within a designated space. The processor 212 may determine the distance between the wearable electronic device 201 and the corresponding electronic device by analyzing the communication signal. The processor 212 may check the location of the wearable electronic device 201 based on the determined distance.

Meanwhile, the processor 212 can confirm the location of the wearable electronic device 201 by analyzing various types of communication signals in addition to the communication signals described above. For example, a method of confirming the location of the wearable electronic device 201 may use various communication signals.

According to one embodiment, the processor 212 may transmit communication signal information to the electronic device 202 through the communication module 218. The processor 212 may receive information about the clustered area for communication signal information from the electronic device 201 through the communication module 218.

According to one embodiment, the processor 212 may identify the first area where the wearable electronic device 201 is located among a plurality of areas included in the designated space, based on information about the clustered area. For example, the first area may refer to an area where a communication signal is received from an access point or an area where the wearable electronic device 201 is located in a designated space.

According to one embodiment, the processor 212 may transmit activity information and information about the first area to the electronic device 202. The electronic device 202 may control the VST device 204 based on activity information and information about the first area.

According to one embodiment, the wearable electronic device 201 may further include a GPS module (not shown). For example, the processor 212 may confirm that the wearable electronic device 201 has entered or is located in a designated space through a GPS module. When it is confirmed that the wearable electronic device 201 enters or is located in a designated space, the processor 212 may start an operation to check the user's activity using the sensor 211.

According to one embodiment, the electronic device 202 may include a processor 220, a memory 230, a GPS module 240, and a communication module 250.

According to one embodiment, the processor 220 may control the overall operation of the electronic device 202. For example, the processor 220 may be implemented identically or similarly to the processor 120 of FIG. 1 .

According to one embodiment, the processor 220 may confirm that the electronic device 202 enters or is located in a designated space through the GPS module 240. For example, the processor 220 may set a geofence in a designated space using a GPS signal. For example, the processor 220 may confirm that the user enters or is located in a designated space based on the GPS signal received through the GPS module 240.

According to one embodiment, when the processor 220 confirms that the electronic device 202 enters or is located in the designated space, it informs the wearable electronic device 201 of entering the designated space through the communication module 250. A notification can be sent.

According to one embodiment, the processor 220 may receive first communication signal information about the communication signal received by the wearable electronic device 201 from the wearable electronic device 201 through the communication module 250. there is.

According to one embodiment, the processor 220 may perform clustering on the first communication signal information. For example, clustering may refer to an operation to confirm or distinguish an area corresponding to first communication signal information among a plurality of areas included in a designated space. For example, clustering performed by the processor 220 may mean a zone clustering operation. The processor 220 may perform clustering to confirm the clustered area corresponding to the first communication signal information. The processor 220 may transmit information about the clustered area to the wearable electronic device 201 through the communication module 250.

According to one embodiment, the processor 220 receives information about the user's first activity and the first area where the wearable electronic device 201 is located from the wearable electronic device 201 through the communication module 250. Information can be received or obtained.

According to one embodiment, the processor 220 may generate or obtain a command for controlling the VST device 204 based on first activity information and information about the first area. The processor 220 may check the pre-learned usage pattern of the VST device 204 based on the first activity and the first area. For example, the pre-learned usage pattern of the VST device 204 may be stored in the memory 230 (eg, memory 130 in FIG. 1). The processor 220 may decide to display specific augmented reality information or execute a specific function based on pre-learned usage patterns. The processor 220 may generate or obtain a command for executing the determined specific augmented reality information or specific function.

According to one embodiment, the processor 220 may transmit a command to the VST device 204 through the communication module 250. When the VST device 204 receives a command, it can display specific augmented reality information corresponding to the command or execute a specific function.

According to one embodiment, the VST device 204 may include a processor 270, a camera 275, a memory 280, a communication module 285, and a display 290. For example, the VST device 204 may be implemented as an HMD device (eg, a VR HMD device).

According to one embodiment, the processor 270 may control the overall operation of the VST device 204. For example, the processor 270 may be implemented identically or similarly to the processor 120 of FIG. 1 .

According to one embodiment, the processor 270 may display an image captured through the camera 275 on the display 290. For example, the processor 270 may display an image captured through the camera 275 on the display 290 in real time.

According to one embodiment, the processor 270 may receive a command from the electronic device 202 through the communication module 285. The processor 270 may display specific augmented reality information corresponding to the command on the display 290. For example, the processor 270 may overlap and display augmented reality information on an image captured through the camera 275. Alternatively, the processor 270 may execute a specific function corresponding to the command. For example, the processor 270 may execute a specific application stored in the memory 280 based on a command.

The VST device according to the comparative example may provide or display augmented reality information based on an image acquired through a camera of the VST device. Alternatively, the VST device according to the comparative example may provide augmented reality information limited to surrounding objects recognized based on images acquired through a camera. The augmented reality information that the VST device according to the comparative example could provide was limited.

According to an embodiment of the present invention, the electronic device 202 analyzes the usage pattern of the VST device 204 based on the user's location and activity confirmed through the wearable electronic device 201, and provides information on the analysis results. Accordingly, suitable augmented reality information may be provided to the user of the VST device 204 or execution of a specific function may be recommended. Through this, the VST device 204 can provide information (or augmented reality information) or functions suitable for the user without relying on image analysis.

Meanwhile, in FIGS. 2A and 2B, the description focuses on the electronic device 202 controlling the VST device 204. However, depending on the implementation, the wearable electronic device 201 may operate the VST device (VST device) without the electronic device 202. 204) can also be controlled. At this time, operations performed by the electronic device 202 may be performed by the wearable electronic device 201.

In addition, in FIGS. 2A and 2B, the description focuses on the control of the VST device 204 by the electronic device 202, but the technical idea of the present invention may not be limited thereto. For example, the electronic device 202 may control other electronic devices (eg, robot vacuum cleaners) that can perform communication functions other than the VST device.

At least some of the operations of the wearable electronic device 201 described below may be performed by the processor 212. At least part of the operation of the electronic device 202 may be performed by the processor 220. Additionally, at least a portion of the operations of the VST device 204 may be performed by the processor 270.

FIG. 3 is a diagram for explaining operations of a wearable electronic device, an electronic device, and a VST device, according to an embodiment.

Referring to FIG. 3 , according to one embodiment, in operation 301, the electronic device 202 may confirm entry into a designated space. In operation 303, the electronic device 202 may transmit an entry notification for the designated space to the wearable electronic device 201.

According to one embodiment, in operation 305, the wearable electronic device 201 may check the user's activities. For example, the user's activities include walking, running, stretching, sleeping, brushing teeth, washing hands, flushing the toilet, and eating. This can include doing things, typing on a keyboard, studying or working, and reading.

According to one embodiment, in operation 307, the wearable electronic device 201 may receive a communication signal from an access point located within a designated space based on checking the user's activity.

According to one embodiment, in operation 309, the wearable electronic device 201 may transmit first communication signal information (eg, BSSID, SSID, RSSI) about the communication signal to the electronic device 202.

According to one embodiment, in operation 311, the electronic device 202 may perform clustering on the first communication signal information. The electronic device 202 may confirm the clustered area corresponding to the first communication signal information based on the result of performing clustering.

Meanwhile, operations 307 to 311 describe a method of performing a clustering operation based on a WIFI signal or a BLE signal, but the technical idea of the present invention may not be limited thereto. For example, the method by which the wearable electronic device 201 performs clustering based on communication signal information may use various communication signals (eg, UWB communication signals).

According to one embodiment, in operation 313, the electronic device 202 may transmit information about the clustered area to the wearable electronic device 201.

According to one embodiment, in operation 315, the wearable electronic device 201 may check the first area in which the wearable electronic device 201 is located in the designated space based on information about the clustered area.

According to one embodiment, in operation 317, the wearable electronic device 201 may transmit activity information and information about the first area to the electronic device 202. For example, information about the first area may mean information indicating in which area the wearable electronic device 201 is located within the designated space, based on information about the clustered area. For example, information about the first area may indicate that the wearable electronic device 201 is located in a specific area (eg, a bathroom, living room, study, or hallway) among a plurality of areas included in the home.

According to one embodiment, in operation 319, the electronic device 202 displays augmented reality information according to a user pattern (e.g., a usage pattern for the VST device 204) based on the activity information and information about the first area. can be decided. Additionally, the electronic device 202 may generate a command for the VST device 204 to display the determined augmented reality information.

According to one embodiment, in operation 321, the electronic device 202 may transmit a command to the VST device 204.

According to one embodiment, in operation 323, the VST device 204 may display augmented reality information based on a command. Alternatively, the VST device 204 may perform a specific function based on a command.

Depending on the implementation, there may be a time difference between performing operations 302 to 313 described above and performing operations 315 to 323 described above. For example, after operations 313 are performed in operations 302, operation 315 may be performed after a certain period of time has elapsed.

FIG. 4A is a flow chart for explaining an operation in which a wearable electronic device checks a user's activity, according to an embodiment.

Referring to FIG. 4A , according to one embodiment, in operation 401, the wearable electronic device 201 may acquire a sensing signal through the sensor 211.

According to one embodiment, in operation 403, the wearable electronic device 201 may extract features of the sensing signal.

According to one embodiment, in operation 405, the wearable electronic device 201 may check the user's activity through a machine learning-based classifier model. For example, the wearable electronic device 201 may execute a machine learning-based classifier model. The wearable electronic device 201 may input features of the extracted sensing signal as input values for a machine learning-based classifier model. The wearable electronic device 201 may obtain activity information as an output value of a machine learning-based classifier model.

Meanwhile, the operation of the wearable electronic device 201 to check the user's activity through a machine learning-based classifier model will be described in detail in FIG. 4B below.

FIG. 4B is a diagram of a machine learning-based classifier model used by a wearable electronic device to identify a user's activity, according to an embodiment.

Referring to FIG. 4B, according to one embodiment, the machine learning-based classifier model 410 may be a software module running on the processor 212 or a hardware module including an electrical circuit. For example, the machine learning-based classifier model 410 includes a signal pre-processing unit 415, a feature extraction unit 420, an activity learning unit 430, and a classifier modeling unit 440. ), and may include at least some or all of the classifier model 450. According to one embodiment, the signal preprocessor 415, feature extraction unit 420, activity learning unit 430, classifier modeling unit 440, and classifier model 450 are each software executed by the processor 212. It may be a program or a hardware module containing electrical circuits.

The signal preprocessor 415 according to one embodiment may obtain sensor data by performing preprocessing on the sensing signal obtained by the sensor 211. For example, the signal preprocessor 415 may acquire acceleration and gyro sensor data from the sensing signal detected through the sensor 211, correct the sensor axis, and perform filtering to remove noise.

The feature extractor 420 according to one embodiment may extract feature information from data such as an inertial sensor, an atmospheric pressure sensor, an HR sensor, and a sound sensor. For example, hand posture information, frequency characteristic information for hand motion, and size and direction information for hand motion movement can be obtained from acceleration and gyro sensor data.

The activity learning unit 430 according to one embodiment may learn an activity (or a pattern of an activity) through a machine learning technique based on extracted feature information.

The classifier modeling unit 440 according to one embodiment may model a classifier that classifies activities by type based on activity learning results and provide classifier modeling information.

The classifier model 450 according to one embodiment can identify activities corresponding to the extracted feature information based on the feature information extracted from the feature extraction unit 420 and the classifier modeling information provided from the classifier modeling unit 440. there is. For example, the classifier modeling unit 440 checks whether the extracted feature information corresponds to a specific activity (e.g., brushing teeth) or whether the extracted feature information corresponds to an activity other than a specific activity (e.g., keyboard typing). You can check it.

According to one embodiment. The operation of the feature extraction unit 420 and the activity learning unit 430 is a pre-training operation (or training phase) and is performed by an external electronic device (e.g., electronic device 202) other than the processor 212 of the wearable electronic device 201. )) or can be performed on an external server. When the operations of the feature extraction unit 420 and the activity learning unit 430 are performed externally, pre-trained learning results may be provided to the wearable electronic device 201. The wearable electronic device 201 may perform a recognition operation (recognition phase) using pre-trained learning results and the classifier model 450. The wearable electronic device 201 may output confirmed activity information as a result of performing the recognition operation.

FIG. 5A is a block diagram for explaining an operation in which an electronic device controls a VST device, according to an embodiment.

Referring to FIG. 5A , according to one embodiment, in operation 501, the electronic device 202 receives first communication signal information about the communication signal received by the wearable electronic device 201 from the wearable electronic device 201. It can be obtained.

According to one embodiment, in operation 503, the electronic device 202 may perform clustering on the first communication signal information. For example, the electronic device 202 may perform clustering to identify a clustered area corresponding to the first communication signal information among a plurality of areas included in the designated space. The electronic device 202 may transmit information about the clustered area to the wearable electronic device 201.

According to one embodiment, in operation 505, the electronic device 202 may check the user's activity and the first area where the wearable electronic device 201 is located. For example, the electronic device 202 may check the wearable electronic device 201. Information about the user's activities and information about the first area can be received from (201).

According to one embodiment, in operation 507, the electronic device 202 may check a usage pattern for the VST device 204 that has been learned in advance based on the activity and the first area. In operation 509, the electronic device 202 may determine augmented reality information to be displayed by the VST device 204 through the display 290 based on the usage pattern. For example, augmented reality information may include information about a virtual background screen and/or virtual information about some areas. Alternatively, the electronic device 202 may determine a specific function to be executed by the VST device 204 based on a usage pattern.

According to one embodiment, in operation 511, the electronic device 202 generates a command for the VST device 204 to display the determined augmented reality information or execute a specific function, and transmits the generated command to the VST device. there is. For example, when the confirmed user's activity is brushing teeth and the first area is a bathroom, the electronic device 202 sends a command to the VST device 204 to display virtual reality or augmented reality tooth brushing guide information. ) can be transmitted. The VST device 204 may display virtual reality or augmented reality tooth brushing guide information based on the received command.

FIG. 5B is a flow chart to explain an operation in which an electronic device learns a usage pattern for a user's VST device, according to an embodiment.

Referring to FIG. 5B , in operation 551 according to one embodiment, the electronic device 202 receives first communication signal information about the communication signal received by the wearable electronic device 201 from the wearable electronic device 201. It can be obtained.

According to one embodiment, in operation 553, the electronic device 202 may perform clustering on first communication signal information. For example, the electronic device 202 may perform clustering to identify a clustered area corresponding to the first communication signal information among a plurality of areas included in the designated space. The electronic device 202 may transmit information about the clustered area to the wearable electronic device 201.

According to one embodiment, in operation 555, the electronic device 202 may check the user's activity and the first area where the wearable electronic device 201 is located. For example, the electronic device 202 may receive information about the user's activity and information about the first area from the wearable electronic device 201.

According to one embodiment, in operation 557, the electronic device 202 may label the first area based on information about devices surrounding the wearable electronic device 201. For example, the labeling operation may refer to the operation of giving the user's meaning to the first area. For example, the labeling operation may mean an operation of designating a name or designation of the first area and an operation of storing and managing related information between the first area and peripheral devices.

According to one embodiment, in operation 559, the electronic device 202 may obtain information about the application executed on the VST device 204 from the VST device 204. Additionally, the electronic device 202 may obtain information about the setting value set in the VST device 204 from the VST device 204.

According to one embodiment, in operation 561, the electronic device 202 monitors the user's use based on the user's activity, the first area where the wearable electronic device 201 is located, and the application executed on the VST device 204. Patterns (or behavior patterns) can be learned.

According to one embodiment, in operation 563, the electronic device 202 may store the learned usage pattern in the memory 230. Thereafter, the electronic device 202 can control the VST device 204 using the previously learned usage pattern.

FIG. 6 is a flow chart to explain an operation in which an electronic device performs clustering, according to an embodiment.

Referring to FIG. 6 , in operation 601 according to one embodiment, the electronic device 202 receives first communication signal information about the communication signal received by the wearable electronic device 201 from the wearable electronic device 201. It can be obtained. The electronic device 202 may count the number of communication signal information acquired during a specified period, including first communication signal information.

According to one embodiment, in operation 603, the electronic device 202 may check whether the number of communication signal information acquired during a specified period is greater than the first threshold. For example, the electronic device 202 may check whether the number of communication signal information accumulated during a specified period, including first communication signal information, is greater than the first threshold. For example, the first threshold may be a value representing a sufficient number to perform clustering on communication signal information. For example, the first threshold may be automatically determined by the processor 220 or determined by the user.

According to one embodiment, if the number of communication signal information acquired during a specified period is not greater than the first threshold (No in operation 603), the electronic device 202 may collect the communication signal information as much as the number greater than the first threshold. You can continue to obtain it.

According to one embodiment, if the number of communication signal information acquired during a specified period is greater than the first threshold (example of operation 603), in operation 605, the electronic device 202 determines whether the number of parasitic generated clusters is plural. You can check whether or not (or whether the number of preset clusters is more than 1).

According to one embodiment, if the number of pre-generated clusters is plural (example of operation 605), in operation 607, the electronic device 202 may check the similarity between the first communication signal information and the pre-generated clusters. .

According to one embodiment, in operation 609, the electronic device 202 may check whether the similarity between the first communication signal information and the pre-generated cluster is greater than the second threshold. For example, the second threshold may be a value indicating that the first communication signal information is similar to elements (eg, a plurality of communication signal information) included in the corresponding cluster. For example, the second threshold may be automatically determined by the processor 220 or determined by the user.

According to one embodiment, if the similarity is confirmed to be greater than the second threshold (No in operation 609), in operation 611, the electronic device 202 determines that the first communication signal information is included in the area corresponding to the cluster. This can be confirmed. In operation 619, the electronic device 202 may update the result.

According to one embodiment, if it is confirmed that the similarity is not greater than the second threshold (No in operation 609), in operation 613, the electronic device 202 may store the first communication signal information as an outlier. For example, if the number of communication signal information stored as outliers is greater than the specified number, the electronic device 202 may additionally perform clustering on the corresponding communication signal information. The electronic device 202 may generate additional clusters for communication signals stored in outliers through additional clustering.

According to one embodiment, if the number of pre-generated clusters is not plural (No in operation 605), in operation 615, the electronic device 202 may perform clustering to create a new cluster. For example, the electronic device 202 may perform region clustering on communication signal information obtained using an unsupervised learning method. For example, the electronic device 202 may generate at least one cluster by performing clustering on a plurality of accumulated communication signal information including first communication signal information. Additionally, the electronic device 202 may check the clustered area corresponding to the first communication signal information.

According to one embodiment, in operation 617, the electronic device 202 may evaluate the clustered area corresponding to the first communication signal information. In operation 619, the electronic device may update the clustered area corresponding to the first communication signal information based on the evaluation result.

FIG. 7A is a diagram for explaining an operation in which an electronic device performs clustering, according to an embodiment.

Referring to (a) of FIG. 7A, according to one embodiment, the electronic device 202 may perform region clustering on communication signal information obtained using an unsupervised learning method. For example, the electronic device 202 may generate at least one cluster by performing clustering on a plurality of accumulated communication signal information including first communication signal information. For example, as shown in (a) of FIG. 7A, a cluster of elements corresponding to a plurality of communication signal information may be created in a three-dimensional space. For example, four clusters can be created.

Referring to (b) of FIG. 7A, according to one embodiment, the electronic device 202 may perform silhouette analysis on clustered areas as shown in (a) of FIG. 7A.

Referring to (b) of FIG. 7A, the electronic device 202 may perform silhouette analysis on a plurality of communication signal information including first communication signal information based on Equation 1. For example, one communication signal information may correspond to one element.

Here, a ⁽ⁱ⁾ may mean the average of the distance between a specific element in the ith cluster and all elements included in the cluster. b ⁽ⁱ⁾ may mean the average of the distance between a specific element of the ith cluster and all elements not included in the cluster. b ⁽ⁱ⁾ -a ⁽ⁱ⁾ can mean how separated the clusters are. The electronic device 202 may divide the corresponding value by max(b ⁽ⁱ⁾ -a ⁽ⁱ⁾ ) to normalize the silhouette calculation result. max(b ⁽ⁱ⁾ -a ⁽ⁱ⁾ ) may mean the maximum value of b ⁽ⁱ⁾ -a ⁽ⁱ⁾ . N may mean the number of all elements.

According to one embodiment, the electronic device 202 may obtain the average value of the silhouette calculation results for all elements using Equation 1, and perform and evaluate clustering by repeatedly ensuring that the obtained average value has the maximum value. there is. For example, the average value of the silhouette calculation results may be 0.5667, indicated by the center line. For example, s(i) representing the silhouette coefficient may be determined by the processor 220.

Referring to (c) of FIG. 7A, according to one embodiment, the electronic device 202 may perform cluster similarity analysis on areas clustered as shown in (a) of FIG. 7A. For example, a method of analyzing cluster similarity may be a method using cosine similarity or a method using Pearson correlation coefficient. For example, as shown in Figure 7a (c), in the matrix of Figure 7a (c), the regions (1,1), (2,2), (3,3), and (4,4) are clearly different from other regions. can be displayed distinctly. If the regions (1,1), (2,2), (3,3), and (4,4) are clearly distinguished from other regions, the similarity between each region may be low. That is, the electronic device 202 can confirm that the clustered areas are appropriately divided through the matrix in (c) of FIG. 7A.

According to one embodiment, the electronic device 202 may update the silhouette analysis and/or similarity analysis results for the clustered areas.

FIG. 7B is a diagram showing clustered areas in a designated space, according to one embodiment.

Referring to FIG. 7B, according to one embodiment, the electronic device 201 may divide the designated space into a plurality of clustered areas based on the clustering result. The electronic device 201 may divide the designated space into a bathroom 710, a dining room 720, a bedroom 730, and a study room 740, based on the clustering result.

According to one embodiment, the electronic device 201 selects the clustered area corresponding to the first communication signal information to any one of the bathroom 710, the dining room 720, the bedroom 730, and the study room 740. You can check this. The electronic device 201 may transmit information about the clustered area to the wearable electronic device 201.

FIG. 8 is a diagram illustrating an operation of labeling at least one area included in a designated space by an electronic device, according to an embodiment.

Referring to FIG. 8 , according to one embodiment, in operation 801, the wearable electronic device 201 may check the user's activity sensed through the sensor 211.

According to one embodiment, in operation 803, the wearable electronic device 201 may transmit first communication signal information based on a communication signal received from an access point to the electronic device 202.

According to one embodiment, in operation 805, the electronic device 202 may perform clustering on the first communication signal information. The electronic device 202 may confirm the clustered area corresponding to the first communication signal information based on the clustering result.

According to one embodiment, in operation 807, the electronic device 202 may transmit information about the clustered area to the wearable electronic device 201.

According to one embodiment, in operation 809, the wearable electronic device 201 may check the first area in the designated space based on the clustered area. For example, the wearable electronic device 201 may determine or confirm that the wearable electronic device has entered or is located in the first area in the designated space.

According to one embodiment, in operation 811, the wearable electronic device 201 may obtain information about devices (hereinafter referred to as peripheral devices) surrounding the wearable electronic device 201 in the first area. For example, the wearable electronic device 201 may obtain a communication signal from a peripheral device (eg, a device capable of performing a communication function) through the communication module 218. The wearable electronic device 201 can obtain information about nearby devices from communication signals. For example, if information about nearby devices is pre-stored, the wearable electronic device 201 may update information about nearby devices by analyzing communication signals. For example, updates may be performed periodically or aperiodically. For example, information obtained from a peripheral device may include address information of the device (e.g., BSSID or MAC address), communication signal strength information (e.g., RSSI), and/or the distance between the wearable electronic device 201 and the surrounding device. May contain information. For example, the wearable electronic device 201 searches for home appliances around the wearable electronic device 201 through wireless communication technology (e.g., BLE, UWB, Wi-Fi), and connects the wearable electronic device 201 with the home appliance. The distance between them can be calculated. The wearable electronic device 201 can manage home appliances by dividing them into movable devices (e.g., movable devices) and non-movable devices (e.g., fixed devices) through device type information. For example, the wearable electronic device 201 may classify an air purifier or robot vacuum cleaner as a movable device, and may classify a refrigerator or TV as a fixed device that rarely moves.

According to one embodiment, in operation 813, the wearable electronic device 201 may transmit information about a nearby device to the electronic device 202.

According to one embodiment, in operation 815, the electronic device 202 may label the first area using information about nearby devices.

According to one embodiment, in operation 817, the electronic device 202 may adjust parameters related to the next labeling time point. For example, the electronic device 202 may adjust parameters according to weights for nearby devices. For example, the electronic device 202 may determine the weight according to the distance between the wearable electronic device 201 and the surrounding device and/or the type of the surrounding device.

According to one embodiment, the electronic device 202 may determine the labeling time. For example, when it is confirmed that the labeling time has arrived, the electronic device 202 may perform a labeling operation on at least one area clustered in the designated space. For example, the electronic device 202 can adjust parameters that determine the labeling time using Equation 2.

here, may mean a parameter for performing area labeling. for example, may mean a threshold parameter for the accumulated number of data to determine when to perform labeling. A is the previously collected area information, A' is the currently additionally collected information (activity information and peripheral device information), W is the weight vector for peripheral devices belonging to the area, and C is It may mean information for calibration. W may be a set including wi (i is a natural number), which is the weight of each peripheral device. (Example: W= [w ₁ (refrigerator)=0.3, w ₂ (washing machine)=0.1, ...])

According to one embodiment, the electronic device 202, when newly collected information (activity information and peripheral device information) has no previous collection history, =1 can be determined. That is, the electronic device 202 is: You can keep the value as the previously calculated value.

If the electronic device 202 has a history of previously collecting newly collected information (activity information and peripheral device information), can be determined as a specified constant other than 1. At this time, parameters that determine the labeling timing may be adjusted. For example, the more history a nearby device has been searched for in the area, the more the weight w _i increases. If the nearby device moves to another area and is not searched at its original location, the weight w _i may decrease. If the weight wi is below a certain threshold, the weight w _i is - By applying the previous It can be smaller than the value. Or, if the weight wi is not below a certain threshold, w _i Previous by applying It can be larger than the value. That is, the electronic device 202 is: As the value increases, the parameters are adjusted so that new labeling is performed only when the accumulated number of data is large. If the value becomes small, the parameter can be adjusted to perform new labeling even if the accumulated number of data is small.

According to one embodiment, the electronic device 202 may determine the labeling time based on the adjusted parameter.

FIG. 9A is a block diagram of an inference model in which an electronic device labels at least one area included in a designated space, according to an embodiment.

Referring to FIG. 9A , according to one embodiment, the electronic device 202 may execute an inference model 901 to label at least one area included in a designated space.

According to one embodiment, the inference model 901 may include an encoding unit 910, an inference unit 920, and a decoding unit 930. For example, the inference unit 920 may include a knowledge base 922 and an inference engine 925.

According to one embodiment, the electronic device 202 may input the user's activity and information on nearby devices as input values to the inference model 901. The encoding unit 910 may encode the input value to match the rules of the inference engine 925 and provide the encoded input value to the inference engine 925 of the inference unit 920 .

According to one embodiment, the inference engine 925 may be modeled based on the knowledge base 922. Alternatively, the inference engine 925 may be modeled based on deep learning. The inference engine 925 can obtain the evaluation result of the premise numerically by applying the encoded input value to the premise of the rule. The output engine 925 may unify the numerically obtained evaluation results and output them to the decoder 930. The decoder 930 may decode the unified evaluation result and output it as an inference result.

According to one embodiment, the input value of the inference model 901 may include an activity, a nearby device, and a distance between the wearable electronic device 201 and the nearby device. For example, the input value of the inference model 901 is

According to one embodiment, the output value of the inference model 901 may include the name of the area included in the designated space. For example, the output value of the inference model 901 is Y, and at this time, Y = {bathroom (A), dining room (B), bedroom (C), study (D), ..., unknown (Z)}. It can be implemented.

Meanwhile, the inference result output by the inference model 901 will be exemplarily described in FIG. 9B.

FIG. 9B is a table showing the results of labeling at least one area included in a designated space by an electronic device, according to one embodiment.

Referring to FIG. 9B, the electronic device 202 may label the first area where the wearable electronic device 201 is located through the inference model 901.

According to one embodiment, if the input value It can be. If the input value If the input value If the input value If the input value It can be.

Referring to FIG. 9B, according to one embodiment, the electronic device 202 may determine the name of the first area as bathroom, dining room, bedroom, or study. The electronic device 202 may store and manage information about peripheral devices in the first area (type, number, and distance of nearby devices) along with the name of the first area.

FIG. 10 is a flow chart to explain an operation in which an electronic device learns a usage pattern for a user's VST device, according to an embodiment.

Referring to FIG. 10 , according to one embodiment, in operation 1001, the wearable electronic device 201 may check the user's activity.

According to one embodiment, in operation 1003, the wearable electronic device 201 may check the first area in which the wearable electronic device 201 is located.

According to one embodiment, in operation 1005, the wearable electronic device 201 may request transmission of application information and setting information to the VST device 204.

According to one embodiment, in operation 1007, the VST device 204 transmits application information and/or setting information executed on the VST device 204 to the electronic device 202 according to a request from the wearable electronic device 201. Can be transmitted.

According to one embodiment, in operation 1009, the wearable electronic device 201 may transmit activity information and information about the first area to the electronic device 202.

According to one embodiment, in operation 1011, the electronic device 201 uses activity information, information about the first area, application information and/or setting information of the VST device 204 to set the user's VST device 204. You can learn usage patterns (or behavior patterns) for ). For example, the electronic device 201 may learn a specific application or specific function executed by the VST device 204 while the user is performing a specific activity in the first area. Alternatively, the electronic device 201 may learn the settings set in the VST device 204 while the user is performing a specific activity in the first area. Thereafter, the electronic device 201 may control the VST device 204 based on the usage pattern of the VST device 204 learned in advance as described above.

FIG. 11 is a diagram illustrating an operation in which a wearable electronic device controls a VST device, according to an embodiment.

Referring to FIG. 11, according to one embodiment, the wearable electronic device 201 may control the VST device 204 without the electronic device 202. At this time, the wearable electronic device 201 may perform at least some of the operations performed by the electronic device 202 described above.

According to one embodiment, in operation 1101, the wearable electronic device 201 may check whether the user has entered a designated space through a GPS module (not shown).

According to one embodiment, in operation 1103, when the wearable electronic device 201 confirms that the wearable electronic device 201 enters the designated space, it can check the user's activity through the sensor 211.

According to one embodiment, in operation 1105, the wearable electronic device 201 may check the first area in the designated space where the wearable electronic device 201 is located. For example, when the user's activity information is confirmed, the wearable electronic device 201 may receive a communication signal from an access point located in a designated space through the communication module 218. The wearable electronic device 201 may obtain first communication signal information about the communication signal. The wearable electronic device 201 may perform clustering on the first communication signal information and confirm the clustered area corresponding to the first communication signal information. The wearable electronic device 201 may identify a first area where the wearable electronic device 201 is located among a plurality of areas included in the designated space based on the clustered area.

According to one embodiment, in operation 1107, the wearable electronic device 201 collects specific augmented reality information according to a user pattern (e.g., a usage pattern for the VST device 204) based on the user's activity and the first area. (or a specific function) can be determined. Additionally, in operation 1107, the wearable electronic device 201 may generate a command to cause the VST device 204 to display specific augmented reality information (or execute a specific function).

According to one embodiment, in operation 1109, the wearable electronic device 201 may transmit a command to the VST device 204. In operation 1111, the VST device 204 may display specific augmented reality information (or execute a specific function) based on the received command.

FIGS. 12A to 12D are diagrams for explaining an operation of a VST device to display augmented reality information, according to an embodiment.

Referring to FIGS. 12A to 12D, according to one embodiment, the wearable electronic device 201 and the VST device 204 may be worn by the user.

Referring to FIG. 12A, according to one embodiment, the wearable electronic device 201 may sense the keyboard typing operation performed by the user on the laptop 1210 and determine or confirm the user's activity through keyboard typing. . The wearable electronic device 201 can confirm that the user or the wearable electronic device 201 has entered or is located in the study. Additionally, the wearable electronic device 201 can obtain information about the laptop 1210.

According to one embodiment, the electronic device 201 may receive information about keyboard typing and information about the study from the wearable electronic device 201. The electronic device 201 may check the pre-learned usage pattern of the VST device 204 of the user based on the received information. For example, the electronic device 202 may learn in advance the pattern of displaying a specific augmented reality screen (e.g., virtual extended display screen) on the VST device 204 when the user performs work using the laptop 1210 in the study. You can. The electronic device 201 may transmit a command to display a specific augmented reality screen to the VST device 204 based on a previously learned usage pattern.

According to one embodiment, the VST device 204 may display the first screen 1201 based on an image captured through the camera 275 on the display 290. The VST device 204 can display a specific augmented reality screen 1215 on the first screen 1201. Alternatively, the VST device 204 may recommend to the user to display a specific augmented reality screen 1215. The VST device 204 may display a specific augmented reality screen 1215 on the first screen 1201 based on user input for the corresponding recommendation.

Referring to FIG. 12B, according to one embodiment, the wearable electronic device 201 may determine or confirm the user's activity as walking by sensing the walking motion performed by the user. The wearable electronic device 201 can confirm that the user or the wearable electronic device 201 has entered or is located in the study.

According to one embodiment, the electronic device 201 may receive information about walking and information about the study from the wearable electronic device 201. The electronic device 201 may check the pre-learned usage pattern of the VST device 204 of the user based on the received information. For example, the electronic device 202 may learn in advance a pattern of displaying a specific augmented reality screen (e.g., a world-famous library background screen) on the VST device 204 after the user enters the library. The electronic device 201 may transmit a command to display a specific augmented reality screen to the VST device 204 based on a previously learned usage pattern.

According to one embodiment, the VST device 204 may display the second screen 1202 based on the image captured through the camera 275 on the display 290. The VST device 204 can display a specific augmented reality screen 1225 on the second screen 1202. Alternatively, the VST device 204 may recommend to the user to display a specific augmented reality screen 1225. The VST device 204 may display a specific augmented reality screen 1225 on the second screen 1202 based on user input for the corresponding recommendation.

Referring to FIG. 12C, according to one embodiment, the VST device 204 may display augmented reality information by further considering information received from another external electronic device (or wearable electronic device).

According to one embodiment, the VST device 204 may receive information from a user's electronic device in the room. At this time, the user's electronic device in the room may transmit the corresponding information to the VST device 204 according to the user's request. Alternatively, the user's electronic device in the room may sense the user's activities (e.g., exercise, work, study, sleep) and automatically transmit the corresponding information to the VST device 204. For example, the VST device 204 may display augmented reality information 1235 indicating that there may be an obstruction on the visit 1230 included in the third screen 1203 based on the image captured through the camera 274. . Alternatively, the VST device 204 may display an image representing emotional information on the door 1230 .

Referring to FIG. 12D, according to one embodiment, the wearable electronic device 201 may confirm that the user or the wearable electronic device 201 enters the living room included in the designated space. The wearable electronic device 201 can obtain information about the nearby robot vacuum cleaner 1240 and TV 1245. The VST device 204 may display the fourth screen 1204 for the living room area based on the image captured through the camera 275.

According to one embodiment, the VST device 204 may display augmented reality information based on information received from the robot vacuum cleaner 1240. For example, information about the robot vacuum cleaner 1240 may be transmitted directly to the VST device 204 or may be transmitted to the VST device 204 through the wearable electronic device 201 or electronic device 202. For example, the part 1250 cleaned by the robot cleaner in a specific area can be displayed as augmented reality information. The VST device 204 may display an interface for controlling the robot vacuum cleaner. Based on user input on the interface displayed on VST device 204, VST device 204 may control the robot vacuum cleaner to resume cleaning portions of a particular area that have not yet been cleaned.

According to one embodiment, the electronic device 202, based on the user's activity and the first area obtained through the wearable electronic device 201, prevents the robot vacuum cleaner 1240 from entering a specific area. ) can be controlled. At this time, the electronic device 202 may transmit a command to the VST device 204 to display a guide informing the VST device 204 that the robot cleaner 1240 cannot enter a specific area. Additionally, the electronic device 202 may further obtain information about the user's status from the VST device 204 and control the robot cleaner based on the obtained information. For example, the electronic device 202 may control the robot cleaner 1240 to resume cleaning a specific area according to the user's activity and user's state.

The electronic device 202 according to one embodiment may include a GPS module 240, a communication module 250, a memory 230, and a processor 220. The processor according to one embodiment may be set to confirm that the electronic device enters a designated space including a plurality of areas through the GPS module. The processor according to one embodiment may be set to receive first communication signal information about a communication signal received by the wearable electronic device 201 from the wearable electronic device 201 through the communication module. The processor according to one embodiment may be set to perform clustering on the first communication signal information and confirm a clustered area corresponding to the first communication signal information. The processor according to an embodiment, through the communication module, sends information about the clustered area to the wearable electronic device so that the wearable electronic device can identify a first area in which the wearable electronic device is located among the plurality of areas. It can be set to transmit to the device. The processor according to one embodiment may be set to obtain information about the user's first activity and information about the first area from the wearable electronic device through the communication module. The processor according to one embodiment, based on the first activity information and information about the first area, sends a first command for augmented reality information of the head mounted display (HMD) device 204. It can be set to obtain. The processor according to one embodiment may be set to transmit the first command to the HMD device through the communication module so that the HMD device displays the augmented reality information through the display 290 of the HMD device. .

According to one embodiment, the processor may be set to check a pre-learned usage pattern for the HMD device based on the first activity and the first area.

According to one embodiment, the processor may be set to determine the augmented reality information based on the pre-learned usage pattern.

According to one embodiment, the processor may be set to confirm the clustered area by comparing the first communication signal information with a plurality of communication signal information stored in the memory.

According to one embodiment, the processor, if the similarity between any one of the at least one cluster obtained by clustering the plurality of communication signal information and the first communication signal information is greater than a threshold, the one It may be set to determine an area corresponding to a cluster as the clustered area.

According to one embodiment, the processor may be set to store the first location information in the memory. According to one embodiment, the processor may be set to divide the designated space into at least one area by clustering a plurality of communication signal information including the first communication signal information stored in the memory.

According to one embodiment, the processor may be set to determine the at least one area by further performing silhouette analysis and/or similarity analysis on the plurality of clustered communication signal information.

According to one embodiment, the processor may be set to obtain information about at least one device located around the wearable electronic device from the wearable electronic device through the communication module. According to one embodiment, the processor may be set to obtain information about the user's activity sensed by the wearable electronic device in the at least one area. According to one embodiment, the processor may be set to label the at least one area based on information about the activity and information about the at least one device.

According to one embodiment, when it is confirmed by the wearable electronic device that the user is located in the first area while performing the first activity, the processor processes the first application executed on the HMD device from the HMD device. It can be set to obtain information about. According to one embodiment, the processor may be set to learn a usage pattern of the HMD device by the user based on the first activity, the first area, and the first application.

According to one embodiment, the processor may be set to determine the augmented reality information related to the first application based on the first activity and the first area.

A method of operating an electronic device 202 according to an embodiment includes confirming that the electronic device enters a designated space including a plurality of areas through the GPS module 240 included in the electronic device. can do. A method of operating an electronic device according to an embodiment includes first communication signal information about a communication signal received by the wearable electronic device 201 from a wearable electronic device 201 through a communication module 250 included in the electronic device. It may include a receiving operation. A method of operating an electronic device according to an embodiment may include performing clustering on the first communication signal information and confirming a clustered area corresponding to the first communication signal information. A method of operating an electronic device according to an embodiment includes, through the communication module, providing information about the clustered areas so that the wearable electronic device identifies a first area in which the wearable electronic device is located among the plurality of areas. It may include transmitting to the wearable electronic device. A method of operating an electronic device according to an embodiment may include obtaining information about a user's first activity and information about the first area from the wearable electronic device through the communication module. A method of operating an electronic device according to an embodiment includes, based on the first activity information and information about the first area, augmented reality information of a head mounted display (HMD) device 204. It may include an operation of obtaining a first command. A method of operating an electronic device according to an embodiment includes transmitting the first command to the HMD device through the communication module so that the HMD device displays the augmented reality information through the display 290 of the HMD device. may include.

According to one embodiment, the operation of obtaining the first command may include the operation of checking a pre-learned usage pattern for the HMD device based on the first area and the first activity.

According to one embodiment, the operation of obtaining the first command may include the operation of determining the augmented reality information based on the pre-learned usage pattern.

According to one embodiment, the operation of checking the clustered area may include an operation of checking the clustered area by comparing the first communication signal information with a plurality of communication signal information stored in the memory.

According to one embodiment, the operation of checking the clustered area is performed when the similarity between any one of the at least one cluster obtained by clustering the plurality of communication signal information and the first communication signal information is greater than a threshold. If it is large, it may include an operation of determining an area corresponding to one of the clusters as the clustered area.

According to one embodiment, the method of operating the electronic device may further include storing the first location information in the memory. According to an embodiment, the method of operating the electronic device further includes dividing the designated space into at least one area by clustering a plurality of communication signal information including the first communication signal information stored in the memory. can do.

According to one embodiment, the operation of dividing the designated space into the at least one area includes further performing silhouette analysis and/or similarity analysis on the plurality of clustered communication signal information to determine the at least one area. It may include an operation to determine one area.

According to one embodiment, the method of operating the electronic device may further include obtaining information about at least one device located around the wearable electronic device from the wearable electronic device through the communication module. . According to one embodiment, the method of operating the electronic device may further include obtaining information about the user's activity sensed by the wearable electronic device in the at least one area. According to one embodiment, the method of operating the electronic device may further include labeling the at least one area based on information about the activity and information about the at least one device.

According to one embodiment, the method of operating the electronic device includes, when the wearable electronic device determines that the user is located in the first area while performing the first activity, the method is executed on the HMD device. It may further include an operation of obtaining information about the first application. According to one embodiment, the method of operating the electronic device may further include learning a usage pattern of the HMD device by the user based on the first activity, the first area, and the first application. You can.

The wearable electronic device 201 according to one embodiment may include a sensor 211, a GPS module, a communication module 218, and a processor 212. The processor according to one embodiment may be set to confirm that the electronic device enters a designated space including a plurality of areas through the GPS module. The processor according to one embodiment may be set to check the user's activity information through the sensor. The processor according to one embodiment may be set to obtain first communication signal information about the communication signal received through the communication module when the activity information is confirmed. The processor according to one embodiment may be set to perform clustering on the first communication signal information and confirm a clustered area corresponding to the first communication signal information. The processor according to an embodiment may be set to identify a first area in which the wearable electronic device is located among the plurality of areas based on the clustered area. The processor according to one embodiment, based on the first activity information and information about the first area, sends a first command for augmented reality information of the head mounted display (HMD) device 204. It can be set to obtain. The processor according to one embodiment may be set to transmit the first command to the HMD device through the communication module so that the HMD device displays the augmented reality information through the display 290 of the HMD device. .

Electronic devices according to various embodiments disclosed in this document may be of various types. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of this document are not limited to the above-described devices.

The various embodiments of this document and the terms used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various changes, equivalents, or replacements of the embodiments. In connection with the description of the drawings, similar reference numbers may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the above items, unless the relevant context clearly indicates otherwise. As used herein, “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “A Each of phrases such as “at least one of , B, or C” may include any one of the items listed together in the corresponding phrase, or any possible combination thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish one component from another, and to refer to those components in other respects (e.g., importance or order) is not limited. One (e.g., first) component is said to be “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicatively.” When mentioned, it means that any of the components can be connected to the other components directly (e.g. wired), wirelessly, or through a third component.

The term “module” used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as logic, logic block, component, or circuit, for example. It can be used as A module may be an integrated part or a minimum unit of the parts or a part thereof that performs one or more functions. For example, according to one embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document are one or more instructions stored in a storage medium (e.g., built-in memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). It may be implemented as software (e.g., program 140) including these. For example, a processor (e.g., processor 120) of a device (e.g., electronic device 101) may call at least one command among one or more commands stored from a storage medium and execute it. This allows the device to be operated to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves). This term refers to cases where data is stored semi-permanently in the storage medium. There is no distinction between cases where it is temporarily stored.

According to one embodiment, methods according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)), or through an application store (e.g. Play Store ^TM ) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smart phones) or online. In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

According to various embodiments, each component (e.g., module or program) of the above-described components may include a single or plural entity, and some of the plurality of entities may be separately placed in other components. there is. According to various embodiments, one or more of the components or operations described above may be omitted, or one or more other components or operations may be added. Alternatively or additionally, multiple components (eg, modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component of the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, or omitted. Alternatively, one or more other operations may be added.

201: Wearable electronic devices
202: Electronic device
204: VST device (or HMD device)

Claims (20)

In the electronic device 202,
GPS module 240;
Communication module 250;
memory(230); and
Includes a processor 220, where the processor:
Through the GPS module, the electronic device confirms that it enters a designated space including a plurality of areas,
Receiving first communication signal information about a communication signal received by the wearable electronic device 201 from the wearable electronic device 201 through the communication module,
Clustering is performed on the first communication signal information to identify a clustered area corresponding to the first communication signal information,
Through the communication module, information about the clustered area is transmitted to the wearable electronic device so that the wearable electronic device identifies a first area in which the wearable electronic device is located among the plurality of areas,
Obtain information about the user's first activity and information about the first area from the wearable electronic device through the communication module,
Based on the first activity information and the information about the first area, obtain a first command for augmented reality information of a head mounted display (HMD) device 204,
An electronic device configured to transmit the first command to the HMD device through the communication module so that the HMD device displays the augmented reality information through a display (290) of the HMD device. The method of claim 1, wherein the processor:
An electronic device configured to check a pre-learned usage pattern for the HMD device based on the first activity and the first area. The method of any one of claims 1 to 2, wherein the processor:
An electronic device configured to determine the augmented reality information based on the pre-learned usage pattern. The method of any one of claims 1 to 3, wherein the processor:
An electronic device configured to confirm the clustered area by comparing the first communication signal information with a plurality of communication signal information stored in the memory. The method of any one of claims 1 to 4, wherein the processor:
If the similarity between any one of the at least one cluster obtained by clustering the plurality of communication signal information and the first communication signal information is greater than a threshold, the area corresponding to the one cluster is selected as the clustered area. An electronic device set to make decisions. The method of any one of claims 1 to 5, wherein the processor:
Store the first location information in the memory,
An electronic device configured to divide the designated space into at least one area by clustering a plurality of communication signal information including the first communication signal information stored in the memory. The method of any one of claims 1 to 6, wherein the processor:
An electronic device configured to determine the at least one area by further performing silhouette analysis and/or similarity analysis on the plurality of clustered communication signal information. The method of any one of claims 1 to 7, wherein the processor:
Obtain information about at least one device located around the wearable electronic device from the wearable electronic device through the communication module,
Obtaining information about the user's activity sensed by the wearable electronic device in the at least one area,
An electronic device configured to label the at least one area based on the information about the activity and the information about the at least one device. The method of any one of claims 1 to 8, wherein the processor:
When it is confirmed by the wearable electronic device that the user is located in the first area while performing the first activity, obtain information about the first application executed on the HMD device from the HMD device,
An electronic device configured to learn a usage pattern of the HMD device by the user based on the first activity, the first area, and the first application. The method of any one of claims 1 to 9, wherein the processor:
An electronic device configured to determine the augmented reality information related to the first application based on the first activity and the first area. In a method of operating an electronic device 202,
Confirming that the electronic device enters a designated space including a plurality of areas through the GPS module 240 included in the electronic device;
An operation of receiving first communication signal information about a communication signal received by the wearable electronic device from the wearable electronic device 201 through a communication module 250 included in the electronic device;
An operation of performing clustering on the first communication signal information and confirming a clustered area corresponding to the first communication signal information;
transmitting, through the communication module, information about the clustered areas to the wearable electronic device so that the wearable electronic device identifies a first area in which the wearable electronic device is located among the plurality of areas;
Obtaining information about a user's first activity and information about the first area from the wearable electronic device through the communication module;
Obtaining a first command for augmented reality information of a head mounted display (HMD) device 204 based on the first activity information and information about the first area; and
A method of operating an electronic device, including transmitting the first command to the HMD device through the communication module so that the HMD device displays the augmented reality information through the display 290 of the HMD device. The method of claim 11, wherein the operation of obtaining the first command includes:
A method of operating an electronic device further comprising checking a pre-learned usage pattern for the HMD device based on the first area and the first activity. The method according to any one of claims 11 to 12, wherein the operation of obtaining the first command includes:
A method of operating an electronic device further comprising determining the augmented reality information based on the pre-learned usage pattern. The method of any one of claims 11 to 13, wherein the operation of confirming the clustered area includes:
A method of operating an electronic device comprising comparing the first communication signal information with a plurality of communication signal information stored in the memory to confirm the clustered area. The method of any one of claims 11 to 14, wherein the operation of checking the clustered area includes:
If the similarity between any one of the at least one cluster obtained by clustering the plurality of communication signal information and the first communication signal information is greater than a threshold, the area corresponding to the one cluster is selected as the clustered area. A method of operating an electronic device including the operation of determining. According to any one of claims 11 to 15,
storing the first location information in the memory; and
A method of operating an electronic device further comprising dividing the designated space into at least one area by clustering a plurality of communication signal information including the first communication signal information stored in the memory. The method of any one of claims 11 to 16, wherein the operation of dividing the designated space into the at least one area includes:
A method of operating an electronic device comprising determining the at least one area by further performing silhouette analysis and/or similarity analysis on the plurality of clustered communication signal information. According to any one of claims 11 to 17,
Obtaining information about at least one device located around the wearable electronic device from the wearable electronic device through the communication module;
Obtaining information about the user's activity sensed by the wearable electronic device in the at least one area; and
A method of operating an electronic device further comprising labeling the at least one area based on the information about the activity and the information about the at least one device. According to any one of claims 11 to 18,
When the wearable electronic device determines that the user is located in the first area while performing the first activity, obtaining information about a first application executed on the HMD device from the HMD device; and
A method of operating an electronic device further comprising learning a usage pattern of the HMD device by the user based on the first activity, the first area, and the first application. In the wearable electronic device 201,
sensor 211;
GPS module;
communication module 218; and
Comprising a processor 212, wherein the processor:
Through the GPS module, the electronic device confirms that it enters a designated space including a plurality of areas,
Through the sensor, the user's activity information is checked,
When the activity information is confirmed, obtain first communication signal information about the communication signal received through the communication module,
Clustering is performed on the first communication signal information to identify a clustered area corresponding to the first communication signal information,
Identifying a first area where the wearable electronic device is located among the plurality of areas based on the clustered area,
Based on the first activity information and the information about the first area, obtain a first command for augmented reality information of a head mounted display (HMD) device 204,
A wearable electronic device configured to transmit the first command to the HMD device through the communication module so that the HMD device displays the augmented reality information through the display 290 of the HMD device.

Images