KR20230029487A - Electronic device for controlling external devices and operating method thereof - Google Patents

Abstract

An electronic device according to one embodiment comprises: a communication module which performs communications with a remote control device through one or more antennas; a sensor module which measures an azimuth (yaw) of the electronic device; a memory which stores computer-executable instructions; and a processor which accesses the memory to execute instructions. Instructions of the electronic device, according to one embodiment, may be configured to: determine a relative location information of the remote control device for the electronic device; based on the relative location information, the azimuth information of the electronic device measured by the sensor module, and azimuth information of the remote control device, calculate a coordinate of the remote control device and an angle (yaw) in an orientation of the remote control device on a global coordinate system having the electronic device as a reference; and based on the coordinate and the angle, determine a control target device which is to be controlled by the remote control device. Other various embodiments may be provided. An electronic device, which sets a global coordinate system to obtain information on an orientation of a remote control device, may be provided.

Description

Electronic device for controlling external electronic devices and its operation method

The following embodiments relate to an electronic device that controls an external electronic device and an operating method thereof.

UWB communication is a communication technology that transmits signals using very short pulses (several nanoseconds) with low power over a wide band compared to conventional communication. In the past, it was used for military purposes such as military radar and remote detection, but since the U.S. Federal Communications Commission (FCC) permitted commercial use by limiting it to the field of indoor wireless communication in 2002, it has expanded its activities in various fields. Through UWB communication, the time of arrival (TOA), which is the time the pulse arrives at the target, and the angle of arrival (AOA) of the pulse at the transmitting device can be accurately measured, and accordingly, precise measurements with an error of several tens of cm can be made indoors. Distance and location recognition became possible.

In an apparatus for determining a device to be controlled according to a pointing direction using UWB indoors, a device to be controlled may be determined according to a pointing direction of a remote control device. In order to obtain information about the pointing direction of the remote control device, a virtual coordinate system may be established.

According to various embodiments of the present disclosure, an electronic device may be provided that obtains information about a pointing direction of a remote control device by setting a global coordinate system based on a reference electronic device in an indoor environment.

An electronic device according to an embodiment includes a communication module that communicates with a remote control device through one or more antennas; a sensor module for measuring a yaw of the electronic device; a memory in which computer-executable instructions are stored; and a processor accessing the memory to execute the commands, wherein the commands determine relative location information of the remote control device with respect to the electronic device, and the relative location information is measured by the sensor module. The coordinates of the remote control device based on the azimuth information of the electronic device and the azimuth information of the remote control device and the angle of the orientation direction of the remote control device on a global coordinate system based on the electronic device. (yaw) may be calculated, and a control target device to be controlled by the remote control apparatus may be determined based on the coordinates and the angle.

According to an embodiment, the instructions may be further configured to control the controlling device through the remote control apparatus when the controlling device is determined.

According to an embodiment, the commands may include the angle (yaw) of the pointing direction of the remote control device, the pitch of the remote control device, and the roll of the remote control device on the global coordinate system. It may be further configured to correct orientation information of the remote control device based on a roll.

According to an embodiment, the relative location information may include information about a distance from the remote control device to the electronic device; and information about a direction from the remote control device to the electronic device and an angle generated in the pointing direction of the remote control device.

According to an embodiment, the relative location information of the remote control device with respect to the electronic device may include information about a distance from each of a plurality of antennas included in the remote control device to the electronic device; And it may be determined based on information about the distance between the plurality of antennas.

According to an embodiment, the azimuth (yaw) of the remote control device may be measured through a sensor module included in the remote control device.

According to an embodiment, the electronic device may perform ultra wide band (UWB) communication with the remote control device.

According to an embodiment, the commands are based on the coordinates of the remote control device on the global coordinate system based on the electronic device, the angle of the orientation direction, and a pre-registered detectable area. It may be configured to determine the control target device.

According to an embodiment, the remote control device may include two antennas, and the directing direction of the remote control device may be determined as a direction perpendicular to a straight line connecting the two antennas.

An operating method of an electronic device according to an embodiment includes determining relative location information of a remote control device with respect to the electronic device; Based on the relative position information, the azimuth information of the electronic device, and the azimuth information of the remote control device, the coordinates of the remote control device and the coordinates of the remote control device on a global coordinate system based on the electronic device. calculating an angle (yaw) of the direction of orientation; and determining a control target device to be controlled by the remote control apparatus based on the coordinates and the angle.

According to an embodiment, when the control target device is determined, the method may further include controlling the control target device through the remote control apparatus.

According to an embodiment, the method may include the angle (yaw) of the pointing direction of the remote control device, the pitch of the remote control device, and the roll of the remote control device on the global coordinate system. An operation of correcting orientation information of the remote control device based on a roll may be further included.

According to an embodiment, the relative location information may include information about a distance from the remote control device to the electronic device; and information about a direction from the remote control device to the electronic device and an angle generated in the pointing direction of the remote control device.

According to an embodiment, the relative location information of the remote control device with respect to the electronic device may include information about a distance from each of a plurality of antennas included in the remote control device to the electronic device; And it may be determined based on information about the distance between the plurality of antennas.

According to an embodiment, the azimuth (yaw) of the remote control device may be measured through a sensor module included in the remote control device.

According to an embodiment, the electronic device may perform ultra wide band (UWB) communication with the remote control device.

According to an embodiment, the operation of determining the control target device may include the coordinates of the remote control device on the global coordinate system based on the electronic device, the angle of the pointing direction, and pre-registered An operation of determining based on the detectable region may be included.

According to an embodiment, the remote control device may include two antennas, and the directing direction of the remote control device may be determined as a direction perpendicular to a straight line connecting the two antennas.

According to an embodiment, a computer-readable recording medium in which a program for controlling an operation of an electronic device is recorded includes an operation of determining relative position information of a remote control device with respect to the electronic device; Based on the relative position information, the azimuth information of the electronic device, and the azimuth information of the remote control device, the coordinates of the remote control device and the coordinates of the remote control device on a global coordinate system based on the electronic device. calculating an angle (yaw) of the direction of orientation; and a program for performing an operation of determining a control target device to be controlled by the remote control apparatus based on the coordinates and the angle may be recorded.

According to an embodiment, the operation of determining the control target device may include the coordinates of the remote control device on the global coordinate system based on the electronic device, the angle of the pointing direction, and pre-registered A program may be recorded that includes an operation of determining based on a detectable region, and causes the electronic device to further control the control target device through the remote control device when the control target device is determined. there is.

According to various embodiments of the present disclosure, an electronic device may be provided that obtains information about a pointing direction of a remote control device by setting a global coordinate system based on a reference electronic device in an indoor environment.

According to various embodiments, an electronic device that controls a control target device through a remote control device may be provided without a separate space setting process for determining the location of the remote control device in an indoor environment.

In addition to this, various effects identified directly or indirectly through this document may be provided.

1A and 1B are block diagrams of a reference electronic device according to an embodiment.
2A and 2B are diagrams for explaining a remote control device according to an exemplary embodiment.
3 is a diagram for explaining an indoor environment in which a reference electronic device, one or more devices, and a remote control device for controlling the devices are located, according to an embodiment.
4 is a diagram for explaining a method of measuring a distance between a reference electronic device and a remote control device according to an exemplary embodiment.
5A to 5C are diagrams for explaining an operation of determining relative position information of a remote control device with respect to a reference electronic device according to an embodiment.
6A and 6B are diagrams for explaining an operation of determining a directing direction of a remote control device on a global coordinate system based on a reference electronic device according to an exemplary embodiment.
7 is a flowchart illustrating an operation of a reference electronic device according to an exemplary embodiment.
8 is a flowchart illustrating an operation of determining relative position information of a remote control device with respect to a reference electronic device according to an exemplary embodiment.
9A to 9C are diagrams for explaining an operation of determining a directing direction of a remote control device on a global coordinate system based on a reference electronic device according to another embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same reference numerals are given to the same components regardless of reference numerals, and overlapping descriptions thereof will be omitted.

< Reference electronic device >

1A and 1B are block diagrams of a reference electronic device according to an embodiment.

As shown in FIG. 1A , a reference electronic device 100 according to an embodiment may include a memory 120, a processor 130, a communication unit 150, and a sensing unit 191. However, not all illustrated components are essential components. The reference electronic device 100 may be implemented with more or fewer components than illustrated components.

For example, as shown in FIG. 1B , the reference electronic device 100 according to an embodiment includes a memory 120, a processor 130, a communication unit 150, a sensing unit 191, and a display 110 ), a tuner unit 140, a sensing unit 160, an input/output unit 170, a video processing unit 180, an audio processing unit 115, an audio output unit 126, and a power supply unit 190. there is.

Hereinafter, we look at the above components.

The processor 130 controls overall operation of the reference electronic device 100 and signal flow between internal components of the reference electronic device 100 and processes data. The processor 130 may execute an operation system (OS) and various applications stored in the memory 120 when there is a user's input or when a pre-set and stored condition is satisfied.

The processor 130 stores signals or data input from the outside of the reference electronic device 100, or RAM used as a storage area corresponding to various tasks performed in the reference electronic device 100, the reference electronic device 100 It may include a ROM and a processor in which a control program for control of is stored.

The processor 130 may include a graphic processing unit (not shown) for graphic processing corresponding to video. The processor 130 may be implemented as a system on chip (SoC) in which a core (not shown) and a GPU (not shown) are integrated. The processor 130 may include a single core, a dual core, a triple core, a quad core, and multiple cores thereof.

Also, the processor 130 may include a plurality of processors. For example, the processor may be implemented as a main processor (not shown) and a sub processor (not shown) operating in a sleep mode.

According to an embodiment, the processor 130 executes one or more instructions stored in the memory 120, and through the sensing unit 191 including the at least one sensor, at least one sensor corresponding to the at least one sensor. A sensing value can be detected.

In addition, according to an embodiment, the processor 130 executes one or more instructions stored in the memory 120, and when it is determined that at least one detected sensing value is equal to or greater than a preset threshold value, the remote control device 200 ) may determine that the reference electronic device 100 has been touched.

Also, according to an embodiment, the processor 130 executes one or more instructions stored in the memory 120, thereby at least one sensed value detected and the sensed value of the remote control device received from the remote control device 200. By comparing , it may be determined that the remote control device 200 has touched the reference electronic device 100 .

Also, according to an embodiment, the processor 130 requests identification information of the remote control device 200 by executing one or more instructions stored in the memory 120, and sends the identification information of the remote control device 200. can receive Also, the processor 130 may confirm that the remote control device 200 is a device pre-registered in the reference electronic device 100 based on the identification information of the remote control device 200 .

Also, according to an embodiment, the processor 130 executes one or more instructions stored in the memory 120 so that the remote control device 200 is placed on the reference electronic device 100 based on at least one sensed value. It is possible to determine a touch area touched on .

Also, according to an embodiment, the processor 130 compares at least one sensing value detected corresponding to at least one sensor by executing one or more instructions stored in the memory 120, and based on the comparison result , One or more sensors determined to be disposed close to a point where the remote control device 200 is touched on the reference electronic device 100 may be determined. Also, the processor 130 may determine the touch area based on the determined one or more sensors.

Also, according to an embodiment, the processor 130 may receive state information about an operation being executed in the remote control device 200 from the remote control device 200 through the communication unit 150 .

In addition, according to an embodiment, the processor 130 executes one or more instructions stored in the memory 120, based on the state information received on the remote control device 200, to set a predetermined touch area corresponding to the determined touch area. function can be performed.

Also, according to an embodiment, the processor 130 executes one or more instructions stored in the memory 120 so that the remote control device 200 is placed on the reference electronic device 100 based on at least one sensed value. It is possible to determine the number of touches touched on .

Also, according to an embodiment, the processor 130 may perform a preset function corresponding to the number of touches by executing one or more instructions stored in the memory 120 .

Also, according to an embodiment, the processor 130 executes one or more instructions stored in the memory 120 to preliminarily perform a function corresponding to at least one touch area on the reference electronic device 100 based on a user input. can be set

Also, the processor 130 may preset a function corresponding to the number of touches performed on at least one touch region on the reference electronic device 100 based on a user input by executing one or more instructions stored in the memory 120 . can

The memory 120 may store various data, programs, or applications for driving and controlling the reference electronic device 100 under the control of the processor 130 . The memory 120 includes a video processing unit 180, a display 110, an audio processing unit 115, an audio output unit 126, a power supply unit 130, a tuner unit 140, a communication unit 150, and a sensing unit 160. , input/output signals or data corresponding to driving of the input/output unit 170 may be stored.

The memory 120 includes an operating system 121 for controlling the reference electronic device 100 and the processor 130, an application 122 initially provided by a manufacturer or downloaded from the outside, a graphical user interface (GUI) related to the application, Objects for providing a GUI (eg, image text, icons, buttons, etc.), user information, documents, databases, or related data may be stored.

In addition, the memory 120 receives an input signal from the remote control device 200 and controls a channel corresponding to the input signal accordingly, or switches to a channel scroll user interface mode when the input signal corresponds to a predetermined input. A TV viewer module 123 including one or more instructions for entering, a character recognition module 124 including one or more instructions for recognizing information from content received from an external device (not shown), an external device (not shown) ) may include an MBR module 125 that includes one or more instructions for channel control from .

The memory 120 includes ROM, RAM, or a memory card (eg, micro SD card, USB memory, not shown) mounted in the reference electronic device 100 . Also, the memory 120 may include a non-volatile memory, a volatile memory, a hard disk drive (HDD), or a solid state drive (SSD).

According to an embodiment, the memory 120 may be a flash memory type, a hard disk type, a multimedia card micro type, or a card type memory (eg SD or XD memory, etc.), RAM (RAM, Random Access Memory), SRAM (Static Random Access Memory), ROM (ROM, Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory) ), a magnetic memory, a magnetic disk, and an optical disk may include at least one type of storage medium.

The display 110 displays a video included in a broadcast signal received through the tuner unit 140 on the screen under the control of the processor 130 . Also, the display 110 may display content (eg, video) input through the communication unit 150 or the input/output unit 170 . The display 110 may output an image stored in the memory 120 under the control of the processor 130 .

The display 110 generates driving signals by converting image signals, data signals, OSD signals, and control signals processed by the processor 130 . The display 110 may be implemented with a plasma display panel (PDP), a liquid crystal display (LCD), an organic light emitting diode (OLED), a cathode ray tube (CRT), a flexible display, and the like. It can be implemented as a 3D display. In addition, the display 110 may be configured as a touch screen and used as an input device in addition to an output device.

The tuner unit 140 determines the frequency of a channel desired to be received by the reference electronic device 100 among many radio wave components through amplification, mixing, resonance, etc. of a broadcast signal received by wire or wirelessly. You can select only by tuning. The broadcast signal includes audio, video, and additional information (eg, Electronic Program Guide (EPG)).

The tuner unit 140 controls a channel according to a user input (eg, a control signal received from the remote control device 200, for example, a channel number input, a channel up-down input, and a channel input on an EPG screen). A broadcast signal can be received in a frequency band corresponding to the number.

The tuner unit 140 may receive broadcast signals from various sources such as terrestrial broadcasting, cable broadcasting, satellite broadcasting, and Internet broadcasting. The tuner unit 140 may receive a broadcasting signal from a source such as analog broadcasting or digital broadcasting. The broadcast signal received through the tuner unit 140 is decoded (eg, audio decoding, video decoding, or additional information decoding) and separated into audio, video, and/or additional information. The separated audio, video and/or additional information may be stored in the memory 120 under the control of the processor 130 .

The tuner unit 140 of the reference electronic device 100 may be one or plural. The tuner unit 140 is implemented as an all-in-one with the reference electronic device 100, or a separate device having a tuner unit electrically connected to the reference electronic device 100 (for example, a set-top box). (set-top box, not shown), and a tuner unit (not shown) connected to the input/output unit 170).

The communication unit 150 may connect the reference electronic device 100 with an external device (eg, an audio device) (not shown) under the control of the processor 130 . The processor 130 may transmit/receive content to/from an external device (not shown) connected through the communication unit 150, download an application from the external device (not shown), or perform web browsing. The communication unit 150 may include one of wireless LAN 151 , Bluetooth 152 , and wired Ethernet 153 corresponding to the performance and structure of the reference electronic device 100 . Also, the communication unit 150 may include a combination of wireless LAN 151, Bluetooth 152, and wired Ethernet 153.

Also, the communication unit 150 may receive a control signal of the remote control device 200 under control of the processor 130 . The control signal may be implemented as a Bluetooth type, an RF signal type, or a Wi-Fi type.

In addition, the communication unit 150 may perform other short-range communication (eg, near field communication (NFC), bluetooth low energy (BLE), UWB communication) other than Bluetooth.

According to an embodiment, the communication unit 150 may include an antenna for performing communication. For example, the reference electronic device 100 may include a UWB antenna for performing UWB communication. According to embodiments, the communication unit 150 of the reference electronic device 100 may include one or more antennas.

The sensing unit 160 detects a user's voice, a user's video, or a user's interaction, and may include a microphone 161, a camera unit 162, and a light receiving unit 163.

The microphone 161 receives the user's utterance. The microphone 161 may convert the received voice into an electrical signal and output it to the processor 130 . The user's voice may include, for example, a voice corresponding to a menu or function of the reference electronic device 100 .

The camera unit 162 may obtain an image bezel such as a still image or a moving image. Images captured through the image sensor may be processed through the processor 130 or a separate image processing unit (not shown).

The image bezel processed by the camera unit 162 may be stored in the memory 120 or transmitted to the outside through the communication unit 150 . Two or more camera units 162 may be provided according to the configuration of the reference electronic device 100 .

The light receiving unit 163 receives an optical signal (including a control signal) received from the external remote control device 200 . The light receiving unit 163 may receive an optical signal corresponding to a user input (eg, touch, pressure, touch gesture, voice, or motion) from the remote control device 200 . A control signal may be extracted from the received optical signal under the control of the processor 130 . For example, the light receiving unit 163 may receive a control signal corresponding to a channel up/down button for channel switching from the remote control device 200 .

The input/output unit 170 receives video (eg, video), audio (eg, voice, music, etc.) and additional information from the outside of the reference electronic device 100 under the control of the processor 130. (eg, EPG, etc.) and the like are received. The input/output unit 170 includes at least one of a High-Definition Multimedia Interface port (HDMI) port 171, a component jack 172, a PC port 173, and a USB port 174. can do. The input/output unit 170 may include a combination of at least one of an HDMI port 171 , a component jack 172 , a PC port 173 , and a USB port 174 . An external video providing device (not shown) may be connected through the HDMI port 171 .

The video processor 180 processes video data received by the reference electronic device 100 . The video processing unit 180 may perform various image processing such as decoding, scaling, noise filtering, bezel rate conversion, and resolution conversion on video data.

The graphic processing unit 181 uses a calculation unit (not shown) and a rendering unit (not shown) to create a screen including various objects such as icons, images, and text. The calculation unit (not shown) calculates attribute values such as coordinate values, shapes, sizes, colors, and the like of each object to be displayed according to the layout of the screen using the user input sensed through the sensing unit 160 . The rendering unit (not shown) creates screens of various layouts including objects based on the attribute values calculated by the calculation unit (not shown). The screen created by the rendering unit (not shown) is displayed within the display area of the display 110 .

The audio processing unit 115 processes audio data. The audio processor 115 may perform various processes such as decoding or amplifying audio data and filtering noise. Meanwhile, the audio processing unit 115 may include a plurality of audio processing modules to process audio corresponding to a plurality of contents.

The audio output unit 126 outputs audio included in the broadcast signal received through the tuner unit 140 under the control of the processor 130 . The audio output unit 126 may output audio (eg, voice, sound) input through the communication unit 150 or the input/output unit 170 . Also, the audio output unit 126 may output audio stored in the memory 120 under the control of the processor 130 . The audio output unit 126 may include at least one of a speaker 127, a headphone output terminal 128, and a Sony/Philips Digital Interface (S/PDIF) output terminal 129. The audio output unit 126 may include a combination of at least one of a speaker 127 , a headphone output terminal 128 , and a S/PDIF output terminal 129 .

The power supply unit 190 supplies power input from an external power source to components inside the reference electronic device 100 under the control of the processor 130 . In addition, the power supply unit 190 may supply power output from one or more batteries (not shown) located inside the reference electronic device 100 to internal components under the control of the processor 130 .

The sensing unit 191 may detect a state of the reference electronic device 100 or a state around the reference electronic device 100 and transmit the sensed information to the processor 130 .

The sensing unit 191 includes a magnetic sensor 192, an acceleration sensor 193, a temperature/humidity sensor 194, an infrared sensor 195, a gyroscope sensor 196, and a position sensor. (eg, a GPS) 197, an air pressure sensor 198, a proximity sensor 199, and an RGB sensor (illuminance sensor) 201, but may include at least one, but is not limited thereto.

According to an embodiment, the sensing unit 191 may include a hall sensor (not shown). The Hall sensor may be a transducer that generates an electrical signal (eg, voltage) in response to a magnetic field. The hall sensor may generate an electrical signal of relatively high intensity when the intensity of the magnetic field is high, and may generate an electrical signal of relatively low intensity when the intensity of the magnetic field is low. The processor 130 may receive an electrical signal according to magnetic field detection from a hall sensor.

According to an embodiment, the sensing unit 191 may be an electronic compass (E-compass) capable of detecting the angle of the reference electronic device 100 using the geomagnetic sensor 192, the acceleration sensor 193, and the hall sensor, or It may include a fluxgate compass. For example, the geomagnetic sensor 192 may measure the direction using a magnetic field and lines of force, and the hall sensor may measure the angle of the reference electronic device 100 by detecting the strength of the magnetic field.

According to an embodiment, the processor 130 may measure the yaw, pitch, and roll values of the reference electronic device 100 through motion data obtained from the sensing unit 191. . Motion data according to an embodiment includes 3-axis motion data (x1, y1, z1) obtained from the acceleration sensor 193 or obtained by additionally using the gyroscope sensor 196 and the geomagnetic sensor 192. It can include 9-axis motion data.

According to an embodiment, the processor 130 may obtain yaw, pitch, and/or roll value information measured from 9-axis motion data.

The sensing unit 191 according to an embodiment may detect an external impact applied to the reference electronic device 100 . For example, when the remote control device 200 touches the reference electronic device 100, the sensing unit 191 of the reference electronic device 100 may output a sensed value.

Also, the reference electronic device 100 including the display 110 may be electrically connected to a separate external device (eg, a set-top box, not shown) including the tuner unit 140 .

In addition, the reference electronic device 100 may be implemented as an analog TV, digital TV, 3D-TV, smart TV, LED TV, OLED TV, plasma TV, monitor, etc., but is not limited thereto. It will be easily understood by those who have knowledge.

Meanwhile, the block diagram of the illustrated reference electronic device 100 is a block diagram for one embodiment. Each component of the block diagram may be integrated, added, or omitted according to specifications of the reference electronic device 100 that is actually implemented. That is, if necessary, two or more components may be combined into one component, or one component may be subdivided into two or more components. In addition, the functions performed in each block are for explaining the embodiments, and the specific operation or device does not limit the scope of the present invention.

< Remote control device >

2A and 2B are diagrams for explaining a remote control device according to an exemplary embodiment.

Referring to FIG. 2A , a configuration diagram of a remote control device 200 according to an embodiment is shown. According to an embodiment, the remote control device 200 includes a plurality of antennas (eg, a first antenna 251 and a second antenna 251) for performing communication (eg, UWB communication) with the reference electronic device 100. An antenna 253) may be included.

According to an embodiment, the directing direction 255 of the remote control device 200 may be determined based on the first antenna 251 and the second antenna 253. According to an embodiment, when viewing an indoor environment in which the reference electronic device 100 and the remote control device 200 operate from above, the indoor environment may be expressed as a plane (eg, an X-Y plane viewed from the +Z axis). and the directing direction 255 of the remote control device 200 may be determined as one direction on a plane.

According to one embodiment, the directing direction 255 may be determined in a direction perpendicular to the straight line 257 corresponding to the first antenna 251 and the second antenna 253, but is not limited thereto, and the straight line 257 It may be a direction of a certain angle ψ _d with According to one embodiment, the angle ψ _d may be about 90°.

Referring to FIG. 2B , a configuration diagram of a remote control device 200 according to an embodiment is shown. According to one embodiment, the remote control device 200 may include at least some of the device configurations disclosed in FIGS. 1A and 1B.

As shown in FIG. 2B , the remote control device 200 according to an embodiment may include a display 210, a memory 220, a communication unit 250, a sensing unit 291, and a processor 230. . However, not all of the components shown in FIG. 2B are essential components of the remote control device 200 . The remote control device 200 may be implemented with more or less components than those shown in FIG. 2B.

According to an embodiment, the display 210 of the remote control device 200 converts a video signal, a data signal, an OSD signal, a control signal, and the like processed by the processor 230 to generate a driving signal.

Also, the display 210 may display content (eg, video) input through the communication unit 250 or an input/output unit (not shown). The display 210 may output an image stored in the memory 220 under the control of the processor 230 .

According to an embodiment, the memory 220 of the remote control device 200 may store a program for processing and control of the processor 230, and may be input to the remote control device 200 or stored in the remote control device 200. You can also store data output from .

According to one embodiment, the processor 230 typically controls the overall operation of the remote control device 200 . For example, the processor 230 may generally control the sensing unit 291 and the communication unit 250 by executing programs stored in the memory 220 . The processor 230 may control the operation of the remote control device 200 to be described with reference to FIGS. 3 to 10 .

The processor 230 may be composed of one or a plurality of processors. In this case, the one or more processors may be a general-purpose processor such as a CPU, AP, digital signal processor (DSP), or the like, or a graphics-only processor such as a GPU or a vision processing unit (VPU). One or more processors control input data to be processed according to predefined operation rules stored in a memory.

According to an embodiment, the processor 230 may receive a radio signal transmitted by the reference electronic device 100 through the communication unit 250 .

Also, the processor 230 may receive a request for identification information of the remote control device 200 from the reference electronic device 100 through the communication unit 250 . The processor 230 may control identification information of the remote control device 200 to be transmitted to the reference electronic device 100 through the communication unit 250 .

Also, the processor 230 may control the sensing value detected by the sensing unit 291 to be transmitted to the reference electronic device 100 through the communication unit 250 .

Also, the processor 230 may receive a request for state information about an operation being executed in the remote control device 200 through the communication unit 250 . The processor 230 may generate state information about the running operation. Also, the processor 230 may control transmission of state information about an operation being executed to the reference electronic device 100 through the communication unit 250 .

In addition, the memory 220 may be a flash memory type, a hard disk type, a multimedia card micro type, or a card type memory (for example, SD or XD memory). , RAM (RAM, Random Access Memory) SRAM (Static Random Access Memory), ROM (ROM, Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, It may include a storage medium of at least one type of a magnetic disk and an optical disk.

Also, according to an embodiment, the communication unit 250 may include one or more components that allow the remote control device 200 to communicate with the outside. For example, the communication unit 250 may include a short-distance communication unit (not shown), a mobile communication unit (not shown), and a broadcast reception unit (not shown).

The short-range wireless communication unit includes a Bluetooth communication unit, a Bluetooth Low Energy (BLE) communication unit, a Near Field Communication unit (WLAN) communication unit, a Zigbee communication unit, an infrared (IrDA) It may include a Data Association (Data Association) communication unit, a Wi-Fi Direct (WFD) communication unit, an ultra wideband (UWB) communication unit, an Ant+ communication unit, etc., but is not limited thereto.

According to an embodiment, the communication unit 250 may include an antenna for performing communication. For example, the remote control device 200 may include a UWB antenna for performing UWB communication. Depending on the embodiment, the communication unit 250 of the remote control device 200 may include one or two or more antennas.

The mobile communication unit transmits and receives radio signals with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the radio signal may include a voice call signal, a video call signal, or various types of data according to text/multimedia message transmission/reception.

The broadcast reception unit receives a broadcast signal and/or broadcast-related information from the outside through a broadcast channel. Broadcast channels may include satellite channels and terrestrial channels. Depending on implementation, the remote control device 200 may not include a broadcast receiver.

The sensing unit 291 may detect a state of the remote control device 200 or a state around the remote control device 200 and transmit the sensed information to the processor 230 .

The sensing unit 291 includes a magnetic sensor, an acceleration sensor, a temperature/humidity sensor, an infrared sensor, a gyroscope sensor, a location sensor (eg, GPS), an illuminance sensor, a proximity sensor, and an RGB sensor. (illuminance sensor) may include at least one, but is not limited thereto. The sensing unit 291 according to an embodiment may correspond to the sensing unit 191 described with reference to FIG. 1B , and duplicate descriptions are omitted. As described with reference to FIG. 1B , the processor 230 may obtain yaw, pitch, and/or roll value information measured from 9-axis motion data through the sensing unit 291. there is.

According to an embodiment, the sensing unit 291 may detect an external impact applied to the remote control device 200 . According to one embodiment, an acceleration sensor built into the remote control device 200 may detect an impact caused by a touch operation with the reference electronic device 100 . Also, an acceleration sensor built into the remote control device 200 may sense the moving speed and acceleration of the remote control device 200 .

In addition, the display 210, the memory 220, the communication unit 250, the sensing unit 291, and the processor 230 of the remote control device 200 are the display of the reference electronic device 100 described with reference to FIGS. 1A and 1B. Descriptions of 110 , memory 120 , communication unit 150 , sensing unit 191 , and processor 130 may be referred to.

3 is a diagram for describing an indoor environment in which a reference electronic device, one or more devices, and an electronic device for controlling the devices are located, according to an embodiment.

Operations to be described with reference to FIGS. 3 to 8 may be performed by the processor 230 of the remote control device 200 or the processor 130 of the reference electronic device 100 . Alternatively, related data may be transmitted to an external server through the reference electronic device 100, and operations may be performed in the server.

Referring to FIG. 3 , an indoor environment includes a reference electronic device 100 (eg, a TV), a remote control device 200, a first device 310 (eg, a refrigerator), and a second device 320. (eg air conditioning) may be present. As described with reference to FIG. 2B , the remote control device 200 may communicate with the reference electronic device 100 , the first device 310 and the second device 320 , and the user may use the remote control device 200 The operation of the reference electronic device 100 , the first device 310 , and the second device 320 may be controlled with the .

According to one embodiment, the remote control apparatus 200 is one or more devices (eg, the first device 310 and the second device 320) and Bluetooth, BLE (Bluetooth Low Energy), near field communication (Near Field Communication) unit), WLAN (Wi-Fi), Zigbee, and infrared (IrDA, Infrared Data Association) communication.

According to an embodiment, the reference electronic device 100 may perform ultra wideband (UWB) communication with the remote control device 200, set a virtual coordinate system in an indoor environment based on the UWB communication, and remotely control the coordinate system. An orientation direction 255 of the device 200 may be determined. A method of setting a virtual coordinate system and a method of determining a directing direction according to various embodiments will be described with reference to FIGS. 5A to 6B.

According to an embodiment, the coordinate system may be set to 2D or 3D according to the antennas mounted on the reference electronic device 100 and the remote control device 200 . According to an embodiment, the antenna corresponding to the position of the reference electronic device 100 for setting the coordinate system may be implemented in a form included in a device distinct from the reference electronic device 100 . For example, an antenna corresponding to the location of the reference electronic device 100 may be included in a separate electronic device, and the corresponding device may be attached to or mounted on the reference electronic device 100 .

When the coordinate system is set in two dimensions, as described with reference to FIG. 2A, the indoor environment can be expressed as a plane (eg, an X-Y plane viewed from the +Z axis), and each device (100, 200, 310, 320 ) may be determined as a point, a line, or an area on the coordinate system, and the orientation direction 255 of the remote control device 200 may be determined as one direction on the coordinate system.

According to an embodiment, setting the coordinate system may be performed by the processor 130 of the reference electronic device 100 or the processor of the remote control device 200 . According to another embodiment, angle of arrival (AoA), time of flight (TOF), or TDOA ( The information on the distance and angle can be measured by a method such as time difference of arrival), and the coordinate system can be set in the server by transmitting the information on the distance or angle to an external server. Information on the set coordinate system is stored in the memory 120 of the reference electronic device 100 or the memory 230 of the remote control device 200, or is stored in an external server and transferred to the processor 130 of the reference electronic device 100 or It may be transmitted according to the request of the processor 230 of the remote control device 200. A distance measurement method based on UWB communication according to various embodiments will be described with reference to FIG. 4 .

According to an embodiment, before controlling the devices 310 and 320 with the remote control device 200, the user sets a virtual envelope detectable by the remote control device 200 for each device on the coordinate system. can register. For example, the user may register the detectable region of the device on the coordinate system through a user input through the remote control apparatus 200 at the location of the device to register the detectable region.

The processor 130 of the reference electronic device 100 selects the control target device based on the detectable area on the coordinate system of each device 310 320, the position of the remote control device 200 on the coordinate system, and the heading direction 255. can decide

A process of determining relative position information of the remote control device 200 with respect to the reference electronic device 100 according to an embodiment will be described with reference to FIGS. 5A to 5C. A process of determining the directing direction 255 of the remote control device 200 on the global coordinate system based on the reference electronic device 100 according to an embodiment will be described with reference to FIGS. 6A and 6B.

4 is a diagram for explaining a method of measuring a distance between a reference electronic device and a remote control device according to an exemplary embodiment.

According to various embodiments, the reference electronic device 100 or the processor 130 of the reference electronic device 100 may calculate a distance to an external electronic device through a UWB signal. As described with reference to FIG. 1B , the reference electronic device 100 may include a UWB antenna for performing UWB communication. The UWB antenna of the reference electronic device 100 may include at least one processor distinct from the processor 130, and a distance to the external electronic device may be calculated based on the UWB signal. At least one processor included in the UWB antenna may generate data or information including time information based on the UWB signal, and may provide the data or information to the processor 130 of the reference electronic device 100 . The processor 130 may calculate a distance to an external electronic device based on the provided data or information.

Although the distance calculation method described below has been described based on the reference electronic device 100 or the processor 130 of the reference electronic device 100, it may be performed by at least one processor of an antenna distinct from the processor 130.

According to an embodiment, the processor 130 of the reference electronic device 100 calculates an angle of arrival (AoA), a time of arrival (ToA), a time difference of arrival (TDoA), etc. to calculate a distance to an external electronic device. method can be used. Referring to FIG. 4 , a process of calculating a distance between the reference electronic device 100 and the remote control device 200 based on the ToA method according to an embodiment will be described.

Referring to FIG. 4 , the processor 130 of the reference electronic device 100 uses a two way ranging (TWR) method for exchanging signals between the reference electronic device 100 and the remote control device 200, The distance between the device 100 and the remote control device 200 can be calculated. The remote control device 200 may be a UWB tag device, and the reference electronic device 100 may be a UWB anchor device.

According to an embodiment, the remote control device 200 may transmit a poll signal to the reference electronic device 100 . Upon receiving the poll signal, the reference electronic device 100 may transmit a response signal to the remote control device 200 . Upon receiving the response signal, the remote control device 200 may transmit a final signal to the reference electronic device 100 . T _{round T} , which is a round trip time (RTT) of a signal transmitted from the remote control device 200 , may be measured through a poll signal transmission time T _SP and a response signal reception time T _RR . T _{reply A} , which is the response delay time of the reference electronic device 100, may be measured through the poll signal reception time T _RP and the response signal transmission time T _SR . Since the reference electronic device 100 can transmit values of T _RP and T _SR together with a response signal, the remote control device 200 can calculate T _{reply A} of the reference electronic device 100 . T _{round A} , which is the RTT of the signal transmitted from the reference electronic device 100, may be measured through the time T _SR at which the response signal is transmitted and the time T _RF at which the final signal is received. T _P , which is the time of arrival (TOA) of a signal between the remote control device 200 and the reference electronic device 100, may be calculated through [Equation 1] below.

A distance between the remote control device 200 and the reference electronic device 100 may be calculated using T _p and the speed of the signal.

However, the distance between the reference electronic device and the remote control device may be measured based on various communication methods other than the UWB communication method described with reference to FIG. 4 . For example, the distance between the reference electronic device and the remote control device may be measured based on Bluetooth or WiFi.

5A to 6B are diagrams for explaining a process of setting a global coordinate system based on the reference electronic device 100 in an indoor environment (eg, the indoor environment of FIG. 3 ) according to an embodiment. As described for the direction 255 of the remote control device 200 of FIG. 2A, FIGS. 5A to 6B show an indoor environment in which the reference electronic device 100 and the remote control device 200 are operated are viewed from above. can be based on For example, FIGS. 5A to 6B may be an X-Y plane looking down on an indoor environment from the +Z axis of FIG. 2A.

5A to 5C are diagrams for explaining an operation of determining relative position information of a remote control device with respect to a reference electronic device according to an embodiment.

Referring to FIG. 5A , the processor 130 of the reference electronic device 100 may determine relative position information of the remote control device 200 with respect to the reference electronic device 100 . The relative location information includes the distance d from the remote control device 200 to the reference electronic device 100, the direction from the remote control device 200 to the reference electronic device 100 (the direction of the dotted line corresponding to d), and the remote control device. An angle ψ ₁ generated in a direction (the direction of the straight line 257) based on the positions of the plurality of antennas 251 and 253 in (200) may be included.

According to an embodiment, as described with reference to FIG. 1B, the reference electronic device 100 may include a UWB antenna 155, and as described with reference to FIGS. 2A and 2B, the remote control device 200 includes a UWB antenna. Antennas 251 and 253 may be included. As described with reference to FIG. 4, information on the distance b, distance d1, and distance d2 between the UWB antennas 155, 251, and 253 may be calculated. Information on the distance b, distance d1, and distance d2 may be obtained from the processor 130 of the reference electronic device 130 or obtained from the processor 230 of the remote control device 200 and transmitted to the reference electronic device 100. can

Referring to FIG. 5B , a distance d and an angle ψ ₁ may be determined based on the distance b, the distance d1, and the distance d2. According to an embodiment, since the distances d1 and d2 are greater than the distance b by a threshold level or more, it can be said that the straight line corresponding to the distance d1 and the straight line corresponding to the distance d2 are parallel. Therefore, the distance a = d1 - d2, and the distance d can be determined to be equal to either d1 or d2. In addition, the angle ψ ₁ may be determined as in [Equation 2] below.

Referring to FIG. 5C , a distance d and an angle ψ ₁ may be determined in a method different from that of FIG. 5B. Triangle ABC in FIG. 5C shows a triangle composed of antenna 155, antenna 251, and antenna 253 in FIG. 5A, point A is antenna 155, point B is antenna 251, and point C is It may correspond to the antenna 253.

Relative position information of the remote control device 200 with respect to the reference electronic device 100 may include a distance d and an angle ψ ₁ based on midpoints M of the antennas 251 and 253 . As described above, since the processor 130 of the reference electronic device 100 may acquire information about the lengths d1, b, and d2 of the three sides of the triangle ABC, for example, according to the second cosine law, the angle ∠ of the triangle ABC Information about C can be calculated.

According to an embodiment, since the processor 130 may also obtain information about the lengths b/2 and d2 of the two sides of the triangle AMC, the processor 130 calculates the length information and the angle ∠C of the two sides of the triangle AMC. Based on the information about , information about the angle ψ ₁ may be calculated according to, for example, the second cosine law.

As described with reference to FIGS. 5A to 5C , when relative position information (eg, distance d and angle ψ ₁ ) of the remote control device 200 with respect to the reference electronic device 100 is determined, the processor 130 A global coordinate system based on the reference electronic device 100 may be set. A process of setting a global coordinate system based on the reference electronic device 100 will be described with reference to FIGS. 6A and 6B.

6A and 6B are diagrams for explaining an operation of determining a directing direction of a remote control device on a global coordinate system based on a reference electronic device according to an exemplary embodiment.

Referring to FIG. 6A, when viewed from above, the reference electronic device 100 and the remote control device 200 are located based on magnetic north (N in FIG. 6A) (eg, an indoor environment in FIG. 3). indoor environment) is shown.

According to an embodiment, the processor 130 of the reference electronic device 100 may obtain or calculate information about the distance d1, the distance d2, the distance d, and the angle ψ ₁ as described with reference to FIGS. 4 and 5A. . According to an embodiment, as described with reference to FIG. 2A, the directing direction 255 of the remote control device 200 is an angle ψ _d direction with respect to a straight line 257 corresponding to the plurality of antennas 251 and 253. can be

According to an embodiment, as described with reference to FIG. 1B , the processor 130 determines the azimuth (yaw) ψ of the reference electronic device 100 corresponding to the straight line 610 along the magnetic north direction based on the sensing unit 191 . Information on ₃ can be obtained. As described with reference to FIG. 2B , the processor 230 of the remote control device 200 determines the azimuth angle ψ ₂ of the remote control device 200 corresponding to the straight line 650 along the magnetic north direction based on the sensing unit 291 . information can be obtained. The processor 130 of the reference electronic device 100 may receive information about the azimuth ψ ₂ of the remote control device 200 from the remote control device 200 .

FIG. 6B is a diagram in which FIG. 6A is rotated based on the reference electronic device 100 in an indoor environment (eg, the indoor environment of FIG. 3 ), and a global view based on the reference electronic device 100 according to an embodiment. It can be a coordinate system. According to an embodiment, the processor 130 of the reference electronic device 100 may obtain information about the coordinates 670 of the remote control device 200 on the global coordinate system based on the reference electronic device 100. . For example, the coordinates 670 may be determined as (−d×cos(ψ ₁ +ψ ₂ +ψ ₃ ), −d×sin(ψ ₁ +ψ ₂ +ψ ₃ )).

According to an embodiment, the processor 130 may calculate the angle of the pointing direction 255 of the remote control device 200 on the global coordinate system based on the reference electronic device 100 . For example, the angle of the pointing direction 255 of the remote control device 200 on the global coordinate system based on the reference electronic device 100 may be ψ ₁ + ψ ₂ + ψ ₃ .

According to an embodiment, the angle of the directing direction 255 of the remote control device 200 on the global coordinate system based on the reference electronic device 100 is an angle (yaw) on the X-Y plane as described with reference to FIG. 2A. can According to one embodiment, as described with reference to FIG. 2B , the processor 230 of the remote control device 200 controls the pitch and/or roll of the remote control device 200 through the sensing unit 291 . ) value may be acquired and transmitted to the reference electronic device 100 . According to an embodiment, the processor 130 of the reference electronic device 100 may correct orientation information of the remote control device 200 based on the received information on pitch and roll values of the remote control device 200. there is.

According to an embodiment, the processor 120 of the reference electronic device 100 may determine a control target device based on an angle between the coordinates 670 of the remote control device 200 and the pointing direction 255 on the global coordinate system. there is.

According to an embodiment, as described with reference to FIG. 3 , a coordinate system of devices (eg, the first device 310 and the second device 320 of FIG. 3 ) of an indoor environment (eg, the indoor environment of FIG. 3 ) An image detectable region may be pre-registered. In FIGS. 6A and 6B , the sensing area 690 of the device is shown in the form of a device for convenience of description, but the sensing area on the coordinate system may be registered in an arbitrary shape.

According to an embodiment, a device control input (eg, power on) through the remote control device 200 may be transmitted to the reference electronic device 100, and the processor 130 may be configured in FIGS. 5A to 50 As described with reference to 6b, information about the position 670 of the remote control device 200 on the coordinate system at the time of receiving the input and the angle of the direction 255 may be calculated. According to an embodiment, the processor 130 pre-registers devices in an indoor environment (eg, the first device 310 and the second device 320 of FIG. 3 ) based on the coordinates 670 and the angle. A device corresponding to an overlapping detectable region among the detectable regions may be determined as a control target device.

According to an embodiment, the processor 130 may control a control target device according to a user input. For example, the processor 130 may turn on the power of the controlled device according to a power-on input to the remote control device 200 .

By setting the global coordinate system based on the reference electronic device 100 in an indoor environment, the user controls the target device through the remote control device 200 without a separate space setting process for determining the location of the remote control device 200. can control.

<Operation method of reference electronic device>

7 is a flowchart illustrating an operation of a reference electronic device according to an exemplary embodiment.

Operations to be described with reference to FIGS. 7 to 8 (operations 710 to 730 and operations 810 to 830) are the processor 130 of the reference electronic device 100 described above with reference to FIGS. 1A and 1B or FIG. 2A. And it may be performed by the processor 230 of the remote control device 200 described above with reference to FIG. 2B. For convenience of description, the following operation will be described based on the processor 130 of the reference electronic device 100 . For concise description, descriptions overlapping those described with reference to FIGS. 1A to 6B may be omitted.

According to an embodiment, in operation 710, the processor 130 may determine relative position information of the remote control device 200 with respect to the reference electronic device 100. As described with reference to FIGS. 5A to 5C , the relative position information includes the distance d from the remote control device 200 to the reference electronic device 100, the direction from the remote control device 200 to the reference electronic device 100, and An angle ψ ₁ generated in a direction based on the positions of the plurality of antennas of the remote control device 200 may be included. A method of determining relative position information of a remote control device with respect to a reference electronic device will be described in detail with reference to FIG. 8 .

According to an embodiment, in operation 720, the processor 130 determines the reference electronic device 100 based on the relative position information determined in operation 710, the azimuth information of the reference electronic device 100, and the azimuth information of the remote control device 200. It is possible to calculate the coordinates of the remote control device 200 on the global coordinate system based on , and the angle of the pointing direction of the remote control device 200 .

As described with reference to FIG. 6A , the processor 130 may obtain information about the azimuth angle ψ ₃ of the reference electronic device 100 and information about the azimuth angle ψ ₂ of the remote control device 200 .

As described with reference to FIG. 6B , the processor 130 transmits information about the coordinates of the remote control device 200 (eg, coordinates 670 of FIG. 6B ) on the global coordinate system based on the reference electronic device 100 . can be obtained For example, the coordinates are (-d×cos(ψ ₁ +ψ ₂ +ψ ₃ ), -d× It can be determined as sin(ψ ₁ +ψ ₂ +ψ ₃ )). As described with reference to FIG. 6B , the processor 130 may calculate the angle of the pointing direction 255 of the remote control device 200 on the global coordinate system based on the reference electronic device 100 . For example, the angle of the pointing direction 255 of the remote control device 200 on the global coordinate system based on the reference electronic device 100 may be ψ _d +ψ ₂ +ψ ₃ .

According to an embodiment, in operation 730, the processor 130 may determine a control target device to be controlled by the remote control device based on the coordinates of the remote control device on the global coordinate system and the angle of the direction in which the remote control device is directed.

As described with reference to FIG. 6B, a device control input (eg, power ON) through the remote control device 200 may be transmitted to the reference electronic device 100, and the processor 130 may input the input It is possible to calculate information about the coordinates on the coordinate system of the remote control device 200 at the time of receiving and the angle of the heading direction. Based on the coordinates and angles, the processor 130 may detect an overlap among pre-registered detectable regions of devices in an indoor environment (eg, the first device 310 and the second device 320 of FIG. 3 ). A device corresponding to the region may be determined as a control target device.

According to an embodiment, the processor 130 may control a control target device according to a user input. For example, the processor 130 may turn on the power of the controlled device according to a power-on input to the remote control device 200 .

8 is a flowchart illustrating an operation of determining relative position information of a remote control device with respect to a reference electronic device according to an exemplary embodiment.

According to an embodiment, operations 810 to 830 may correspond to operations for determining relative location information of the remote control device with respect to the reference electronic device described with reference to FIG. 7 (eg, operation 710 of FIG. 7 ).

According to an embodiment, in operation 810, the processor 130 may obtain information about a distance from each of the plurality of antennas of the remote control device 200 to the reference electronic device 100.

According to an embodiment, in operation 820, the processor 130 may obtain information about distances between a plurality of antennas of the remote control device 200.

As described with reference to FIG. 4 , for example, the distance d1 , the distance d2 , and the distance b of FIG. 5A may be measured based on the ToA method of UWB communication, and in operations 810 and 820, the processor 130 determines the distance information can be obtained.

According to an embodiment, in operation 830, the processor 130 may determine relative location information of the remote control device 200 with respect to the reference electronic device 100. As described with reference to FIGS. 5B and 5C , the processor 130 determines the distance from each of the plurality of antennas of the remote control device 200 to the reference electronic device 100 (eg, distance d1 and distance d2 of FIG. 5A ). ) and the distance between the plurality of antennas (eg, the distance b of FIG. 5A ), the distance d from the remote control device 200 to the reference electronic device 100, the remote control device ( 200 to the reference electronic device 100 and information about an angle ψ ₁ generated in a direction based on the positions of a plurality of antennas of the remote control device 200 may be determined.

9A to 9C are diagrams for explaining an operation of determining the orientation direction of the remote control device 200 on the global coordinate system based on the reference electronic device 100 according to another embodiment.

Descriptions overlapping with those described above with reference to FIGS. 1 to 8 will be omitted. For example, the electronic device 100 may include at least some of the components described above with reference to FIGS. 1A and 1B, and the description described above with reference to FIGS. 2A and 2B may be applied to the remote control device 200. can As another example, the embodiment described with reference to FIGS. 9A to 9C may be performed by the processor 130 of the reference electronic device 100 in the indoor environment described above with reference to FIG. 3 .

9A to 9C may be diagrams for explaining a process of setting a global coordinate system based on the reference electronic device 100 in an indoor environment (eg, the indoor environment of FIG. 3 ) according to an embodiment. As described for the direction 255 of the remote control device 200 of FIG. 2A, FIGS. 9A to 9C show an indoor environment in which the reference electronic device 100 and the remote control device 200 are operated are viewed from above. can be based on For example, FIGS. 9A to 9C may be X-Y planes looking down at an indoor environment from the +Z axis of FIG. 2A .

An embodiment described with reference to FIGS. 9A to 9C may be different from the method described above with reference to FIGS. 5A to 6B. For example, in the method for determining the directing direction of the remote control device 200 described above with reference to FIGS. 5A to 6B, one antenna of the reference electronic device 100 (eg, FIGS. 5A, 6A and 6B) is used. 6b, the antenna 155), but in the embodiment described with reference to FIGS. 9A to 9C, the reference electronic device 100 may have two antennas 951 and 953. However, in the embodiments described with reference to FIGS. 9A to 9C , the number of antennas of the reference electronic device 100 is not limited to two, and more than two antennas may be used to improve positioning accuracy.

In addition, in the method described above with reference to FIGS. 5A to 6B , azimuth information of the reference electronic device 100 (eg, ψ ₃ in FIGS. 6A and 6B ) and azimuth information of the remote control device 200 (eg, FIG. 6A ) and ψ ₂ of FIG. 6B are used, but azimuth information may not be used in the embodiment to be described with reference to FIGS. 9A to 9C .

In the embodiment described with reference to FIGS. 9A to 9C , the operation is described based on the processor 130 of the reference electronic device 100, but the processor 230 of the remote control device 200 or the server based on communication Actions can be performed in 9A to 9C, the reference electronic device 100 includes a plurality of antennas 951 and 953 for UWB communication, and the remote control device 200 includes a plurality of antennas 251 and 253 for UWB communication. can be included However, the communication method is not limited to UWB communication, and various communication methods such as Bluetooth and WiFi may be used.

Referring to FIG. 9A, a process of obtaining the distance d _A and the angle ψ _A will be described. Referring to FIG. 9B, a process of obtaining the distance d _B and the angle ψ _B will be described.

Referring to FIG. 9A , the processor 130 of the reference electronic device 100 determines the distance d A from the reference electronic device 100 to the remote control device 200 , the distance d _A from the reference electronic device 100 to the remote control device 200 Information on an angle ψ _A generated by a straight line (a straight line corresponding to d _A ) and a straight line 957 based on the positions of the plurality of antennas 951 and 953 of the reference electronic device 100 may be obtained. When distance information is obtained based on UWB communication, information on d _A may be obtained based on the above with reference to FIG. 4 .

Since the process of obtaining information on the distance d _A and the angle ψ _A is duplicated with that described with reference to FIGS. 5A to 5C , a detailed description thereof will be omitted. Information on the distance d _A and the angle ψ _A may be acquired by the processor 130 of the reference electronic device 100 or obtained by the processor 230 of the remote control device 200 and transmitted to the reference electronic device 100. can

Referring to FIG. 9B , the processor 130 of the reference electronic device 100 determines the distance d B from the remote control device 200 to the reference electronic device 100 , the distance d _B from the remote control device 200 to the reference electronic device 100 Information on an angle _ψ B generated by a straight line (a straight line corresponding to d _B ) and a straight line 257 based on the positions of the plurality of antennas 251 and 253 of the remote control device 200 can be obtained. When distance information is obtained based on UWB communication, information on d _B may be obtained based on the above with reference to FIG. 4 .

Since the process of acquiring information on the distance d _B and the angle ψ _B is duplicated with that described with reference to FIGS. 5A to 5C , a detailed description thereof will be omitted. Information on the distance d _B and the angle ψ _B may be acquired by the processor 130 of the reference electronic device 100 or obtained by the processor 230 of the remote control device 200 and transmitted to the reference electronic device 100. can

Referring to FIG. 9C , a method of determining the orientation direction of the remote control device 200 on the global coordinate system based on the reference electronic device 100 will be described.

According to an embodiment, the processor 130 of the reference electronic device 100 may obtain information about distances d _A and d _B and angles ψ _{A and} ψ _B as described with reference to FIGS. 9A and 9B . . According to an embodiment, the distance from the midpoint 900 of the two antennas 951 and 953 of the reference electronic device 100 to the midpoint 910 of the two antennas 251 and 253 of the remote control device 200 is D. Since the distance between the two antennas 951 and 953 or the two antennas 251 and 253 is negligibly small compared to D, D is the distance d _A or the distance d described above with reference to FIGS. 9A and 9B. may be the same as _B.

According to one embodiment, as described with reference to FIG. 2A, the directing direction 255 of the remote control device 200 is an angle ψ _d with respect to a straight line 257 corresponding to the plurality of antennas 251 and 253, For example, it may be a direction forming 90 degrees.

9C may be a global coordinate system based on the reference electronic device 100 according to an embodiment, and the processor 130 of the reference electronic device 100 is based on distance D and angles ψ _A, ψ _{B, and} ψ _d . Thus, the directing direction 255 of the remote control device 200 may be determined.

Referring to FIG. 9C , the processor 130 of the reference electronic device 100 provides information about the coordinates 910 of the remote control device 200 on the global coordinate system whose origin is the center 900 of the reference electronic device 100. can be obtained. For example, the coordinates 910 may be determined as (-D×cos(ψ _A ), -D×sin(ψ _A )).

According to an embodiment, the processor 130 may calculate the angle of the pointing direction 255 of the remote control device 200 on the global coordinate system based on the reference electronic device 100 . For example, the angle of the pointing direction 255 of the remote control device 200 on the global coordinate system based on the reference electronic device 100 may be ψ _A -ψ _B + ψ _d .

According to an embodiment, the angle of the directing direction 255 of the remote control device 200 on the global coordinate system based on the reference electronic device 100 is an angle (yaw) on the X-Y plane as described with reference to FIG. 2A. can According to one embodiment, as described with reference to FIG. 2B , the processor 230 of the remote control device 200 controls the pitch and/or roll of the remote control device 200 through the sensing unit 291 . ) value may be acquired and transmitted to the reference electronic device 100 . According to an embodiment, the processor 130 of the reference electronic device 100 may correct orientation information of the remote control device 200 based on the received information on pitch and roll values of the remote control device 200. there is.

According to an embodiment, the processor 120 of the reference electronic device 100 may determine a control target device based on an angle between the coordinates 670 of the remote control device 200 and the pointing direction 255 on the global coordinate system. there is.

As described above with reference to FIGS. 6A and 6B , on the coordinate system of devices (eg, the first device 310 and the second device 320 of FIG. 3 ) of an indoor environment (eg, the indoor environment of FIG. 3 ) A detectable region may be pre-registered.

According to an embodiment, a device control input (eg, power on) through the remote control device 200 may be transmitted to the reference electronic device 100, and the processor 130 may perform the operation of FIGS. 9A to 9 As described with reference to 9c, information about the position 910 of the remote control device 200 on the coordinate system at the time of receiving the input and the angle of the direction 255 may be calculated.

According to an embodiment, the processor 130 pre-registers devices in an indoor environment (eg, the first device 310 and the second device 320 of FIG. 3 ) based on the coordinates 910 and the angle. Among the detectable regions, a device 990 corresponding to an overlapping detectable region may be determined as a control target device.

According to an embodiment, the processor 130 may control a control target device according to a user input. For example, the processor 130 may turn on the power of the controlling device 990 according to a power-on input to the remote control apparatus 200 .

By setting the global coordinate system based on the reference electronic device 100 in an indoor environment, the user controls the target device through the remote control device 200 without a separate space setting process for determining the location of the remote control device 200. can control.

The embodiments described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), programmable logic units (PLUs), microprocessors, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. You can command the device. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination, and the program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in the art of computer software. may be Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

The hardware device described above may be configured to operate as one or a plurality of software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on this. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

100: reference electronic device
200: remote control device
130: processor

Claims (20)

In electronic devices,
a communication module that communicates with the remote control device through one or more antennas;
a sensor module for measuring a yaw of the electronic device;
a memory in which computer-executable instructions are stored; and
A processor that accesses the memory and executes the instructions
including,
These commands are
determining relative location information of the remote control device with respect to the electronic device;
A global coordinate system based on coordinates of the remote control device and the electronic device based on the relative position information, the azimuth information of the electronic device and the azimuth information of the remote control device measured by the sensor module Calculate the angle (yaw) of the directional direction of the remote control device,
Determines a control target device to be controlled by the remote control device based on the coordinates and the angle
configured to
electronic device.
According to claim 1,
These commands are
When the control target device is determined,
Control the control target device through the remote control device
further configured to
electronic device.
According to claim 1,
These commands are
The remote control device is based on the angle (yaw) of the pointing direction of the remote control device on the global coordinate system, the pitch of the remote control device, and the roll of the remote control device. correct the orientation information of
further configured to
electronic device.
According to claim 1,
The relative location information is
information about a distance from the remote control device to the electronic device; and
Information about the direction from the remote control device to the electronic device and the angle generated in the orientation direction of the remote control device
including,
electronic device.
According to claim 1,
The relative location information of the remote control device with respect to the electronic device,
information about a distance from each of the plurality of antennas included in the remote control device to the electronic device; and
Information on the distance between the plurality of antennas
determined based on
electronic device.
According to claim 1,
The azimuth (yaw) of the remote control device is,
Measured through a sensor module included in the remote control device,
electronic device.
According to claim 1,
The electronic device performs ultra wide band (UWB) communication with the remote control device.
electronic device.
According to claim 1,
These commands are
The control target device is determined based on the coordinates of the remote control device on the global coordinate system based on the electronic device, the angle of the pointing direction, and a pre-registered detectable area.
configured to
electronic device.
According to claim 1,
The remote control device includes two antennas,
The directing direction of the remote control device is determined as a direction perpendicular to a straight line connecting the two antennas.
electronic device.
In the operating method of the electronic device,
determining relative location information of a remote control device with respect to the electronic device;
Based on the relative position information, the azimuth information of the electronic device, and the azimuth information of the remote control device, the coordinates of the remote control device and the coordinates of the remote control device on a global coordinate system based on the electronic device. calculating an angle (yaw) of the direction of orientation; and
An operation of determining a control target device to be controlled by the remote control apparatus based on the coordinates and the angle
including,
Methods of operating electronic devices.
According to claim 10,
When the control target device is determined,
Controlling the control target device through the remote control device
Including more,
Methods of operating electronic devices.
According to claim 10,
The remote control device is based on the angle (yaw) of the pointing direction of the remote control device on the global coordinate system, the pitch of the remote control device, and the roll of the remote control device. Operation of correcting orientation information of
Including more,
Methods of operating electronic devices.
According to claim 10,
The relative location information is
information about a distance from the remote control device to the electronic device; and
Information about the direction from the remote control device to the electronic device and the angle generated in the orientation direction of the remote control device
including,
Methods of operating electronic devices.
According to claim 10,
The relative location information of the remote control device with respect to the electronic device,
information about a distance from each of the plurality of antennas included in the remote control device to the electronic device; and
Information on the distance between the plurality of antennas
determined based on
Methods of operating electronic devices.
According to claim 10,
The azimuth (yaw) of the remote control device is,
Measured through a sensor module included in the remote control device,
Methods of operating electronic devices.
According to claim 10,
The electronic device performs ultra wide band (UWB) communication with the remote control device.
Methods of operating electronic devices.
According to claim 10,
The operation of determining the control target device,
An operation of determining based on the coordinates of the remote control device on the global coordinate system based on the electronic device, the angle of the orientation direction, and a pre-registered detectable area.
including,
Methods of operating electronic devices.
According to claim 10,
The remote control device includes two antennas,
The directing direction of the remote control device is determined as a direction perpendicular to a straight line connecting the two antennas.
Methods of operating electronic devices.
A computer-readable recording medium on which a program for controlling the operation of an electronic device is recorded,
the electronic device,
determining relative location information of a remote control device with respect to the electronic device;
Based on the relative position information, the azimuth information of the electronic device, and the azimuth information of the remote control device, the coordinates of the remote control device and the coordinates of the remote control device on a global coordinate system based on the electronic device. calculating an angle (yaw) of the direction of orientation; and
An operation of determining a control target device to be controlled by the remote control apparatus based on the coordinates and the angle
There is a program written to do
A computer-readable recording medium.
According to claim 19,
The operation of determining the control target device,
An operation of determining based on the coordinates of the remote control device on the global coordinate system based on the electronic device, the angle of the orientation direction, and a pre-registered detectable area.
including,
When the control target device is determined,
the electronic device,
Controlling the control target device through the remote control device
There is a program written to do more,
A computer-readable recording medium.

Images