KR20220035767A - Electronic apparatus and controlling method thereof - Google Patents

Abstract

A method for controlling an electronic device is disclosed. The control method of an electronic device according to the present disclosure includes displaying a UI for controlling a first robot located in a first space, and transmitting a control command for controlling the first robot to the first robot based on a user manipulation command. If it is difficult for the first robot to move to a second space different from the first space, confirming the second robot located in the second space and displaying a UI for controlling the confirmed second robot. Includes.

Description

Electronic device and control method thereof { ELECTRONIC APPARATUS AND CONTROLLING METHOD THEREOF }

The present disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device for controlling a robot and a control method thereof.

As robot-related technology develops, various robots are appearing, and services using robots are also appearing. For example, even if users do not visit places such as museums or shopping centers in person, they can view works in museums, art galleries, or purchase items in shopping centers using the vision technology of robots present in museums, art galleries, or shopping centers. While doing so, you can enjoy the same effects as visiting places such as museums, art galleries, and shopping centers in person.

In addition, by using robots at construction sites or work sites where it is dangerous for people to enter directly, users can perform the desired tasks.

Meanwhile, in the case of such robots, they are often manufactured as robots that can pass obstacles such as stairs and escalators because they must be easy to move under user control.

However, in the case of a robot that passes obstacles, it is expensive in that it requires devices that require complex technology, such as devices for the robot's joints and devices for maintaining the robot's parallelism, and in the process of the robot passing the obstacle, The downside is that there is a possibility of it being overturned.

The present disclosure is intended to solve the above-mentioned problems and relates to an electronic device and a control method thereof that provide a UI for controlling a plurality of robots so that a user can receive uninterrupted service even in situations where robots cannot move.

A method of controlling an electronic device according to an embodiment of the present disclosure includes displaying a UI for controlling a first robot located in a first space, and a control command for controlling the first robot based on a user manipulation command. transmitting to the first robot, if it is difficult for the first robot to move to a second space different from the first space, confirming a second robot located in the second space, and the confirmed second robot It includes the step of displaying a UI for controlling the robot.

Meanwhile, an electronic device according to another embodiment of the present disclosure controls the display to display a communication interface, a display, and a UI for controlling a first robot located in a first space, and controls the display based on a user manipulation command. The communication interface is controlled to transmit a control command for controlling the first robot to the first robot, and if it is difficult for the first robot to move to a second space different from the first space, it is located in the second space. and a processor that controls the display to check a second robot and display a UI for controlling the confirmed second robot.

1 is a diagram for explaining a system according to an embodiment of the present disclosure;
2 is a block diagram for explaining the configuration of an electronic device according to an embodiment of the present disclosure;
3 is a block diagram for explaining the configuration of a robot according to an embodiment of the present disclosure;
4 is a sequence diagram illustrating an electronic device that communicates with a server and an electronic device according to an embodiment of the present disclosure;
Figure 5a is a diagram for explaining a robot existing in a space where obstacles exist;
5B is a diagram for explaining an electronic device that displays an image received from a robot;
FIG. 5C is a diagram for explaining an electronic device that displays an image containing an obstacle;
5D is a diagram for explaining an electronic device that displays another robot existing in a space different from the space where the robot exists;
FIG. 5E is a diagram for explaining an electronic device that displays an image received from another robot existing in a different space of FIG. 5D;
FIG. 6A is a diagram for explaining an electronic device that receives an image containing an obstacle from a robot and displays it;
FIG. 6B is a diagram for explaining an electronic device that identifies an obstacle in an image containing the obstacle and displays an image of another space;
7 is a diagram for explaining a server that collects and stores spatial information according to an embodiment of the present disclosure;
8 is a diagram for explaining a space collection device according to an embodiment of the present disclosure, and
FIG. 9 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present disclosure.

Hereinafter, various embodiments of this document are described with reference to the attached drawings. However, this is not intended to limit the technology described in this document to specific embodiments, and should be understood to include various modifications, equivalents, and/or alternatives to the embodiments of this document. . In connection with the description of the drawings, similar reference numbers may be used for similar components.

In this document, expressions such as “have,” “may have,” “includes,” or “may include” refer to the existence of the corresponding feature (e.g., a numerical value, function, operation, or component such as a part). , and does not rule out the existence of additional features.

In this document, expressions such as “A or B,” “at least one of A or/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, “A or B,” “at least one of A and B,” or “at least one of A or B” (1) includes at least one A, (2) includes at least one B, or (3) it may refer to all cases including both at least one A and at least one B.

Expressions such as “first,” “second,” “first,” or “second,” used in this document can modify various components regardless of order and/or importance, and can refer to one component. It is only used to distinguish from other components and does not limit the components. For example, a first user device and a second user device may represent different user devices regardless of order or importance. For example, a first component may be renamed a second component without departing from the scope of rights described in this document, and similarly, the second component may also be renamed the first component.

Terms such as “module,” “unit,” and “part” used in this document are terms to refer to components that perform at least one function or operation, and these components are implemented in hardware or software. Alternatively, it can be implemented through a combination of hardware and software. In addition, a plurality of "modules", "units", "parts", etc. are integrated into at least one module or chip, except in cases where each needs to be implemented with individual specific hardware, and is integrated into at least one processor. It can be implemented as:

A component (e.g., a first component) is “(operatively or communicatively) coupled with/to” another component (e.g., a second component). When referred to as being “connected to,” it should be understood that any component may be directly connected to the other component or may be connected through another component (e.g., a third component). On the other hand, when a component (e.g., a first component) is said to be “directly connected” or “directly connected” to another component (e.g., a second component), It may be understood that no other component (e.g., a third component) exists between other components.

Terms used in this document are merely used to describe specific embodiments and may not be intended to limit the scope of other embodiments. Singular expressions may include plural expressions, unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by a person of ordinary skill in the technical field described in this document. Among the terms used in this document, terms defined in general dictionaries may be interpreted to have the same or similar meaning as the meaning they have in the context of related technology, and unless clearly defined in this document, have an ideal or excessively formal meaning. It is not interpreted as In some cases, even terms defined in this document cannot be interpreted to exclude embodiments of this document.

Meanwhile, in the present disclosure, the term user may refer to a person using an electronic device or a device (eg, an artificial intelligence electronic device) using the electronic device.

Hereinafter, the present disclosure will be described in detail with reference to the drawings.

1 is a diagram for explaining a system according to an embodiment of the present disclosure.

As shown in FIG. 1, the system 1000 according to an embodiment of the present disclosure includes an electronic device 100, a first robot 200-1, a second robot 200-2, and a server 300. do. Although only two robots, the first robot 200-1 and the second robot 200-2, are shown in FIG. 1, the system 1000 may include N robots.

The electronic device 100 may communicate with the first robot 200-1, the second robot 200-2, and the server 300.

Although the electronic device 100 in FIG. 1 is shown as a smartphone, the electronic device 100 according to various embodiments of the present disclosure may be a tablet PC, a mobile phone, a TV, an electronic whiteboard, a monitor, a laptop PC, a camera, or a wearable device (e.g. , watches, glasses, and head-mounted-device (HMD)). Meanwhile, the electronic device 100 is not necessarily limited to this, and any device that includes a display and can communicate with an external device can be the electronic device 100 of the present disclosure.

The electronic device 100 can control the first robot 200-1 or the second robot 200-2. Specifically, the electronic device 100 may transmit a control command to the first robot 200-1 or the second robot 200-2. To this end, the electronic device 100 may display a user interface (UI) for controlling the first robot 200-1 or the second robot 200-2.

The electronic device 100 may control the first robot 200-1 or the second robot 200-2 to capture an image of a specific space while moving in the specific space. And, the electronic device 100 receives the image captured by the first robot 200-1 or the second robot 200-2 from the first robot 200-1 or the second robot 200-2. It can be displayed. Accordingly, even if the electronic device 100 does not exist in the space where the first robot 200-1 or the second robot 200-2 is located, the electronic device 100 is connected to the first robot 200-1 or the second robot 200-2. Through 2), an image of a specific space where the first robot 200-1 or the second robot 200-2 is located can be displayed.

The first robot 200-1 or the second robot 200-2 may exist in the same space, but may also exist in different spaces. For example, the second robot 200-2 may exist in a space where the first robot 200-1 cannot move.

The first robot 200-1 or the second robot 200-2 may operate according to control commands received from the electronic device 100. The first robot 200-1 or the second robot 200-2 can move a specific space or take an image of a specific space according to a control command. In addition, sounds generated in a specific space can be collected and transmitted to the electronic device 100, or voice signals uttered by the user of the electronic device 100 can be received and output.

The first robot 200-1 or the second robot 200-2 may have various forms. Specifically, the first robot 200-1 or the second robot 200-2 may be an articulated robot (walking robot) similar to the shape of a human, a wheeled/caterpillar type robot, or a flying robot (e.g., Drone), it may be a swimming robot that operates underwater. However, it is not necessarily limited to this, and any device that can receive a control command from the user of the electronic device 100 and operate according to the received control command can be a robot according to the present disclosure.

Meanwhile, the electronic device 100 may receive information about the first robot 200-1 or the second robot 200-2 from the server 300. The server 300 is a device that can process various requests received from the electronic device 100, and can receive various information from the electronic device 100 or transmit various information to the electronic device 100.

Specifically, the electronic device 100 may request information about a robot located in a specific space from the server 300, and in response to the request, the first robot 200-1 or the second robot may be sent from the server 300. Location information of (200-2) can be received.

Alternatively, the electronic device 100 may receive an image of the space where the first robot 200-1 or the second robot 200-2 is located from the server 300. The electronic device 100 receives the first robot 200-1 or the second robot 200-2 from the server 300 before receiving the image from the first robot 200-1 or the second robot 200-2. 2) You can receive an image of the space where it exists.

Meanwhile, the server 300 may receive location information of the first robot 200-1 or the second robot 200-2 from the first robot 200-1 or the second robot 200-2. . Specifically, the server 300 periodically receives location information of the first robot 200-1 or the second robot 200-2 from the first robot 200-1 or the second robot 200-2. can do. Or, the server 300 receives image information of the space where the first robot 200-1 or the second robot 200-2 is located from the first robot 200-1 or the second robot 200-2. You may.

Hereinafter, the operations of the electronic device 100, robot 200, and server 300 of the present disclosure will be described in detail through the drawings.

Figure 2 is a block diagram for explaining the configuration of an electronic device according to an embodiment of the present disclosure.

As shown in FIG. 2 , the electronic device 100 may include a communication interface 110, a display 120, and a processor 130.

The communication interface 110 is a component that allows the electronic device 100 to communicate with an external electronic device (not shown) such as the robot 200 or the server 300. The electronic device 100 may transmit a control command to the robot 200 through the communication interface 110 and receive an image captured by the robot 200 from the robot 200 . In addition, the electronic device 100 may request information about a specific space or information about a robot located in a specific space from the server 300 through the communication interface 110, and receive information about the specific space from the server 300. Alternatively, information about a robot located in a specific space may be received.

The communication interface 110 may include various communication modules, such as a wired communication module (not shown), a short-range wireless communication module (not shown), and a wireless communication module (not shown).

Here, the wired communication module is a module for communicating with an external device (not shown) according to a wired communication method, such as wired Ethernet. Additionally, the short-range wireless communication module is a module for communicating with an external device (not shown) located nearby according to a short-range wireless communication method such as Bluetooth (BT), BLE (Bluetooth Low Energy), or ZigBee. Additionally, a wireless communication module is a module that is connected to an external network and communicates with an external device (not shown) according to a wireless communication protocol such as WiFi or IEEE. In addition, wireless communication modules comply with various mobile communication standards such as 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), and 5G Networks. It may further include a mobile communication module that connects to a mobile communication network and performs communication.

The display 120 can provide various content screens that can be provided through the electronic device 100. In the present disclosure, the display 120 may display a UI for controlling the robot 200 or may display a UI including an image received from the robot 200. Alternatively, the display 120 may display an augmented reality image generated by the electronic device 100.

The display 120 may be implemented as various types of displays, such as Liquid Crystal Display (LCD), Organic Light Emitting Diodes (OLED) display, Plasma Display Panel (PDP), Wall, Micro LED, etc. The display 120 may also include a driving circuit and a backlight unit that may be implemented in the form of a-si TFT, low temperature poly silicon (LTPS) TFT, or organic TFT (OTFT). Meanwhile, the display 120 may be implemented as a touch screen combined with a touch sensor, a flexible display, a 3D display, etc.

The processor 130 is electrically connected to a memory (not shown) and can control the overall operation and functions of the electronic device 100. For example, the processor 130 can control hardware or software components connected to the processor 130 by running an operating system or application program, and can perform various data processing and calculations. Additionally, the processor 130 may load and process commands or data received from at least one of the other components into volatile memory and store various data in non-volatile memory.

To this end, the processor 130 is a dedicated processor (e.g., embedded processor) or a general-purpose processor (e.g., CPU (Central)) capable of performing the operations by executing one or more software programs stored in a memory device. Processing Unit) or application processor (AP).

In the present disclosure, the processor 130 may be implemented as a digital signal processor (DSP), a microprocessor, or a time controller (TCON) that processes digital signals. However, it is not limited to this, and includes central processing unit (CPU), MCU (Micro Controller Unit), MPU (micro processing unit), controller, application processor (AP), GPU It may include one or more of a graphics-processing unit (graphics-processing unit), a communication processor (CP), or an ARP processor (Address Resolution Protocol processor), or may be defined by those terms. Additionally, the processor 130 may be implemented as a System on Chip (SoC) or large scale integration (LSI) with a built-in processing algorithm, or may be implemented in the form of a Field Programmable Gate Array (FPGA).

The processor 130 may control the first robot 200-1 located in the first space. Specifically, the processor 130 may control the communication interface 110 to generate a control command for controlling the robot 200 and transmit it to the robot 200.

To this end, the processor 130 may control the display 120 to display a UI for controlling the first robot.

The processor 130 may control the image captured by the first robot 200-1 to be transmitted to the electronic device 100 and display a UI including the image received from the first robot 200-1. The display 120 can be controlled.

Meanwhile, if it is difficult for the first robot 200-1 to move from the first space to a second space different from the first space, the processor 130 may check the second robot located in the second space. At this time, the reason why it is difficult for the first robot 200-1 to move from the first space to the second space may vary depending on the embodiment. For example, this may be the case when the first robot 200-1 encounters an obstacle that it cannot pass while moving from the first space to the second space. Alternatively, when the battery of the first robot 200-1 is below a preset value or the second space is an area where control commands for the first robot 200-1 do not reach, the first robot 200-1 It may be difficult for the first robot to move from the first area to the second area depending on the status of 1) (battery, failure, etc.) or the communication status transmitting a control command to the first robot 200-1.

At this time, the processor 130 may request information about the robot located in the second space from the server 300.

When the processor 130 receives information about the robot located in the second space from the server 300, it is connected to the second robot 200-2 located in the second space and controls the second robot 200-2. can do.

At this time, the processor 130 may control the display 120 to display a UI for controlling the second robot.

Hereinafter, specific operations of the processor 130 will be described in detail with reference to FIG. 4.

Meanwhile, at least one component shown in the electronic device 100 of FIG. 2 may be added, changed, or deleted in accordance with the performance and/or type of the electronic device 100. Additionally, it will be easily understood by those skilled in the art that the positions of the components may be changed in response to the performance or structure of the electronic device 100.

Figure 3 is a block diagram for explaining the configuration of the robot 200 according to an embodiment of the present disclosure.

As shown in FIG. 3, the robot 200 includes a communication interface 210, a driving device 220, a sensor 230, a memory 240, a camera 250, a microphone 260, a speaker 270, and May include a processor 280.

The communication interface 210 is a component that allows the robot 200 to communicate with the electronic device 100 or the server 300. The robot 200 may transmit the image captured by the robot 200 to the electronic device 100 or the server 300 through the communication interface 210. Additionally, the robot 200 may transmit location information of the robot 200 to the server 300 through the communication interface 210. Additionally, the robot 200 may receive control commands from the electronic device 100 through the communication interface 210.

The driving device 220 can move the robot 200. To this end, the driving device 220 may be connected to one or more wheels and may be provided with a driving unit such as a motor capable of rotating the wheels. In addition, the driving device 220 may perform driving operations such as moving, stopping, and changing direction of the robot 200 according to the control signal from the processor 280.

The sensor 230 can detect obstacles around the robot 200. Specifically, the sensor 230 can detect the location of obstacles and the distance to the obstacles around the robot 200 using an ultrasonic sensor, an infrared sensor, an RF sensor, a camera, etc. Additionally, the sensor 230 may further include a collision sensor that detects an obstacle through collision with the obstacle.

The memory 240 can store various programs and data necessary for the operation of the robot 200. The memory 240 may be implemented as non-volatile memory, volatile memory, flash-memory, hard disk drive (HDD), or solid state drive (SSD).

The camera 250 can capture an image of the space where the robot 200 is located. Camera 250 may include one or more image sensors, lenses, and image signal processors. The camera 250 is generally located on the front of the robot 200, but is not necessarily limited to this, and may be located on the rear or side of the robot 200.

The microphone 260 is configured to receive sound input from the outside of the robot 200 and generate a signal corresponding to the input sound. The processor 280 may receive sound from the space where the robot 200 is located through the microphone 260 and generate a signal corresponding thereto.

The speaker 270 is configured to output various notification sounds or voice messages as well as various audio data on which various processing tasks such as decoding, amplification, and noise filtering have been performed by an audio processor.

The processor 280 is electrically connected to a memory (not shown) and can control the overall operation and functions of the robot 200. For example, the processor 280 can control hardware or software components connected to the processor 130 by running an operating system or application program, and can perform various data processing and calculations. Additionally, the processor 280 may load and process commands or data received from at least one of the other components into volatile memory and store various data in non-volatile memory.

To this end, the processor 280 is a dedicated processor (e.g., embedded processor) or a general-purpose processor (e.g., CPU (Central)) capable of performing the corresponding operations by executing one or more software programs stored in a memory device. Processing Unit) or application processor (AP).

The processor 280 may control the operation and function of each component of the robot 200 so that the robot 200 can operate according to control commands received through the communication interface 210.

For example, the processor 280 may control the driving device 220 to move the robot 200 according to a control command, or control the camera 250 to capture an image of the space where the robot 200 is located.

FIG. 4 is a sequence diagram illustrating an electronic device that communicates with a server and an electronic device according to an embodiment of the present disclosure.

The first robot 200-1 and the second robot 200-2 may transmit location information to the server 300 (S401, S402). Specifically, the first robot 200-1 and the second robot 200-2 detect position such as an acceleration sensor and a gyro sensor included in the first robot 200-1 and the second robot 200-2. The positions of the first robot 200-1 and the second robot 200-2 can be identified using a sensor, and the identified position information can be periodically transmitted to the server 300. Additionally, the server 300 may store the positions of the first robot 200-1 and the second robot 200-2.

Additionally, the first robot 200-1 and the second robot 200-2 may transmit information about the first robot 200-1 and the second robot 200-2 to the server 300. For example, the first robot 200-1 and the second robot 200-2 provide the type, image information, and identification information of the first robot 200-1 and the second robot 200-2 to the server 300. , location information, battery information, and information about the movable area can be transmitted. Additionally, the server 300 may store various information received from the first robot 200-1 and the second robot 200-2.

Meanwhile, the processor 130 included in the electronic device 100 may be connected to the first robot 200-1 located in the first space through the communication interface 110 (S403).

Specifically, when the electronic device 100 wants to connect with a robot located in the first space, the processor 130 uses a communication interface to transmit a signal requesting information about the robot located in the first space to the server 300. (110) can be controlled.

The server 300, which has received a signal requesting information about the robot from the electronic device 100, identifies the robot existing in the first space and places it in the first space based on the signal received from the first robot 200-1. It is possible to identify that the first robot 200-1 exists, and to transmit information about the first robot 200-1. At this time, the information about the first robot may include at least one of the image, type, identification information, location information, battery information, and information about the movable area of the first robot.

Upon receiving information about the first robot 200-1 located in the first space from the server 300, the processor 130 may request control authority from the first robot 200-1. Specifically, the processor 130 may request control authority from the first robot 200-1 by transmitting identification information, address information, and user identification information of the electronic device 100. Additionally, the first robot 200-1 may grant control authority to the electronic device 100 based on information about the electronic device 100 received from the electronic device 100. Accordingly, the electronic device 100 can be connected to the first robot 200-1.

Meanwhile, when the first robot 200-1 is connected to the electronic device 100, the first robot 200-1 captures an image of the first space where the first robot 200-1 is located (S404) , the captured image can be transmitted to the electronic device 100 (S405).

And, the processor 130 can remotely control the first robot 200-1 based on the image received from the first robot 200-1.

To this end, the processor 130 may control the display 120 to display a UI for controlling the first robot 200-1 located in the first space (S406). At this time, the UI for controlling the first robot 200-1 may include a UI including an image captured by the first robot 200-1. Additionally, the processor 130 may generate an augmented reality image based on the image captured by the first robot 200-1 and control the display 120 to display a UI including the generated augmented reality image. .

The processor 130 may receive a user input for controlling the first robot 200-1 through a UI for controlling the first robot 200-1. And, the processor 130 may transmit a control signal for controlling the first robot to the first robot 200-1 based on the user input (S407).

The first robot 200-1 may operate based on a control signal received from the electronic device 100. Specifically, the first robot 200-1 may move in the first space according to a control signal or take a picture of an object located in the first space and transmit it to the electronic device 100. Additionally, the first robot 200-1 may collect sound generated in the first space according to a control signal and transmit it to the electronic device 100. Alternatively, the first robot 200-1 may receive a user voice uttered by the user of the electronic device 100 and output the received voice.

Meanwhile, a case may occur in which movement of the first robot 200-1 is impossible depending on the control signal transmitted by the electronic device 100. For example, when the first robot 200-1 encounters an obstacle that is difficult to pass, when the battery of the first robot 200-1 is below a preset value, or when the first robot 200-1 breaks down, the first robot 200-1 If the communication status for receiving control commands for the robot 200-1 is poor or depending on the movement area limit set for the first robot 200-1, the first robot may not be able to move to the second space (S408 ). Here, the second space represents a space where the first robot 200-1 cannot move due to such reasons.

If it is impossible to move from the first space to the second space, the first robot 200-1 may transmit a message to the electronic device 100 indicating that it is impossible to move to the second space (S409).

When it is difficult for the first robot 200-1 to move from the first space to the second space, the processor 130 may check the second robot located in the second space. Specifically, the processor 130 may request information about a robot existing in the second space from the server 300 (S410).

And, the server 300 can identify the robot existing in the second space (S411). Specifically, the server 300 determines that the second robot 200-2 exists in the second space based on the location information of the second robot 200-2 received from the second robot 200-2 in step S402. It can be identified that

And, the server 300 may transmit information about the second robot to the electronic device 100. At this time, when a plurality of second robots exist in the second space, the server 300 may provide information about the plurality of second robots 200-2, but the plurality of second robots 200-2 By selecting one of the options, information on the selected second robot 200-2 can be provided to the electronic device 100.

For example, the server 300 selects the second robot 200-2 closest to the first robot 200-1 among the plurality of second robots 200-2 located in the second space, and selects the second robot 200-2 that is closest to the first robot 200-1. Information on the robot 200-2 may be transmitted to the electronic device 100. Alternatively, the server 300 selects the second robot 200-2 with the largest battery capacity among the plurality of second robots 200-2 located in the second space and sends information about the selected second robot to an electronic device ( 100). At this time, the information about the second robot 200-2 includes image information of the second robot 200-2, type of the second robot 200-2, identification information, location information, battery information, and movable area. It may contain at least one piece of information about.

Upon receiving information about the second robot from the server 300, the processor 130 may control the display 120 to display the received information about the second robot 200-2 (S413). Specifically, the processor 130 is configured to display at least one of identification information, location information, battery information, or information about a movable area of the second robot 200-2, including an image of the second robot 200-2. The display 120 can be controlled.

And, the processor 130 may receive a user input for connecting to the second robot (S414). Specifically, the processor 130 may receive a user input for selecting information about the second robot 200-2 displayed on the display 120. For example, the processor 130 may receive a user input for selecting an image of the second robot displayed on the display 120.

Upon receiving a user input of information about the second robot, the processor 130 may connect to the second robot 200-2 (S415). Specifically, the processor 130 may request control authority from the second robot 200-2 by transmitting at least one of identification information, address information, and user identification information of the electronic device 100. Then, the second robot 200-2 transmits a control permission approval message to the electronic device 100 based on the information received from the electronic device 100 and sends the second robot 200-2 to the electronic device 100. Control authority can be granted. Accordingly, the electronic device 100 can be connected to the second robot 200-2 and control the second robot 200-2.

Meanwhile, upon receiving a control authority approval message from the second robot 200-2, the processor 130 sends a control command to the first robot 200-1 to move the first robot 200-1 to a preset position. ) can be controlled to transmit to the communication interface 110. At this time, the preset location may be the station of the first robot 200-1, or may be a point where the first space is connected to a space other than the second space.

When connected to the electronic device 100, the second robot 200-2 captures an image of the second space (S416), and captures the image of the second space captured by the second robot 200-2 with the electronic device 100. ) can be transmitted (S417).

And, the processor 130 can remotely control the second robot 200-2 based on the image received from the second robot 200-2. Specifically, the processor 130 may control the display to display a UI for controlling the second robot 200-2 existing in the second space (S418). Here, the UI for controlling the second robot 200-2 may include a UI including an image captured by the second robot 200-2. Additionally, the processor 130 may generate an augmented reality image based on the image captured by the second robot 200-2 and control the display 120 to display a UI including the generated augmented reality image. .

Through this process, the electronic device 100 can acquire an image captured by the first robot 200-1 while controlling the first robot 200-1 located in the first space, and control the first robot 200-1 located in the first space. When it is difficult for 200-1) to move to the second space, the image captured by the second robot 200-2 can be acquired while controlling the second robot 200-2.

5A to 5E show images captured by the first robot 200-1 and the second robot 200-2 being received from the first robot 200-1 and the second robot 200-2. This is a diagram to explain an electronic device that displays an image.

Figure 5A is a diagram for explaining a robot existing in a space where obstacles exist, for example, a diagram showing a robot existing in an art gallery or museum where obstacles such as stairs exist. In the present disclosure, an obstacle is an object that restricts the movement of a robot to prevent it from moving, and may be of various forms such as stairs, escalators, elevators, doors, and fences. In addition, of course, the space in FIG. 5A can be not only an art gallery or museum, but also various spaces.

Here, the area where the first robot 200-1 is located (e.g., the area where Figures A and Figure B exist) is referred to as the first space, and an area where the first robot 200-1 cannot move due to obstacles. (For example, the area where Figure C and Figure D exist) can be described as the second space.

The first robot 200-1 may receive a control signal from the electronic device 100 and capture images while moving in the first space according to the received control signal. For example, based on the control signal received from the electronic device 100, the first robot 200-1 photographs picture A or picture B and transmits the image of the captured picture A or picture B to the electronic device 100. You can.

Additionally, the processor 130 may receive an image captured by the first robot 200-1 from the first robot 200-1 and control the display 120 to display a UI including the received image. there is.

For example, as shown in FIG. 5B, the processor 130 receives the image of picture A captured by the first robot 200-1 and displays the UI including the image including the received picture A. (120) can be controlled.

Meanwhile, as shown in FIG. 5C, the processor 130 receives an image captured while the first robot 200-1 is moving from the first space to the second space, and the first robot 200-1 ) can be controlled to display the image received from the display 120 on the screen.

At this time, the processor 130 may identify an obstacle in an image captured while the first robot 200-1 is moving from the first space to the second space. Specifically, the processor 130 may identify obstacles in an image displayed on the screen using an object recognition program or an artificial intelligence model learned to recognize objects included in the image.

Additionally, the processor 130 may determine that it is difficult for the first robot 200-1 to move to the second space when the identified obstacle is located in a preset area of the screen. This will be explained in detail in FIGS. 6A and 6B.

Alternatively, the processor 130 may receive information about obstacles directly from the first robot 200-1. The processor 130 receives information about the obstacle identified by the first robot 200-1 while moving from the first space to the second space, and based on the received information, the first robot 200-1 It may be confirmed that it is difficult to move from space 1 to space 2. In this regard, the first robot 200-1 detects obstacles located in front of the first robot 200-1 through various sensors such as an infrared sensor, an optical sensor, and a camera included in the first robot 200-1. can be identified and calculated whether the identified obstacle can be bypassed or crossed. Additionally, information on the obstacle, including information on the identified obstacle based on the calculation result and whether the obstacle can be crossed or passed through, may be transmitted to the electronic device 100.

If it is determined that it is difficult for the first robot 200-1 to move from the first space to the second space, the processor 130 may check the second robot 200-2 located in the second space.

Specifically, as described above in FIG. 4, the processor 130 requests information about the robot existing in the second space from the server 300, and receives information about the second robot 200 existing in the second space from the server 300. -2) information may be received, and the received information of the second robot 200-2 may be displayed.

More specifically, the processor 130 generates a virtual image for the second robot 200-2 based on the information about the second robot 200-2 received from the server 300, and the generated virtual image Based on this, an augmented reality image can be created.

Specifically, as shown in FIG. 5D, the processor 130 stores information about the second robot 200-2 received from the server 300 in the image containing the obstacle captured by the first robot 200-1. An augmented reality image can be generated by rendering a virtual image, and the display 120 can be controlled to display the generated augmented reality image. Accordingly, the user of the electronic device 100 can visually recognize that the second robot 200-2 exists in the second space.

Meanwhile, the processor 130 may receive a user input for selecting information about the second robot 200-2 from the augmented reality image displayed on the display 120. For example, the processor 130 may receive a user input for selecting a virtual image for the second robot 200-2 as shown in FIG. 5D.

Upon receiving a user input for selecting a virtual image for the second robot 200-1, the processor 130 may transmit a signal requesting control authority to the second robot 200-2. And, upon receiving control authority for the second robot 200-2 from the second robot 200-2, it is connected to the second robot 200-2 and can control the second robot 200-2. .

And, the processor 130 uses a communication interface 110 to transmit a control command to the second robot 200-2 to cause the second robot 200-2 to move to an area adjacent to the first robot 200-1. can be controlled. Here, the 'adjacent area' is an area of the second space where the first robot 200-1 cannot move and represents an area where the first robot 200-1 meets an obstacle located in front of the first robot 200-1. Accordingly, the second robot 200-2 can move to an area where the second space meets an obstacle.

When the second robot 200-2 moves to an area adjacent to the first robot 200-1, the processor 130 receives the image captured by the second robot 200-2 from the second robot 200-2. The display 120 may be controlled to receive the image and display a UI including the received image. For example, when the second robot 200-2 moves in front of picture C, which is an area adjacent to the first robot 200-1, the processor 130 causes the second robot 200-2 to display the image of the second space. The second robot 200-2 can be controlled to capture images and transmit the captured images. And, as shown in FIG. 5E, the display 120 is configured to display a UI including an image captured by the second robot 200-2 (e.g., the image in Figure C). can be controlled.

Accordingly, the electronic device 100 can provide the user with an image of a specific space without restrictions on obstacles by using a plurality of robots existing in the specific space even if there are obstacles in the specific space, and the user can provide the user with information about the space. You will be able to receive continuous services without restrictions.

FIG. 6 is a diagram for explaining an electronic device that identifies obstacles in images captured by a robot and displays a virtual image. FIG. 6A is a diagram for explaining an electronic device that receives and displays an image containing an obstacle from a robot, and FIG. 6B is a diagram for explaining an electronic device that identifies an obstacle in an image containing an obstacle and displays an image of another space. This is a drawing for

As described above, the first robot 200-1 may capture an image while moving in the first space and transmit the captured image to the electronic device 100. Additionally, the processor 130 may receive an image captured by the first robot 200-1 from the first robot 200-1 and control the display 120 to display the received image.

Meanwhile, the processor 130 may identify an obstacle in an image captured by the first robot 200-1. Specifically, the processor 130 identifies obstacles included in the first robot 200-1 using an object recognition program stored in memory (not shown) or an artificial intelligence model learned to recognize objects included in an image. can do.

Also, when the identified obstacle is displayed in the preset area 610 of the screen of the display 120, the processor 130 may determine that the first robot 200-1 cannot overcome the obstacle. Here, the preset area 610 is an area for determining whether the robot 200 can overcome the obstacle by roughly identifying the size of the obstacle located in front of the robot 200, and includes the center area of the display screen. This is an area that occupies more than a certain percentage of the total screen area.

The processor 130 displays obstacles included in the image captured by the first robot 200-1 within a preset area 610 of the screen, and displays an area excluding the obstacle from the preset area 610 (shaded area). If the area of is less than a predefined value, it can be confirmed that the first robot 200-1 cannot move over the obstacle.

For example, the processor 130 identifies a door in the image captured by the first robot 200-1, displays the door within a preset area 610 on the screen, and displays an area excluding the door from the preset area 610. If the area of is less than a predefined value, it may be determined that the first robot 200-1 cannot move over the obstacle.

When it is confirmed that the first robot 200-1 cannot move from the first space to the second space due to an obstacle, the processor 130 may request information about the second space from the server 300.

Specifically, the processor 130 transmits identification information of the first robot 200-1 to the server 300, and identifies the first robot 200-1 based on the identification information of the first robot 200-1. You can request information about the location of and the adjacent second space. Here, the information about the second space may include image information of the second space or image information of an object included in the second space. Additionally, the processor 130 may request information about a robot located in the second space from the server 300.

As described above in FIG. 4, the first robot 200-1 and the second robot 200-2 periodically transmit identification information and location information to the server 300. Location information of the first robot 200-1 and the second robot 200-2 may be stored. Therefore, the server 300, which has received the identification information of the first robot 200-1 from the electronic device 100, can identify the location information of the first robot based on the identification information of the first robot 200-1. there is.

Additionally, the server 300 may obtain information about the second space adjacent to the first robot 200-1 based on the location information of the first robot 200-1. To this end, the server 300 can store spatial information about a plurality of spaces. Specifically, the server 300 may receive spatial information from a spatial collection device located in each of a plurality of spaces and store the received spatial information. In this regard, the operation of the server 300 will be described in detail with reference to FIGS. 7 and 8.

The server 300 may transmit the acquired information about the second space and information about the robot located in the second space to the electronic device 100. Specifically, the server 300 may provide at least one of image information of the second space, image information of the second robot 200-2 located in the second space, identification information, location information, battery information, and information about the movable area. One can be transmitted to the electronic device 100.

Meanwhile, based on information about the robot stored in the server 300, it may be confirmed that the robot does not exist in the second space. In this case, the processor 130 may request information about the third space connected to the second space from the server 300. Specifically, the processor 130 may receive location information of the third space and information about a robot located in the third space. Additionally, it is possible to check whether the third robot located in the third space can move to the second space, and to display a UI for controlling the third robot.

Meanwhile, when there is a robot located in the second space, the processor 130 creates an augmented reality image based on the information about the second space received from the server 300 and the information about the robot located in the second space. The display 120 can be controlled to generate and display it.

Specifically, the processor 130 renders the image of the second space and the image of the second robot 200-2 received from the server 300 on the image received from the first robot 200-1 to create an augmented reality image. can be created. At this time, the processor 130 renders the image of the second space and the image of the second robot 200-2 at the location of the obstacle displayed in the preset area 610 in the image received from the first robot 200-1. The display 120 can be controlled to do so. Additionally, the processor 130 may control the display 120 to provide graphic effects corresponding to obstacles while rendering images of the second space and the image of the second robot 200-2. For example, the processor 130 may control the display 120 to provide a graphic effect of a door opening while rendering an image of the second space and an image of the second robot 200-2, as shown in FIG. 6B. there is.

Accordingly, when the first robot 200-1 encounters an obstacle and has difficulty moving from the first space to the second space, the processor 130 generates an image of the second space and an image of the second robot existing in the second space. The display 120 can be controlled to display a UI including.

FIG. 7 is a diagram illustrating a server that collects and stores spatial information according to an embodiment of the present disclosure.

A spatial information collection device 500 may be located in each space of the present disclosure. Here, the spatial information collection device 500 is a device that generates an image of the space by capturing an image of the space in which the spatial information collection device 500 is placed, and refers to various imaging devices including cameras and CCTVs.

Specifically, as shown in FIG. 8, there may be a space collection device 500-1 for imaging the first space and a space collection device 500-2 for imaging the second space based on the area where the obstacle is located. there is. Meanwhile, in FIG. 8, there is one space collection device 500-1 for the first space and one space collection device 500-2 for the second space, but each space has a plurality of space collection devices. can be located

The spatial information collection device 500 may collect spatial information about the space in which the spatial information collection device 500 is located (S710). Specifically, the spatial information collection device 500 may capture a space where the spatial information collection device 500 is located and generate an image including robots or various objects existing in the space.

For example, referring to FIG. 8 , the first space collection device 500-1 may collect spatial information about the first space by photographing the first space and robots or various objects located in the first space. Additionally, the second space collection device 500-2 may collect spatial information about the second space by photographing the second space and robots or various objects located in the second space.

The spatial collection device 500 may transmit the collected spatial information to the server 300 (S720). Specifically, the spatial collection device 500 may be connected to the server 300 and periodically transmit collected spatial information to the server 300.

The server 300 can reconstruct and store the spatial information received from the spatial information collection device 500 through image processing (S730). Specifically, the server 300 performs Euclidean geometric transformation to enlarge, reduce, and rotate the image included in the spatial information received from the spatial information collection device 500, performs color correction or color conversion of the image, or 1 Each space is processed through various image processing, such as combining a plurality of images received from the space collection device 500-1 (or the second space collection device 500-2) to create one image for the first space. Spatial information about can be reconstructed and stored.

Then, the server 300 may transmit the reconstructed spatial information to the spatial information collection device 500 (S740). In this case, the spatial information collection device 500 may re-collect spatial information based on the reconstructed spatial information received from the server 300.

Meanwhile, based on the spatial information stored in the server 300, a plurality of spatial information may be connected (S750). Specifically, the manager of the server 300 may connect a plurality of spatial information based on geographical information for each space.

As another example, when geographical information for each space is input to the server 300, the server 300 may connect each of a plurality of spatial information based on the input geographical information. For example, if the geographical information received by the server 300 indicates that the first space and the second space are connected, and the second space and the third space are connected, the server 300 provides spatial information about the first space and the third space. Spatial information for the second space can be mapped as one pair, and spatial information for the second space and spatial information for the third space can be mapped as one pair.

Additionally, the server 300 may store a plurality of interconnected spatial information in the server 300 (S760).

Accordingly, when the server 300 receives identification information of the first robot 500-1 from the electronic device 100, the server 300 selects the first robot 500-1 based on the identification information of the first robot 500-1. ), and information about the second space adjacent to the first robot 500-1 can be identified based on the confirmed location information of the first robot 500-1.

FIG. 9 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present disclosure.

The electronic device 100 can be connected to the first robot 200-1 located in the first space. Specifically, the electronic device 100 may transmit a control permission request message to the first robot 200-1 and receive control permission from the first robot 200-1.

When the electronic device 100 is connected to the first robot 200-1, a UI for controlling the first robot can be displayed (S910).

Specifically, when connected to the first robot 200-1, the image captured by the first robot 200-1 is received from the first robot 200-1, and a UI including the received image is displayed. You can. At this time, the first robot 200-1 may receive an image captured while moving from the first space to the second space, and display a UI including the received image.

Also, a control command for controlling the first robot 200-1 may be transmitted to the first robot 200-1 based on the user manipulation command. Accordingly, the first robot 200-1 can capture images of the first space while moving through the first space according to a control command.

Also, it can be confirmed whether the first robot 200-1 can move to a second space that is different from the first space. Specifically, the first robot 200-1 identifies an obstacle in an image captured while moving from the first space to the second space, and when the identified obstacle is located in a preset area of the screen, the first robot 200-1 It may be determined that it is difficult to move from the first space to the second space.

According to another embodiment, information about an obstacle identified while the first robot 200-1 moves from the first space to the second space is received from the first robot 200-1, and information about the received obstacle is received. Based on the information, it can be confirmed whether the first robot 200-1 can move from the first space to the second space. To this end, the first robot 200-1 can identify obstacles through various sensors such as an infrared sensor, an optical sensor, and a camera while moving according to the control command of the electronic device 100. Additionally, the first robot 200-1 may transmit information about the identified obstacle to the electronic device 100.

If it is difficult for the first robot 200-1 to move to a second space different from the first space, the second robot 200-2 located in the second space can be confirmed (S930).

When receiving information about an obstacle identified while the first robot 200-1 moves from the first space to the second space from the first robot 200-1, the first robot 200-1 based on the received information -1) If it is confirmed that it is difficult to move from the first space to the second space, it can be confirmed whether the robot exists in the second space.

Specifically, if it is difficult for the first robot 200-1 to move from the first space to the second space, information about the robot located in the second space may be requested from the server 300. And, information about the second robot 200-2 located in the second space can be received from the server 300. At this time, the information about the second robot 200-2 may include at least one of the image, type, identification information, location information, battery information, and information about the movable area of the second robot 200-2.

Meanwhile, at this time, if it is confirmed that there is no robot in the second space, information about the third space connected to the second space is requested from the server 300, and the third space is requested based on the information received from the server 300. It can be confirmed whether the third robot 200-3 located in the space can move to the second space. Also, when the third robot 200-3 can move to the second space, a UI for controlling the third robot 200-3 may be displayed.

When information about the second robot 200-2 is received from the server 300, the received information about the second robot 200-2 can be displayed. For example, image information of the second robot 200-2 may be displayed.

And, when a user input for selecting the displayed second robot 200-2 is received, the second image captured by the second robot 200-2 can be displayed.

Specifically, upon receiving a user input for selecting the second robot 200-2, control authority may be requested from the second robot 200-2. At this time, when control authority approval is received from the second robot 200-2, it is connected to the second robot 200-2 and a control command can be transmitted to the second robot 200-2.

And, a UI for controlling the confirmed second robot 200-2 can be displayed (S940).

At this time, a control command that causes the second robot 200-2 to move to an area adjacent to the first robot 200-1 may be transmitted to the second robot 200-2. When the second robot 200-2 moves to an area adjacent to the first robot 200-1, it receives the image captured by the second robot 200-2 and 2 A UI including an image received from the robot 200-2 can be displayed.

When control authority approval is received from the second robot (200-2) and the second robot (200-2) moves to an area adjacent to the first robot (200-1), the first robot (200-1) A control command to move the robot 200-1 to a preset location, such as a station, may be transmitted to the first robot 200-1.

The various operations described above as being performed through at least one of the electronic device 100 or the server 200 are performed by one or more electronic devices in the form of a control method of an electronic device or a control method or operation method of a system including an electronic device. It can be performed through .

Meanwhile, the various embodiments described above may be implemented in a recording medium that can be read by a computer or similar device using software, hardware, or a combination thereof.

According to hardware implementation, embodiments described in this disclosure include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), and field programmable gate arrays (FPGAs). ), processor, controller, micro-controllers, microprocessor, and other electrical units for performing functions.

In some cases, embodiments described herein may be implemented in the processor itself. According to software implementation, embodiments such as procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules described above may perform one or more functions and operations described herein.

Meanwhile, computer instructions for performing processing operations on a user device or administrator device according to various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. You can. The computer instructions stored in such a non-transitory computer-readable medium, when executed by a processor of a specific device, cause the specific device to perform processing operations of the user device and/or the administrator device according to the various embodiments described above.

A non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as registers, caches, and memories. Specifically, the various applications or programs described above may be stored and provided on non-transitory readable media such as CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, etc.

In addition, although the preferred embodiments of the present disclosure have been shown and described above, the present disclosure is not limited to the specific embodiments described above, and the technical field belonging to the disclosure is not departing from the gist of the disclosure as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical idea or perspective of the present invention.

100: Electronic device 200-1: First robot
100-2: Second robot 300: Server
110: communication interface 120: display
130: processor

Claims (20)

In a method of controlling an electronic device,
Displaying a UI for controlling a first robot located in a first space;
Transmitting a control command for controlling the first robot to the first robot based on a user operation command;
If it is difficult for the first robot to move to a second space different from the first space, confirming a second robot located in the second space; and
A control method comprising: displaying a UI for controlling the confirmed second robot. According to paragraph 1,
A control method further comprising transmitting a control command to the second robot to cause the confirmed second robot to move to an area adjacent to the first robot. According to paragraph 1,
When the second robot moves to an area adjacent to the first robot, receiving a second image captured by the second robot from the second robot,
The step of displaying a UI for controlling the second robot,
A control method comprising displaying a UI including the received second image. According to paragraph 1,
It further includes receiving a first image captured by the first robot from the first robot,
The step of displaying a UI for controlling the first robot,
A control method comprising: displaying a UI including the received first image. According to paragraph 1,
The step of displaying a UI for controlling the first robot,
Receiving a first image captured by the first robot while moving from the first space toward the second space, and displaying a UI including the received first image,
identifying an obstacle in the captured first image; and
A control method including; determining that it is difficult for the first robot to move to the second space when the identified obstacle is displayed in a preset area of the screen. According to paragraph 1,
Receiving information from the first robot about an obstacle identified by the first robot while moving from the first space to the second space,
The above confirmation steps are:
If it is determined that it is difficult for the first robot to move to the second space based on the received information, confirming whether the robot exists in the second space. A control method comprising a. According to clause 6,
When it is confirmed that there is no robot in the second space, requesting information about a third space connected to the second space from a server;
Checking whether a third robot located in the third space can move to the second space based on information received from the server;
If the third robot can move to the second space, displaying a UI for controlling the third robot. According to paragraph 1,
The step of checking the second robot is,
If it is difficult for the first robot to move from the first space to the second space, requesting information about the robot located in the second space from a server; and
Receiving information about a second robot located in the second space from the server; Including,
Information about the second robot,
A control method including at least one of an image of the second robot, a type of the second robot, identification information, location information, battery information, and information about a movable area. According to clause 8,
The step of displaying a UI for controlling the second robot,
Upon receiving information about the second robot, displaying the received information about the second robot;
Upon receiving a user input for selecting information on the displayed second robot, displaying a second image captured by the second robot; and
A control method including; displaying a UI for controlling the position or direction of the second robot. According to paragraph 1,
Requesting control authority from the identified second robot; and
Upon receiving a control authority approval message from the second robot, transmitting a control command to the first robot to cause the first robot to move to a preset position. A control method comprising a.
In electronic devices,
communication interface;
display; and
Controlling the display to display a UI for controlling a first robot located in a first space,
Controlling the communication interface to transmit a control command for controlling the first robot to the first robot based on a user operation command,
If it is difficult for the first robot to move to a second space different from the first space, check the second robot located in the second space,
An electronic device comprising; a processor controlling the display to display a UI for controlling the confirmed second robot. According to clause 11,
The processor,
An electronic device that controls the communication interface to transmit a control command to the second robot to cause the identified second robot to move to an area adjacent to the first robot.
According to clause 11,
The processor,
When the second robot moves to an area adjacent to the first robot, a second image captured by the second robot is received from the second robot,
An electronic device that controls the display to display a UI including the received second image. According to clause 11,
The processor,
Receiving a first image captured by the first robot from the first robot,
An electronic device that controls the display to display a UI including the received first image. According to clause 11,
The processor,
Receiving a first image captured by the first robot while moving from the first space toward the second space, and controlling the display to display a UI including the received first image,
Identifying an obstacle in the captured first image,
The electronic device determines that it is difficult for the first robot to move to the second space when the identified obstacle is displayed in a preset area of the screen. According to clause 11,
The processor,
Receiving information about obstacles identified by the first robot while moving from the first space to the second space from the first robot,
If it is determined that it is difficult for the first robot to move to the second space based on the received information, the electronic device checks whether the robot exists in the second space. According to clause 11,
The processor,
If it is difficult for the first robot to move from the first space to the second space, request information about the robot located in the second space from the server,
Receiving information about a second robot located in the second space from the server,
Information about the second robot,
An electronic device comprising at least one of an image of the second robot, a type of the second robot, identification information, location information, battery information, and information about a movable area. According to clause 17,
The processor,
When receiving information about the second robot, display the received information about the second robot,
When receiving a user input for selecting information on the displayed second robot, the display is configured to display a UI including a second image captured by the second robot and a UI for controlling the position or direction of the second robot. Controlling electronic device. According to clause 11,
The processor,
Request control authority from the confirmed second robot,
An electronic device that controls the communication interface to transmit a control command to the first robot to move the first robot to a preset position when receiving a control authority approval message from the second robot. In the computer-readable recording medium including a program for executing a control method of an electronic device,
The control method is,
Displaying a UI for controlling a first robot located in a first space;
Transmitting a control command for controlling the first robot to the first robot based on a user operation command;
If it is difficult for the first robot to move to a second space different from the first space, confirming a second robot located in the second space; and
A recording medium comprising: displaying a UI for controlling the confirmed second robot.

Images