KR20240020865A - Electronic device, systems and methods for robot control - Google Patents

Abstract

The electronic device mounted on the robot according to the present invention includes a sensing unit that acquires image information about the indoor space in order to perform visual positioning for the robot to run in the indoor space, and an interface that performs data communication between the electronic device and the robot. It includes a communication unit that communicates with the unit and the server, and transmits image information to the server so that the server performs visual positioning using the image information, and receives a control signal generated based on the result of the visual positioning from the server. In response, the control signal received from the server may be transmitted to the robot through the interface unit so that the robot runs in the indoor space based on the control signal.

Description

Electronic devices, systems and methods for robot control {ELECTRONIC DEVICE, SYSTEMS AND METHODS FOR ROBOT CONTROL}

The present invention relates to electronic devices, systems and methods for robot control. More specifically, about an electronic device, system, and method that is mounted on a robot, obtains information about the indoor space in which the robot runs, and controls the movement of the robot through a control signal generated by a server to control the movement of the robot. will be.

As technology develops, the spread of electronic devices (e.g., smartphones, tablet PCs, automated devices, etc.) has become popular, and as a result, dependence on electronic devices is gradually increasing in many aspects of daily life.

Furthermore, recently, as artificial intelligence technology, cloud technology, etc. have developed, brainless robot systems are being developed to efficiently control robots.

The brainless robot system includes a plurality of brainless robots and brain servers. In the brainless robot system, each brainless robot reports sensor information and the status of various robots to a brain server located remotely or at the edge and moves, etc. Control commands are received and processed.

At this time, in order for the brainless robot to receive and process control commands from the brain server, it must be equipped with sensors, communication devices, and processors that can be linked to a specific brain server, and cost and effort are required to develop new robots. .

Additionally, in the case of low-spec robots or robots manufactured by a third party, there may be difficulties in linking with a specific brain server.

Accordingly, in Republic of Korea Patent Publication No. 10-2021-0062684 (Separate vehicle lidar data collection pod), a pod that is mounted on a vehicle to collect environmental/vehicle data and communicate with a remote server is disclosed, but the indoor space It does not disclose a device that is mounted on a robot traveling and transmits the acquired indoor image information to a server and transmits the control signal generated by the server to the robot.

Accordingly, there is a need for a device that controls the robot's indoor driving by being mounted on the robot, collecting indoor image information and transmitting it to the server, and transmitting control signals generated through visual positioning using the information received by the server to the robot. It still exists.

The present invention provides an electronic device, system, and method for controlling the movement of a robot. More specifically, the present invention relates to an electronic device, a system, and It provides a method.

Furthermore, the present invention transmits image information acquired by an electronic device mounted on the robot and robot image information acquired by the robot to a server, and the server transmits a control signal generated based on the image information and robot image information to the robot. To provide electronic devices, systems and methods.

The electronic devices mounted on the robot to solve the problems discussed above include a sensing unit that acquires image information about the indoor space in order to perform visual localization for the robot to run in the indoor space, electronic devices, and the robot. An interface unit that performs data communication between devices and a communication unit that communicates with the server and transmits image information to the server so that the server performs visual positioning using the image information, and generates data from the server based on the results of the visual positioning. In response to receiving the control signal, the control signal received from the server may be transmitted to the robot through the interface unit so that the robot runs in the indoor space based on the control signal.

In addition, the robot control system includes a server that generates control signals to control the movement of the robot in an indoor space, and an electronic device that is mounted on the robot, obtains information related to the robot, and transmits information related to the robot to the server through the robot. The electronic device acquires image information about the indoor space through a sensing unit so that visual positioning for driving control of the robot is performed in the server, and communicates data with the robot so that the image information is transmitted to the server through the robot. Transmits image information to the robot through an interface unit connected to perform, and the server, in response to receiving image information acquired from an electronic device from the robot, performs visual positioning for the robot based on the image information, and performs visual Based on the results of the positioning, a control signal for controlling the robot's driving in an indoor space is generated, and a control signal for controlling the robot's driving generated based on image information acquired from an electronic device can be transmitted to the robot. .

In addition, the robot control method includes obtaining image information about the indoor space through a sensing unit of an electronic device mounted on the robot to perform visual positioning for the robot to run in the indoor space, and allowing the server to perform visual positioning. , transmitting image information to a server and, in response to receiving a control signal generated based on the result of visual positioning from the server, transmitting a control signal to the robot so that the robot runs in an indoor space based on the control signal. May include steps.

Furthermore, it is a program that is executed by one or more processes in an electronic device and stored in a computer-readable recording medium, and the program provides visual positioning for the robot to run in an indoor space through a sensing unit of the electronic device mounted on the robot. In order to perform, obtaining image information about the indoor space, transmitting the image information to the server so that the server performs visual positioning, and receiving a control signal generated based on the result of the visual positioning from the server. In response to doing so, it may be characterized as including instructions for transmitting a control signal to the robot so that the robot runs in an indoor space based on the control signal.

As seen above, the electronic device and robot control system mounted on the robot according to the present invention are mounted on the robot to obtain information related to the running of the robot, and communicate with the server using the obtained information to determine the type of robot or Regardless of specifications, a robot control environment controlled by a server can be provided.

Furthermore, the electronic device and robot control system mounted on the robot according to the present invention use the information acquired by the electronic device and the information acquired by the robot together to generate a control signal through visual positioning. By doing so, the precision of robot control by the server can be improved.

In addition, the electronic device and robot control system mounted on the robot according to the present invention control the robot's driving not only through control signals provided from the server, but also through driving assistance information based on information acquired by the electronic device, thereby controlling the robot's driving in an indoor space. It is possible to provide a robot control environment that can flexibly respond to obstacles.

1 and 2 are conceptual diagrams for explaining an electronic device and a robot control system mounted on a robot according to the present invention.
Figure 3 is a flowchart showing a method by which an electronic device controls a robot, according to an embodiment.
Figure 4 is a flowchart showing a method for a server to generate a control signal to control a robot, according to an embodiment.
FIG. 5 is a flowchart showing how the robot control system of FIG. 2 provides control signals for controlling the running of the robot to the robot.
FIG. 6 is a flowchart showing how the robot control system of FIG. 2 provides a control signal generated using robot image information to the robot.
FIG. 7 is a conceptual diagram for explaining an electronic device and a robot control system further including a second camera, an inertial sensor, and a processor, according to an embodiment.
FIG. 8 is a flowchart showing a method of acquiring driving assistance information based on depth information and inertia information acquired by the sensing unit of FIG. 7 and transmitting it to the robot.
FIG. 9 is a flowchart showing how the robot control system of FIG. 7 provides control signals for controlling the running of the robot to the robot.
FIG. 10 is a flowchart showing how the robot control system of FIG. 7 provides a control signal generated using robot image information to the robot.
Figure 11 is a conceptual diagram for explaining an electronic device mounted on a robot and a robot control system according to another embodiment.
FIG. 12 is a flowchart showing a method by which the electronic device of FIG. 11 transmits image information to a robot.
FIG. 13 is a flowchart showing how the robot control system of FIG. 11 provides control signals for controlling the running of the robot to the robot.
FIG. 14 is a flowchart showing how the robot control system of FIG. 11 provides a control signal generated using robot image information to the robot.

Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings. However, identical or similar components will be assigned the same reference numbers regardless of drawing symbols, and duplicate descriptions thereof will be omitted. The suffixes “module” and “part” for components used in the following description are given or used interchangeably only for the ease of preparing the specification, and do not have distinct meanings or roles in themselves. Additionally, in describing the embodiments disclosed in this specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed descriptions will be omitted. In addition, the attached drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed in this specification is not limited by the attached drawings, and all changes included in the spirit and technical scope of the present invention are not limited. , should be understood to include equivalents or substitutes.

Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

The present invention provides an electronic device mounted on a robot, a robot control system, and a method, More specifically, the present invention relates to an electronic device, a system, and It's about method.

Below, we will look at the electronic devices and robot control system mounted on the robot along with the attached drawings. 1 and 2 are conceptual diagrams for explaining an electronic device and a robot control system mounted on a robot according to the present invention.

For example, as shown in Figure 1, as technology develops, the utilization of robots is gradually increasing. Conventionally, robots have been used in special industrial fields (for example, fields related to industrial automation), but they are gradually transforming into service robots that can perform useful tasks for humans or facilities.

A robot capable of providing various services as described above may be configured to travel in a space 10 as shown in FIG. 1 in order to perform assigned tasks. There is no limit to the type of space in which the robot drives, and it can be made to drive in at least one of indoor space and outdoor space as needed. For example, indoor spaces may be various spaces such as department stores, airports, hotels, schools, buildings, subway stations, train stations, bookstores, etc. In this way, robots can be deployed in various spaces to provide useful services to humans. However, for convenience of explanation below, it is assumed that space 10 refers to an indoor space.

Meanwhile, in order to provide various services using robots, accurate control of the robot is a very important factor. Accordingly, the present invention relates to a method of controlling the robot 100 through the server 20 using the electronic device 110 mounted on the robot 100, regardless of the specifications and type of the robot 100. suggest.

As shown in FIGS. 1 and 2, the robot control system of the present invention generates control signals for controlling the electronic device 110 mounted on the robot 100 and the movement of the robot 100 in the space 10. It may include a server 20 that does. At this time, the electronic device 110 can control the driving of the robot 100 through communication with the server 20.

According to one embodiment, the electronic device 110 includes image information received from the sensing unit 120 of the electronic device 110, image information received from the robot 100, and image information received from the sensor provided in the robot 100. By utilizing at least one of the available information, the driving of the robot 100 can be controlled or appropriate control of the robot 100 can be performed.

Referring to FIG. 2 , the electronic device 110 according to an embodiment may include at least one of a sensing unit 120, a communication unit 130, an interface unit 140, and an authentication unit 150.

According to one embodiment, the communication unit 130 may communicate with the server 20 by wire or wirelessly. The communication unit 130 may transmit image information acquired using the sensing unit 120 to the server 20 and receive a control signal generated from the server 20 through communication with the server 20 .

Here, the server 20 may be configured to include at least one of a cloud server or a database. Meanwhile, the server 20 may perform data processing or data operations on data received from the electronic device 110, and the present invention does not impose any special restrictions on this method.

In addition, the server 20 may exist in a remote location from the space 10 in which the robot 100 runs, but is not limited to this. In the case of an installed robot, the server 20 is located within the space 10. It may be placed together with the robot 100 and connected to the electronic device 110 and/or the robot 100 by wire.

Meanwhile, the communication unit 130 can support various communication methods depending on the communication standard of the communicating device.

For example, the communication unit 130 supports wireless LAN (WLAN), wireless-fidelity (Wi-Fi), wireless fidelity (Wi-Fi) Direct, digital living network alliance (DLNA), wireless broadband (WiBro), and WiMAX ( World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G (5th Generation Mobile Telecommunication) , Bluetooth™ RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi Direct, Wireless USB (Wireless Universal Serial Bus) ), and can be made to communicate with the server 20 using at least one of Ethernet technology.

Next, the sensing unit 120 according to one embodiment may be configured to obtain information about the space 10 in which the robot 100 runs. According to one embodiment, the sensing unit 120 includes at least one camera and a sensor, and can obtain information about the space 10 in which the robot 100 runs through the at least one camera and sensor. For example, the sensing unit 120 may use the first camera 121 to obtain image information about the indoor space 10 in which the robot 100 runs.

At this time, the first camera 121 may be arranged on the electronic device 110 to be aligned with the front of the robot 100. Additionally, the first camera 121 may include a wide-angle lens or a fisheye lens that can acquire image information about the front, side, and top of the robot 100, but is not limited thereto. However, a detailed description of this will be provided later in the description of FIG. 7 below.

The sensing unit 120 according to one embodiment may transmit the acquired information to the communication unit 130. Specifically, the sensing unit 120 may transmit the acquired information to the communication unit 130 so that the acquired information is transmitted to the server 20 through the communication unit 130. The sensing unit 120 according to another embodiment may transmit the acquired information to the authentication unit 150. Specifically, the sensing unit 120 may transmit the acquired information to the authentication unit 150 so that the acquired information is transmitted to the communication unit 130 through the authentication unit 150.

Next, the interface unit 140 may be configured to perform data communication between the robot 100 and the electronic device 110. In the present invention, the interface unit 140 can communicate with the robot 100 as shown.

According to one embodiment, the interface unit 140 may be configured to transmit a control signal obtained from the server 20 through the communication unit 130 to the robot 100 through data communication with the robot 100. .

According to another embodiment, the interface unit 140 may be configured to transmit information received from the authentication unit 150 to the robot 100 through data communication with the robot 100.

According to another embodiment, the interface unit 140 may receive information acquired by the robot 100 from the robot 100. For example, the electronic device 110 may receive robot image information acquired by a sensor disposed on the robot 100 from the robot 100 through the interface unit 140.

Meanwhile, the interface unit 140 can support various communication methods depending on the communication standard of the communicating device.

For example, the interface unit 140 supports WLAN, Wi-Fi, Wi-Fi Direct, DLNA, Bluetooth™ RFID, infrared communication, UWB, ZigBee, NFC, Wi-Fi Direct, Wireless USB, and Ethernet. ), CAN (controller area network), and UART (universal asynchronous receiver/transmitter) may be configured to communicate with the robot 100 using at least one of communication technologies.

Next, the authentication unit 150 may be configured to encrypt the information obtained by the sensing unit 120 and insert authentication information that the server 20 can authenticate into the encrypted data.

The authentication unit 150 according to one embodiment may perform encryption on image information so that the server 20 with preset verification authority can verify image information acquired through the sensing unit 120.

Additionally, the authentication unit 150 according to one embodiment may insert authentication information that can identify at least one of the electronic device 110 and the robot 100 into the encrypted data. At this time, the authentication information includes the unique serial number of the electronic device 110, the unique serial number of the robot 100, the type of the robot 100, the specifications of the robot 100, Among the mission information assigned to the robot 100, status information of the robot 100 (e.g., power status, failure status, battery status, etc.), information regarding the communication status between the electronic device 110 and the robot 100 It may include at least one, but is not limited to this, and may be understood as various information that can identify at least one of the electronic device 110 and the robot 100.

According to one embodiment, the server 20 may confirm image information by performing decoding on data (or image information) received through the communication unit 130. Additionally, the server 20 can identify at least one of the electronic device 110 and the robot 100 by identifying recognition information included in the image information. However, a detailed explanation of this will be provided later.

Figure 3 is a flowchart showing a method by which an electronic device controls a robot, according to an embodiment. Figure 4 is a flowchart showing a method for a server to generate a control signal to control a robot, according to an embodiment. FIG. 5 is a flowchart showing how the robot control system of FIG. 2 provides control signals for controlling the running of the robot to the robot.

Referring to FIGS. 2 to 5 together, the electronic device 110 according to one embodiment transmits image information acquired through the sensing unit 120 to the server 20 through the communication unit 130, and the server ( By transmitting the control signal generated by 20 to the robot 100 through the interface unit 140, the movement of the robot 100 in the space 10 can be controlled.

According to one embodiment, in operation 301, the electronic device 110 uses the sensing unit 120 to perform visual localization for driving the robot 100 in the indoor space 10. Image information about the indoor space 10 in which 100 drives can be obtained.

For example, the electronic device 110 may acquire an image of the indoor space 10 through the first camera 121 included in the sensing unit 120 in operation 301. For another example, the electronic device 110 may obtain the traveling speed of the robot through the sensing unit 120 in operation 301.

According to one embodiment, the electronic device 110 may generate encrypted data by encrypting image information through the authentication unit 150 in operation 501 so that the server 20 can verify the image information. In addition, the electronic device 110 according to one embodiment is configured to use an encrypted code through the authentication unit 150 in operation 501 so that the server 20 can identify at least one of the electronic device 110 and the robot 100. Authentication information can be inserted into the data.

According to one embodiment, in operation 303, the electronic device 110 may transmit the image information acquired through operation 301 to the server 20 through the communication unit 130. For example, in operation 303, the electronic device 110 sends image information about at least one of the front, side, and top of the robot 100 obtained through the first camera 121 to the server through the communication unit 130. It can be transmitted to (20).

At this time, the image information transmitted to the server 20 in operation 303 may be understood as data that is encrypted through the authentication unit 150 in operation 501 and into which authentication information is inserted.

According to one embodiment, in operation 503, the server 20 may confirm the image information by performing decoding on the image information in response to receiving image information from the electronic device 110 through operation 303. Additionally, in operation 503, the server 20 verifies the authentication information inserted in the encrypted image information in response to receiving image information from the electronic device 110, thereby enabling the electronic device 110 and the robot 100 to At least one can be identified.

According to another embodiment (not shown), the encryption operation of operation 501 and the decryption operation of operation 503 may be omitted.

According to one embodiment, the server 20 may perform visual localization for the robot 100 in the indoor space 10 using image information received from the electronic device 110 in operation 401. . Specifically, in operation 401, the server 20 may perform visual positioning to estimate the location of the robot 100 in the indoor space 10 using image information received from the electronic device 110.

Furthermore, the server 20 may compare the image information received from the electronic device 110 and the map information stored in the server 20 to extract location information corresponding to the current location of the robot 100. At this time, the map information about the space 10 may include a map created in advance by at least one robot moving the space 10 based on SLAM (Simultaneous Localization and Mapping). In particular, the map for the space 10 may be a map previously generated based on previously acquired image information for the space 10. In this case, the server 20 i) uses image comparison between the image information and the images constituting the previously generated map to specify the image most similar to the image information, and ii) provides location information matched to the specified image. The location information of the robot 100 can be specified by obtaining it.

That is, in operation 401, the server 20 may estimate the current location of the robot 100 within the space 10 through visual positioning using image information received from the electronic device 110.

According to one embodiment, in operation 403, the server 20 may generate a control signal for controlling the movement of the robot 100 in the indoor space 10 based on the result of the visual positioning in operation 401. . Specifically, in operation 403, the server 20 sends a control signal for controlling the running of the robot 100 in the indoor space 10 based on the location information of the robot 100 obtained through visual positioning in operation 401. can be created.

At this time, the control signal may include at least one of the traveling direction, traveling speed, and traveling path of the robot 100, but is not limited thereto, and is understood to include various control commands for traveling the robot 100. It can be.

According to one embodiment, the server 20 may transmit a control signal for controlling the driving of the robot 100 to the electronic device 110 in operation 305. Meanwhile, in operation 305, the electronic device 110 may receive a control signal generated based on the result of visual positioning from the server 20 through the communication unit 130.

According to one embodiment, the electronic device 110 may transmit a control signal to the robot 100 through the interface unit 140 in operation 307. More specifically, the electronic device 110 may transmit the control signal received from the server 20 in operations 307 and 305 to the robot 100 through the interface unit 140.

In operation 307, the electronic device 110 according to one embodiment transmits the control signal received from the server 20 through the communication unit 130 to the robot 100 through the interface unit 140, thereby maintaining the indoor space ( 10) The running of the robot 100 within the space can be controlled.

As described above, the electronic device 110 according to the present invention transmits image information about the indoor space 10 to the server 20, and the server 20 generates control based on visual positioning using the image information. By transmitting a signal to the robot 100, the driving of the robot 100 in the indoor space 10 can be controlled.

FIG. 6 is a flowchart showing how the robot control system of FIG. 2 provides a control signal generated using robot image information to the robot.

Referring to FIGS. 2 to 6 together, the electronic device 110 according to an embodiment may transmit robot image information received from the robot 100 to the server 20 along with the image information. More specifically, the electronic device 110 transmits the robot image information received from the robot 100 to the server 20 together with the image information, thereby generating a control signal based on visual positioning using the image information and the robot image information. can be transmitted to the robot 100. However, the same reference numbers are used for components that are identical or substantially the same as the above-described components, and overlapping descriptions are omitted.

According to one embodiment, the robot 100 may acquire robot image information through a robot sensor disposed on the robot 100 in operation 601. At this time, the robot image information can be understood as image information acquired about the indoor space 10 in which the robot 100 runs through a robot sensor disposed on the robot 100.

Meanwhile, the robot sensor disposed on the robot 100 may be disposed to be spaced apart from the first camera 121 of the electronic device 110. For example, the robot sensor is arranged to have a first height from the ground on the robot 100, and the first camera 121 of the electronic device 110 has a second height higher than the first height on the electronic device 110. It can be arranged to have For another example, the first camera 121 of the electronic device 110 may be arranged to face the front of the robot 100, and the robot sensor of the robot 100 may be arranged to face the rear of the robot 100. However, the arrangement of the first camera 121 and the robot sensor is not limited to the above-described examples.

According to one embodiment, the electronic device 110 may receive robot image information acquired through a sensor of the robot 100 from the robot 100 through the interface unit 140 in operation 603.

Furthermore, in operation 605, the electronic device 110 may transmit both image information and robot image information to the server 20 through the communication unit 130. More specifically, in operation 605, the electronic device 110 transmits the image information acquired through the sensing unit 120 and the robot image information acquired from the robot 100 to the server 20 through the communication unit 130. Can be transmitted. At this time, the image information and robot image information may be transmitted sequentially or simultaneously.

The server 20 according to one embodiment performs visual positioning using image information and robot image information received from the electronic device 110, and controls the driving of the robot 100 based on the results of the visual positioning. A control signal can be generated for More specifically, the server 20 may perform visual positioning using image information and robot image information acquired at different locations from the electronic device 110, and generate a control signal based on the results of the visual positioning. there is.

That is, the electronic device 110 according to the present invention transmits the image information acquired through the sensing unit 120 and the robot image information received from the robot 100 to the server 20, and the server 20 transmits the image information. And by transmitting a control signal generated based on the result of visual positioning using robot image information to the robot 100, the driving of the robot 100 in the space 10 can be controlled.

FIG. 7 is a conceptual diagram for explaining an electronic device and a robot control system further including a second camera, an inertial sensor, and a processor, according to an embodiment. FIG. 8 is a flowchart showing a method of acquiring driving assistance information based on depth information and inertia information acquired by the sensing unit of FIG. 7 and transmitting it to the robot. FIG. 9 is a flowchart showing how the robot control system of FIG. 7 provides control signals for controlling the running of the robot to the robot.

Referring to FIGS. 7 to 9 together, the electronic device 110 according to one embodiment includes image information acquired through the first camera 121, depth information acquired through the second camera 122, and an inertial sensor. At least part of the inertial information obtained through 123 may be transmitted to the server 20, and a control signal generated based on visual positioning using the transmitted information may be transmitted from the server 20 to the robot 100. However, the same reference numbers are used for components that are identical or substantially the same as the above-described components, and overlapping descriptions are omitted.

According to one embodiment, the electronic device 110 may include a processor 160 connected to the sensing unit 120.

The processor 160 according to one embodiment may be configured to control the overall operation of the electronic device 110 mounted on the robot 100 related to the present invention. For example, the processor 160 may process signals, data, information, etc. input or output through the components of the electronic device 110 discussed above.

In particular, the processor 160 may process information obtained through the sensing unit 120 and generate driving assistance information to assist the autonomous driving of the robot 100. At this time, the driving assistance information may include information about obstacles adjacent to the robot 100 in the space 10 in which the robot 100 drives.

According to one embodiment, the sensing unit 120 includes a first camera 121 that acquires image information about the space 10, a second camera 122 that acquires depth information about the space 10, and a robot ( It may include at least one of the inertial sensors 123 that acquire inertial information of 100).

According to one embodiment, the electronic device 110 may acquire depth information about the space 10 and inertia information of the robot 100 through the sensing unit 120 in operation 801. More specifically, in operation 801, the electronic device 110 acquires depth information about the space 10 through the second camera 122 and acquires inertial information of the robot 100 through the inertial sensor 123. You can.

According to one embodiment, the first camera 121 included in the sensing unit 120 is used to obtain image information about the space 10 for the server 20 to perform visual positioning for the robot 100. You can. At this time, the first camera 121 can acquire image information in front, on the sides, and above the robot 100. For example, the first camera 121 may include a lens (eg, a wide-angle lens, a fisheye lens) having a field of view (FOV) of 130 degrees or more vertically and 180 degrees or more horizontally.

According to one embodiment, the first camera 121 of the sensing unit 120 is a 1-1 camera 121A arranged to be aligned with the front of the robot 100 and a first camera 121A arranged on the upper surface of the electronic device 110. It may include 1-2 cameras (121B). At this time, the electronic device 110 acquires image information about the front and sides of the robot 100 through the 1-1 camera 121A and the upper side of the robot 100 through the 1-2 camera 121B. You can obtain image information about . That is, the sensing unit 120 acquires image information about the front, side, and top of the indoor space 10 where the robot 100 runs through the 1-1 camera 121A and the 1-2 camera 121B. can do.

According to one embodiment, the second camera 122 included in the sensing unit 120 may be understood as a depth camera that acquires depth information about the indoor space 10. For example, the second camera 122 may be a stereo depth camera using a three-dimensional image generated using two or more image sensors, a time-of-flight (ToF) depth camera using the delay time of an optical signal, Alternatively, it may be referred to as one of the structured pattern type depth cameras that project structured light onto an object, but is not limited to this.

According to one embodiment, the second camera 122 included in the sensing unit 120 is positioned to face the rear of the robot 100 and the 2-1 camera 122A is arranged to face the front of the robot 100. It may include a 2-2 camera (122B) disposed. Furthermore, the electronic device 110 acquires depth information about the front space of the robot 100 through the 2-1 camera 122A and the rear space of the robot 100 through the 2-2 camera 122B. Depth information can be obtained. At this time, the 2-1 camera 122A and the 2-2 camera 122B may each include a lens having a vertical field of view (FOV) within 90 degrees and a horizontal field of view within 90 degrees. You can.

According to one embodiment, the sensing unit 120 may include an inertial sensor 123 (or an inertial measurement unit (IMU) sensor) that acquires inertial information about the running of the robot 100. The electronic device 110 according to one embodiment may measure the inertial force of the robot 100 through the inertial sensor 123 included in the sensing unit 120.

The inertial sensor 123 according to one embodiment may include an acceleration sensor, a gyro sensor, and a geomagnetic sensor. The inertial sensor 123 according to an embodiment measures acceleration using an acceleration sensor, measures angular velocity using a gyro sensor, and corrects the acceleration and angular velocity obtained through a geomagnetic sensor, thereby increasing the inertial force of the robot 100. Alternatively, the tilt angle can be measured.

According to one embodiment, in operation 803, the processor 160 assists the running of the robot 100 using the depth information received from the second camera 122 and the inertial information received from the inertial sensor 123. Driving assistance information can be generated. For example, the processor 160 uses the depth information about the space 10 received from the second camera 122 and the inertial information received from the inertial sensor 123, within the space 10 in operation 803. Information about obstacles adjacent to the robot 100 (e.g., location of obstacle, size of obstacle) may be generated.

According to one embodiment, the electronic device 110 may transmit depth information and inertia information to the server 20 through the communication unit 130 in operation 901. More specifically, the electronic device 110 transmits depth information and inertial information to the server 20 so that the server 20 generates a control signal based on visual positioning using depth information and inertial information along with image information. You can.

According to one embodiment, the server 20 drives the robot 100 based on visual positioning using image information received from the electronic device 110 in operations 401 and 403, and at least one of depth information and inertial information. A control signal to control can be generated. For example, the server 20 may generate a control signal based on visual positioning using image information and depth information received from the electronic device 110. For another example, the server 20 may generate a control signal based on visual positioning using image information received from the electronic device 110 and inertial information of the robot 100.

According to one embodiment, in operation 805, the electronic device 110 may transmit at least one of driving assistance information and a control signal to the robot 100 through the interface unit 140. For example, in operation 805, the electronic device 110 may transmit driving assistance information and a control signal to the robot 100 through the interface unit 140. For another example, the electronic device 110 may sequentially transmit driving assistance information and control signals to the robot 100 through the interface unit 140 in operation 805. For another example, the electronic device 110 transmits the driving assistance information generated through operation 803 to the robot 100, and after a certain time has elapsed, the electronic device 110 transmits the control signal transmitted from the server 20 to the robot 100. It can be sent to . However, the order or cycle in which the electronic device 110 transmits driving assistance information and control signals to the robot 100 is not limited to the above-described examples.

As described above, the electronic device 110 according to the present invention transmits at least one of depth information and inertial information along with image information about the indoor space 10 to the server 20, so that the server 20 can use the indoor space ( The precision of visual positioning for estimating the position of the robot 100 in 10) can be improved. Furthermore, the electronic device 110 transmits a control signal generated by the server 20 based on visual positioning using image information, depth information, and inertial information to the robot 100, thereby controlling the robot (100) in the indoor space (100). 100) driving can be controlled.

Furthermore, the electronic device 110 according to the present invention controls the driving of the robot 100 not only through a control signal provided from the server 20, but also through driving assistance information based on information acquired by the electronic device 110. , it is possible to provide a robot control environment that can flexibly cope with obstacles in the indoor space 10.

FIG. 10 is a flowchart showing how the robot control system of FIG. 7 provides a control signal generated using robot image information to the robot.

Referring to FIGS. 7 to 10 together, the electronic device 110 according to an embodiment may transmit robot image information received from the robot 100 to the server 20 along with the image information. More specifically, the electronic device 110 transmits the robot image information received from the robot 100 to the server 20 together with the image information, thereby generating a control signal based on visual positioning using the image information and the robot image information. can be transmitted to the robot 100. However, the same reference numbers are used for components that are identical or substantially the same as the above-described components, and overlapping descriptions are omitted.

According to one embodiment, the robot 100 may acquire robot image information through a robot sensor disposed on the robot 100 in operation 601. At this time, the robot image information can be understood as image information acquired through a robot sensor placed on the robot 100 about the indoor space 10 in which the robot 100 runs.

The server 20 according to an embodiment performs visual positioning using at least some of the image information, robot image information, depth information, and inertial information received from the electronic device 110, and based on the results of the visual positioning, A control signal for controlling the driving of the robot 100 can be generated.

That is, the electronic device 110 according to the present invention transmits the image information, depth information, and inertia information acquired through the sensing unit 120 and the robot image information received from the robot 100 to the server 20, and the server (20) transmits the control signal generated based on the results of visual positioning using image information, depth information, inertial information, and robot image information to the robot 100, thereby driving the robot 100 in space 10. can be controlled.

As described above, the electronic device 110 according to the present invention transmits the robot image information measured by the robot 100 together with the image information about the indoor space 10 to the server 20, so that the server 20 The precision of visual positioning that estimates the position of the robot 100 in space 10 can be improved. Furthermore, the electronic device 110 transmits a control signal generated by the server 20 based on visual positioning using image information and robot image information to the robot 100, thereby controlling the robot 100 in the indoor space 10. You can control the driving of .

Figure 11 is a conceptual diagram for explaining an electronic device and a robot control system mounted on a robot, according to another embodiment. FIG. 12 is a flowchart showing a method by which the electronic device of FIG. 11 transmits image information to a robot. FIG. 13 is a flowchart showing how the robot control system of FIG. 11 provides control signals for controlling the running of the robot to the robot.

As shown in FIG. 13, the robot control system of the present invention includes an electronic device 1210 mounted on the robot 100 and a server ( 20) may be included. At this time, the robot 100 may transmit image information through communication with the server 20 and receive a control signal generated by the server 20.

Referring to FIGS. 11 to 13 together, the electronic device 1210 according to one embodiment includes a sensing unit 1220 that acquires image information about the space 10 in which the robot 100 runs, and encryption for the image information. It may include an authentication unit 1250 that performs and inserts authentication information, and an interface unit 1240 that performs data communication with the robot 100.

At this time, the sensing unit 1220, the authentication unit 1250, and the interface unit 1240 have substantially the same configuration as the sensing unit 120, the authentication unit 150, and the interface unit 140 of FIG. 2 or 7, respectively. It can be understood as Accordingly, overlapping descriptions of components that are substantially the same as those described above will be omitted.

At this time, the robot 100 may include substantially the same configuration as the communication unit 130 included in the electronic device 110 of FIG. 2 or 7. For example, the robot 100 may be communicatively connected to the server 20 through communication technologies such as LTE, LTE-A, 5G, and Ethernet.

According to one embodiment, the electronic device 1210 determines the indoor area where the robot 100 runs, obtained through the sensing unit 1220 (or the first camera 1221 included in the sensing unit 1220) in operation 1201. Image information about the space 10 can be obtained. At this time, the first camera 1221 may be understood to have substantially the same configuration as the first camera 121 of FIG. 2 or 7.

According to one embodiment, the authentication unit 1250 can generate encrypted data by encrypting the image information obtained from the sensing unit 1220 so that the server 20 with preset confirmation authority can verify the image information. there is. In addition, the authentication unit 1250 provides authentication related to the robot 100 and the electronic device 1210 to the encrypted data so that the server 20 can identify at least one of the robot 100 and the electronic device 1210. Information can be added.

According to one embodiment, the electronic device 1210 may transmit image information to the robot 100 through the interface unit 1240 so that the image information is transmitted to the server 20 in operation 1203.

According to one embodiment, the robot 100 may transmit image information received from the electronic device 1210 to the server 20 in operation 1205.

According to one embodiment, in operation 1207, the server 20 performs visual positioning using image information received from the robot, and based on the results of the visual positioning, moves the robot 100 in the indoor space 10. A control signal for control can be generated.

According to one embodiment, in operation 1209, the server 20 generates a control signal based on image information acquired from the electronic device 1210 and transmits a control signal for controlling the running of the robot 100 to the robot 100. It is possible.

As described above, the present invention provides that when the robot 100 is communicatively connected to the server 20 or includes a configuration capable of communicating with the server 20, the information acquired by the electronic device 1210 is transmitted to the robot 100. ), the server 20 can generate a control signal for controlling the running of the robot 100 and transmit it to the robot 100.

FIG. 14 is a flowchart showing how the robot control system of FIG. 11 provides a control signal generated using robot image information to the robot.

Referring to FIGS. 11 to 14 together, the robot 100 according to one embodiment transmits the robot image information acquired using the robot sensor disposed on the robot 100 and the image information received from the electronic device 1210 to the server. It can be transmitted to (20). The robot 100 transmits the robot image information and the image information received from the electronic device 1210 to the server 20, so that the server 20 generates a control signal based on the robot image information and visual positioning using the image information. can receive.

However, the same reference numbers are used for components that are identical or substantially the same as the above-described components, and overlapping descriptions are omitted.

According to one embodiment, the robot 100 may acquire robot image information through a robot sensor disposed on the robot 100 in operation 1401. At this time, the robot image information can be understood as image information acquired through a robot sensor placed on the robot 100 with respect to the indoor space 10 in which the robot 100 runs.

According to one embodiment, the robot 100 may transmit image information and robot image information received from the electronic device 1210 to the server 20 in operation 1403.

In operation 1207, the server 20 according to an embodiment performs visual positioning using at least some of the image information and robot image information received from the electronic device 1210 through the robot 100, and provides a result of the visual positioning. Based on this, a control signal for controlling the running of the robot 100 can be generated.

The server 20 according to one embodiment may transmit the control signal generated through operation 1207 to the robot 100 in operation 1209. The robot 100 may receive a control signal from the server 20 in operation 1209.

The robot 100 according to the present invention transmits image information acquired from the electronic device 1210 and robot image information obtained through the robot sensor to the server 20, and the server 20 transmits the image information and robot image information. A control signal generated based on the results of visual positioning can be received.

As described above, the robot 100 according to the present invention transmits the robot image information measured by the robot 100 together with the image information received from the electronic device 1210 to the server 20, so that the server 20 The accuracy of visual positioning for estimating the position of the robot 100 in the indoor space 10 can be improved. Furthermore, the robot 100 may receive a control signal generated by the server 20 based on visual positioning using image information and robot image information.

Meanwhile, the present invention discussed above can be implemented as a program that is executed by one or more processes on a computer and can be stored in a medium that can be read by such a computer.

Furthermore, the present invention discussed above can be implemented as computer-readable codes or instructions on a program-recorded medium. That is, the present invention may be provided in the form of a program.

Meanwhile, computer-readable media includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is.

Furthermore, the computer-readable medium may be a server or cloud storage that includes storage and can be accessed by electronic devices through communication. In this case, the computer can download the program according to the present invention from a server or cloud storage through wired or wireless communication.

Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, that is, a CPU (Central Processing Unit), and there is no particular limitation on its type.

Meanwhile, the above detailed description should not be construed as restrictive in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (16)

In the electronic device mounted on the robot,
A sensing unit that acquires image information about the indoor space in order to perform visual positioning for the robot to run in the indoor space;
an interface unit that performs data communication between the electronic device and the robot; and
Comprising a communication unit that communicates with a server and transmits the image information to the server so that the server performs the visual positioning using the image information,
In response to receiving, from the server, a control signal generated based on the result of the visual positioning, the control signal received from the server is transmitted to the interface so that the robot runs in the indoor space based on the control signal. An electronic device, which passes through the unit to the robot. According to paragraph 1,
The sensing unit includes a first camera that acquires the image information,
The first camera is arranged on the electronic device to be aligned with the front of the robot. According to paragraph 2,
The robot includes a robot sensor arranged to be spaced apart from the first camera of the electronic device,
The interface unit receives robot image information obtained from the robot through the robot sensor disposed on the robot,
The communication unit transmits the image information and the robot image information acquired through the first camera to the server. According to paragraph 3,
In the server, the visual positioning is performed using the image information and the robot image information acquired through the first camera on the electronic device,
In response to the communication unit receiving the control signal generated based on the image information and the result of the visual positioning using the robot image information from the server, the control signal is transmitted to the robot through the interface unit. , electronic devices. According to paragraph 1,
It further includes an authentication unit that performs encryption on the video information and generates encrypted data so that the server with preset confirmation authority can confirm the video information,
The authentication unit inserts authentication information capable of identifying at least one of the electronic device and the robot into the encrypted data,
An electronic device that transmits the encrypted data to the server through the communication unit. According to paragraph 1,
The electronic device generates the driving assistance information using the sensing information obtained through the sensing unit so that the driving of the robot is controlled based on the control signal received from the server and the driving assistance information generated by the electronic device. Create a ,
The sensing information includes the image information, depth information about the indoor space, and inertial information of the robot. According to clause 6,
The sensing unit further includes at least one of a second camera for acquiring the depth information and an inertial sensor for acquiring the inertial information,
The electronic device further includes a processor that generates the driving assistance information to assist the robot in driving in the indoor space, based on the depth information and the inertial information,
The control signal received from the server and the driving assistance information received from the processor are transmitted to the robot through the interface unit. In the robot control system,
A server that generates control signals to control the robot's driving in an indoor space; and
It includes an electronic device mounted on the robot, acquiring information related to the robot, and transmitting the information related to the robot to the server through the robot,
The electronic device:
The server acquires image information about the indoor space through a sensing unit so that visual localization for driving control of the robot is performed,
Transmitting the image information to the robot through an interface unit connected to perform data communication with the robot so that the image information is transmitted to the server through the robot,
Said server:
In response to receiving the image information acquired in the electronic device from the robot, perform the visual localization for the robot based on the image information,
Generating a control signal to control the movement of the robot in the indoor space based on the results of the visual positioning,
A robot control system that transmits the control signal for controlling the running of the robot, which is generated based on the image information acquired from the electronic device, to the robot. According to clause 8,
The electronic device adds authentication information to the image information so that the server can identify at least one of the electronic device and the robot through an authentication unit and transmits it to the robot,
Said server:
Identifying at least one of the electronic device and the robot from the authentication information received from the robot and added to the image information,
A robot control system that generates the control signal for controlling the driving of the robot in the indoor space based on the authentication information and the image information. According to clause 9,
The authentication unit generates encrypted data by performing encryption on the image information so that the server with preset confirmation authority can verify the image information,
A robot control system, wherein the server confirms the image information by performing decryption on the image information in response to receiving the encrypted data through the robot. According to clause 8,
The server is:
Receiving robot image information obtained from the robot through a robot sensor disposed on the robot,
Performing the visual positioning using the image information and the robot image information,
A robot control system that generates the control signal for controlling the movement of the robot in the indoor space based on the results of the visual positioning. According to clause 8,
The sensing unit further includes a first camera that acquires the image information, a second camera that acquires depth information about the indoor space, and an inertial sensor that acquires inertial information of the robot,
The electronic device further includes a processor that obtains driving assistance information that assists the robot in driving in the indoor space based on the depth information and the inertial information.
The electronic device transmits the driving assistance information received from the processor to the robot through the interface unit,
The server is a robot control system that transmits the control signal generated based on the image information to the robot. In the robot control method,
Obtaining image information about the indoor space through a sensing unit of an electronic device mounted on the robot to perform visual localization for driving the robot in the indoor space;
transmitting the image information to the server so that the server performs the visual localization; and
In response to receiving, from the server, a control signal generated based on the result of the visual localization, transmitting the control signal to the robot to cause the robot to travel in the indoor space based on the control signal. How to control a robot. According to clause 13,
Obtaining depth information about the indoor space and inertia information of the robot through the sensing unit;
Based on the depth information and the inertia information, obtaining driving assistance information to assist the robot in driving in the indoor space; and
A robot control method comprising transmitting the driving assistance information and the control signal received from the server to the robot . According to clause 13,
Receiving robot image information acquired by a robot sensor disposed on the robot from the robot;
Transmitting the image information and the robot image information together to the server; and
In response to the server receiving, from the server, the control signal generated based on the image information and a result of the visual positioning using the robot image information, transmitting the control signal to the robot. , Robot control method. A program that is executed by one or more processes in an electronic device and stored on a computer-readable recording medium,
The above program is,
Obtaining image information about the indoor space through a sensing unit of an electronic device mounted on the robot to perform visual localization for driving the robot in the indoor space,
transmitting the image information to the server so that the server performs the visual localization, and
In response to receiving, from the server, a control signal generated based on a result of the visual localization, transmitting the control signal to the robot so that the robot runs in the indoor space based on the control signal. A program stored on a computer-readable recording medium that includes instructions to be executed.

Images