KR20230015144A - Different typed robot system having edge server and cloud server and control method thereof - Google Patents

Abstract

The present invention relates to a cloud-based robot control method for controlling a plurality of robots which are positioned in a plurality of spaces divided arbitrarily. The method comprises the steps of: generating a control base model which can be applied to the plurality of robots in a cloud server; distributing the control base model to edge servers allocated to respective spaces; upgrading the control base model in accordance with the plurality of robots of the spaces, in the edge server; directly transmitting the upgraded control model from the edge server to another edge server; and controlling the plurality of robots by means of the upgraded control model in the edge server. Therefore, by sharing a deep-learning model among edge servers, supporting heterogeneous robots and heterogeneous services is possible. Moreover, a base deep-learning model from the cloud server is tuned into a customized deep-learning model to be suitable for respective robots in the edge server, and the deep-learning model is upgraded to an adaptive deep-learning model to be suitable for a service provided by respective robots. Therefore, an optimized service can be provided.

Description

Different typed robot system having edge server and cloud server and control method thereof

The present invention provides a robot system including a plurality of edge servers that control heterogeneous robots and a cloud server that performs transmission and reception. Specifically, the present invention relates to a method for controlling a heterogeneous robot system using a cloud server providing a base model.

Robots have been developed for industrial use and have been in charge of a part of factory automation. In recent years, the field of application of robots has been further expanded, and medical robots, space robots, etc. have been developed, and household robots that can be used at home are also being made. Among these robots, those that can drive on their own are called artificial intelligence robots.

With the increase in the use of robots, there is an increasing demand for robots that can provide various information, fun, and services beyond the repetitive performance of simple functions.

Accordingly, various robots are being developed that provide convenience to people by being placed in homes, restaurants, shops, public places, and the like.

In addition, heterogeneous robot systems are being developed in which different types of robots are arranged in a defined space to perform respective tasks.

Such a heterogeneous robot system can be defined as the convergence of Information and Communication Technologies (ICT) and the existing service industry.

For example, the heterogeneous robot system is based on technologies such as Internet of Things, Big data, Cloud computing, and CPS (Cyber-Physical System) for various events in the space. It is defined as a system of processing and communication.

Cloud computing utilizes Internet technology to provide virtualized ICT resources as a service.

Users can use information and communication technology resources (eg, servers, storage, networks, and software) as needed through cloud computing.

In addition, the user can receive service scalability in real time according to the load of the service provided through cloud computing, and pays for the provision of the service.

When a cloud server controls numerous heterogeneous robots through cloud computing, an edge server may be further included for each space for control or communication convenience.

As such, a robot system including an edge server that controls a plurality of robots at a short distance and a cloud server that transmits and receives data to and from the plurality of edge servers has been developed.

This robot system generates a lot of data traffic between the cloud server and the edge server.

US Patent Publication No. 2020-0050951 discloses that one of the edge nodes generates a specification of a machine learning model and distributes the specification to a plurality of edge nodes.

That is, a model is updated by performing machine learning with each other through parameter exchange between a plurality of edge nodes.

However, when parameters are exchanged between edge nodes, it is difficult to support heterogeneous robots or heterogeneous services.

In addition, in the case of US Patent Publication No. 2020-0079898, the edge server performs a specific action through reasoning through a machine learning model, evaluates whether the specific action is accurate, and if the specific action is incorrect, the edge server responds to the specific action It is disclosed to collect and transmit related data to a cloud server, and to learn a new machine learning model using the data collected in the cloud server.

However, in this case, since a large amount of data must be transmitted and received between the edge server and the cloud server, a lot of time and cost are incurred, and a new machine learning model must be learned, which is very inefficient in the edge server.

Meanwhile, Korean Patent Publication No. 2020-0063340 discloses that an edge server processes data requiring real-time, and a central processing server processes advanced machine learning and large-capacity data. At this time, the central processing server starts converting the general-purpose deep learning neural network into a deep learning neural network of a product similar to the specific product to be learned, and relearning the converted deep learning neural network as a training data set of the specific product. .

However, in this case, there is a disadvantage in that a lot of time and processing are required for re-learning.

US Patent Publication No. 2020-0050951, published on February 13, 2020 US Patent Publication No. 2020-0079898, published on March 14, 2019 Korean Patent Publication No. 2020-0063340, published on June 05, 2020

A first object of the present invention is to provide a service for efficient intelligence augmentation by sharing a deep learning model for controlling heterogeneous robots between edge servers.

To this end, the second task of the present invention is to tune the base deep learning model to a custom deep learning model suitable for each robot, and upgrade to an adaptive deep learning model suitable for the service provided by each robot. It is to provide a control method using a server.

In addition, the third task of the present invention is to select a deep learning model necessary for intelligence augmentation between edge servers, share it directly, and replace and expand the existing deep learning model with the shared deep learning model in the edge server, It is to provide a control method capable of minimizing the processing data passing through the server.

The present invention includes a plurality of robots disposed in a plurality of spaces that are arbitrarily divided; a cloud server generating and distributing a control base model applicable to the plurality of robots; And a plurality of edge servers allocated to each space, transmitting and receiving data to and from the cloud server, receiving the control base model, upgrading the plurality of robots in the space, and controlling the plurality of robots with the upgraded control model. And, the edge server provides a cloud-based robot system, characterized in that the upgraded control model is directly transmitted to other edge servers.

The plurality of robots may include different types of robots.

As the control base model, each control model for a plurality of functions of the robot of a different type may be packaged.

The edge server may receive the control base model, tune and execute the control base model according to the type of robot controlled by the edge server.

The edge server may generate an upgraded control model by performing deep learning error learning on the tuned control base model.

The cloud server may obtain information about the upgraded control model, select another edge server to which the upgraded control model is applied, and transmit the upgraded control model directly between the edge servers.

The cloud server may transmit the upgraded control model by selecting the edge server including the robot to which the upgraded control model is applied.

The edge server may generate the upgraded control model by performing the deep learning error learning when an error value exceeding a threshold value is equal to or greater than a predetermined number while executing the control base model.

The cloud server may classify the plurality of edge servers and manage them into a plurality of groups.

One group can be located within a predetermined distance or within a predetermined response time.

On the other hand, another embodiment of the present invention, in the cloud-based robot control method for controlling a plurality of robots disposed in a plurality of spaces that are arbitrarily divided, a control base model applicable to the plurality of robots in the cloud server generating; distributing the control base model to an edge server allocated to each space; upgrading the control base model according to the plurality of robots in the space at the edge server; directly transmitting the upgraded control model from the edge server to another edge server; and controlling the plurality of robots with the upgraded control model in the edge server.

The plurality of robots may include different types of robots.

In the control base model generating step, each control model for a plurality of functions of the robot of a different type may be generated and packaged.

The method may further include receiving the control base model from the edge server and tuning the control base model according to a type of robot controlled by the edge server.

In the upgrading, an upgraded control model may be generated by performing deep learning error learning on the tuned control base model.

Obtaining information on the upgraded control model from the cloud server, selecting another edge server to which the upgraded control model is applied, transmitting information of the selected other edge server to the upgraded edge server, and directly transmitting the upgraded control model from the upgraded edge server to the selected other edge server.

In the step of selecting another edge server, the cloud server may select an edge server including a robot to which the upgraded control model is applied.

In the upgrading step, the upgraded control model may be generated by performing the deep learning error learning when an error value exceeding a threshold value is equal to or greater than a predetermined number while executing the control base model in the edge server.

The method may further include classifying and grouping a plurality of edge servers in the cloud server.

One group can be located within a predetermined distance or within a predetermined response time.

Through the above solution, the present invention makes it possible to support heterogeneous robots and heterogeneous services by sharing deep learning models between edge servers.

In addition, the base deep learning model from the cloud server is tuned to a custom deep learning model suitable for each robot on the edge server, and upgraded to an adaptive deep learning model suitable for the service provided by each robot, thereby providing optimized services. it is possible to provide

Between the edge servers, the deep learning model required for intelligence augmentation is selected and directly shared, and the existing deep learning model in the edge server is replaced and expanded with the shared deep learning model, thereby minimizing the processing data passing through the cloud server. can

In addition, it is possible to efficiently increase knowledge by learning by updating an existing deep learning model with a shared deep learning model of another edge server based on one base model from the cloud server.

1 is a schematic diagram illustrating a cloud and edge-based environment according to an embodiment of the present invention.
2 is a conceptual diagram illustrating zone classification of a cloud server according to the present invention.
3 is a detailed view for explaining a heterogeneous robot system based on one edge server of the present invention.
4 is a configuration diagram according to an application example of the cloud server or edge server of FIG. 3 .
5 is a conceptual diagram illustrating modeling of a cloud and an edge server according to an embodiment of the present invention.
6 is an overall flowchart of the robot cloud system of the present invention.
FIG. 7 is a detailed flow chart illustrating how one edge server enters the system of FIG. 6 .
8 is a flowchart illustrating that one edge server of FIG. 6 upgrades a base model.
9 is a flowchart illustrating a method for sharing an upgrade model of another edge server according to an embodiment of the present invention.

Expressions referring to directions such as “front (F) / back (R) / left (Le) / right (Ri) / up (U) / down (D)” mentioned below are defined as shown in the drawings, , This is only for explaining the present invention so that it can be clearly understood, and each direction may be defined differently depending on where the reference is placed, of course.

The use of terms with expressions such as 'first, second' in front of the components mentioned below is only to avoid confusion between the components referred to, and has nothing to do with the order, importance, or master-servant relationship between the components. . For example, an invention including only the second component without the first component may be implemented.

In the drawings, the thickness or size of each component is exaggerated, omitted, or schematically illustrated for convenience and clarity of explanation. Also, the size and area of each component do not entirely reflect the actual size or area.

In addition, angles and directions mentioned in the process of describing the structure of the present invention are based on those described in the drawings. In the description of the structure in the specification, when reference points and positional relationships for angles are not clearly mentioned, reference is made to related drawings.

1 is a schematic diagram for explaining a cloud and edge-based environment according to an embodiment of the present invention, FIG. 2 is a conceptual diagram showing zone classification of a cloud server of the present invention, and FIG. 3 is an edge server of the present invention It is a detailed diagram for explaining the heterogeneous robot system based on .

Referring to FIG. 1 , a cloud-based robot system according to an embodiment of the present invention may include a cloud server 10, a plurality of edge servers 20, and a plurality of robots 30.

The cloud-based robot 30 system integrally manages and controls a plurality of robots 30 distributed in various spaces spaced apart from each other.

At this time, each space is functionally or geographically separated, and each edge server 20 for controlling a plurality of robots 30 disposed in each space is set.

The robots 30 of the same type may be disposed in each space, but unlike this, different types of robots 30 may be disposed.

For example, an edge server 1 (21) is set in space 1 (A1), and one ordering robot 1 (313) and two guide robots 311 and 312 may be disposed. Meanwhile, the edge server 2 22 may be set in space 2, and two delivery robots 321 and 323 and one guide robot 322 may be disposed. Meanwhile, in space 3, an edge server n 23 may be set, and an entertaining robot 331 and a guide robot 332 may be disposed.

The heterogeneous or homogeneous robots 30 disposed in each space transmit/receive with the edge server 20 set in the corresponding space, and interact with users at home or business sites to provide assigned services.

Preferably, in the cloud robot system according to an embodiment of the present invention, a plurality of heterogeneous robots 30 and each edge server 20 are set in each space, transmit and receive data with each edge server 20, and manage the integrated It includes a cloud server 10 that provides.

The edge server 20 can monitor and control the status of the plurality of robots 30 remotely, so that the cloud robot system can provide more effective services using the plurality of robots 30 .

The plurality of heterogeneous robots 30, the edge server 20, and the cloud server 10 are equipped with communication means (not shown) supporting one or more communication standards, and can communicate with each other.

For example, the plurality of heterogeneous robots 30, the edge server 20, and the cloud server 10 are wireless devices such as IEEE 802.11 WLAN, IEEE 802.15 WPAN, UWB, Wi-Fi, Zigbee, Z-wave, and Blue-Tooth. It may be implemented in wireless communication as a communication technology. The robot 30 may vary depending on the communication method of the server 2 or another device to communicate with.

In particular, the plurality of robots 30 may implement wireless communication with other robots 30 and/or the edge server 20 and the cloud server 10 through a 5G network. When the robot 30 wirelessly communicates through a 5G network, an ultra-low latency/ultra-high-capacity data transmission network can be performed.

In more detail, the 5G network is a communication technology that provides a transmission speed of several tens of Gbps in a wireless section. It is a technology capable of ultra-low-latency data transmission. Such a 5G network can provide network quality equivalent to that of a high-speed wired network while simultaneously providing the advantage of being wireless.

This 5G network is a cloud machine learning / deep learning-based robot system between the robot 30 and the edge server 20, between the edge server 20 and the cloud server 10, and various types of robots optimized for each of the heterogeneous robots 30. control to provide services.

In addition, the plurality of robots 30 and the edge server 20 may communicate in a message queuing telemetry transport (MQTT) method or in a hypertext transfer protocol (HTTP) method, but are not limited thereto.

Depending on the case, two or more communication standards are supported between the plurality of robots 30, the edge server 20, and the cloud server 10, and the optimal communication standard according to the type of communication data and the type of device participating in communication can be used.

The user can check information about the robots 30 in the robot 30 system while transmitting and receiving with the edge server 20 .

In this specification, a 'user' is a person who uses a service through a plurality of robots 30, an individual customer who purchases or rents robots 30 and uses them in a workplace, and provides services to employees or customers using robots 30. This may include managers, employees of the companies providing them, and customers who use the services provided by these companies. Accordingly, 'users' may include individual customers (Business to Consumer: B2C) and corporate customers (Business to Business: B2B).

The cloud server 10 designs/creates a model for controlling a plurality of physically distributed robots 30 and distributes the model to the edge servers 20 set in each space.

Specifically, the cloud server 10 creates a model or engine in the cloud 10 . In the environment of the heterogeneous robot system, the general-purpose model or general-purpose engine applied to the heterogeneous robot 30 is a base model, and is not designed as a specific model for a specific situation.

The cloud server 10 can form and package a plurality of models or engines that are functionally or structurally distinguished for a plurality of heterogeneous robots 30, respectively, and complex machine learning / deep learning can be performed.

Machine learning (i.e., machine learning) is a field of artificial intelligence, and refers to a technology that generates a model that can generalize, classify, and evaluate numerous data based on algorithms and techniques that allow computers to learn on their own.

By generating such a base model and distributing it to a plurality of edge servers 20, it is possible to control a plurality of heterogeneous robots 30 that are physically separated.

In addition, the cloud server 10 may classify and group a plurality of edge servers 20 by zone, and manage the grouped edge servers 20 by group.

At this time, the zones to be grouped can be set according to the physical distance or response time from the cloud server 10 to the edge server 20 .

As shown in FIG. 2 , a plurality of edge servers 21 and 22 existing within a first distance d1 or a first response time from the cloud server 10 are classified into a first zone z1, and the first distance d1 ) Classify a plurality of edge servers 23, 24, and 25 existing within a second distance d2 greater than or a second response time longer than the first response time into the second zone z2, and other edge servers ( 26 and 27) can be classified into the third zone z3 while being classified into the third distance d3 or the third response time.

In this case, the distances between the first distance d1, the second distance d2, and the third distance d3 may be equal to each other, but are not limited thereto.

In addition, the difference between the first to third response times may be equal to each other, but is not limited thereto. At this time, the response time may be defined as the number of network nodes until the packet arrives or the response time coming through ping, but is not limited thereto.

In this way, when the plurality of edge servers 20 are classified according to zones, modeling distribution scheduling of each edge server 20 in the cloud server 10 can be usefully applied.

The edge server 20 is set in each space as shown in FIG. 3, and tunes and upgrades the base model to an adaptive model to be suitable for a plurality of heterogeneous or homogeneous robots 30 disposed in the space, so that each robot (30) to control.

The general edge server 20 is a technology that processes data in the edge area (Edge) where data is created, departing from the way data is processed in the cloud server 10. Data that requires real time is processed by the edge server 20 It is implemented by communicating with the central cloud as a secondary task when necessary.

However, in the present invention, the cloud server 10 performs base deep learning modeling, which is a general-purpose model or engine for the heterogeneous robot 30 and service, receives the base model from the edge server 20, and It is tuned to suit and provides service as an adaptive deep learning model.

Specifically, the edge server 20 utilizes learning data of a corresponding space previously collected from a plurality of various robots 30 to transfer the base deep learning model from the cloud server 10 to each robot 30 and service. Customize and upgrade to suit your needs.

At this time, the edge server 20 may consider various environmental information about the space together, and can upgrade to an optimal customized deep learning model.

In addition, when providing the service, the edge server 20 transmits feedback (error, performance information, etc.) from the plurality of heterogeneous robots 30 to the cloud server 10, and sends the upgraded model to the other edge server 20. can be shared Accordingly, the other shared edge server 20 removes the existing model and updates the shared model to perform tuning suitable for the robots 30 in the corresponding space.

As shown in FIG. 3, each edge server 20 is disposed in one divided virtual space or physically separated space A1, and a plurality of robots disposed in the space A1 to provide services ( 30) and the service provided by the robot 30 is controlled.

As an example, as shown in FIG. 3, the edge server 21 may control a plurality of robots 30: 31, 32, 33, 34, and 35 providing services in a smart restaurant, and the plurality of robots 30 may include a guide robot, an ordering robot, a delivery robot, an entertaining robot, a cleaning robot or cooking robot, a barista robot, and the like.

In this way, the plurality of heterogeneous robots 30 disposed in the corresponding space A1 are subordinate to the corresponding edge server 21 and provide services while transmitting and receiving data.

The cloud server 10 or the edge server 21 may have a generally similar configuration.

4 is a configuration diagram according to an application example of the cloud server 10 or the edge server 20 of FIG. 3 .

Referring to FIG. 4 , the edge server 20 or the cloud server 10 may include a memory, a computing data collection database 110, a control unit 100, and a communication unit 120.

Specifically, the cloud server 10 or the edge server 20 may be implemented as at least one computing device including a memory and a processor that reads a program stored in the memory and performs a specific function.

First, in the case of the edge server 20, the memory 110 can store and manage data obtained based on IoT technology from a plurality of the same or heterogeneous robots 30 in the space, and the cloud server 10 A base deep learning model received from may be stored.

In the case of the cloud server 10, an algorithm capable of performing processing for base deep learning may be stored.

In addition, the memory 110 may store any one or more of application programs, data, and commands necessary for functional operations according to an embodiment of the present invention.

The memory 110 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and may be a web storage that performs the storage function of the memory 110 on the Internet. may be

Also, in some embodiments, software components stored in memory 110 may include an operating system, a communication module (or set of instructions), a contact/motion module (or set of instructions), a graphics module (or set of instructions), A text input module (or set of instructions), a global positioning system (GPS) module (or set of instructions), and applications (or sets of instructions).

In the case of the edge server 20, the processor 100 may control and drive overall operations of the plurality of robots 30.

These processors 100 include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, It may be implemented using at least one of micro-controllers, microprocessors, and electrical units for performing other functions.

The processor 100 of the edge server 20 can analyze various data obtained from the plurality of robots 30 and perform appropriate processing, and performs machine learning/deep learning on its own to obtain the corresponding base deep learning modeling. You can perform learning for upgrade.

The edge server 20 may perform machine learning/deep learning based on data received from the robot 30, data input by a user, and the like, and then transmit upgraded data to the robot 30. Accordingly, a control program of the robot 30 may be updated.

The processor 100 of the edge server 20 may analyze each of the data obtained from the plurality of robots 30 and apply the analyzed data to a corresponding model or engine.

The processor 100 of the cloud server 10 may have a more advanced processor capable of performing machine learning/deep learning.

Meanwhile, the cloud server 10 or the edge server 20 may include the communication unit 120, and as described above, various communications are possible according to communication methods required in the environment. In particular, communication between the robot 30-edge server 20, edge server 20-cloud server 10, and edge server 20-edge server 20 is possible using a 5G network, and large-capacity deep The algorithm itself of learning modeling can transmit and receive.

On the other hand, various heterogeneous or homogeneous robots 30 disposed in each space transfer space, object, and usage-related data to the edge server 20. can transmit

Here, the data is space, and the object-related data is recognition-related data of the space and object recognized by the robot 30, or the space and object acquired by the image acquisition unit ( It may be image data for Object).

According to an embodiment, the robot 30 and the edge server 20 are artificial neural networks (ANNs) in the form of software or hardware learned to recognize at least one of the properties of objects such as user, voice, property of space, and obstacles. ) may be included.

According to an embodiment of the present invention, the cloud server 10 and the edge server 20 use deep learning-trained convolutional neural networks (CNNs), recurrent neural networks (RNNs), and deep belief networks (DBNs). A deep neural network (DNN) may be included. For example, a deep neural network structure (DNN) such as a convolutional neural network (CNN) may be installed in the processor of the edge server 20 .

In addition, the usage related data is data obtained according to the use of the robot 30, and may correspond to usage history data, a detection signal obtained from the sensor unit 110, and the like.

The robot 30 may receive input data for recognition through the learned deep neural network structure (DNN), recognize attributes of people, objects, and spaces included in the input data, and output the result.

In addition, the learned deep neural network structure (DNN) receives input data for recognition, analyzes and learns usage related data of the robot 30, and recognizes usage patterns, usage environments, etc. .

Meanwhile, data related to space, objects, and usage may be transmitted to the edge server 20 through the communication unit of the robot 30 .

The edge server 20 may train a deep neural network (DNN) based on the received data, and then transmit updated deep neural network (DNN) structure data to the corresponding robot 30 so that it is updated.

The robot 30 according to an embodiment of the present invention may be a mobile robot 30 that travels in each space and performs a set service or mission.

Such a robot 30 may include a control unit for controlling overall operation, a storage unit for storing various data, and a communication unit for transmitting and receiving data to and from other devices such as the edge server 20 .

The control unit may control the overall operation of the robot 30 by controlling the communication unit and various sensors in the robot 30 .

The communication unit includes at least one communication module so that the artificial intelligence robot 30 can be connected to the Internet or a predetermined network and communicate with other devices.

In addition, the communication unit may process data transmission and reception between the robot 30 and the edge server 20 by connecting to a communication module provided in the edge server 20 .

The robot 30 according to an embodiment of the present invention may further include a voice input unit for receiving a user's voice input through a microphone.

The voice input unit includes a processing unit that converts analog sound into digital data or is connected to the processing unit, and converts the user input voice signal into data so that the control unit or the edge server 20 can recognize it.

Meanwhile, the controller may control the robot 30 to perform a predetermined operation based on the voice recognition result.

Meanwhile, the robot 30 may include various modules that display predetermined information as an image or output sound according to its type.

The robot 30 may include a display for displaying information corresponding to a command input, a processing result corresponding to a user's command input, an operation mode, an operation state, an error state, and the like as images.

Depending on the embodiment, at least a part of the display may be configured as a touch screen by forming a mutual layer structure with a touch pad. In this case, a display composed of a touch screen may be used as an input device capable of inputting information by a user's touch in addition to an output device.

In addition, the sound output unit may output a notification message such as a warning sound, an operation mode, an operation state, an error state, information corresponding to a user's command input, a processing result corresponding to a user's command input, and the like according to the control of the control unit. there is. The audio output unit may convert an electrical signal from the control unit into an audio signal and output the converted audio signal. To this end, a speaker or the like may be provided.

According to the embodiment, the robot 30 may further include an image acquisition unit capable of capturing a predetermined range.

The image acquisition unit captures the surroundings of the artificial intelligence robot 30 and the external environment, and may include a camera module. Several such cameras may be installed for each part for shooting efficiency.

The image acquisition unit may capture an image for user recognition. The controller may determine an external situation or recognize a user (guidance target) based on an image acquired by the image acquisition unit.

In addition, when the robot 30 is an artificial intelligence robot 30, the control unit may control the robot 30 to drive based on an image obtained by photographing by the image acquisition unit.

The robot 30 may further include a driving unit for movement, and the driving unit may move the main body under the control of the controller.

The driving unit may be disposed within the driving unit of the robot 30 and may include at least one driving wheel (not shown) for moving the main body. The driving unit may include a driving motor (not shown) connected to the driving wheel to rotate the driving wheel. Driving wheels may be provided on the left and right sides of the main body, respectively, and are hereinafter referred to as a left wheel and a right wheel, respectively.

The left wheel and the right wheel may be driven by a single drive motor, but a left wheel drive motor for driving the left wheel and a right wheel drive motor for driving the right wheel may be provided, respectively, if necessary. The driving direction of the main body can be switched to the left or right by making a difference in the rotational speed of the left wheel and the right wheel.

Meanwhile, the robot 30 may include a sensor unit including sensors for sensing various data related to the operation and state of the robot 30 .

The sensor unit may further include a motion detection sensor that senses motion of the robot 30 and outputs motion information. For example, as a motion detection sensor, a gyro sensor, a wheel sensor, an acceleration sensor, or the like may be used.

The sensor unit may include an obstacle detection sensor for detecting an obstacle, and the obstacle detection sensor may include an infrared sensor, an ultrasonic sensor, an RF sensor, a geomagnetic sensor, a Position Sensitive Device (PSD) sensor, and the presence of a cliff on the floor in the driving area. It may include a cliff detection sensor that detects whether or not, a LiDAR (Light Detection and Ranging: Lidar), and the like.

Meanwhile, the obstacle detection sensor detects an object, particularly an obstacle, present in the driving (moving) direction of the artificial intelligence robot 30 and transmits obstacle information to the control unit. At this time, the controller may control the movement of the robot 30 according to the detected position of the obstacle.

On the other hand, the control unit may control to transmit the operating state of the artificial intelligence robot 30 or a user input to the edge server 20 or the like through the communication unit.

As shown in FIG. 3, such a robot 30 may have a structure in which a specific task such as a delivery robot, an ordering robot, and a guide robot is assigned to one space, and a work unit may be separately included to suit the task.

Each robot 30 may be controlled from each edge server 20 according to assigned tasks.

Hereinafter, control of the robot 30 system including the heterogeneous robot 30 will be described in detail with reference to FIGS. 5 to 7 .

5 is a conceptual diagram illustrating modeling of a cloud and an edge server 20 according to an embodiment of the present invention, FIG. 6 is an overall flow chart of a robot 30 cloud system of the present invention, and FIG. 7 is a single edge server (20) is a detailed flow chart showing how to enter the system of FIG.

Referring to FIG. 6 , in the cloud system of the robot 30 according to an embodiment of the present invention, model creation/distribution/sharing is performed between the cloud server 10 and a plurality of edge servers 20 that transmit/receive.

Specifically, the cloud server 10 is all robots 30 controlled by the corresponding cloud server 10, that is, robots 30 controlled from the corresponding cloud server 10 or connected to store information, or A base model that can be universally applied to the robots 30 associated with the edge server 20 is created (S10).

First, the cloud server 10 performs machine learning/deep learning in the processor 100 to generate a model that can be used universally for controlling the plurality of heterogeneous robots 30 .

When the heterogeneous robot 30 controlled through the edge server 20 transmitting and receiving with the cloud server 10 is a delivery robot, a barista robot, an ordering robot, a cooking robot, a guide robot, etc. as shown in FIG. 5, each robot As a model that can be selectively used in (30), object recognition model 101, navigation model 102, voice recognition model 103, location recognition model 104, emotion recognition model 105, control intelligence model (106), etc., respectively.

Such models 101 to 106 are selectively applicable to a specific robot 30, and in the case of the cloud server 10, all models 101 to 106 required as a union concept are generated, respectively.

Each such deep learning modeling provides a general-purpose packaged base model to the edge server 20, and each edge server 20 optimizes the process by tuning/upgrading according to each environment and robot 30 learned through

Specifically, the cloud server 10 creates a generated packaged base model and distributes the packaged base model to each registered edge server 20 (S10).

Meanwhile, the cloud server 10 registers the plurality of edge servers 20 and performs grouping to classify into a specific zone (S20).

As shown in FIG. 7, zones are set according to the distance or response time of each registered edge server 20, and information about zones allocated to a plurality of edge servers 20 in the same zone and registered in the same zone Information about other edge servers 20 there is transmitted (S100).

At this time, when a service registration request is received from the new edge server 22, the cloud server 10 performs zone assignment for the corresponding new edge server 22 (S101).

That is, the physical location of the edge server 22 is determined through the IP address of the new edge server 22, and the response time is measured through ping or traceroute (S102).

The cloud server 10 reads the distance to the cloud server 10 according to the response time or physical location and allocates the new edge server 20 to a specific zone (S103).

In this way, the cloud server 10 transmits information about zone allocation, for example, the range of the corresponding zone, and other edge servers 21, 23,,,, registered in the corresponding zone to the new edge server 22 Do (S104). In addition, it transmits that the new edge server 22 has been registered to other edge servers 21 and 23 in the corresponding zone (S105).

Returning to FIG. 6 , the cloud server 10 distributes the packaged base model to the registered new edge server 22 (S31).

At this time, the distributed base model is a packaged model or module related to various functions capable of managing or controlling various heterogeneous robots 30, and an application for upgrading or tuning the base model is also included when the base model is transmitted. can be distributed

Upon receiving the base model, the new edge server 22 tunes the base model to be suitable for the plurality of heterogeneous robots 30 managed by the edge server 22 (S40).

In this tuning, in FIG. 5, when the new edge server 22 controls only the delivery robot 30, the ordering robot 30, and the guide robot 30, the control intelligence model among the received base models is not activated. , can be tuned to activate only the object recognition model 101, the navigation model 102, the voice recognition model 103, the location recognition model 104, and the emotion recognition model 105, which are other models.

In other words, the process of activating only necessary models in each area and storing unnecessary models in an inactive state is defined as tuning.

The new edge server 22 selects and activates only models necessary for the heterogeneous robots 30 in the zone A2, thereby reducing operation programs and reducing cost and time.

For example, in the case of the first edge server 21 of FIG. 5 , when a delivery robot, a barista robot, an ordering robot, a cooking robot, and a guide robot are disposed in the zone A1, all models packaged as a base model can be activated.

Like the third edge server 23, when there are only barista robots, ordering robots, and cooking robots, the robots do not move, so the navigation model 102, the location recognition model 104, etc. are not activated. control can proceed.

Next, the edge server 20 may upgrade each base model when predetermined data is secured while performing control of each heterogeneous robot 30 by driving the tuned base model (S50).

The upgrade of the base model will be described in detail later with reference to FIGS. 8 and 9 .

When an upgrade of the base model is performed in one edge server 20, the edge server 20 transmits the corresponding upgrade information to the cloud server 10 (S61).

The cloud server 10 receives the upgrade information, selects an edge server 20 requiring an upgraded base model, and transmits information about the selected edge server 20 to the edge server 20 (S62).

The edge server 20 may share the upgraded base model with other edge servers 20 by transmitting the upgraded base model to the selected edge server 20 (S63).

At this time, the selected edge server 20 that has received the upgraded base model can control the robot 30 within the allocated area by deleting the previous base model, converting and updating the upgraded base model (S70). .

As such, the control system of the heterogeneous robot 30 using the cloud server 10 of the present invention includes the edge server 20 for controlling the remote robots 30, and each edge server 20 has a cloud server While the base model provided by (10) is suitably tuned and used, the base model is upgraded and upgraded by performing machine learning/deep learning.

In this way, the base model upgraded to be specialized for each environment in the edge server 20 can be directly transmitted and received to other edge servers 20 that require it.

In this way, since the upgraded base model is not transmitted and received through the cloud server 10, but the upgraded base model is directly transmitted and received from the edge server 20 to another edge server 20 that is necessary, unnecessary traffic can be reduced. , It is possible to sufficiently secure the storage space of the cloud server 10.

Hereinafter, upgrade of the base model in each edge server 20 and sharing of the upgraded base model will be described with reference to FIGS. 8 and 9 .

8 is a flowchart illustrating that one edge server of FIG. 6 upgrades a base model, and FIG. 9 is a flowchart illustrating a method of sharing an upgrade model of another edge server according to an embodiment of the present invention.

First, referring to FIG. 8 , when one edge server 20 receives a grouped base model from the cloud server 10, the base model is transmitted to a plurality of heterogeneous robots 30 controlled by the corresponding edge server 20. tune to fit

For example, if all models are required, as in the first edge server 21 of FIG. 5, it can be used without additional tuning, and can be tuned by inactivating the control intelligence model as in the second edge server 22. there is.

For the base model thus tuned, when applying control to a plurality of robots 30, a threshold value and minimum data number for deep learning error learning are set (S51).

Specifically, a threshold value (Eth) and a minimum data number (N) may be set.

The threshold value (Eth) may be defined as a reference value for selecting values classified as error values among values received for each base model, and the minimum number of data (N) is the minimum number of data capable of learning the deep learning model. can be defined

In this case, the threshold value (Eth) and the minimum data number (N) may be set differently for each model.

Next, the edge server 20 utilizes all of the respective models to control the plurality of heterogeneous robots 30 (S52).

At this time, the edge server 20 receives an input value transmitted to each robot 30, an output value output from each robot 30, and an error value E.

Such an input value, an output value, and an error value E can be received whenever each service or function progresses, and can be received whenever an event occurs.

The edge server 20 compares each error value (E) with a threshold value (Eth) for one model (S53).

In this case, if the error value E is greater than the threshold value Eth, it is determined to be true, stored as learning data, and the number of data (n=n+1) is counted (S54.)

In this way, while determining whether the error value (E) for the consecutive data is greater than the threshold value (Eth), if the number of counted data does not meet the minimum number of data (N), the next data is read and the error value (E) , and if the number of data with an error value greater than the threshold value (Eth) meets the minimum number of data (N), it is determined that the deep learning model can be trained, and the model is upgraded through deep learning learning. It is performed (S55).

At this time, the edge server 20 can be upgraded while changing the number of nodes, the number of layers, and activation function values along with the added error values. At this time, the activation function can store and utilize a plurality of candidate groups in advance. do (S56).

For example, when an error value greater than or equal to a minimum data value occurs while applying the corresponding voice recognition model, a program may be performed to minimize the error by applying another activation function accordingly.

The upgraded model may be stored in the memory 101 of the edge server 20 as version 2 (V2), and may be used by replacing the previous version, version 1 (V1).

When a change is made to the upgraded model, the edge server 20 transmits the change history to the cloud server 10 and controls the heterogeneous robot 30 again (S57).

In this way, each edge server 20 upgrades the base model according to its own feedback, so that the most optimized version for the environment can be modeled, and the base model itself is upgraded by receiving and upgrading the base model to the cloud server 10. It does not require very many operations to create.

In addition, since each activation function and change option are already stored, it is possible to upgrade by changing them, enabling a relatively simple upgrade.

In this way, the upgraded base model in a specific edge server 20 can be shared with other edge servers 20 that control similar robots 30 .

Specifically, referring to FIG. 9 , when a model upgrade occurs in the edge server A 21 (S200), the edge server A 21 transmits such upgrade change details to the cloud server 10 (S201).

At this time, the edge server A 21 may notify the other edge server 22 that an event requiring communication has occurred.

That is, since a lot of errors occur in the model and similar errors are expected to occur in other edge servers 22, the need for model upgrade to prevent such errors is transmitted to the cloud server 10.

In addition, the edge server A 21 transmits the changed points and the model name of the robot 30 related to the upgraded model, not the entire upgraded base model (S202).

That is, by transmitting only the information on the upgraded change point and the related robot 30 model, the cloud server 10 selects the edge server 20 for which the application of the upgraded base model is required based on the information ( S203).

That is, the cloud server 10 may select other edge servers 22 that control the same robot 30, and may select an edge server 22 that mainly utilizes the upgraded model.

At this time, the cloud server 10 may select only the edge servers 22 in the zone to which the upgraded edge server A 21 belongs by referring to the relevance of each edge server 20 for each zone. It is not limited.

In this way, the cloud server 10 selects the required edge server 20 and transmits information about the corresponding edge server 22 to the edge server A 21 (S204).

The edge server A 21 receives information about the selected edge servers 22, that is, IP addresses, etc., and prepares to transmit the upgraded base model to the corresponding edge servers 22.

Meanwhile, the cloud server 10 schedules transmission of the upgrade base model to the selected edge servers 22 (S205).

That is, each selected edge server 22 is notified that an upgrade of the corresponding base model has occurred, and transmission of the upgraded base model from the edge server A 21 is notified.

Each of the selected edge servers 22 prepares to receive the upgraded base model by notification from the cloud server 10 .

At this time, the edge server A 21 may notify the selected edge servers 22 of the deep learning model change and the model name of the related robot 30, and request preparation of an update to the upgraded base model (S206). .

The selected edge servers 22 move the currently stored deep learning base model to temp and temporarily store it (S207).

Next, each of the selected edge servers 22 returns a completion of preparation to the edge server A 21 and transmits an update request (S208).

The edge server A 21 distributes the upgraded deep learning model to each selected edge server 22 (S209).

Upon receiving the upgraded deep learning model, the selected edge server 22 stores and test-runs the model.

If there is no error in the implementation of the upgraded deep learning model, temp is discarded to delete the previous deep learning base model (S210).

In this way, the deep learning base model is updated, and the version of the updated model is notified to the cloud server 10 as version 2 (V2) (S211).

At this time, the completion of the model update is notified to the edge server A 21 (S212).

As such, the selected edge server 22 tunes and executes the base model received from the cloud server 10, and when upgrade information and selection information are received from other edge servers 20, the upgraded model is received and tuned. The updated base model can be updated to the next version and the robots 30 in the corresponding area can be controlled with the upgraded version.

In addition, even when controlling to the upgraded version, each edge server 20 can upgrade the model by deep learning to the next version by counting a predetermined number of data and determining an error, as shown in FIG. It can be shared with the server 20.

In an embodiment of the present invention, by sharing intelligence between two or more edge servers 20, it is possible to continuously upgrade by sharing an evolved model while evolving in each environment.

Accordingly, since only feature points and history are stored in the cloud server 10 without storing a model to be upgraded, the storage space of the cloud server 10 can be secured.

In addition, adaptive modeling is possible by performing an upgrade at the edge server 20 that is actually necessary.

The robot system according to the present invention is not limited to the configuration and method of the embodiments described above, but all or part of each embodiment is selectively combined so that various modifications can be made. may be configured.

Meanwhile, the control method of the robot system according to an embodiment of the present invention can be implemented as a processor-readable code on a processor-readable recording medium. The processor-readable recording medium includes all types of recording devices in which data readable by the processor is stored. Examples of the processor-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., and also include those implemented in the form of carrier waves such as transmission through the Internet. . In addition, the processor-readable recording medium is distributed in computer systems connected through a network, so that the processor-readable code can be stored and executed in a distributed manner.

In addition, although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. Of course, various modifications are possible by those skilled in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

10: cloud server 20: edge server
30: robot

Claims (20)

A plurality of robots disposed in a plurality of spaces arbitrarily divided;
a cloud server generating and distributing a control base model applicable to the plurality of robots; and
A plurality of edge servers assigned to each space, transmitting and receiving with the cloud server, receiving the control base model and upgrading according to the plurality of robots in the space to control the plurality of robots with the upgraded control model
Including,
The edge server is a cloud-based robot system, characterized in that the upgraded control model is directly transmitted to another edge server.
According to claim 1,
The plurality of robots are cloud-based robot systems, characterized in that they include different types of robots.
According to claim 2,
The control base model is a cloud-based robot system, characterized in that each control model for a plurality of functions of the robot of a different type is packaged.
According to claim 1,
The cloud-based robot system, characterized in that the edge server receives the control base model, tunes and executes the control base model according to the type of robot controlled by the edge server.
According to claim 4,
The cloud-based robot system, characterized in that the edge server generates an upgraded control model by performing deep learning error learning on the tuned control base model.
According to claim 5,
The cloud server obtains information on the upgraded control model, selects another edge server to which the upgraded control model is applied, and transmits the upgraded control model directly between the edge servers Cloud, characterized in that based robotic system.
According to claim 6,
The cloud-based robot system, characterized in that the cloud server selects the edge server containing the robot to which the upgraded control model is applied and transmits the upgraded control model.
According to claim 7,
Wherein the edge server generates the upgraded control model by performing the deep learning error learning when an error value exceeding a threshold value is more than a predetermined number while executing the control base model Cloud-based robot system.
According to claim 1,
The cloud-based robot system, characterized in that the cloud server classifies a plurality of the edge servers and manages them into a plurality of groups.
According to claim 9,
One group is a cloud-based robot system, characterized in that located within a predetermined distance or predetermined response time.
In the cloud-based robot control method for controlling a plurality of robots disposed in a plurality of spaces that are arbitrarily divided,
generating a control base model applicable to the plurality of robots in a cloud server;
distributing the control base model to an edge server allocated to each space;
upgrading the control base model according to the plurality of robots in the space at the edge server;
directly transmitting the upgraded control model from the edge server to another edge server; and
Controlling the plurality of robots with the upgraded control model in the edge server.
Cloud-based robot control method comprising a.
According to claim 11,
The plurality of robots cloud-based robot control method, characterized in that including different types of robots.
According to claim 12,
The control base model generating step is a cloud-based robot control method, characterized in that for packaging by generating each control model for a plurality of functions of the different types of the robot.
According to claim 11,
The cloud-based robot control method further comprising receiving the control base model from the edge server and tuning the control base model according to the type of robot controlled by the edge server.
According to claim 14,
The upgrade step is
A cloud-based robot control method, characterized in that for generating an upgraded control model by performing deep learning error learning on the tuned control base model.
According to claim 15,
Obtaining information about the upgraded control model from the cloud server;
Selecting another edge server to which the upgraded control model is applied;
Transmitting information of other selected edge servers to the upgraded edge server, and
Transmitting the upgraded control model directly from the upgraded edge server to the selected other edge server.
Cloud-based robot control method comprising a.
According to claim 16,
The step of selecting the other edge server,
The cloud-based robot control method, characterized in that the cloud server selects an edge server containing a robot to which the upgraded control model is applied.
According to claim 17,
The upgrade step is
When the error value exceeding the threshold value is more than a predetermined number while executing the control base model in the edge server, the deep learning error learning is performed to generate the upgraded control model. Cloud-based robot control method.
According to claim 11,
Cloud-based robot control method further comprising the step of classifying and grouping a plurality of the edge servers in the cloud server.
According to claim 19,
A cloud-based robot control method, characterized in that one group is located within a predetermined distance or predetermined response time.

Images