WO2023115927A1

WO2023115927A1 - Cloud robot mapping method, system, device and storage medium

Info

Publication number: WO2023115927A1
Application number: PCT/CN2022/106943
Authority: WO
Inventors: 陈莹; 马世奎; 董文锋
Original assignee: 达闼机器人股份有限公司
Priority date: 2021-12-24
Filing date: 2022-07-21
Publication date: 2023-06-29
Also published as: CN114373148A

Abstract

Embodiments of the present application provide a cloud robot mapping method, a system, a device, and a storage medium. In the mapping system for the cloud robot, the robot may interact with a terminal device by means of a cloud server, so that the terminal device may receive real-time video data sent by the robot deployed in a space to be mapped, output the real-time video data, and in response to a robot control operation initiated according to the real-time video data, send a corresponding control instruction to the robot by means of the cloud server. Furthermore, the robot may collect mapping data according to the control instruction and send the collected mapping data to the cloud server, so that the cloud server creates a map of a target space according to the mapping data. According to such an implementation, the robot may be sensed and controlled remotely by means of the terminal device, and there is no need for relevant personnel to go to the space to be mapped in order to scan and control the robot, thereby saving labor costs.

Description

Mapping method, system, device and storage medium for cloud robot

cross reference

This application refers to the Chinese patent application No. 202111602631.7, which was submitted on December 24, 2021 and is entitled "Mapping method, system, device and storage medium for cloud robots", which is fully incorporated by reference into this application.

technical field

The embodiments of the present application relate to the field of robot technology, and in particular, to a method, system, device and storage medium for building a map of a cloud robot.

Background technique

In the field of robotics, in order to realize the automatic walking function of the robot in the actual physical space, it is necessary to create a map of the physical space in advance based on the map construction method. For example, SLAM maps of physical spaces can be created based on SLAM (simultaneous localization and mapping, instant positioning and map construction) methods for use by robots.

In the prior art, after the robot is deployed in the service place (ie, the physical space), the robot is usually manually controlled to scan and construct the map. The labor cost of this method is relatively high. Therefore, it is urgent to propose a solution.

Contents of the invention

Embodiments of the present application provide a cloud-based robot mapping method, system, device, and storage medium for perceptually controlling the robot at a remote location to reduce labor costs.

The embodiment of the present application provides a cloud-based robot mapping method, which is suitable for terminal devices, including: determining the robot deployed in the target space to be mapped; receiving the real-time video of the target space sent by the robot through the cloud server in real time data, and output the real-time video data; in response to the robot control operation initiated according to the real-time video data, send corresponding control instructions to the robot through the cloud server, so as to collect the robot in the target space The action of the mapping data is controlled remotely.

Further optionally, in response to the robot control operation initiated according to the real-time video data, sending a corresponding control instruction to the robot through the cloud server includes: responding to any target direction initiated according to the real-time video data For motion control operations, a motion control command is sent to the robot through the cloud server; the motion control command includes: a forward command, a backward command, a turn command or a stop command in the target direction.

Further optionally, the mapping method of the cloud robot further includes: obtaining a map of the target space from the cloud server; the map is generated by the cloud server according to the mapping data collected by the robot; responding For the editing operation of the map, the map is updated, and the updated map is sent to the cloud server.

Further optionally, updating the map in response to an editing operation on the map includes at least one of the following: updating obstacle information at a corresponding position on the map in response to an obstacle update operation on the map The obstacle update operation includes: obstacle deletion operation, obstacle addition operation or obstacle movement operation; in response to any point of interest labeling operation on the map, corresponding to the point of interest on the map mark the mark of the point of interest; respond to the operation of setting a virtual wall at any position on the map, draw the virtual wall at the corresponding position on the map; respond to any area on the map The marker operation for adds an area marker for the area on the map.

Further optionally, the mapping method of the cloud robot further includes: obtaining the mapping data of the target space from the cloud server; the mapping data is collected by the robot from the target space and uploaded to the The cloud server; displaying the mapping data on the same screen as the real-time video data of the target space, so as to control the robot to move in the target space according to the comparison result of the real-time video data and the mapping data.

Further optionally, the mapping method of the cloud robot further includes: receiving device monitoring data sent by the robot through the cloud server; the device monitoring data includes: battery data, network status data, and abnormalities of the robot At least one of the event data; when the device monitoring data indicates that the robot needs operation and maintenance processing, an operation and maintenance prompt message is output.

The embodiment of the present application also provides a cloud-based robot mapping method, which is suitable for robots, including: sending the collected real-time video data of the target space to the terminal device in real time through the cloud server; The control instruction sent by the server; the control instruction is sent according to the real-time video data; according to the control instruction, the mapping data is collected in the target space; the collected mapping data is sent to the cloud server for making the cloud server create a map of the target space according to the mapping data.

Further optionally, the mapping data includes: at least one of pose data, ranging data, mileage data, laser point cloud data, collision data, image data and drop detection data of the robot.

Further optionally, the mapping method of the cloud robot further includes: acquiring its own device monitoring data; the device monitoring data includes: at least one of battery data, network status data, and abnormal event data of the robot; The device monitoring data is sent to the terminal device through the cloud server, so as to perform operation and maintenance processing on the robot according to the device monitoring data.

Further optionally, the mapping method of the cloud robot further includes: during the process of collecting mapping data in the target space, sending the trajectory of the robot in the target space to the cloud server, so that the cloud server generates the motion trajectory required by the robot to perform tasks in the target space according to the motion trajectory.

The embodiment of the present application also provides a cloud-based robot mapping system, including: a robot, a cloud server, and a terminal device; wherein the terminal device establishes a communication connection with the robot through the cloud server; wherein the terminal device mainly It is used for: receiving the real-time video data of the target space sent by the robot in real time through the cloud server, and outputting the real-time video data; responding to the robot control operation initiated according to the real-time video data, sending the robot a Send corresponding control instructions; the robot is mainly used to send the collected real-time video data of the target space to the terminal device in real time through the cloud server; receive the control sent by the terminal device through the cloud service instruction; according to the control instruction, collect mapping data in the target space; send the collected mapping data to the cloud server, so that the cloud server creates the target space according to the mapping data map.

The embodiment of the present application also provides a terminal device, including: a memory, a processor, a communication component, and a display component; wherein, the memory is used to: store one or more computer instructions; and the processor is used to execute the one or more computer instructions. A plurality of computer instructions for: executing the steps in the methods described in the embodiments of the present application.

The embodiment of the present application also provides a robot device, including: a memory, a processor, and a communication component; wherein, the memory is used to: store one or more computer instructions; the processor is used to execute the one or more computer instructions The instruction is used for: executing the steps in the method described in the embodiment of the present application.

The embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is caused to implement the steps in the method described in the embodiment of the present application.

Embodiments of the present application provide a cloud-based robot mapping method, system, device, and storage medium. The robot can interact with the terminal device through the cloud server, so that the terminal device can receive real-time information sent by the robot deployed in the space to be mapped. Video data, output real-time video data, and respond to the robot control operation initiated according to the real-time video data, and send corresponding control instructions to the robot through the cloud server. Furthermore, the robot can collect mapping data according to the control instruction, and send the collected mapping data to the cloud server, so that the cloud server can create a map of the target space according to the mapping data. Through this implementation, the robot can be remotely sensed and controlled through the terminal device, and there is no need for relevant personnel to scan and control the robot in the space to be mapped, thereby saving labor costs.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic structural diagram of a mapping system for a cloud robot provided by an exemplary embodiment of the present application;

FIG. 2 is a schematic flow diagram of a cloud robot mapping method on the terminal device side provided by an exemplary embodiment of the present application;

FIG. 3 is a schematic flowchart of a cloud robot mapping method on the robot device side provided by another exemplary embodiment of the present application;

FIG. 4 is a schematic diagram of a terminal device provided by an exemplary embodiment of the present application;

Fig. 5 is a schematic diagram of a robot device provided by an exemplary embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

In the prior art, after the robot is deployed to the service site (i.e., the physical space), the robot is usually manually controlled to scan and construct the map, which requires high labor costs. Aiming at the above technical problems, some embodiments of the present application provide a solution. The technical solutions provided by each embodiment of the present application will be described in detail below with reference to the accompanying drawings.

FIG. 1 is a schematic structural diagram of a cloud robot mapping system provided by an exemplary embodiment of the present application. As shown in FIG. 1 , the cloud robot mapping system 100 includes: a terminal device 10 , a cloud server 20 and a robot 30 .

Wherein, the terminal device 10 can be implemented as a device held by the robot manager and capable of remote communication, such as a smart phone, a smart watch, a tablet computer, a notebook computer, and a smart wearable device. A software program for managing the robot may run on the terminal device 10, or a browser for accessing a robot management page may run.

Wherein, the cloud server 20 may be implemented as a cloud host, a virtual center in the cloud, an elastic computing instance in the cloud, etc., which is not limited in this embodiment. Wherein, the composition of the cloud server 20 mainly includes a processor, a hard disk, a memory, a system bus, etc., and is similar to a general computer architecture, and will not be repeated here.

In the cloud robot mapping system 100 , wireless communication connections can be established between the terminal device 10 and the cloud server 20 , and between the robot 30 and the cloud server 20 , and the specific communication connection methods may depend on different application scenarios. In some embodiments, the wireless communication connection can be implemented based on a dedicated virtual network (Virtual Private Network, VPN) to ensure communication security.

Wherein, the robot 30 is deployed in the target space to be mapped, which can be any space in the real world, such as libraries, restaurants, hotels, residences, etc., and the robot can provide services in the target space. Before providing the service, the robot 30 may perform a scanning operation in advance to construct a map of the target space.

After the deployment of the robot 30 is completed, in order to facilitate the remote control of the scanning operation of the robot 30 , the terminal device 10 may establish a corresponding relationship with the robot 30 . The robot manager can at first obtain the identity identification mark of robot 30, such as account name, identity identification number ID (Identity document, identity identification number), IP (Internet Protocol, Internet Interconnection Protocol) address, MAC (Multiple Access Channel, multiple address access channel) address, etc.; next, input the identification mark into the terminal device 10, and send a request to establish a binding relationship with the robot 30 to the cloud server 20 through the terminal device 10. After receiving the request, the cloud server 20 can establish a binding relationship between the terminal device 10 and the robot 30 , and forward communication data between the terminal device 10 and the robot 30 in the subsequent communication process.

In the mapping system 100 of the cloud robot, the robot 30 can collect real-time video data of the target space, and can send the collected real-time video data of the target space to the terminal device 10 in real time through the cloud server 20 . Wherein, the real-time video data can be collected by at least one image collection device on the robot 30 . The real-time video data may include continuous frame images captured by at least one image capture device.

Correspondingly, the terminal device 10 can receive the real-time video data of the target space sent by the robot 30 through the cloud server 20 in real time, and output the real-time video data through the display component and/or the audio component. Through this implementation, the robot manager can use the terminal device 10 to observe the real-time video data collected by the robot 30 in real time, and can remotely sense the environment where the robot is located, so as to control the movement route of the robot in the target space more accurately.

Based on the above steps, the robot manager can initiate a control operation on the robot 30 according to the real-time video data output by the terminal device 10 . In response to the control operation, the terminal device 10 can send a corresponding control command to the robot 30 through the cloud server 20, so as to remotely control the robot's action of collecting mapping data in the target space.

For example, the robot needs to move to position A to collect data. The robot manager knows that the current robot is at position B through the real-time video data output by the terminal device 10. The robot manager can send a series of control commands to make the robot move from position B to position B. Location A collects data. The series of control commands can be forwarded to the robot 30 via the cloud server 20 . Through this implementation, the robot can be remotely sensed and controlled based on real-time video data.

Correspondingly, the robot 30 can receive the control instruction sent by the terminal device 10 through the cloud server 20 . After receiving the control instruction, the robot 30 can collect mapping data in the target space according to the control instruction, and send the collected mapping data to the cloud server 20 .

Among them, the mapping data refers to the data used to construct the map corresponding to the target space. The mapping data includes but is not limited to: robot pose data, ranging data, mileage data, laser point cloud data, collision data, image data and at least one of drop detection data.

Wherein, the laser point cloud data can be collected by the lidar on the robot 30; the pose data of the robot can be collected by the attitude sensor on the robot 30; the ranging data can be collected by the ultrasonic sensor or infrared sensor on the robot 30; The mileage data can be collected by the odometer on the robot 30 ; the collision data can be collected by the anti-collision sensor on the robot 30 ; the drop detection data can be collected by the anti-drop sensor on the robot 30 .

Correspondingly, after receiving the mapping data, the cloud server 20 can create a map of the target space according to the mapping data. Wherein, the map can be implemented as a grid map or a topological map, etc., which is not limited in this embodiment.

In this embodiment, the robot can interact with the terminal device through the cloud server, so that the terminal device can receive the real-time video data sent by the robot deployed in the space to be mapped, output the real-time video data, and respond to the real-time video data initiated by the robot Control operation, send corresponding control instructions to the robot through the cloud server. Furthermore, the robot can collect mapping data according to the control instruction, and send the collected mapping data to the cloud server, so that the cloud server can create a map of the target space according to the mapping data. Through this implementation, the robot can be remotely sensed and controlled through the terminal device, and there is no need for relevant personnel to scan and control the robot in the space to be mapped, thereby saving labor costs.

In some optional embodiments, in response to the robot control operation initiated according to the real-time video data, when the cloud server 20 sends a corresponding control command to the robot, the terminal device 10 can respond to any target direction initiated according to the real-time video data. In the motion control operation, the cloud server 20 sends a motion control command to the robot 30, so that the robot 30 moves according to the motion control command and collects mapping data. Wherein, taking the world coordinate system where the robot is located (the horizontal axis is x, and the vertical axis is y) as an example, the target direction may include at least one of the following: positive x-axis direction, positive y-axis direction, negative x-axis direction, and negative y-axis direction , the 45° direction between the positive direction of the x-axis and the positive direction of the y-axis, the 45° direction between the positive direction of the x-axis and the negative direction of the y-axis, the 45° between the negative direction of the x-axis and the negative direction of the y-axis, and the x-axis The 45° direction between the negative direction and the positive direction of the y-axis. Wherein, the motion control instruction includes: a forward instruction, a backward instruction, a turn instruction or a stop instruction in the target direction.

In some optional embodiments, when the robot 30 collects the mapping data in the target space, the robot 30 may send the trajectory of the robot 30 in the target space to the cloud server 20 .

Correspondingly, after receiving the movement trajectory, the cloud server 20 can generate the movement trajectory required for the robot to perform tasks in the target space according to the movement trajectory. For example, when the mapping system of the above-mentioned cloud robot is applied in a hotel (the target space is a hotel), the robot can follow the motion trajectory J1 from room F1 to room F2, the motion trajectory J2 from room F2 to room F3, and the motion from F1 to room F3 The trajectory J3 generates the movement trajectory J4 required for the robot 30 to perform tasks in the hotel. Based on this, in the subsequent task execution scenario, when the cleaning robot performs the cleaning task, it can perform the cleaning tasks of room F1, room F2, and room F3 along the trajectory J4; The motion track J4 executes the food delivery tasks in the room F1, the room F2 and the room F3.

Through this embodiment, by generating the movement trajectory required for the robot to perform the task in the target space, the robot can perform the task according to the movement trajectory, which improves the task execution efficiency of the robot.

In some optional embodiments, after the cloud server 20 creates the map of the target space according to the mapping data collected by the robot 30 , the terminal device 10 may obtain the map of the target space from the cloud server 20 . The map can then be edited by those who edit the map. Correspondingly, the terminal device 10 may update the map in response to the editing operation on the map, and send the updated map to the cloud server 20 .

Optionally, after acquiring the map, the terminal device 10 can display the map on the same screen as the real-time video data of the target space. For example, display live video data on the left and a map on the right. Relevant personnel who edit the map can edit the map more intuitively according to the comparison results of the map and real-time video data.

In some optional embodiments, the "update the map in response to the editing operation on the map" described in the foregoing embodiments includes at least one of the following:

1) In response to the obstacle update operation on the map, update the obstacle information at the corresponding position on the map. The obstacle update operation includes: obstacle deletion operation, obstacle addition operation or obstacle movement operation. Taking the operation of adding obstacles as an example, because there may be sensor blind spots when the robot collects mapping data, some obstacles in the target space are not generated on the map, so the relevant personnel who edit the map can use real-time video data Observe obstacles that are not generated on the map, and perform the operation of adding obstacles. The terminal device 10 may generate the obstacle at a corresponding position on the map in response to the operation.

2) In response to an operation of marking any point of interest on the map, mark the mark of the point of interest on the map corresponding to the point of interest. For example, in response to the labeling operation of the entertainment interest point, the mark of "entertainment" can be marked on the location corresponding to the TV and the location corresponding to the game console on the map.

3) In response to the operation of setting a virtual wall at any position on the map, draw the virtual wall at the corresponding position on the map. For example, when the target space is a factory, there may be poor security areas or forbidden areas in the generated map. At this time, editors can set virtual walls in the poor security areas on the map according to the actual situation. The terminal device 10 may respond to the operation of setting a virtual wall, and draw a virtual wall in an area with poor security on the map.

4) In response to the marking operation on any area on the map, add an area mark for the area on the map. For example, in the scenario where the target space is a large hotel, the terminal device 10 can respond to the marking operation on area A on the map, and add an area mark of "dining area" to area A on the map; Marking operation, add an area mark of "rest area" to area B in the map.

Through the above implementation, through the same screen display, the relevant personnel who edit the map can more intuitively compare the map and the actual situation of the target space, and can remotely edit the map in a variety of ways to optimize the map, thereby improving the quality of the map. Accuracy and efficiency of map editing.

In some optional embodiments, in order to improve the mapping quality of the target space, the robot manager can control the robot to move in the target space according to the comparison result of the real-time video data and the mapping data. Further description will be given below.

The terminal device 10 can obtain the mapping data of the target space from the cloud server 20, and can display the mapping data on the same screen as the real-time video data of the target space. For example, display real-time video data on the left, and display mapping data on the right. Furthermore, the robot manager can control the robot to move in the target space according to the comparison result of the real-time video data and the mapping data.

For example, the real-time video data shows that the robot is going straight towards the path L. Since there is an obstacle A on the left side of the path L in the blind area of the image acquisition device, the robot cannot capture the obstacle A, that is, it cannot perceive the obstacle A through the real-time video data. The obstacle A. Therefore, this obstacle A will be missed in the generated map of the target space. However, the odometry data in the mapping data can show that there is an obstacle A on the left side of the robot. By comparing the two data, the operator can control the robot to turn left, so that the image acquisition device can capture the obstacle A. Through this implementation, by comparing the real-time video data with the mapping data, the movement of the robot can be controlled more accurately, reducing the probability of the robot wrongly collecting the mapping data.

In some scenarios, during the process of collecting mapping data, the robot 30 may have insufficient power, downtime, network abnormalities, etc., resulting in interruption of the mapping process. In order to ensure the smooth execution of the mapping process, in some embodiments, remote operation and maintenance of the robot 30 can be performed. An exemplary description will be given below.

Optionally, the robot 30 may acquire its own device monitoring data, wherein the device monitoring data includes: at least one of battery data, network status data, and abnormal event data of the robot 30 . Among them, the abnormal event data may include abnormal situations such as robot falling, robot falling, robot collision, movement device damage, and system downtime. Wherein, the network status data may include: network delay, network packet loss rate, network jitter and so on. Wherein, the battery data may include: battery life, battery capacity, current battery power and so on.

After the device monitoring data is obtained, the device monitoring data can be sent to the terminal device 10 through the cloud server 20 . Correspondingly, after receiving the device monitoring data, the terminal device 10 may output an operation and maintenance prompt message when the device monitoring data indicates that the robot 30 needs operation and maintenance processing.

Optionally, the terminal device 10 may compare the device monitoring data with a preset monitoring threshold, or judge whether the robot 30 needs operation and maintenance processing according to other preset rules or a baseline prediction model based on machine learning.

Optionally, when the device monitoring data indicates that the robot 30 needs operation and maintenance processing, the terminal device 10 may output an operation and maintenance prompt message through voice prompts, vibration prompts, or text prompts.

Taking the battery data as an example, the power threshold is 10%. If the current battery power of the robot 30 is 9%, which is less than the power threshold, it can be determined that the robot 30 needs operation and maintenance processing at this time. The terminal device 10 may output an operation and maintenance processing message of "the battery is low, please charge it in time".

Taking the network status data as an example, the network delay threshold is 30ms. If the network delay of the robot 30 is 31ms, which is greater than the power threshold, it can be determined that the robot 30 needs operation and maintenance processing at this time. The terminal device 10 may output an operation and maintenance processing message of "the network delay is high, please deal with it in time".

Taking abnormal event data as an example, the default rule is that the operation and maintenance request is made when the system is down. If the system of the robot 30 is down at this time, it can be determined that the robot 30 needs operation and maintenance processing at this time. The terminal device 10 may output an operation and maintenance processing message of "the system is down, please deal with it in time".

Through this implementation mode, there is no need for operation and maintenance personnel to inspect the robot 30, and through the equipment monitoring data of the robot 30 itself, the status of the robot 30 can be deeply perceived, and the operation and maintenance of the robot 30 can be performed in a timely manner, which simplifies the operation and maintenance process, improving the efficiency of operation and maintenance.

In addition to the cloud robot mapping system provided by the above embodiments, the embodiment of the present application also provides a cloud robot mapping method, which will be described below with reference to the accompanying drawings.

Fig. 2 is a schematic flowchart of a cloud robot mapping method provided by an exemplary embodiment of the present application. When the method is executed on the terminal device side, it may include the steps shown in Fig. 2:

Step 201. Determine the robots deployed in the target space to be mapped.

Step 202, receiving the real-time video data of the target space sent by the robot through the cloud server in real time, and outputting the real-time video data.

Step 203: In response to the robot control operation initiated according to the real-time video data, send corresponding control instructions to the robot through the cloud server, so as to remotely control the robot's action of collecting mapping data in the target space.

Further optionally, in response to the robot control operation initiated according to the real-time video data, sending a corresponding control instruction to the robot through the cloud server includes: responding to a motion control operation in any target direction initiated according to the real-time video data, A motion control command is sent to the robot through the cloud server; the motion control command includes: a forward command, a backward command, a turn command or a stop command in the target direction.

Further optionally, the mapping method of the cloud robot further includes: obtaining the map of the target space from the cloud server; the map is generated by the cloud server according to the mapping data collected by the robot; and responding to editing operations on the map , update the map, and send the updated map to the cloud server.

Further optionally, updating the map in response to an editing operation on the map includes at least one of the following: updating obstacle information at a corresponding position on the map in response to an obstacle update operation on the map; the obstacle The update operation includes: an obstacle deletion operation, an obstacle addition operation or an obstacle movement operation; in response to any point of interest labeling operation on the map, mark the mark of the point of interest on the map at the position corresponding to the point of interest ;In response to the operation of setting a virtual wall at any position on the map, draw the virtual wall at the corresponding position on the map; in response to the marking operation of any area on the map, add an area for the area on the map mark.

Further optionally, the mapping method of the cloud robot also includes: obtaining the mapping data of the target space from the cloud server; the mapping data is collected by the robot from the target space and uploaded to the cloud server; The real-time video data of the space displays the mapping data on the same screen, so as to control the robot to move in the target space according to the comparison result between the real-time video data and the mapping data.

Further optionally, the mapping method of the cloud robot further includes: receiving device monitoring data sent by the robot through the cloud server; the device monitoring data includes: at least one of the robot's battery data, network status data, and abnormal event data One: output an operation and maintenance prompt message when the device monitoring data indicates that the robot needs operation and maintenance processing.

In this embodiment, the robot can interact with the terminal device through the cloud server, so that the terminal device can receive the real-time video data sent by the robot deployed in the space to be mapped, output the real-time video data, and respond to the real-time video data initiated by the robot Control operation, send corresponding control instructions to the robot through the cloud server. Furthermore, the robot can collect mapping data according to the control instructions, and send the collected mapping data to the cloud server, so that the cloud server can create a map of the target space based on the mapping data. Through this implementation, the robot can be remotely sensed and controlled through the terminal device, and there is no need for relevant personnel to scan and control the robot in the space to be mapped, thereby saving labor costs.

The embodiment of the present application also provides a cloud robot mapping method, which will be described below with reference to the accompanying drawings.

Fig. 3 is a schematic flow chart of a cloud robot mapping method provided by an exemplary embodiment of the present application. When the method is executed on the robot side, it may include the steps shown in Fig. 3:

Step 301. Send the collected real-time video data of the target space to the terminal device in real time through the cloud server.

Step 302, receiving a control instruction sent by the terminal device through the cloud server; the control instruction is sent according to the real-time video data.

Step 303, collect mapping data in the target space according to the control instruction.

Step 304, sending the collected mapping data to the cloud server, so that the cloud server creates a map of the target space according to the mapping data.

Further optionally, the mapping method of the cloud robot also includes: obtaining its own device monitoring data; the device monitoring data includes: at least one of the robot's battery data, network status data, and abnormal event data; The server sends the device monitoring data to the terminal device for operation and maintenance of the robot based on the device monitoring data.

Further optionally, the mapping method of the cloud robot further includes: during the process of collecting mapping data in the target space, sending the trajectory of the robot in the target space to the cloud server, so that the cloud server According to the motion trajectory, the motion trajectory required by the robot to perform tasks in the target space is generated.

It should be noted that the subject of execution of each step of the method provided in the foregoing embodiments may be the same device, or the method may also be executed by different devices. For example, the execution subject of steps 201 to 203 may be device A; for another example, the execution subject of

steps

201 and 202 may be device A, and the execution subject of step 203 may be device B; and so on.

In addition, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appearing in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear herein or executed in parallel , the serial numbers of the operations, such as 201, 202, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. Additionally, these processes can include more or fewer operations, and these operations can be performed sequentially or in parallel.

It should be noted that the descriptions of "first" and "second" in this article are used to distinguish different messages, devices, modules, etc. are different types.

FIG. 4 is a schematic structural diagram of a terminal device provided by an exemplary embodiment of the present application. As shown in FIG. 4 , the terminal device includes: a memory 401 , a processor 402 and a communication component 403 .

The memory 401 is used to store computer programs, and can be configured to store other various data to support operations on the terminal device. Examples of such data include instructions for any application or method operating on the terminal device, contact data, phonebook data, messages, pictures, videos, etc.

The processor 402, coupled with the memory 401, is used to execute the computer program in the memory 401, so as to: determine the robot deployed in the target space to be mapped; receive the real-time video of the target space sent by the robot in real time through the cloud server data, and output the real-time video data; in response to the robot control operation initiated according to the real-time video data, the corresponding control instructions are sent to the robot through the cloud server, so that the robot collects the mapping data in the target space. remote control.

Further optionally, when the processor 402 sends a corresponding control instruction to the robot through the cloud server in response to the robot control operation initiated according to the real-time video data, it is specifically configured to: respond to any target operation initiated according to the real-time video data. For motion control operations in the direction, a motion control command is sent to the robot through the cloud server; the motion control command includes: a forward command, a backward command, a turn command or a stop command in the target direction.

Further optionally, the processor 402 is also configured to: obtain a map of the target space from the cloud server; the map is generated by the cloud server according to the mapping data collected by the robot; in response to an editing operation on the map, the The map is updated, and the updated map is sent to the cloud server.

Further optionally, when updating the map in response to an editing operation on the map, the processor 402 is specifically configured to: update obstacle information at a corresponding position on the map in response to an obstacle update operation on the map; The obstacle update operation includes: an obstacle deletion operation, an obstacle addition operation or an obstacle movement operation; in response to any point of interest marking operation on the map, mark the point of interest on the map at a position corresponding to the point of interest In response to the operation of setting a virtual wall at any position on the map, the virtual wall is drawn at the corresponding position on the map; in response to the marking operation of any area on the map, the virtual wall is drawn on the map for the Regions add region markers.

Further optionally, the processor 402 is also configured to: obtain the mapping data of the target space from the cloud server; the mapping data is collected by the robot from the target space and uploaded to the cloud server; The video data displays the mapping data on the same screen, so as to control the robot to move in the target space according to the comparison result between the real-time video data and the mapping data.

Further optionally, the processor 402 is also configured to: receive device monitoring data sent by the robot through the cloud server; the device monitoring data includes: at least one of battery data, network status data, and abnormal event data of the robot; When the device monitoring data indicates that the robot needs operation and maintenance processing, an operation and maintenance prompt message is output.

Further, as shown in FIG. 4 , the terminal device further includes: a display component 404 , a power supply component 405 , an audio component 406 and other components. FIG. 4 only schematically shows some components, which does not mean that the terminal device only includes the components shown in FIG. 4 .

In this embodiment, the terminal device can interact with the robot through the cloud server, so that the terminal device can receive the real-time video data sent by the robot deployed in the space to be mapped, output the real-time video data, and respond to the robot control initiated according to the real-time video data Operation, send corresponding control instructions to the robot through the cloud server. Furthermore, the robot can collect mapping data according to the control instruction, and send the collected mapping data to the cloud server, so that the cloud server can create a map of the target space according to the mapping data. Through this implementation, the robot can be controlled remotely through the terminal device, and there is no need for relevant personnel to scan and control the robot in the space to be mapped, thus saving labor costs.

Correspondingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program. When the computer program is executed, the steps that can be executed by the terminal device in the foregoing method embodiments can be implemented.

Fig. 5 shows a schematic structural diagram of a robot device provided by an exemplary embodiment of the present application, and the robot device is suitable for the cloud robot mapping system provided by the foregoing embodiments. As shown in FIG. 5 , the robotic device includes: a memory 501 , a processor 502 and a communication component 503 .

The memory 501 is used to store computer programs, and can be configured to store other various data to support operations on the server. Examples of such data include instructions for any application or method operating on the server, contact data, phonebook data, messages, pictures, videos, etc.

The processor 502, coupled with the memory 501, is used to execute the computer program in the memory 501, so as to: send the collected real-time video data of the target space to the terminal device in real time through the cloud server; receive the terminal device through the cloud The control instruction sent by the server; the control instruction is sent according to the real-time video data; according to the control instruction, the mapping data is collected in the target space; the collected mapping data is sent to the cloud server, so that the cloud server according to The mapping data creates a map of the target space.

Further optionally, the processor 502 is also configured to: obtain its own device monitoring data; the device monitoring data includes: at least one of battery data, network status data, and abnormal event data of the robot; The device monitoring data is sent to the terminal device for operation and maintenance of the robot based on the device monitoring data.

Further optionally, the processor 502 is also configured to: during the process of collecting mapping data in the target space, send the motion track of the robot in the target space to the cloud server, so that the cloud server can Trajectories generate the motion trajectories required for the robot to perform tasks in this target space.

Further, as shown in FIG. 5 , the robotic device further includes: a display component 504 , a power supply component 505 , an audio component 506 and other components. Fig. 5 only schematically shows some components, which does not mean that the robotic device only includes the components shown in Fig. 5 .

Correspondingly, the embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed, the steps that can be performed by the robot device in the above method embodiments can be realized.

The memory in above-mentioned Fig. 4 and Fig. 5 can be realized by any type of volatile or non-volatile memory device or their combination, as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EPROM) EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

The above-mentioned communication components in FIG. 4 and FIG. 5 are configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on communication standards, such as WiFi, 2G, 3G, 4G or 5G, or a combination thereof. In one exemplary embodiment, the communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component may be based on Near Field Communication (NFC) technology, Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies to fulfill.

The above-mentioned display assembly in FIGS. 4 and 5 includes a screen, and the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or swipe action, but also detect the duration and pressure associated with the touch or swipe action.

The audio components in Figures 4 and 5 above can be configured to output and/or input audio signals. For example, the audio component includes a microphone (MIC), which is configured to receive an external audio signal when the device on which the audio component is located is in an operation mode, such as a calling mode, a recording mode, and a speech recognition mode. The received audio signal may be further stored in a memory or sent via a communication component. In some embodiments, the audio component further includes a speaker for outputting audio signals.

The power supply components in the above-mentioned Fig. 4 and Fig. 5 provide power for various components of the equipment where the power supply components are located. A power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component resides.

Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include non-permanent storage in computer readable media, in the form of random access memory (RAM) and/or nonvolatile memory such as read-only memory (ROM) or flash RAM. Memory is an example of computer readable media.

Computer-readable media, including both volatile and non-volatile, removable and non-removable media, can be implemented by any method or technology for storage of information. Information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, A magnetic tape cartridge, disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media excludes transitory computer-readable media, such as modulated data signals and carrier waves.

It should also be noted that the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes Other elements not expressly listed, or elements inherent in the process, method, commodity, or apparatus are also included. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

The above descriptions are only examples of the present application, and are not intended to limit the present application. For those skilled in the art, various modifications and changes may occur in this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

A method for building a map of a cloud robot, suitable for a terminal device, characterized in that it includes:

Identify the robots deployed in the target space to be mapped;

receiving the real-time video data of the target space sent by the robot in real time through the cloud server, and outputting the real-time video data;

In response to the robot control operation initiated according to the real-time video data, a corresponding control command is sent to the robot through the cloud server, so as to remotely control the action of the robot collecting mapping data in the target space.
The method according to claim 1, characterized in that, in response to the robot control operation initiated according to the real-time video data, sending corresponding control instructions to the robot through the cloud server, including:

In response to a motion control operation in any target direction initiated according to the real-time video data, a motion control command is sent to the robot through the cloud server; the motion control command includes: a forward command in the target direction, Back command, turn command or stop command.
The method according to claim 1, further comprising:

Obtaining the map of the target space from the cloud server; the map is generated by the cloud server according to the mapping data collected by the robot;

In response to an editing operation on the map, the map is updated, and the updated map is sent to the cloud server.
The method according to claim 3, wherein updating the map in response to an editing operation on the map includes at least one of the following:

In response to an obstacle update operation on the map, update obstacle information at a corresponding position on the map; the obstacle update operation includes: an obstacle deletion operation, an obstacle addition operation, or an obstacle movement operation;

Responding to an operation of marking any point of interest on the map, mark the mark of the point of interest on the map at a position corresponding to the point of interest;

drawing the virtual wall at a corresponding position on the map in response to an operation of setting a virtual wall at any position on the map;

In response to a marking operation on any area on the map, an area mark is added for the area on the map.
The method according to claim 1, further comprising:

Obtaining the mapping data of the target space from the cloud server; the mapping data is collected by the robot from the target space and uploaded to the cloud server;

and displaying the mapping data on the same screen as the real-time video data of the target space, so as to control the robot to move in the target space according to a comparison result between the real-time video data and the mapping data.
The method according to any one of claims 1-5, further comprising:

receiving device monitoring data sent by the robot through the cloud server; the device monitoring data includes: at least one of battery data, network status data, and abnormal event data of the robot;

When the device monitoring data indicates that the robot needs operation and maintenance processing, an operation and maintenance prompt message is output.
A method for building a map of a cloud robot, suitable for a robot, characterized in that it includes:

Send the collected real-time video data of the target space to the terminal device in real time through the cloud server;

receiving the control instruction sent by the terminal device through the cloud server; the control instruction is sent according to the real-time video data;

Collecting mapping data in the target space according to the control instruction;

Sending the collected mapping data to the cloud server, so that the cloud server creates a map of the target space according to the mapping data.
The method according to claim 7, wherein the mapping data includes: pose data, ranging data, mileage data, laser point cloud data, collision data, image data and drop detection data of the robot. at least one of .
The method according to claim 7, further comprising:

Obtaining its own device monitoring data; the device monitoring data includes: at least one of battery data, network status data and abnormal event data of the robot;

The device monitoring data is sent to the terminal device through the cloud server, so as to perform operation and maintenance processing on the robot according to the device monitoring data.
The method according to any one of claims 7-9, further comprising:

In the process of collecting mapping data in the target space, the motion trajectory of the robot in the target space is sent to the cloud server, so that the cloud server generates The motion trajectory required to perform the task in the target space.
A mapping system for a cloud robot, characterized in that it comprises:

A robot, a cloud server, and a terminal device; wherein, the terminal device establishes a communication connection with the robot through the cloud server;

Wherein, the terminal device is mainly used to: receive the real-time video data of the target space sent by the robot in real time through the cloud server, and output the real-time video data; respond to the robot control operation initiated according to the real-time video data, through The cloud server sends corresponding control instructions to the robot;

The robot is mainly used for sending the collected real-time video data of the target space to the terminal device in real time through the cloud server; receiving the control instruction sent by the terminal device through the cloud service; according to the control An instruction to collect mapping data in the target space; and send the collected mapping data to the cloud server, so that the cloud server creates a map of the target space according to the mapping data.
A terminal device, characterized by comprising: a memory, a processor, a communication component, and a display component;

Wherein, the memory is used to: store one or more computer instructions;

The processor is configured to execute the one or more computer instructions for: performing the steps in the method of any one of claims 1-6.
A robot device, characterized in that it includes: a memory, a processor, and a communication component;

Wherein, the memory is used to: store one or more computer instructions;

The processor is configured to execute the one or more computer instructions for: performing the steps in the method of any one of claims 7-10.
A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, the processor is caused to implement the steps in the method of any one of claims 1-10.