WO2023123457A1

WO2023123457A1 - Robot control method and apparatus, method and apparatus for controlling robot to return to base, and robot

Info

Publication number: WO2023123457A1
Application number: PCT/CN2021/143927
Authority: WO
Inventors: 李琳; 刘昂; 吴帆
Original assignee: 深圳市闪至科技有限公司
Priority date: 2021-12-31
Filing date: 2021-12-31
Publication date: 2023-07-06
Also published as: CN117835884A

Abstract

A robot control method and apparatus, a method and apparatus for controlling a robot to return to a base, and a robot. The robot control method comprises: acquiring the position of a ground point where a target object comes in contact with the ground (S202); planning an operation area on the basis of the position of the ground point where the target object comes in contact with the ground (S204); and controlling a robot to operate in the operation area (S206). An accurate position in an actual environment is transmitted to a robot in a convenient and fast manner, such that an operation area of the robot is planned on the basis of the accurate position, and the robot can be accurately controlled in a simple and fast manner.

Description

Robot control method, method and device for controlling robot to return to base, and robot

technical field

The present application relates to the technical field of robots, in particular, to a method for controlling a robot, a method and a device for controlling a robot to return to a base, and a robot.

Background technique

With the development of technology, various intelligent robots are widely used in various fields. In the process of the robot performing tasks, the user needs to control the robot. At present, users mostly control the robot through body buttons, remote control, mobile app, or body movements, but these control methods are either cumbersome to operate, inconvenient for users to operate, or difficult to transmit accurate location information. That is, in the current control method, it is impossible to quickly transmit the precise position in the real environment to the robot, so that the robot can perform tasks based on the position.

Contents of the invention

In view of this, the present application provides a robot control method, a method and a device for controlling the robot to return to the base, and the robot.

According to a first aspect of the present application, a method for controlling a robot is provided, the method comprising:

Obtain the position of the ground point where the target object touches the ground;

planning an operation area based on the position of the ground point where the target object is in contact with the ground;

The robot is controlled to work in the working area.

According to the second aspect of the present application, a robot control device is provided, the device includes a processor, a memory, and a computer program stored in the memory that can be executed by the processor. When the calculation program is executed, The following steps can be implemented:

The robot is controlled to work in the working area.

According to a third aspect of the present application, a robot is provided, and the robot includes the image processing device mentioned in the second aspect above.

According to the fourth aspect of the present application, there is provided a method for controlling a robot to return to a base, the robot is provided with an image acquisition device, and the base is provided with a target mark, the method comprising:

acquiring the image of the base captured by the image capture device;

Recognizing the target identifier from the image, and acquiring a first relative pose between the robot and the target identifier;

The robot is controlled to enter the base from the entrance based on the first relative posture and a second relative posture between the target mark and the entrance of the base.

According to the fifth aspect of the present application, there is provided a method for controlling a robot to return to a base, the robot is provided with a three-dimensional laser radar, and the method includes:

Obtaining spatial information of the base of the robot through the three-dimensional laser radar;

determining a relative pose of the robot to an entrance of the base based on the spatial information;

The robot is controlled to enter the base from the entrance based on the relative pose.

Applying the solution provided by this application, in the process of controlling the robot, the contact between the target object and the ground can be used, and the precise position information can be indicated through the ground point where the target object contacts the ground, so that the control device for controlling the robot can be based on The position of the target object determines the position of the ground point, and then plans the operation area based on the position of the ground point, so as to control the robot to perform operations in the area. In this way, the precise location in the actual environment can be transmitted to the robot in a convenient and quick way, so that the robot's work area can be planned based on the precise location, and the robot can be precisely controlled in a simple and quick way.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

Fig. 1(a)-Fig. 1(c) are schematic diagrams of the installation positions of the control device in different embodiments of the present application.

Fig. 2 is a flowchart of a robot control method according to an embodiment of the present application.

3-6 are schematic diagrams of determining an operation area based on ground points in multiple different embodiments of the present application.

Fig. 7 is a schematic diagram of controlling a robot to move to a target position in an embodiment of the present application.

Fig. 8 is a schematic diagram of a logic structure of a robot control device according to an embodiment of the present application.

Fig. 9 is a flowchart of a method for controlling a robot to return to a base according to an embodiment of the present application.

Fig. 10 is a flowchart of a method for controlling a robot to return to a base according to another embodiment of the present application.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some of the embodiments of the application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

With the development of technology, various intelligent robots are widely used in various fields. In the process of the robot performing tasks, the user needs to control the robot. At present, users mostly control the robot through body buttons, remote control, mobile phone App and other means. These control methods are not convenient to operate, and the user interaction experience is poor. There are also ways to control the robot through the user's limbs, for example, using gestures and limbs to control the robot. Although this method is convenient for the user to operate, it is difficult to transmit accurate position information.

Taking the sweeping robot as an example, when the user finds that a certain position on the ground is dirty, he can control the sweeping robot to move to the position through the mobile app. This method requires the user to constantly adjust the position of the robot, and the entire control process is cumbersome and time-consuming. longer. Or the user can click or select the location on the map through the App, and then set it as the cleaning area, so that the sweeping machine can automatically move to the location and perform cleaning. However, this method requires the user to accurately map the location in the actual environment to the map, which is difficult to implement and generally has certain deviations. Another way that is easy to think of is that the user can point the user's finger to the position, and then control the robot to move to the position, but since the position pointed by the finger is only an approximate range, it is difficult to align a precise position, and it is still impossible to achieve fast and accurate The location information is accurately transmitted to the robot so as to control the precise operation of the robot.

For another example, for some scenarios, it is necessary to use the robot to place objects in certain specific locations. Since it is impossible to quickly transmit accurate location information to the robot, it is also difficult to quickly and accurately control the robot.

All in all, none of the current control methods can accurately transmit the position determined by the user in the actual environment to the robot in a fast and convenient way, so that the robot can determine the working range based on the position and carry out the work.

Based on this, the embodiment of the present application provides a robot control method. In the process of controlling the robot, the target object can be used to contact the ground, and the precise position information can be indicated through the ground point where the target object contacts the ground, so that The control device for controlling the robot can determine the position of the ground point based on the position of the target object, and then plan an operation area based on the position of the ground point, so as to control the robot to perform operations in the area. In this way, the precise location in the actual environment can be transmitted to the robot in a convenient and quick way, so that the robot's work area can be planned based on the precise location, and the robot can be precisely controlled in a simple way.

The robot control method provided in the embodiment of the present application can be executed by a control device. In some embodiments, as shown in FIG. It could be a chip set on the robot. In some implementations, as shown in Figure 1(b), the control device can also be set on other equipment other than the robot. For example, the user can control and manage the robot through a certain control terminal, and the control device can also It can be set on the control terminal. In some embodiments, as shown in Figure 1(c), the control device can also include two, one is set on the robot, and the other is set on the control terminal of the robot, and some steps in the control method provided by the application can be It is executed by the control device installed on the robot, and some steps are executed by the control device installed on the control terminal. For example, the position of the ground point where the target object is in contact with the ground can be obtained through the control device on the control terminal, and then sent to the control device on the robot, so that the control device on the robot can plan the operation area based on the ground point and control the robot during operation. work in the area. It is worth noting that the control terminal can be a remote controller matched with the robot, or a butler robot or monitoring equipment with monitoring function matched with the robot.

It is not difficult to understand that the execution subject of each step of the method can be flexibly set based on actual application scenarios, which is not limited in this embodiment of the present application.

The target object in the embodiment of the present application can be various objects that can be in contact with the ground, for example, it can be the user's body parts, such as the user's feet (the user uses the foot to touch the ground), the user's finger (the user uses the finger to point Ground), or various auxiliary tools used by users, for example, users can use crutches to touch the ground, or disabled people can use various auxiliary tools to touch the ground and so on.

The robot in the embodiment of the present application may be various mobile intelligent robots, for example, the robot may be a sweeping robot, an electronic pet, a smart car, or other robots with certain specific functions.

Specifically, as shown in Figure 2, the robot control method provided in the embodiment of the present application may include the following steps:

S202. Obtain the position of the ground point where the target object contacts the ground;

In step S202, the position of the ground point where the target object contacts the ground may be acquired. Since the target object is in contact with the ground, the position of the target object can be determined first, and then the position of the ground point where the target object contacts the ground can be obtained. Wherein, the ground point may be one ground point or multiple ground points, which is not limited in this embodiment of the present application.

The position of the ground point can be determined in a variety of ways, for example, by collecting the image of the target object, predicting the position of the ground point based on the image and the pre-trained neural network, or by using some spatial information collection devices to determine the distance between the target object and The position information of the ground point in contact with the ground, for example, determine the position of the target object based on the spatial information collected by the depth sensor, laser radar, etc., and then determine the position of the ground point where the target object contacts the ground. It is not difficult to understand that various manners for determining the position of the ground point where the target object contacts the ground are within the protection scope of the embodiments of the present application.

S204. Planning an operation area based on the position of the ground point where the target object contacts the ground;

In step S204, after the position of the ground point where the target object is in contact with the ground is acquired, the operation area may be planned based on the position of the ground point. The method of planning the operation area based on the position of the ground point can also be flexibly set. For example, the position of the ground point is directly used as the operation position for operation. For example, the ground point is used as the target drop point, and the robot is controlled to drop the target on the ground. point. Alternatively, an area within a certain range around the ground point may also be used as an operation area, etc., which is not limited in this embodiment of the present application.

S206. Control the robot to work in the work area.

In step S206, after the planning of the work area is completed, the robot can be controlled to work in the work area. For example, the robot is a sweeping robot, that is, the work area can be cleaned, or the robot can be controlled to transport the object to the work area. Inner, etc., are not limited by the embodiment of this application.

The robot control method provided in the embodiment of the present application can accurately transmit position information by utilizing the contact between the target object and the ground, so that the control device can accurately determine the position of the ground point where the target object is in contact with the ground, and then based on the position of the ground point The location is used to plan the working area, so that the simple and convenient method can be used to accurately control the working area of the robot.

In some embodiments, when the target object is detected to be in contact with the ground, the above-mentioned process of planning the operation area based on the ground point and controlling the robot operation can be performed. For example, the target object can be some specific auxiliary tools, usually It does not come into contact with the ground, and only when the user uses it to control the robot, it will come into contact with the ground. Therefore, the above operations can be triggered when the target object is detected to be in contact with the ground. In some embodiments, in order to avoid false triggering when the user does not need to control the robot, causing the control device to perform the above-mentioned operation of planning the operation area and controlling the operation of the robot, it may also be that the control device receives the user's trigger instruction Then perform the above operations. For example, before the user wants to use the target object to control the robot, he can send a trigger command first. The control device will only execute the process of obtaining the ground point and planning the operation area after receiving the trigger command from the user.

In some embodiments, the user can issue a trigger instruction through a control terminal (such as an APP on a mobile phone). In some embodiments, in order to facilitate user operations, the user can also perform a specified action or pose a specified gesture as a trigger command. Touching the ground twice is used as a trigger command, and the control device determines that the trigger command has been received after recognizing the specified action or gesture.

In some embodiments, in order to prevent the user’s inadvertent action from being mistakenly recognized as a trigger instruction, the trigger instruction can also be that the user performs a specified action for a preset duration. After it is recognized that the time for the user to perform the specified action reaches the preset time, it is determined that the trigger instruction is received. In some embodiments, the trigger instruction can also be that the target object presents a specified posture. For example, the target object can be an auxiliary tool, such as a crutch. When the user wants to control the robot, the target object can be placed in a specified position. Gesture, after the control device recognizes that the target object presents a designated posture, it determines that a trigger instruction has been received. Of course, the above are only exemplary examples, and it is not difficult to understand that other various ways of transmitting information can be used to send the trigger instruction.

In some embodiments, the ground point includes multiple ground points successively contacting the target object, and the operation area planned based on the ground point may be an area formed by connecting the multiple ground points. For example, as shown in Figure 3, taking the sweeping robot as an example, when the user finds that a certain area on the ground is dirty and wants to clean the area, the user can touch the ground with his feet and click multiple ground points on the ground (as shown in the figure) ground 1, ground point 2...ground point 5), select the area, and the control device can connect the multiple ground points to form the working area after the control device recognizes the positions of the multiple ground points .

In some embodiments, the ground point may be multiple ground points that successively contact the target object, and there may be multiple operating areas, and each operating area is determined based on one of the multiple ground points. For example, as shown in Figure 4, taking the floor-sweeping robot as an example, when the user finds that there are dirt in different locations on the ground, he hopes that the floor-sweeping robot will clean these dirty locations, and the user can walk to each dirty location place, and then send out a trigger command, and touch the ground with your feet. At this time, the control device will identify the positions of multiple ground points (for example, ground point 1, ground point 2, and ground point 3) that are in contact with the feet, and Record the positions of these ground points, and then determine a work area based on each ground point. For example, you can use each ground as a vertex and specify a length as a side length to determine a work area. Or you can use the ground point as the center of the circle, specify the length as the radius, and determine a working area, so that multiple working areas corresponding to the dirty positions can be determined (for example, working area 1, working area 2, and working area 3 in the figure) .

In some embodiments, the ground point may be a plurality of ground points in contact with the target object. When planning an operation area according to the ground point, the operation area may be determined based on one or more of the plurality of ground points. For example, the user may Use the target object to draw a line or a circle on the ground, so that the work area can be determined based on the line or the circle. For example, users can use their feet to draw a straight line on the ground, and then use one end point of the line as the center and the other end point as the radius to determine the work area. Or when the dirt is a line-type dirt, the user can draw a line along the dirt with his feet, extend the two endpoints of the line horizontally to form a nearly rectangular area, and use this area as the work area. Or the user directly circles a nearly circular area with his feet on the ground, and uses the circular area as a working area. Or, as shown in Figure 5, when the user finds that the ground is dirty, the user can use his feet to draw two straight lines on both sides of the dirt on the ground, and then use the area between the two straight lines as the work area.

In some embodiments, the target object may be the two feet of the user. When planning the work area based on the ground points, the positions of the two ground points where the two feet of the user are in contact with the ground may be determined, and two The distance between ground points, and then determine the work area based on the distance between two ground points. For example, the ground point where one of the feet is in contact with the ground can be used as the center of the circle, and the distance can be used as the radius to determine the working area, or the midpoint of the line connecting the two grounds can be used as the center, and the distance can be used as the side length to determine a square area. As shown in Figure 6, taking the sweeping robot as an example, when the user finds dirt, the two feet can be moved a certain distance near the dirt, so that the sweeping robot can recognize the position of the two feet, and based on the position of the feet The distance between them determines the working area. By determining the working area based on the distance between the two feet of the user, the user can adjust the distance between the feet according to the size of the dirty area, thereby controlling the size of the working area.

In some embodiments, the trigger instruction may be a trigger instruction issued by the user through the target object, and the ground point may be a ground point that is in contact with the target object when the user sends the trigger instruction. When planning the operation area according to the position of the ground point , the first position can be determined according to the position of the ground point first, the first position can be the position where the ground point is located, or the position at a first preset distance from the ground point, and then the operation area is planned based on the first position . For example, take a sweeping robot as an example. The user can touch the ground twice with the foot to trigger the control of the robot. When the control device recognizes that the user has touched the ground twice with the foot, it can determine that the current foot is in contact with the ground. The position of the ground point, the user can leave after touching the ground with his feet, so that the sweeping robot can directly move to the position of the ground point, and then plan the operation area. Or, after the user touches the ground with their feet, they can also stay at this position. At this time, the sweeping robot can determine the first position that is a certain distance away from this position, and move to the first position to avoid touching the user's feet. Department, and then plan the operation area based on the first position.

In some embodiments, when the operation area is planned based on the first position, the operation area may be determined based on the first position and a second preset distance. For example, take the first position as the center and the second preset distance as the side length to determine a square area, or use the first position as the center and the second preset distance as the radius to determine a circular area, or use the first position as the vertex , the second preset distance is the side length to determine the working area.

In some embodiments, when determining the position of the ground point, the relative position of the ground point to the robot may be determined. For example, the spatial information of the space where the robot is located can be collected by the spatial information acquisition device, and then the relative position of the target object and the robot can be determined based on the spatial information, and then the relative position of the ground point where the target object contacts the ground and the robot can be determined. For example, the spatial information acquisition device can be a laser radar, which can collect point cloud information of the space where the robot is located, identify the target object based on the point cloud information, and determine the relative position of the ground point where the target object contacts the ground and the robot. Alternatively, the space information collection device can be a combination of a color camera and a depth camera, and the color image and the depth image of the space where the robot is collected are respectively collected by the color camera and the depth camera, that is, the target object can be identified from the image, and the target object can be determined to be in contact with the ground The relative position of the ground point of .

In some implementations, when determining the position of the ground point, the absolute position of the ground point in the pre-stored map may also be determined. Usually, the map of the robot's usual work area (such as a map of the home, a map of the office, etc.) can be stored in the robot or the control terminal of the robot. The map can be a two-dimensional map or a three-dimensional ground. Therefore, the The ground point position is projected onto the map in order to determine the absolute position of the ground point on the map.

In some embodiments, when determining the absolute position of the ground point on the map, the relative position between the ground point and the robot can be determined first based on the spatial information of the space where the robot is located, and then based on the current absolute position of the robot on the map and the relative position Position determines the absolute position of the ground point on the map. Since the real-time position of the robot on the map is known, the relative position of the ground point and the robot can be determined first, and then the position of the robot on the map determines the absolute position of the ground point on the ground.

In some embodiments, when determining the absolute position of the ground point in the map, the scene represented by the spatial information of the space where the robot is located can also be matched with the scene in the map, and the absolute position of the ground point in the map can be determined based on the matching result . For example, the map usually contains the scenes in the robot's usual work area and the distance between the scenes, so the spatial information of the space where the robot is located, such as point cloud information, depth images, etc., can be obtained, and then the scenes in the spatial information can be used Match the scene with the map to estimate the absolute position of the ground point on the map.

After the location of the ground point is determined, the robot can be controlled to move to the ground point or a location near the ground point, and then the operation area can be planned, or the operation area can be planned based on the location of the ground point, and then the robot can be controlled to move. For example, in some embodiments, before the operation area is planned based on the position of the ground point, the robot may be controlled to move to a third position. The location of the preset distance, and then plan the operation area based on the third location.

In some embodiments, the operation area can be planned based on the position of the ground point where the target object is in contact with the ground, and then the robot is controlled to move to the fourth position, wherein the fourth position can be the position where the ground point is located, or it can be related to the ground point. The position of the first preset distance from the ground point, or the position determined based on the planned operation area. For example, after planning the operation area, you can select the position closest to the robot in the operation area as the fourth position, and control the robot to move to After the fourth position, plan the operation area with the fourth position as the starting point.

In some embodiments, the third position and/or the fourth position may be collectively referred to as a target position, and when the robot is controlled to move to the target position, the robot may be controlled to move to the target position based on the relative position between the robot and the target position. For example, based on the relative position of the ground point and the robot, the relative position of the target position and the robot can be determined, and then the robot can be directly controlled to move to the target position based on the relative position.

In some embodiments, when controlling the robot to move to the target position, the absolute position of the target position in the pre-stored map can also be determined based on the relative position between the robot and the target position, and then according to the absolute position of the target position in the map, And the map plans the movement path, and controls the robot to move to the target position according to the movement path. By first projecting the target position onto the map, and then planning the movement path based on the map, the position of the obstacle can be determined from the map, and then the optimal path that can avoid the obstacle can be planned to improve the safety of the robot during the movement process .

In some embodiments, before controlling the robot to move to the target position, it is also possible to determine whether there is an obstacle between the robot and the target position based on the spatial information of the space where the machine is located. The relative position of the position controls the robot to move to the target position, so that there is no need to project the target position to the map first, and then plan the path, which can improve the processing efficiency.

If there is an obstacle between the robot and the target position, in order to improve the safety of the robot during the movement and avoid touching the obstacle, the target position can be projected onto the map first to determine the absolute position of the target position on the map. Then plan the movement path that can avoid obstacles according to the map, and control the robot to move to the target position based on the movement path. As shown in Figure 7, taking the sweeping robot as an example, after the user finds the dirt, he can touch the ground with his feet and control the sweeping robot to move to the dirty position for cleaning. Determine whether there are obstacles (such as steps, furniture, etc.) between the sweeping robot and the ground point. If not, control the movement of the robot directly based on the relative position of the two. If there is, project the ground point to a pre-stored map , then plan the movement path based on the ground and the location of the ground point on the map, and then control the robot to move to the location of the ground point based on the movement path.

The position of the ground point can be determined based on the spatial information of the space where the robot is collected by the space acquisition device, and the space acquisition device can be set on the control terminal of the robot or on the robot. In some embodiments, the spatial information collection device can be set on the robot, because when the user wants to control the operation of the robot, the robot may not necessarily be able to collect the spatial information of the target object. For example, take the floor sweeping robot as an example. Suppose the robot is in the living room and the user is in the room. The user finds that the room is dirty, and hopes to control the robot to move to the dirty position by touching the ground with its feet, and check the dirty position. Clean up. Since the robot and the user are in different rooms, the spatial information of the user cannot be collected.

In order to realize the scene where the target object cannot be collected by the space acquisition device, the position information can also be transmitted by the way the target object touches the ground to control the robot. In some embodiments, the user can first control the robot to move to a position where the spatial information of the target object can be collected through the control terminal, and then use the above method to control the robot to clean the dirty area. For example, the control device may receive a control command sent by the user through the control terminal, and control the robot to move to a first designated position based on the control command, wherein when the robot is located at the first designated position, the spatial information collected by the spatial information collection device Include the target object. By using the control terminal to control the robot to move to the position where the spatial information of the target object can be collected, and then use the target object to touch the ground to control the robot's operation, so that the robot can be precisely controlled when the user and the robot are located in different rooms .

In some embodiments, in the scene where the robot cannot collect the spatial information of the target object, the target user can also be identified first based on the spatial information collected by the robot, and follow the target user to move to the second designated position, and the second designated position is the same as the target user. The position where the target object stops moving is separated by a third preset distance, and after moving to the second specified position, the position of the ground point where the target object contacts the ground is obtained, and the operation area is planned based on the position of the ground point. For example, when the user and the robot are located in different rooms, the robot cannot obtain the spatial information of the target object. At this time, the robot can follow the user to a position where the spatial information of the target object can be collected, and then use the target object to touch the ground to control the robot. Work in a designated area.

In some embodiments, the robot can be a sweeping robot, and the sweeping robot is provided with a spatial information collection device. When the robot is controlled to work in the work area, it can first identify the operation based on the spatial information of the space where the sweeping robot is collected by the spatial information collection device. The dirty position in the area, and then control the sweeping robot to clean the dirty position. For example, the image acquisition device on the sweeping robot can collect the image of the ground, and then identify the dirty position from the image, and then clean the dirty position.

In some embodiments, the sweeping robot can also first identify the type of dirt in the work area based on the spatial information of the space where the sweeping robot is collected by the spatial information collection device, and determine the cleaning mode of the sweeping robot based on the type of dirt, to Use this cleaning mode to clean the dirt in the working area. For example, dirt includes liquid dirt and solid dirt, and for different types of dirt, different cleaning modes can be used for cleaning, so as to obtain a better cleaning effect.

In some embodiments, the cleaning robot includes a first cleaning device for cleaning liquid dirt and a second cleaning device for cleaning solid dirt. When determining the cleaning mode of the cleaning robot based on the type of dirt, if it is identified If there is liquid dirt in the working area, the cleaning mode of the sweeping robot can be switched to the mopping mode, wherein, when the sweeping robot is in the mopping mode, the first cleaning device is exposed to the liquid dirt before the second cleaning device. For example, the first cleaning device can be a mop, and the second cleaning device can be a side brush. When it is recognized that the dirt is liquid dirt, the mop can be used to clean the liquid dirt first, and then use the side brush to clean it. Determining the cleaning mode based on the type of dirt can achieve better cleaning results.

It should be pointed out that, in the case of no conflict, the solutions described in the embodiments of the present application can be freely combined, and due to space reasons, no examples are given here.

Corresponding to the above robot control method, the embodiment of the present application also provides a robot control device. As shown in FIG. The computer program executed by the processor 81, when the calculation program is executed, the following steps can be realized:

The robot is controlled to work in the working area.

In some embodiments, when the processor is used to acquire the position of the ground point where the target object is in contact with the ground, it is specifically used for:

In response to a user's trigger instruction, the position of the ground point where the target object is in contact with the ground is acquired.

In some embodiments, the trigger instruction includes one or more of the following:

The specified action or gesture of the user, the duration of the user's execution of the specified action reaches a preset duration, and the target object presents the specified gesture.

In some embodiments, the ground point includes multiple ground points that successively contact the target object, and the operation area is an area formed by connecting the multiple ground points.

In some embodiments, the ground point includes a plurality of ground points successively in contact with the target object, there are multiple operating areas, and each operating area is based on one ground point in the plurality of ground points Sure.

In some embodiments, when the processing is used to plan the operation area based on the position of the ground point where the target object is in contact with the ground, it is specifically used for:

The operation area is planned based on at least one location of a ground point where the target object is in contact with the ground.

In some embodiments, the target object includes two feet of the user, and the processor is used to plan the operation area based on the position of the ground point where the target object contacts the ground, specifically for:

The working area is determined based on the distance between two ground points where the two feet of the user are respectively in contact with the ground.

In some embodiments, the trigger instruction is a trigger instruction sent by the user through the target object, and the ground point includes a ground point that is in contact with the target object when the user sends the trigger instruction;

When the processor is used to plan the operation area based on the position of the ground point where the target object is in contact with the ground, it is specifically used for:

determining a first position based on the position of the ground point, where the first position is the position of the ground point, or a position at a first preset distance from the ground point;

The operation area is planned based on the first position.

In some embodiments, when the processor is configured to plan the operation area based on the first position, it is specifically configured to:

The working area is determined based on the first position and a second preset distance.

In some embodiments, the locations of the ground points include:

the relative position of the ground point to the robot, or

The absolute position of the ground point in the map pre-stored by the robot.

In some embodiments, the absolute position of the ground point in the map is determined based on the following methods:

Determine the relative position of the ground point and the robot based on the spatial information of the space where the robot is located, and determine that the ground point is in the map based on the current absolute position of the robot in the map and the relative position. the absolute position of the ; or

The scene represented by the spatial information of the space where the robot is located is matched with the scene in the map, and the absolute position of the ground point in the map is determined based on the matching result.

In some embodiments, before planning the operation area based on the position of the ground point where the target object contacts the ground, the processor is further configured to:

The robot is controlled to move to a third position, and the third position is the position where the ground point is located, or a position separated from the ground point by a first preset distance.

In some embodiments, after the operation area is planned based on the position of the ground point where the target object contacts the ground, the processor is further configured to:

Controlling the robot to move to a fourth position, the fourth position includes at least the position where the ground point is located, a position at a first preset distance from the ground point, and one of positions determined based on the work area kind.

In some embodiments, the target position includes the third position and/or the fourth position, and when the processor is used to control the robot to move to the target position, it is specifically used for:

controlling movement of the robot to the target location based on the relative position of the robot to the target location; or

Determining the absolute position of the target position in a map pre-stored by the robot based on the relative position between the robot and the target position, planning a movement path based on the absolute position of the target position in the map and the map ; controlling the robot to move to the target position based on the movement path.

In some embodiments, before controlling the robot to move to a target position in the work area, the processor is further configured to:

determining whether there is an obstacle between the robot and the target position based on the spatial information of the space where the machine is located;

if not, performing the step of controlling the robot to move to the target position based on the relative position of the robot to the target position;

If it exists, determine the absolute position of the target position in the map based on the relative position of the robot and the target position, plan the movement based on the absolute position of the target position in the map and the map path, and controlling the robot to move to the target position based on the moving path.

In some embodiments, the position of the ground point where the target object contacts the ground is determined based on spatial information of the space where the robot is located.

In some embodiments, the spatial information collection device is set on the robot, and the processor is also used for:

Receive the control instruction sent by the user through the control terminal;

Controlling the robot to move to a first designated position based on the control instruction, wherein when the robot is located at the first designated position, the spatial information collected by the spatial information collection device includes the target object.

Identifying a target user based on the spatial information collected by the robot, and following the target user to move to a second specified location, the second specified location is a third preset distance away from the location where the target user stops moving;

After moving to the second specified position, the acquisition of the position of the ground point where the target object contacts the ground is performed, and the operation area is planned based on the position of the ground point where the target object contacts the ground, and the robot is controlled in the The steps of the job within the job area.

In some embodiments, the robot is a sweeping robot, the sweeping robot is provided with a spatial information collection device, and the processor is used to control the robot to work in the working area, specifically for:

Identifying dirty locations in the work area based on the spatial information of the space where the sweeping robot is collected by the spatial information collection device;

The sweeping robot is controlled to clean the dirty location.

identifying the type of dirt in the work area based on the spatial information of the space where the sweeping robot is collected by the spatial information collection device;

The cleaning mode of the cleaning robot is determined based on the type of dirt, so as to use the cleaning mode to clean the dirt in the working area.

In some embodiments, the cleaning robot includes a first cleaning device for cleaning liquid dirt and a second cleaning device for cleaning solid dirt by the user, and the processor is used to determine the cleaning function of the cleaning robot based on the type of dirt. In cleaning mode, it is specifically used for:

When it is recognized that there is liquid contamination in the working area, the cleaning mode of the sweeping robot is switched to the mopping mode, wherein, when the sweeping robot is in the mopping mode, the first cleaning device is more efficient than the second cleaning device. The second cleaning device is first exposed to the liquid dirt.

Further, an embodiment of the present application also provides a robot, the robot includes the control device according to any one of the above embodiments. The robot can be a variety of intelligent robots such as sweeping robots, smart cars, and electronic pets.

After the robot completes the task, the robot needs to be controlled to return to the base to charge the robot. At present, when controlling the robot to return to the base, infrared positioning is mostly used. This method requires additional infrared sensors on the base and the robot, which is cumbersome and will increase costs.

Based on this, the embodiment of the present application provides a method for controlling the robot to return to the base. A target mark can be set on the base, wherein the target mark can be located around the entrance of the base, or inside the entrance of the base, or any other Position, only need to pre-calibrate the relative positional relationship between the target mark and the entrance of the base, and then collect the image of the base through the image acquisition device installed on the robot, and determine the relative position of the robot relative to the target by the posture of the target mark on the image. Then, based on the relative posture of the target mark and the entrance of the base, the relative posture of the robot and the entrance of the base is determined, so that the robot is controlled to enter the base from the entrance of the base based on the relative posture. In this way, automatic control of the robot back into the base can be realized without adding special sensors.

The robot in the embodiments of the present disclosure may be a sweeping robot, or other robots that need to be docked with the base, so as to return to the base for charging or perform other tasks. Described method is shown in Figure 9, comprises the following steps:

S902. Acquire the image of the base collected by the image collection device;

S904. Identify the target mark from the image, and acquire a first relative pose between the robot and the target mark;

S906. Based on the first relative posture and a second relative posture between the target mark and the entrance of the base, control the robot to enter the base from the entrance.

First, the image of the base can be collected by the image acquisition device on the robot, and then the target mark can be recognized from the image. The target mark can be a marker with a specified shape, or it can be a plurality of points arranged in a fixed way. The marks can be distributed on the outer surface of the base, or can also be distributed inside the entrance of the base, as long as it is convenient to recognize the target mark from the collected images and determine the relative posture to the target mark.

After the target mark is recognized from the image, a first relative pose between the robot and the target mark can be determined based on the pose presented by the target mark in the image. Among them, the robot can be pre-collected images of the base at different positions and from different angles, and determine the pose of the target mark in the image when the robot does not collect the base image through the angle at different positions, so as to determine the pose and the position of the target mark. The corresponding relationship between the position and the angle of the robot, and then based on the corresponding relationship and the collected image of the base, the first relative posture between the robot and the target mark can be determined. Alternatively, a large number of images containing target identifications carrying label information can be used as training samples, the label information is used to indicate the relative pose of the robot and the target identification, and then the training samples are used to train the neural network. After the image of the base is collected, the image can be input into the neural network, and the neural network is used to predict the first relative posture.

Since the second relative posture of the target mark and the base entrance position can be calibrated in advance, the relative posture of the robot and the base entrance can be determined based on the first relative posture and the second relative posture, and then the robot adjustment can be controlled based on the relative posture. Move the direction so that it is aligned with the pedestal entrance and enters the pedestal from the pedestal entrance.

In some embodiments, the relative position of the robot and the base can be determined based on the collected spatial information of the base (for example, point cloud or depth image), and then the robot is controlled to move to the vicinity of the base based on the relative position, and then the above-mentioned The method controls the robot to enter the base from the base entrance.

In addition, the embodiment of the present application also provides another method for controlling the robot to return to the base, as shown in Figure 10, the method may include the following steps:

S102. Obtain spatial information of the base of the robot through the three-dimensional laser radar;

S104. Determine the relative posture of the robot and the entrance of the base based on the spatial information;

S106. Control the robot to enter the base from the entrance based on the relative posture.

The spatial information of the base, such as point cloud information, can be obtained by means of the 3D laser radar installed on the robot, and then the relative attitude between the robot and the entrance of the base can be determined according to the spatial information, and the moving direction of the robot can be adjusted based on the relative attitude for alignment. the base entrance, and enter into the base from the base entrance.

Through the above two methods, the robot can be controlled to automatically return to the base without additional sensors, which can save costs.

Correspondingly, the embodiments of this specification further provide a computer storage medium, where a program is stored in the storage medium, and when the program is executed by a processor, the method in any of the foregoing embodiments is implemented.

Embodiments of the present description may take the form of a computer program product embodied on one or more storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) having program code embodied therein. Computer usable storage media includes both volatile and non-permanent, removable and non-removable media, and may be implemented by any method or technology for information storage. Information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for computers include, but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cartridge, tape magnetic disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

As for the device embodiment, since it basically corresponds to the method embodiment, for related parts, please refer to the part description of the method embodiment. The device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without creative effort.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is a relationship between these entities or operations. There is no such actual relationship or order between them. The term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements but also other elements not expressly listed elements, or also elements inherent in such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

The methods and devices provided by the embodiments of the present invention have been described in detail above. The principles and implementation methods of the present invention have been explained by using specific examples in this paper. The descriptions of the above embodiments are only used to help understand the methods and methods of the present invention. core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation and scope of application. In summary, the content of this specification should not be construed as limiting the present invention .

Claims

A control method for a robot, characterized in that the method comprises:

Obtain the position of the ground point where the target object touches the ground;

planning an operation area based on the position of the ground point where the target object is in contact with the ground;

The robot is controlled to work in the working area.
The method according to claim 1, wherein said obtaining the position of the ground point where the target object contacts the ground comprises:

In response to a user's trigger instruction, the position of the ground point where the target object is in contact with the ground is acquired.
The method according to claim 2, wherein the trigger instruction includes one or more of the following:

The specified action or gesture of the user, the duration of the user's execution of the specified action reaches a preset duration, and the target object presents the specified gesture.
The method according to any one of claims 1-3, wherein the ground point includes a plurality of ground points that successively contact the target object, and the operation area is connected by the plurality of ground points constituted area.
The method according to any one of claims 1-3, wherein the ground point includes a plurality of ground points successively in contact with the target object, and there are multiple operating areas, each of the operating areas The determination is based on a ground point of the plurality of ground points.
The method according to any one of claims 1-3, wherein the planning of the operation area based on the position of the ground point where the target object contacts the ground comprises:

The operation area is planned based on at least one location of a ground point where the target object is in contact with the ground.
The method according to any one of claims 1-3, wherein the target object includes two feet of the user, and planning the operation area based on the position of the ground point where the target object contacts the ground includes:

The working area is determined based on the distance between two ground points where the two feet of the user are respectively in contact with the ground.
The method according to claim 2, wherein the trigger instruction is a trigger instruction sent by the user through the target object, and the ground point includes the ground that is in contact with the target object when the user sends the trigger instruction point;

The operation area is planned based on the position of the ground point where the target object is in contact with the ground, including:

determining a first position based on the position of the ground point, where the first position is the position of the ground point, or a position at a first preset distance from the ground point;

The operation area is planned based on the first position.
The method according to claim 8, wherein planning the operation area based on the first position comprises:

The working area is determined based on the first position and a second preset distance.
The method according to any one of claims 1-9, wherein the position of the ground point comprises:

the relative position of the ground point to the robot, or

The absolute position of the ground point in the pre-stored map.
The method according to claim 10, wherein the absolute position of the ground point in the map is determined based on the following method:

Determine the relative position of the ground point and the robot based on the spatial information of the space where the robot is located, and determine that the ground point is in the map based on the current absolute position of the robot in the map and the relative position. the absolute position of the ; or

The scene represented by the spatial information of the space where the robot is located is matched with the scene in the map, and the absolute position of the ground point in the map is determined based on the matching result.
The method according to claim 1, wherein, before planning the operation area based on the position of the ground point where the target object is in contact with the ground, the method further comprises:

The robot is controlled to move to a third position, and the third position is the position where the ground point is located, or a position separated from the ground point by a first preset distance.
The method according to claim 1, characterized in that, after planning the operation area based on the position of the ground point where the target object is in contact with the ground, the method further comprises:

Controlling the robot to move to a fourth position, the fourth position includes at least the position where the ground point is located, a position at a first preset distance from the ground point, and one of positions determined based on the work area kind.
The method according to claim 12 or 13, wherein the target position includes the third position and/or the fourth position, and controlling the robot to move to the target position comprises:

controlling movement of the robot to the target location based on the relative position of the robot to the target location; or

Determining the absolute position of the target position in a pre-stored map based on the relative position between the robot and the target position, planning a movement path based on the absolute position of the target position in the map and the map; The moving path controls the robot to move to the target position.
The method according to claim 14, wherein before controlling the robot to move to a target position in the work area, further comprising:

determining whether there is an obstacle between the robot and the target position based on the spatial information of the space where the machine is located;

if not, performing the step of controlling the robot to move to the target position based on the relative position of the robot to the target position;

If it exists, determine the absolute position of the target position in the map based on the relative position of the robot and the target position, plan the movement based on the absolute position of the target position in the map and the map path, and controlling the robot to move to the target position based on the moving path.
The method according to any one of claims 1-15, characterized in that the position of the ground point where the target object is in contact with the ground is determined based on spatial information of the space where the robot is located.
The method according to claim 16, wherein the spatial information collection device is arranged on the robot, and the method further comprises:

Receive the control instruction sent by the user through the control terminal;

Controlling the robot to move to a first designated position based on the control instruction, wherein when the robot is located at the first designated position, the spatial information collected by the spatial information collection device includes the target object.
The method according to claim 16, wherein the spatial information collection device is arranged on the robot, and the method further comprises:

Identifying a target user based on the spatial information collected by the robot, and following the target user to move to a second specified location, the second specified location is a third preset distance away from the location where the target user stops moving;

After moving to the second specified position, the acquisition of the position of the ground point where the target object contacts the ground is performed, and the operation area is planned based on the position of the ground point where the target object contacts the ground, and the robot is controlled in the The steps of the job within the job area.
The method according to any one of claims 1-18, wherein the robot is a sweeping robot, and the sweeping robot is provided with a spatial information collection device, and the robot is controlled to work in the working area, including :

Identifying dirty locations in the work area based on the spatial information of the space where the sweeping robot is collected by the spatial information collection device;

The sweeping robot is controlled to clean the dirty location.
The method according to any one of claims 1-19, wherein the robot is a sweeping robot, and the sweeping robot is provided with a spatial information collection device, and the robot is controlled to work in the working area, including :

identifying the type of dirt in the work area based on the spatial information of the space where the sweeping robot is collected by the spatial information collection device;

The cleaning mode of the cleaning robot is determined based on the type of dirt, so as to use the cleaning mode to clean the dirt in the working area.
The method according to claim 20, wherein the sweeping robot includes a first cleaning device for cleaning liquid dirt and a second cleaning device for cleaning solid dirt by the user, and the sweeping robot is determined based on the type of dirt The cleaning mode of the robot, including:

When it is recognized that there is liquid contamination in the working area, the cleaning mode of the sweeping robot is switched to the mopping mode, wherein, when the sweeping robot is in the mopping mode, the first cleaning device is more efficient than the second cleaning device. The second cleaning device is first exposed to the liquid dirt.
A control device for a robot, characterized in that the device includes a processor, a memory, and a computer program stored in the memory that can be executed by the processor. When the calculation program is executed, the following steps can be realized:

Obtain the position of the ground point where the target object touches the ground;

planning an operation area based on the position of the ground point where the target object is in contact with the ground;

The robot is controlled to work in the working area.
The device according to claim 22, wherein when the processor is used to obtain the position of the ground point where the target object contacts the ground, it is specifically used for:

In response to a user's trigger instruction, the position of the ground point where the target object is in contact with the ground is acquired.
The device according to claim 23, wherein the trigger instruction includes one or more of the following:

The specified action or gesture of the user, the duration of the user's execution of the specified action reaches a preset duration, and the target object presents the specified gesture.
The device according to any one of claims 22-24, wherein the ground point includes a plurality of ground points successively in contact with the target object, and the operation area is connected by the plurality of ground points constituted area.
The device according to any one of claims 22-24, wherein the ground point includes a plurality of ground points that successively contact the target object, and there are multiple operating areas, each of the operating areas The determination is based on a ground point of the plurality of ground points.
The device according to any one of claims 22-24, wherein when the processing is used to plan the operation area based on the position of the ground point where the target object contacts the ground, it is specifically used for:

The operation area is planned based on at least one location of a ground point where the target object is in contact with the ground.
The device according to any one of claims 22-24, wherein the target object includes two feet of the user, and the processor is used for planning operations based on the position of the ground point where the target object contacts the ground area, specifically for:

The working area is determined based on the distance between two ground points where the two feet of the user are respectively in contact with the ground.
The device according to claim 23, wherein the trigger command is a trigger command sent by the user through the target object, and the ground point includes the ground that is in contact with the target object when the user sends the trigger command point;

When the processor is used to plan the operation area based on the position of the ground point where the target object contacts the ground, it is specifically used for:

determining a first position based on the position of the ground point, where the first position is the position of the ground point, or a position at a first preset distance from the ground point;

The operation area is planned based on the first position.
The device according to claim 29, wherein when the processor is used to plan the operation area based on the first position, it is specifically used to:

The working area is determined based on the first position and a second preset distance.
The device according to any one of claims 22-30, wherein the positions of the ground points include:

the relative position of the ground point to the robot, or

The absolute position of the ground point in the pre-stored map.
The device according to claim 31, wherein the absolute position of the ground point in the map is determined based on the following method:

Determine the relative position of the ground point and the robot based on the spatial information of the space where the robot is located, and determine that the ground point is in the map based on the current absolute position of the robot in the map and the relative position. the absolute position of the ; or

The scene represented by the spatial information of the space where the robot is located is matched with the scene in the map, and the absolute position of the ground point in the map is determined based on the matching result.
The device according to claim 22, wherein before planning an operation area based on the position of the ground point where the target object contacts the ground, the processor is further configured to:

The robot is controlled to move to a third position, and the third position is the position where the ground point is located, or a position separated from the ground point by a first preset distance.
The device according to claim 22, wherein after planning the operation area based on the position of the ground point where the target object contacts the ground, the processor is further configured to:

Controlling the robot to move to a fourth position, the fourth position includes at least the position where the ground point is located, a position at a first preset distance from the ground point, and one of positions determined based on the work area kind.
The method according to claim 33 or 34, wherein the target position includes the third position and/or the fourth position, and the processor is used to control the robot to move to the target position, specifically Used for:

controlling movement of the robot to the target location based on the relative position of the robot to the target location; or

Determining the absolute position of the target position in a pre-stored map based on the relative position between the robot and the target position, planning a movement path based on the absolute position of the target position in the map and the map; The moving path controls the robot to move to the target position.
The device according to claim 35, wherein before controlling the robot to move to a target position in the work area, the processor is further configured to:

determining whether there is an obstacle between the robot and the target position based on the spatial information of the space where the machine is located;

if not, performing the step of controlling the robot to move to the target position based on the relative position of the robot to the target position;

If it exists, determine the absolute position of the target position in the map based on the relative position of the robot and the target position, plan the movement based on the absolute position of the target position in the map and the map path, and controlling the robot to move to the target position based on the moving path.
The device according to any one of claims 22-36, wherein the position of the ground point where the target object contacts the ground is determined based on spatial information of the space where the robot is located.
The device according to claim 37, wherein the spatial information collection device is arranged on the robot, and the processor is also used for:

Receive the control instruction sent by the user through the control terminal;

Controlling the robot to move to a first designated position based on the control instruction, wherein when the robot is located at the first designated position, the spatial information collected by the spatial information collection device includes the target object.
The device according to claim 37, wherein the spatial information collection device is arranged on the robot, and the processor is also used for:

Identifying a target user based on the spatial information collected by the robot, and following the target user to move to a second specified location, the second specified location is a third preset distance away from the location where the target user stops moving;

After moving to the second specified position, the acquisition of the position of the ground point where the target object contacts the ground is performed, and the operation area is planned based on the position of the ground point where the target object contacts the ground, and the robot is controlled in the The steps of the job within the job area.
The device according to any one of claims 22-39, characterized in that, the robot is a sweeping robot, the sweeping robot is provided with a spatial information collection device, and the processor is used to control the When working in the area, it is specifically used for:

Identifying dirty locations in the work area based on the spatial information of the space where the sweeping robot is collected by the spatial information collection device;

The sweeping robot is controlled to clean the dirty location.
The device according to any one of claims 22-40, wherein the robot is a sweeping robot, the sweeping robot is provided with a spatial information collection device, and the processor is used to control the When working in the area, it is specifically used for:

Identifying the type of dirt in the work area based on the spatial information of the space where the sweeping robot is collected by the spatial information collection device;

The cleaning mode of the cleaning robot is determined based on the type of dirt, so as to use the cleaning mode to clean the dirt in the working area.
The device according to claim 41, wherein the sweeping robot includes a first cleaning device for cleaning liquid dirt and a second cleaning device for cleaning solid dirt by the user, and the processor is used for cleaning based on dirt When the type determines the cleaning mode of the sweeping robot, it is specifically used for:

When it is recognized that there is liquid contamination in the working area, the cleaning mode of the sweeping robot is switched to the mopping mode, wherein when the sweeping robot is in the mopping mode, the first cleaning device is more efficient than the second cleaning device. The second cleaning device is first exposed to the liquid dirt.
A robot, characterized in that the robot comprises the device according to any one of claims 22-42.
A method for controlling a robot to return to a base, wherein the robot is provided with an image acquisition device, and the base is provided with a target mark, and the method comprises:

acquiring the image of the base captured by the image capture device;

Recognizing the target identifier from the image, and acquiring a first relative pose between the robot and the target identifier;

The robot is controlled to enter the base from the entrance based on the first relative posture and a second relative posture between the target mark and the entrance of the base.
A method for controlling a robot to return to a base, wherein the robot is provided with a three-dimensional laser radar, and the method comprises:

acquiring spatial information of the base of the robot through the three-dimensional laser radar;

determining a relative pose of the robot to an entrance of the base based on the spatial information;

The robot is controlled to enter the base from the entrance based on the relative pose.