CN110370273B

CN110370273B - Robot obstacle avoidance method, device and system

Info

Publication number: CN110370273B
Application number: CN201910566538.1A
Authority: CN
Inventors: 徐学军
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2019-06-27
Filing date: 2019-06-27
Publication date: 2021-04-09
Anticipated expiration: 2039-06-27
Also published as: WO2020259524A1; CN110370273A

Abstract

The embodiment of the application discloses a robot obstacle avoidance method, device and system, relates to the field of electronic equipment, and solves the problems that the labor cost is high and the accuracy of a trained model is low in the process of using a machine learning method to identify a dilemma. The specific scheme is as follows: the robot obtains the sample information of N moments in the moving process. Wherein, the sample information of each moment in N moments comprises: the robot comprises environment image information used for indicating the environment where the robot is located at the moment, position information used for indicating the position where the robot is located at the moment, and label information used for indicating whether the robot is in a trapped state at the moment. And the robot sends the sample information of M times in the N times to the server. The robot receives from the server a dilemma recognition model trained from sample information collected by one or more robots. And the robot avoids the obstacle in the moving process according to the predicament recognition model. The embodiment of the application is used in the moving process of the robot.

Description

Robot obstacle avoidance method, device and system

Technical Field

The application relates to the field of electronic equipment, in particular to a robot obstacle avoidance method, device and system.

Background

Robots (e.g., sweeping robots) have been widely used in people's daily lives. Since the operation of a robot is almost completely autonomous, some trapped situations are often encountered during the operation. At present, a robot can be trained to recognize a predicament by using a machine learning method so as to avoid obstacles.

Among them, the more common machine learning methods include supervised learning and reinforcement learning. In supervised learning, a large amount of sample data needs to be manually collected, corresponding trapped or non-trapped labels are manually marked on the sample data, and then supervised learning model training is performed on the marked sample data to obtain a corresponding model, so that the robot can judge whether obstacle avoidance needs to be performed according to the model.

For example, the robot is a sweeping robot, and the sample data is environment information where the sweeping robot is located. As shown in fig. 1A, because the sweeping robot is trapped by the connecting wires of the socket and cannot move freely, a worker can mark the trapped tag on the sample data. For another example, as shown in fig. 1B, the sweeping robot is not trapped by the socket and the connecting line thereof, and can move freely, so that a worker can mark the sample data with an un-trapped label. For another example, in the environment shown in fig. 2A, the sweeping robot is located at an included angle between the wall and the fixed object, and cannot move freely, so that a worker can put a trapped label on the sample data. For another example, in the environment shown in fig. 2B, the sweeping robot is located far away from the wall and the fixed object, and can continue to move freely, so that the user can manually mark the sample data with a label that is not trapped. For another example, in the environment shown in fig. 3A, the sweeping robot is in a gap between a fixed object and the ground, and cannot move freely, so that a person can manually mark a trapped tag on the sample data. Correspondingly, as shown in fig. 3B, the robot is far away from the fixed object and the gap between the fixed object and the ground, and can move freely, so that the worker can mark the sample data with an un-trapped label. According to a large amount of labeled sample data similar to those shown in fig. 1A-3B, supervised learning model training is performed to obtain a corresponding model for obstacle avoidance of the sweeping robot.

Reinforcement learning refers to a method in which a robot learns in a "trial and error" manner. The robot determines the behavior with the highest reward fed back by the environment through continuous trial and error according to a large amount of manually collected sample data and the trapped or non-trapped labels manually printed on the sample data. And marking according to the reward result and a trial and error method corresponding to the reward result, and taking the marked result as a reference for next work.

It can be seen that no matter supervised learning or reinforcement learning is used for dilemma recognition, a large amount of sample data is required. Currently, the sample data needs to be collected manually, such as by web crawler search. In order to distinguish the state of the robot corresponding to the collected sample data (for example, whether the robot is in a trapped state), the collected sample data needs to be manually identified and marked with a corresponding label. This results in a large investment in labor costs. In addition, because there is no uniform sample data collection standard, the quality of the collected sample data is difficult to control, which may result in low labeling quality of the label, such as labeling error. And the sample data collected by the web crawler, such as the above environment information, are mostly environment pictures shot at the angle of the user, and the environment where the robot is located cannot be intuitively and accurately reflected. Both of these can lead to inaccuracies in the trained model.

Disclosure of Invention

The embodiment of the application provides a robot obstacle avoidance method, device and system, and solves the problems that the labor cost is high and the accuracy of a trained model is low in the process of using a machine learning method to identify a predicament.

In order to achieve the above purpose, the embodiment of the present application adopts the following technical solutions:

in a first aspect of the embodiments of the present application, a robot obstacle avoidance method is provided, where the method includes: the method comprises the following steps that a robot obtains sample information of N moments in the moving process of the robot, wherein N is an integer larger than 0; wherein, the sample information of each moment in N moments comprises: environment image information indicating an environment in which the robot is located at the moment, position information indicating a position of the robot in the environment indicated by the environment image information at the moment, and tag information indicating a state of the robot at the moment; the label information comprises a first label or a second label, the first label is used for indicating that the robot is in an un-trapped state, and the second label is used for indicating that the robot is in a trapped state; the robot sends the sample information of M moments in the N moments to a server, wherein M is a positive integer less than or equal to N; the method comprises the following steps that the robot receives a predicament recognition model from a server, wherein the predicament recognition model is obtained by training sample information collected by one or more robots; and the robot avoids the obstacle in the moving process according to the predicament recognition model.

Therefore, the robot can automatically acquire the environmental image information and the position information and acquire 100% correct label information for identifying whether the robot is in the trapped state or not according to the current state, so that the accuracy of the sample information is ensured and a large amount of manpower is avoided. The robot sends the sample information of N moments to the server, acquires the predicament recognition model through training, and is used for guiding the robot to avoid the obstacle.

With reference to the first aspect and the foregoing possible implementation manners, in another possible implementation manner, the environment image information at each time includes: and at the moment, the robot is in the environment, and the image information of objects on the moving route and around the moving route of the robot is obtained. In this way, it is ensured that the environment image information may include information of an object that is most likely to be an obstacle to the movement of the robot.

With reference to the first aspect and the foregoing possible implementation manners, in another possible implementation manner, the location information at each time includes: the environmental image information at this point in time indicates relative position information between the robot and an object on and around the moving path of the robot in the environment indicated by the environmental image information. In this way, the specific position of the robot in the environment at the moment can be determined, so that the robot can accurately predict the position of the robot in the environment at the next moment.

With reference to the first aspect and the foregoing possible implementation manners, in another possible implementation manner, a method for a robot to send sample information of M times of N times to a server includes: the robot determines sample information at N moments, wherein label information included in the sample information at the Nth moment is used for indicating that the robot is in a trapped state at the Nth moment; and the robot sends the sample information of the first M-1 time of the Nth time and the sample information of the Nth time to the server. Therefore, for the training of the predicament recognition model, the quality of the trapped sample information and the quality of the sample information in a period of time before the trapped sample information are higher, so that the transmission of high-quality sample information is ensured, the total amount of system information transmission is reduced, and the communication load of the system is reduced.

With reference to the first aspect and the foregoing possible implementation manners, in another possible implementation manner, a robot performs obstacle avoidance in a moving process according to a predicament recognition model, including: the robot acquires environmental image information at the current moment and position information at the current moment; the robot acquires environment image information of the next moment and position information of the next moment according to the environment image information of the current moment, the position information of the current moment and the moving direction and the moving speed of the robot at the current moment; the robot determines the probability of the robot being trapped at the next moment according to the environment image information at the next moment, the position information at the next moment and the predicament recognition model; and the robot determines that the probability of the trapped robot at the next moment is greater than a preset threshold value, and the robot changes a motion strategy to avoid obstacles. Therefore, whether the robot is trapped at the next moment or not can be accurately judged by predicting the environment image information and the position information of the environment where the robot is located at the next moment and combining the trapping recognition model, the obstacle is avoided according to the result, and the situation that the robot is trapped in the moving process can be effectively avoided.

In a second aspect of the embodiments of the present application, an obstacle avoidance device is provided, where the obstacle avoidance device may include: the device comprises an acquisition unit, a communication unit and an obstacle avoidance unit. The acquisition unit is used for acquiring sample information of N moments in the moving process of the robot, wherein N is an integer larger than 0; wherein, the sample information of each moment in N moments comprises: environment image information indicating an environment in which the robot is located at the moment, position information indicating a position of the robot in the environment indicated by the environment image information at the moment, and tag information indicating a state of the robot at the moment; the label information comprises a first label or a second label, the first label is used for indicating that the robot is in an un-trapped state, and the second label is used for indicating that the robot is in a trapped state; the communication unit is used for sending the sample information of M moments in the N moments to the server, wherein M is a positive integer less than or equal to N; the communication unit is also used for receiving a predicament recognition model from the server, and the predicament recognition model is obtained by training sample information collected by one or more robots; and the obstacle avoidance unit is used for avoiding obstacles in the moving process of the robot according to the dilemma recognition model.

With reference to the second aspect and the foregoing possible implementation manners, in another possible implementation manner, the environment image information at each time includes: and at the moment, the robot is in the environment, and the image information of objects on the moving route and around the moving route of the robot is obtained.

With reference to the second aspect and the foregoing possible implementation manners, in another possible implementation manner, the location information at each time includes: the environmental image information at this point in time indicates relative position information between the robot and an object on and around the moving path of the robot in the environment indicated by the environmental image information.

With reference to the second aspect and the foregoing possible implementation manners, in another possible implementation manner, the determining unit is configured to determine, of the sample information at N times, that the sample information at the nth time includes label information for indicating that the machine is in a trapped state at the nth time; the communication unit is used for sending the sample information of M moments in N moments to the server, and comprises: and the communication unit is used for sending the sample information of the first M-1 time of the Nth time and the sample information of the Nth time to the server.

With reference to the second aspect and the foregoing possible implementation manners, in another possible implementation manner, the obtaining unit is further configured to obtain environmental image information at the current time and position information at the current time, and obtain environmental image information at a next time and position information at the next time according to the environmental image information at the current time, the position information at the current time, and the moving direction and the moving speed of the robot at the current time; keep away barrier unit for keep away the barrier according to the predicament recognition model at robot removal in-process, include: and the obstacle avoidance unit is used for determining the probability of the robot being trapped at the next moment according to the environment image information at the next moment, the position information at the next moment and the predicament recognition model, changing the motion strategy of the robot and avoiding obstacles when determining that the probability of the robot being trapped at the next moment is greater than a preset threshold value.

In a third aspect of the embodiments of the present application, an obstacle avoidance system for a robot is provided, where the obstacle avoidance system may include: one or more robots, and a server. The robot is used for acquiring the sample information of N moments in the moving process of the robot and sending the sample information of the N moments to the server; wherein N is an integer greater than 0, and the sample information at each of the N times includes: environment image information indicating an environment in which the robot is located at the moment, position information indicating a position of the robot in the environment indicated by the environment image information at the moment, and tag information indicating a state of the robot at the moment; the label information comprises a first label or a second label, the first label is used for indicating that the robot is in an un-trapped state, and the second label is used for indicating that the robot is in a trapped state; the server is used for receiving the sample information from the one or more robots, training the received sample information of the one or more robots, obtaining a predicament recognition model, and sending the predicament recognition model to the robots; the robot is further used for receiving the predicament recognition model and avoiding obstacles in the moving process according to the predicament recognition model.

With reference to the third aspect and the foregoing possible implementation manners, in another possible implementation manner, the server receives the sample information including the first type sample information and the second type sample information; the server is used for training the received sample information of one or more robots to obtain a predicament recognition model, and the predicament recognition model comprises the following steps: the server is used for training the first type of sample information to obtain an initial model, verifying the accuracy of the initial model according to the second type of sample information, determining the initial model as a predicament recognition model if the verification accuracy is greater than a preset threshold, receiving new sample information sent by the robot by the server if the verification accuracy is less than the preset threshold, continuing training according to the new sample information and the initial model, verifying the model obtained by continuing training according to the second type of sample information until the verification accuracy is greater than the preset threshold, and determining the model with the verification accuracy greater than the preset threshold as the predicament recognition model; if the state of whether the robot indicated by the result obtained by inputting the environmental image information and the position information in the second type sample information into the initial model is the same as the state of whether the robot indicated by the label information in the second type sample information is trapped, the sample information is determined to be verified accurately; if the state of whether the robot indicated by the result obtained by inputting the environmental image information and the position information of each piece of sample information in the second type of sample information into the initial model is different from the state of whether the robot indicated by the label information in the second type of sample information is trapped, determining that the verification of the sample information is inaccurate; and determining the verification accuracy rate according to the verification result of each sample information in the second type of sample information. Therefore, the server can acquire an accurate predicament recognition model through training and verification of the first type of sample information and the second type of sample information and send the predicament recognition model to the robot for effective obstacle avoidance.

In a fourth aspect, embodiments of the present application provide a robot, where the robot may include a processor, configured to connect to a memory, and call a program stored in the memory to perform the robot obstacle avoidance method according to the first aspect or any one of the possible implementation manners of the first aspect.

In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, including: computer software instructions; when the computer software instructions are run in the obstacle avoidance apparatus, the obstacle avoidance apparatus is caused to execute the robot obstacle avoidance method according to the first aspect or any one of the possible implementation manners of the first aspect.

In a sixth aspect, embodiments of the present application provide a computer program product, which when run on a computer, causes the computer to execute the robot obstacle avoidance method according to the first aspect or any one of the possible implementation manners of the first aspect.

It is to be understood that the above-mentioned obstacle avoidance apparatus of the second aspect, the robot obstacle avoidance system of the third aspect, the robot of the fourth aspect, the computer readable storage medium of the fifth aspect, and the computer program product of the sixth aspect are all configured to perform the corresponding methods provided above, and therefore, the beneficial effects that can be achieved by the above-mentioned method can be referred to and are not described herein again.

Drawings

FIG. 1A is an environmental schematic diagram of a robot during movement;

FIG. 1B is a schematic diagram of another environment of the robot during movement;

FIG. 2A is a schematic diagram of another environment of the robot during movement;

FIG. 2B is a schematic diagram of another environment of the robot during movement;

FIG. 3A is a schematic diagram of another environment of the robot during movement;

FIG. 3B is a schematic diagram of another environment of the robot during movement;

fig. 4 is a simplified schematic diagram of a robot obstacle avoidance system according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of the robot assembly provided herein;

fig. 6 is a schematic flowchart of a robot obstacle avoidance method according to an embodiment of the present disclosure;

fig. 7 is a schematic flowchart of another robot obstacle avoidance method according to an embodiment of the present disclosure;

FIG. 8 is a schematic diagram illustrating a composition of a dilemma recognition model according to an embodiment of the present disclosure;

fig. 9 is a schematic diagram of obstacle avoidance of a robot according to an embodiment of the present application;

fig. 10 is a simplified schematic diagram of an obstacle avoidance device according to an embodiment of the present application.

Detailed Description

The existing method for identifying the dilemma of the robot (such as a sweeping robot) needs to manually collect sample data, and the collected sample data needs to be manually labeled, so that the investment of large labor cost is brought. And the manual labeling process may also be subject to error. For example, the robot is taken as a sweeping robot. For example, when the sweeping robot is in the position shown in fig. 3A, the person can think that the sweeping robot is stuck in the gap between the fixed object and the ground and cannot move any more when viewing from the angle shown in fig. 3A, and at this time, the person can put a trapped label on the robot. However, the sweeping robot may not be blocked by the gap and can continue to work in the gap, and thus, a situation that label information of the sample data is wrongly labeled occurs. In addition, most of the currently collected sample data are environment pictures shot from the angle of the user, and the environment of the robot cannot be intuitively and accurately reflected. These all result in inaccuracies in the dilemma recognition model obtained by training.

The application provides a robot obstacle avoidance method, which has the following basic principle: the robot acquires sample information at N times (N is an integer greater than 0) during the movement of the robot, and sends the sample information at the N times to the server. The robot receives a distress recognition model from the server, wherein the distress recognition model is trained by sample information collected by one or more robots. The robot can avoid obstacles in the moving process according to the predicament recognition model.

The sample information corresponding to each of the N times includes environment image information of an environment in which the robot is located at the time, and position information of a position of the robot in the environment indicated by the environment image information at the time. The information is acquired by the robot, and the environment where the robot is located at the moment can be accurately identified. Meanwhile, each sample information further includes label information indicating whether the robot is in a trapped state at that time. Similarly, the label information is also the state mark which is recognized by the robot according to the label information, and the current state of the robot can be accurately reflected. In this way, by transmitting such sample information to the server for the server to train the dilemma recognition model, the accuracy of the dilemma recognition model can be improved. Meanwhile, the environment image information in the working process of the robot does not need to be collected manually, and the environment image information does not need to be labeled manually, so that a large amount of labor cost input is saved.

The robot obstacle avoidance method provided by the application is described in detail below with reference to the accompanying drawings.

Please refer to fig. 4, which is a simplified schematic diagram of a robot obstacle avoidance system according to an embodiment of the present disclosure. As shown in fig. 4, the robot obstacle avoidance system may include: a robot 401, and a server 402. It should be noted that the robot 401 may include one or more robots, such as the robot 1-the robot n shown in fig. 4.

Each of the robots 401 may be configured to acquire sample information at N times during its own movement, and send the acquired sample information at N times to the server 402 through wireless communication. Wherein, the sample information of each time may include: the environmental image information at the time, the position information at the time and the label information at the time. The environment image information indicates the environment in which the robot is located at that time. The position information indicates a position of the robot in the environment indicated by the environment image information at that time. The tag information is used to indicate the state of whether the robot is trapped at that time.

The server 402 may train the received sample information of the one or more robots according to the received sample information from the one or more robots, obtain the dilemma recognition model, and send the dilemma recognition model to each robot. So that the robot 401 can avoid obstacles in the moving process according to the received predicament recognition model.

For example, the process of obtaining the dilemma recognition model by the server 402 through the sample information can be implemented by the following methods: the server 402 may classify the obtained sample information into two categories, such as referred to as first type sample information and second type sample information. The first type of sample information may be used for training of the dilemma recognition model, and the second type of sample information may be used for detecting the trained dilemma recognition model to determine the accuracy of the trained dilemma recognition model. The process of the server 402 obtaining the dilemma recognition model is described in detail in the following embodiments, and is not described herein in detail.

The robot 401 may be a sweeping robot, a robot providing self-service in the service industry such as a bank, or other electronic devices capable of moving autonomously.

Please refer to fig. 5, which provides a schematic diagram of a robot 500. As shown in fig. 5, the robot 500 may include an image acquisition module 501, a sensor 502, a processor 503, a memory 504, and a communication module 505.

Among other things, the processor 503 may include one or more processing units, such as: the processor 503 may include an Application Processor (AP), a modem processor, a Graphics Processing Unit (GPU), an Image Signal Processor (ISP), a controller, a memory, a video codec, a Digital Signal Processor (DSP), a baseband processor, and/or a neural-Network Processing Unit (NPU), etc. The different processing units may be separate devices or may be integrated into one or more processors.

Among the controllers may be the neural center and the command center of the robot 500. The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of instruction fetching and instruction execution.

A memory may also be provided in the processor 503 for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. The memory may hold instructions or data that have just been used or recycled by the processor. If the processor needs to reuse the instruction or data, it can be called directly from the memory. Avoiding repeated accesses reduces the latency of the processor 503 and thus increases the efficiency of the system.

In some embodiments, the processor 503 may include one or more interfaces. The interface may include an integrated circuit (I2C) interface, a universal asynchronous receiver/transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a general-purpose input/output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and/or a Universal Serial Bus (USB) interface, etc.

The interface is a bi-directional synchronous serial bus comprising a serial data line (SDA) and a Serial Clock Line (SCL). In some embodiments, the processor may include multiple sets of I2C buses. The processor may be coupled to the sensor, camera, etc. through different I2C bus interfaces.

The UART interface is a universal serial data bus used for asynchronous communications. The bus may be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is typically used to connect the processor 503 and the communication module 505. For example: the processor 503 communicates with the bluetooth module in the wireless communication module through the UART interface to implement the bluetooth function.

The MIPI interface may be used to connect a processor with peripheral devices such as cameras. The MIPI interface includes a Camera Serial Interface (CSI) and the like. In some embodiments, the processor and camera communicate through a CSI interface to implement the camera functions of the robot 500.

The GPIO interface may be configured by software. The GPIO interface may be configured as a control signal and may also be configured as a data signal. In some embodiments, a GPIO interface may be used to connect the processor with a camera, wireless communication module, sensor module, or the like. The GPIO interface may also be configured as an I2C interface, a UART interface, a MIPI interface, and the like.

The USB interface is an interface which accords with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface and the like. The USB interface may be used to connect a charger to charge the robot 500, and may also be used to transmit data between the robot 500 and peripheral devices. The interface may also be used to connect other robots 500, etc.

The image capturing module 501 may capture image information of the periphery of the robot 500, for example, capturing a picture or capturing a video. The robot 500 may implement image capture functions via an ISP, a camera, a video codec, a GPU, and an application processor, among others.

The ISP is used for processing data fed back by the camera. For example, when a photo is taken, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converting into an image visible to naked eyes. The ISP can also perform algorithm optimization on noise, brightness, etc. of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In some embodiments, the ISP may be provided in a camera.

The camera is used to capture still images or video. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The light sensing element converts the optical signal into an electrical signal, which is then passed to the ISP where it is converted into a digital image signal. And the ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into image signal in standard RGB, YUV and other formats. In some embodiments, robot 500 may include 1 or N cameras, N being a positive integer greater than 1.

The sensor 502 may acquire information such as a moving speed, a moving direction, and a distance from a peripheral object of the robot 500. Illustratively, the sensors 502 may include a gyroscope sensor, a velocity sensor, an acceleration sensor, a distance sensor, and the like.

Among other things, a gyro sensor may be used to determine the motion pose of the robot 500. In some embodiments, the angular velocity of the robot 500 about three axes (i.e., x, y, and z axes) may be determined by a gyroscope sensor. The gyro sensor may be used for photographing anti-shake. Illustratively, when the robot 500 is performing image acquisition, the gyroscope sensor detects a shake angle of the robot 500, calculates a distance to be compensated for the lens module according to the shake angle, and allows the lens to counteract the shake of the robot 500 through reverse movement, thereby achieving anti-shake. The gyro sensor may also be used for navigation, determining whether the robot 500 is trapped in a scene, etc.

The speed sensor is used for measuring the moving speed. In some embodiments, the robot 500 measures the moving speed at the current time through a speed sensor, and may predict the environment in which the robot 500 is located at the next time according to the environment in which the robot 500 is located at the current time through a distance sensor.

The acceleration sensor may detect the magnitude of acceleration of the robot 500 in various directions (typically three axes). The magnitude and direction of gravity may be detected when the robot 500 is stationary.

A distance sensor for measuring a distance. The robot 500 may measure the distance by infrared or laser. In some embodiments, a scene is photographed, and robot 500 may utilize range sensor ranging to achieve fast focus.

The memory 504 may include an external memory as well as an internal memory. The external memory interface may be used to connect an external memory card, such as a Micro SD card, to extend the storage capability of the robot 500. The external memory card communicates with the processor through the external memory interface to realize the data storage function. For example, the sample information file is saved in an external memory card.

The internal memory may be used to store computer-executable program code, which includes instructions. The processor executes various functional applications of the robot 500 and data processing by executing instructions stored in the internal memory. The internal memory may include a program storage area and a data storage area. The storage program area can store an operating system and an application program required by at least one function. The storage data area may store data created during use of the robot 500. In addition, the internal memory may include a high-speed random access memory, and may further include a nonvolatile memory, such as at least one of a magnetic disk storage device, a flash memory device, a universal flash memory (UFS), and the like.

The wireless communication function of the robot 500 may be implemented by the communication module 505. For example, through the communication module 505, the robot 500 may enable communication with other devices, such as a server. As an example, the communication module 505 may include an antenna 1, an antenna 2, a mobile communication module, a wireless communication module, a modem processor, a baseband processor, and the like.

The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the robot 500 may be used to cover a single or multiple communication bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module may provide a solution including wireless communication of 2G/3G/4G/5G, etc. applied to the robot 500. The mobile communication module may include at least one filter, a switch, a power amplifier, a Low Noise Amplifier (LNA), and the like. The mobile communication module can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit the electromagnetic waves to the modem processor for demodulation. The mobile communication module can also amplify the signal modulated by the modulation and demodulation processor and convert the signal into electromagnetic wave to be radiated by the antenna 1. In some embodiments, at least part of the functional modules of the mobile communication module may be provided in the processor. In some embodiments, at least part of the functional modules of the mobile communication module may be provided in the same device as at least part of the modules of the processor.

The wireless communication module may provide a solution for wireless communication applied to the robot 500, including Wireless Local Area Networks (WLANs) (e.g., wireless fidelity (Wi-Fi) networks), Bluetooth (BT), Global Navigation Satellite System (GNSS), Infrared (IR), and the like. The wireless communication module may be one or more devices integrating at least one communication processing module. The wireless communication module receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on electromagnetic wave signals, and transmits the processed signals to the processor. The wireless communication module can also receive a signal to be sent from the processor, frequency-modulate and amplify the signal, and the signal is converted into electromagnetic waves through the antenna 2 and radiated out.

In some embodiments, antenna 1 of robot 500 is coupled to a mobile communication module and antenna 2 is coupled to a wireless communication module, such that robot 500 may communicate with servers and other devices via wireless communication techniques. The wireless communication technology may include global system for mobile communications (GSM), General Packet Radio Service (GPRS), code division multiple access (code division multiple access, CDMA), Wideband Code Division Multiple Access (WCDMA), time-division code division multiple access (time-division code division multiple access, TD-SCDMA), Long Term Evolution (LTE), LTE, BT, GNSS, WLAN, NFC, FM, and/or IR technologies, etc. The GNSS may include a Global Positioning System (GPS), a global navigation satellite system (GLONASS), a beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS), and/or a Satellite Based Augmentation System (SBAS).

It is to be understood that the structure illustrated in the present embodiment does not specifically limit the robot 500. In other embodiments, robot 500 may include more or fewer components than shown, or combine certain components, or split certain components, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The robot obstacle avoidance method provided by the embodiment of the application can be implemented in the system shown in fig. 4 and the robot shown in fig. 5.

Fig. 6 is a schematic flow chart of a robot obstacle avoidance method according to an embodiment of the present application. Referring to fig. 6, the method may include S601-S605.

S601, the robot acquires sample information of N moments in the moving process, wherein N is an integer larger than 0.

The sample information of each moment comprises environment image information used for indicating the environment where the robot is located at the moment, position information used for indicating the position of the robot in the environment indicated by the environment image information at the moment, and label information used for indicating the state of the robot at the moment, wherein the label information comprises a first label or a second label, the first label is used for indicating that the robot is in an unbolted state, and the second label is used for indicating that the robot is in a stranded state.

For example, the robot may acquire environment image information and position information of an environment where the robot is located through a module provided on the robot. Meanwhile, the robot can judge whether the robot is trapped or not according to the moving state at the current moment so as to obtain the label information at the moment. For example, when the robot cannot move continuously, it is determined that the robot is in the trapped state at the current moment, and accordingly, the tag information at the moment can be obtained as the second tag. Otherwise, the robot is determined to be in the state of not being trapped at the current moment, and accordingly the label information at the moment can be obtained to be the first label. Therefore, the robot can obtain the sample information of the robot at N moments in the moving process by acquiring the information in real time in the moving process. Wherein N is an integer greater than 0.

S602, the robot sends the sample information of M times in the N times to a server, wherein M is a positive integer less than or equal to N.

The robot may send all the acquired samples at N times to the server, that is, M equals to N. For example, the robot may send the sample information of a time to the server in real time after acquiring the sample information of the time, or may send the sample information of the N times to the server together.

It can be understood that the robot can acquire a large amount of sample information during the moving process, and some of the sample information has high quality, so that the robot can send some of the acquired sample information with high quality to the robot, that is, send the sample information at M times out of N times, where M is a positive integer smaller than N. For example, the robot may send sample information when the robot is trapped and sample information that the robot is not trapped before being trapped to the server. Therefore, the information quality can be improved on the premise of ensuring the sample information to be sufficient, and the system information load is reduced.

S603, the server trains sample information collected by one or more robots to obtain a predicament recognition model.

Because training of the predicament recognition model requires a large amount of sample information, in this embodiment, a plurality of robots may be capable of reporting sample information acquired by the robots to the server. Therefore, the server can perform model training according to the received sample information collected by the plurality of robots so as to obtain a more accurate predicament recognition model.

S604, the robot receives the predicament recognition model from the server.

The server may transmit the dilemma recognition model trained based on the sample information acquired by one or more robots to the robot in real time, or may transmit the dilemma recognition model to the robot at predetermined time intervals. The robot may receive the distress recognition model through a communication module and store in memory.

And S605, the robot carries out obstacle avoidance in the moving process according to the predicament recognition model.

In the moving process of the robot, the moving direction and the moving speed of the robot at the current moment can be obtained, and the environment image information and the position information of the environment where the robot is located at the next moment are predicted according to the environment image information and the position information of the current moment. So that the robot determines whether the robot is in a trapped state at the next moment or determines the probability of the robot being trapped at the next moment according to the predicament recognition model. Therefore, the robot can determine whether to avoid the obstacle at the next moment according to the determination result.

For example, in some embodiments, the environmental image information and the position information of the environment in which the robot is located at the next time are used as the input of the predicament recognition model, and it may be determined whether the robot is in the trapped state at the next time. When the robot determines that the robot is trapped at the next moment, the robot can avoid the obstacle at the current moment so as to avoid the trapping at the next moment. In other embodiments, the environmental image information and the position information of the environment where the robot is located at the next moment are used as the input of the predicament recognition model, so that the probability that the robot is trapped at the next moment can be determined. When the robot determines that the probability that the robot is trapped at the next moment is greater than the preset threshold value, the robot can perform obstacle avoidance operation.

According to the robot obstacle avoidance method provided by the embodiment of the application, the robot acquires the environment image information and the position information at N moments in the moving process of the robot, and the environment of the robot at each moment in the N moments can be accurately identified. Meanwhile, the robot autonomously judges whether the robot is in a trapped state or not, so that the environment of the corresponding moment is automatically labeled according to the judgment result, and whether the robot is trapped at the corresponding moment or not can be accurately reflected. The robot transmits the information to the server as sample information, and the server trains the predicament recognition model, so that the accuracy of the predicament recognition model can be improved. Meanwhile, the environment image information in the working process of the robot does not need to be collected manually, and the environment image information does not need to be labeled manually, so that a large amount of labor cost input is saved.

Please refer to fig. 7, which is another obstacle avoidance method for a robot according to an embodiment of the present disclosure. As shown in fig. 7, the method may include S701-S706.

S701, the robot acquires sample information of N moments in a moving process, wherein the sample information of each moment in the N moments comprises environment image information of the moment, position information of the moment and label information of the moment.

And the environment image information of the moment is used for indicating the environment of the robot at the moment. The position information at the time point is used to indicate the position of the robot in the environment indicated by the environment image information at the time point. The tag information at that time is used to indicate the state of the robot at that time. The tag information comprises a first tag or a second tag, the first tag is used for indicating that the robot is in an un-trapped state, and the second tag is used for indicating that the robot is in a trapped state.

Because the robot is constantly moving during work, the robot may be in different environments, different positions and different trapped states at different times. In order to enable the server to obtain enough sample information to train the distress recognition model, the robot can obtain the sample information of N moments in the moving process of the robot according to the environment, the position and whether the robot is in the trapped state in the moving process. Wherein N is an integer greater than 0.

For example, the following describes an acquisition process of sample information at a certain time, taking the robot as an example to acquire the sample information at the first time in the moving process. The first time is any one of the N times.

In the moving process, the robot acquires environment image information of the environment where the robot is located at the first moment.

For example, in conjunction with fig. 5, the robot 500 may acquire environment image information of an environment in which the robot 500 is located through an image acquisition module 501 including a camera, which is disposed on the robot 500.

In some embodiments, the environment image information may be a picture of the environment in which the robot is located taken by the robot at the first moment.

Generally, the robot records the working state or feeds back working information to a user by recording videos in the working process. Therefore, in some other embodiments, the robot may use the image of the first time in the recorded video as the environment image information of the environment where the robot is located at the first time. It will be appreciated that video may be decomposed into a plurality of temporally successive images. And a video corresponds to a playing time, so each image in a video may correspond to a time in the playing time. Then, each image corresponding to the time can be used as the environment image information of the environment where the robot is located at the time. For example, assume that the robot records a video including image a at time 1, image B at time 2, and image C at time 3. The robot can then use image a as the environment image information 1 of the environment in which the robot is located at the first moment. Similarly, image B is taken as environment image information B of the environment in which the robot is located at other times, such as the second time, and image C is taken as environment image information C of the environment in which the robot is located at other times, such as the third time.

It should be noted that, since the environment image information is directly acquired by the robot, the environment where the robot is located can be reflected more accurately. In addition, during the movement of the robot, since the object on or around the moving line of the robot has the highest possibility of obstructing the movement of the robot, the environment image information acquired by the robot may include the image information of the object on or around the moving line. Thus, the environment image information can provide more accurate reference for whether the robot is possibly trapped or not. The moving route of the robot may be determined according to the current time, such as the moving direction and the moving speed of the robot at the first time. In some scenarios, the moving route of the robot may be planned in advance, and then the robot may also obtain information of the moving route by querying the moving route planned in advance.

During the moving process, the robot acquires the position information of the robot in the environment at the first moment.

The position information of the robot in the environment at the first moment can be relative position information between the robot and other objects in the environment at the first moment. The environment in which the robot is located at the first time may refer to an environment indicated by the environment image information at the first time.

The environment in which the robot is located differs at different times, and therefore the distance between the robot and the same object also differs at different times. For example, in conjunction with fig. 5, the robot 500 may acquire relative position information between the robot and other objects in the environment where the robot is located at a first time by the sensor 502 disposed on the robot 500. For example, the relative position information may be distance information between the robot 500 and the above-described object. The distance information may include a distance between the robot and an object on or around the moving route at the first time.

Of course, in some embodiments, the robot 500 may also obtain, through the sensor 502, longitude and latitude information and altitude information or altitude information of the robot at the current time (e.g., the first time), which may also be used as the above-mentioned position information. Thus, the robot can more accurately determine the position of the robot at that time based on the position information. Then, when the robot is at another time (e.g., the second time), and the acquired position information is the same as the position information, it can be accurately determined that the environment in which the robot is located at the second time is identical to the environment in which the robot is located at the first time. The dilemma recognition model used as the robot obstacle avoidance guidance is obtained by training the information including the sample information collected by the robot at the first moment, so that the robot can more accurately judge the environment where the robot is located at the second moment through the dilemma recognition model (namely, judge whether the robot is trapped at the second moment).

In the moving process, the robot acquires label information which represents the state of the robot at the first moment. For example, the tag information may include the first tag or the second tag.

It is understood that the robot can accurately recognize whether or not it is trapped at the current time (e.g., the first time). For example, when the robot has a situation where the wheels are stuck, the robot is stuck around, the robot turns in a fixed place for a certain time, or the like, the robot may be considered to be in a stuck state. The trapped information acquired by the method can accurately identify the current state of the robot. Similarly, when the robot is not trapped, for example, the robot is not trapped at the first moment, the robot can accurately determine the current state. After the robot determines to be trapped or not trapped at the first moment, the tag information corresponding to the state where the robot is located at the first moment can be acquired. Then, we can consider that the robot automatically labels the environment image information (such as surrounding picture information) and the position information (such as distance information) collected by the robot at the first time.

After the environment image information at the first time, the position information at the first time and the label information at the first time are obtained, the robot can use the information as sample information at the first time. Similarly, by repeating the above process during the moving process, sample information of N moments during the moving process of the robot can be obtained.

S702, the robot sends the sample information of M times in the N times to a server, wherein M is a positive integer less than or equal to N.

In some embodiments, M may be equal to N. The robot may send the acquired sample information for N times to the server. Therefore, the server can be ensured to be trained according to enough sample information, and an accurate predicament recognition model is obtained.

It will be appreciated that the robot is not trapped most of the time during the movement, and therefore, the amount of exemplar information with the first label may be much greater than the amount of exemplar information with the second label. In addition, the sample information which is not trapped for a period of time before the robot is trapped has a high value, so in the embodiment of the application, the robot can screen the sample information at N moments and then send the sample information to the server. That is, the robot may select sample information of M times among the N times to transmit to the server. Wherein the robot is trapped at the Nth moment, and the robot is not trapped at M-1 moments before the Nth moment.

Thus, in other embodiments, M may be less than N. The robot may transmit sample information of M times having higher values among the N times to the server. Therefore, the data volume of communication between the robot and the server can be effectively reduced while the server can acquire an accurate predicament recognition model.

For example, the robot determines sample information at N times, where the sample information at the nth time includes tag information for indicating that the robot is in a trapped state at the nth time, that is, the sample information at the nth time may include a second tag identifying that the robot is in a trapped state at the nth time. And the robot sends the sample information of the first M-1 time of the Nth time and the sample information of the Nth time to the server.

For example, the memory of the robot may be a buffer memory (buffer), the robot may store the sample information acquired during the moving process in the buffer, and monitor whether the tag information in the sample information acquired at the current time is the second tag, and if the tag information in the sample information acquired at the current time is the second tag, the robot sends the sample information in the buffer to the server. If the label information in the sample information acquired at the current moment does not include the second label, the robot may continue to acquire the sample information and store the sample information in the buffer until the robot acquires the sample information with the second label. For example, assuming that the buffer of the robot can store 10 sample information at most, if the 10 th sample information in the buffer includes the second tag, the robot may send the 10 sample information in the buffer to the server together. If none of the 10 sample information stored in the buffer includes the sample information of the second label, the robot may continue to acquire the sample information, delete the sample information at the earliest time among the 10 sample information already in the buffer, store the new sample information in the buffer, and loop until the sample information including the second label appears.

Alternatively, the robot may empty the buffer after sending the sample information to the server.

When the robot and the server are in a normal communication state, the robot may transmit the acquired sample information to the server in various forms. For example, the robot may send multiple sample information together to the server. As another example, the robot may send the plurality of sample information to the server several times. As another example, the robot may send a plurality of sample information to the server at regular intervals.

When communication between the robot and the server is not possible, such as the robot going offline or the server going offline, the robot may store the sample information in a memory provided on the robot. After the robot resumes communication with the server, the robot may send the sample information stored in the memory to the server.

S703, the server trains sample information collected by one or more robots to obtain a predicament recognition model.

For example, the server may divide the acquired sample information into two parts (e.g., the first type sample information and the second type sample information). The first type of sample information can be used for training to obtain the predicament recognition model, and the second type of sample information can be used for detecting whether the detection result of the predicament recognition model is accurate or not.

For example, it is assumed that the sample information obtained by the server includes X first-type sample information and Y second-type sample information, where X and Y are both integers greater than 0, and X + Y ═ N. The method comprises the steps of taking environmental image information and position information included in sample information as input, training X first-class sample information, taking the probability that the robot is trapped under the current environment or the specific result of whether the robot is trapped or not as output, comparing the output result with label information included in the sample information, and performing cyclic training until the comparison result converges to an expected level so as to obtain an initial model. After the initial model is obtained by training according to the X first-type sample information, the environmental image information and the position information in each sample information of the Y second-type sample information can be input into the predicament recognition model to detect the initial model, so as to detect the accuracy of the predicament recognition model. For example, if the state of whether the robot indicated by the result obtained by inputting the environmental image information and the position information of a certain sample information in the second type of sample information into the initial model is trapped is the same as the state of whether the robot indicated by the label information in the second type of sample information is trapped, it is determined that the verification of the initial model for the sample information is accurate; and if the state of whether the robot indicated by the result obtained by inputting the environmental image information and the position information of certain sample information in the second type of sample information into the initial model is not the same as the state of whether the robot indicated by the label information in the second type of sample information is trapped, determining that the verification result for the sample information is inaccurate. Because the second type sample information can be a plurality of pieces, the accuracy of the initial model after each detection is integrated, and the verification accuracy of the initial model can be obtained. When the accuracy is greater than a preset threshold, the initial model can be used as a predicament identification model. When the accuracy is smaller than the preset threshold, the initial model is considered to be not accurate enough and needs to be trained continuously. The server can add new sample information from the robot into the training, and further train the initial model to obtain a model with higher accuracy. Thus, the server can determine a model with an accuracy greater than a preset threshold and send the model to the robot as a dilemma recognition model.

Illustratively, the training of the X first type sample information may be according to a loss function (loss function) or a cost function (cost function). A loss function (loss function) or cost function (cost function) is a function that maps a random event or its associated random variable values to non-negative real numbers to represent the "risk" or "loss" of the random event. In the embodiment of the present application, the solution can be performed by minimizing a loss function (cost function). When the minimum value of the loss function is solved, iterative solution can be performed step by step through a gradient descent method to obtain the minimized loss function and the model parameter value, so that the predicament identification model (initial model) can be obtained.

For example, the dilemma recognition model obtained by training may be a model based on a convolutional neural network. For example, as shown in FIG. 8, the dilemma identification model may include an input layer, a volume base layer, a pooling layer, a fully connected layer, and an output layer. In the figure, only two convolutional layers, two pooling layers, and one full-link layer are illustrated. The dilemma recognition model in the embodiment of the present application may include a plurality of convolutional layers, a plurality of pooling layers, and a plurality of fully-connected layers.

It should be noted that the greater the types and the number of the sample information are, the more accurate the dilemma recognition model obtained by training and learning is. Therefore, the sample information may be sample information acquired during one movement of one robot, sample information acquired during multiple movements of one robot, or sample information acquired during multiple movements of multiple robots. The embodiments of the present application are not limited thereto.

S704, the robot receives the predicament recognition model from the server.

The server trains and obtains the predicament recognition model according to a plurality of sample information sent by one or more robots, and issues the predicament recognition model to the robots. In conjunction with fig. 5, the robot 500 may receive the distress recognition model through a communication module 505 provided on the robot and store the distress recognition model in a memory 504. To avoid obstacles by using the stored predicament recognition model, the following steps S705-S706 are executed.

S705, the robot acquires the moving direction and the moving speed of the robot at the current moment.

During the moving process of the robot, the environment of the robot at the next moment can be predicted. For example, the robot may acquire the moving direction and the moving speed of the robot at the current time through a sensor provided on the robot. And obtaining the environmental image information and the position information at the current moment. The processor of the robot can acquire the environment image information of the environment where the robot is located at the next moment and the position information in the environment according to the moving direction and the moving speed of the robot at the current moment and the environment image information and the position information at the current moment.

Assume that the robot is at the position shown in (a) in fig. 9 at the present time, and the robot is moving in the direction 1. Then, second position information of the robot at the next time can be obtained based on the first position information of the robot at the current time, for example, information of the distance between the robot and the object 1 and information of the distance between the robot and the object 2, the moving direction, and the moving speed, and environment image information of the environment where the robot is located at the second time can be obtained by performing triangulation based on the position information and the environment image information of the robot at the current time, for example, as shown in (b) of fig. 9.

For example, at the current time, the robot acquires the environmental image information on its moving route as P1 (e.g., a picture of the environment where the robot is located) through the image acquisition module, and acquires the position information as S1 (e.g., the distance between the robot and the object in the environment) through the sensor. The robot can acquire the moving direction and the moving speed at the current moment through the sensor. Further, the positional information S2 at the next time can be calculated from the formula S1-S2 ═ t × V. Where S1 is the distance from the robot to the object ahead at the current time, S2 is the physical distance from the robot to the front at the next time, t is the time interval between the current time and the next time, and V is the moving speed of the robot at the current time.

In addition, the robot can also calculate and acquire the image information P2 of the environment where the robot is located at the next moment by utilizing triangulation according to the moving distance from the current moment to the next moment (e.g., S1-S2).

Thus, the robot acquires the environment image information P2 of the environment at the next time and the position information S2 of the robot in the environment at the next time.

And S706, predicting whether the robot is trapped at the next moment according to the environment image information, the position information and the predicament recognition model of the environment where the robot is located at the next moment, and avoiding the obstacle.

With reference to the above example in S705, the robot may obtain, through calculation, the probability that the robot is trapped at the next time, using the image information P2 of the environment where the robot is located at the next time and the position information S2 as inputs of the predicament recognition model. The robot can compare the trapped probability with a preset threshold, and when the robot determines that the trapped probability at the next moment is greater than the preset threshold, the robot can change the current motion strategy (such as the motion direction at the current moment) to avoid the obstacle, so that the situation that the robot is trapped in the moving process is avoided. When the robot determines that the probability of being trapped at the next moment is smaller than the preset threshold, the robot can be considered not to be trapped at the next moment, and then the robot can continue to execute the current motion strategy to move.

For example, referring to fig. 9, assuming that the robot is in the environment shown in (a) of fig. 9 at the current moment, when the robot determines that the robot will be trapped at the next moment according to the distress recognition model, the robot may change the original motion strategy at the moment, for example, changing the original movement in the direction 1 to the movement in the direction 2 shown in (c) of fig. 9, so as to avoid the robot being trapped at the next moment.

In addition, S706 is described as an example of the probability that the output of the distress recognition model is trapped at the next time. In the embodiment of the application, the predicament recognition model may further output a prediction result of the determination of whether the robot will be trapped at the next time, that is, the robot will be trapped at the next time or will not be trapped at the next time. When the robot determines that the robot is trapped at the next moment according to the predicament recognition model, the robot can change the current motion strategy (such as the motion direction at the current moment) to avoid the obstacle, so that the situation that the robot is trapped in the moving process is avoided. When the robot determines that the robot is not trapped at the next moment according to the predicament recognition model, the robot can continuously execute the current motion strategy to move.

In the obstacle avoidance method for the robot provided by the embodiment of the application, the process of the robot performing the above-mentioned S701-S706 is not a single pass, but can be cyclic. For example, the robot may continuously obtain sample information at different times and send the sample information to the server, so that the server obtains the updated dilemma recognition model through incremental training according to the updated sample information and the dilemma recognition model. The incremental training process is similar to the initial training process, except that in the incremental training process, the loaded model is the previously trained model, and the weight of the loaded model is also trained. And at the time of initial training, the weight value of the loaded model is a default value. After the incremental training is finished, the second type sample information detection model is used, and if the detected accuracy is higher than that in the prior art, the quality of the model subjected to the incremental training is high. At this time, the server can issue the newly trained model to the robot, so that the robot can recognize the model according to the updated predicament for guiding the following movement, and more effective obstacle avoidance is performed.

In this way, the robot can autonomously acquire sample information at N times, and the sample information corresponding to each of the N times includes environment image information of an environment in which the robot is located at the time and position information of a position of the robot in the environment indicated by the environment image information at the time. Because the information is acquired by the robot, the environment where the robot is located at the moment can be accurately identified. Meanwhile, each sample information further includes label information indicating whether the robot is in a trapped state at that time. Similarly, the label information is also the state mark which is recognized by the robot according to the label information, and the current state of the robot can be accurately reflected. The robot transmits the sample information to the server for the server to train the predicament recognition model, so that the accuracy of the predicament recognition model can be improved. Meanwhile, the environment image information in the working process of the robot does not need to be collected manually, and the environment image information does not need to be labeled manually, so that a large amount of labor cost input is saved. The server can also perform incremental training on the predicament recognition model according to the updated sample information to acquire a more accurate predicament recognition model and send the model to the robot, so that the robot can continuously acquire the sample information in the moving process and send the sample information to the server, and the updated predicament recognition model is acquired to perform more accurate obstacle avoidance guidance on the moving process of the robot.

The above description mainly introduces the solution provided by the embodiments of the present application from the perspective of a robot. It is understood that the robot includes hardware structures and/or software modules for performing the above functions, and the hardware structures and/or software modules for performing the functions may form one robot. Those of skill in the art will readily appreciate that the present application is capable of hardware or a combination of hardware and computer software implementing the various illustrative algorithm steps described in connection with the embodiments disclosed herein. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiment of the present application, the robot may be divided into the functional modules according to the method example, for example, the robot may include an obstacle avoidance device, the obstacle avoidance device may divide each functional module corresponding to each function, or may integrate two or more functions into one processing module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. It should be noted that, in the embodiment of the present application, the division of the module is schematic, and is only one logic function division, and there may be another division manner in actual implementation.

In the case of dividing each function module according to each function, fig. 10 shows a schematic diagram of a possible composition of the obstacle avoidance device in the above embodiment, as shown in fig. 10, the obstacle avoidance device includes: an acquisition unit 1001, a communication unit 1003 and an obstacle avoidance unit 1004. For example, the obtaining unit 1001 may also be the image capturing module 501 and the sensor 502 shown in fig. 5, the communication unit 1003 may also be the communication module 505 shown in fig. 5, and the obstacle avoidance unit 1004 may also be the processor 503 shown in fig. 5.

The obtaining unit 1001 is configured to obtain sample information of N times during a moving process of the robot, where N is an integer greater than 0. Wherein, the sample information of each moment in N moments comprises: environment image information indicating an environment in which the robot is located at the moment, position information indicating a position of the robot in the environment indicated by the environment image information at the moment, and tag information indicating a state of the robot at the moment; the tag information comprises a first tag or a second tag, the first tag is used for indicating that the robot is in an un-trapped state, and the second tag is used for indicating that the robot is in a trapped state. For example, the obtaining unit 1001 may be configured to execute S601 of the robot obstacle avoidance method shown in fig. 6. The obtaining unit 1001 may also be configured to execute S701 of the robot obstacle avoidance method shown in fig. 7.

A communication unit 1003, configured to send sample information of M times out of the N times to a server, where M is a positive integer less than or equal to N. Illustratively, the communication unit 1003 may be configured to execute S602 of the robot obstacle avoidance method shown in fig. 6. The communication unit 1003 may also be configured to execute S702 of the robot obstacle avoidance method shown in fig. 7.

The communication unit 1003 is further configured to receive a predicament recognition model from the server, where the predicament recognition model is trained by sample information collected by one or more robots. Illustratively, the communication unit 1003 may also be configured to execute S604 of the robot obstacle avoidance method shown in fig. 6. The communication unit 1003 may also be configured to execute S704 of the robot obstacle avoidance method shown in fig. 7.

And the obstacle avoidance unit 1004 is used for avoiding obstacles in the moving process of the robot according to the predicament recognition model. For example, the obstacle avoidance unit 1004 may be further configured to execute S605 of the robot obstacle avoidance method shown in fig. 6. The obstacle avoidance unit 1004 may also be configured to perform S706 of the robot obstacle avoidance method shown in fig. 7.

Further, the obstacle avoidance device may further include: a determination unit 1002.

The determining unit 1002 is configured to determine, in the sample information at the N times, that the sample information at the nth time includes tag information for indicating that the machine is in a trapped state at the nth time.

It should be noted that all relevant contents of each step related to the above method embodiment may be referred to the functional description of the corresponding functional module, and are not described herein again. The obstacle avoidance device provided by the embodiment of the application is used for executing the robot obstacle avoidance method, so that the same effect as the robot obstacle avoidance method can be achieved.

In the case of an integrated unit, another possible composition of the obstacle avoidance device in the above embodiments may include: the device comprises an acquisition module, a processing module and a communication module.

The acquisition module is used to acquire information needed for obstacle avoidance by the robot, e.g., the acquisition module is used to support the robot to perform S601 in fig. 6, S701, S705 in fig. 7, and/or other processes for the techniques described herein. The processing module is used to control and manage the actions of the robot, e.g., the processing module is used to support the robot to perform S605 in fig. 6, S706 in fig. 7, and/or other processes for the techniques described herein. The communication module is used to support communication between the robot and other network entities, for example, the server shown in fig. 6 or fig. 7, and other functional modules or network entities. The robot may also include a storage module for storing sample information, a dilemma identification model, and other program code and data for the robot.

The obstacle avoidance device provided by the embodiment of the application is used for executing the robot obstacle avoidance method, so that the same effect as the robot obstacle avoidance method can be achieved.

Through the above description of the embodiments, it is clear to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional modules is merely used as an example, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device may be divided into different functional modules to complete all or part of the above described functions.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, a module or a unit may be divided into only one logic function, and may be implemented in other ways, for example, a plurality of units or components may be combined or integrated into another apparatus, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed to a plurality of different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially or partially contributed to by the prior art, or all or part of the technical solutions may be embodied in the form of a software product, where the software product is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above is only an embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A robot obstacle avoidance method is characterized by comprising the following steps:

the robot acquires sample information of N moments in the moving process of the robot, wherein N is an integer greater than 0; wherein, the sample information of each moment in N moments comprises: environment image information indicating an environment in which the robot is located at the time, position information indicating a position of the robot in the environment indicated by the environment image information at the time, and tag information indicating a state of the robot at the time; the tag information comprises a first tag and a second tag, the first tag is used for indicating that the robot is in an un-trapped state, and the second tag is used for indicating that the robot is in a trapped state;

the robot sends sample information of M moments in the N moments to a server, wherein M is a positive integer less than or equal to N;

the robot receives a predicament recognition model from the server, wherein the predicament recognition model is obtained by training sample information collected by one or more robots;

and the robot avoids obstacles in the moving process according to the predicament recognition model.

2. The method of claim 1,

the environment image information at each time includes: and at the moment, the robot is in the environment, and the image information of objects on the moving route and around the moving route of the robot is obtained.

3. The method of claim 1,

the position information for each time includes: and in the environment indicated by the environment image information at that time, relative position information between the robot and an object on and around the moving route of the robot.

4. The method of any one of claims 1-3, wherein the robot sends sample information for M of the N moments to a server, comprising:

the robot determines that the machine is in a trapped state at the Nth moment in the sample information at the N moments, wherein the label information included in the sample information at the Nth moment is used for indicating that the machine is in the trapped state at the Nth moment;

and the robot sends the sample information of the first M-1 time of the Nth time and the sample information of the Nth time to the server.

5. The method according to any one of claims 1-3, wherein the robot performs obstacle avoidance during movement according to the dilemma recognition model, comprising:

the robot acquires environmental image information at the current moment and position information at the current moment;

the robot acquires environment image information of the next moment and position information of the next moment according to the environment image information of the current moment, the position information of the current moment, and the moving direction and the moving speed of the robot at the current moment;

the robot determines the probability of the robot being trapped at the next moment according to the environment image information at the next moment, the position information at the next moment and the predicament recognition model;

and the robot determines that the probability of the trapped robot at the next moment is greater than a preset threshold value, and the robot changes a motion strategy to avoid obstacles.

6. An obstacle avoidance device, for use with a robot, the device comprising: the system comprises an acquisition unit, a communication unit and an obstacle avoidance unit;

the acquisition unit is used for acquiring sample information of N moments in the moving process of the robot, wherein N is an integer greater than 0; wherein, the sample information of each moment in N moments comprises: environment image information indicating an environment in which the robot is located at the time, position information indicating a position of the robot in the environment indicated by the environment image information at the time, and tag information indicating a state of the robot at the time; the tag information comprises a first tag and a second tag, the first tag is used for indicating that the robot is in an un-trapped state, and the second tag is used for indicating that the robot is in a trapped state;

the communication unit is used for sending the sample information of M moments in the N moments to the server, wherein M is a positive integer less than or equal to N;

the communication unit is further used for receiving a predicament recognition model from the server, wherein the predicament recognition model is obtained by training sample information collected by one or more robots;

and the obstacle avoidance unit is used for avoiding obstacles in the moving process of the robot according to the dilemma identification model.

7. The apparatus of claim 6,

8. The apparatus of claim 6,

9. The apparatus according to any one of claims 6-8, further comprising: a determination unit;

the determining unit is configured to determine, in the sample information at the N times, that the sample information at the nth time includes tag information for indicating that the machine is in a trapped state at the nth time;

the communication unit is configured to send sample information of M times out of the N times to a server, and includes:

the communication unit is configured to send the sample information of the M-1 time before the nth time and the sample information of the nth time to the server.

10. The apparatus according to any one of claims 6-8,

the acquiring unit is further configured to acquire environmental image information at a current moment and position information at the current moment, and acquire environmental image information at a next moment and position information at the next moment according to the environmental image information at the current moment, the position information at the current moment, and a moving direction and a moving speed of the robot at the current moment;

the obstacle avoidance unit is used for avoiding obstacles according to the dilemma recognition model in the moving process of the robot, and comprises the following components:

and the obstacle avoidance unit is used for determining the probability of the robot being trapped at the next moment according to the environment image information at the next moment, the position information at the next moment and the predicament recognition model, determining that the probability of the robot being trapped at the next moment is greater than a preset threshold value, changing the robot motion strategy and avoiding obstacles.

11. A robot obstacle avoidance system, comprising: one or more robots, and a server;

the robot is used for acquiring sample information of N moments in the moving process of the robot and sending the sample information of the N moments to the server; wherein N is an integer greater than 0, and the sample information at each of the N times includes: environment image information indicating an environment in which the robot is located at the time, position information indicating a position of the robot in the environment indicated by the environment image information at the time, and tag information indicating a state of the robot at the time; the tag information comprises a first tag and a second tag, the first tag is used for indicating that the robot is in an un-trapped state, and the second tag is used for indicating that the robot is in a trapped state;

the server is used for receiving the sample information from the one or more robots, training the received sample information of the one or more robots, obtaining a predicament recognition model, and sending the predicament recognition model to the robots;

the robot is further used for receiving the predicament recognition model and avoiding obstacles in the moving process according to the predicament recognition model.

12. The system of claim 11, wherein the server receives the sample information comprising a first type of sample information and a second type of sample information;

the server is used for training the received sample information of the one or more robots to obtain a predicament recognition model, and the predicament recognition model comprises the following steps:

the server is used for training the first type of sample information to obtain an initial model, verifying the accuracy of the initial model according to the second type of sample information to obtain verification accuracy, determining the initial model as the predicament recognition model if the verification accuracy is greater than a preset threshold, receiving new sample information sent by the robot by the server if the verification accuracy is less than the preset threshold, continuing to train the new sample information and the initial model, verifying the accuracy of the model obtained by continuing to train according to the second type of sample information until the verification accuracy of the currently obtained model is greater than the preset threshold, and determining the model as the predicament recognition model;

wherein if the state of whether the robot is trapped, which is indicated by the result obtained by inputting the environmental image information and the position information of the sample information included in the second type of sample information into the initial model, is the same as the state of whether the robot is trapped, which is indicated by the label information in the sample information, it is determined that the initial model is accurate; if the state of whether the robot is trapped or not, which is indicated by the result obtained by inputting the environmental image information and the position information of the sample information included in the second type of sample information into the initial model, is different from the state of whether the robot is trapped or not, which is indicated by the label information in the second type of sample information, it is determined that the initial model is inaccurate; and determining the verification accuracy of the initial model according to the verification result of each sample information in the second type of sample information on the accuracy of the initial model.