JP6884312B1 - Robot obstacle avoidance processing method, device, robot - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a robot obstacle avoidance processing method, an apparatus and a robot. SOLUTION: A robot obstacle avoidance processing method acquires an environmental image collected by an image sensing module (S102), identifies an obstacle in the environmental image, and positions a positioning sensing module with respect to the obstacle. Confirmed (S104), based on the positioning feature information collected by the positioning sensing module, it is determined whether or not the obstacle is within the target obstacle area (S106), and if YES, the positioning feature information is confirmed. Failure avoidance processing is performed based on the position information (S108). [Selection diagram] Fig. 1

Description

The present invention relates to the field of robot technology, and more particularly to robot obstacle avoidance processing methods, devices and robots.

With the development of robot technology, the working environment of robots is becoming more and more complicated, and in such a complicated, changing and unpredictable working environment, the tasks that the robot should complete become more and more complicated, and at the same time, the user. The demand for robots is also constantly increasing. Robots that can sense the surroundings and make decisions are needed so that they can be applied to complex and changing work environments, and such robots build models with related algorithms based on the surrounding environmental parameters and intend to do so. It can be decided and finally the command issued by the user can be executed.

As for robots, the processing of obstacle avoidance plans is a very basic and extremely important task in the work process. Taking a robot in a storage warehouse environment as an example, when a robot executes an order task, it moves back and forth between warehouse display shelves according to a command. There are not only moving obstacles, but also fixed obstacles such as display shelves and walls. How quickly a decision can be made so that a robot can avoid a collision during the working process of the robot is an important point and a difficult point for the robot to execute an order task.

One or more embodiments of the present specification provide a robotic failure avoidance processing method. The robot obstacle avoidance processing method is
Acquiring the environmental image collected by the image sensing module
Identifying an obstacle in the environmental image and determining a positioning sensing module that positions the obstacle.
Based on the positioning feature information collected by the positioning sensing module, it is determined whether or not the obstacle is within the target obstacle area.
If YES, the failure avoidance process is included based on the position information determined by the positioning feature information.

Optionally, identifying obstacles in the environmental image above can be
This includes inputting the environmental image into an image identification model, performing feature division and feature identification on the environmental image by the image identification model, and outputting an obstacle type of an obstacle included in the environmental image.

Optionally, determining a positioning sensing module that positions the obstacle above
When the obstacle type is a fixed obstacle, it includes determining that the image sensing module is a positioning sensing module that positions the obstacle.
The image sensing module includes a monocular camera.
Correspondingly, it is possible to determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module.
The image coordinate position corresponding to the obstacle is determined based on the environment image, and the target image area corresponding to the preset reference image area in the environment image is determined.
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, the step of performing the failure avoidance process based on the position information determined by the above-mentioned positioning feature information is executed.
Optionally, determining a positioning sensing module that positions the obstacle above
When the obstacle type is a movement obstacle, it includes determining that the depth image sensor and the image sensing module are positioning sensing modules that position the obstacle.
Correspondingly, it is possible to determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module.
Based on the environmental image and the depth image collected by the depth image sensor, the image coordinate position corresponding to the obstacle in the depth image is determined.
Determining the target image area corresponding to the preset reference image area in the depth image, and
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, the step of performing the failure avoidance process based on the position information determined by the above-mentioned positioning feature information is executed.
Optionally, determining a positioning sensing module that positions the obstacle above
Determining the feature occupancy of the obstacle based on the environmental image
Judging whether or not the feature occupancy rate of the obstacle is larger than the predetermined feature occupancy threshold value,
If YES, confirm that the wireless sensor is a positioning sensing module that positions the obstacle.
If NO, it involves determining that the image sensing module is a positioning sensing module that positions the obstacle.
The image sensing module includes a monocular camera.

It is possible to optionally determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module.
Based on the position data of the obstacle collected by the wireless sensor, the physical coordinate position corresponding to the position data is determined.
The physical coordinate position includes determining whether or not the physical coordinate position is within a pre-installed physical obstacle area.
If YES, the step of performing the failure avoidance process based on the position information determined by the above-mentioned positioning feature information is executed.

It is possible to optionally determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module.
The image coordinate position corresponding to the obstacle is determined based on the environment image, and the target image area corresponding to the preset reference image area in the environment image is determined.
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, the step of performing the failure avoidance process based on the position information determined by the above-mentioned positioning feature information is executed.
Optionally, determining a positioning sensing module that positions the obstacle above
To calculate the initial distance corresponding to the obstacle based on the environmental image,
Based on the correspondence between the distance and the positioning sensing module established in advance, the first positioning sensing module corresponding to the initial distance is determined as the positioning sensing module for positioning the obstacle.

Arbitrarily, the robot obstacle avoidance processing method further
Based on the positioning feature information of the obstacle collected by the positioning sensing module corresponding to the distance, the second distance corresponding to the obstacle is calculated.
Regarding the correspondence between the distance and the positioning sensing module, it is determined whether or not the second positioning sensing module corresponding to the second distance matches the positioning sensing module corresponding to the initial distance.
If NO, this includes switching the positioning sensing module from the first positioning sensing module to the second positioning sensing module.

Arbitrarily, it is possible to perform the failure avoidance process based on the position information determined by the above-mentioned positioning feature information.
Generating and executing a driving command to decelerate,
It includes defining a traveling route based on the location information and updating the original traveling route based on the traveling route.
Arbitrarily, defining a travel route based on the above-mentioned position information is not possible.
Determining the maximum obstacle outline of the obstacle based on the environmental image,
It includes defining a traveling route based on the maximum obstacle outline and the position information so that the traveling width of the traveling route becomes larger than the numerical value of the maximum obstacle outline.
Arbitrarily, after executing the step of acquiring the environmental image collected by the image sensing module, and determining the positioning sensing module that identifies the obstacle in the environmental image and positions the obstacle. Before performing the steps to
Detecting the target object features included in the environmental image and
The target object feature further includes detecting whether or not it is an obstacle.
If YES, the step of identifying the obstacle in the environmental image and determining the positioning sensing module that positions the obstacle is performed.

One or more embodiments of the present specification further include.
An environmental image acquisition module configured to acquire environmental images collected by the image sensing module, and
An obstacle identification module configured to identify an obstacle in the environmental image and determine a positioning sensing module that positions the obstacle.
An obstacle determination module configured to determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning detection module is provided.
If YES, the robot fault avoidance processing device that operates the fault avoidance processing module configured to perform the fault avoidance processing based on the position information determined by the positioning feature information is provided.

One or more embodiments of the present specification further include.
Equipped with a processor, image sensing module and positioning sensing module
The image sensing module is configured to collect environmental images.
The processor acquires an environmental image collected by the image sensing module, identifies an obstacle in the environmental image, and the obstacle is in the target obstacle area based on the positioning feature information collected by the positioning sensing module. If YES, the failure avoidance process is performed based on the position information determined by the positioning feature information.
The positioning sensing module provides a robot that is determined by the processor and configured to position against the obstacle.

Optionally, identifying obstacles in the environmental image above can be
This includes inputting the environmental image into an image identification model, performing feature division and feature identification on the environmental image by the image identification model, and outputting an obstacle type of an obstacle included in the environmental image.

Optionally, if the obstacle type is a fixed obstacle, the image sensing module is determined to be the positioning sensing module.
The image sensing module includes a monocular camera.
Correspondingly, it is possible to determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module.
The image coordinate position corresponding to the obstacle is determined based on the environment image, and the target image area corresponding to the preset reference image area in the environment image is determined.
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, perform the next step.

Optionally, if the obstacle type is a movement obstacle, the depth image sensor and the image sensing module are determined to be the positioning sensing module.
Correspondingly, it is possible to determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module.
Based on the environmental image and the depth image collected by the depth image sensor, the image coordinate position corresponding to the obstacle in the depth image is determined.
Determining the target image area corresponding to the preset reference image area in the depth image, and
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, perform the next step.

Optionally, the processor
The feature occupancy rate of the obstacle is determined based on the environmental image, and the feature occupancy rate is determined.
It is determined whether or not the feature occupancy rate of the obstacle is larger than the predetermined feature occupancy threshold value.
If YES, it is determined that the wireless sensor is a positioning sensing module that positions the obstacle.
If NO, it is determined that the image sensing module is a positioning sensing module that positions the obstacle.
The positioning sensing module is confirmed by executing the operation, and the positioning sensing module is confirmed.
The image sensing module includes a monocular camera.

It is possible to optionally determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module.
Based on the position data of the obstacle collected by the wireless sensor, the physical coordinate position corresponding to the position data is determined.
The physical coordinate position includes determining whether or not the physical coordinate position is within a pre-installed physical obstacle area.
If YES, perform the next step.

It is possible to optionally determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module.
The image coordinate position corresponding to the obstacle is determined based on the environment image, and the target image area corresponding to the preset reference image area in the environment image is determined.
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, perform the next step.

Optionally, the processor
The initial distance corresponding to the obstacle is calculated based on the environmental image, and the initial distance is calculated.
Based on the pre-established correspondence between the distance and the positioning sensing module, the first positioning sensing module corresponding to the initial distance is determined as the positioning sensing module.
The positioning sensing module is confirmed by performing the operation.

Optionally, the processor further
Based on the positioning feature information of the obstacle collected by the positioning sensing module corresponding to the distance, the second distance corresponding to the obstacle is calculated.
In the correspondence relationship between the distance and the positioning sensing module, it is determined whether or not the second positioning sensing module corresponding to the second distance matches the positioning sensing module corresponding to the initial distance.
If NO, the positioning sensing module is configured to switch from the first positioning sensing module to the second positioning sensing module.

Arbitrarily, it is possible to perform the failure avoidance process based on the position information determined by the above-mentioned positioning feature information.
Generating and executing a driving command to decelerate,
It includes defining a traveling route based on the location information and updating the original traveling route based on the traveling route.

Arbitrarily, defining a travel route based on the above-mentioned position information is not possible.
Determining the maximum obstacle outline of the obstacle based on the environmental image,
It includes defining a traveling route based on the maximum obstacle outline and the position information so that the traveling width of the traveling route becomes larger than the numerical value of the maximum obstacle outline.

Optionally, the processor further
The target object feature included in the environmental image is detected and
Detecting whether or not the target object feature is an obstacle,
If YES, it is configured to proceed to the next step.

One or more embodiments of the present specification are further storage media for storing computer-executable commands.
When the computer-executable command is executed,
Acquires the environmental image collected by the image sensing module and
An obstacle in the environmental image is identified, and a positioning sensing module that positions the obstacle is determined.
Based on the positioning feature information collected by the positioning sensing module, it is determined whether or not the obstacle is within the target obstacle area.
If YES, failure avoidance processing is performed based on the position information determined by the positioning feature information.
A storage medium that realizes a flow is provided.

In order to more clearly explain the technical plan in one or more examples or the prior art of the present specification, the drawings necessary for the description of the embodiment or the prior art will be briefly introduced below. It goes without saying that the drawings described below are only a portion of the embodiments described herein, and those skilled in the art will be able to obtain yet other drawings based on these drawings without the need for creative labor. ..

FIG. 1 is a flowchart of a robot failure avoidance processing method processing provided by one or more embodiments of the present specification. FIG. 2 is a flowchart of a robot failure avoidance processing method processing applied to a storage warehouse scene provided by one or more embodiments of the present specification. FIG. 3 is a schematic view of a robotic failure avoidance processing device provided by one or more embodiments of the present specification. FIG. 4 is a schematic structural diagram of a robot provided by one or more embodiments of the present specification.

To help those skilled in the art better understand the technical schemes in one or more embodiments of the present specification, the following, together with the drawings in one or more embodiments of the present specification, may be combined with one or more of the present specification. The technical proposals in the plurality of embodiments will be clearly and completely described. Of course, the examples described are merely a part of the examples of the present specification, not all the examples. Based on one or more embodiments herein, it is understood that all other embodiments obtained by those skilled in the art without creative labor also fall within the scope of this document.

Examples of the robot failure avoidance processing method provided in the present specification are as follows:

FIG. 1 shows a flowchart of the robot failure avoidance processing method processing provided by this embodiment. FIG. 2 shows a flowchart of the robot failure avoidance processing method processing applied to the storage warehouse scene provided by this embodiment.

With reference to FIG. 1, the robot failure avoidance processing method provided by this embodiment specifically includes the following steps S102 to S108.

Step S102, Acquire the environmental image collected by the image sensing module.

The robot obstacle avoidance processing method provided in the present specification first collects an environmental image of the work environment in which the robot is located by an image sensing module, and when the obstacle included in the environmental image is identified, the type of the obstacle and the obstacle. Based on the height of the robot or the distance between the robot and the robot, the positioning sensing module that positions the robot is determined, and thus different positioning sensing modules are adopted for different obstacle situations to perform more accurate positioning. After positioning the obstacle, it is determined whether or not the obstacle is in the target obstacle area that affects the robot work, and if the obstacle is in the target obstacle area, the position information of the obstacle is obtained. To ensure that the robot's work is performed smoothly by performing obstacle avoidance processing on the robot.

The image sensor of this embodiment refers to an image sensor provided on the robot itself, and is, for example, a monocular camera provided on the robot. The image of the working environment in which the robot is collected by the image sensor is the environment image, and the purpose of collecting the environment image is to detect an obstacle in the robot traveling process. Therefore, the environment collected by the image sensor here. It can be understood that the image is an environmental image of the work environment area in front of the robot traveling route.

In addition, the image sensor can be a movable camera that collaborates with the robot in the work environment where the robot is present, or a plurality of fixed cameras that are provided at different positions in the robot work environment and cooperate with each other. Can be.

In practical applications, in order to improve the speed and accuracy of detection of obstacles in the robot, the method adopted is often to increase the frequency of the image sensor provided in the robot, and to collect higher frequency images. And while obstacle detection and positioning avoids the robot from colliding with obstacles in the work environment, higher frequency image collection and obstacle detection and positioning identification exerts processing pressure on the robot, provided by this embodiment. In the optional embodiment, in order to improve the work efficiency of the robot, after the image sensing module collects the environmental image, the target object feature included in the environmental image is first detected, and then the target object feature is further determined. Detects whether or not it is an obstacle, and if YES, executes step S104 below, and if NO, indicates that there is no obstacle in the environmental image collected by the image sensor installed in the current robot. In other words, it means that there are no obstacles in the current working environment of the robot that affect its running, and it is not necessary to process the robot.

Step S104, the obstacle in the environmental image is identified, and the positioning sensing module for positioning with respect to the obstacle is determined.

Obstacles in the environmental image are identified based on the environmental image collected by the image collecting module. Alternatively, when an obstacle included in the above environmental image is detected, the obstacle is identified. Specifically, in the identification process, the environmental image is input to the image identification model, feature division and feature identification are performed on the environmental image by the image identification model, and the obstacle type of the obstacle included in the environmental image. Is output.

For example, in order to improve the identification efficiency and the identification accuracy rate, an image identification model adopting the YOLO algorithm is trained, and obstacle identification is performed on the environmental image collected using the image identification model. In the image identification model, in the process of identifying obstacles for an environmental image, it is necessary to perform feature division by going back and forth before identification. That is, the obstacle feature is divided among the complex image features of the environment image, and the obstacle is identified by the obstacle feature acquired by the feature division. Similarly, from the angle of efficiency improvement, by adding a module that divides the features of the environment image to the image identification model, and using the obstacle features output by the module as an input for obstacle identification to be performed later. , Input the environment image into the image identification model, perform feature division and obstacle identification, and output and identify the acquired obstacle types such as human legs (sorting workers), robots, chargers, display shelves and walls. Can be realized.

The positioning sensing module of this embodiment refers to a sensor used for positioning in a sensor provided in the robot. For example, a monocular camera, a depth camera, and an infrared sensor provided in the robot. When positioning is performed based on the environmental image collected by the monocular camera in the positioning process, the monocular camera is determined to be the positioning sensing module. Similarly, when positioning is performed based on the depth image collected by the depth camera or the position data collected by the infrared sensor in the positioning process, the depth camera or the infrared sensor is determined to be the positioning sensing module.

In practical applications, the robot working environment may have various types of obstacles, such as moving obstacles such as sorting workers and other robots, and fixed obstacles such as display shelves and walls. Based on the different characteristics of different types of obstacles, in the optional embodiments provided by this embodiment, the appropriate positioning sensing module is determined based on the obstacle type. That is, positioning is performed on an obstacle by selecting an appropriate positioning method based on the obstacle type. In this way, the accuracy of obstacle positioning is improved.

Specifically, if the obstacle type output by the image identification model is a fixed obstacle, it is determined that the image sensing module is a positioning sensing module that positions the obstacle. For example, for fixed obstacles of the type such as display shelves, walls, etc., a monocular camera that still collects previous environmental images is adopted as a positioning sensing module that positions the obstacles. That is, the obstacle is positioned based on the environmental image collected by the monocular camera.

If the obstacle type is a movement obstacle, it is determined that the depth image sensor and the image sensing module are positioning sensing modules that position the obstacle. For example, for a type of moving obstacle such as a sorting worker or another robot, positioning is performed on the obstacle by simultaneously adopting a depth camera and a monocular camera provided in the robot. That is, the depth image collected by the depth camera and the environment image collected by the monocular camera are combined to position the obstacle.
When specifically implemented, considering that the height of obstacles has a certain effect on the identification and positioning of obstacles in many scenes, for example, in the storage warehouse scene, the height of the sorting robot is low and relative. In addition, the field of view of the image sensor provided in the sorting robot may also be low, and in this situation, in the optional embodiment provided by this embodiment, appropriate positioning is performed based on the height of the obstacle. The sensing module is fixed and different positioning methods are adopted for obstacles of different heights, thus improving the accuracy of obstacle positioning. Specifically, it is realized as follows:
The feature occupancy rate of the obstacle is determined based on the environmental image, and the feature occupancy rate is determined.
It is determined whether or not the feature occupancy rate of the obstacle is larger than the predetermined feature occupancy threshold value.
If YES, it is determined that the wireless sensor is a positioning sensing module that positions the obstacle.
If NO, it is determined that the image sensing module is a positioning sensing module that positions the obstacle.

For example, in the environment image of the work environment collected by the sorting robot in the storage warehouse scene, when the height occupancy rate of the image feature of a certain obstacle (display shelf) in the environment image exceeds 1/2, the front of the storage warehouse robot. The height of the obstacle called the display shelf is high. In such a case, it will be more accurate if it is positioned with respect to an obstacle such as a display shelf by a wireless detection and ranging sensor (radar), an ultrasonic wave sensor or an infrared sensor. If the height occupancy of a certain obstacle (other sorting robot) is less than 1/2, it means that the height of the obstacle called the sorting robot in front of the storage warehouse robot is low, and the monocular camera still installed in the robot. Positioning may be performed using.

In addition to this, when concretely implemented, considering that the distance between the obstacle and the robot also has a certain influence on the identification and positioning of the obstacle in many scenes, in this situation, this embodiment In an optional embodiment provided by, a suitable positioning method is selected based on the distance of the obstacle, thus increasing the accuracy of the positioning of the obstacle. Specifically, the initial distance corresponding to the obstacle is calculated based on the environmental image, and then the first positioning sensing module corresponding to the initial distance is referred to based on the correspondence relationship between the distance established in advance and the positioning sensing module. Determined as a positioning sensing module that positions obstacles.

For example, when the distance between the obstacle and the robot is greater than 6 m, the corresponding positioning sensing module is a ranged detection sensor such as a wireless detection and ranging sensor (radar), an ultrasonic wave sensor or an infrared sensor. When the distance between the obstacle and the robot is less than 6m, the corresponding positioning sensing module is a depth camera. In such a case, not only the depth image collected by the depth camera is used for positioning the obstacle, but also the depth image and the environment image collected by the monocular camera are combined to position the obstacle with higher accuracy. be able to.

It should be noted that there is a possibility that the accuracy of positioning the obstacle by adopting different positioning sensing modules at different stages of performing the obstacle avoidance process for the same obstacle may be higher, and in the work running process of the robot. In order to further improve the positioning accuracy, the present embodiment further provides the following positioning sensing module switching operation. Specifically, based on the positioning feature information of the obstacle collected by the positioning sensing module corresponding to the distance, the second distance corresponding to the obstacle is calculated, and based on this, the distance corresponds to the positioning sensing module. In relation to this, the second positioning sensing module corresponding to the second distance determines whether the initial distance matches the corresponding positioning sensing module, and if YES, it does not need to be processed, and if NO, the positioning The sensing module is switched from the first positioning sensing module to the second positioning sensing module.

Step S106, based on the positioning feature information collected by the positioning sensing module, it is determined whether or not the obstacle is within the target obstacle area.

Based on the determination of the positioning sensing module in step S104 above, here, based on the positioning feature information collected by the determined positioning sensing module, it is determined whether or not the obstacle is within the target obstacle area, and if. If YES, step S108 is executed, and the obstacle avoidance process is performed based on the position information in which the positioning feature information is determined. If NO, it indicates that the obstacle is not within the target obstacle area of the robot, and the process may not be performed.

The positioning feature information collected by the positioning sensing module can be the positioning feature information collected by the determined positioning sensor after the positioning sensor is confirmed. For example, when the determined positioning sensing module is a depth camera, positioning is performed. The positioning feature information collected by the sensing module is a depth image collected by the depth camera. For example, when the determined positioning sensing module is a monocular camera, the positioning feature information collected by the positioning sensing module is the environment collected by the monocular camera. The environmental image becomes an image, and the environmental image may be an environmental image collected before the monocular camera is determined to be the positioning sensing module, or may be an environmental image newly collected after the monocular camera is determined to be the positioning sensing module.

The target obstacle area of the present embodiment refers to a specific area in front of the robot traveling in spatial latitude, and for example, a rectangular area with a width of 0.7 m and a length of 1.2 m in front of the robot can be designated as the target obstacle area. In addition, the target obstacle area can further include a specific area in the image at image latitude. For example, a rectangular area or a fan-shaped area in an environmental image collected by a monocular camera can be designated as a target obstacle area, or a rectangular area or a fan-shaped area in a depth image collected by a depth camera can be designated as a target obstacle area.

Corresponding to the implementation method of determining the positioning detection module based on the above obstacle type, if the obstacle type output by the image identification model is a fixed obstacle, the image detection module is applied to the obstacle. After confirming that it is a positioning detection module that performs positioning, it is determined whether or not an obstacle is within the target obstacle area by the following method:
The image coordinate position corresponding to the obstacle is determined based on the environment image, and the target image area corresponding to the preset reference image area in the environment image is determined.
It is determined whether or not the image coordinate position is within the target image area, and the image coordinate position is determined.
If YES, the following step S108 is executed, and the failure avoidance process is performed based on the position information in which the positioning feature information is confirmed.
If NO, it means that the obstacle is not within the robot's target obstacle area and should not be processed.

Similar to this, if the obstacle type output by the image identification model is a moving obstacle, after determining that the image sensing module is a positioning sensing module that positions the obstacle, the following method is used. To determine if an obstacle is within the target obstacle area:
The image coordinate position corresponding to the obstacle is determined based on the environment image, and the target image area corresponding to the preset reference image area in the environment image is determined.
After that, it is determined whether or not the image coordinate position is within the target image area, and the image coordinate position is determined.
If YES, the following step S108 is executed, and the failure avoidance process is performed based on the position information in which the positioning feature information is confirmed.
If NO, it means that the obstacle is not within the robot's target obstacle area and should not be processed.

In addition, corresponding to the realization method of determining the appropriate positioning sensing module based on the height of the obstacle, if the determined positioning sensing module is a wireless sensor, the obstacle is the target obstacle by the following method. Determine if it is in the area:
Based on the position data of the obstacle collected by the wireless sensor, the physical coordinate position corresponding to the position data is determined.
It is determined whether or not the physical coordinate position is within the pre-installed physical obstacle area, and the physical coordinate position is determined.
If YES, the following step S108 is executed, and the failure avoidance process is performed based on the position information in which the positioning feature information is confirmed.
If NO, it means that the obstacle is not within the robot's target obstacle area and should not be processed.

Similarly, if the above-determined positioning sensing module is an image sensor, the following method is used to determine whether an obstacle is within the target obstacle area:
The image coordinate position corresponding to the obstacle is determined based on the environment image, and the target image area corresponding to the preset reference image area in the environment image is determined.
It is determined whether or not the image coordinate position is within the target image area, and the image coordinate position is determined.
If YES, the following step S108 is executed, and the failure avoidance process is performed based on the position information in which the positioning feature information is confirmed.
If NO, it means that the obstacle is not within the robot's target obstacle area and should not be processed.

In step S108, the failure avoidance process is performed based on the position information in which the positioning feature information is determined.

The premise that this step is carried out is that the determination result of determining whether or not the obstacle is within the target obstacle area in step S106 is YES, and specifically, the position information in which the positioning feature information is determined. In the process of performing obstacle avoidance processing based on, first, a traveling command for decelerating travel is generated and executed, then a traveling route is defined based on the position information, and the original traveling route is updated based on the traveling route. is there.

In actual application, there are various complicated situations in the robot work scene, and the volume of cargo placed on the sorting robot may be larger than the volume of the sorting robot, taking the storage warehouse scene as an example. Although there is no collision with obstacles between the sorting robots, the cargo placed on the sorting robots can cause collisions or collisions with obstacles. In response to such a situation, in the traveling route defining process, in the traveling route defining process, the maximum obstacle outline of the obstacle is first determined based on the environmental image, and the traveling width of the traveling route is larger than the maximum obstacle outline value. Therefore, the travel route is defined based on the maximum obstacle outline and the position information. In this way, the probability of colliding with an obstacle in the traveling process of the descending robot is reduced, and the robot's ability to respond to complicated scenes is enhanced.

The robot failure avoidance processing method provided by the present embodiment will be further described below by taking as an example the application of the robot failure avoidance processing method provided by the present embodiment in the storage warehouse scene. With reference to FIG. 2, the robot failure avoidance processing method applied to the storage warehouse scene specifically includes steps S202 to S226.

In step S202, the environmental image collected by the monocular camera provided in the sorting robot is acquired.

In addition, the monocular camera can be a movable camera that works in collaboration with the sorting robot in a work environment with a sorting robot, or a plurality of fixed cameras that are provided at different positions in the sorting robot working environment and cooperate with each other. It can be a camera.
Step S204, the target object feature included in the environmental image is detected.

In step S206, the target object feature detects whether it is an obstacle.

If the detection result of the target object feature indicates that it is an obstacle, perform step S208 below, and vice versa, the obstacle is in the environmental image collected by the monocular camera installed in the current sorting robot. It means that it does not exist, that is, the sorting robot means that there are no obstacles in the current working environment that affect its running, and it does not have to be processed.

Step S208, the environmental image is input to the image identification model, feature division and feature identification are performed, and the obstacle type of the obstacle included in the environmental image is output.

Step S210, if the obstacle type is a fixed obstacle, determine that it is a positioning sensing module that positions the monocular camera with respect to the obstacle.

Step S212, the spatial coordinate position corresponding to the obstacle is determined based on the environment image.

Step S214, determines whether the spatial coordinate position is within the spatial obstacle area, if YES, execute step S224, and if NO, do not process.

Step S216, if the obstacle type is a moving obstacle, determine that both the depth camera and the monocular camera are positioning sensing modules that position the obstacle.

Step S218, Based on the environmental image and the depth image collected by the depth camera, determine the corresponding image coordinate position in the fusion image of the obstacle. Among them, the fused image can be acquired after being fused by the environment image and the depth image.

Step S220, the target image area corresponding to the preset reference image area in the fused image is determined.

In step S222, it is determined whether or not the image coordinate position is within the target image area. If YES, step S224 is executed, and if NO, the process may not be performed.

Step S224, generate and execute a running command to decelerate.

Step S226, the travel route is defined based on the position information, and the route is updated for the sorting robot.

Examples of the robot failure avoidance processing device provided in the present specification are as follows:
In the above embodiment, the robot failure avoidance processing method is provided. Correspondingly, a robot failure avoidance processing device is further provided. It will be described below together with the drawings.

FIG. 3 shows a schematic view of the robot failure avoidance processing device provided by this embodiment.

Since the device embodiment corresponds to the method embodiment, it will be described relatively briefly. For the related part, the corresponding description of the method embodiment provided above can be referred to. The device embodiments described below are merely schematic.

This embodiment provides a robot failure avoidance processing device.

An environmental image acquisition module 302 configured to acquire environmental images collected by the image sensing module, and
An obstacle identification module 304 configured to identify an obstacle in the environmental image and determine a positioning sensing module that positions the obstacle.
An obstacle determination module 306 configured to determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module is provided.
If YES, the failure avoidance processing module 308 configured to perform the failure avoidance processing based on the position information in which the positioning feature information is determined is operated.

Optionally, the obstacle identification module 304
The environment image is input to the image identification model, the environment image is feature-divided and feature-identified by the image identification model, and the obstacle type of the obstacle included in the environment image is output. Includes an identifier module.

Optionally, the obstacle identification module 304
Includes a first determiner module configured to determine that the image sensing module is a positioning sensing module that positions the obstacle when the obstacle type is a fixed obstacle.
The image sensing module includes a monocular camera and
Correspondingly, the obstacle determination module 306 specifically determines the image coordinate position corresponding to the obstacle based on the environment image, and corresponds to the reference image area set in advance in the environment image. The target image area to be used is determined, and it is configured to determine whether or not the image coordinate position is within the target image area.
If YES, the failure avoidance processing module 308 is operated.

Optionally, the obstacle identification module 304
When the obstacle type is a movement obstacle, a second determinant module configured to determine that the depth image sensor and the image sensing module are positioning sensing modules that position the obstacle. Including
Correspondingly, the obstacle determination module 306 is specifically based on the environment image and the depth image collected by the depth image sensor, and the obstacle is at the corresponding image coordinate position in the depth image. Is determined, the target image area corresponding to the preset reference image area in the depth image is determined, and it is configured to determine whether or not the image coordinate position is within the target image area.
If YES, the failure avoidance processing module 308 is operated.

Optionally, the obstacle identification module 304
A feature occupancy determinant module configured to determine the feature occupancy of the obstacle based on the environmental image.
Includes a feature occupancy determiner module configured to determine if the feature occupancy of the obstacle is greater than a predetermined feature occupancy threshold.
If YES, it is determined that the wireless sensor is a positioning sensing module that positions the obstacle.
If NO, it is determined that the image sensing module is a positioning sensing module that positions the obstacle.
The image sensing module includes a monocular camera.

Arbitrarily, the obstacle determination module 306 determines the physical coordinate position corresponding to the position data based on the position data of the obstacle collected by the wireless sensor, and the physical coordinate position is set to the physical coordinate position. It is configured to determine if it is within a pre-installed physical failure area,
If YES, the failure avoidance processing module 308 is operated.

Optionally, the obstacle determination module 306 specifically determines the image coordinate position corresponding to the obstacle based on the environment image, and corresponds to a preset reference image area in the environment image. It is configured to determine the target image area and determine whether the image coordinate position is within the target image area.
If YES, the failure avoidance processing module 308 is operated.

Optionally, the obstacle identification module 304
An initial distance calculator module configured to calculate the corresponding initial distance of the obstacle based on the environmental image.
Based on the correspondence between the distance and the positioning sensing module established in advance, the first positioning sensing module configured to determine the first positioning sensing module corresponding to the initial distance as the positioning sensing module that positions the obstacle. Includes a sensing module determinant module and.

Optionally, the robot fault avoidance processing device
A second distance determination module configured to calculate the second distance corresponding to the obstacle based on the positioning feature information of the obstacle collected by the positioning sensing module corresponding to the distance.
In the correspondence relationship between the distance and the positioning sensing module, the second positioning sensing module corresponding to the second distance is configured to determine whether or not the initial distance matches the corresponding positioning sensing module. With more modules,
If NO, the positioning sensing module is switched from the first positioning sensing module to the second positioning sensing module.

Optionally, the failure avoidance processing module 308
A travel command executor module configured to generate and execute travel commands for decelerating travel,
It includes a travel route regulator module configured to define a travel route based on the location information and update the original travel route based on the travel route.

Optionally, the travel route regulator module
A maximum obstacle outline determination unit configured to determine the maximum obstacle outline of the obstacle based on the environmental image,
The traveling route defining unit is provided so as to define a traveling route based on the maximum obstacle outline and the position information so that the traveling width of the traveling route becomes larger than the maximum obstacle outline numerical value.

Optionally, the robot fault avoidance processing device
A target object feature detection module configured to detect a target object feature included in the environmental image, and a target object feature detection module.
The target object feature further comprises an obstacle detection module configured to detect whether or not it is an obstacle.
If YES, run the obstacle identification module 304.

Examples of robots provided herein are:
Corresponding to the robotic failure avoidance processing method described above, based on the same technical concept, one or more embodiments of the present specification further provide robots. The robot is used to execute the above-mentioned robot obstacle avoidance processing method. FIG. 4 is a schematic structural diagram of the robot provided by one or more embodiments of the present specification.

The robot provided by this embodiment is
Equipped with processor 401, image sensing module 402 and positioning sensing module 403
The image sensing module 402 is configured to collect environmental images.
The processor 401 acquires an environmental image collected by the image sensing module 402, identifies an obstacle in the environmental image, and based on the positioning feature information collected by the positioning sensing module 403, the obstacle is a target obstacle area. Judge whether it is inside,
If YES, the fault avoidance process is configured to be performed based on the position information in which the positioning feature information is fixed.
The positioning sensing module 403 was determined by the processor 401 and configured to position against the obstacle.

Optionally, identifying obstacles in the environmental image above can be
This includes inputting the environmental image into an image identification model, performing feature division and feature identification on the environmental image by the image identification model, and outputting an obstacle type of an obstacle included in the environmental image.

Optionally, if the obstacle type is a fixed obstacle, the image sensing module 402 is determined to be the positioning sensing module 403.
The image sensing module 402 includes a monocular camera.
Correspondingly, it is possible to determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module 403.
The image coordinate position corresponding to the obstacle is determined based on the environment image, and the target image area corresponding to the preset reference image area in the environment image is determined.
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, do the following:

Optionally, if the obstacle type is a moving obstacle, the depth image sensor and the image sensing module 402 are determined to be the positioning sensing module 403.
Correspondingly, it is possible to determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module 403.
Based on the environmental image and the depth image collected by the depth image sensor, it is determined that the obstacle is at the corresponding image coordinate position in the depth image.
A target image area corresponding to a preset reference image area in the depth image is determined, and the target image area is determined.
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, do the following:

Optionally, the processor
The feature occupancy rate of the obstacle is determined based on the environmental image, and the feature occupancy rate is determined.
It is determined whether or not the feature occupancy rate of the obstacle is larger than the predetermined feature occupancy threshold value.
If YES, the wireless sensor is determined to be the positioning sensing module 403,
If NO, the image sensing module 402 is determined to be the positioning sensing module 403.
The positioning sensing module 403 is confirmed by executing the operation, and the positioning sensing module 403 is confirmed.
The image sensing module 402 includes a monocular camera.

It is possible to optionally determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module 403.
Based on the position data of the obstacle collected by the wireless sensor, the physical coordinate position corresponding to the position data is determined.
The physical coordinate position includes determining whether or not the physical coordinate position is within a pre-installed physical obstacle area.
If YES, do the following:

It is possible to optionally determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module 403.
The image coordinate position corresponding to the obstacle is determined based on the environment image, and the target image area corresponding to the preset reference image area in the environment image is determined.
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, do the following:

Optionally, the processor
Based on the environmental image, the initial distance corresponding to the obstacle is calculated.
Based on the correspondence between the distance and the positioning sensing module established in advance, the first positioning sensing module corresponding to the initial distance is determined as the positioning sensing module 403.
The positioning sensing module 403 is confirmed by executing the operation.

Optionally, the processor 401 further
Based on the positioning feature information of the obstacle collected by the positioning sensing module corresponding to the distance, the second distance corresponding to the obstacle is calculated.
In the correspondence relationship between the distance and the positioning sensing module, the second positioning sensing module corresponding to the second distance determines whether or not the initial distance matches the corresponding positioning sensing module.
If NO, the positioning sensing module 403 is configured to switch from the first positioning sensing module to the second positioning sensing module.

Arbitrarily, it is possible to perform the failure avoidance process based on the position information in which the above-mentioned positioning feature information is determined.
Generate and execute a running command to decelerate,
It includes defining a traveling route based on the location information and updating the original traveling route based on the traveling route.

Arbitrarily, defining a travel route based on the above-mentioned position information is not possible.
Based on the environmental image, the maximum obstacle outline of the obstacle is determined.
It includes defining a traveling route based on the maximum obstacle outline and the position information so that the traveling width of the traveling route becomes larger than the maximum obstacle outline numerical value.

Optionally, the processor further
The target object feature included in the environmental image is detected and
Detecting whether or not the target object feature is an obstacle,
If YES, it is configured to proceed.

Examples of storage media provided herein are:
Corresponding to the robotic failure avoidance processing method described above, based on the same technical concept, one or more embodiments of the present specification further provide a storage medium.

The storage medium provided by this embodiment is used to store a computer-executable command, and the computer-executable command is executed when the computer-executable command is executed.
Acquires the environmental image collected by the image sensing module and
An obstacle in the environmental image is identified, and a positioning sensing module that positions the obstacle is determined.
Based on the positioning feature information collected by the positioning sensing module, it is determined whether or not the obstacle is within the target obstacle area.
If YES, a flow is realized in which the failure avoidance process is performed based on the position information in which the positioning feature information is determined.

Optionally, identifying obstacles in the environmental image above can be
This includes inputting the environmental image into an image identification model, performing feature division and feature identification on the environmental image by the image identification model, and outputting an obstacle type of an obstacle included in the environmental image.

Optionally, determining a positioning sensing module that positions the obstacle above
If the obstacle type is a fixed obstacle, it includes determining that the image sensing module is a positioning sensing module that positions the obstacle.
The image sensing module includes a monocular camera and
Correspondingly, it is possible to determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module.
The image coordinate position corresponding to the obstacle is determined based on the environment image, and the target image area corresponding to the preset reference image area in the environment image is determined.
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, the flow of performing the failure avoidance process is executed based on the position information in which the above-mentioned positioning feature information is confirmed.

Optionally, determining a positioning sensing module that positions the obstacle above
If the obstacle type is a movement obstacle, it includes determining that the depth image sensor and the image sensing module are positioning sensing modules that position the obstacle.
Correspondingly, it is possible to determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module.
Based on the environmental image and the depth image collected by the depth image sensor, it is determined that the obstacle is at the corresponding image coordinate position in the depth image.
A target image area corresponding to a preset reference image area in the depth image is determined, and the target image area is determined.
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, the flow of performing the failure avoidance process is executed based on the position information in which the above-mentioned positioning feature information is confirmed.

Optionally, determining a positioning sensing module that positions the obstacle above
The feature occupancy rate of the obstacle is determined based on the environmental image, and the feature occupancy rate is determined.
It is determined whether or not the feature occupancy rate of the obstacle is larger than the predetermined feature occupancy threshold value.
If YES, it is determined that the wireless sensor is a positioning sensing module that positions the obstacle.
If NO, it involves determining that the image sensing module is a positioning sensing module that positions the obstacle.
The image sensing module includes a monocular camera.

It is possible to optionally determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module.
Based on the position data of the obstacle collected by the wireless sensor, the physical coordinate position corresponding to the position data is determined.
The physical coordinate position includes determining whether or not the physical coordinate position is within a pre-installed physical obstacle area.
If YES, the flow of performing the failure avoidance process is executed based on the position information in which the above-mentioned positioning feature information is confirmed.

It is possible to optionally determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module.
The image coordinate position corresponding to the obstacle is determined based on the environment image, and the target image area corresponding to the preset reference image area in the environment image is determined.
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, the flow of performing the failure avoidance process is executed based on the position information in which the above-mentioned positioning feature information is confirmed.

Optionally, determining a positioning sensing module that positions the obstacle above
Based on the environmental image, the initial distance corresponding to the obstacle is calculated.
Based on the correspondence between the distance and the positioning sensing module established in advance, the first positioning sensing module corresponding to the initial distance is determined as the positioning sensing module that positions the obstacle.

Optionally, the computer-executable command further, when executed,
Based on the positioning feature information of the obstacle collected by the positioning sensing module corresponding to the distance, the second distance corresponding to the obstacle is calculated.
In the correspondence relationship between the distance and the positioning sensing module, the second positioning sensing module corresponding to the second distance determines whether or not the initial distance matches the corresponding positioning sensing module.
If NO, the positioning sensing module is switched from the first positioning sensing module to the second positioning sensing module.
Realize the flow.

Arbitrarily, it is possible to perform the failure avoidance process based on the position information in which the above-mentioned positioning feature information is determined.
Generate and execute a running command to decelerate,
It includes defining a traveling route based on the location information and updating the original traveling route based on the traveling route.

Arbitrarily, defining a travel route based on the above-mentioned position information is not possible.
Based on the environmental image, the maximum obstacle outline of the obstacle is determined.
It includes defining a traveling route based on the maximum obstacle outline and the position information so that the traveling width of the traveling route becomes larger than the maximum obstacle outline numerical value.

Optionally, the computer-executable command further, when executed,
The target object feature included in the environmental image is detected and
Detecting whether or not the target object feature is an obstacle,
If YES, identify the obstacle in the above environmental image and determine the positioning sensing module that positions the obstacle.
Execute the flow.

It should be noted that the examples relating to the storage medium in the present specification and the examples relating to the robot failure avoidance processing method in the present specification are based on the same concept of the invention. Therefore, since the specific implementation of the embodiment can refer to the implementation of the above-mentioned corresponding method, the repeated description will be omitted.
The specific examples of the present specification have been described above. Other embodiments fall within the scope of the accompanying claims. In some cases, the actions or steps described in the claims can be performed in a different order than in the examples, and the desired result can be achieved. In addition, the processes illustrated in the drawings do not necessarily mean that the desired results cannot be achieved unless they are in a specific order or a continuous order. In some embodiments, multitasking and parallel processing may be possible, or these may be beneficial.

In the 1930s, with respect to one technological improvement, the most obvious is a hardware improvement (eg, an improvement in the circuit structure of a diode, a transistor, a switch, etc.) or a software improvement (for a method flow). Improvement). However, with the development of technology, many current method flow improvements can already be regarded as direct improvements to the hardware circuit structure. The designer can obtain a suitable hardware circuit structure by programming a nearly improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of one method flow cannot be realized by the hardware entity module. For example, a programmable logic device (PLD) (eg, Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logical function is for the user to program the device. Is confirmed by. Designers can "integrate" a single digital system into a single PLD by programming themselves, without having to ask the chip manufacturer to design and produce a dedicated integrated circuit chip. Also, nowadays, instead of manual manufacturing of integrated circuit chips, many such programs are realized by replacing them with "logic compiler" software, which is similar to the software compiler used when creating program development. , The original code before compiling must also be written in a specific programming language, and here it is called Hardware Description Language (HDL), but there are many types of HDL, not just one type. Yes, for example ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby) Hardware Description Language), etc., and the most popular ones at present are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. It is clear to those skilled in the art that a hardware circuit that realizes the logical method flow can be easily obtained by slightly logically programming the method flow in some of the above-mentioned hardware description languages and programming it into an integrated circuit.

The controller can be implemented in any suitable manner, eg, a computer-readable medium, logical gate that stores a microprocessor or processor and computer-readable program code (eg, software or firmware) that can be executed by the (micro) processor. , Switches, Application Specific Integrated Circuits (ASICs), programmable logic controllers and embedded microcontrollers can be adopted, and examples of controllers are the following microprocessors: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone. Memory controllers may also be implemented as part of memory control logic, including but not limited to Labs C8051F320. Other than implementing the controller with a pure computer readable program code method, those skilled in the art can logically program the method steps to make the controller a logic gate, switch, dedicated integrated circuit, programmable logic controller, embedded microcontroller, etc. I also understand that the same functionality can be fully achieved by the format of. Therefore, such a controller can be regarded as a kind of hardware member, and a device for realizing each function included in the controller can also be regarded as a structure in the hardware member. Alternatively, the device for realizing each function can be regarded as both a software module for realizing the method and a structure in the hardware member.

The system, device, module or unit described in the above embodiment can be specifically realized by a computer chip or an entity, or can be realized by a product having a certain function, and one typical realization device is a computer. Specifically, computers include, for example, parsol computers, laptop computers, cellular phones, videophones, smartphones, personal digital assistants, media players, navigation devices, email devices, game consoles, tablet computers, wearable devices, or these devices. It can be any combination of devices in.

For the sake of clarity, when the above devices are described, they will be described separately for each unit according to the function. Of course, when implementing the embodiments herein, the functionality of each unit can be implemented in the same or more software and / or hardware.

It will be apparent to those skilled in the art that one or more embodiments of the specification can be provided as a method, system or computer program product. Thus, one or more embodiments herein may adopt the form of a complete hardware embodiment, a complete software embodiment, or a combination of software and hardware embodiments. In addition, the present specification is carried out on one or more computer-usable storage media (including, but not limited to, magnetic disk memory, CD-ROM, optical memory, etc.) including computer-usable program code. The format of the computer program product to be used can be adopted.

This specification is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products based on the embodiments of the present specification. It can be understood that computer program commands realize each flow and / or block in the flowchart and / or block diagram, and a combination of the flow and / or block in the flowchart and / or block diagram. These computer program commands can be provided to the processor of a general purpose computer, dedicated computer, embedded processor or other programmable data processing device as if manufacturing one device, thereby processing the computer or other programmable data. The command executed by the processor of the device manufactures a device for realizing the function specified by one flow or a plurality of flows and / or a block diagram in one block or a plurality of blocks.

These computer program commands may be stored in a computer-readable memory that can lead the computer or other programmable data processing device to operate in a particular manner, so that the commands stored in the computer-readable memory can be stored. A manufactured product including a command device is manufactured, and the command device realizes a function specified by one flow or a plurality of flows in a flowchart and / or a block or a plurality of blocks in a block diagram.

These computer program commands can also be loaded into a computer or other programmable data processing device, thereby causing a series of operational steps on the computer or other programmable device so that the processing achieved by the computer occurs. And thus the command executed on a computer or other programmable device implements the function specified in one or more flows and / or blocks in the diagram. Provides steps for.

In one typical arrangement, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces and internal memory.

Internal memory may include forms such as non-permanent memory, random access memory (RAM) and / or non-volatile internal memory in computer readable media, such as read-only memory (ROM) or flash memory (flash RAM). is there. Internal memory is an example of a computer-readable medium.

Computer-readable media include permanent and non-permanent, mobile and non-movable media, and information storage can be achieved by any method or technique. The information can be computer readable directives, data structures, program modules or other data. Examples of computer storage media include phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), Rewritable Read Only Memory (EEPROM), Flash ROM or Other Internal Memory Technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc Disc (DVD) or Other Optical Memory, Cassette Magnetic Tape, Magnetic Tape These include, but are not limited to, magnetic disk memory or other magnetic storage devices or any other non-transmission medium, which can be used to store information accessed by a computer. As defined in the text, computer readable media does not include transient computer readable media, such as modulated data signals and carrier waves.

It should be noted that the terms "provide", "include" or any other alternative term mean to cover non-exclusive inclusion, and processes, methods, goods or equipment containing a series of elements are not limited to those elements. , Other elements not explicitly listed, or elements specific to these processes, methods, goods or equipment. In the absence of more restrictions, an element limited by the term "contains one ..." does not preclude the existence of yet another same element in a process, method, commodity or device that includes said element.

One or more examples herein can be described in the general context of computer-executable commands executed by a computer. For example, a program module. In general, a program module includes routines, programs, objects, assemblies, data structures, etc. that perform specific tasks or realize specific abstract data. Further, one or more examples of the specification may be practiced in the distributed calculation environment, and in these distributed calculation environments, the task can be executed by the remote processing device connected by the communication network. In a distributed computing environment, program modules can be located on local and remote computer storage media, including storage devices.

Each of the examples in the present application is described by the cumulative method, and the same similar parts may be referred to each other among the examples, and the emphasis on each example is the same as that of the other examples. It's a different part. In particular, as for the system embodiment, since the basics are similar to the method embodiment, the explanation is relatively simple, and the related parts may refer to the partial explanation of the method embodiment.

The above contents are not for limiting the present application, but merely for examples of the present application. For those skilled in the art, this application may have various modifications and variations. Any amendments, equal replacements, improvements, etc. made within the ideas and principles of the present application are all within the scope of the claims of the present application.

Claims (24)

Acquiring the environmental image collected by the image sensing module
Identifying an obstacle in the environmental image and determining a positioning sensing module that positions the obstacle.
Based on the positioning feature information collected by the positioning sensing module, it is determined whether or not the obstacle is within the target obstacle area.
If YES if, viewed contains a and performing the failure avoidance processing based on the determined position information by the positioning characteristic information,
Determining the positioning sensing module that positions the obstacle
Determining the feature occupancy of the obstacle based on the environmental image
Judging whether or not the feature occupancy rate of the obstacle is larger than the predetermined feature occupancy threshold value,
If YES, confirm that the wireless sensor is a positioning sensing module that positions the obstacle.
If NO, it involves determining that the image sensing module is a positioning sensing module that positions the obstacle.
The image sensing module includes a monocular camera.
A robot obstacle avoidance processing method characterized by this. Identifying obstacles in the environmental image
This includes inputting the environmental image into an image identification model, performing feature division and feature identification on the environmental image by the image identification model, and outputting an obstacle type of an obstacle included in the environmental image.
The robot failure avoidance processing method according to claim 1. Determining the positioning sensing module that positions the obstacle
When the obstacle type is a fixed obstacle, it includes determining that the image sensing module is a positioning sensing module that positions the obstacle.
The image sensing module includes a monocular camera.
Correspondingly, it is possible to determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module.
The image coordinate position corresponding to the obstacle is determined based on the environment image, and the target image area corresponding to the preset reference image area in the environment image is determined.
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, the step of performing the failure avoidance process based on the position information determined by the positioning feature information is executed.
The robot failure avoidance processing method according to claim 2. Determining the positioning sensing module that positions the obstacle
When the obstacle type is a movement obstacle, it includes determining that the depth image sensor and the image sensing module are positioning sensing modules that position the obstacle.
Correspondingly, it is possible to determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module.
Based on the environmental image and the depth image collected by the depth image sensor, the image coordinate position corresponding to the obstacle in the depth image is determined.
Determining the target image area corresponding to the preset reference image area in the depth image, and
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, the step of performing the failure avoidance process based on the position information determined by the positioning feature information is executed.
The robot failure avoidance processing method according to claim 2. Based on the positioning feature information collected by the positioning sensing module, it is possible to determine whether or not the obstacle is within the target obstacle area.
Based on the position data of the obstacle collected by the wireless sensor, the physical coordinate position corresponding to the position data is determined.
The physical coordinate position includes determining whether or not the physical coordinate position is within a pre-installed physical obstacle area.
If YES, the step of performing the failure avoidance process based on the position information determined by the positioning feature information is executed.
The robot failure avoidance processing method according to claim 1. Based on the positioning feature information collected by the positioning sensing module, it is possible to determine whether or not the obstacle is within the target obstacle area.
The image coordinate position corresponding to the obstacle is determined based on the environment image, and the target image area corresponding to the preset reference image area in the environment image is determined.
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, the step of performing the failure avoidance process based on the position information determined by the positioning feature information is executed.
The robot failure avoidance processing method according to claim 1. Determining the positioning sensing module that positions the obstacle
To calculate the initial distance corresponding to the obstacle based on the environmental image,
Based on the correspondence between the distance and the positioning sensing module established in advance, the first positioning sensing module corresponding to the initial distance is determined as the positioning sensing module for positioning the obstacle.
The robot failure avoidance processing method according to claim 1. Based on the positioning feature information of the obstacle collected by the positioning sensing module corresponding to the distance, the second distance corresponding to the obstacle is calculated.
Regarding the correspondence between the distance and the positioning sensing module, it is determined whether or not the second positioning sensing module corresponding to the second distance matches the positioning sensing module corresponding to the initial distance.
If NO, further comprises switching the positioning sensing module from the first positioning sensing module to the second positioning sensing module.
The robot failure avoidance processing method according to claim 7. Performing failure avoidance processing based on the position information determined by the positioning feature information is not possible.
Generating and executing a driving command to decelerate,
Including defining a travel route based on the location information and updating the original travel route based on the travel route.
The robot failure avoidance processing method according to claim 1. Defining a travel route based on the location information
Determining the maximum obstacle outline of the obstacle based on the environmental image,
Including defining a travel route based on the maximum obstacle outline and the position information so that the travel width of the travel route is larger than the numerical value of the maximum obstacle outline.
The robot failure avoidance processing method according to claim 9. After executing the step of acquiring the environmental image collected by the image sensing module, and before executing the step of identifying the obstacle in the environmental image and determining the positioning sensing module that positions the obstacle.
Detecting the target object features included in the environmental image and
The target object feature further includes detecting whether or not it is an obstacle.
If YES, perform the step of identifying an obstacle in the environmental image and determining a positioning sensing module that positions the obstacle.
The robot failure avoidance processing method according to claim 1. An environmental image acquisition module configured to acquire environmental images collected by the image sensing module, and
An obstacle identification module configured to identify an obstacle in the environmental image and determine a positioning sensing module that positions the obstacle.
An obstacle determination module configured to determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning detection module is provided.
If YES, the failure avoidance processing module configured to perform the failure avoidance processing based on the position information determined by the positioning feature information is operated .
Determining the positioning sensing module that positions the obstacle
Determining the feature occupancy of the obstacle based on the environmental image
Judging whether or not the feature occupancy rate of the obstacle is larger than the predetermined feature occupancy threshold value,
If YES, confirm that the wireless sensor is a positioning sensing module that positions the obstacle.
If NO, it involves determining that the image sensing module is a positioning sensing module that positions the obstacle.
The image sensing module includes a monocular camera.
A robot obstacle avoidance processing device characterized by this. Equipped with a processor, image sensing module and positioning sensing module
The image sensing module is configured to collect environmental images.
The processor acquires an environmental image collected by the image sensing module, identifies an obstacle in the environmental image, and the obstacle is in the target obstacle area based on the positioning feature information collected by the positioning sensing module. If YES, the failure avoidance process is performed based on the position information determined by the positioning feature information.
The positioning sensing module is determined by the processor and is configured to position against the obstacle .
The processor
The feature occupancy rate of the obstacle is determined based on the environmental image, and the feature occupancy rate is determined.
It is determined whether or not the feature occupancy rate of the obstacle is larger than the predetermined feature occupancy threshold value.
If YES, the wireless sensor is confirmed to be the positioning sensing module.
If NO, the image sensing module is determined to be the positioning sensing module.
The positioning sensing module is confirmed by executing the operation, and the positioning sensing module is confirmed.
The image sensing module includes a monocular camera.
A robot characterized by that. Identifying obstacles in the environmental image
This includes inputting the environmental image into an image identification model, performing feature division and feature identification on the environmental image by the image identification model, and outputting an obstacle type of an obstacle included in the environmental image.
13. The robot according to claim 13. When the obstacle type is a fixed obstacle, the image sensing module is determined to be the positioning sensing module, and the image sensing module is determined to be the positioning sensing module.
The image sensing module includes a monocular camera.
Correspondingly, it is possible to determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module.
The image coordinate position corresponding to the obstacle is determined based on the environment image, and the target image area corresponding to the preset reference image area in the environment image is determined.
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, take the next step,
14. The robot according to claim 14. When the obstacle type is a movement obstacle, the depth image sensor and the image sensing module are determined to be the positioning sensing module.
Correspondingly, it is possible to determine whether or not the obstacle is within the target obstacle area based on the positioning feature information collected by the positioning sensing module.
Based on the environmental image and the depth image collected by the depth image sensor, the image coordinate position corresponding to the obstacle in the depth image is determined.
Determining the target image area corresponding to the preset reference image area in the depth image, and
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, take the next step,
14. The robot according to claim 14. Based on the positioning feature information collected by the positioning sensing module, it is possible to determine whether or not the obstacle is within the target obstacle area.
Based on the position data of the obstacle collected by the wireless sensor, the physical coordinate position corresponding to the position data is determined.
The physical coordinate position includes determining whether or not the physical coordinate position is within a pre-installed physical obstacle area.
If YES, take the next step,
16. The robot according to claim 16. Based on the positioning feature information collected by the positioning sensing module, it is possible to determine whether or not the obstacle is within the target obstacle area.
The image coordinate position corresponding to the obstacle is determined based on the environment image, and the target image area corresponding to the preset reference image area in the environment image is determined.
The image coordinate position includes determining whether or not the image coordinate position is within the target image area.
If YES, take the next step,
16. The robot according to claim 16. The processor
The initial distance corresponding to the obstacle is calculated based on the environmental image, and the initial distance is calculated.
Based on the pre-established correspondence between the distance and the positioning sensing module, the first positioning sensing module corresponding to the initial distance is determined as the positioning sensing module.
The positioning sensing module is confirmed by performing the operation.
13. The robot according to claim 13. The processor further
Based on the positioning feature information of the obstacle collected by the positioning sensing module corresponding to the distance, the second distance corresponding to the obstacle is calculated.
In the correspondence relationship between the distance and the positioning sensing module, it is determined whether or not the second positioning sensing module corresponding to the second distance matches the positioning sensing module corresponding to the initial distance.
If NO, the positioning sensing module is configured to switch from the first positioning sensing module to the second positioning sensing module.
The robot according to claim 19. Performing failure avoidance processing based on the position information determined by the positioning feature information is not possible.
Generating and executing a driving command to decelerate,
Including defining a travel route based on the location information and updating the original travel route based on the travel route.
13. The robot according to claim 13. Defining a travel route based on the location information
Determining the maximum obstacle outline of the obstacle based on the environmental image,
Including defining a travel route based on the maximum obstacle outline and the position information so that the travel width of the travel route is larger than the numerical value of the maximum obstacle outline.
21. The robot according to claim 21. The processor further
The target object feature included in the environmental image is detected and
Detecting whether or not the target object feature is an obstacle,
If YES, configured to proceed to the next step,
13. The robot according to claim 13. A storage medium for storing computer-executable commands.
When the computer-executable command is executed,
Acquires the environmental image collected by the image sensing module and
An obstacle in the environmental image is identified, and a positioning sensing module that positions the obstacle is determined.
Based on the positioning feature information collected by the positioning sensing module, it is determined whether or not the obstacle is within the target obstacle area.
If YES, the failure avoidance process is executed based on the position information determined by the positioning feature information.
The feature occupancy rate of the obstacle is determined based on the environmental image, and the feature occupancy rate is determined.
It is determined whether or not the feature occupancy rate of the obstacle is larger than the predetermined feature occupancy threshold value.
If YES, the wireless sensor is confirmed to be the positioning sensing module.
If NO, the image sensing module is determined to be the positioning sensing module.
The positioning sensing module is confirmed by performing the operation.
Realize the flow ,
The image sensing module includes a monocular camera.
A storage medium characterized by that.

Images