WO2021249568A1 - 仓储机器人的控制方法、装置、设备及可读存储介质 - Google Patents

仓储机器人的控制方法、装置、设备及可读存储介质 Download PDF

Info

Publication number
WO2021249568A1
WO2021249568A1 PCT/CN2021/102865 CN2021102865W WO2021249568A1 WO 2021249568 A1 WO2021249568 A1 WO 2021249568A1 CN 2021102865 W CN2021102865 W CN 2021102865W WO 2021249568 A1 WO2021249568 A1 WO 2021249568A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
task
storage location
location
handling
Prior art date
Application number
PCT/CN2021/102865
Other languages
English (en)
French (fr)
Inventor
李汇祥
郑睿群
陈宇奇
Original Assignee
深圳市海柔创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市海柔创新科技有限公司 filed Critical 深圳市海柔创新科技有限公司
Priority to JP2022576011A priority Critical patent/JP2023531391A/ja
Publication of WO2021249568A1 publication Critical patent/WO2021249568A1/zh
Priority to US18/064,609 priority patent/US20230106134A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G05D1/0248—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means in combination with a laser
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/20—Instruments for performing navigational calculations
    • G01C21/206—Instruments for performing navigational calculations specially adapted for indoor navigation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G1/00—Storing articles, individually or in orderly arrangement, in warehouses or magazines
    • B65G1/02—Storage devices
    • B65G1/04—Storage devices mechanical
    • B65G1/137—Storage devices mechanical with arrangements or automatic control means for selecting which articles are to be removed
    • B65G1/1373—Storage devices mechanical with arrangements or automatic control means for selecting which articles are to be removed for fulfilling orders in warehouses
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G2203/00—Indexing code relating to control or detection of the articles or the load carriers during conveying
    • B65G2203/02—Control or detection
    • B65G2203/0208—Control or detection relating to the transported articles
    • B65G2203/0233—Position of the article
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G2203/00—Indexing code relating to control or detection of the articles or the load carriers during conveying
    • B65G2203/04—Detection means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G2203/00—Indexing code relating to control or detection of the articles or the load carriers during conveying
    • B65G2203/04—Detection means
    • B65G2203/041—Camera
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G2203/00—Indexing code relating to control or detection of the articles or the load carriers during conveying
    • B65G2203/04—Detection means
    • B65G2203/042—Sensors
    • B65G2203/044—Optical
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds

Definitions

  • the present invention relates to the technical field of intelligent storage, in particular to a control method, device, equipment and readable storage medium of a storage robot.
  • warehousing logistics has a very important position in the process of enterprise generation management.
  • intelligent warehousing it is becoming more and more common for warehousing robots to replace workers to carry goods.
  • the material box may shift in the storage location or fall from the shelf.
  • the storage robot may It will collide with the material box. Therefore, there are potential safety hazards when the storage robot accesses the bins.
  • the invention provides a control method, device, equipment and readable storage medium of a storage robot, which are used to solve the problem of low safety of the storage robot.
  • One aspect of the present invention is to provide a control method of a storage robot, the storage robot having a handling device and an image acquisition device, including:
  • the collecting image data of the target storage location corresponding to the transportation task by the image collecting device includes:
  • the image acquisition device When the storage robot moves to the target location corresponding to the target storage location, control the image acquisition device to start and collect the image data of the target storage location; or, when the storage robot moves to the target storage location When the surrounding area is within a preset range, the image acquisition device is controlled to start and collect the image data of the target storage location.
  • the image acquisition device is disposed on the transport device, and before controlling the image acquisition device to start and collect the image data of the target storage location, the method further includes:
  • controlling the transportation device to perform the transportation task includes:
  • the status information of the target storage location includes at least one of the following:
  • Obstacle information on the transport path of the target storage location is free.
  • the state information of the target object includes at least one of the following:
  • the identity information of the target object The identity information of the target object; the posture information of the target object; the size information of the target object; the damage degree information of the target object; the deformation degree information of the target object.
  • the handling task is a pickup task
  • the execution condition of the handling task includes at least one of the following:
  • the identity information, posture information and size information of the target object meet the pick-up conditions; the damage degree of the target object is within the first preset safety threshold range; The deformation degree of the target object is within the second preset safety threshold range.
  • the handling task is a delivery task
  • the execution condition of the handling task includes at least one of the following:
  • the target storage location is free; the size of the target storage location satisfies the delivery condition; there is no obstacle on the delivery path of the target storage location.
  • an error message is sent to the server, wherein the error information includes at least one of the following: status information of the target location, target The status information of the object and the execution condition items that are not met.
  • the method further includes:
  • the storage robot is controlled to perform corresponding error handling behaviors.
  • the error handling behavior is any one of the following:
  • collecting image data of the target storage location by the image acquisition device includes at least one of the following:
  • the method before the collecting image data of the target storage location corresponding to the transportation task by the image collecting device, the method further includes:
  • the storage robot In response to the execution instruction of the handling task, the storage robot is controlled to move to the target location.
  • Another aspect of the present invention is to provide a control device for a storage robot, which is applied to a storage robot.
  • the storage robot includes a handling device and an image acquisition device, including:
  • a data acquisition module for acquiring image data of a target storage location corresponding to a transportation task through the image acquisition device
  • the control module is configured to control the transport device to execute the transport task if it is determined that the execution condition of the transport task is satisfied according to the image data of the target storage location.
  • the data acquisition module is further used for:
  • the image acquisition device When the storage robot moves to the target location corresponding to the target storage location, control the image acquisition device to start and collect the image data of the target storage location; or, when the storage robot moves to the target storage location When the surrounding area is within a preset range, the image acquisition device is controlled to start and collect the image data of the target storage location.
  • the image acquisition device is provided on the transport device, and the control module is further used for:
  • control module is further used to:
  • the status information of the target storage location includes at least one of the following:
  • Obstacle information on the transport path of the target storage location is free.
  • the state information of the target object includes at least one of the following:
  • the identity information of the target object The identity information of the target object; the posture information of the target object; the size information of the target object; the damage degree information of the target object; the deformation degree information of the target object.
  • the handling task is a pickup task
  • the execution condition of the handling task includes at least one of the following:
  • the identity information, posture information and size information of the target object meet the pick-up conditions; the damage degree of the target object is within the first preset safety threshold range; The deformation degree of the target object is within the second preset safety threshold range.
  • the handling task is a delivery task
  • the execution condition of the handling task includes at least one of the following:
  • the target storage location is free; the size of the target storage location satisfies the delivery condition; there is no obstacle on the delivery path of the target storage location.
  • control module is further used to:
  • an error message is sent to the server, wherein the error information includes at least one of the following: status information of the target location, target The status information of the object and the execution condition items that are not met.
  • control module is further used to:
  • the storage robot is controlled to perform corresponding error handling behaviors.
  • the error handling behavior is any one of the following:
  • the data acquisition module is further configured to perform at least one of the following:
  • control module is further configured to control the storage robot to move to the target location in response to the execution instruction of the handling task.
  • Another aspect of the present invention is to provide a storage robot, including:
  • a transport device an image acquisition device, a processor, a memory, and a computer program stored on the memory and running on the processor;
  • Another aspect of the present invention is to provide a computer-readable storage medium in which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned control method of a storage robot is realized.
  • the control method, device, equipment and readable storage medium of the storage robot provided by the present invention collect the image data of the target storage location corresponding to the handling task through the image acquisition device before performing the handling task, and according to the image data of the target storage location, Determine whether the current execution conditions of the handling task are met. When it is determined that the execution conditions of the handling task are satisfied, that is, when the handling device performs the handling task without danger, controlling the handling device to perform the handling task can avoid danger and improve The safety of cargo pick-up and place reduces the chance of cargo damage and shelf dumping.
  • FIG. 1 is a flow chart of a control method of a storage robot provided by Embodiment 1 of the present invention
  • Embodiment 2 is a flow chart of a control method of a storage robot provided by Embodiment 2 of the present invention.
  • FIG. 3 is a schematic structural diagram of a control device for a storage robot provided by Embodiment 3 of the present invention.
  • Fig. 4 is a schematic structural diagram of a storage robot provided by Embodiment 5 of the present invention.
  • the present invention is specifically applied to an intelligent storage system.
  • the intelligent storage system includes a storage robot, a dispatching system, a warehouse, etc.
  • the warehouse includes a plurality of storage locations for placing bins, goods and other objects. Warehousing robots can replace workers to carry goods.
  • the dispatch system communicates with the storage robot. For example, the dispatch system can issue a handling task to the storage robot, and the storage robot can send task execution status information to the dispatch system, and so on.
  • the material box may shift in the storage location or fall from the shelf.
  • the storage robot may It will collide with the material box. Therefore, there are potential safety hazards when storing and unloading bins of storage robots.
  • the control method of the storage robot provided by the present invention aims to solve the above technical problems.
  • Fig. 1 is a flow chart of a method for controlling a warehouse robot according to the first embodiment of the present invention.
  • the method in this embodiment is applied to a warehouse robot.
  • the method can also be applied to other equipment.
  • a storage robot is used as an example for schematic illustration.
  • the execution subject of the method in this embodiment may be a processor used to control the storage robot to perform handling tasks, for example, it may be a processor of a terminal device loaded on the storage robot.
  • the specific steps of the method are as follows:
  • Step S101 Collect image data of the target storage location corresponding to the transportation task through the image acquisition device.
  • the handling task includes the information of the corresponding target location, task type, and other information required to perform the current task.
  • the types of handling tasks can include pickup tasks and unloading tasks.
  • the storage robot is equipped with a handling device for picking up and/or placing the goods.
  • the handling device refers to a device for picking up or placing goods in a storage location, such as a fork.
  • the image acquisition device is a device that is installed on the storage robot and can collect image data of a target storage location.
  • the image acquisition device may be a 2D camera, a 3D camera, a lidar, or the like.
  • a 2D camera refers to a camera whose photographed data is plane data
  • 2D cameras commonly include ordinary color cameras and black and white cameras.
  • a 3D camera refers to a camera whose photographed data is stereo data. Its principle can be that structured light is reflected by an object, and the field of view is poor through a binocular camera. Common 3D cameras include Kinect, RealSense, etc.
  • the image acquisition device can be set on the handling device of the storage robot.
  • the image acquisition device installed on the handling device can be Collect the image data of the target location.
  • the processor may obtain the image data of the target storage location to determine whether the current execution condition of the transportation task is satisfied according to the image data of the target storage location.
  • the processor controls the image acquisition device to collect image data of the target storage location, and sends the image data of the target storage location to the processor.
  • the processor receives the image data of the target storage location sent by the image acquisition device, so that the image data of the target storage location can be obtained in real time.
  • Step S102 According to the image data of the target storage location, if it is determined that the execution condition of the transportation task is satisfied, the transportation device is controlled to perform the transportation task.
  • the processor can perform detection processing on the image data of the target location to detect the status information of the target location and the status information of the object in the target location; and according to the status of the target location Information and the status information of the objects in the target location to determine whether the current conditions for the execution of the handling task are met.
  • the status information of the target storage location may include whether the target storage location is free, the size, whether the handling device picks up the goods from the target storage location or whether there is an obstacle on the path of the target storage location, and so on.
  • the status information of the objects in the target location may include identity, size, posture, degree of damage, degree of deformation, and so on.
  • the information detected according to the image data of the target storage location may be changed according to the needs of the actual application scenario, which is not specifically limited in this embodiment.
  • the transportation device If it is determined that the execution conditions of the transportation task are met, it means that under the current conditions, the transportation device will not be dangerous when performing the transportation task, and the transportation device is controlled to perform the transportation task.
  • the handling device may be dangerous to perform the handling task, and the handling device will not be controlled to perform the handling task to avoid danger.
  • the image data of the target location corresponding to the transfer task is collected by the image acquisition device before the transfer task is performed. According to the image data of the target location, it is determined whether the current execution condition of the transfer task is satisfied. When the conditions are executed, the handling device may be dangerous when performing the handling task, so the handling task will not be executed temporarily to avoid danger and improve the safety of the storage robot.
  • Fig. 2 is a flowchart of a control method of a storage robot provided in the second embodiment of the present invention.
  • the transport device is controlled to perform the transport task, including: detecting the image data of the target location Process, determine the status information of the target storage location and/or target object; according to the status information of the target storage location and/or target object, if it is determined that the execution condition of the handling task is satisfied, the handling device is controlled to perform the handling task.
  • an error message is sent to the server.
  • the specific steps of the method are as follows:
  • Step S201 In response to the execution instruction of the handling task, control the storage robot to move to the target location corresponding to the handling task.
  • the execution instruction of the handling task may be the instruction information sent by the scheduling system to the storage robot for triggering the storage robot to perform the handling task.
  • the handling task includes the information of the target location, the task type, and other information needed to perform the current task.
  • the types of handling tasks can include pickup tasks and unloading tasks.
  • the processor controls the storage robot to move to the target location corresponding to the handling task according to the newness of the target location corresponding to the handling task.
  • the target location refers to the location corresponding to the handling task
  • the target object refers to the handling target of the current handling task.
  • the target location refers to the location from which the goods need to be picked up, and the picked goods and/or containers are the target objects;
  • the handling task is to unload, the objects that need to be stored are Target object, the target location refers to the location where the target object needs to be placed.
  • Step S202 Collect image data of the target storage location through the image acquisition device.
  • the storage robot is equipped with a handling device for picking up goods
  • the handling device refers to a device for picking up or placing goods in a storage location, such as a fork.
  • the image acquisition device is a device that is set on the storage robot and can collect the image data of the target storage location.
  • the image acquisition device may be an image sensor such as a black-and-white camera, a color camera, and a depth camera.
  • the image acquisition device may be a 2D camera, a 3D camera, a lidar, or the like.
  • the image acquisition device can be set on the handling device of the storage robot.
  • the image acquisition device installed on the handling device can be Collect the image data of the target location.
  • the image acquisition device can be arranged on the handling device of the storage robot, facing the front of the handling device, so that when the handling device is aligned with the target location, the image capture device can be aligned with the target location and can accurately capture the target.
  • the image data of the location can be arranged on the handling device of the storage robot, facing the front of the handling device, so that when the handling device is aligned with the target location, the image capture device can be aligned with the target location and can accurately capture the target.
  • the processor can also control the handling device to align with the target storage location by analyzing the relative position between the current location of the handling device and the target storage location.
  • the processor controls the image acquisition device to start and collect the image data of the target storage location.
  • the processor may control the handling device to move to the target storage location, so that the image acquisition device installed on the handling device can be aligned with the target storage location.
  • the processor controls the image acquisition device to start in advance and collect images of the target storage location Data, in order to obtain the image data of the target location in advance and perform detection processing, so that it can be determined as early as possible whether the current execution conditions of the handling task are met, so that the handling task can be completed in advance, which can improve efficiency.
  • the preset range can be set according to actual application scenarios, which is not specifically limited in this embodiment.
  • the transportation device is controlled to perform the transportation task. Specifically, the following steps S203-S204 can be used.
  • Step S203 Perform detection processing on the image data of the target storage location, and determine the status information of the target storage location and/or the status information of the target object.
  • the status information of the target location includes at least one of the following:
  • Obstacle information on the transportation path of the target location includes a pick-up path and/or a delivery path.
  • the status information of the target object includes at least one of the following:
  • the identity information of the target object The identity information of the target object; the posture information of the target object; the size information of the target object; the damage degree information of the target object; the deformation degree information of the target object.
  • the information detected in this step may be different according to different transport tasks.
  • the information detected in this step can be used in subsequent steps to determine whether the execution conditions of the current handling task are met. On the premise that it can be judged whether the execution conditions of the current handling task are met, the less information detected, the higher the efficiency.
  • the detection processing of image data may include: image filtering, feature extraction, target segmentation, deep learning, point cloud filtering, point cloud extraction, point cloud clustering, point cloud segmentation, point cloud deep learning and other algorithms
  • the processing may also include other image processing algorithms in the image processing field, which will be described in detail in the subsequent step S204, and will not be repeated here.
  • Step S204 According to the status information of the target storage location and/or the status information of the target object, it is determined that the execution condition of the transportation task is satisfied.
  • the execution conditions of the handling task include at least one of the following:
  • the identity information, posture information and size information of the target object meet the pick-up conditions; the damage degree of the target object is within the first preset safety threshold; the deformation degree of the target object is in the first 2. Within the preset safety threshold range.
  • the pick-up path means that before the storage robot moves to the shelf, the handling device takes the container (or the target object in the container) from the target location, and moves the container (or target object) to the storage robot.
  • the route taken during the process of the location (such as the cache location on the storage robot).
  • the handling task is picking up goods. If the posture and size of the bin meets the picking conditions and there are no obstacles in the picking path, the fork can be extended to pick up the goods.
  • the execution conditions of the handling task include at least one of the following:
  • the target storage location is free; the size of the target storage location meets the delivery conditions; there are no obstacles in the delivery path of the target storage location.
  • the handling task is placing goods. If the target location is empty, the size of the target location meets the size requirements of the bin, and there are no obstacles in the delivery path, the fork can be extended to carry out Delivery behavior.
  • the delivery path means that before the storage robot moves to the shelf, the handling device takes the cargo box (or the target object in the cargo box) from the designated location on the storage robot (such as the cache location on the storage robot), and puts the cargo box ( (Or target object) in the process of moving to the target location (or the cargo box of the target location).
  • the two-dimensional image data of the target storage location may be collected by the first camera.
  • the first photographing device may be a photographing device capable of collecting two-dimensional image data, such as a 2D camera.
  • the target location is in the field of view of the first camera Within range.
  • the processor controls the first camera to turn on, so that the first camera captures images within its field of view. Since the target location is within the field of view of the first camera, the image data that can be captured by the first camera includes The target storage location, that is, the image data of the target storage location that can be captured by the first camera, and sent to the processor.
  • the step may be: the processor performs filtering and noise reduction processing on the received image data, extracts regions in the image that meet specific conditions, and uses a deep learning algorithm to extract and separate objects in the image to identify The target location in the image, the objects in the target location, and other obstacles, etc., and based on the image processing and recognition results to determine whether the current carrying task execution conditions are met; it can also be: the use of deep learning methods for image registration , To determine whether it meets the execution conditions of the handling task; this article does not impose restrictions on this.
  • the filtering and noise reduction processing can be the application of filtering algorithms such as Gaussian filtering, mean filtering, and median filtering.
  • the specific condition may include at least one of the following: specific color, position in the image, pixel value size, and so on.
  • the specific conditions can be set and adjusted according to the specific characteristics of the target to be recognized in the actual application scenario, and this embodiment does not specifically limit it here.
  • feature extraction is performed on the image, and the extracted features may include at least one of the following: a straight line of the edge of the bin, a feature point on the surface of the bin, a specific pattern on the surface of the bin, and the color of the surface of the bin.
  • the result of feature extraction may include at least one of the following: the enclosing area of the bin line, the coordinates of the intersection of the bin line, the number of feature points, the area of a specific figure, and the like.
  • the size of the container meets the condition by judging whether the size of the enclosed area of the straight line of the container meets the preset threshold; or, it can also be the use of deep learning methods to directly identify and determine whether the size of the container meets the preset threshold.
  • Condition by judging whether a specific graphic is a preset graphic; determining whether the target is the target location specified in the handling task or the target object to be picked up.
  • any step can be increased or decreased based on the specific conditions of the actual application scenario, and the sequence can be changed, or other algorithms can be inserted to improve the detection effect based on the actual situation, which is not specifically limited in this embodiment.
  • the three-dimensional point cloud data of the target storage location may be collected by the second camera.
  • the second photographing device may be a photographing device capable of collecting three-dimensional point cloud data, such as a 3D camera, a 3D lidar, or a 2D lidar.
  • 2D lidar can obtain 3D point cloud data through movement.
  • the target location is in the field of view of the second camera Within range.
  • the processor controls the second camera to turn on, so that the second camera captures images within its field of view. Since the target location is within the field of view of the second camera, the image data that can be captured by the second camera includes The target storage location, that is, the image data of the target storage location that can be captured by the second camera, and sent to the processor.
  • the processor performs processing on the received three-dimensional point cloud data, performs noise reduction processing on the sampled point cloud, extracts the target area in the point cloud, and clusters the point cloud, Determine whether there are obstacles in the current picking/unloading path and whether the size of the storage location meets the execution conditions of the handling task through the clustering results; extract the state information of the object (bin) in the target area, and pass the state of the object (bin) The information determines whether the object and the location meet the execution conditions of the handling task.
  • extracting the target area in the point cloud may be based on whether the 3D coordinates of the point cloud fall within the preset spatial area, to extract the part where the 3D coordinates fall within the preset spatial area.
  • the preset space area can be set and adjusted according to actual application scenarios, which is not specifically limited in this embodiment.
  • the state information of the object may include at least one of the following: posture, size, flatness, and texture.
  • judging whether the object and the storage location meet the execution conditions of the transport task through the state information of the object (the bin) includes at least one of the following:
  • the material box in the field of view of the second camera can be identified. If the status information of the material box in the field of view can be captured, it is considered that there is a material box in front; if the state of the material box in the field of view is empty, it is considered There is no bin in front, and the delivery conditions are met; if the size of the bin is smaller than the size threshold, it is considered to meet the pick-up condition; if the current placement angle of the bin is within the safe range of the bin placement angle, the pick-up condition is considered to be met .
  • the size threshold and the safety range of the placement angle of the bin can be set and adjusted according to actual application scenarios, which are not specifically limited in this embodiment.
  • any step can be increased or decreased based on the specific conditions of the actual application scenario, and the sequence can be changed, or other algorithms can be inserted to improve the detection effect based on the actual situation, which is not specifically limited in this embodiment.
  • the third possible implementation manner collect two-dimensional point cloud data of the target storage location through a lidar device, or obtain two-dimensional point cloud data through movement through a single-point laser rangefinder.
  • the target storage location is within the field of view of the lidar device.
  • the processor controls the lidar device to turn on, so that the lidar device scans the image within its field of view. Since the target location is within the field of view of the lidar device, the image data that can be captured by the lidar device includes the target location. That is, the lidar device can capture the image data of the target storage location and send it to the processor.
  • the processor performs processing on the received two-dimensional point cloud data, performs noise reduction processing on the sampled point cloud, extracts the target area in the point cloud, and clusters the point cloud , Through the clustering results to determine whether there are obstacles in the current pick/release path and whether the size of the location meets the execution conditions of the handling task; extract the status information of the object (bin) in the target area, and pass the object (bin) The status information determines whether the object and the storage location meet the execution conditions of the handling task.
  • the length threshold and the angle threshold can be determined according to the size of the bin, which is not specifically limited in this embodiment.
  • the state information of the object may include at least one of the following: angle, size, and flatness.
  • judging whether the object and the storage location meet the execution conditions of the transport task through the state information of the object (the bin) includes at least one of the following:
  • the material box in the field of view of the lidar device can be identified. If the status information of the material box in the field of view can be captured, it is considered that there is a material box in front; if the state of the material box in the field of view is empty, it is considered to be forward If there is no bin, the delivery conditions are met; if the size of the bin is smaller than the size threshold, the pick-up condition is considered to be met; if the current placement angle of the bin is within the safe range of the bin's placement angle, the pick-up condition is considered to be met.
  • the size threshold and the safety range of the placement angle of the bin can be set and adjusted according to actual application scenarios, which are not specifically limited in this embodiment.
  • any step can be increased or decreased based on the specific conditions of the actual application scenario, and the sequence can be changed, or other algorithms can be inserted to improve the detection effect based on the actual situation, which is not specifically limited in this embodiment.
  • Step S205 If it is determined that the execution condition of the transport task is satisfied, the transport device is controlled to perform the transport task.
  • step S204 if it is determined that the execution conditions of the transportation task are satisfied, it means that under the current conditions, the transportation device performs the transportation task without danger, and the transportation device is controlled to perform the transportation task. For example, control the extension of the forks to pick up the cargo and take out the cargo box from the target location; or control the extension of the fork to release the cargo and place the cargo box on the target location.
  • Step S206 If it is determined that the execution condition of the transportation task is not satisfied, an error message is sent to the server.
  • the error information includes at least one of the following: status information of the target storage location, status information of the target object, and unsatisfied execution condition items.
  • the posture of the bin exceeds the safe range
  • the size of the bin exceeds the set range
  • the degree of damage of the bin exceeds the threshold for safe pick-up, and so on.
  • step S204 if it is determined that the execution condition of the transportation task is not met, it means that under the current conditions, the transportation device may be dangerous when performing the transportation task, and the transportation device will not be controlled to perform the transportation task to avoid danger.
  • the processor may send error information to the server of the dispatching system, so that the dispatching system guides the manual completion of the recovery of the working condition of the storage robot.
  • the dispatch system can send information to the terminal equipment of the corresponding technician to inform the worker how to complete the restoration of the working condition.
  • the current handling task is a pick-up task
  • workers can be told to remove obstacles in the storage location, adjust the posture of the bin, remove severely damaged bins, and so on.
  • the current handling task is a delivery task
  • workers can be told to modify the size of the current location, remove obstacles in the location, and remove the bins in the location.
  • Step S207 Control the storage robot to perform corresponding error handling actions according to the scheduling instructions of the server.
  • the processor may also control the storage robot to execute the corresponding error handling behavior according to the scheduling instruction of the server.
  • the error handling behavior is any of the following:
  • the storage robot maintains the posture before carrying out the handling task (collecting or discharging), and does not perform any action until the working condition is restored, and stands by.
  • the target point refers to any point on the map that does not interfere with the walking of other robots.
  • the processor may control the storage robot to move to the target point closest to its current position to improve efficiency.
  • Skip the current handling task and execute the next handling task refers to abandoning the acquisition of the current bin or abandoning the storage of the current bin, and enters the process of picking/unloading the next bin.
  • the processor can also control the device to execute the corresponding error handling behavior according to the preset error handling strategy. That is, the error handling strategy configuration can be set for the storage robot in advance. When an error occurs during the execution of the handling task, the corresponding error handling behavior can be directly executed according to the preset error handling strategy.
  • the embodiment of the present invention collects the image data of the target storage location through the image acquisition device on the storage robot, and uses it as the basic data for judging whether the execution condition of the handling task is satisfied.
  • Various types of warehousing systems improve the versatility and flexibility of warehousing robots, and greatly reduce the cost and deployment costs; further, warehousing robots can be directly applied to a variety of warehousing systems, compared to existing warehouses.
  • Sensors such as sonic radar and gravity meter are set in the location.
  • the target location is collected by an image acquisition device (which can be a 2D camera, a 3D camera, a 3D lidar, a 2D lidar, a single-point laser rangefinder, etc.) Based on these image data, the target location and target bin are detected based on the 2D or 3D image data, which improves the detection accuracy, so that the conditions that do not meet the carrying task execution conditions can be determined more accurately, and the danger can be better avoided The situation happened, which improved the safety of the storage robot.
  • an image acquisition device which can be a 2D camera, a 3D camera, a 3D lidar, a 2D lidar, a single-point laser rangefinder, etc.
  • Fig. 3 is a schematic structural diagram of a control device for a storage robot provided in the third embodiment of the present invention.
  • the control device of the storage robot provided in the embodiment of the present invention can execute the processing flow provided in the embodiment of the control method of the storage robot.
  • the control device 30 of the storage robot includes: a control module 301 and a data acquisition module 302.
  • control module 301 is configured to control the storage robot to move to the target location of the handling task in response to the execution instruction of the handling task;
  • the data acquisition module 302 is used to collect the image data of the target storage location through the image acquisition device;
  • the control module 301 is also used for: according to the image data of the target storage location, if it is determined that the execution condition of the transportation task is satisfied, then the transportation device is controlled to perform the transportation task.
  • the device provided in the embodiment of the present invention may be specifically used to execute the method embodiment provided in the first embodiment above, and the specific functions are not repeated here.
  • the image data of the target storage location is collected by the image acquisition device before the carrying task is performed, and according to the image data of the target storage location, it is determined whether the execution condition of the carrying task is currently satisfied.
  • the handling device performs the handling task, it may be dangerous, and the handling task will not be performed temporarily to avoid danger and improve the safety of the storage robot.
  • control module is also used for:
  • the data acquisition module is also used to:
  • control the image acquisition device When the storage robot moves to the target location corresponding to the target storage location, control the image acquisition device to start and collect the image data of the target storage location; or, when the storage robot moves to the target location corresponding to the target storage location, control the image acquisition device to start And collect the image data of the target location.
  • the image acquisition device is provided on the transport device, and the control module is further used for:
  • the status information of the target location includes at least one of the following:
  • Obstacle information on the pickup/delivery path of the target storage location size information of the target storage location; whether objects are placed in the target storage location.
  • the state information of the target object includes at least one of the following:
  • the identity information of the target object The identity information of the target object; the posture information of the target object; the size information of the target object; the damage degree information of the target object; the deformation degree information of the target object.
  • the execution condition of the handling task includes at least one of the following:
  • the identity, posture and size of the target object meet the pick-up conditions; the damage degree of the target object is within the first preset safety threshold; the deformation degree of the target object is in the second preset Within the safety threshold.
  • the execution condition of the handling task includes at least one of the following:
  • the target storage location is free; the size of the target storage location meets the delivery conditions; there are no obstacles on the delivery path of the target storage location.
  • control module is also used to:
  • an error message is sent to the server, where the error information includes at least one of the following: status information of the target location, status information of the target object, unsatisfied Execution condition item.
  • control module is also used to:
  • the storage robot is controlled to perform corresponding error handling behaviors.
  • the error handling behavior is any of the following:
  • the data acquisition module is further configured to perform at least one of the following:
  • the two-dimensional image data of the target storage location is collected by the first camera; the three-dimensional point cloud data of the target storage location is collected by the second camera; the two-dimensional point cloud data of the target storage location is collected by the lidar device.
  • the device provided in the embodiment of the present invention may be specifically used to execute the method embodiment provided in the second embodiment above, and the specific functions are not repeated here.
  • the embodiment of the present invention collects the image data of the target storage location through the image acquisition device on the storage robot, and uses it as the basic data for judging whether the execution condition of the handling task is satisfied.
  • Various types of warehousing systems improve the versatility and flexibility of warehousing robots, and greatly reduce the cost and deployment costs; further, warehousing robots can be directly applied to a variety of warehousing systems, compared to existing warehouses.
  • Sensors such as sonic radar and gravity meter are set in the location.
  • the target location is collected by an image acquisition device (which can be a 2D camera, a 3D camera, a 3D lidar, a 2D lidar, a single-point laser rangefinder, etc.) Based on these image data, the target location and target bin are detected based on the 2D or 3D image data, which improves the detection accuracy, so that the conditions that do not meet the carrying task execution conditions can be determined more accurately, and the danger can be better avoided The situation happened, which improved the safety of the storage robot.
  • an image acquisition device which can be a 2D camera, a 3D camera, a 3D lidar, a 2D lidar, a single-point laser rangefinder, etc.
  • Fig. 4 is a schematic structural diagram of a storage robot provided by Embodiment 5 of the present invention.
  • the device 100 includes a processor 1001, a memory 1002, and a computer program that is stored on the memory 1002 and can run on the processor 1001.
  • the processor 1001 implements the storage robot control method provided in any of the foregoing method embodiments when the processor 1001 runs the computer program.
  • the embodiment of the present invention collects the image data of the target storage location through the image acquisition device on the storage robot, and uses it as the basic data for judging whether the execution condition of the handling task is satisfied.
  • Various types of warehousing systems improve the versatility and flexibility of warehousing robots, and greatly reduce the cost and deployment costs; further, warehousing robots can be directly applied to a variety of warehousing systems, compared to existing warehouses.
  • Sensors such as sonic radar and gravity meter are set in the location.
  • the target location is collected by an image acquisition device (which can be a 2D camera, a 3D camera, a 3D lidar, a 2D lidar, a single-point laser rangefinder, etc.) Based on these image data, the target location and target bin are detected based on the 2D or 3D image data, which improves the detection accuracy, so that the conditions that do not meet the carrying task execution conditions can be determined more accurately, and the danger can be better avoided The situation happened, which improved the safety of the storage robot.
  • an image acquisition device which can be a 2D camera, a 3D camera, a 3D lidar, a 2D lidar, a single-point laser rangefinder, etc.
  • an embodiment of the present invention also provides a computer-readable storage medium.
  • the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the storage robot control method provided by any of the foregoing method embodiments.

Abstract

一种仓储机器人的控制方法、装置、设备及可读存储介质。控制方法包括，在执行搬运任务之前，通过图像采集装置采集所述目标库位的图像数据，根据所述目标库位的图像数据，确定当前是否满足所述搬运任务的执行条件，在确定满足搬运任务的执行条件，也就是搬运装置执行搬运任务不会发生危险时，控制搬运装置执行所述搬运任务。该方法可以避免发生危险，提高了仓储机器人的安全性。

Description

仓储机器人的控制方法、装置、设备及可读存储介质
本申请要求于2020年06月12日提交中国专利局、申请号为202010537646.9、申请名称为“仓储机器人的控制方法、装置、设备及可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及智能仓储技术领域，尤其涉及一种仓储机器人的控制方法、装置、设备及可读存储介质。
背景技术
随着智能制造和仓储物流领域的网络化和智能化，仓储物流在企业生成管理过程中具有非常重要的地位，在智能仓储领域，仓储机器人代替工人进行货物的搬运变得越来越普遍。
现有的智能仓储系统中，由于货架震动或人为的操作失误等都会导致料箱在库位内发生偏移或者从货架掉落，仓储机器人在取该料箱或者在经过该料箱时，可能会与料箱发生碰撞。因此，仓储机器人存取料箱时存在安全隐患。
发明内容
本发明提供一种仓储机器人的控制方法、装置、设备及可读存储介质，用以解决仓储机器人安全性低的问题。
本发明的一个方面是提供一种仓储机器人的控制方法，所述仓储机器人具有搬运装置和图像采集装置，包括：
通过所述图像采集装置采集搬运任务对应的目标库位的图像数据；根据所述目标库位的图像数据，若确定满足所述搬运任务的执行条件，则控制所述搬运装置执行所述搬运任务。
在一种可能的实施方式中，所述通过所述图像采集装置采集搬运任务对应的目标库位的图像数据，包括：
当所述仓储机器人移动至所述目标库位对应的目标位置时，控制所述图像采集装置启动并采集所述目标库位的图像数据；或者，当所述仓储机器人移动至所述目标库位周围的预设范围内时，控制所述图像采集装置启动并采集所述目标库位的图像数据。
在一种可能的实施方式中，所述图像采集装置设置于所述搬运装置上，控制所述图像采集装置启动并采集所述目标库位的图像数据之前，还包括：
控制所述搬运装置对准所述目标库位。
在一种可能的实施方式中，所述根据所述目标库位的图像数据，若确定满足所述搬运任务的执行条件，则控制所述搬运装置执行所述搬运任务，包括：
对所述目标库位的图像数据进行检测处理，确定所述目标库位的状态信息和/或目标物体的状态信息；根据所述目标库位的状态信息和/或目标物体的状态信息，若确定满足所述搬运任务的执行条件，则控制所述搬运装置执行所述搬运任务。
在一种可能的实施方式中，所述目标库位的状态信息包括以下至少一项：
所述目标库位的搬运路径上的障碍物信息；所述目标库位的尺寸信息；所述目标库位是否空闲。
在一种可能的实施方式中，所述目标物体的状态信息包括以下至少一项：
所述目标物体的身份信息；所述目标物体的姿态信息；所述目标物体的尺寸信息；所述目标物体的破损程度信息；所述目标物体的形变程度信息。
在一种可能的实施方式中，所述搬运任务为取货任务，所述搬运任务的执行条件包括以下至少一项：
所述目标库位的取货路径上没有障碍物；所述目标物体的身份信息、姿态信息和尺寸信息满足取货条件；所述目标物体的破损程度在第一预设安全阈值范围内；所述目标物体的形变程度在第二预设安全阈值范围内。
在一种可能的实施方式中，所述搬运任务为放货任务，所述搬运任务的执行条件包括以下至少一项：
所述目标库位空闲；所述目标库位的尺寸满足放货条件；所述目标库位的放货路径上没有障碍物。
在一种可能的实施方式中，还包括：
根据所述目标库位的图像数据，若确定不满足所述搬运任务的执行条件，则向服务器发送错误信息，其中所述错误信息包括以下至少一项：所述目标库位的状态信息、目标物体的状态信息、未满足的执行条件项。
在一种可能的实施方式中，向服务器发送错误信息之后，还包括：
根据所述服务器的调度指示，控制所述仓储机器人执行对应的错误处理行为。
在一种可能的实施方式中，所述错误处理行为是以下任意一种：
停留在当前位置，等待指示；移动到目标点；跳过当前的所述搬运任务，执行下一个搬运任务。
在一种可能的实施方式中，通过所述图像采集装置采集所述目标库位的图像数据，包括以下至少一项：
通过第一拍摄装置采集所述目标库位的二维图像数据；通过第二拍摄装置采集所述目标库位的三维点云数据；通过激光雷达装置采集所述目标库位的二维点云数据。
在一种可能的实施方式中，所述通过所述图像采集装置采集搬运任务对应的所述目标库位的图像数据之前，还包括：
响应于搬运任务的执行指令，控制仓储机器人向所述目标库位移动。
本发明的另一个方面是提供一种仓储机器人的控制装置，应用于仓储机器人，所述仓储机器人包括搬运装置和图像采集装置，包括：
数据获取模块，用于通过所述图像采集装置采集搬运任务对应的目标库位的图像数据；
控制模块，用于根据所述目标库位的图像数据，若确定满足所述搬运任务的执行条件，则控制搬运装置执行所述搬运任务。
在一种可能的实施方式中，所述数据获取模块还用于：
当所述仓储机器人移动至所述目标库位对应的目标位置时，控制所述图像采集装置启动并采集所述目标库位的图像数据；或者，当所述仓储机器人移动至所述目标库位周围的预设范围内时，控制所述图像采集装置启动并采集所述目标库位的图像数据。
在一种可能的实施方式中，所述图像采集装置设置于所述搬运装置上，所述控制模块还用于：
控制所述搬运装置对准所述目标库位。
在一种可能的实施方式中，所述控制模块还用于：
对所述目标库位的图像数据进行检测处理，确定所述目标库位的状态信息和/或目标物体的状态信息；根据所述目标库位的状态信息和/或目标物体的状态信息，若确定满足所述搬运任务的执行条件，则控制所述搬运装置执行所述搬运任务。
在一种可能的实施方式中，所述目标库位的状态信息包括以下至少一项：
所述目标库位的搬运路径上的障碍物信息；所述目标库位的尺寸信息；所述目标库位是否空闲。
在一种可能的实施方式中，所述目标物体的状态信息包括以下至少一项：
所述目标物体的身份信息；所述目标物体的姿态信息；所述目标物体的尺寸信息；所述目标物体的破损程度信息；所述目标物体的形变程度信息。
在一种可能的实施方式中，所述搬运任务为取货任务，所述搬运任务的执行条件包括以下至少一项：
所述目标库位的取货路径上没有障碍物；所述目标物体的身份信息、姿态信息和尺寸信息满足取货条件；所述目标物体的破损程度在第一预设安全阈值范围内；所述目标物体的形变程度在第二预设安全阈值范围内。
在一种可能的实施方式中，所述搬运任务为放货任务，则所述搬运任务的执行条件包括以下至少一项：
所述目标库位空闲；所述目标库位的尺寸满足放货条件；所述目标库位的放货路径上没有障碍物。
在一种可能的实施方式中，所述控制模块还用于：
根据所述目标库位的图像数据，若确定不满足所述搬运任务的执行条件，则向服务器发送错误信息，其中所述错误信息包括以下至少一项：所述目标库位的状态信息、目标物体的状态信息、未满足的执行条件项。
在一种可能的实施方式中，所述控制模块还用于：
根据所述服务器的调度指示，控制所述仓储机器人执行对应的错误处理行为。
在一种可能的实施方式中，所述错误处理行为是以下任意一种：
停留在当前位置，等待指示；移动到目标点；跳过当前的所述搬运任务， 执行下一个搬运任务。
在一种可能的实施方式中，所述数据获取模块还用于执行以下至少一项：
通过第一拍摄装置采集所述目标库位的二维图像数据；通过第二拍摄装置采集所述目标库位的三维点云数据；通过激光雷达装置采集所述目标库位的二维点云数据。
在一种可能的实施方式中，所述控制模块还用于响应于搬运任务的执行指令，控制仓储机器人向所述目标库位移动。
本发明的另一个方面是提供一种仓储机器人，包括：
搬运装置，图像采集装置，处理器，存储器，以及存储在所述存储器上并可在所述处理器上运行的计算机程序；
其中，所述处理器运行所述计算机程序时实现上述所述的仓储机器人的控制方法。
本发明的另一个方面是提供一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机程序，所述计算机程序被处理器执行时实现上述所述的仓储机器人的控制方法。
本发明提供的仓储机器人的控制方法、装置、设备及可读存储介质，在执行搬运任务之前，通过图像采集装置采集搬运任务对应目标库位的图像数据，根据所述目标库位的图像数据，确定当前是否满足所述搬运任务的执行条件，在确定满足搬运任务的执行条件，也就是搬运装置执行搬运任务不会发生危险时，控制搬运装置执行所述搬运任务，可以避免发生危险，提高了货物取放的安全性，降低了货物损坏以及货架倾倒的几率。
附图说明
图1为本发明实施例一提供的仓储机器人的控制方法流程图；
图2为本发明实施例二提供的仓储机器人的控制方法流程图；
图3为本发明实施例三提供的仓储机器人的控制装置的结构示意图；
图4为本发明实施例五提供的仓储机器人的结构示意图。
通过上述附图，已示出本发明明确的实施例，后文中将有更详细的描述。这些附图和文字描述并不是为了通过任何方式限制本发明构思的范围， 而是通过参考特定实施例为本领域技术人员说明本发明的概念。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本发明相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本发明的一些方面相一致的装置和方法的例子。
术语“第一”、“第二”等仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。在以下各实施例的描述中，“多个”的含义是两个以上，除非另有明确具体的限定。
本发明具体应用于智能仓储系统，智能仓储系统包括仓储机器人，调度系统，仓库等，仓库包括多个用于放置料箱、货物等物体的库位。仓储机器人能够代替工人进行货物的搬运。调度系统与仓储机器人进行通信，例如调度系统可以向仓储机器人下发搬运任务，仓储机器人可以向调度系统发送任务执行的状态信息，等等。
现有的智能仓储系统中，由于货架震动或人为的操作失误等都会导致料箱在库位内发生偏移或者从货架掉落，仓储机器人在取该料箱或者在经过该料箱时，可能会与料箱发生碰撞。因此，仓储机器人的存取料箱时存在安全隐患。
本发明提供的仓储机器人的控制方法，旨在解决如上的技术问题。
下面以具体地实施例对本发明的技术方案以及本申请的技术方案如何解决上述技术问题进行详细说明。下面这几个具体的实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例中不再赘述。下面将结合附图，对本发明的实施例进行描述。
实施例一
图1为本发明实施例一提供的仓储机器人的控制方法流程图。本实施例中的方法应用于仓储机器人。在其他实施例中，该方法还可应用于其他设备，本实施例以仓储机器人为例进行示意性说明。本实施例中的方法的执行主体可以是用于控制仓储机器人执行搬运任务的处理器，例如可以是仓储机器人 上装载的终端设备的处理器等。如图1所示，该方法具体步骤如下：
步骤S101、通过图像采集装置采集搬运任务对应的目标库位的图像数据。
其中，搬运任务包括对应的目标库位的信息，任务类型，以及执行当前任务所需的其他信息。其中，搬运任务的类型可以包括取货任务和放货任务。
仓储机器人配置有用于取货和/或放货的搬运装置，搬运装置是指用于从库位取货或者向库位中放货的装置，例如货叉等。
图像采集装置是设置于仓储机器人上的、能够采集目标库位的图像数据的装置，例如，图像采集装置可以是2D相机、3D相机、激光雷达等。本公开中，2D相机是指拍摄的数据为平面数据的相机，2D相机常见有普通的彩色相机，黑白相机。3D相机是指拍摄的数据为立体数据的相机，其原理可为通过物体反射结构光，通过双目摄像头的视野差等，3D相机常见有Kinect、RealSense等。
可选地，图像采集装置可以设置于仓储机器人的搬运装置上，当仓储机器人向目标库位移动的过程中，或者仓储机器人移动至目标库位附近时，安装于搬运装置上的图像采集装置可以采集到目标库位的图像数据。
在执行搬运任务之前，处理器可以获取目标库位的图像数据，以根据目标库位的图像数据，确定当前是否满足搬运任务的执行条件。
具体地，处理器控制图像采集装置采集目标库位的图像数据，并将目标库位的图像数据发送给处理器。处理器接收图像采集装置发送的目标库位的图像数据，从而可以实时地获取到目标库位的图像数据。
步骤S102、根据目标库位的图像数据，若确定满足搬运任务的执行条件，则控制搬运装置执行搬运任务。
在获取到目标库位的图像数据之后，处理器可以对目标库位的图像数据进行检测处理，以检测目标库位的状态信息和目标库位内物体的状态信息；并根据目标库位的状态信息和目标库位内物体的状态信息，确定当前是否满足搬运任务的执行条件。
示例性地，目标库位的状态信息可以包括目标库位是否空闲、尺寸、搬运装置从目标库位取货或者向目标库位放货的路径上是否有障碍物，等等。目标库位中的物体的状态信息可以包括身份、尺寸、姿态、破损程度、变形程度等等。
另外，根据目标库位的图像数据所检测的信息可以根据实际应用场景的需要发生变化，本实施例此处不做具体限定。
如果确定满足搬运任务的执行条件，则说明在当前的条件下，搬运装置执行搬运任务不会发生危险，控制搬运装置执行搬运任务。
如果确定不满足搬运任务的执行条件，则说明在当前的条件下，搬运装置执行搬运任务可能会发生危险，不会控制搬运装置执行搬运任务，以避免发生危险。
本发明实施例通过在执行搬运任务之前，通过图像采集装置采集搬运任务对应目标库位的图像数据，根据目标库位的图像数据，确定当前是否满足搬运任务的执行条件，在不满足搬运任务的执行条件时，搬运装置执行搬运任务可能会发生危险，则暂不执行搬运任务，以避免发生危险，提高了仓储机器人的安全性。
实施例二
图2为本发明实施例二提供的仓储机器人的控制方法流程图。在上述实施例一的基础上，本实施例中，根据目标库位的图像数据，若确定满足搬运任务的执行条件，则控制搬运装置执行搬运任务，包括：对目标库位的图像数据进行检测处理，确定目标库位和/或目标物体的状态信息；根据目标库位和/或目标物体的状态信息，若确定满足搬运任务的执行条件，则控制搬运装置执行搬运任务。进一步地，根据目标库位的图像数据，若确定不满足搬运任务的执行条件，则向服务器发送错误信息。如图2所示，该方法具体步骤如下：
步骤S201、响应于搬运任务的执行指令，控制仓储机器人向搬运任务对应的目标库位移动。
其中，搬运任务的执行指令可以是调度系统向仓储机器人发送的用于触发仓储机器人执行搬运任务的指令信息。
搬运任务包括目标库位的信息，任务类型，以及执行当前任务所需的其他信息。其中，搬运任务的类型可以包括取货任务和放货任务。
当接收到的搬运任务的执行指令时，处理器根据搬运任务对应的目标库位的新，控制仓储机器人向搬运任务对应的目标库位移动。
本实施例中，目标库位是指搬运任务对应的库位，目标物体是指本次搬 运任务的搬运目标。例如，如果搬运任务为取货，则目标库位是指需要从哪个库位上取货，所取的货物和/或货箱就是目标物体；如果搬运任务为放货，则需要存放的物体就是目标物体，目标库位是指需要将目标物体放置于的那个库位。
步骤S202、通过图像采集装置采集目标库位的图像数据。
其中，仓储机器人配置有用于取货的搬运装置，搬运装置是指用于从库位取货或者向库位中放货的装置，例如货叉等。
图像采集装置是设置于仓储机器人上的、能够采集目标库位的图像数据的装置。图像采集装置可以是黑白相机，彩色相机，深度相机等图像传感器。例如，图像采集装置可以是2D相机、3D相机、激光雷达等。
可选地，图像采集装置可以设置于仓储机器人的搬运装置上，当仓储机器人向目标库位移动的过程中，或者仓储机器人移动至目标库位附近时，安装于搬运装置上的图像采集装置可以采集到目标库位的图像数据。
可选地，图像采集装置可以设置于仓储机器人的搬运装置上，朝向搬运装置的前方，以使搬运装置对准目标库位时，图像采集装置可以对准目标库位，能够准确地拍摄到目标库位的图像数据。
进一步地，处理器还可以通过分析搬运装置当前所在位置与目标库位的相对位置，控制搬运装置对准目标库位。
示例性地，当仓储机器人移动至目标库位对应的目标位置时，处理器控制图像采集装置启动并采集目标库位的图像数据。
进一步地，当仓储机器人移动至目标库位对应的目标位置时，处理器可以控制搬运装置移动到目标库位，以使安装于搬运装置上的图像采集装置可以对准目标库位。
示例性地，在仓储机器人向目标库位对应的目标位置移动过程中，当仓储机器人移动至目标库位周围的预设范围内时，处理器控制图像采集装置提前启动并采集目标库位的图像数据，以提前获得目标库位的图像数据并进行检测处理，从而可以尽早判断当前是否满足搬运任务的执行条件，以提前完成搬运任务，能够提高效率。其中预设范围可以根据实际应用场景进行设定，本实施例此处不做具体限定。
本实施例中，根据目标库位的图像数据，若确定满足搬运任务的执行条 件，则控制搬运装置执行搬运任务，具体可以采用如下步骤S203-S204实现。
步骤S203、对目标库位的图像数据进行检测处理，确定目标库位的状态信息和/或目标物体的状态信息。
其中，目标库位的状态信息包括以下至少一项：
目标库位的搬运路径上的障碍物信息；目标库位的尺寸信息；目标库位是否空闲。其中，搬运路径包括取货路径和/或放货路径。
目标物体的状态信息包括以下至少一项：
目标物体的身份信息；目标物体的姿态信息；目标物体的尺寸信息；目标物体的破损程度信息；目标物体的形变程度信息。
本实施例中，根据搬运任务的不同，该步骤中所检测的信息可以不同。该步骤中所检测的信息能够用于后续步骤中判断是否满足当前搬运任务的执行条件即可，在能够判断是否满足当前搬运任务的执行条件的前提下，检测的信息越少，效率越高。
示例性地，图像数据进行的检测处理可以包括：图像学滤波、特征提取、目标分割、深度学习、点云滤波、点云提取、点云聚类、点云分割、点云的深度学习等算法的处理，还可以包括图像处理领域的其他图像处理的算法，具体在后续步骤S204中进行详细说明，此处不再赘述。
步骤S204、根据目标库位的状态信息和/或目标物体的状态信息，判断满足搬运任务的执行条件。
具体地，若搬运任务为取货，则搬运任务的执行条件包括以下至少一项：
目标库位的取货路径上没有障碍物；目标物体的身份信息、姿态信息和尺寸信息满足取货条件；目标物体的破损程度在第一预设安全阈值范围内；目标物体的形变程度在第二预设安全阈值范围内。
其中，取货路径是指仓储机器人移动到货架前，搬运装置从目标库位中取到货箱(或货箱内的目标物体)，并将货箱(或目标物体)移动到仓储机器人上指定位置(如仓储机器人上的缓存位置)的过程中所经过的路线。
例如，以目标物体为料箱为例，搬运任务为取货，若料箱姿态和尺寸满足取货条件，且取货路径中无障碍物，则可以伸出货叉进行取货行为。
若搬运任务为放货，则搬运任务的执行条件包括以下至少一项：
目标库位空闲；目标库位的尺寸满足放货条件；目标库位的放货路径上 没有障碍物。
例如，以目标物体为料箱为例，搬运任务为放货，若目标库位为空，目标库位大小满足料箱尺寸要求，且放货路径中无障碍物，则可以伸出货叉进行放货行为。
其中，放货路径是指仓储机器人移动到货架前，搬运装置从仓储机器人上指定位置(如仓储机器人上的缓存位置)取到货箱(或货箱内的目标物体)，并将货箱(或目标物体)移动到目标库位(或者目标库位的货箱)中的过程中所经过的路线。
在一种可能的实施方式中，可以通过第一拍摄装置采集目标库位的二维图像数据。
其中，第一拍摄装置可以是2D相机等能够采集二维图像数据的拍摄装置。
具体的，通过对第一拍摄装置所在的仓储机器人和/或搬运装置的位置的调整，以及对第一拍摄装置在仓储机器人上的安装位置的调整，使得目标库位在第一拍摄装置的视野范围内。
处理器通过控制第一拍摄装置开启，使得第一拍摄装置拍摄其视野范围内的图像，由于目标库位在第一拍摄装置的视野范围内，因此第一拍摄装置拍摄能够拍摄的图像数据中包括目标库位，也即是第一拍摄装置能够拍摄到目标库位的图像数据，并发送给处理器。
对于这种实施方式，该步骤可以是：处理器对接收到的图像数据进行滤波降噪处理，提取图像中满足特定条件的区域，运用深度学习算法对图像中的目标进行提取分离，以识别出图像中的目标库位，目标库位内的物体，以及其他障碍物等目标，并根据图像处理及识别结果判断当前是否满足搬运任务的执行条件；也可以是：运用深度学习方法进行图像配准，判断是否符合搬运任务的执行条件；本文对此不加以限制。
其中，滤波降噪处理可为应用高斯滤波、均值滤波、中值滤波等滤波算法。
示例性地，特定条件可以包括以下至少一项：特定颜色、在图像中的位置、像素值大小等。特定条件可以根据实际应用场景中待识别目标的具体特点进行设定和调整，本实施例此处不做具体限定。
示例性地，对图像进行特征提取，所提取的特征可以包括以下至少一项： 料箱的边缘直线，料箱表面的特征点，料箱表面的特定图形，料箱表面的颜色。特征提取的结果可以包括以下至少一项：料箱直线的包围面积、料箱直线的交点坐标、特征点数量、特定图形面积等。根据特征提取结果，可以是通过判断料箱直线的包围面积的大小是否符合预设阈值，可以确定料箱尺寸是否满足条件；或者，也可以是运用深度学习方法，直接识别判断料箱尺寸是否满足条件；通过判断特定图形是为预设图形；以确定目标是否是搬运任务中指定的目标库位或待取货的目标物体等。
该实施方式中，所提取的特征包括哪些特征，特征提取的结果包括哪些信息，以及根据具体的特征提取结果判断当前是否满足搬运任务的执行条件的规则，可以根据实际应用场景进行调整，本实施例此处不做具体限定。
该实施方式中，任意步骤均可基于实际应用场景的具体情况进行增减、改变顺序，也可基于现实情况插入其他算法提升检测效果，本实施例此处不做具体限定。
在另一种可能的实施方式中，可以通过第二拍摄装置采集目标库位的三维点云数据。
其中，第二拍摄装置可以是3D相机、3D激光雷达或者2D激光雷达等能够采集三维点云数据的拍摄装置。其中，2D激光雷达通过移动能够获得3D点云数据。
具体的，通过对第二拍摄装置所在的仓储机器人和/或搬运装置的位置的调整，以及对第二拍摄装置在仓储机器人上的安装位置的调整，使得目标库位在第二拍摄装置的视野范围内。
处理器通过控制第二拍摄装置开启，使得第二拍摄装置拍摄其视野范围内的图像，由于目标库位在第二拍摄装置的视野范围内，因此第二拍摄装置拍摄能够拍摄的图像数据中包括目标库位，也即是第二拍摄装置能够拍摄到目标库位的图像数据，并发送给处理器。
对于这种实施方式，该步骤中，处理器对接收到的三维点云数据进行采用处理，对采样后的点云进行降噪处理，提取点云中的目标区域，对点云进行聚类，通过聚类结果判断当前取/放货路径中是否有障碍物以及库位尺寸是否满足搬运任务的执行条件；提取目标区域中的物体(料箱)的状态信息，通过物体(料箱)的状态信息判断物体和库位是否满足搬运任务的执行条件。
示例性地，提取点云中的目标区域可以为依据点云的3D坐标是否落入预设的空间区域，来提取出3D坐标落入预设的空间区域的部分。其中，预设的空间区域可以根据实际应用场景进行设定和调整，本实施例此处不做具体限定。
示例性地，若聚类后在目标区域中存在点云物体类别，则确定取/放货路径中存在障碍物，或库位尺寸不满足放货的要求。反之，若聚类后在目标区域中不存在点云物体类别，则确定取/放货路径中不存在障碍物，库位尺寸满足放货的要求。
示例性地，以物体为料箱为例，物体的状态信息可以包括以下至少一种：姿态，尺寸，平整度，纹理。
示例性地，以物体为料箱为例，通过物体(料箱)的状态信息判断物体和库位是否满足搬运任务的执行条件，包括以下至少一种：
根据料箱的状态信息，可以识别出第二拍摄装置视野内的料箱，如果可捕获到视野内的料箱的状态信息，则认为前方有料箱；如果视野内料箱状态为空，则认为前方无料箱，满足放货条件；如果料箱尺寸小于尺寸阈值，则认为满足可取货条件；如果料箱当前的摆放角度在料箱摆放角度的安全范围内，则认为满足可取货条件。
其中，尺寸阈值、料箱摆放角度的安全范围可以根据实际应用场景进行设定和调整，本实施例此处不做具体限定。
该实施方式中，任意步骤均可基于实际应用场景的具体情况进行增减、改变顺序，也可基于现实情况插入其他算法提升检测效果，本实施例此处不做具体限定。
第三种可能的实施方式：通过激光雷达装置采集目标库位的二维点云数据，或者通过单点激光测距仪通过运动获得二维点云数据。
具体的，通过对激光雷达装置所在的仓储机器人和/或搬运装置的位置的调整，以及对激光雷达装置在仓储机器人上的安装位置的调整，使得目标库位在激光雷达装置的视野范围内。
处理器通过控制激光雷达装置开启，使得激光雷达装置扫描其视野范围内的图像，由于目标库位在激光雷达装置的视野范围内，因此激光雷达装置拍摄能够拍摄的图像数据中包括目标库位，也即是激光雷达装置能够拍摄到 目标库位的图像数据，并发送给处理器。
对于这种实施方式，该步骤中，处理器对接收到的二维点云数据进行采用处理，对采样后的点云进行降噪处理，提取点云中的目标区域，对点云进行聚类，通过聚类结果判断当前取/放货路径中是否有障碍物以及库位尺寸是否满足搬运任务的执行条件；提取目标区域中的物体(料箱)的状态信息，通过物体(料箱)的状态信息判断物体和库位是否满足搬运任务的执行条件。
示例性地，若聚类后在目标区域中存在点云物体类别，则确定取/放货路径中存在障碍物，或库位尺寸不满足放货的要求。反之，若聚类后在目标区域中不存在点云物体类别，则确定取/放货路径中不存在障碍物。另外通过计算库位边线长度、库位边线间所成的角度，根据边线长度和所成的角度是否符合预设长度阈值和角度阈值，来判断库位尺寸是否满足放货的要求。其中，长度阈值和角度阈值可以根据料箱的尺寸确定，本实施例此处不做具体限定。
示例性地，以物体为料箱为例，物体的状态信息可以包括以下至少一种：角度，尺寸，平整度。
示例性地，以物体为料箱为例，通过物体(料箱)的状态信息判断物体和库位是否满足搬运任务的执行条件，包括以下至少一种：
根据料箱的状态信息，可以识别出激光雷达装置视野内的料箱，如果可捕获到视野内的料箱的状态信息，则认为前方有料箱；如果视野内料箱状态为空，则认为前方无料箱，满足放货条件；如果料箱尺寸小于尺寸阈值，则认为满足可取货条件；如果料箱当前的摆放角度在料箱摆放角度的安全范围内，则认为满足可取货条件。其中，尺寸阈值、料箱摆放角度的安全范围可以根据实际应用场景进行设定和调整，本实施例此处不做具体限定。
该实施方式中，任意步骤均可基于实际应用场景的具体情况进行增减、改变顺序，也可基于现实情况插入其他算法提升检测效果，本实施例此处不做具体限定。
步骤S205、若确定满足搬运任务的执行条件，则控制搬运装置执行搬运任务。
在上述步骤S204中，若确定满足搬运任务的执行条件，则说明在当前的条件下，搬运装置执行搬运任务不会发生危险，控制搬运装置执行搬运任务。例如，控制货叉伸出进行取货行为，从目标库位取出货箱；或者控制货叉伸 出进行放货行为，将货箱放到目标库位上。
步骤S206、若确定不满足搬运任务的执行条件，则向服务器发送错误信息。
其中，错误信息包括以下至少一项：目标库位的状态信息、目标物体的状态信息、未满足的执行条件项。
例如，库位取/放货路径有障碍物、料箱姿态超出安全范围、料箱尺寸超出设定范围、料箱的破损程度超出安全取货的阈值等等。
在上述步骤S204中，若确定不满足搬运任务的执行条件，则说明在当前的条件下，搬运装置执行搬运任务可能会发生危险，不会控制搬运装置执行搬运任务，以避免发生危险。
进一步地，处理器可以向调度系统的服务器发送错误信息，以使调度系统引导人工完成仓储机器人工况的恢复。例如，调度系统可以向对应技术人员的终端设备发送信息，告知工人如何完成工况恢复。
例如，如果当前的搬运任务为取货任务，可以告知工人移除库位中的障碍物、调整料箱的姿态、移除严重破损的料箱等等。如果当前的搬运任务为放货任务，可以告知工人修改当前库位的尺寸、移除库位中的障碍物、移除库位内料箱等。
步骤S207、根据服务器的调度指示，控制仓储机器人执行对应的错误处理行为。
本实施例中，若确定不满足搬运任务的执行条件，处理器还可以根据服务器的调度指示，控制仓储机器人执行对应的错误处理行为。
其中，错误处理行为是以下任意一种：
停留在当前位置，等待指示；移动到目标点；跳过当前的搬运任务，执行下一个搬运任务。
其中，停留在当前位置，等待指示：是指仓储机器人保持执行搬运任务(取货或者放货)前的姿态，并且在工况恢复前不执行任何动作，原地待命。
目标点是指地图中不干扰其他机器人行走的任意点。可选地，处理器可以控制仓储机器人移动到距离其当前位置最近的目标点，以提高效率。
跳过当前的搬运任务，执行下一个搬运任务：是指放弃获取当前料箱或者放弃存放当前料箱，进入下一个料箱的取/放货的流程。
本实施例的另一实施方式中，若确定不满足搬运任务的执行条件，则处理器向服务器发送错误信息之后，还可以根据预设错误处理策略，控制设备执行对应的错误处理行为。也就是可以预先为仓储机器人设定好错误处理策略配置，当执行搬运任务过程中出现错误之后，可以直接根据预设的错误处理策略，执行对应的错误处理行为。
本发明实施例通过仓储机器人上的图像采集装置采集目标库位的图像数据，以此作为判断搬运任务的执行条件是否满足的基础数据，无需在每个库位上设置传感器，可以灵活地应用于各种类型的仓储系统，提高了仓储机器人的通用性和灵活性，并且大大降低了造价成本和部署成本；进一步地，仓储机器人可以直接与应用于多种仓储系统，对相对于现有的库位中设置的声波雷达、重力测计等传感器，本实施例中通过图像采集装置(可以是2D相机、3D相机、3D激光雷达、2D激光雷达、单点激光测距仪等)采集目标库位的2D或者3D图像数据，基于这些图像数据对目标库位和目标料箱等进行检测，提高了检测精准度，从而可以更加准确地确定不满足搬运任务执行条件的情况，可以更好地避免危险情况发生，提高了仓储机器人的安全性。
实施例三
图3为本发明实施例三提供的仓储机器人的控制装置的结构示意图。本发明实施例提供的仓储机器人的控制装置可以执行仓储机器人的控制方法实施例提供的处理流程。如图3所示，该仓储机器人的控制装置30包括：控制模块301和数据获取模块302。
具体地，控制模块301用于响应于搬运任务的执行指令，控制仓储机器人向搬运任务的目标库位移动；
数据获取模块302用于通过图像采集装置采集目标库位的图像数据；
控制模块301还用于：根据目标库位的图像数据，若确定满足搬运任务的执行条件，则控制搬运装置执行搬运任务。
本发明实施例提供的装置可以具体用于执行上述实施例一所提供的方法实施例，具体功能此处不再赘述。
本发明实施例通过在执行搬运任务之前，通过图像采集装置采集目标库位的图像数据，根据目标库位的图像数据，确定当前是否满足搬运任务的执行条件，在确定不满足搬运任务的执行条件时，搬运装置执行搬运任务可能 会发生危险，暂不执行搬运任务，以避免发生危险，提高了仓储机器人的安全性。
实施例四
在上述实施例三的基础上，本实施例中，控制模块还用于：
对目标库位的图像数据进行检测处理，确定目标库位和/或目标物体的状态信息；根据目标库位和/或目标物体的状态信息，若确定满足搬运任务的执行条件，则控制搬运装置执行搬运任务。
在一种可能的实施方式中，数据获取模块还用于：
当仓储机器人移动至目标库位对应的目标位置时，控制图像采集装置启动并采集目标库位的图像数据；或者，在仓储机器人向目标库位对应的目标位置移动过程中，控制图像采集装置启动并采集目标库位的图像数据。
在一种可能的实施方式中，图像采集装置设置于搬运装置上，控制模块还用于：
控制搬运装置对准目标库位。
在一种可能的实施方式中，目标库位的状态信息包括以下至少一项：
目标库位的取/放货路径上的障碍物信息；目标库位的尺寸信息；目标库位内是否放置有物体。
在一种可能的实施方式中，目标物体的状态信息包括以下至少一项：
目标物体的身份信息；目标物体的姿态信息；目标物体的尺寸信息；目标物体的破损程度信息；目标物体的形变程度信息。
在一种可能的实施方式中，若搬运任务为取货，则搬运任务的执行条件包括以下至少一项：
目标库位的取货路径上没有障碍物；目标物体的身份、姿态和尺寸满足取货条件；目标物体的破损程度在第一预设安全阈值范围内；目标物体的形变程度在第二预设安全阈值范围内。
在一种可能的实施方式中，若搬运任务为放货，则搬运任务的执行条件包括以下至少一项：
目标库位空闲；目标库位的尺寸满足放货条件；目标库位的放货路径上没有障碍物。
在一种可能的实施方式中，控制模块还用于：
根据目标库位的图像数据，若确定不满足搬运任务的执行条件，则向服务器发送错误信息，其中错误信息包括以下至少一项：目标库位的状态信息、目标物体的状态信息、未满足的执行条件项。
在一种可能的实施方式中，控制模块还用于：
根据服务器的调度指示，控制仓储机器人执行对应的错误处理行为。
在一种可能的实施方式中，错误处理行为是以下任意一种：
停留在当前位置，等待指示；移动到目标点；跳过当前的搬运任务，执行下一个搬运任务。
在一种可能的实施方式中，数据获取模块还用于执行以下至少一项：
通过第一拍摄装置采集目标库位的二维图像数据；通过第二拍摄装置采集目标库位的三维点云数据；通过激光雷达装置采集目标库位的二维点云数据。
本发明实施例提供的装置可以具体用于执行上述实施例二所提供的方法实施例，具体功能此处不再赘述。
本发明实施例通过仓储机器人上的图像采集装置采集目标库位的图像数据，以此作为判断搬运任务的执行条件是否满足的基础数据，无需在每个库位上设置传感器，可以灵活地应用于各种类型的仓储系统，提高了仓储机器人的通用性和灵活性，并且大大降低了造价成本和部署成本；进一步地，仓储机器人可以直接与应用于多种仓储系统，对相对于现有的库位中设置的声波雷达、重力测计等传感器，本实施例中通过图像采集装置(可以是2D相机、3D相机、3D激光雷达、2D激光雷达、单点激光测距仪等)采集目标库位的2D或者3D图像数据，基于这些图像数据对目标库位和目标料箱等进行检测，提高了检测精准度，从而可以更加准确地确定不满足搬运任务执行条件的情况，可以更好地避免危险情况发生，提高了仓储机器人的安全性。
实施例五
图4为本发明实施例五提供的仓储机器人的结构示意图。如图4所示，该设备100包括：处理器1001，存储器1002，以及存储在存储器1002上并可在处理器1001上运行的计算机程序。
其中，处理器1001运行计算机程序时实现上述任一方法实施例提供的仓储机器人的控制方法。
本发明实施例通过仓储机器人上的图像采集装置采集目标库位的图像数据，以此作为判断搬运任务的执行条件是否满足的基础数据，无需在每个库位上设置传感器，可以灵活地应用于各种类型的仓储系统，提高了仓储机器人的通用性和灵活性，并且大大降低了造价成本和部署成本；进一步地，仓储机器人可以直接与应用于多种仓储系统，对相对于现有的库位中设置的声波雷达、重力测计等传感器，本实施例中通过图像采集装置(可以是2D相机、3D相机、3D激光雷达、2D激光雷达、单点激光测距仪等)采集目标库位的2D或者3D图像数据，基于这些图像数据对目标库位和目标料箱等进行检测，提高了检测精准度，从而可以更加准确地确定不满足搬运任务执行条件的情况，可以更好地避免危险情况发生，提高了仓储机器人的安全性。
另外，本发明实施例还提供一种计算机可读存储介质，计算机可读存储介质中存储有计算机程序，计算机程序被处理器执行时实现上述任一方法实施例提供的仓储机器人的控制方法。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本发明的其它实施方案。本发明旨在涵盖本发明的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本发明的一般性原理并包括本发明未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本发明的真正范围和精神由下面的权利要求书指出。
应当理解的是，本发明并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本发明的范围仅由所附的权利要求书来限制。

Claims (16)

  1. 一种仓储机器人的控制方法，所述仓储机器人具有搬运装置和图像采集装置，其特征在于，包括：
    通过所述图像采集装置采集搬运任务对应的目标库位的图像数据；
    根据所述目标库位的图像数据，若确定满足所述搬运任务的执行条件，则控制所述搬运装置执行所述搬运任务。
  2. 根据权利要求1所述的方法，其特征在于，所述通过所述图像采集装置采集搬运任务对应的目标库位的图像数据，包括：
    当所述仓储机器人移动至所述目标库位对应的目标位置时，控制所述图像采集装置启动并采集所述目标库位的图像数据；
    或者，
    当所述仓储机器人移动至所述目标库位周围的预设范围内时，控制所述图像采集装置启动并采集所述目标库位的图像数据。
  3. 根据权利要求2所述的方法，其特征在于，所述图像采集装置设置于所述搬运装置上，控制所述图像采集装置启动并采集所述目标库位的图像数据之前，还包括：
    控制所述搬运装置对准所述目标库位。
  4. 根据权利要求1所述的方法，其特征在于，所述根据所述目标库位的图像数据，若确定满足所述搬运任务的执行条件，则控制所述搬运装置执行所述搬运任务，包括：
    对所述目标库位的图像数据进行检测处理，确定所述目标库位的状态信息和/或目标物体的状态信息；
    根据所述目标库位的状态信息和/或目标物体的状态信息，若确定满足所述搬运任务的执行条件，则控制所述搬运装置执行所述搬运任务。
  5. 根据权利要求4所述的方法，其特征在于，所述目标库位的状态信息包括以下至少一项：
    所述目标库位的搬运路径上的障碍物信息；
    所述目标库位的尺寸信息；
    所述目标库位是否空闲。
  6. 根据权利要求5所述的方法，其特征在于，所述目标物体的状态信息 包括以下至少一项：
    所述目标物体的身份信息；
    所述目标物体的姿态信息；
    所述目标物体的尺寸信息；
    所述目标物体的破损程度信息；
    所述目标物体的形变程度信息。
  7. 根据权利要求6所述的方法，其特征在于，所述搬运任务为取货任务，所述搬运任务的执行条件包括以下至少一项：
    所述目标库位的取货路径上没有障碍物；
    所述目标物体的身份信息、姿态信息和尺寸信息满足取货条件；
    所述目标物体的破损程度在第一预设安全阈值范围内；
    所述目标物体的形变程度在第二预设安全阈值范围内。
  8. 根据权利要求6所述的方法，其特征在于，所述搬运任务为放货任务，所述搬运任务的执行条件包括以下至少一项：
    所述目标库位空闲；
    所述目标库位的尺寸满足放货条件；
    所述目标库位的放货路径上没有障碍物。
  9. 根据权利要求1-8中任一项所述的方法，其特征在于，还包括：
    根据所述目标库位的图像数据，若确定不满足所述搬运任务的执行条件，则向服务器发送错误信息，其中所述错误信息包括以下至少一项：所述目标库位的状态信息、目标物体的状态信息、未满足的执行条件项。
  10. 根据权利要求9所述的方法，其特征在于，向服务器发送错误信息之后，还包括：
    根据所述服务器的调度指示，控制所述仓储机器人执行对应的错误处理行为。
  11. 根据权利要求10所述的方法，其特征在于，所述错误处理行为是以下任意一种：
    停留在当前位置，等待指示；
    移动到目标点；
    跳过当前的所述搬运任务，执行下一个搬运任务。
  12. 根据权利要求1-8中任一项所述的方法，其特征在于，通过所述图像采集装置采集所述目标库位的图像数据，包括以下至少一项：
    通过第一拍摄装置采集所述目标库位的二维图像数据；
    通过第二拍摄装置采集所述目标库位的三维点云数据；
    通过激光雷达装置采集所述目标库位的二维点云数据。
  13. 根据权利要求1-8中任一项所述的方法，其特征在于，所述通过所述图像采集装置采集搬运任务对应的所述目标库位的图像数据之前，还包括：
    响应于搬运任务的执行指令，控制仓储机器人向所述目标库位移动。
  14. 一种仓储机器人的控制装置，其特征在于，应用于仓储机器人，所述仓储机器人包括搬运装置和图像采集装置，包括：
    数据获取模块，用于通过所述图像采集装置采集搬运任务对应的目标库位的图像数据；
    控制模块，用于根据所述目标库位的图像数据，若确定满足所述搬运任务的执行条件，则控制搬运装置执行所述搬运任务。
  15. 一种仓储机器人，其特征在于，包括：
    处理器，存储器，以及存储在所述存储器上并可在所述处理器上运行的计算机程序；
    其中，所述处理器运行所述计算机程序时实现如权利要求1至13中任一项所述的方法。
  16. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有计算机程序，所述计算机程序被处理器执行时实现如权利要求1至13中任一项所述的方法。
PCT/CN2021/102865 2020-06-12 2021-06-28 仓储机器人的控制方法、装置、设备及可读存储介质 WO2021249568A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2022576011A JP2023531391A (ja) 2020-06-12 2021-06-28 倉庫ロボットの制御方法、装置、機器、および読み取り可能記憶媒体
US18/064,609 US20230106134A1 (en) 2020-06-12 2022-12-12 Warehouse robot control method and apparatus, device, and readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010537646.9A CN111674817B (zh) 2020-06-12 2020-06-12 仓储机器人的控制方法、装置、设备及可读存储介质
CN202010537646.9 2020-06-12

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/064,609 Continuation US20230106134A1 (en) 2020-06-12 2022-12-12 Warehouse robot control method and apparatus, device, and readable storage medium

Publications (1)

Publication Number Publication Date
WO2021249568A1 true WO2021249568A1 (zh) 2021-12-16

Family

ID=72435544

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/102865 WO2021249568A1 (zh) 2020-06-12 2021-06-28 仓储机器人的控制方法、装置、设备及可读存储介质

Country Status (4)

Country Link
US (1) US20230106134A1 (zh)
JP (1) JP2023531391A (zh)
CN (2) CN114044298A (zh)
WO (1) WO2021249568A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114282841A (zh) * 2021-12-31 2022-04-05 广东利元亨智能装备股份有限公司 调度方法、装置、系统、控制设备及可读存储介质
CN115407355A (zh) * 2022-11-01 2022-11-29 小米汽车科技有限公司 库位地图的验证方法、装置及终端设备

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114044298A (zh) * 2020-06-12 2022-02-15 深圳市海柔创新科技有限公司 仓储机器人的控制方法、装置、设备及可读存储介质
CN112407726B (zh) * 2020-11-20 2022-07-08 深圳市海柔创新科技有限公司 货物存放方法、装置、机器人、仓储系统及存储介质
CN112407729A (zh) * 2020-11-20 2021-02-26 深圳市海柔创新科技有限公司 货物取放方法、装置、仓储机器人和仓储系统
CN112407722A (zh) * 2020-11-20 2021-02-26 深圳市海柔创新科技有限公司 货物存放空间异常处理方法、装置、设备及仓储系统
CN112429456B (zh) * 2020-11-20 2022-12-30 深圳市海柔创新科技有限公司 取出货物异常处理方法、装置、设备、系统及存储介质
CN114326740B (zh) * 2021-12-30 2023-06-27 杭州海康机器人股份有限公司 协同搬运处理方法、装置、电子设备及系统
CN116402895A (zh) * 2023-06-05 2023-07-07 未来机器人（深圳）有限公司 安全校验方法、无人叉车及存储介质
CN117430026B (zh) * 2023-12-20 2024-02-20 国网浙江省电力有限公司金华供电公司 基于5g技术仓位智能管理的智能行吊控制方法
CN117550273B (zh) * 2024-01-10 2024-04-05 成都电科星拓科技有限公司 基于蜂群算法的多搬运机器人协作方法
CN117592764B (zh) * 2024-01-18 2024-04-09 瑞熙(苏州)智能科技有限公司 出入库调度处理方法、装置、电子设备及可读存储介质

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN207810578U (zh) * 2017-12-26 2018-09-04 天津市天地申通物流有限公司 搬运机器人以及分拣系统
CN109071114A (zh) * 2017-09-05 2018-12-21 深圳蓝胖子机器人有限公司 一种自动装卸货的方法、设备及具有存储功能的装置
CN109154825A (zh) * 2016-07-28 2019-01-04 X开发有限责任公司 库存管理
CN109607031A (zh) * 2019-01-14 2019-04-12 青岛舍科技有限公司 基于无人机全景的智能仓储系统及方法
CN110015528A (zh) * 2018-01-10 2019-07-16 德国邮政股份公司 用于接收、临时储存以及发放运送物的运送物仓库以及方法
US10542219B2 (en) * 2017-01-20 2020-01-21 Toshiba Tec Kabushiki Kaisha Information collecting apparatus, information collecting system and information collecting method
CN210504192U (zh) * 2019-04-24 2020-05-12 深圳市海柔创新科技有限公司 智能仓储系统
CN111222827A (zh) * 2019-12-31 2020-06-02 云南电网有限责任公司楚雄供电局 货位管理方法、装置、存储介质及电子设备
CN111348361A (zh) * 2020-01-21 2020-06-30 深圳市海柔创新科技有限公司 取、放货控制方法、装置、搬运装置及搬运机器人
CN111674817A (zh) * 2020-06-12 2020-09-18 深圳市海柔创新科技有限公司 仓储机器人的控制方法、装置、设备及可读存储介质

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9300430B2 (en) * 2013-10-24 2016-03-29 Harris Corporation Latency smoothing for teleoperation systems
US9656806B2 (en) * 2015-02-13 2017-05-23 Amazon Technologies, Inc. Modular, multi-function smart storage containers
CN105835029A (zh) * 2016-05-30 2016-08-10 上海发那科机器人有限公司 具有区域移动能力的协作机器人及其工作方法
CN106276009B (zh) * 2016-08-11 2020-06-19 中国科学院宁波材料技术与工程研究所 全向移动搬运机器人
US10899015B2 (en) * 2017-09-01 2021-01-26 Siemens Aktiengesellschaft Method and system for dynamic robot positioning
CN108622590B (zh) * 2018-05-14 2019-12-13 福建中科兰剑智能装备科技有限公司 一种物流仓库用的智能运输机器人
CN109230148A (zh) * 2018-08-02 2019-01-18 李丹 基于机器人的无人化智能仓储系统
CN209023571U (zh) * 2018-09-07 2019-06-25 深圳市海柔创新科技有限公司 一种搬运机器人
CN109866201B (zh) * 2019-04-08 2021-03-30 清华大学 双目视觉系统、移动抓取机器人和自动取货方法
CN110482098B (zh) * 2019-07-18 2023-12-08 深圳市海柔创新科技有限公司 一种基于搬运机器人、系统的取放货方法
CN110421542B (zh) * 2019-08-02 2024-04-05 浙江创联信息技术股份有限公司 一种箱类包裹装卸智能机器人
CN111232524B (zh) * 2020-03-09 2023-06-13 深圳市海柔创新科技有限公司 一种控制搬运机器人的方法、装置及搬运机器人

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109154825A (zh) * 2016-07-28 2019-01-04 X开发有限责任公司 库存管理
US10542219B2 (en) * 2017-01-20 2020-01-21 Toshiba Tec Kabushiki Kaisha Information collecting apparatus, information collecting system and information collecting method
CN109071114A (zh) * 2017-09-05 2018-12-21 深圳蓝胖子机器人有限公司 一种自动装卸货的方法、设备及具有存储功能的装置
CN207810578U (zh) * 2017-12-26 2018-09-04 天津市天地申通物流有限公司 搬运机器人以及分拣系统
CN110015528A (zh) * 2018-01-10 2019-07-16 德国邮政股份公司 用于接收、临时储存以及发放运送物的运送物仓库以及方法
CN109607031A (zh) * 2019-01-14 2019-04-12 青岛舍科技有限公司 基于无人机全景的智能仓储系统及方法
CN210504192U (zh) * 2019-04-24 2020-05-12 深圳市海柔创新科技有限公司 智能仓储系统
CN111222827A (zh) * 2019-12-31 2020-06-02 云南电网有限责任公司楚雄供电局 货位管理方法、装置、存储介质及电子设备
CN111348361A (zh) * 2020-01-21 2020-06-30 深圳市海柔创新科技有限公司 取、放货控制方法、装置、搬运装置及搬运机器人
CN111674817A (zh) * 2020-06-12 2020-09-18 深圳市海柔创新科技有限公司 仓储机器人的控制方法、装置、设备及可读存储介质

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114282841A (zh) * 2021-12-31 2022-04-05 广东利元亨智能装备股份有限公司 调度方法、装置、系统、控制设备及可读存储介质
WO2023125076A1 (zh) * 2021-12-31 2023-07-06 广东利元亨智能装备股份有限公司 调度方法、装置、系统、控制设备及可读存储介质
CN115407355A (zh) * 2022-11-01 2022-11-29 小米汽车科技有限公司 库位地图的验证方法、装置及终端设备
CN115407355B (zh) * 2022-11-01 2023-01-10 小米汽车科技有限公司 库位地图的验证方法、装置及终端设备

Also Published As

Publication number Publication date
CN114044298A (zh) 2022-02-15
JP2023531391A (ja) 2023-07-24
US20230106134A1 (en) 2023-04-06
CN111674817A (zh) 2020-09-18
CN111674817B (zh) 2021-12-17

Similar Documents

Publication Publication Date Title
WO2021249568A1 (zh) 仓储机器人的控制方法、装置、设备及可读存储介质
US11288810B2 (en) Robotic system with automated package registration mechanism and methods of operating the same
CN110054121B (zh) 一种智能叉车以及容器位姿偏移检测方法
KR101699672B1 (ko) 컨테이너 크레인을 사용하여 랜딩 타깃 상에 컨테이너들을 자동적으로 랜딩하기 위한 방법 및 시스템
US10860855B2 (en) Instruction projecting device, package sorting system and instruction projecting method
JP2023529878A (ja) コンテナの取り出し方法、装置、システム、ロボットおよび記憶媒体
US20230410362A1 (en) Target object detection method and apparatus, and electronic device, storage medium and program
CN114241037A (zh) 混合尺寸卸盘
JP2022181174A (ja) 回転補正付きの物体ビンピッキング
CN114170442A (zh) 机器人空间抓取点的确定方法及装置
US20190099784A1 (en) Projection instruction device, parcel sorting system, and projection instruction method
Mrovlje et al. Automatic detection of the truck position using stereoscopy
WO2024053150A1 (ja) ピッキングシステム
CN116385533A (zh) 一种基于二维和三维成像的叉式agv目标位姿检测方法
KR20230174126A (ko) 디팔레타이징 방법
US10635869B2 (en) Projection instruction device, parcel sorting system, and projection instruction method
CN117474892A (zh) 货架识别方法、移动机器人及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21821613

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022576011

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21821613

Country of ref document: EP

Kind code of ref document: A1