CN114355869A

CN114355869A - AGV trolley control method and device, electronic equipment and storage medium

Info

Publication number: CN114355869A
Application number: CN202011031273.4A
Authority: CN
Inventors: 王浩; 高坚; 吴登禄
Original assignee: SF Technology Co Ltd
Current assignee: SF Technology Co Ltd
Priority date: 2020-09-27
Filing date: 2020-09-27
Publication date: 2022-04-15

Abstract

The application provides a method and a device for controlling an AGV (automatic guided vehicle), electronic equipment and a storage medium, wherein the method for controlling the AGV comprises the following steps: detecting whether a target bottom mark matched with a target cage car exists above the AGV car by using a first camera; if the first camera detects that a target bottom mark matched with the target cage car exists above the AGV car, determining first relative pose information and preset relative pose information of the AGV car and the target cage car based on the target bottom mark; when the first relative pose information does not meet the preset relative pose information, adjusting the relative pose information of the AGV trolley and the target cage trolley to the preset relative pose information; and controlling the AGV to lift the target cage car. According to the method and the device, the AGV trolley is adjusted when the relative pose of the AGV trolley and the target cage trolley does not meet the requirements, the AGV trolley and the target cage trolley are accurately positioned, and the safety of the AGV trolley during lifting of the cage trolley is improved.

Description

AGV trolley control method and device, electronic equipment and storage medium

Technical Field

The application relates to the technical field of AGV control, in particular to a method and a device for controlling an AGV, electronic equipment and a storage medium.

Background

In recent years, with the rapid development of electronic commerce, online shopping is becoming more popular. "Shuangele" is one of a large festival of pulling the electric commerce sales volume every year, and a store and an electric commerce platform launch a series of preferential measures, and consumers buy the electric commerce platform too soon this day, and a large amount of quick packages are stocked in the period. China becomes a big express country, and the billion breaking time of package delivery is shortened from 48 hours in 2013 to 12 hours in 2017. Cats only spend 107 minutes in 2018 to break billions. The traditional material transportation system mainly comprises a forklift, a trailer, a conveyor belt, various lifting machines and the like, is low in transportation efficiency, poor in equipment flexibility, low in safety and high in degree of dependence on personnel, and cannot well meet the requirement of modern enterprises on high production efficiency.

When goods are stored and transported, the goods need to be transported to a designated position by using two hands or a cart; neither of these scenarios frees up both hands, which can increase the workload of the dispenser. Therefore, in the logistics distribution field, make the commodity circulation car can transport the cage that fills with the goods to appointed position automatically, will promote the efficiency of production transportation by a wide margin. The cage car butt joint is an indispensable part of the cage car butt joint to complete the function of automatically carrying goods. The cage car docking refers to controlling an Automated Guided Vehicle (AGV for short, also commonly referred to as an AGV car) to move to the bottom of a cage car, lifting the cage car and moving the cage car to a designated position. However, when AGV moved to the cage bottom, can't carry out accurate location to the cage car, the cage car took place to empty easily behind the AGV dolly lifting cage car.

That is, AGV dolly can't carry out accurate location to the cage car after getting into cage car bottom among the prior art, and the security when AGV dolly lifts the cage car is not high.

Disclosure of Invention

The application aims at providing a control method and device of an AGV, electronic equipment and a storage medium, and aims at solving the problems that in the prior art, the AGV cannot accurately position a cage after entering the bottom of the cage, and the safety of the AGV lifts the cage is not high.

On one hand, the AGV is used for carrying cage cars in a preset scene, a bottom mark is arranged at the bottom of a chassis of each cage car, goods are loaded above the chassis of each cage car, an accommodating space for accommodating the AGV is arranged at the bottom of the chassis of each cage car, and a first camera is arranged at the top of the AGV and used for taking pictures above the AGV; the AGV control method comprises the following steps:

detecting whether a target bottom mark matched with a target cage car exists above the AGV car by using the first camera;

if the first camera detects that a target bottom mark matched with the target cage car exists above the AGV car, determining first relative pose information and preset relative pose information of the AGV car and the target cage car based on the target bottom mark;

when the first relative pose information does not meet the preset relative pose information, adjusting the relative pose information of the AGV trolley and the target cage trolley to the preset relative pose information;

and controlling the AGV to lift the target cage car.

The lateral side of the cage trolley is provided with a lateral part mark, the lateral side of the AGV trolley is provided with a second camera, and the second camera is used for photographing the lateral side of the AGV trolley;

utilize whether the top that first camera detected the AGV dolly has the target bottom sign that matches with target cage car, include:

detecting whether a target side part identification matched with the target cage car exists in the preset scene by using the second camera;

and if the second camera detects that the target side part identification matched with the target cage car does not exist in the preset scene, the first camera is utilized to detect whether the target bottom identification matched with the target cage car exists above the AGV.

The AGV control method further comprises the following steps:

if the second camera detects that a target side identification matched with the target cage car exists in the preset scene, second relative position and posture information of the target side identification and the AGV car, and third relative position and posture information of the target side identification and the target cage car prestored in the target side identification are obtained based on the target side identification;

determining fourth relative position information of the AGV trolley and the target cage trolley based on the second relative position information and the third relative position information;

and controlling the AGV to move a preset distance to the target cage car based on the fourth relative attitude information.

Wherein, the determining the first relative pose information and the preset relative pose information of the AGV and the target cage based on the target bottom mark comprises:

acquiring fifth relative pose information of the target bottom identifier and the AGV based on the target bottom identifier, and sixth relative pose information of the target cage trolley and the target bottom identifier pre-stored in the target bottom identifier;

determining the first relative pose information based on the fifth relative pose information and the sixth relative pose information.

Acquiring the target bottom identifier and fifth relative pose information of the AGV car based on the target bottom identifier, wherein the sixth relative pose information of the target cage car and the target bottom identifier pre-stored in the target bottom identifier comprises:

performing information identification on the target bottom identifier to acquire first position and orientation information of the target bottom identifier in a bottom identifier coordinate system and second position and orientation information of the target bottom identifier in a first camera image coordinate system;

acquiring a preset conversion relation between a first camera coordinate system and an AGV body coordinate system;

determining third pose information of the bottom identifier under the first camera coordinate system based on the first pose information and the second pose information;

and determining the fifth relative pose information based on the third pose information and the preset conversion relation.

The top of the AGV trolley is also provided with a lighting device;

controlling the lighting device to be turned on;

and detecting whether a target bottom mark matched with the target cage car exists above the AGV car by using the second camera.

Wherein the area of the bottom mark is larger than the area of the side mark.

The detecting, by using the second camera, whether a target side identifier matched with the target cage car exists in the preset scene includes:

acquiring request information for carrying the target cage car, wherein the request information comprises cage car information corresponding to the target cage car;

detecting whether the side part identification exists in the preset scene by using the second camera;

if the second camera detects that the side part identification exists in the preset scene, the cage information stored in the detected side part identification is matched with the cage information of the target cage so as to determine whether the target side part identification matched with the target cage exists in the preset scene.

On one hand, the AGV is used for carrying cage cars in a preset scene, a bottom mark is arranged at the bottom of a chassis of each cage car, goods are loaded above the chassis of each cage car, an accommodating space for accommodating the AGV is arranged at the bottom of the chassis of each cage car, and a first camera is arranged at the top of the AGV and used for taking pictures above the AGV; the AGV controlling means includes:

the detection unit is used for detecting whether a target bottom mark matched with a target cage car exists above the AGV car by using the first camera;

the determining unit is used for determining first relative pose information and preset relative pose information of the AGV trolley and the target cage trolley based on the target bottom mark if the second camera detects that the target bottom mark matched with the target cage trolley exists above the AGV trolley;

the adjusting unit is used for adjusting the relative pose information of the AGV trolley and the target cage trolley to the preset relative pose information when the first relative pose information does not meet the preset relative pose information;

a control unit for controlling the AGV to lift the target cage car

the detection unit is used for detecting whether a target side part identifier matched with the target cage car exists in the preset scene by using the second camera;

The detection unit is further configured to, if the second camera detects that a target side identifier matched with the target cage car exists in the preset scene, obtain, based on the target side identifier, second relative position and orientation information of the target side identifier and the AGV car, and third relative position and orientation information of the target side identifier and the target cage car, which are pre-stored in the target side identifier;

The determining unit is configured to acquire, based on the target bottom identifier, fifth relative pose information of the target bottom identifier and the AGV, and sixth relative pose information of the target bottom identifier and the target cage car, which are pre-stored in the target bottom identifier;

The determining unit is used for performing information identification on the target bottom identifier to acquire first position and attitude information of the target bottom identifier in a bottom identifier coordinate system and second position and attitude information of the target bottom identifier in a first camera image coordinate system;

The top of the AGV trolley is also provided with a lighting device;

the detection unit is used for controlling the lighting device to be turned on;

Wherein the area of the bottom mark is larger than the area of the side mark.

The detection unit is used for acquiring request information for carrying the target cage car, wherein the request information comprises cage car information corresponding to the target cage car;

In one aspect, the present application further provides an electronic device, including:

one or more processors;

a memory; and

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the method of controlling an AGV of any of the first aspects.

In one aspect, the present application further provides a computer readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the steps of the method for controlling an AGV cart according to any one of the first aspect.

The application provides a control method of many AGV dollies, set up first camera at the top of AGV dolly, the chassis bottom of cage car sets up the bottom sign, detect the bottom sign of cage car bottom through first camera, and confirm according to the bottom sign whether the relative position appearance of AGV dolly and target cage car meets the requirements, thereby adjust the AGV dolly when the relative position appearance of AGV dolly and target cage car does not meet the requirements, the realization is to the accurate positioning of AGV dolly and target cage car, the security when the AGV dolly lifts the cage car is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic diagram illustrating a scenario of an AGV control system according to an embodiment of the present disclosure;

FIG. 2 is a flowchart illustrating an exemplary method for controlling an AGV according to an exemplary embodiment of the present disclosure;

FIG. 3 is a schematic flow chart diagram illustrating another exemplary method for controlling an AGV according to an exemplary embodiment of the present disclosure;

FIG. 4 is a schematic diagram of an exemplary AGV control system according to an exemplary embodiment of the present disclosure;

fig. 5 is a schematic structural diagram of an embodiment of an electronic device provided in the embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In the description of the present application, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like indicate orientations or positional relationships based on those shown in the drawings, and are used merely for convenience of description and for simplicity of description, and do not indicate or imply that the referenced device or element must have a particular orientation, be constructed in a particular orientation, and be operated, and thus should not be considered as limiting the present application. Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", may explicitly or implicitly include one or more of the described features. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.

In this application, the word "exemplary" is used to mean "serving as an example, instance, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the application. In the following description, details are set forth for the purpose of explanation. It will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In other instances, well-known structures and processes are not set forth in detail in order to avoid obscuring the description of the present application with unnecessary detail. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

It should be noted that, since the method in the embodiment of the present application is executed in the electronic device, the processing objects of each electronic device all exist in the form of data or information, for example, time, which is substantially time information, and it is understood that, if the size, the number, the position, and the like are mentioned in the following embodiments, all corresponding data exist so as to be processed by the electronic device, and details are not described herein.

Embodiments of the present application provide a method and an apparatus for controlling an AGV, an electronic device, and a storage medium, which are described in detail below.

Referring to fig. 1, fig. 1 is a schematic view of a control system of an AGV according to an embodiment of the present disclosure, where the control system of an AGV may include an electronic device 100, and a control device of the AGV, such as the electronic device in fig. 1, is integrated in the electronic device 100.

Wherein, the controlling means of AGV passes through internet access with the AGV dolly to make the controlling means of AGV can carry out information interaction with the AGV dolly, thereby control the AGV dolly.

In this embodiment of the application, the electronic device 100 may be an independent server, or may be a server network or a server cluster composed of servers, for example, the electronic device 100 described in this embodiment of the application includes, but is not limited to, a computer, a network host, a single network server, multiple network server sets, or a cloud server composed of multiple servers. Among them, the Cloud server is constituted by a large number of computers or web servers based on Cloud Computing (Cloud Computing).

Those skilled in the art will appreciate that the application environment shown in fig. 1 is only one application scenario of the present application, and does not constitute a limitation to the application scenario of the present application, and that other application environments may further include more or less electronic devices than those shown in fig. 1, for example, only 1 electronic device is shown in fig. 1, and it is understood that the control system of the AGV cart may further include one or more other electronic devices, which is not limited herein.

In addition, as shown in FIG. 1, the AGV control system may further include a memory 200 for storing data, such as preset relative pose information.

It should be noted that the scenario diagram of the control system of the AGV cart shown in fig. 1 is only an example, and the control system and the scenario of the AGV cart described in the embodiment of the present application are for more clearly illustrating the technical solution of the embodiment of the present application, and do not form a limitation on the technical solution provided in the embodiment of the present application.

First, an embodiment of the present application provides a method for controlling an AGV, where an execution main body of the method is a control device of the AGV, the control device of the AGV is applied to an electronic device, and the method includes:

detecting whether a target bottom mark matched with a target cage car exists above the AGV car by using a first camera;

and controlling the AGV to lift the target cage car.

Referring to fig. 2, fig. 2 is a schematic flowchart illustrating an embodiment of a method for controlling an AGV according to an embodiment of the present disclosure. As shown in FIG. 2, the AGV control method comprises the following steps:

s201, detecting whether a target bottom mark matched with a target cage car exists above the AGV car by using a first camera.

The AGV dolly is arranged in the transport to predetermine cage car in the scene, can have 1 or a plurality of cage car in predetermineeing the scene, and the chassis top of cage car is used for bearing the weight of the goods, and the chassis bottom of cage car is equipped with the accommodation space who holds the AGV dolly, thereby AGV can get into cage car bottom and lift the cage car. The bottom of the chassis of the cage vehicle is provided with a bottom mark. The bottom mark can be a pattern mark such as a two-dimensional code or a bar code capable of storing information. Two-dimensional codes, also known as two-dimensional bar codes, are an encoding mode which is super-popular on mobile devices in recent years. The two-dimensional bar code/two-dimensional code is a pattern which is distributed on a plane according to a certain rule by using a certain specific geometric figure, is black and white and is alternated and records data symbol information; the concept of '0' and '1' bit stream which forms the internal logic base of computer is skillfully utilized in coding, a plurality of geometric shapes corresponding to binary system are used for representing literal numerical information, and the information is automatically read by an image input device or an optoelectronic scanning device so as to realize the automatic processing of the information: it has some commonality of barcode technology: each code system has its specific character set; each character occupies a certain width; has certain checking function and the like. Meanwhile, the method also has the function of automatically identifying information of different rows and processing the graph rotation change points. A bar code (barcode) is a graphic identifier in which a plurality of black bars and spaces having different widths are arranged according to a certain coding rule to express a set of information. A common bar code is a pattern of parallel lines of black and white bars of widely differing reflectivity. The bar code can mark many information such as the producing country, the manufacturer, the commodity name, the production date, the book classification number, the starting and stopping place of the mail, the class, the date and the like of the article, so the bar code is widely applied to many fields such as commodity circulation, book management, postal management, bank systems and the like.

In a specific embodiment, the bottom mark is a two-dimensional code. For example, the bottom mark is a square two-dimensional code with 0.03m by 0.03m, and the bottom mark is attached to the center of the bottom of the cage chassis, so that the AGV trolley can enter the bottom of the cage from any direction and can quickly detect the bottom mark. Of course, in other embodiments, the shape of the bottom mark may also be other shapes such as a rectangle, a triangle, and a circle, the size of the bottom mark may also be other sizes, and the bottom mark may also be attached to other positions of the bottom of the cage car chassis, which is not limited in this application.

In the embodiment of the application, the top of the AGV trolley is provided with a first camera which can be a monocular camera, a binocular camera and the like. Preferably, the first camera is arranged at the center of the top of the AGV trolley, so that when the AGV trolley is positioned under the cage trolley chassis, the first camera can still detect the bottom mark, and accurate positioning is guaranteed.

In the embodiment of the application, the bottom identification can store cage information of a cage to which the bottom identification belongs. The cage information can be the code, height, cargo capacity and the like of the cage. For convenience of description, a square two-dimensional code with a bottom mark L × L is taken as an example for description, for example, L is 0.03mm, and L × L is the real size of the square two-dimensional code in a world coordinate system, and can be obtained by measurement in advance and stored in the bottom mark.

In this application embodiment, whether the top that utilizes first camera to detect the AGV dolly has the target bottom sign that matches with the target cage car, can include: cage information corresponding to the target cage vehicle is acquired, and a chassis of the cage vehicle is scanned by using a first camera so as to detect whether a bottom mark exists above the AGV. The cage information corresponding to the target cage vehicle may include a code, a height, a cargo capacity, and the like of the target cage vehicle, for example, the code of the target cage vehicle is cage vehicle a. And if the bottom mark exists above the AGV trolley, matching the cage information stored in the scanned bottom mark with the cage information corresponding to the target cage to determine whether the target bottom mark matched with the target cage exists above the AGV trolley. And if the bottom mark does not exist above the AGV trolley, sending prompt information to prompt a worker to check or finish the current task.

In this application embodiment, the cage car information that stores in the bottom sign that will scan obtained matches with the cage car information that the target cage car corresponds to whether there is the target bottom sign that matches with the target cage car in the top of confirming the AGV dolly, can include: matching the cage information stored in the scanned bottom identification with cage information corresponding to a target cage, if the cage information stored in the scanned bottom identification is the same as the cage information corresponding to the target cage, judging that a target bottom identification matched with the target cage exists above the AGV, and determining the scanned bottom identification as the target bottom identification; and if the cage car information stored in the bottom identification obtained by scanning is different from the cage car information corresponding to the target cage car, judging that the target bottom identification matched with the target cage car does not exist above the AGV.

Furthermore, the top of the AGV trolley is also provided with a lighting device. The lighting device may be an LED lamp. Whether a target bottom mark matched with a target cage car exists above the AGV car is detected by using the first camera, and the method comprises the following steps: controlling the lighting device to be turned on; and detecting whether a target bottom mark matched with the target cage car exists above the AGV car by using a first camera. That is, reach cage bottom of the car before the AGV dolly and detect the bottom sign and carry out the light filling through opening lighting device, can make first camera clearly scan the bottom sign to further improve the discernment rate of accuracy, and then improve the location precision.

S202, if the first camera detects that a target bottom mark matched with the target cage car exists above the AGV, determining first relative pose information and preset relative pose information of the AGV and the target cage car based on the target bottom mark.

In the embodiment of the application, the pose information includes position information and posture information. The position information represents the displacement of 3 degrees of freedom, and the attitude information represents the spatial Rotation (Rotation) of 3 degrees of freedom, which together are called the Pose (position).

The first relative pose information is the current relative pose of the AGV trolley and the target cage trolley, and the preset relative pose information is the preset relative pose enabling the AGV trolley to lift the cage trolley. The preset relative pose information can be set according to specific conditions, for example, the preset relative pose information can be that the center of the AGV trolley and the center of the cage trolley are located on the same vertical line, the preset relative pose information can be that the distance between the center of the AGV trolley and the center of the cage trolley is smaller than a preset value, and the like, and the method is not limited in the application.

In this embodiment of the application, determining the first relative pose information of the AGV and the target cage based on the target bottom identifier may include: acquiring fifth relative pose information of the bottom mark and the AGV based on the target bottom mark, and sixth relative pose information of the target cage car and the target bottom mark prestored in the bottom mark; and determining first relative pose information of the AGV trolley and the target cage trolley based on the fifth relative pose information and the sixth relative pose information. For example, the fifth relative pose information is the relative pose of the target bottom marker with respect to the AGV cart, represented by matrix a; the sixth relative pose information is the relative pose of the target cage car relative to the target bottom mark and is represented by a matrix B; and adding the relative pose of the target bottom mark relative to the AGV trolley and the relative pose of the target cage trolley relative to the target bottom mark to obtain first relative pose information, wherein the first relative pose information is (A + B).

And determining sixth relative position information of the target bottom mark and the target cage car according to the installation position of the target bottom mark on the target cage car. When the bottom mark is attached to the cage car, the relative pose information of the bottom mark and the cage car is calculated and stored in the bottom mark.

In a specific embodiment, the obtaining, based on the target bottom identifier, fifth relative pose information of the target bottom identifier and the AGV, and sixth relative pose information of the target cage cart and the target bottom identifier pre-stored in the target bottom identifier may include the following steps:

(1) and performing information identification on the target bottom mark to acquire first position and attitude information of the target bottom mark in a bottom mark coordinate system and second position and attitude information of the target bottom mark in a first camera image coordinate system.

The image coordinate system (Pixel coordinate system) is used to store digital images collected by a camera in a computer as an array, and the value of each element (Pixel) in the array is the brightness (gray scale) of an image point. A rectangular coordinate system u-v is defined on the image, and the coordinates (u, v) of each pixel are the column number and the row number of the pixel in the array respectively. Therefore, (u, v) is the image coordinate system coordinate in pixel units. The first camera image coordinate system is an image coordinate system of the first camera.

The AGV body coordinate system can be a coordinate system established by taking the AGV body center as an original point, and can describe the relative position of a target object and the AGV body center in the real world.

Camera coordinate system (camera coordinate system): the coordinate system established on the camera is defined to describe the object's position from the camera's perspective as the middle loop that communicates the AGV body coordinate system with the image coordinate system. The first camera coordinate system is a camera coordinate system of the first camera.

The bottom mark coordinate system can be a three-dimensional coordinate system established by taking the center of the target bottom mark as an origin, and can describe the relative pose of the target object and the target bottom mark in the real world. Specifically, the first pose information includes three-dimensional coordinate information of at least three reference points in the target bottom mark in a bottom mark coordinate system. For example, a bottom mark coordinate system is established with the center of the target bottom mark as an origin and two adjacent sides of the bottom mark as XY axes. The three-dimensional coordinates of the 4 end points of the target bottom mark in the bottom mark coordinate system are shown in formula (1),

wherein the content of the first and second substances,

are the 4 endpoints identified at the bottom of the target.

The second position and posture information of the target bottom mark in the first camera image coordinate system comprises two-dimensional coordinate information of at least three reference points in the target bottom mark in the first camera image coordinate system. For example, the two-dimensional coordinate information of at least three reference points in the target bottom mark under the first camera image coordinate system is respectivelyIs composed of

So that 4 groups of 3D-2D matching pairs can be obtained

(2) And acquiring a preset conversion relation between the first camera coordinate system and the AGV body coordinate system.

The method comprises the steps of determining a preset conversion relation between a first camera coordinate system and an AGV body coordinate system in advance according to the installation pose of a first camera on the AGV, storing the preset conversion relation in a control device of the AGV, and reading the preset conversion relation between the first camera coordinate system and the AGV body coordinate system. For example, the predetermined transformation relationship between the first camera coordinate system and the AGV body coordinate system includes a first rotation matrix

And a first translation matrix

For rotation of the first camera relative to the body center of the AGV cart,

is the translation of the first camera relative to the center of the body of the AGV cart.

(3) And determining third attitude information of the target bottom identifier under the camera coordinate system based on the first attitude information and the second attitude information.

Specifically, third pose information of the bottom mark under a camera coordinate system is determined based on the first pose information and the second pose information through a preset camera pose estimation algorithm. The preset camera pose estimation algorithm can be epipolar geometry, PnP, ICP and other algorithms. The PnP (peer-n-point) problem is that a known point pair corresponding to n spatial 3D points and an image 2D point calculates a camera pose or an object pose, and the two are equivalent. The PnP problem solves the problem of 3D-2D camera pose estimation when the positions of map points under a world reference system and projection points under a camera reference system are known, can obtain better motion estimation in fewer matching points (at least 3 points, a P3P method), and is the most important pose estimation method. If the map points under the world reference frame are known, and the map points under the camera reference frame are known at the same time, the attitude can be solved by an ICP method. This constitutes the most important epipolar geometry in stereo vision, the three most commonly used pose estimation methods of PnP and ICP.

For example, the third pose information identified at the bottom of the target in the first camera coordinate system by the PNP algorithm may be: position t_m→cAnd attitude R_m→c。

(4) And determining fifth relative pose information based on the third pose information and a preset conversion relation.

Specifically, the fifth relative pose information includes a position t of the bottom marker relative to the AGV_tarAnd attitude R_tar. The fifth relative pose information is determined by formula (2) and formula (3).

And S203, when the first relative pose information does not meet the preset relative pose information, adjusting the relative pose information of the AGV trolley and the target cage trolley to the preset relative pose information.

In this embodiment of the application, when the first relative pose information does not satisfy the preset relative pose information, it indicates that the relative pose information of the AGV car and the target cage car does not satisfy the preset requirement, and further positioning is required, the relative pose information of the AGV car and the target cage car is adjusted to the preset relative pose information, and S204 is executed when the first relative pose information satisfies the preset relative pose information. When the first relative pose information meets the preset relative pose information, it indicates that the relative pose information of the AGV car and the target cage car meets the preset requirement, and S204 may be directly executed.

Specifically, the AGV trolley is controlled to adjust the self pose, so that the relative pose information of the AGV trolley and the target cage trolley is adjusted to the preset relative pose information. For example, the center position of the AGV car is used as a starting point, and the center position of the target cage car is used as an end point to navigate the AGV, so that the AGV car is controlled to adjust the self pose, or the AGV car is controlled to rotate, incline and the like to adjust the self pose. Of course, in other embodiments, the relative pose information between the AGV trolley and the target cage trolley may be adjusted to the preset relative pose information by controlling the cage trolley to move, which is not limited in this application.

And S204, controlling the AGV to lift the target cage car.

In this application implementation, be equipped with the lifting device of liftable on the AGV dolly, control the AGV dolly and rise the lifting device in order to lift target cage car.

Referring to fig. 3, fig. 3 is a schematic flowchart of another embodiment of a method for controlling an AGV according to an embodiment of the present application. As shown in FIG. 3, the AGV control method comprises the following steps:

s301, detecting whether a target side part identification matched with the target cage car exists in a preset scene by using a second camera.

In the embodiment of the application, the side of the cage car is provided with the lateral part identification, the side of the AGV car is provided with the second camera, and the second camera is used for photographing the side of the AGV car. The second camera may be the same as or different from the first camera, and is not limited herein. Wherein, the second camera can be 1 also can be a plurality of, for example, the quantity of second camera is 4, sets up respectively on four sides of AGV dolly to can realize no dead angle scanning to AGV dolly environment all around.

In a specific embodiment, the side portion is identified as a two-dimensional code. For example, the square two-dimensional code with the side marks of 0.08m by 0.08m has 4 side marks, and the side marks are attached to four sides of the cage car, so that the AGV can quickly detect the side marks from any one direction of the sides of the cage car. Of course, in other embodiments, the shape of the bottom mark may be other shapes such as a rectangle, a triangle, and a circle, and the size of the side mark may also be other sizes, which is not limited in this application. The side part identification can store cage car information of the cage car to which the side part identification belongs. The cage information can be the code, height, cargo capacity and the like of the cage.

In this application embodiment, whether there is a target side identifier matching with the target cage car in the preset scene by using the second camera to detect may include:

acquiring a request instruction for carrying a target cage car, wherein the request information comprises cage car information corresponding to the target cage car; detecting whether a side part identifier exists in a preset scene by using a second camera; and if the second camera detects that the side part identification exists in the preset scene, matching the cage information stored in the detected side part identification with the cage information of the target cage to determine whether the target side part identification matched with the target cage exists in the preset scene. And if the second camera detects that the side part identification does not exist in the preset scene, sending out prompt information to prompt a worker to check.

Specifically, the cage car information that will detect the storage in the obtained lateral part sign matches with the cage car information of target cage car to confirm whether have the target lateral part sign that matches with target cage car in presetting the scene, include: matching the cage car information stored in the detected side part identification with cage car information corresponding to the target cage car, if the cage car information stored in the detected side part identification is the same as the cage car information corresponding to the target cage car, judging that a target side part identification matched with the target cage car exists in a preset scene, and determining the detected side part identification as the target side part identification; and if the cage car information stored in the detected side part identification is different from the cage car information corresponding to the target cage car, judging that the target side part identification matched with the target cage car does not exist in the preset scene.

In the embodiment of the application, if a target side part identifier matched with a target cage car exists in a preset scene, executing S302; if the target side part identifier matched with the target cage car does not exist in the preset scene, it indicates that the target cage car may have entered the bottom of the cage car, and at this time, the second camera cannot function, then S305 is executed.

S302, second relative position information of the target side identification and the AGV trolley, third relative position information of the target side identification and the target cage trolley, which are prestored in the target side identification, are obtained based on the target side identification.

In this embodiment of the application, the step of obtaining the second relative pose information of the side identifier and the AGV based on the side identifier may refer to the step of obtaining the fifth relative pose information of the bottom identifier and the AGV based on the bottom identifier in S202, which is not described herein again.

S303, determining fourth relative position information of the AGV trolley and the target cage trolley based on the second relative position information and the third relative position information.

And S304, controlling the AGV to move to the target cage car for a preset distance based on the fourth relative attitude information.

The preset distance is set according to specific conditions, such as 0.1m, 0.2m, and the like. And after the AGV trolley is controlled to move to the target cage trolley for the preset distance based on the fourth relative pose information, the relative poses of the AGV trolley and the target cage trolley are changed, S301 is repeatedly executed, and the action route of the AGV trolley can be corrected in real time.

S305, detecting whether a target bottom mark matched with a target cage car exists above the AGV car by using a first camera.

In this embodiment of the application, S305 is the same as S201, and the specific implementation process of S201 may be referred to for the implementation of S305, which is not described herein again.

In a preferred embodiment, the area of the bottom indicator is smaller than the area of the side indicator. For example, a square two-dimensional code with a bottom labeled 0.03m by 0.03m and a side labeled 0.08m by 0.08 m. The area of the bottom mark is smaller than that of the side mark, so that the AGV trolley can detect the complete bottom mark as soon as possible after entering the bottom of the cage trolley.

S306, if the first camera detects that a target bottom mark matched with the target cage car exists above the AGV car, determining first relative pose information and preset relative pose information of the AGV car and the target cage car based on the target bottom mark.

In this embodiment of the application, S306 is the same as S202, and the specific implementation process of S202 may be referred to for implementation of S306, which is not described herein again.

And S307, when the first relative pose information does not meet the preset relative pose information, adjusting the relative pose information of the AGV trolley and the target cage trolley to the preset relative pose information.

In this embodiment of the application, S307 is the same as S203, and the specific implementation process of S203 may be referred to for implementation of S307, which is not described herein again.

And S308, controlling the AGV to lift the target cage car.

In this embodiment of the application, S308 is the same as S204, and the specific implementation process of S204 may be referred to for implementation of S308, which is not described herein again.

In order to better implement the method for controlling an AGV according to an embodiment of the present application, based on the method for controlling an AGV, an embodiment of the present application further provides a device for controlling an AGV, as shown in fig. 4, fig. 4 is a schematic structural diagram of an embodiment of the device for controlling an AGV according to an embodiment of the present application, and the device for controlling an AGV includes:

the detection unit 401 is configured to detect whether a target bottom identifier matched with a target cage car exists above the AGV by using a first camera;

a determining unit 402, configured to determine, based on a target bottom identifier, first relative pose information and preset relative pose information of the AGV trolley and a target cage trolley if a target bottom identifier matching the target cage trolley exists above the AGV trolley detected by the second camera;

an adjusting unit 403, configured to adjust the relative pose information of the AGV car and the target cage car to the preset relative pose information when the first relative pose information does not satisfy the preset relative pose information;

a control unit 404 for controlling the AGV to lift the target cage car

a detection unit 401, configured to detect whether a target side identifier matching the target cage car exists in a preset scene by using a second camera;

and if the second camera detects that the target side part identification matched with the target cage car does not exist in the preset scene, the first camera is used for detecting whether the target bottom identification matched with the target cage car exists above the AGV.

The detection unit 401 is further configured to, if the second camera detects that a target side identifier matching the target cage car exists in the preset scene, obtain, based on the target side identifier, second relative position and orientation information of the target side identifier and the AGV car, and third relative position and orientation information of the target cage car and a target side identifier pre-stored in the target side identifier;

and controlling the AGV to move to the target cage car for a preset distance based on the fourth relative attitude information.

The determining unit 402 is configured to obtain, based on the target bottom identifier, a fifth relative pose information of the target bottom identifier and the AGV, and a sixth relative pose information of the target cage cart and a target bottom identifier pre-stored in the target bottom identifier;

determining first relative pose information based on the fifth relative pose information and the sixth relative pose information.

The determining unit 402 is configured to perform information identification on the target bottom identifier, and acquire first position and orientation information of the target bottom identifier in a bottom identifier coordinate system and second position and orientation information of the target bottom identifier in a first camera image coordinate system;

determining third position information of the bottom identifier under the first camera coordinate system based on the first position information and the second position information;

and determining fifth relative pose information based on the third pose information and a preset conversion relation.

Wherein, the top of the AGV trolley is also provided with a lighting device;

a detection unit 401 for controlling the lighting device to turn on;

and detecting whether a target bottom mark matched with the target cage car exists above the AGV car by using a second camera.

Wherein, the area of the bottom mark is larger than that of the side mark.

The detection unit 401 is configured to obtain request information for carrying a target cage car, where the request information includes cage car information corresponding to the target cage car;

detecting whether a side part identifier exists in a preset scene by using a second camera;

and if the second camera detects that the side part identification exists in the preset scene, matching the cage information stored in the detected side part identification with the cage information of the target cage to determine whether the target side part identification matched with the target cage exists in the preset scene.

The embodiment of the application also provides electronic equipment which integrates the control device of any AGV provided by the embodiment of the application. As shown in fig. 5, a schematic structural diagram of an electronic device according to an embodiment of the present application is shown, specifically:

the electronic device may include components such as a processor 601 of one or more processing cores, memory 602 of one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will appreciate that the electronic device configurations shown in the figures do not constitute limitations of the electronic device, and may include more or fewer components than shown, or some components in combination, or a different arrangement of components. Wherein:

the processor 601 is a control center of the electronic device, connects various parts of the whole electronic device by using various interfaces and lines, and performs various functions of the electronic device and processes data by operating or executing software programs and/or modules stored in the memory 602 and calling data stored in the memory 602, thereby performing overall monitoring of the electronic device. Optionally, processor 601 may include one or more processing cores; preferably, the processor 601 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 601.

The memory 602 may be used to store software programs and modules, and the processor 601 executes various functional applications and data processing by operating the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to use of the electronic device, and the like. Further, the memory 602 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

The electronic device further comprises a power supply 603 for supplying power to the various components, and preferably, the power supply 603 is logically connected to the processor 601 through a power management system, so that functions of managing charging, discharging, power consumption, and the like are realized through the power management system. The power supply 603 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.

The electronic device may further include an input unit 604, and the input unit 604 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

Although not shown, the electronic device may further include a display unit and the like, which are not described in detail herein. Specifically, in this embodiment, the processor 601 in the electronic device loads the executable file corresponding to the process of one or more application programs into the memory 602 according to the following instructions, and the processor 601 runs the application program stored in the memory 602, thereby implementing various functions as follows:

and controlling the AGV to lift the target cage car.

It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.

To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like. The AGV control system comprises a AGV, a computer, a controller, a storage device, a display device and a control device, wherein the AGV comprises a first AGV and a second AGV, and the first AGV is connected with the controller and the second storage device through the control device. For example, the computer program may be loaded by a processor to perform the steps of:

and controlling the AGV to lift the target cage car.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and parts that are not described in detail in a certain embodiment may refer to the above detailed descriptions of other embodiments, and are not described herein again.

In a specific implementation, each unit or structure may be implemented as an independent entity, or may be combined arbitrarily to be implemented as one or several entities, and the specific implementation of each unit or structure may refer to the foregoing method embodiment, which is not described herein again.

The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.

The method, the device, the electronic device and the storage medium for controlling the AGV provided by the embodiment of the present application are introduced in detail, and a specific example is applied in the present application to explain the principle and the implementation of the present application, and the description of the embodiment is only used to help understanding the method and the core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. The method for controlling the AGV trolley is characterized in that the AGV trolley is used for carrying cage trolleys in a preset scene, a bottom mark is arranged at the bottom of a chassis of each cage trolley, goods are loaded above the chassis of each cage trolley, an accommodating space for accommodating the AGV trolley is arranged at the bottom of the chassis of each cage trolley, and a first camera is arranged at the top of the AGV trolley and used for taking pictures above the AGV trolley; the AGV control method comprises the following steps:

and controlling the AGV to lift the target cage car.

2. The method of claim 1, wherein a side marker is provided on a side of said cage car, and a second camera is provided on a side of said AGV car, said second camera being used to photograph a side of said AGV car;

3. The method of controlling an AGV according to claim 2, further comprising:

4. The method of any one of claims 1-3, wherein said determining first and preset relative pose information for the AGV and the target cage based on the target bottom markers comprises:

5. The AGV control method according to claim 4, wherein said obtaining fifth relative pose information of the target bottom flag and the AGV based on the target bottom flag, and sixth relative pose information of the target bottom flag and the target cage pre-stored in the target bottom flag comprises:

6. The method of controlling an AGV according to claim 1, wherein a lighting device is further provided on the top of the AGV;

controlling the lighting device to be turned on;

7. The AGV cart control method of claim 2, wherein said bottom flag has an area greater than an area of said side flag.

8. The AGV cart control method of claim 2, wherein said detecting with said second camera whether there is a target side id matching said target cage cart in said predetermined scenario comprises:

9. The control device of the AGV trolley is characterized in that the AGV trolley is used for carrying cage trolleys in a preset scene, a bottom mark is arranged at the bottom of a chassis of each cage trolley, goods are loaded above the chassis of each cage trolley, an accommodating space for accommodating the AGV trolley is arranged at the bottom of the chassis of each cage trolley, and a first camera is arranged at the top of the AGV trolley and used for taking pictures above the AGV trolley; the AGV controlling means includes:

and the control unit is used for controlling the AGV to lift the target cage car.

10. An electronic device, characterized in that the electronic device comprises:

one or more processors;

a memory; and

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the method of controlling an AGV of any of claims 1 to 8.

11. A computer-readable storage medium, having stored thereon a computer program which is loaded by a processor to perform the steps of the method for controlling an AGV cart according to any one of claims 1 to 8.