TWI799051B - Automatic guided vehicle and method for forking pallet - Google Patents

Abstract

An automatic guided vehicle (AGV) and a method for forking a pallet are provided. The method includes: capturing a depth map by a depth camera, wherein the depth map includes the pallet; obtaining a plurality of images respectively corresponding to a plurality of depths from the depth map; inputting the plurality of images to a machine learning model to generate 3-dimensional location information of the pallet; and controlling a driving wheel and a fork of the AGV to move the pallet according to the 3-dimensional location information. The invention includes an automatic guided vehicle (AGV) for forking a pallet.

Description

Automatic carrier for forking pallets and method for forking pallets

The invention relates to an automatic guided vehicle (AGV) for moving pallets and a method for forking pallets.

Under the trend of smart factories, the use of automatic pallet fork picking technology can increase the autonomous operation mode of automatic pallet trucks, effectively saving the labor and time required for logistics personnel to carry goods or put goods on shelves, and accelerate the development of logistics industry or traditional industries. automation. In recent years, AI machine learning technology has become more and more developed. There are many models for image recognition. The recognition rate and recognition speed also have a certain foundation, and the correction model can be continuously optimized for multiple applications.

At present, most of the existing automatic fork-picking pallet technology is to walk to the designated position by the AGV positioning method, and judge whether the fork-picking is successful through the Touch Sensor and infrared sensor; in addition, there are also 3D vision cameras and computing models to identify pallets. However, the cost of equipment is high, and the placement requires a specific prominent placement method to identify pallets for fork picking. In order to enable the automatic pallet truck to move the pallet autonomously, the automatic pallet truck must be given the function of identifying the pallet. However, when the logistics warehouse environment is small and the environment decoration is relatively When it is messy, traditional machine vision technology cannot accurately determine the position of the pallet. In addition, the material of the pallet will also affect the object recognition result.

The invention provides an automatic transport vehicle for moving a pallet and a method for forking the pallet, which can judge the position of the pallet through a depth map.

An automatic transport vehicle for forking pallets according to the present invention includes a tooth fork, a driving wheel, a depth camera, a storage medium and a processor. The depth camera captures a depth map, where the depth map includes pallets. The storage medium stores the machine learning model. The processor is coupled to the fork, the drive wheel, the depth camera and the storage medium, and is configured to perform: obtaining a plurality of images respectively corresponding to a plurality of depths from the depth map; inputting the plurality of images into a machine learning model to generate The three-dimensional positioning information of the pallet; and controlling the driving wheel and the tooth fork to move the pallet according to the three-dimensional positioning information.

In an embodiment of the present invention, the above-mentioned plurality of images includes a first image, wherein the machine learning model detects the first image to generate a first bounding box and a second bounding box corresponding to the pallet, wherein the processor calculates a first distance between the center point of the first image and the first bounding box, and calculates a second distance between the center point and the second bounding box, wherein the processor responds to the first distance being less than Select the first bounding frame from the first bounding frame and the second bounding frame according to the second distance, and generate 3D positioning information according to the selected first bounding frame.

In an embodiment of the present invention, the above machine learning model detects the first image to generate the first confidence level corresponding to the first bounding box and the second bounding box A second confidence level of block, wherein the processor compares the first confidence level to a second confidence level in response to the first distance being the same as the second distance, wherein the processor responds to the first confidence level being greater than the second confidence level The first bounding frame is selected from the first bounding frame and the second bounding frame, and three-dimensional positioning information is generated according to the selected first bounding frame.

In an embodiment of the present invention, the above-mentioned multiple images include a first image corresponding to a first depth, wherein the first image includes pixels, and the processor depth map obtains the abscissa value of the pixel, the vertical axis of the pixel The coordinate value and the distance between the pixel and the depth camera, wherein the processor judges that the pixel corresponds to the first depth according to the abscissa value, the ordinate value and the distance.

In an embodiment of the present invention, the above-mentioned processor calculates the second distance between the coordinate origin and the pixel according to the abscissa value and the ordinate value, calculates the arc cosine value of the ratio of the second distance to the distance, and calculates the distance Multiplied by the arccosine to calculate the first depth.

In an embodiment of the present invention, the processor controls the driving wheel to reduce the angle in response to the angle between the fork of the pallet and the optical axis of the depth camera being greater than an angle threshold.

In an embodiment of the present invention, the above-mentioned automatic transport vehicle further includes a speed sensor. The speed sensor is coupled to the processor, wherein the speed sensor obtains the linear velocity and angular velocity of the automatic transport vehicle, and the processor controls the driving wheel according to the linear velocity, angular velocity and three-dimensional positioning information.

In an embodiment of the present invention, the above-mentioned automatic transport vehicle further includes driven wheels. The distance between the first axle of the driving wheel and the second axle of the driven wheel, wherein the processor Control the driving wheel according to the distance.

In an embodiment of the present invention, adjacent depths among the aforementioned plurality of depths are apart from each other by a preset distance.

In an embodiment of the present invention, the above-mentioned processor determines the preset distance according to the length of the pallet.

In an embodiment of the present invention, the above-mentioned depth camera includes at least one of the following: a camera with an RGBD lens, a camera with an infrared lens, or a camera with multiple lenses.

In an embodiment of the present invention, the above-mentioned automatic transport vehicle further includes a transceiver. The transceiver is coupled to the processor, wherein the processor controls the driving wheel to move to a specified location in response to receiving a command through the transceiver, and captures a depth map at the specified location.

In an embodiment of the present invention, the above-mentioned machine learning model is a YOLO model.

A method for forking pallets of the present invention is suitable for automatic transport vehicles, wherein the method includes: using a depth camera to capture a depth map, wherein the depth map includes pallets; obtaining multiple maps corresponding to multiple depths from the depth map image; input multiple images into the machine learning model to generate three-dimensional positioning information of the pallet; and control the driving wheel and the tooth fork of the automatic pallet truck to move the pallet according to the three-dimensional positioning information.

Based on the above, the automatic transport vehicle of the present invention can detect the three-dimensional positioning information of the pallet, and can automatically move the pallet according to the three-dimensional positioning information, thereby saving a lot of labor cost and time cost for the factory. The present invention combines the depth camera and AI machine learning, allowing the logistics industry to achieve low equipment construction costs and low environmental deployment requirements. Under certain circumstances, the position and angle of the pallet can be effectively identified. Through the layered processing of depth images and AI deep learning, the present invention can avoid the situation that pallets cannot be identified due to environmental changes or changes in light, different pallet materials, and the like.

100:Automatic pallet truck

110: Processor

120: storage media

121:Machine Learning Models

130: Transceiver

140: Depth camera

141: optical axis

150: tooth fork

160: wheels

161: driving wheel

162: driven wheel

170: Speed sensor

200: Pallet

210: fork

211: central axis

300, 400, 500: bounding box

310, 410, O: center point

61, 62: axle

71, 72: Image

A1, A2, d, d2, d3, D: distance

d1: depth

H: height

L: Length

m: default spacing

P0: coordinate origin

P1: pixel

R: radius of rotation

W: width

x _J , x _K , z _J , z _K : coordinate values

δ , δ _J , δ _K : yaw angle

:角度 θ ,

:angle

S301, S302, S303, S304, S305, S306, S307, S308, S309, S310, S311, S312, S313, S314, S315, S901, S902, S903, S904: steps

FIG. 1 shows a schematic diagram of an automatic transport vehicle for forking pallets according to an embodiment of the present invention.

FIG. 2 is a schematic diagram illustrating a wheel configuration of an automatic transport vehicle according to an embodiment of the present invention.

FIG. 3 shows a flowchart of a method for moving a pallet according to an embodiment of the invention.

FIG. 4 is a schematic diagram illustrating the rotation radius and yaw angle of the automatic transport vehicle according to an embodiment of the present invention.

FIG. 5 shows a schematic diagram of an automatic transport vehicle and pallets according to an embodiment of the present invention.

FIG. 6 is a schematic diagram of determining the depth of an object according to an embodiment of the present invention.

7 and 8 are schematic diagrams illustrating images corresponding to different depths according to an embodiment of the present invention.

FIG. 9 shows a flowchart of a method for forking a pallet according to an embodiment of the present invention.

In order to make the content of the present invention more understandable, the following special examples are implemented Examples are given as examples of how the invention can actually be practiced. In addition, wherever possible, elements/components/steps using the same reference numerals in the drawings and embodiments represent the same or similar parts.

FIG. 1 shows a schematic diagram of an automatic transport vehicle 100 for forking pallets according to an embodiment of the present invention. The automated pallet truck 100 may include a processor 110 , a storage medium 120 , a transceiver 130 , a depth camera 140 , a fork 150 and wheels 160 . In an embodiment, the automatic pallet truck 100 may further include a speed sensor 170 . The automatic pallet truck 100 is, for example, a forklift or a fork truck.

The processor 110 is, for example, a central processing unit (central processing unit, CPU), or other programmable general purpose or special purpose micro control unit (micro control unit, MCU), microprocessor (microprocessor), digital signal processing Digital Signal Processor (DSP), Programmable Controller, Application Specific Integrated Circuit (ASIC), Graphics Processing Unit (GPU), Image Signal Processor (ISP) ), image processing unit (image processing unit, IPU), arithmetic logic unit (arithmetic logic unit, ALU), complex programmable logic device (complex programmable logic device, CPLD), field programmable logic gate array (field programmable gate array , FPGA) or other similar components or a combination of the above components. The processor 110 can be coupled to the storage medium 120, the transceiver 130, the depth camera 140, the fork 150, the wheel 160, and the speed sensor 170, and access and execute multiple modules and various application.

The storage medium 120 is, for example, any type of fixed or removable random access memory (random access memory, RAM), read-only memory (read-only memory, ROM), flash memory (flash memory) , hard disk drive (hard disk drive, HDD), solid state drive (solid state drive, SSD) or similar components or a combination of the above components, and are used to store multiple modules or various application programs that can be executed by the processor 110 . In this embodiment, the storage medium 120 can store a plurality of modules including the machine learning model 121, and its functions will be described later.

The transceiver 130 transmits and receives signals in a wireless or wired manner. The transceiver 130 may also perform operations such as low noise amplification, impedance matching, frequency mixing, up or down frequency conversion, filtering, amplification, and the like.

The depth camera 140 is used to capture the depth map of the object. The depth camera 140 may include a camera with an RGBD lens, a camera with an infrared lens, or a camera with multiple lenses, and the present invention is not limited thereto.

Tooth fork 150 is used to fork and take pallet. When the automatic pallet truck 100 intends to move the pallet, the processor 110 can control the tooth fork 150 to insert into the bay of the pallet. The processor 110 can control the fork 150 to move vertically to move the pallet closer to or away from the ground.

The speed sensor 170 may include an accelerometer or a gyroscope. The speed sensor 170 is used to sense the linear velocity and the angular velocity of the automatic pallet truck 100 , and report the linear velocity and the angular velocity to the processor 110 .

The wheels 160 may include a driving wheel 161 and a driven wheel 162 . FIG. 2 is a schematic diagram illustrating a wheel configuration of an automatic transport vehicle according to an embodiment of the present invention. In this embodiment, it is assumed that the automatic transport vehicle 100 travels The ground is the XZ plane of the Cartesian coordinate system and the Y axis is perpendicular to the ground. The automatic pallet truck 100 may have a single driving wheel 161 disposed at the front and two driven wheels 162 disposed at the rear. There is a distance D between the axle 61 of the driving wheel 161 and the axle 62 of the driven wheel. The processor 110 can be coupled to the driving wheel 161 and control the rotation speed and yaw angle (yaw) of the driving wheel, so as to control the moving speed and moving direction of the automatic pallet truck 100 . The present embodiment forklift is three-wheeled, and its driving wheel has one round, directly controls the moving angle of forklift, and its dynamics is the same with tricycle; If it is four-wheeled, its driving wheel is two-wheeled, and its dynamics is the same with general automobile.

FIG. 3 shows a flowchart of a method for moving pallets according to an embodiment of the present invention, wherein the method can be implemented by the automatic transport vehicle 100 shown in FIG. 1 . In step S301 , the processor 110 may receive a command through the transceiver 130 , wherein the command instructs the automatic pallet truck 100 to move to a designated location. For example, if the user wants to transport pallets at a designated location, the user can operate a terminal device (such as a computer or a smart phone) to send a command corresponding to the designated location to the automatic pallet truck 100 .

In step S302 , the processor 110 may control the driving wheels 161 according to the command to move the automatic transport vehicle 100 to a designated location. The processor 110 can control the movement of the automatic transport vehicle 100 according to the method disclosed in Taiwan Patent Publication No. I671610. FIG. 4 shows a schematic diagram of the rotation radius R and the yaw angle δ of the automatic transport vehicle 100 according to an embodiment of the present invention. Assuming that the coordinates of the automatic pallet truck 100 are ( x _J , z _J ) and the yaw angle is δ _J , the processor 110 can perform equations (1), (2), (3), (4) and (5) to The rotational speed and the yaw angle of the driving wheel 161 are controlled, so as to move the automatic transport vehicle 100 to the coordinate ( x _K , z _K ) at the yaw angle δ _K .

其中R為自動搬運車100的旋轉半徑，ω為自動搬運車100的角速度，v為自動搬運車100的線速度，D為主動輪161的輪軸61與從動輪162的輪軸62之間的距離，且△t為自動搬運車100從座標(x _J ,z _J )移動到座標(x _K ,z _K )所花費的時間。處理器110可通過控制主動輪161的轉速來調整線速度v，並可通過控制主動輪161的轉速來和偏擺角來調整角速度ω或旋轉半徑R。

Wherein R is the radius of rotation of the automatic transport vehicle 100, ω is the angular velocity of the automatic transport vehicle 100, v is the linear velocity of the automatic transport vehicle 100, D is the distance between the axle 61 of the driving wheel 161 and the axle 62 of the driven wheel 162, And Δt is the time it takes for the automatic transport vehicle 100 to move from the coordinates ( x _J , z _J ) to the coordinates ( x _K , z _K ). The processor 110 can adjust the linear velocity v by controlling the rotational speed of the driving wheel 161 , and can adjust the angular velocity ω or the radius of rotation R by controlling the rotational speed of the driving wheel 161 and the yaw angle.

Returning to FIG. 3, after the automatic transport vehicle 100 moves to the designated location, in step S303, the processor 110 can capture a depth map (depth map) through the depth camera 140, and obtain depth maps corresponding to multiple depths from the depth map. Multiple images, where the depth map can contain pallets. FIG. 5 shows a schematic diagram of the automatic transport vehicle 100 and the pallet 200 according to an embodiment of the present invention. The pallet 200 has a length L, a width W and a height H. As shown in FIG. The pallet 200 may include a fork 210 for receiving the fork 150 . The depth camera 140 can capture a depth map including the pallet 200 . The processor 110 may determine a plurality of depths according to the preset distance m. Adjacent depths in multiple depths can be separated from each other by a preset distance m.

For example, assume that the optical axis 141 of the depth camera 140 is parallel to the Z axis. The processor 110 can sample the image corresponding to depth Z=0, the image corresponding to depth Z=m, the image corresponding to depth Z=2m and the image corresponding to depth Z=2m from the depth map according to the preset interval m. The images of 3m and the like respectively correspond to a plurality of images of different depths. In order to avoid that the pallet 200 does not appear in all the sampled images, the processor 110 may determine the preset distance m according to the size of the pallet 200 . In one embodiment, the processor 110 can determine the preset distance m according to equations (6) and (7), where L is the length of the pallet 200, d is the offset distance of the pallet 200, and α is a coefficient (for example: α =3), and θ is the angle between the optical axis 141 of the depth camera 140 and the central axis 211 of the fork 210, wherein the central axis 211 of the fork 210 can be connected to the short side of the pallet 200 (that is: the length is W sides) are parallel.

d = L . sin θ ... (6)

m = α . d ...(7)

Each pixel on the image extracted by the processor 110 from the depth map corresponds to the same depth. FIG. 6 is a schematic diagram of determining the depth of an object according to an embodiment of the present invention. Take the pixel P1 corresponding to the vertex of the pallet 200 as an example. Assume that the optical axis 141 of the depth camera 140 is parallel to the Z axis. If the processor 110 captures the vertex of the pallet 200 through the depth camera 140, the depth map generated by the depth camera 140 may include the abscissa value (ie: X coordinate value) of the pixel P1 corresponding to the vertex, and the ordinate value of the pixel P1 (ie: Y coordinate value) and information such as the distance d3 between the pixel P1 and the depth camera 140 , as shown in FIG. 6 . Processor 110 may be based on the equation (8) and (9) determine that the pixel P1 corresponds to the depth d1 according to the abscissa value of the pixel P1, the ordinate value of the pixel P1, and the distance d3. Therefore, if the processor 110 wants to capture an image corresponding to the depth d1, the processor 110 can extract the pixel P1 from the depth map to generate the image. The image extracted by the processor 110 from the depth map may be parallel to the XY plane.

其中d2為像素P1與座標原點P0之間的距離，並且d3為像素P1與深度攝影機140之間的距離。一般的深度攝影機將光軸的橫座標值和縱座標值設為零。也就是說，上述的座標原點P0可位於深度攝影機140的光軸141上。

Where d2 is the distance between the pixel P1 and the coordinate origin P0 , and d3 is the distance between the pixel P1 and the depth camera 140 . A typical depth camera sets the abscissa and ordinate values of the optical axis to zero. That is to say, the aforementioned coordinate origin P0 may be located on the optical axis 141 of the depth camera 140 .

Returning to FIG. 3 , after obtaining multiple images respectively corresponding to multiple depths, in step S304, the processor 110 may input the multiple images into the machine learning model 121 to generate at least a certain depth corresponding to the pallet 200. A bounding box, wherein the at least certain bounding box is used to generate three-dimensional positioning information of the pallet 200 . The machine learning model 121 is, for example, an object detection model including the YOLO model, but the present invention is not limited thereto. 7 and 8 are schematic diagrams illustrating images corresponding to different depths according to an embodiment of the present invention. Assume that the multiple images extracted by the processor 110 from the depth map include image 71 and image 72 respectively corresponding to different depths, wherein image 71 corresponds to depth Z=2m and image 72 corresponds to depth Z=3m . The processor 110 can input the image 71 to the machine learning model 121 to generate the bounding box 300 and the bounding box 400 . In addition, the machine learning model 121 can also generate a confidence level corresponding to the bounding box 300 score) and the confidence level corresponding to the bounding box 400. On the other hand, the processor 110 may input the image 72 to the machine learning model 121 to generate the bounding box 500 and the confidence level corresponding to the bounding box 500 .

Returning to FIG. 3 , in step S305 , the processor 110 may select an image from a plurality of images. The selected image can include the pallet 200 and can be used to determine the three-dimensional positioning information of the pallet 200 . In one embodiment, the processor 110 may select the image according to the confidence of the bounding box corresponding to the image. Specifically, the processor 110 may select an image including a bounding box corresponding to a maximum reliability from the plurality of images as the selected image. Taking Fig. 7 and Fig. 8 as an example, it is assumed that the reliability of bounding box 300 in image 71 is greater than that of bounding box 400, and the reliability of bounding box 300 in image 71 is also greater than that in Fig. Confidence of the bounding box 500 as in 72. Accordingly, the processor 110 may select the image 71 from the images 71 and 72 as the selected image.

In step S306, the processor 110 may determine whether the selected image contains multiple bounding boxes. If the selected image contains multiple bounding boxes, go to step S308. If the selected image contains a single bounding box but does not contain multiple bounding boxes, go to step S307. Take image 71 as an example. Assuming that the image 71 is the selected image, since the image 71 includes two bounding boxes 300 and 400 , the processor 110 can determine that the selected image includes multiple bounding boxes. Accordingly, the processor 110 may execute step S308. Take image 72 as an example. Assuming that the image 72 is the selected image, since the image 72 includes one bounding box 500 , the processor 110 can determine that the selected image does not include multiple bounding boxes. Accordingly, the processor 110 may execute step S307.

In step S307, the processor 110 may, according to the bounding box in the selected image Generate three-dimensional positioning information of the pallet 200, wherein the three-dimensional positioning information may include the horizontal axis coordinate value (ie: X coordinate value), vertical coordinate value (ie: Y coordinate value) and depth (ie: Z coordinate value) of the pallet 200 . Taking the image 72 in FIG. 8 as an example, it is assumed that the image 72 is the selected image. Since the processor 110 knows that the image 72 corresponds to a depth of Z=3m, the processor 110 can determine that the depth of the pallet 200 is Z=3m. On the other hand, the processor 110 may determine the abscissa value and the ordinate value of the pallet 200 according to the bounding box 500 . After obtaining the abscissa value, ordinate value and depth of the pallet 200 , the processor 110 may generate three-dimensional positioning information including the abscissa value, ordinate value and depth of the pallet 200 .

In step S308, the processor 110 may calculate the distance between each bounding box (or the center point of the bounding box) and the center point of the image to generate a plurality of distances, and determine whether the plurality of distances are the same. If the multiple distances are the same, go to step S310. If the multiple distances are not the same, go to step S309. Taking the image 71 as an example, the processor 110 can calculate the distance A1 between the center point 310 of the bounding box 300 and the center point O of the image 71 and the distance A1 between the center point 410 of the bounding box 400 and the center point O of the image 71 The distance between A2. If the distance A1 is the same as the distance A2, go to step S310. If the distance A1 is different from the distance A2, go to step S309.

In step S309, the processor 110 may select the bounding box closest to the center point of the image from the plurality of bounding boxes as the selected bounding box, and generate the three-dimensional positioning information of the pallet 200 according to the selected bounding box . Taking the image 71 as an example, the processor 110 may select the bounding box 300 from the bounding box 300 and the bounding box 400 in response to the distance A1 corresponding to the bounding box 300 being smaller than the distance A2 corresponding to the bounding box 400 . as the selected bounding box. Accordingly, the processor 110 can generate the frame of the pallet 200 according to the bounding box 300 3D positioning information.

In step S310, the processor 110 may determine whether the multiple confidence levels respectively corresponding to the multiple bounding boxes are the same. If the reliability levels are the same, go to step S312. If the multiple credibility levels are not the same, go to step S311. Taking the image 71 as an example, if the reliability of the bounding frame 300 is the same as that of the bounding frame 400 , go to step S312 . If the reliability of the bounding box 300 is not the same as that of the bounding box 400, go to step S311.

In step S311 , the processor 110 may select a bounding frame with the greatest reliability from the plurality of bounding frames as the selected bounding frame, and may generate three-dimensional positioning information of the pallet 200 according to the selected bounding frame. If the distances between the bounding boxes in the image and the center point of the image are the same, it means that the processor 110 cannot select a better bounding box according to the distance. Therefore, the processor 110 will select the bounding box with the greatest confidence as the selected bounding box. Taking the image 71 as an example, the processor 110 may select the bounding frame 300 from the bounding frame 300 and the bounding frame 400 as the subject in response to the reliability of the bounding frame 300 being greater than the reliability of the bounding frame 400 . Select a bounding box. Accordingly, the processor 110 can generate the three-dimensional positioning information of the pallet 200 according to the bounding box 300 .

In step S312 , the processor 110 may select a selected bounding frame from a plurality of bounding frames according to a preset rule (or randomly), and generate three-dimensional positioning information of the pallet 200 according to the selected bounding frame. The default rule is, for example, "select the bounding box with the smallest abscissa value as the selected bounding box". Taking the image 71 as an example, the processor 110 may select the bounding box 300 from the bounding box 300 and the bounding box 400 as the subject in response to the abscissa value of the bounding box 300 being smaller than the abscissa value of the bounding box 400 . Select a bounding box. Accordingly, The processor 110 can generate three-dimensional positioning information of the pallet 200 according to the bounding box 300 .

In order to reduce the difficulty of identifying the pallet 200 or inserting the fork 150 into the fork opening 210, the processor 110 can determine whether to move the automatic transport vehicle according to the angle θ between the central axis 211 of the fork opening 210 and the optical axis 141 of the depth camera 140 100. Specifically, in step S313 , the processor 110 may determine whether the angle θ between the central axis 211 of the fork 210 and the optical axis 141 of the depth camera 140 is greater than an angle threshold. If the angle θ is greater than the angle threshold, go to step S314. If the angle θ is less than or equal to the angle threshold, go to step S315.

In step S314 , the processor 110 can control the driving wheel 161 to move the automated truck 100 , so as to reduce the angle θ until the angle θ is less than or equal to the angle threshold. Next, the processor 110 may re-execute the process of FIG. 3 to attempt to fork the pallet 200 .

In step S315 , the processor 110 can control the driving wheel 161 and the tooth fork 150 to move the pallet 200 according to the three-dimensional positioning information of the pallet 200 . Specifically, the processor 110 can control the driving wheel 161 to move the automatic pallet truck 100 to insert the tooth fork 150 into the fork opening 210 . Next, the processor 110 can control the fork 150 to lift up the pallet 200 .

FIG. 9 shows a flow chart of a method for forking pallets according to an embodiment of the present invention, wherein the method can be implemented by the automatic transport vehicle 100 shown in FIG. 1 . In step S901, a depth image is captured by a depth camera, wherein the depth image includes pallets. In step S902, a plurality of images respectively corresponding to a plurality of depths are acquired from the depth map. In step S903, a plurality of images are input into a machine learning model to generate three-dimensional positioning information of pallets. In step S904, the automatic moving is controlled according to the three-dimensional positioning information. The driving wheels of the transport cart and the forks are used to move the pallet.

To sum up, the automatic transport vehicle of the present invention can automatically move to a designated location according to commands, and detect whether there is a pallet at the designated location. If the angle between the fork of the pallet and the fork of the automatic pallet truck is too large, the automatic pallet truck can move to reduce the angle so that the pallet can be picked up by the forks of the pallet. The automatic pallet truck can extract multiple images from the depth map generated by the depth camera through the image layering algorithm, and judge the position of the fork of the pallet based on these images, so as to increase the automatic pallet truck's ability to stack in different environments board or the recognition accuracy of pallets of different materials. The automatic pallet truck can control the wheels and tooth forks according to the position of the fork to move the pallet automatically.

S901, S902, S903, S904: steps

Claims (14)

An automatic carrier for forking pallets, comprising: Tooth fork; driving wheel; a depth camera for capturing a depth map, wherein the depth map includes the pallet; storage media for storing machine learning models; and a processor, coupled to the fork, the drive wheel, the depth camera and the storage medium, and configured to: obtaining a plurality of images respectively corresponding to a plurality of depths from the depth map; inputting the plurality of images into the machine learning model to generate 3D positioning information of the pallet; and The driving wheel and the tooth fork are controlled according to the three-dimensional positioning information to move the pallet. The automatic pallet truck as claimed in claim 1, wherein said plurality of images comprises a first image, wherein The machine learning model detects the first image to generate a first bounding box and a second bounding box corresponding to the pallet, wherein the processor calculates a first distance between a center point of the first image and the first bounding box, and calculates a second distance between the center point and the second bounding box, in The processor selects the first bounding box from the first bounding box and the second bounding box in response to the first distance being less than the second distance, and based on the selected The first bounding box generates the three-dimensional positioning information. The automatic transport vehicle as described in claim 2, wherein The machine learning model detects the first image to generate a first confidence level corresponding to the first bounding box and a second confidence level corresponding to the second bounding box, wherein The processor compares the first confidence level to the second confidence level in response to the first distance being the same as the second distance, wherein The processor selects the first bounding box from the first bounding box and the second bounding box in response to the first confidence level being greater than the second confidence level, and based on The selected first bounding box generates the 3D positioning information. The automated pallet truck of claim 1, wherein the plurality of images includes a first image corresponding to a first depth, wherein the first image includes pixels, wherein The processor acquires the abscissa value of the pixel, the ordinate value of the pixel, and the distance between the pixel and the depth camera from the depth map, wherein The processor judges that the pixel corresponds to the first depth according to the abscissa value, the ordinate value and the distance. The automatic transport vehicle according to claim 4, wherein the processor calculates the second distance between the coordinate origin and the pixel according to the abscissa value and the ordinate value, and calculates the second distance and an arccosine of a ratio of the distances, and multiplying the distance by the arccosine to calculate the first depth. The automatic transport vehicle as described in claim 1, wherein The processor controls the drive wheel to reduce the angle in response to an angle between the fork of the pallet and the optical axis of the depth camera being greater than an angle threshold. The automatic transfer vehicle as described in claim 1, further comprising: a speed sensor, coupled to the processor, wherein the speed sensor obtains the linear velocity and angular velocity of the automatic transport vehicle, wherein the processor is based on the linear velocity, the angular velocity and the three-dimensional Positioning information controls the driving wheel. The automatic transfer vehicle as described in claim item 7, further comprising: For the driven wheel, the first axle of the driving wheel is at a distance from the second axle of the driven wheel, wherein the processor controls the driving wheel according to the distance. The automated pallet truck as claimed in claim 1, wherein adjacent ones of the plurality of depths are spaced apart from each other by a preset distance. The automatic transport vehicle according to claim 9, wherein the processor determines the preset distance according to the length of the pallet. The automatic transport vehicle according to claim 1, wherein the depth camera includes at least one of the following: a camera with RGBD lens, a camera with infrared lens or a camera with multiple lenses. The automatic transfer vehicle as described in claim 1, further comprising: a transceiver coupled to the processor, wherein the processor controls the drive wheel to move to a specified location in response to receiving a command through the transceiver, and captures the depth map at the specified location. The automatic transport vehicle according to claim 1, wherein the machine learning model is a YOLO model. A method for forking pallets, suitable for automatic transport vehicles, wherein the method includes: capturing a depth map using a depth camera, wherein the depth map includes the pallet; obtaining a plurality of images respectively corresponding to a plurality of depths from the depth map; inputting the plurality of images into a machine learning model to generate 3D positioning information of the pallet; and According to the three-dimensional positioning information, the driving wheel and the tooth fork of the automatic transport vehicle are controlled to move the pallet.

Images