JP3242392U - autonomous vehicle - Google Patents

Abstract

An autonomous vehicle capable of easily identifying the structure of a shelf is provided.
An autonomously traveling vehicle (10) includes an imaging device (12) that captures an area to the left of the autonomously traveling vehicle or an area to the right of the autonomously traveling vehicle, a laser light irradiation device (13) that includes a laser light source that emits laser light, A recognition device 14 that recognizes a plurality of markers arranged in a matrix in the front-rear direction and the vertical direction on the left or right of an imaging device, and acquires image data including a plurality of markers and laser light from the imaging device. And prepare.
[Selection drawing] Fig. 1

Description

The present invention relates to autonomous vehicles that recognize markers.

As a device or invention related to an autonomous vehicle that recognizes a conventional marker, for example, a measurement system described in Patent Literature 1 is known. In this measurement system, CAD data of a shelf arranged in a warehouse is used to extract shape data of a luggage area in the shelf.

WO2017/175312

By the way, in the measurement system described in Patent Literature 1, there is a demand to easily identify the structure of the shelf without using complicated data such as CAD data.

SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide an autonomous vehicle that can easily identify the structure of a shelf.

An autonomous vehicle according to one aspect of the present invention includes:
an autonomous vehicle,
an imaging device that captures an area to the left of the autonomous vehicle or an area to the right of the autonomous vehicle;
a laser light irradiation device including a laser light source that emits laser light;
A recognition device that recognizes a plurality of markers arranged in a matrix in the front-rear direction and the vertical direction on the left or right from the imaging device, and acquires image data including the plurality of markers and the laser beam from the imaging device. a recognition device that
Prepare.

According to the autonomous vehicle according to the present invention, it is possible to easily identify the structure of the shelf.

FIG. 1 is a top view of the inside of a warehouse. FIG. 2 is a top view of the interior of the warehouse. FIG. 3 shows image data generated by the imaging device 12 . FIG. 4 is a flow chart executed by the recognition device 14 . FIG. 5 shows image data generated by the imaging device 12 . FIG. 6 shows image data generated by the imaging device 12 . FIG. 7 shows image data generated by the imaging device 12 . FIG. 8 shows image data generated by the imaging device 12 . FIG. 9 is a flowchart executed by the recognition device 14a. FIG. 10 shows image data generated by the imaging device 12 . FIG. 11 shows image data generated by the imaging device 12 . FIG. 12 shows image data generated by the imaging device 12 .

[First embodiment]
[Structure of recognition device]
A structure of an autonomous vehicle 10 according to an embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a top view of the inside of a warehouse. FIG. 2 is a top view of the interior of the warehouse. FIG. 3 shows image data generated by the imaging device 12 .

In this specification, the longitudinal direction of the autonomous vehicle 10 is simply referred to as the longitudinal direction. The left-right direction of the autonomous vehicle 10 is simply referred to as the left-right direction. The vertical direction of the autonomous vehicle 10 is simply referred to as the vertical direction.

As shown in FIG. 1, three rows of shelves T1 to T3 extend in the front-rear direction in the warehouse. The shelves T1 to T3 are arranged in this order from left to right. Since the shelves T1 to T3 have the same structure, the shelf T1 will be described as an example. As shown in FIG. 3, the shelf T1 has a lattice structure by combining a plurality of frames F extending in the front-back direction and the up-down direction. Thus, the shelf T1 has a structure in which rectangular spaces are arranged in a matrix in the vertical direction and the front-rear direction. Markers M11, M21, M31, M41, M12, M22, M32, and M42 are positioned at respective lattice points of the lattice structure. As a result, the markers M11, M21, M31, M41, M12, M22, M32, and M42 are arranged in a matrix on the left of the imaging device 12 in the front-back direction and the up-down direction. A plurality of markers M11, M21, M31, M41, M12, M22, M32, and M42 are arranged in m rows and n columns. m and n are integers of 2 or more. In this embodiment, the plurality of markers M11, M21, M31, M41, M12, M22, M32, M42 are arranged in 4 rows and 2 columns.

Here, when a marker Mij is written, i indicates a row. j indicates a column. i is an integer of 1 or more and m or less. j is an integer of 1 or more and n or less. A plurality of markers other than the markers M11, M21, M31, M41, M12, M22, M32, and M42 are arranged at the intersections of the lattice structure. However, description of these markers will be omitted. The markers M11, M21, M31, M41, M12, M22, M32, and M42 are, for example, AR markers, QR codes (registered trademark), or the like. Markers M11, M21, M31, M41, M12, M22, M32, M42 therefore contain information.

A plurality of markers are also arranged on the shelf T2. That is, the plurality of markers are arranged in a matrix in the front-rear direction and the up-down direction even on the right side of the imaging device 12 . However, since the plurality of markers on shelf T2 are the same as the plurality of markers on shelf T1, description thereof will be omitted. A plurality of markers are also arranged on the shelf T3.

The autonomous traveling vehicle 10 is, for example, a forklift, an AGV (Automatic Guided Vehicle), a cleaning robot, or the like. The autonomous vehicle 10 carries cargo into the shelves T1 to T3. Also, the autonomous vehicle 10 unloads packages from the shelves T1 to T3. After traveling between the shelves T1 and T2, the autonomous vehicle 10 travels between the shelves T2 and T3. The autonomous vehicle 10 includes an imaging device 12 , a laser beam irradiation device 13 and a recognition device 14 .

The imaging device 12 is a solid-state imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The imaging device 12 captures an area to the left of the autonomous vehicle 10 and an area to the right of the autonomous vehicle 10, as shown in FIG.

The laser light irradiation device 13 includes a laser light source (not shown) that emits laser light L. As shown in FIG. The laser light L travels straight. The position of the laser beam irradiation device 13 in the vertical direction is different from the position of the imaging device 12 in the vertical direction. In this embodiment, the laser beam irradiation device 13 is positioned below the imaging device 12 . The laser beam irradiation device 13 irradiates the area on the left of the autonomous vehicle 10 and the area on the right of the autonomous vehicle 10 with the laser beam L, as shown in FIG. 2 . In this embodiment, the laser light L travels straight on a horizontal plane perpendicular to the vertical direction.

The recognition device 14 is an arithmetic device such as a CPU (Central Processing Unit) or an MPU (Micro Processor Unit). The recognition device 14 recognizes markers M11, M21, M31, M41, M12, M22, M32, and M42 arranged in a matrix on the left and right sides of the imaging device 12 in the front-rear direction and the vertical direction. The operations performed by the recognition device 14 will be described below with reference to the drawings. FIG. 4 is a flow chart executed by the recognition device 14 . 5 to 7 are image data generated by the imaging device 12. FIG. The recognition device 14 executes the flowchart of FIG. 4 by executing a program stored in a storage device (not shown).

First, the recognition device 14 acquires image data including the markers M11, M21, M31, M41, M12, M22, M32, and M42 and the laser light L from the imaging device 12 (acquisition step: step S1). In the following, image data obtained by capturing an area on the left side of the autonomous vehicle 10 will be described as an example.

Next, as shown in FIG. 3, the recognition device 14 detects two or more markers M11, M31, M41 among the markers M11, M21, M31, M41, M12, M22, M32, M42 based on the image data. , M22 and M32 are recognized as two or more recognition markers m11, m31, m41 and m22 (first recognition step: step S2). In other words, the recognition device 14 does not recognize the markers M21, M12, M32, M42. The reason why the recognition device 14 does not recognize the markers M21, M12, M32 and M42 is, for example, that the illuminance around the markers M21, M12, M32 and M42 is insufficient.

Next, the recognition device 14 identifies a laser beam LB that is an extension of the trajectory of the laser beam L, as shown in FIG. 5 (laser beam identification step: step S3). Note that the recognition device 14 may specify the laser beam LB by storing in advance the position of the laser beam L in the image data, or by analyzing the image data acquired in step S1 (acquisition step). A laser beam LB may be specified.

Next, the recognition device 14 identifies the eye level EL of the imaging device 12 as shown in FIG. 5 (eye level identification step: step S4). Note that the recognition device 14 may specify the eye level EL of the imaging device 12 by storing in advance the position of the eye level EL in the image data, or the image data acquired in step S1 (acquisition step). The analysis may identify the eye level EL.

Next, as shown in FIG. 5, the recognition device 14 identifies a vanishing point O that is the intersection of the eye level EL and the laser beam LB (vanishing point identifying step: step S5).

Next, as shown in FIG. 6, the recognition device 14 recognizes one recognition marker m11 among the two or more recognition markers m11, m31, m41, and m22 recognized in step S2 (first recognition step) and a vanishing point. A first perspective line PL11 passing through O is specified (first perspective line specifying step: step S6). Further, the recognition device 14 also identifies the first perspective lines PL12, PL13, PL14 by repeating the same operation.

Next, the position of one or more unrecognized markers m21 located on the first perspective line PL11 and not recognized in step S2 (first recognition step) is recognized (second recognition step). The second recognition step includes the following column line identification step and position recognition step.

First, as shown in FIG. 7, when the recognition markers m11, m31, and m41 are positioned in the first column (j-th column), the recognition device 14 detects the first line passing through the first column (j-th column). Column line L1 (jth column line) is specified (column line specifying step: S7). The first column line L1 extends vertically. Then, the recognition device 14 recognizes that the unrecognized marker m21 is positioned at the intersection of the first column line L1 (jth column line) and the first perspective line PL12 (position recognition step: step S8). Thus, since the first column line L1 is orthogonal to the eye level EL, if the recognition device 14 can recognize one marker on the first column line L1, the other markers on the first column line L1 can also be recognized. recognizable. Thereby, the recognition device 14 recognizes the marker M21.

By repeating steps S7 and S8, the recognition device 14 recognizes that the unrecognized marker m12 is positioned at the intersection of the second column line L2 (jth column line) and the first perspective line PL11. The recognition device 14 also recognizes that the unrecognized marker m32 is positioned at the intersection of the second column line L2 (jth column line) and the first perspective line PL13. The recognition device 14 also recognizes that the unrecognized marker m42 is positioned at the intersection of the second column line L2 (jth column line) and the first perspective line PL14. Through the above steps S1 to S8, the recognition device 14 recognizes the markers M11, M21, M31, M41, M12, M22, M32 and M42.

After that, the recognition device 14 identifies the structure of the shelf T1 based on the information contained in the markers M11, M21, M31, M41, M12, M22, M32 and M42. Note that the recognition device 14 identifies the structure of the shelf T2 based on the image data obtained by imaging the area to the right of the autonomous vehicle 10 . However, the specific details of the structure of shelf T2 are the same as the specific details of the structure of shelf T1 and will not be described.

[effect]
The recognition device 14 can easily identify the structure of the shelf T1. More specifically, the vanishing point O should be specified in order for the recognition device according to the comparative example to recognize the positions of the unrecognized markers m21, m12, m32, and m42. Thereby, the recognition device according to the comparative example can specify the first perspective line PL11 passing through the vanishing point O and one of the recognition markers m11, m31, m41, and m22. Then, the recognition device according to the comparative example can recognize the position of the unrecognized marker m21 using the first perspective line PL11. In order to specify such a vanishing point O, the recognition device according to the comparative example normally needs to specify two perspective lines and specify the intersection of the two perspective lines as the vanishing point O. be. In this case, the recognition device according to the comparative example needs to recognize four recognition markers in order to specify two perspective lines.

On the other hand, the recognition device 14 identifies the eye level EL of the imaging device 12 . The recognition device 14 also identifies a laser beam LB, which is an extension of the trajectory of the laser beam L. As shown in FIG. As a result, the recognition device 14 can specify the vanishing point O as the intersection of the eye level EL and the laser beam LB. Therefore, the recognition device 14 can recognize the positions of unrecognized markers by recognizing a small number of recognition markers. As a result, the recognition device 14 can easily identify the structure of the shelf T1.

When the recognition device 14 identifies the laser beam LB by analyzing the image data, it is not necessary to mount the laser beam irradiation device 13 at a predetermined mounting position and angle. Therefore, the degree of freedom of the mounting position and angle of the laser beam irradiation device 13 is increased.

When the recognition device 14 identifies the eye level EL by analyzing the image data, it is not necessary to mount the imaging device 12 at a predetermined mounting position and angle. Therefore, the mounting position and angle of the imaging device 12 are more flexible.

[First modification]
The structure of an autonomous vehicle 10a according to a first modified example of the present invention will be described below with reference to the drawings. FIG. 8 shows image data generated by the imaging device 12 . FIG. 9 is a flowchart executed by the recognition device 14a. As for the autonomous vehicle 10a and the recognition device 14a according to the first modification, only the portions different from the autonomous vehicle 10 and the recognition device 14 according to the first embodiment will be explained, and the rest will be omitted.

The autonomous vehicle 10a includes a recognition device 14a instead of the recognition device 14. FIG. The recognition device 14a identifies the virtual markers M01 and M02 located above the first perspective line PL11 located at the top. There are actually no markers at the positions of the virtual markers M01 and M02. That is, the recognition device 14a assumes that markers are present at the positions of the virtual markers M01 and M02. The operations performed by the recognition device 14a in steps S1 to S8 are the same as the operations performed by the recognition device 14 in steps S1 to S8, so description thereof will be omitted.

The recognition device 14a identifies the first perspective line PL11 (second perspective line) passing through the first row (second perspective line identification step: step S9). Next, the recognition device 14a identifies a third perspective line PL3 passing above the first row (third perspective line identifying step: step S10).

Finally, the recognition device 14a recognizes the virtual markers M01 and M02 positioned on the third perspective line PL3 (third recognition step: step S11). Specifically, the recognition device 14a recognizes that the virtual marker M01 is positioned at the intersection of the first column line L1 and the third perspective line PL3. The recognition device 14a recognizes that the virtual marker M02 is positioned at the intersection of the second column line L2 and the third perspective line PL3. Thereby, the recognition device 14a can recognize that there is a space having the virtual markers M01 and M02 and the markers M11 and M12 as four corners.

[Second modification]
The structure of an autonomous vehicle 10b according to a second modified example of the present invention will be described below with reference to the drawings. FIG. 10 shows image data generated by the imaging device 12 . As for the autonomous vehicle 10b and the recognition device 14b according to the second modification, only the portions different from the autonomous vehicle 10 and the recognition device 14 according to the first embodiment will be explained, and the rest will be omitted.

The autonomous vehicle 10b includes a recognition device 14b instead of the recognition device 14. FIG. When no baggage is placed on the shelf T1, the laser light L is irradiated only on the frame F as shown in FIG. Even in such a case, the recognition device 14b may specify the laser beam LB, which is an extension of the trajectory of the laser beam L, in step S3 (laser beam specifying step).

Note that even when a package is placed on the shelf T1, the recognition device 14b detects the laser beam, which is an extension of the trajectory of the laser beam L applied to the frame F, in step S3 (laser beam identification step). LB may be specified.

[Third Modification]
The structure of an autonomous vehicle 10c according to a third modified example of the invention will be described below with reference to the drawings. 11 and 12 are image data generated by the imaging device 12. FIG. As for the autonomous vehicle 10c and the recognition device 14c according to the third modification, only the portions different from the autonomous vehicle 10 and the recognition device 14 according to the first embodiment will be explained, and the rest will be omitted.

The autonomous vehicle 10 c includes a recognition device 14 c instead of the recognition device 14 . In addition, the laser light irradiation device 13 includes a plurality of laser light sources (not shown) that emit laser light L. As shown in FIG. In this case, as shown in FIGS. 11 and 12, the recognition device 14c identifies each of the plurality of laser beams LB, which are extension lines of the trajectories of the plurality of laser beams L (laser beam identification step: step S3 ). This improves the accuracy of identifying the vanishing point O, and improves the accuracy of identifying the structure of the shelf T1.

[Other embodiments]
The recognition device according to the present invention is not limited to the autonomous vehicles 10, 10a to 10c, and can be modified within the scope of the gist thereof. Also, the structures of the autonomous vehicles 10, 10a to 10c may be combined arbitrarily.

In the recognition devices 14, 14a to 14c, the image data represents an image based on the one-point perspective method. However, the image data may represent images in two-point perspective and three-point perspective. In the case of two-point perspective, the recognition devices 14, 14a-14c identify each of the two vanishing points according to the flow chart of FIG. In the case of three-point perspective, the recognition devices 14, 14a-14c identify each of the three vanishing points according to the flow chart of FIG.

Note that the recognition devices 14, 14a to 14c recognize a plurality of markers arranged in a matrix on the left and right sides of the imaging device 12 in the front-rear direction and the vertical direction. However, the recognition devices 14, 14a to 14c may recognize a plurality of markers arranged in a matrix in the front-rear direction and the vertical direction on the left or right of the imaging device 12. FIG. In this case, the recognition devices 14, 14a to 14c, for example, after recognizing a plurality of markers arranged in a matrix in the front-rear direction and the up-down direction on the left side of the imaging device 12, capture an image based on the recognized markers. The arrangement of markers arranged in a matrix in the front-back direction and the up-down direction on the right side of the device 12 may be estimated.

The order of the acquisition step, the first recognition step, the laser beam identification step, the eye level identification step, the vanishing point identification step, the first perspective line identification step, and the second recognition step is not limited to the illustrated order. The order of these steps may be interchanged without conflict. For example, the eye level identification step may be performed before the first recognition step.

Note that the laser light L does not have to travel straight on a horizontal plane perpendicular to the vertical direction. For example, the recognition devices 14, 14a-14c may correct the laser beam LB with reference to the eye level EL.

Note that the laser beam L may pass through at least one of the markers M11, M21, M31, M41, M12, M22, M32, and M42.

Note that the color of the laser light L is not limited to a specific color.

Note that the beam diameter of the laser light L is not limited to a specific size.

Note that the markers M11, M21, M31, M41, M12, M22, M32, and M42 are not limited to being provided on the shelves T1 to T3. For example, by projection mapping, markers M11, M21, M31, M41, M12, M22, M32 and M42 may be projected onto the shelves T1 to T3.

10, 10a-10c: Autonomous vehicle 12: Imaging device 13: Laser light irradiation device 14, 14a-14c: Recognition device EL: Eye level F: Frame L: Laser light L1: First row line L2: Second row line LB: Laser beams M01, M02: Virtual markers M11, M21, M31, M41, M12, M22, M32, M42, Mij: Markers O: Vanishing points PL11 to PL14: First perspective line PL3: Third perspective lines T1 to T3 : shelves m11, m22, m31, m41: recognized markers m12, m21, m32, m42: unrecognized markers

Claims (1)

an autonomous vehicle,
an imaging device that captures an area to the left of the autonomous vehicle or an area to the right of the autonomous vehicle;
a laser light irradiation device including a laser light source that emits laser light;
A recognition device that recognizes a plurality of markers arranged in a matrix in the front-rear direction and the up-down direction on the left or right from the imaging device, and acquires image data including the plurality of markers and the laser beam from the imaging device. a recognition device that
comprising
autonomous vehicle.

Images