JP7415830B2 - unmanned vehicle - Google Patents

Description

The present invention relates to an unmanned vehicle.

SLAM (Simultaneous Localization and Mapping) is a technology for estimating the state (location information, etc.) of unmanned vehicles. When estimating the self-position, SLAM estimates the self-position of an unmanned vehicle by comparing the surrounding environment map (initial map) acquired in advance with the surrounding environment acquired by the sensor. Since the map changes over time, if the initial map is not updated in some way, it will lead to deterioration of self-position estimation accuracy or loss of self-position estimation. For example, in the interlocking system of an automatic guided vehicle and an inventory management system disclosed in Patent Document 1, the map management unit updates the environmental map used by the automatic guided vehicle in real time based on the storage/output information of the inventory management system. Improve the accuracy of environmental maps.

Japanese Patent Application Publication No. 2015-172878

By the way, an inventory management system is required in the case where an environmental map used by an automatic guided vehicle is updated in real time based on the input/output information of the inventory management system. In addition, if there are no existing structures with characteristics in the surrounding environment, SLAM cannot be used in the first place, and when comparing the environmental map with the surrounding environment acquired by an environmental sensor (during matching), the surrounding environment is used as environmental map information. It is better to have as much environmental information as possible.

An object of the present invention is to provide an unmanned vehicle that can estimate its own position even in an environment with few existing structures.

An unmanned vehicle for solving the above problems has a vehicle body, an environmental sensor installed on the vehicle body that detects objects existing around the vehicle body, and geographical location information is linked, and temporary structures are removed. a first sensor map that includes existing structures and temporary structures acquired by the environmental sensor at the current position of the unmanned vehicle; a deletion unit that deletes an existing structure corresponding to the existing structure on the map to generate a second sensor map including the temporary structure, and synthesizes the initial map and the second sensor map at the current position of the unmanned vehicle. A compositing unit that creates a composite map including existing structures and temporary structures, and a composite map linked with geographical location information, are compared with the surrounding environment acquired by the environmental sensor. The gist of the present invention is to include a self-position estimating section that estimates the self-position of the vehicle.

According to this, in the deletion section, the existing structures in the initial map at the current position of the unmanned vehicle are matched from the first sensor map including the existing structures and temporary structures acquired by the environmental sensor at the current position of the unmanned vehicle. The existing structures are deleted and a second sensor map including the temporary structures is generated. In the synthesis section, the initial map at the current position of the unmanned vehicle and the second sensor map are synthesized to create a composite map including existing structures and temporary structures. Then, in the self-position estimating section, the self-position of the unmanned vehicle is estimated by comparing the composite map linked with the geographical position information and the surrounding environment acquired by the environmental sensor.

Therefore, since the self-position of an unmanned vehicle is estimated by comparing a composite map that includes temporary structures and existing structures in the surrounding environment with the surrounding environment acquired by an environmental sensor, the self-position can be estimated even in an environment with few existing structures. Estimates can be made.

Further, in the unmanned vehicle, the synthesis unit may create the composite map every time the vehicle travels a certain distance or every time a certain period of time passes.
Further, in the unmanned vehicle, it is preferable that the synthesis unit adds storage period information to the temporary structure in the synthetic map, and deletes the temporary structure from the synthetic map when the storage period has expired.

According to the present invention, self-position estimation can be performed even in an environment with few existing structures.

FIG. 1 is a side view of an unmanned forklift in an embodiment. FIG. 2 is a block diagram showing the electrical configuration of an unmanned forklift. FIG. 3 is a plan view for explaining object detection by a laser sensor of an unmanned forklift. An explanatory diagram of data processing for explaining the effect. Flowchart for explaining the action.

An embodiment embodying the present invention will be described below with reference to the drawings.
As shown in FIG. 1, an unmanned forklift 10 as an unmanned vehicle includes a vehicle body 11, a driving wheel 12 disposed at the front lower part of the vehicle body 11, and a steering wheel 13 disposed at the rear lower part of the vehicle body 11. . The unmanned forklift 10 includes a cargo handling device 14 in front of a vehicle body 11. The cargo handling device 14 includes a mast 15 erected at the front of the vehicle body 11, a lift bracket 16 attached to the mast 15, and a pair of forks 17 fixed to the lift bracket 16. A load is loaded onto the fork 17. Although not shown, the load is loaded onto the fork 17 while being mounted on a pallet. In this embodiment, the front of the cargo handling device 14 and the front of the vehicle body 11 are aligned. The front of the cargo handling device 14 is the direction in which the fork 17 extends. The cargo handling device 14 includes a lift cylinder 18 that moves the mast 15 up and down. The cargo handling device 14 includes a tilt cylinder 19 that tilts the mast 15. Lift cylinder 18 and tilt cylinder 19 are hydraulic cylinders.

As shown in FIG. 2, the unmanned forklift 10 includes a drive mechanism 21, a hydraulic mechanism 22, a control device 23, and a laser sensor 40 as an environmental sensor. The drive mechanism 21 is a member for running the unmanned forklift 10, and includes a running motor for driving the drive wheels 12 and a steering mechanism for steering the steering wheels 13. The hydraulic mechanism 22 is a member for controlling supply and discharge of hydraulic oil to the lift cylinder 18 and the tilt cylinder 19, and includes a cargo handling motor for driving a pump and a control valve.

The control device 23 includes a processing section 24 and a storage section 25. The processing unit 24 includes a conversion unit 26 , a self-position estimation unit 27 , a deletion unit 28 , and a synthesis unit 29 . The self-position estimating section 27 includes a matching section 30.

The storage unit 25 stores various programs for controlling the unmanned forklift 10. The control device 23 may include dedicated hardware that executes at least some of the various processes, such as an application-specific integrated circuit (ASIC). The control device 23 may be configured as a circuit including one or more processors operating according to a computer program, one or more dedicated hardware circuits such as an ASIC, or a combination thereof. The processor includes a CPU and memory such as RAM and ROM. The memory stores program codes or instructions configured to cause the CPU to perform processes. Memory, or computer readable media, includes anything that can be accessed by a general purpose or special purpose computer.

The control device 23 operates the unmanned forklift 10 by controlling the drive mechanism 21 and the hydraulic mechanism 22. The unmanned forklift 10 of this embodiment is a forklift that automatically travels, steers, and handles cargo under the control of the control device 23 without being operated by a passenger.

The unmanned forklift 10 is used in factories, distribution centers, airports, ports, and other workplaces where it is necessary to transport loads.
As shown in FIG. 1, the laser sensor 40 is disposed at the rear end of a head guard 20 that partitions a driver's seat in the vehicle body 11 so as to face toward the rear of the vehicle body. The laser sensor 40 may be placed at any position on the vehicle body 11.

As shown in FIG. 3, the laser sensor 40 provided on the vehicle body 11 is for detecting an object Obj1 existing around the vehicle body 11, and irradiates the surrounding area with a laser and reflects from the object Obj1 that is hit by the laser. This is a laser range finder that can recognize the surrounding environment by receiving reflected light. As the laser sensor 40 of this embodiment, a two-dimensional laser range finder that emits laser while changing the irradiation angle in the horizontal direction is used. The map is a point group (a collection of points represented by X and Y) obtained in advance with a two-dimensional laser range finder.

Note that as an environmental sensor for acquiring the surrounding environment, a three-dimensional laser sensor, for example, 3DLiDAR, may be used instead of a two-dimensional laser sensor, and the initial map and composite map, which are surrounding environment maps, are prepared in advance using 3DLiDAR, etc. It may be a point group (a collection of points represented by X, Y, and Z) acquired in the above. That is, a three-dimensional laser range finder that changes the irradiation angle in the vertical direction as well as the horizontal direction may be used.

The laser irradiation range of the laser sensor 40 is, for example, a range centered on the axis B extending rearward of the unmanned forklift 10. The laser sensor 40 is arranged to irradiate a laser beam to the rear of the unmanned forklift 10. In the following description, the direction in which the axis B extends is assumed to be the Y direction, and the direction perpendicular to the axis B in the horizontal direction is assumed to be the X direction.

Laser sensor 40 measures the distance to object Obj1. The laser sensor 40 outputs the distance to the object Obj1 to the control device 23 in association with the irradiation angle. It can be said that the measurement result of the laser sensor 40 indicates the relative coordinates of the object Obj1 with respect to the unmanned forklift 10. The control device 23 detects the relative angles θ1 and θ2 of the object Obj1 with respect to the unmanned forklift 10 and the relative distance of the object Obj1 with respect to the unmanned forklift 10 from the measurement results of the laser sensor 40. The relative angle of the object Obj1 with respect to the unmanned forklift 10 is the angle formed by the object Obj1 and the axis B which is the backward movement direction when the unmanned forklift 10 is moving straight, that is, the steering angle of the unmanned forklift 10 is 0. be. The relative distance of the object Obj1 to the unmanned forklift 10 is the distance from the unmanned forklift 10 to the object Obj1.

As shown in FIG. 2, the storage unit 25 of the control device 23 stores (registers) an initial map IM and a composite map CM, which are surrounding environment maps. Both the initial map IM and the composite map CM are stored (registered) with geographical position information linked to them. Geographical location information is information indicating the coordinates of a location. The initial map IM and composite map CM, which are surrounding environment maps, are information regarding the physical structure of the environment around the unmanned forklift 10, such as the shape and size of the environment in which the unmanned forklift 10 is used. The coordinates of the location are set within the map.

In the initial map IM, as a preprocessing, an operator looks at the image and manually creates a temporary structure TS1 from point cloud data acquired in advance, as shown in map data MD1 in FIG. 4 and map data MD2 in FIG. is removed. Specifically, the map data MD1 in FIG. 4 includes an existing structure ES1 and a temporary structure TS1 on a map expressed in an orthogonal two-axis coordinate system (XY coordinate system) behind the unmanned forklift 10. . Temporary structure TS1 is removed from map data MD1 as a preprocess. As a result, in the map data MD2 of FIG. 4, the temporary structure TS1 is excluded and the existing structure ES1 is It will contain.

In this way, the initial map IM in the storage unit 25 is associated with geographical position information and shows the surrounding environment with the temporary structure TS1 removed. An initial map IM in the storage unit 25 is created from the surrounding environment acquired by the laser sensor 40.

The initial map IM, which is a map of the surrounding environment, may be stored in the storage unit 25 in advance if the surrounding environment in which the unmanned forklift 10 is used can be grasped in advance. When storing the initial map IM, which is an environmental map, in the storage unit 25 in advance, the coordinates of existing structures such as walls and pillars of buildings are stored as the environmental map. The initial map IM and composite map CM, which are surrounding environment maps, are created by mapping using SLAM (Simultaneous Localization and Mapping). SLAM simultaneously estimates the self-position of a mobile object and creates an environmental map, and allows the mobile object to create an environmental map in an unknown environment. It uses the map information it has built to perform specific tasks while avoiding obstacles.

The control device 23 moves the unmanned forklift 10 to a desired position by controlling the drive mechanism 21 while performing self-position estimation to estimate the position of the unmanned forklift 10 on the initial map IM and composite map CM, which are surrounding environment maps. It is possible to move 10. The self-position is the coordinates indicating one point on the vehicle body 11, for example, the coordinates of the center of the vehicle body 11 in the horizontal direction.

There are existing structures and temporary structures in the surrounding environment. Existing structures are things that will not change forever (buildings, etc.), and temporary structures are things that will change (vehicles, luggage, etc.). The temporary structure is, for example, a pallet, which is placed into position by, for example, a container truck.

As shown in FIG. 4, the storage unit 25 stores a composite map CM including the existing structure ES1 and the temporary structure TS1. The self-position estimating unit 27 in FIG. It is now possible to estimate the self-location of

Next, the effect will be explained.
FIG. 5 shows the processing executed by the processing unit 24, and the processing executed in the processing unit 24 will be explained using FIG.

In FIG. 4, it is assumed that an initial map indicated by code MD2 at the current position of the unmanned forklift 10 is stored in the storage unit 25 as an environmental map. Further, in FIG. 4, it is assumed that a sensor map, that is, scan data (map coordinate system) is obtained by the laser sensor 40 indicated by symbol SM1 at the current position of the unmanned forklift 10.

The process in FIG. 5 is activated at regular intervals, and the process of step S10→step S11→step S12→step S13→step S14 is executed.
In step S10 of FIG. 5, the conversion unit 26 in FIG. 2 acquires surrounding environment data (scan data) using the laser sensor 40 and temporarily stores the scan data (sensor coordinate system).

In step S11 of FIG. 5, the conversion unit 26 in FIG. 2 converts the scan data (sensor coordinate system) into a map coordinate system using the current coordinates (current position) and temporarily converts the scan data into the map coordinate system. Remember. This scan data (map coordinate system) is represented by the first sensor map SM1 in FIG. The first sensor map SM1 in FIG. 4 includes an existing structure ES10 and a temporary structure TS10 in a two-axis coordinate system (XY coordinate system) orthogonal to each other behind the unmanned forklift 10.

In this way, the conversion unit 26 in FIG. 2 converts the surrounding environment data Ad1 (see FIG. 4) including the existing structure ES10 and the temporary structure TS10 acquired by the laser sensor 40 as an environment sensor into the map coordinate system. A first sensor map SM1 including the existing structure ES10 and the temporary structure TS10 is generated.

The deletion unit 28 in FIG. 2 deletes the scan data irradiated on the existing structure using the obtained initial map (map from which temporary structures have been removed) IM in step S12 in FIG. is temporarily stored as a map coordinate system. That is, from the first sensor map SM1 including the existing structures ES10 and temporary structures TS10 acquired by the laser sensor 40 at the current position of the unmanned forklift 10, to the existing structures ES1 in the initial map IM at the current position of the unmanned forklift 10. A second sensor map SM2 including the temporary structure TS10 is generated by deleting the corresponding existing structure ES10. That is, the first sensor map SM1 and the initial map IM are compared, and an object that is common to both the first sensor map SM1 and the initial map IM is deleted from the first sensor map SM1 as an existing structure.

When deleting scan data irradiated to an existing structure and temporarily storing the scan data as a map coordinate system, scan data whose distance difference from the existing structure ES1 in the initial map IM is 0.5 m or less is deleted. The points are deleted from the scan data as points that were irradiated onto the existing structure. That is, the deletion unit 28 deletes a point detected by the laser sensor 40 whose distance difference from the object Obj1 is equal to or less than a threshold value as a point irradiated onto an existing structure. This is because the scan data is converted to the map coordinate system using the current position (current coordinates), so it contains errors at this point, and this is to prevent unnecessary data from being registered. This is to prevent deterioration of estimation accuracy.

The scan data (map coordinate system) after deletion is represented by the second sensor map SM2 in FIG. In the second sensor map SM2 of FIG. 4, in the orthogonal two-axis coordinate system (XY coordinate system) behind the unmanned forklift 10, the existing structure ES10 is deleted and a temporary structure TS10 is included.

In step S13 of FIG. 5, the synthesizing unit 29 in FIG. 2 synthesizes the initial map (map from which temporary structures have been removed) IM and the scan data converted into the map coordinate system, and stores the synthesized map CM. That is, the initial map IM at the current position of the unmanned forklift 10 and the second sensor map SM2 are combined to create a composite map CM. This is registered in the storage unit 25 in association with geographical position information. To explain using FIG. 4, by combining the initial map IM and the second sensor map SM2, existing structures ES1 and temporary A composite map CM including the structure TS10 is obtained.

At this time, the synthesis unit 29 creates a composite map CM every time the laser sensor 40 acquires data.
In step S14 in FIG. 5, the matching unit 30 of the self-position estimating unit 27 in FIG. 2 performs matching ( For example, the current coordinates (self-position) are estimated by NDT matching). That is, the self-position of the unmanned forklift 10 is estimated by comparing the synthetic map CM with which geographical position information is linked and the surrounding environment acquired by the laser sensor 40.

When the map data MD1 in FIG. 4 and the scan data (first sensor map SM1) are compared, they do not match, so self-position estimation cannot be performed. However, in this embodiment, information on the current temporary structure is added to the map data MD2 and registered. This makes it easier to match the scan data with the environmental map, preventing data loss and increasing accuracy.

That is, conventionally, an inventory management system is required when an environmental map used by an automatic guided vehicle is updated in real time based on input/output information from an inventory management system. Furthermore, if there are no existing structures with characteristics in the surrounding environment, SLAM cannot be used in the first place. Furthermore, when comparing the environmental map and the surrounding environment acquired by the environmental sensor (during matching), it is better to have as much surrounding environment information as the environment map information. Furthermore, in order to estimate one's own position, it is better to have structures nearby, and if there are structures that temporarily exist, the background map information will become invisible from the current position, and the information will be acquired using an environmental map and an environmental sensor. When comparing the surrounding environment, self-position estimation is performed using only what is visible (during matching), leading to loss of self-position and deterioration of self-position estimation accuracy.

In this embodiment, the self-position of an unmanned vehicle is estimated using temporary structures in the surrounding environment by comparing the surrounding environment acquired by an environmental sensor and a map synthesized by a synthesis section. Even in a small environment, it becomes easier to match the surrounding environment with the map synthesized by the compositing section, making it less likely to be lost and increasing accuracy. In addition, by estimating the self-position using temporary structures that match the current scan data, even in environments with large changes in the surrounding environment, it becomes easier to match the surrounding environment with the map synthesized by the compositing section, making it possible to eliminate lost data. The accuracy increases as it becomes more difficult to do so. In this way, self-position estimation becomes possible even in environments with few existing structures and environments with large changes in the surrounding environment.

Specifically, it is used under the following circumstances.
During the busy season, there are many cargoes and trucks coming in and out of a distribution center (the surrounding environment changes significantly). At airports and container yards, there are few existing structures and many temporary structures (for example, containers). Therefore, in particular, there may be a situation where there are few existing structures and the surrounding environment changes significantly.

Even under such an environment, in this embodiment, it is difficult to cause a loss in which the self-position cannot be estimated, even in an environment with large changes in the surrounding environment or in an environment with few existing structures.
According to the above embodiment, the following effects can be obtained.

The configuration of the unmanned forklift 10 as an unmanned vehicle is such that a vehicle body 11, a laser sensor 40 as an environmental sensor that is provided on the vehicle body 11 and detects an object Obj1 existing around the vehicle body 11, and geographical position information are linked. , a storage unit 25 that stores an initial map IM showing the surrounding environment from which the temporary structure TS1 has been removed; A deletion unit 28 that deletes the existing structure ES10 corresponding to the existing structure ES1 in the initial map IM at the current position of the unmanned forklift 10 from the one-sensor map SM1 to generate a second sensor map SM2 including the temporary structure TS10. , a synthesis unit 29 that synthesizes the initial map IM at the current position of the unmanned forklift 10 and the second sensor map SM2 to create a composite map CM including the existing structure ES1 and the temporary structure TS10, and geographic position information. The self-position estimating unit 27 estimates the self-position of the unmanned forklift 10 by comparing the synthetic map CM with which the unmanned forklift 10 is linked and the surrounding environment acquired by the laser sensor 40.

Therefore, in order to estimate the self-position of the unmanned forklift 10 by comparing a composite map CM including temporary structures and existing structures in the surrounding environment with the surrounding environment acquired by the laser sensor 40 as an environment sensor, the existing structure Self-position estimation can be performed even in environments with few objects. Further, since existing structures and temporary structures detected by the laser sensor 40 at the current position of the unmanned forklift 10 are used for matching, self-position estimation can be performed even in an environment with large changes in the surrounding environment.

The embodiment is not limited to the above, and may be embodied as follows, for example.
In the embodiment described above, a composite map is created every time scan data is acquired as shown in FIG. 5, but a composite map may be created every time a certain distance is traveled or every time a certain time elapses.

Since the accuracy of the current coordinates (current position) is not high, temporary structures may also be registered with a difference, and if you use them to match scan data every time data is acquired, the map may gradually shift, but the map may shift over a certain distance. By creating a composite map every time a vehicle travels or after a certain period of time, the accumulation of errors is reduced. Moreover, there is an effect that the calculation processing load is reduced.

In this way, the synthesis unit 29 may create a composite map CM every time the vehicle travels a certain distance or every time a certain period of time passes.
- Regarding how long to store temporary structures, for example, if the temporary structure is a car, there is a possibility that it will be moved soon, so we would like to avoid registering it in the data as much as possible. On the other hand, there is no problem in registering pallets and the like that are unlikely to move for a long period of time. Therefore, storage period information may be added to each temporary structure synthesized on the initial map, and the temporary structure may be deleted from the map when the storage period has expired. The retention period is determined from the viewpoint of reliability. Specifically, a camera captures an image of the surrounding area and uses deep learning to perform image recognition processing to recognize that it is a moving object such as a car or a stationary object such as a pallet. period, and if it is a pallet, it is a long period.

In this case, the accuracy of self-position estimation can be improved, and it is possible to prevent the occurrence of a loss in which the self-position cannot be estimated.
In this way, the synthesizing unit 29 can also add information about the retention period to the temporary structure TS10 in the composite map CM, and delete it from the composite map CM when the retention period has expired.

〇In step S12 in FIG. 5, when deleting the scan data irradiated to the existing structure in order to delete the existing structure ES10 corresponding to the existing structure ES1 in the initial map IM from the first sensor map SM1, Objects that hardly move may be registered on the map as existing structures. In other words, when a stationary object like this is detected by a laser sensor, it is initially treated as a temporary structure, but if it is determined that it hardly moves, it can be treated as an existing structure and registered in the composite map or initial map. good.

In this case, it is possible to improve the accuracy of self-position estimation, and it is possible to prevent the occurrence of a loss in which the self-position cannot be estimated.
A laser sensor was used as the environmental sensor, but other than the laser sensor, for example, a stereo camera or the like may be used.

Although the vehicle was a forklift, the invention is not limited to this, and may be applied to industrial vehicles other than forklifts, or may be applied to vehicles other than industrial vehicles.

10...Unmanned forklift, 11...Vehicle body, 25...Storage unit, 27...Self-position estimation unit, 28...Deletion unit, 29...Composition unit, 40...Laser sensor, CM...Synthesis map, ES1...Existing structure, ES10...Existing installation Structure, IM...Initial map, Obj1...Object, SM1...First sensor map, SM2...Second sensor map, TS10...Temporary structure.

Claims (5)

a vehicle body including a drive mechanism and a hydraulic mechanism ;
a control device that operates the vehicle body by controlling the drive mechanism and the hydraulic mechanism;
The control device includes:
an environmental sensor that is provided on the vehicle body and detects objects existing around the vehicle body;
a storage unit that stores an initial map associated with geographical location information and showing the surrounding environment with temporary structures removed;
Deleting existing structures corresponding to existing structures in the initial map at the current location of the unmanned vehicle from a first sensor map including existing structures and temporary structures acquired by the environmental sensor at the current location of the unmanned vehicle. a deletion unit that generates a second sensor map including the temporary structure;
a synthesis unit that synthesizes the initial map and the second sensor map at the current position of the unmanned vehicle to create a composite map including existing structures and temporary structures;
a self-position estimating unit that estimates the self-position of the unmanned vehicle by comparing the composite map to which geographical position information is linked and the surrounding environment acquired by the environmental sensor ;
The environmental sensor irradiates the surrounding area with a laser and outputs surrounding environment data to the control device as a measurement result of reflected light that hits objects including existing structures and temporary structures, and
The control device further includes a conversion unit that associates geographic position information with the surrounding environment data acquired by the environmental sensor,
The control device performs preprocessing before the self-position estimation unit estimates the self-position,
The preprocessing includes acquiring the surrounding environment data at the current position of the unmanned vehicle with the environmental sensor, and the converting unit associating the acquired surrounding environment data with geographic position information, thereby determining whether the unmanned vehicle exists around the unmanned vehicle. 1. An unmanned vehicle comprising the steps of: creating map data in which an object is shown, and creating the initial map from the map data and storing it in a storage unit . The deletion unit compares the first sensor map and the initial map and deletes an object that is common to both the first sensor map and the initial map from the first sensor map as an existing structure. The unmanned vehicle described in item 1. The unmanned vehicle according to claim 1 or 2, wherein the deletion unit deletes points on the initial map whose distance from an existing structure is equal to or less than a threshold value as points irradiated onto an existing structure. The unmanned vehicle according to any one of claims 1 to 3, wherein the synthesis unit creates the composite map every time a certain distance is traveled or every time a certain period of time has elapsed. Any one of claims 1 to 4, wherein the synthesis unit adds storage period information to the temporary structure in the synthetic map, and deletes the temporary structure from the synthetic map when the storage period expires. Unmanned vehicles as described in Section .