EP4695789A1

EP4695789A1 - Tracking system for industrial vehicles

Info

Publication number: EP4695789A1
Application number: EP24723949.4A
Authority: EP
Inventors: Matteo LUGLI; Salvatore Milazzo
Original assignee: Roboptica Srl
Current assignee: Roboptica Srl
Priority date: 2023-04-12
Filing date: 2024-04-08
Publication date: 2026-02-18
Also published as: IT202300007002A1; WO2024213984A1

Abstract

The tracking system (1) for industrial vehicles comprises acquisition means (2) installed on an industrial vehicle (V) to be tracked and configured to acquire an image (I) of the surrounding environment and/or data relating to the distances of surrounding objects, a storage unit (3) comprising a global map (GM) of an industrial area (A), a processing unit (4) configured to determine the position of the vehicle (V) within said area (A), wherein the global map (GM) comprises global coordinates relating to a plurality of global references (GR), and wherein the processing unit (4) is configured to iteratively repeat the following steps: determining a plurality of local coordinates relating to a plurality of local references (LR); identifying global references (GR) on the global map (GM) corresponding to at least some of the determined local references (LR); determining the position (P) of the vehicle (V) within said area (A) by transforming the local coordinates of the vehicle (V) into corresponding global coordinates within the global map (GM);v updating the global map (GM) by adding/deleting at least one global reference (GR).

Description

TRACKING SYSTEM FOR INDUSTRIAL VEHICLES

Technical Field

The present invention relates to a tracking system for industrial vehicles. Background Art

As is well known, the tracking of industrial vehicles is a key issue for fleet management and for tracing the movements within industrial facilities.

In fact, vehicles such as forklift trucks, shuttles and wagons are used to move things or people both in outdoor yards and inside buildings.

The currently most reliable solution to know the position of a vehicle in real time is to mount on board a Global Navigation Satellite System (GNSS) device, which intercepts the electromagnetic signals from one or more satellite constellations and provides the vehicle’s position in geographic coordinates.

However, the electromagnetic radiation from satellites is severely degraded or even absent inside buildings, making the use of such system ineffective.

Several solutions have been introduced to solve the problem of vehicle tracking in indoor environments, each based on a specific technology of reference.

For example, the use of UltraWideBand (UWB) antennas is well known which are installed on board the vehicle and deployed inside the building to enable the replication of GNSS operation indoors.

Alternatively, the use of Light Imaging, Detection, And Ranging (LIDAR) devices is known together with the installation of several particularly reflective points of reference at suitable positions.

In this way, the position of the vehicle can be determined by means of the reflector distances detected by the LIDAR device.

Finally, there are several solutions developed to determine the position of vehicles with the aid of cameras.

For example, solutions are known which employ a camera mounted on top of the vehicle and facing upwards so as to observe the ceiling of the facility, on which appropriate markers, such as labels or the like, are placed.

In addition, solutions are known which employ a camera installed on board the vehicle and facing the floor. Again, appropriate markers recognizable by the system are installed on the floor.

However, in all of the known solutions described above, the installation and maintenance of an appropriate infrastructure of points of reference is necessary, as the position of the vehicle is determined according to these points of reference. Necessarily, this requires significant effort on the part of the user of the system. In fact, a large number of devices, both active and passive, are required to be deployed at the time of the system startup, while during the life cycle their constant maintenance is necessary so that the system does not degrade the performance thereof.

In addition, under certain conditions the installation of such devices is prohibited or made problematic by the special configurations of the environment.

Description of the Invention

The main aim of the present invention is to devise a tracking system for industrial vehicles which allows the limitations set forth above for known solutions to be overcome by enabling the tracking of a vehicle by means of the natural references within the environment, without requiring either an installation of references at startup or their subsequent maintenance.

The aforementioned objects are achieved by this tracking system for industrial vehicles according to the characteristics described in claim 1.

Brief Description of the Drawings

Other characteristics and advantages of the present invention will become more apparent from the description of a preferred, but not exclusive, embodiment of a tracking system for industrial vehicles, illustrated by way of an indicative, yet non-limiting example, in the accompanying tables of drawings in which:

Figure 1 and Figure 2 show a possible industrial vehicle which can be used with the tracking system according to the invention;

Figure 3 shows a possible industrial environment within which the management of the industrial vehicles and of the materials is carried out using the tracking system according to the invention;

Figure 4 is a block diagram which schematically shows the tracking system according to the invention. Embodiments of the Invention

With particular reference to these figures, reference numeral 1 globally denotes a tracking system for industrial vehicles, which can be employed in particular for the management of the material movement within an industrial area, of the type of a warehouse or the like.

The tracking system 1 comprises acquisition means 2 installed on at least one industrial vehicle V to be tracked and configured to acquire at least one image I of the surrounding environment of the vehicle V and/or data relating to the distances of objects with respect to the vehicle V.

The acquisition means 2 are appropriately positioned so as to observe the surrounding environment of the vehicle V.

According to preferred embodiments, the acquisition means 2 comprise at least one of either a stereoscopic camera or a LIDAR system.

In addition, the tracking system 1 comprises at least one storage unit 3 comprising at least one global map GM of an industrial area A within which the vehicle V is to move.

In addition, the tracking system 1 comprises at least one processing unit 4 operationally connected to the acquisition means 2 and to the storage unit 3 and configured to determine the position of the vehicle V within the area A starting from the acquired image I and/or from the acquired data relating to the distances. With reference to one possible embodiment, shown in the figures, the processing unit 4 is implemented by means of a local processor 5 installed on the vehicle V, while the storage unit 3 is implemented on a remote processor 6. In such a case, the local processor 5 can communicate with the remote processor 6 by means of wireless communication means (Wi-Fi, Bluetooth or the like).

Alternative embodiments cannot however be ruled out wherein the processing unit 4 and the storage unit 3 are implemented and deployed differently.

For example, both the processing unit 4 and the storage unit 3 can be implemented on a remote processor, while the vehicle V, in such a case, can be provided with wireless communication means for sending the images I and/or the collected distance data of the acquisition means 2 to the remote processor. Alternatively, both the processing unit 4 and the storage unit 3 can be installed on the vehicle V.

Conveniently, the tracking system 1 may comprise an on-board computer 7 installed on the vehicle V and configured to interface with the user of the vehicle. Advantageously, according to the invention, the global map GM comprises global coordinates relating to a plurality of global references GR, wherein the global references GR correspond to fixed structural elements E present within the area A and to the material M deposited and stationary within the area A, and wherein the global coordinates are determined with respect to a predefined point of origin O on the global map GM.

In addition, the processing unit 4 is configured to iteratively repeat the following steps: starting from the at least one acquired image I and/or from the acquired data relating to the distances, determining a plurality of local coordinates relating to a plurality of local references LR, wherein the local references LR correspond to fixed structural elements E present within the area A and/or to material M deposited and stationary within the area A, and wherein the local coordinates are determined by considering a point of origin on the vehicle V (step 41 of Figure 4); identifying global references GR on the global map GM corresponding to at least some of the determined local references LR (step 42); starting from the identified global references GR, determining the position P of the vehicle V within the area A by transforming the local coordinates of the vehicle V into corresponding global coordinates within the global map GM (step 43); once the position P of the vehicle V has been determined, if at least one of the determined local references LR does not correspond to any global reference GR on the global map GM and/or if at least one global reference GR is present on the global map GM that is not among the determined local references LR, updating the global map GM by adding/deleting at least one global reference GR (step 44). Specifically, if at least one of the determined local references LR does not correspond to any global reference GR on the global map GM, then the processing unit 4 adds such global reference GR on the global map GM.

If, on the other hand, at least one global reference GR is present on the global map GM that is not among the determined local references LR, then the processing unit 4 deletes such global reference GR from the global map GM.

Regarding the transformation of the local coordinates of the vehicle V into corresponding global coordinates within the global map GM, according to a possible and preferred embodiment, given the correspondences between local and global references, the translations of the centroids, or centers of gravity, of the local points and global points into the origins of the respective coordinates are considered first. Next, the Kabsch’s algorithm is applied to evaluate the rotation. The final transformation is obtained by means of the composition of the previous transformations.

By means of the tracking process outlined above, the tracking system 1 is able to know in real time the position of each vehicle V within the mapped industrial area A and, therefore, can externally transmit such position P to other management systems, e.g., to track the movements of the material M within such area and to evaluate various warehouse performance indices (step 45).

With reference to the above, it should be noted that within this description the adjective “local” means the coordinates describing the points of reference observed by a vehicle V with respect to itself.

Instead, the coordinates of the elements of the mapped industrial area A, decided upon the initialization of the tracking system 1 and choosing the origin of the axes, are denoted as “global”.

Therefore, to know the position P of the vehicle V, it is necessary to determine the transformation that maps the local coordinates into global coordinates and vice versa.

By extension, the term “local map” denotes the set of references observed by the vehicle V and expressed in local coordinates, while the term “global map” denotes the set of global coordinates representing the most recent version of all known references of the facility.

It is also pointed out that the tracking system 1 exclusively records deposited and stationary material M. Thus, it is possible to construct a solid and consistent global map GM of usable references for tracking the vehicles.

As a result, the images I of the surrounding environment acquired from each of the tracked vehicles V allow the global map GM to be modified by addition or removal of references relating to deposited and/or picked material M. In this way, It is possible to have a dynamic management of the global map GM of references. Conversely, an updated global map GM of the material M allows the correct tracking of the vehicles V within the mapped industrial area A.

According to a preferred embodiment of the tracking system 1, the criterion for updating the global map is based on two basic assumptions: vehicles V move while the material M is stationary and material M may be picked up or deposited through the use of a suitably equipped vehicle V of the system, but not otherwise modified or deformed.

In particular, the second hypothesis specifies that the tracking system 1 describes the global map GM of the pre- and post-modification global references GR, but it does not manage any transitional phase between the two. Thus, the global map GM of the global references GR contains only existing and valid material M for subsequent tracking of the vehicles V.

In addition, the processing unit 4 is configured to determine the orientation D of the vehicle V starting from the position P of the vehicle itself and from the global coordinates of the identified global references GR.

The tracking system 1 uses the information of position P and orientation D of the vehicle V to compare the environment observed by the acquisition means 2 and that saved in the global map GM.

In fact, by knowing the position P and orientation D of the vehicle V, it is possible to transpose the observed points of reference into global coordinates and compare them with known ones.

Advantageously, according to a preferred embodiment, the tracking system 1 comprises at least one presence sensor 8 installed on gripping means F of the material M of the vehicle V. The presence sensor 8 is operationally connected to the processing unit 4 and is configured to detect the presence/absence of material M on the gripping means F.

With reference to this preferred embodiment, the processing unit 4 is configured to perform the following steps: detecting a loading/unloading operation of material M on/from the vehicle V by means of the presence sensor 8 (step 46); recording the position P of the vehicle V during such unloading/loading operation (step 47); starting from the recorded position P of the vehicle, determining an unloading/loading position of the material M (step 48); generating an invalidation volume VI within the global map GM (step 49), where said loading/unloading position is located.

Still with reference to the preferred embodiment of the tracking system 1, the step 44 of updating the global map, performed by the processing unit 4, comprises the following steps: if the global reference GR to be added to/removed from the global map GM corresponds to a generated invalidation volume VI, then the processing unit 4 adds/removes the global reference GR to/from the global map GM; if the global reference GR to be added to/removed from the global map GM does not correspond to a generated invalidation volume VI, then the processing unit does not add/remove the global reference GR to/from the global map GM.

Therefore, whenever the presence sensors 7 on the gripping means F detect a loading or unloading operation, the tracking system 1 records the exact position P of the vehicle, determined by means of the tracking algorithm. The position of the gripping means F at the time of loading or unloading allows determining an invalidation volume VI in global coordinates. This volume is invalidated by the logistic operation of the vehicle V. Only the previously invalidated volumes admit modification through the step of updating the global map GM.

Specifically, the global map GM thus obtained consists of: a list of characteristic points that represent valid references for the tracking algorithm, and a scattered and discrete representation of the invalid volumes.

Each logistic operation invalidates a volume of the global map GM. At this point a readout can add features to the global map GM only if they fall within the volumes recorded as invalidated. This operation creates a new global reference in the global map GM and simultaneously validates the map volume. Conversely, if a readout indicates that an invalidated area is empty, this too is validated.

With this criterion, the observations of the camera 2 and the notifications from the presence sensors 7 dynamically keep both the global map GM and the invalidation volume map VI updated in such a way as to prevent spurious modifications due to noise in the readouts and to ensure the accuracy thereof over time.

By using a dynamic global map GM, it is indeed vital for the tracking system 1 not to introduce errors during the updating process. Otherwise, after long periods of execution, the errors become integrated and the global map GM deviates from the truth over time.

Thus, therefore, the proposed tracking system 1 provides for a reliable tracking of the vehicles V at all times, guaranteed by an up-to-date global map GM of the references and assumed to be true.

In fact, this allows preventing errors during the phases of acquisition from leading to incorrect modifications in the global map GM.

Moreover, the integration of errors over time would affect the performance of the tracking system 1, as the new incorrect version of the global map GM would still be used in subsequent correspondence searches.

Preferably, the tracking system 1 comprises a plurality of presence sensors 7 installed on the gripping means F of the vehicle V, configured to detect the presence/absence of material M on the gripping means F and to detect at least one approximation of the volume occupied by the material M.

In particular, the presence of additional presence sensors 7 on board the vehicle V enables more accurate and sensitive detection of the volume of invalidation VI subject to modifications.

Therefore, this makes it possible to generate an invalidation volume VI to be added to the global map GM which has dimensions that conform as closely as possible to the volume actually occupied by the unloaded or loaded material M. The use of different types of sensors in addition to the presence sensors cannot also be ruled out.

For example, with reference to a vehicle consisting of a forklift truck, an additional height sensor 9 positioned on the forks is able to measure at what height the material is being moved.

According to a possible embodiment of the tracking system 1, the local references LR and the global references GR comprise characteristic points (features) which are recognizable by the processing unit 4 from different angles.

Indeed, it is necessary for the same feature to be observable and recognizable by the processing unit 4 from different perspectives.

Therefore, the aforementioned step of determining a plurality of local coordinates relating to a plurality of local references LR comprises processing the image acquired to identify the characteristic points thereof.

Specifically, each characteristic point synthesizes the data provided by the acquisition means 2 for a particular item and provides a well-defined position of that item in space.

Preferably, the characteristic points comprise at least one of the following parameters: orientation, size, color, color gradient.

These features serve to promote the search for correspondences between the local references LR of the local map and the global references GR of the global map GM during the aforementioned step of determining the position of the vehicle V. Specifically, the step of identifying global references GR on the global map GM corresponding to at least some of the determined local references LR comprises comparing and detecting correspondences between the characteristic points of local references LR and the characteristic points of global references GR on the global map GM. According to a possible and preferred embodiment, the step of identifying global references GR on the global map GM corresponding to at least some of the determined local references LR comprises at least the following steps: considering at least two local references LR and evaluating mutual distances and orientations between such local references LR considered; grouping the global references GR into sets of the same number and type as said at least two local references LR considered and evaluating mutual distances and orientations between the grouped global references GR for each set; comparing said at least two local references LR considered with all possible grouped global references GR and considering only those with the same mutual distances and orientations (preferably in accordance with appropriate tolerances).

Therefore, the local and global references are not compared with each other individually, but as pairs or more. This is because the position and orientation of a reference depend on the coordinates in which it is expressed, while the distance between two references or the mutual orientation is the same for all coordinate systems, in particular it is the same in both local map and global map.

Therefore, the observed characteristic points are compared with the known references in the global map GM made available through the tracking system 1. The global map GM itself consists of characteristic points that can be directly compared with those observed.

The tracking system 1 seeks the correspondence between the greatest number of local features and global features.

In addition, in order to determine the degree of reliability of the solution, the tracking system 1 comprises a criterion based on the quality of the references that discards solutions if they have a limited number of correspondences and/or are composed of references of low relevance.

For each solution found, the processing unit 4 is configured to perform at least the following steps: for each observed local reference LR, associating a significance value that represents both the quality of the detection thereof and the sporadicity of its possible observation within the area A; calculating the solution significance as the sum of the significances of each local reference LR, the correspondence of which with a global reference GR has been determined.

A vehicle V is defined as tracked if it has a sufficient number of correspondences between local references LR and global references GR and if the solution has a greater sum of significances than a certain threshold value.

The threshold value for discarding solutions with too low significance can be dynamic over time and depend on the vehicle situation.

In particular, a vehicle, the position of which is unknown, starts with a very high threshold level. As soon as a more or less accurate hypothesis about the position and orientation of the vehicle is available, this threshold can be reduced. At the same time, solutions that are too far from the hypothesis are prohibited. This reduces the size of the allowed global map and, consequently, the possibility of running into ambiguous solutions.

Finally, the ability to estimate a hypothesis of the position before executing the algorithm can greatly improve computational performance. Indeed, by knowing the area of the facility where the vehicle may be present, references outside this area can be neglected. In this way, the search for correspondences is greatly alleviated.

In addition, the startup of the tracking system 1 requires manual setting of the coordinate origin and the activation of a transient and temporary self-learning phase, during which modifications to the global map are allowed which are not justified by loading/unloading events.

Finally, the tracking system 1 may optionally comprise fixed fiducial references arranged within the industrial area at the time of installation and maintained unchanged over time.

For example, such markers can be printed on highly visible surfaces, bearing clearly distinguishable and unique patterns for each reference.

In such a hybrid configuration, position estimation is done by comparing both material and markers observed in the surroundings.

The correspondences found must satisfy both reference maps, that is, the static one consisting of the markers and the dynamic one consisting of the stored material.

This makes it possible to track the vehicle in those places in the facility where there is insufficient or no material at all.

It has in practice been ascertained that the described invention achieves the intended objects.

In particular, the fact is highlighted that the proposed solution introduces a system that is able to track a vehicle by means of the natural references in the environment, requiring neither an installation of references at startup nor their subsequent maintenance.

Specifically, the detected and used references are the material stored within the warehouse.

The innovative aspect of the solution particularly lies in the dynamic management of the reference map.

In fact, in all solutions of known type there are two execution phases: detection of local references and tracking by searching correspondences between the detected references and those known on the global map. In fact, the tracking process involves determining the function that maps the local coordinates to the global coordinates. All known tracking systems require the creation and management of a map of global references that remains constant over time. Modifications in references, such as adding, removing or displacing, require human intervention. Otherwise, the system cannot determine whether deviations between what is known and what is observed are due to a readout error or inaccuracy in the global map.

In a different way, the tracking system according to the invention enables the tracking of a vehicle by means of the natural references within the environment, without requiring either an installation of references at startup or their subsequent maintenance.

Claims

1) Tracking system (1) for industrial vehicles, comprising: acquisition means (2) installed on at least one industrial vehicle (V) to be tracked and configured to acquire at least one image (I) of the surrounding environment and/or data relating to the distances of objects surrounding said vehicle (V); at least one storage unit (3) comprising at least one global map (GM) of an industrial area (A) within which said vehicle (V) is to move; at least one processing unit (4) operationally connected to said acquisition means (2) and to said storage unit (3) and configured to determine the position of said vehicle (V) within said area (A) starting from said at least one image (I) and/or from said data relating to the distances; characterized by the fact that said global map (GM) comprises global coordinates relating to a plurality of global references (GR), wherein said global references (GR) correspond to fixed structural elements (E) present within said area (A) and to material (M) deposited and stationary within said area (A), and wherein said global coordinates are determined with respect to a predefined point of origin (O) on said global map (GM), and by the fact that said processing unit (4) is configured to iteratively repeat the following steps: starting from said at least one acquired image (I) and/or from said data relating to the distances, determining a plurality of local coordinates relating to a plurality of local references (LR), wherein said local references (LR) correspond to fixed structural elements (E) present within said area (A) and/or to material (M) deposited and stationary within said area (A), and wherein said local coordinates are determined by considering a point of origin on said vehicle (V); identifying global references (GR) on the global map (GM) corresponding to at least some of said determined local references (LR); starting from said identified global references (GR), determining the position (P) of said vehicle (V) within said area (A) by transforming the local coordinates of the vehicle (V) into corresponding global coordinates within the global map (GM); once said position (P) of the vehicle (V) has been determined, if at least one of said determined local references (LR) does not correspond to any global reference (GR) on the global map (GM) and/or if at least one global reference (GR) is present on the global map (GM) that is not among the determined local references (LR), updating said global map (GM) by adding/deleting at least one global reference (GR).

2) Tracking system (1) according to claim 1, characterized by the fact that said processing unit (4) is configured to determine the orientation (D) of said vehicle (V) starting from said position (P) of the vehicle (V) and from the global coordinates of said identified global references (GR).

3) Tracking system (1) according to one or more of the preceding claims, characterized by the fact that it comprises at least one presence sensor (8) installed on gripping means (F) of the material (M) of said vehicle (V) and operationally connected to said processing unit (4), said presence sensor (8) being configured to detect the presence/absence of material (M) on said gripping means (F).

4) Tracking system (1) according to claim 3, characterized by the fact that said processing unit (4) is configured to carry out the following steps: detecting a loading/unloading operation of material (M) on/from said vehicle (V) by means of said presence sensor (8); recording the position (P) of said vehicle (V) during said loading/unloading operation; starting from said recorded position (P) of the vehicle (V), determining a loading/unloading position of said material (M); generating an invalidation volume (VI) within said global map (GM) where said loading/unloading position is located.

5) Tracking system (1) according to claim 4, characterized by the fact that said step of updating the global map (GM) comprises the following steps: if said global reference (GR) to be added to/removed from the global map (GM) corresponds to a generated invalidation volume (VI), adding/removing said global reference (GR) to/from the global map (GM); if said global reference (GR) to be added to/removed from the global map (GM) does not correspond to a generated invalidation volume (VI), not adding/removing said global reference (GR) to/from the global map (GM).

6) Tracking system (1) according to one or more of claims 3 to 5, characterized by the fact that it comprises a plurality of presence sensors (8) installed on said gripping means (F) of the vehicle (V), said presence sensors (8) being configured to detect the presence/absence of material (M) on said gripping means (F) and to detect at least one approximation of the volume occupied by said material (M).

7) Tracking system (1) according to one or more of the preceding claims, characterized by the fact that said step of identifying global references (GR) on the global map (GM) corresponding to at least some of the determined local references (LR) comprises at least the following steps: considering at least two local references (LR) and evaluating mutual distances and orientations between said local references (LR) considered; grouping said global references (GR) into sets of the same number and type as said at least two local references (LR) considered and evaluating mutual distances and orientations between said grouped global references (GR) for each set; comparing said at least two local references (LR) considered with all possible grouped global references (GR) and considering only those global references (GR) which have the same mutual distances and orientations as said at least two local references (LR).

8) Tracking system (1) according to one or more of the preceding claims, characterized by the fact that said local references (LR) and said global references (GR) comprise characteristic points which are recognizable from different angles.

9) Tracking system (1) according to claim 8, characterized by the fact that said step of determining a plurality of local coordinates relating to a plurality of local references (LR) comprises processing said at least one image (I) acquired to identify said characteristic points. 10) Tracking system (1) according to one or more of claims 8 and 9, characterized by the fact that these characteristic points comprise at least one of the following parameters: orientation, size, color, color gradient.

11) Tracking system (1) according to one or more of claims 8 to 10, characterized by the fact that said step of identifying global references (GR) on the global map (GM) corresponding to at least some of said determined local references (LR) comprises comparing and detecting correspondences between the characteristic points of local references (LR) and the characteristic points of global references (GR) on the global map (GM). 12) Tracking system (1) according to one or more of the preceding claims, characterized by the fact that said acquisition means (2) comprise at least one of either a stereoscopic camera or a LIDAR system.