DE102020008065A1 - Method and system for determining a feature of a geographic line - Google Patents

Abstract

The invention relates to a method for determining one or more features of a geographic line, GL, or its surroundings. The method comprises: acquiring georeferenced earth observation data representing one or more spatially overlapping image planes covering a particular region of interest, ROI, of the earth's surface from an air or space perspective, the ROI comprising the GL; geometrically transforming the georeferenced observation data, at least in part, into a local internal frame of reference of the GL within the ROI to obtain a mapping of the georeferenced observation data to corresponding coordinates within the local internal frame of reference of the GL; and evaluating the mapped earth observation data as represented in the local internal reference system of the GL according to a classification scheme in order to obtain evaluation data therefrom which represent a classification of one or more properties of the GL or its surroundings according to one or more properties of the GL.

Description

The present invention relates to the field of image processing, and more particularly to the field of extracting and classifying features from images to classify and monitor conditions along geographic lines. More specifically, the invention relates to a method and a system for determining a nature of a geographical line, for example a transport line - such as a road, in particular a dirt road or a railway line - or a boundary line.

Roads are typical examples of land-based transport lines that can be found in practically every country in the world. Specifically, the majority of roads around the world are unpaved. While official figures are only available for a few countries, overall estimates go up to 80%. Although this percentage is negatively correlated with a given country's level of development, even in some of the wealthiest and most developed countries, a significant number of roads are unpaved.

The current condition of such unpaved roads is highly dependent on the current season and weather, vegetation and frequency of use. Snow, ice, flooding, landslides and the constant impact of (heavy) vehicles can make these roads impassable for months. Even though there is much less traffic on these roads, the risk of accidents is much higher than in urban areas. At the same time, however, these transport lines are essential since 50% of the world's population lives along such roads. Their quality is a performance indicator for economic development and a prerequisite for tourism and accessibility to infrastructure for remote communities, e.g. to hospitals, schools and markets.

While unpaved roads are most prone to the above issues, other transportation lines, especially paved roads, can also be affected by problems such as asphalt aging, potholes, geological phenomena, and overgrown vegetation, which require difficult-to-predict maintenance work and cause costly delays or damage and even cost lives, for example when poorly maintained bridges collapse. At least some of these issues may also be relevant to other transportation lines, e.g. railway lines or water transportation lines along rivers, pipelines or power lines.

Referring back to roads as non-limiting examples of geographic lines, assessing road surface quality and the space surrounding it in rural regions has several additional levels of difficulty compared to metropolitan areas.

First of all, it should be noted that these roads are generally used less frequently than roads and paths in cities or major thoroughfares. In addition, monitoring systems based on on-site quality assessment with regular updates are usually not feasible for technical or economic reasons. The technical equipment, staff and funding limit road maintenance actions to actions taken after specific disasters or accidents or involve long-term infrastructure planning. Therefore, for most rural transport lines, especially unpaved roads, up-to-date information collected “on the ground” is unlikely to be available in the foreseeable future due to the nature of the problem presented.

Secondly, the quality of the road surface assessment strongly depends on the analysis of the historical data, i.e. for the assessment of the current condition it is important to know how much precipitation has fallen on a region of interest in the last hours, days or weeks. In addition to the amount of precipitation (rain, snow, any combination thereof), other weather parameters such as temperature and wind can also cause road quality degradation or long-term erosion problems.

Third, the quality and condition of a road is affected not only by what is directly on the trafficable road surface, but also by the shape, condition, and events in the vicinity of the road, such as overgrown vegetation, landslides, avalanches, rockfalls, or flash floods . Some of these problems have a very local reach, in the order of meters - such as the shape of the roadside or the amount of vegetation next to the road - while others can have a reach of several kilometers, such as catastrophic landslides or flooding. Therefore, systems for deriving the current status and situation on certain road sections will usually contain data from a wider spatial area in order to be able to provide reliable information.

The use of multispectral satellite imagery ranges from countless civil applications such as geo graphical mapping, urban planning, precision farming, traffic management to pollution monitoring and climate change analysis. Currently, i.e. in 2020, medium-resolution satellite systems typically provide imagery with a resolution of up to 10 mx 10 m per pixel and a repetition time between 3 and 6 days (e.g. Copernicus Sentinel 2 operated by the European Space Agency), while systems with very high resolution typically provide a resolution of up to 30cm x 30cm per pixel. Even higher resolutions can be achieved with aerial photos from drones or airplanes. Medium-resolution images allow roads and vegetation to be captured and classified, while high-resolution systems can even resolve lane markings and individual trees.

In general, satellite operators provide the data as orthorectified images. Therefore, they transform the raw image data to account for topographical variations of the Earth's surface and the tilt of the satellite or aerial imager used to capture the image. Satellite imagery typically provides intensity values for the visual spectrum (red, green, and blue), but other spectral components that are not visible to humans but still provide information about the observed region are often included (e.g., infrared and radar). Land cover classes such as urban, vegetation or water each have a characteristic spectral response and accordingly indices such as the Normalized Difference Vegetation Index (NDVI) have been developed to standardize corresponding analyses. Such indices combine spectral information from different bands to simplify the assessment of features on the ground.

Satellite imagery has been extensively used to classify land cover and vegetation for precision agriculture or to analyze climate change. Known methods for analyzing such satellite images have in common that they are based on classic images with rectangular grids that observe the earth from above as a continuous region. Preprocessing techniques can be applied to account for satellite inclination, atmospheric distortions or altitude with the aim of reducing image distortions and other image artifacts as much as possible. Thus, each pixel in the image represents features as if seen by a camera directly overhead (zenith).

Typically, these images are georeferenced, i.e. their pixels are aligned to and expressed in relation to a geographic coordinate system, e.g. in coordinates related to longitude and latitude, or in Universal Transverse Mercator (UTM) coordinates. In other scenarios, the satellite's native orientation can be used to avoid expensive pre-processing steps. However, lines of transportation, whether artificial, like roads or railroads or canals, or natural, like rivers, are not generally aligned to common geographic coordinate systems.

In addition, a number of road surface classification methods based on vehicle-mounted sensors have been proposed to assess traction or to optimize suspension. Such systems have been proposed to reduce aquaplaning, determine a road condition, classify winter road surface conditions, or classify road surface roughness using tire sensors.

The subject of the present invention is an improved approach for determining a nature of a geographic line. In particular, it is desirable to improve the efficiency, effectiveness, accuracy, and error rate of monitoring one or more features of a geographic line.

A solution to this problem is provided by the teaching of the independent claims. Various preferred embodiments of the present invention are provided by the teaching of the dependent claims.

A first aspect of the invention relates to a method, in particular a computer-implemented method, for determining one or more properties of a geographic line, GL, or its surroundings. The method comprises: (i) acquiring georeferenced earth observation data representing one or more spatially overlapping image planes covering a particular region of interest, ROI, of the earth's surface from an air or space perspective, the ROI containing the GL; (ii) geometrically transforming the georeferenced observation data, at least in part, into a local internal frame of reference of the GL within the ROI to obtain a mapping of the georeferenced observation data to corresponding coordinates within the local internal frame of reference of the GL; and (iii) evaluating the imaged earth observation data, as represented in the local internal reference system of the GL, according to a classification scheme in order to obtain evaluation data therefrom which classify one or more properties of the GL or its surroundings according to one or more Properties - such as physical, chemical or biological properties - represent GL.

Accordingly, the method can be viewed as a sophisticated measurement method for measuring one or more, in particular physical (e.g. meteorological, geospatial, geomorphological, geological, electromagnetic), chemical or biological properties of the GL, with one or more measurement results being represented by the evaluation data , based on a set of measured georeferenced earth observation data, eg data generated by one or more image sensors (eg satellite or airborne), and optionally further sensor data. In addition to the current measured values for these properties, their development over time and the associated dimensions, e.g. annual averages (e.g. for the appropriate mapping of permafrost soils), can also be determined.

The term "geographical line" as used herein specifically refers to a line, i.e., an elongate straight or curved continuous, predominantly one-dimensional, geometric structure on the surface of the earth. A geographic line can be man-made (e.g., a road, railroad track, power line, dam, bridge, canal, pipeline, or boundary line) or natural, e.g., a river. Concretely, a geographic line can be a transport line, i.e. a line that forms a transport infrastructure or is part of a transport infrastructure, e.g. for the transport of goods, people or energy. A geographic line may correspond to a portion of a more complex network that includes multiple geographic lines, such as a road network with multiple interconnected roads, a railroad system with multiple interconnected railroad lines, or a pipeline system with multiple interconnected pipeline branches.

The term "vicinity" of a geographic line, as used herein, refers specifically to the immediate vicinity of the geographic line, typically ranging from less than a meter to a few hundred meters, measured in a direction perpendicular to the direction of the line any point on this line.

The term "condition of a geographic line" and similar expressions, as used herein, refers specifically to a detected or observable condition, particularly a current condition of the geographic line at one or more locations on the line. Specifically, each condition or condition can be a physical, chemical or biological condition of the geographic line, or a combination of two or more thereof, which condition or combination of conditions can be detected by image analysis or by one or more suitable sensors. Without limitation, the physical or chemical or biological properties of a road surface, e.g. asphalt, the location and extent of potholes in a road, geological phenomena and overgrown vegetation, the occurrence of landslides, avalanches, rockfalls or flash floods, if and to the extent that they are geographical Affect Line Examples of a feature or combination of features of a geographic line.

The term "collecting" of data, e.g Data from an external data source.

The term "georeferenced reference geometry" as used herein refers to a geometry, e.g. as represented by corresponding data, defined or expressed in terms of a geographic coordinate system, such as longitude and latitude or UTM coordinates. In particular, and without limitation, a geometry of a particular road as represented by a reference electronic map, such as an electronic map based on map data for automotive navigation systems or mapping or navigation applications on a computer or handheld device, such as a smartphone or tablet -Computer based, a georeferenced reference geometry. The same naturally also applies to corresponding reference geometries of other geographical lines, e.g. for railway lines, border lines, rivers, etc., which in particular can also be represented by corresponding map data or other geo-referenced geometric data.

The term "Local Internal Reference System of Geographic Line (GL)" as used herein refers specifically to a reference system, such as a coordinate system, which is typically distinct from a georeferenced reference system. The geographic line is in the local internal reference system and is represented therein as a line of a predetermined selected geometry, which can be a straight line in particular. When the geographic line is a street, such a reference frame is often referred to as "Str ßen space”. Correspondingly, a GL which is a boundary line in a geographical reference system (geographical coordinate system) using eg longitude and latitude as coordinates or UTM coordinates can be represented as a straight line in its local internal reference system. For example, in the local internal reference system, the coordinates of a given location on the GL may include: (i) a first coordinate representing a distance of that point from a predefined reference point on the line, the distance being measured along the path of the line ( eg road kilometers), and (ii) a second coordinate expressing a distance of the given point from a (physical or virtual) center line of the GL, measured along a straight line passing through the given point in a direction perpendicular to the GL.

The term "classification scheme" as used herein specifically refers to a scheme for classifying one or more features of a geographic line into a set of classes. By way of example and without limitation, such classes may relate to different physical conditions, e.g. presence or absence of road surface, potholes, ice or snow, rocks or other elements of a road surface, different degrees of vegetation overgrowth over a given GL, current winds - or precipitation conditions, etc.

The terms "first," "second," "third," and the like are used in the specification and claims to distinguish between like elements and not necessarily to describe a sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances, and that the embodiments of the invention described herein may operate in different orders than those described or illustrated herein.

When used in the present specification and claims, the term "comprising" or "including" does not exclude other elements or steps. If an indefinite or definite article is used when referring to a single noun, eg "a", "an", "an" or ""the", "the", "the", this concludes the plural of that noun unless expressly stated otherwise.

Further, unless expressly stated otherwise, "or" refers to an inclusive or and not to an exclusive or. For example, a condition A or B is satisfied by one of the following conditions: A is true (or present) and B is false (or absent), A is false (or absent) and B is true (or present), and both A and B are true (or present).

Accordingly, the method according to the second aspect comprises an effective, normally non-linear, geometric transformation of georeferenced earth observation data from a global space, the observation data relating to a region of interest covering a GL of interest, into a local internal frame of reference of that GL , to get a corresponding view (e.g. image or combined stack of different images) of the GL in this local frame of reference. In particular, this local frame of reference and the way the GL is represented in it can form a very suitable and easy-to-use basis for a normalized representation (e.g. a normalized street space) of GLs, which is particularly advantageous for subsequent classification steps or comparisons with other GLs -related data can be, e.g. B. Comparisons between the respective qualities of two or more different GLs or between the respective qualities of the same GL but at different times. In addition, due to the transformation, the method allows the use of various different earth observation data (in particular image data) as input, in particular independently of the respective orientation or resolution with which this data was recorded. The evaluation data resulting from the classification can be used in many different applications, as will be explained in more detail below.

In particular, the use of earth observation data, such as satellite imagery, can compensate for the lack of data on road surface quality in rural areas where sufficient ground-based data is not available. The method according to the second aspect thus offers a canonical possibility of using satellite images and optionally also supplementary data in order to obtain a normalized view of road sections and other geographic lines.

An additional advantage of the method can be achieved in that otherwise unconnected image regions can be transformed into connected regions based on the geometry of the transport line. This can reveal otherwise unobvious spatial and temporal patterns. In addition, some techniques, such as machine learning, require input data with a consistent structure, which is also made up of results from this method, whereby data with different dimensionality can be projected into the straightened road space.

Preferred embodiments of the method are described below, which can be combined with one another or with other aspects of the present invention as desired, unless such a combination is expressly excluded or is technically impossible.

In some embodiments, geometrically transforming the georeferenced observation data, at least in part, into the GL's local internal frame of reference comprises: (i) defining a plurality of control points within the ROI at different positions along the GL; (ii) determining for each control point a respective local cell grid covering a limited local spatial neighborhood of that control point, the resulting set of local cell grids defining the local internal frame of reference of the GL; and (iii) geometrically transforming the observation data, at least in part, into the local internal frame of reference of GL to obtain a mapping of the observation data onto the set of local cell grids of the control points. This approach is particularly flexible in that the local internal frame of reference is defined by a combination of the local cell grids, each of which can be independently optimized, particularly in terms of its orientation relative to its corresponding control point. In addition, the approach is also very efficient since the set of local cell grids in total need not cover the whole GL, but can only cover a part of it, such as a set of selected segments of the GL within the ROI and their respective neighborhoods, so that the total dataset of the transformed observation data can be effectively limited and in particular can be less than that of the original georeferenced observation data covering the ROI. While one or more of the local cell lattices may have a rectangular pattern, other shapes are also possible, e.g., hexagonal cells in a corresponding hexagonal lattice. In fact, the term "local cell lattice" as used herein even includes the possibility that a local cell lattice has or consists of one or more cells separated by spaces, e.g. several unconnected circular ones Cells arranged in a specific pattern.

In some related embodiments, the method further comprises concatenating the individual control point cell grids to form a data representation of a straightened, particularly regular, overall cell grid that defines the local internal frame of reference of the GL, in which the individual control point cell grids are arranged in a straightened, rectified way are arranged in an order that corresponds to the order of the control points along the GL. This has the advantage of creating a particularly simple, compact and easy-to-handle reference system. In particular, it enables a particularly simple normalized representation of the transformed observation data or the resulting evaluation data, in particular for the above-mentioned purposes of comparing such data with other data represented in each case.

In some of these embodiments, the evaluation of the respective earth observation data takes place on the basis of the representation of the earth observation data as they are displayed on the straightened total cell grid. This means that not only is the end result represented on the basis of the straightened overall cell grid, but that the actual previous evaluation of the transformed observation data (and optionally additional sensor data) is already carried out on the basis of the straightened overall cell grid. This further improves the simplicity and efficiency of the evaluation process, since typically straightened datums can be handled with less effort than most other types of datums, e.g., bound datums. In particular, straightened reference systems can be represented very easily by simple and memory-efficient data structures such as tables or matrices.

In some embodiments, at least one of the local cell lattices is defined as having one or more of the following properties: (i) anisotropic arrangement of at least a subset of the cells within the local cell lattice; (ii) the widths or heights or spacings of the cells vary between at least a subset of the cells within the local cell lattice; (iii) an orientation of at least a subset of the cells within the local cell grid is aligned with the local orientation of the GL according to its georeferenced reference geometry at the control point corresponding to the local cell grid. What all of these specific variants (i) to (iii) have in common is that they enable an optimal adaptation of the associated local one or more cell grids to the specific individual geometry of the GL and the desired type of evaluation to be reflected in the evaluation data. For example, if the GL is strongly curved near a certain first control point, the cell lattice for that first control point could be defined to have an entirely different orientation, variation in cell dimensions and anisotropic arrangement of the cells within this local cell grid than the cell grid for a second control point located at a different location on the GL where the GL is more straight.

In some embodiments, the georeferenced earth observation data represents one or more pixel values per pixel of an original pixel grid covering the at least one image plane. In addition, the geometric transformation of the georeferenced observation data comprises the determination of at least one cell value for each cell of the local cell grid based on an aggregation of a subset of the pixel values corresponding to an area of the ROI covered by the respective cell of the local cell grid. on. This is a particularly effective tool that offers a high level of flexibility and the option of optimally adapting the transformation to the desired type of evaluation. In addition, using a plurality of pixel values per pixel allows for even more accurate or complex evaluations, including evaluations that rely on the combination of more than two pixel values per pixel to determine associated evaluation data for that pixel or a surrounding region around it.

In particular, in some of these embodiments, aggregating the subset of pixel values comprises one or a combination of at least two of the following operations: (i) averaging the pixel values of all pixels that are entirely within the corresponding cell of the local cell grid; (ii) averaging the pixel values of all pixels overlapping the region of the ROI covered by the corresponding cell of the local cell grid, each pixel value being weighted according to the extent of its overlap with that cell of the local cell grid; (iii) weighted averaging of the pixel values, where pixels closer to the cell center of the local cell grid are given higher weights than pixels further from the cell center; (iv) determining the median, minimum and/or maximum of the pixel values among the pixel values covered by the corresponding cell of the local cell grid; (v) determining pixel values representing a specific percentile of the distribution of pixel values within the corresponding cell of the local cell grid; (vi) determining one or more of a range, standard deviation, variance, or any other higher-order function that captures features of the distribution of pixel values within the cell of the local cell grid; (vii) determining a histogram representing the distribution of pixel values within the cell of the local cell grid. The result of the aggregation can be a single value or a data structure, depending on the type of operation being applied. For example, the histogram according to variant (vii) is typically represented by a data structure having multiple values, while an average of point values according to any of variants (i) to (iii) is typically represented by a single value.

In some embodiments, at least one of the local cell lattices is defined by a set of lattice parameters, and the method further comprises determining at least one of the lattice parameters based on a process that uses machine learning in conjunction with cross-validation of intermediate results obtained during the machine learning process includes evaluation data with reference parameter data representing known correct classifications of the relevant one or more properties of the GL or its environment according to the one or more constitutions. In particular, these embodiments enable an automatic definition, even a dynamic adaptation or variation of the local cell lattice with the aid of the machine learning process. The data of the reference parameters serve as training data in the sense of a supervised learning for the machine learning process.

In some embodiments, capturing the georeferenced earth observation data comprises one or more of the following steps: (i) capturing georeferenced earth observation data representing one or more of the image planes, each of these image planes representing images representing the ROI in a different spectral range of the electromagnetic cover spectrum; (ii) acquisition of georeferenced earth observation data representing one or more data planes as further image planes, each of these data planes representing respective sensory measurements or predictions, e.g. simulation results of one or more local conditions of the GL or its surroundings within the ROI; (iii) acquisition of georeferenced earth observation data representing one or more pre-processed image planes resulting from a combination of two or more of said variant (i) image planes or variant (ii) data planes. Both the use of two or more of the image planes per variant (i) or the supplement of this image plane (s) with one or more of the data planes according to variants (ii) or (iii) allows in particular to expand the range of possible types of evaluations or their achievable quality and accuracy.

Specifically, in some of these embodiments, the data planes representing the respective sensory measurements or predictions of one or more local features of the GL or its environment within the ROI represent one or more of the following: (a) precipitation, (b) intensity or wavelength of the local radiation, (c) temperature, (d) barometric pressure, (e) wind strength or direction, (f) if at least in a portion of the ROI the GL is a water-based geographic line: speed and direction of water flow; (g) if in at least one portion of the ROI the GL is a geographic line on land accessible to a land vehicle: vibration, orientation, speed, acceleration or any other vehicle parameter measured by one or more vehicle-based sensors and insight in certain characteristics of the transport line; (h) if the GL is a haulage line: the frequency of use of the haulage line, the maintenance condition of the haulage line or the time since the last maintenance interval, an age or a condition of the surface of the haulage line; (i) climate zone, (j) climate profiles, (k) expected temperatures or precipitation, (i) vulnerability to unexpected events such as accidents, natural disasters, floods, landslides, avalanches, storm damage, tornadoes, hurricanes, typhoons, (m) electromagnetic fields, e.g. signal strength of mobile phone networks covering the GL. This enables a very wide range of different evaluations, depending on which types of measurements or predictions are selected from those mentioned above, and correspondingly positive effects on the achievable quality and accuracy of the resulting evaluation data.

In some embodiments, the method comprises aligning the respective geometry of the GL in the one or more image planes represented by the earth observation data with a georeferenced reference geometry of the GL to form a combined image stack, with at least two of the image planes integrated into the stack, where each of said image planes representing a respective digital image in a given respective spatial resolution, wherein the spatial resolutions of at least two of said image planes differ from each other, at least in an overlapping region, resulting in a combined multi-resolution image stack. Using such an orientation of the respective geometries of the GL enables a particularly simple, efficient and reliable way of performing the geometric transformation, since the respective image data of the planes of the stack can be transformed directly into the local internal reference system of the GL by projection. In addition, the several different resolutions enable the method to be highly efficient, in particular with regard to limiting the amount of data to be processed and thus increasing the performance of the method while at the same time maintaining the desired accuracy. In particular, this can be achieved by selectively using either a higher or a lower of the available resolutions for different planes, depending in particular on the impact of a particular choice on the ability to achieve a required accuracy of a desired type of analysis.

In some embodiments, the method further comprises applying a data compression process to reduce the spatial resolution of a given image plane, the data compression process including creating two or more sub-planes of the image plane by resampling at least a portion of the image plane with a spatial resolution lower than theirs original spatial resolution, while maintaining or resampling the spatial resolution of at least one other region of the image plane such that when the sub-planes are overlapped to collectively define a combined multi-resolution representation of the image plane, the average resolution of the combined multi-resolution representation is lower than the original resolution the image plane. Instead of or in addition to a different resolution across different image planes, data compression is therefore applied within the same image plane according to these embodiments. This can be used to advantage to further increase the efficiency of the method, e.g. with regard to the speed of data processing and the memory efficiency of the method.

In particular, in some of these embodiments, the lower spatial resolution at which at least a region of the image plane is resampled is a function of the region's distance from the GL. In particular, the spatial resolution can be lower for larger distances and higher for smaller distances. In this way, the achievable accuracy for evaluations that relate to the GL itself or its immediate vicinity can be chosen to be higher than for more distant areas in the vicinity of the GL. For example, it might be important to capture vegetation overgrowing the GL with a high level of accuracy, while capturing detail of vegetation further from the GL might be less relevant, so the benefits of compression over necessity high resolution might dominate in these remote locations.

In some of the embodiments in which sub-levels are used, the overlapping of the sub-levels comprises aggregating the values of spatially overlapping pixels of different sub-levels according to a predetermined aggregation method to obtain a pixel value of a pixel that represents a spatial region in the resulting combined multi-resolution representation , which corresponds to the overlapping pixels of the different sublayers. In particular, one or more of the various aggregation methods mentioned above in connection with embodiments comprising aggregating subsets of pixel values may also be applied here.

Furthermore, in some of the embodiments in which sub-planes are used, the pixels of a first of the sub-planes are locally aligned to the respective local orientation of the GL at or near the respective pixel, while the pixels of a second of the sub-planes are aligned prior to compression into one correspond to the regular orientation of the pixels in the given image plane. This has the advantage that, despite the loss of image information caused by the compression, the most relevant image information, e.g. image information relating to the GL or its immediate surroundings, is available with a sufficiently high resolution and in an optimized orientation that corresponds to the course of the GL. remain available. The resulting processed image plane can either be used itself as the desired compressed version of the original image plane, or instead used only as an intermediate image, which is then further processed, specifically straightened, as discussed below, to yield a final compressed version of the original image plane.

Accordingly, in some embodiments, overlapping and aggregating the values of the spatially overlapping pixels of different sub-planes comprises generating the combined multi-resolution representation of the image plane such that the orientation of the resulting pixels of that representation is the same for all such pixels (i.e., this operation results in a straightened , compressed multi-resolution image plane).

In some embodiments, the method further comprises performing one or more of the following operations based on the evaluation data: (i) determining a navigation or informational output of a vehicle navigation system; (ii) controlling a dynamic traffic sign; (iii) setting one or more configurable technical parameters of a vehicle or other transportation system; (iv) initiating a maintenance or inspection process in relation to GL; (v) predicting a future state of the geographic line or its surroundings.

Particularly in the case of variant (i), the output of the vehicle navigation system can relate to the fact that a specific route is selected as the preferred route or as the selected route for navigation purposes from a plurality of possible routes leading to a specific destination. In the case of variant (ii), the dynamic traffic sign could specifically be a sign indicating whether or not a certain section of road is allowed to be driven on, or a warning sign which, depending on its current configuration, can signal two or more different warning levels or categories, which can be controlled, ie selected, on the basis of the evaluation data. In the case of variant (iii), the one or more configurable technical parameters may be specific to a particular suspension configuration of the vehicle or transportation system. In the case of variant (iv), the maintenance or inspection process could be specifically triggered by a corresponding signaling, e.g. to inform the maintenance staff accordingly, or these processes could even be fully or partially automated processes that are automatically triggered by a corresponding trigger signal.

A second aspect of the present invention relates to a data processing system configured to carry out the method of the first aspect. The data processing system can be specifically configured using one or more computer programs to carry out the method according to the first aspect. Additionally or alternatively, the configuration can be implemented entirely or partially by appropriate hardware.

A third aspect of the present invention relates to a computer program or a computer program product containing instructions which, when executed on a data processing system according to the second aspect of the invention, cause the system to carry out the method according to the first aspect of the invention.

The computer program (product) can be executed in particular in the form of a data carrier on which one or more programs for carrying out the method are stored. This is preferably a data carrier device, such as a CD, DVD, or other optical media or flash memory module. This can be advantageous if the computer program product is to be traded as a single product independent of the processor platform on which the one or more programs are to be executed. In another implementation, the computer program product is provided as a file on a data processing unit, in particular on a server, and can be downloaded via a data connection, eg the Internet or a dedicated data connection, such as a proprietary or local network.

Accordingly, the system of the second aspect can have a program memory in which the computer program is stored. Alternatively, the system can also be set up in such a way that it accesses a computer program that is available externally, e.g represent outputs of the computer program.

The explanations, embodiments and advantages described above in connection with the method of the first aspect also apply correspondingly to the other aspects of the invention.

character list

• 1 schematisch eine beispielhafte Ausführungsform eines Verfahrens nach der vorliegenden Erfindung veranschaulicht;
• 2 einen beispielhaften kombinierten Bildstapel mit begradigten Ebenen veranschaulicht;
• 3 einen Satz einer beispielhaften Ausgabe möglicher Auswertungsergebnisse des Verfahrens aus 1 oder 5 veranschaulicht;
• 4 schematisch ein Datenkompressionsverfahren für eine Bildebene, das bestimmten Ausführungsformen des Verfahrens der vorliegenden Erfindung entspricht, veranschaulicht;
• 5 ein detailliertes Flussdiagramm zeigt, das eine Ausführungsform eines Verfahrens gemäß der vorliegenden Erfindung veranschaulicht, wobei das Verfahren insbesondere (i) eine Modellerstellungsphase zur Optimierung eines parametrisierten Modells des Verfahrens und (ii) eine Modellanwendungsphase zur Anwendung des optimierten Modells als Instanziierung des Verfahrens aufweist, um tatsächlich eine Bestimmung von Auswertungsdaten durchzuführen, die eine Klassifizierung einer oder mehrerer Eigenschaften der GL oder ihrer Umgebung gemäß einer oder mehrerer Beschaffenheiten der GL repräsentieren; und
• 6 verschiedene beispielhafte Ausführungsformen der Untergitter 111 veranschaulicht.

Further advantages, features and applications of the present invention are described in the following detailed description and the attached figures, wherein

• 1 schematically illustrates an exemplary embodiment of a method according to the present invention;
• 2 Figure 12 illustrates an example combined image stack with planes straightened;
• 3 a set of an exemplary output of possible evaluation results of the method 1 or 5 illustrated;
• 4 schematically illustrates a data compression method for an image plane, corresponding to certain embodiments of the method of the present invention;
• 5 shows a detailed flow chart illustrating an embodiment of a method according to the present invention, the method in particular having (i) a model creation phase for optimizing a parameterized model of the method and (ii) a model application phase for applying the optimized model as an instantiation of the method to actually perform a determination of evaluation data representing a classification of one or more properties of the GL or its environment according to one or more characteristics of the GL; and
• 6 various exemplary embodiments of sublattices 111 are illustrated.

In relation to 1 an exemplary embodiment 100 of a method of the first aspect of the present invention comprises a step S1 of obtaining georeferenced earth observation data comprising aerial image data collected by one or more drones 101, airplanes 102 or satellites 103 or other devices acquiring multispectral observation data of the earth 104, in particular their surface, have been obtained.

In a further step S2, a specific region of interest (ROI) 105 is selected, eg a specific location, a rectangular region, a specific country, a state, a city or an arbitrarily shaped region or any combination/association of such regions that cannot be contiguous either. Further processing of the obtained observation data will focus on this ROI. In this way, a stack of 106 multiple image planes (each image plane forms an image) is obtained, where in particular each image plane can correspond to a respectively associated spectral band, so that the stack has a total of several spectral bands in different image planes. For example, a typical RGB picture plane would contain three channels. Satellite imagery may contain additional infrared bands, e.g. the Sentinel-2 satellite imagery offers 12 bands. Active sensors - radar or lidar - with an active signal emission component can provide one or more additional image planes independent of the natural light sources. In addition, these image planes may contain pre-processed images generated by combining image data from different channels or by pre-processing methods to obtain digital elevation models. The image stack may contain images from the same source, e.g. from a specific satellite system, but acquired at different points in time (time series), or even from different source systems, e.g. from different satellite systems or airborne or drone-based earth observation systems. Other image planes of the stack 106 may be data planes representing sensor data collected from sensors other than image sensors, e.g., land-based sensors or even vehicle-based sensors, such as vibration sensors.

In a further step S3, one or more georeferenced geographical lines GL or networks 107 of GLs, such as road networks (or other transport line networks or geographical boundaries), are aligned with the georeferenced image stack 106 to obtain a combined image stack 108. The transport line network can be obtained (a) manually, e.g. by an operator that defines the geo-coordinates of the transport line to be evaluated, (b) from existing databases such as "OpenStreetMaps", (c) via any image processing method that derives vector data from images, or ( d) via automatically recorded vehicle GPS tracks (or Galileo or Glonass etc.). The transport line network can be defined by a graph consisting of a series of nodes (defined by their latitude and longitude coordinates on the globe) and their connections. Although this is the most common case, it does not exclude other types of line definition such as Bezier curves, splines, or other parametric curves.

2 12 shows an example combined image stack 200 (ie, an example instantiation of the combined image stack 108 discussed above) composed of six different image planes 210-260. The combined image stack 200 has already been aligned in step S3 with respect to a GL for which an evaluation according to the method 100 is to be carried out. As an example, plane 210 may represent an image in the visual portion of the electromagnetic spectrum, eg, expressed in RGB color space or some other color space suitable for that spectral range. The level 210 imagery may have been provided by a satellite system such as Landsat or Sentinel. Level 220 can represent an elevation/topology image that was supplied by a digital elevation model and was generated by combining image data from different channels, eg also from a radar channel. The further plane 230 can represent an image showing a spatial distribution of vegetation and optionally the type of vegetation for the region of coverage (ROI). Another layer 240 can represent an image that indicates the type of surface in the ROI at the different locations of the ROI. In particular, this plane may be derived from image analysis of plane 210 or by combining data from multiple image planes and/or other data sources such as ground sensors, e.g. to determine the nature of the surface of the GL itself, particularly when the GL is a road . Level 250 may represent evaporation data over the entire ROI, which may be particularly relevant for detecting features of a GL's environment, eg, areas covered with forest, meadows, rivers, lakes, snow, and so on. Level 260 may represent weather data for the ROI, eg, precipitation forecast data. Other levels can represent historical weather data for the ROI, for example. Accordingly, it is noted that an image plane (“plane” for short) is not limited to representing an optical image, but can also represent other data that can be used to characterize a state of the ROI. This data may be data provided directly by one or more image sensors or other sensors, or it may be derived solely from such sensor data, eg by computation and/or by combining two or more different types of input data , eg two or more other image planes, to derive the data for the image plane in question.

Referring again to 1 the method 100 also has a step S4, in which for each connected line segment 110 of the network 107, ie for at least one GL in the network 107, a sample of control points 109 is defined, for example by selecting evenly distributed points on the line segment 110. Depending on In this case, other sampling plans can be advantageous. For example, if the GL to be monitored contains intrinsic control points, one can use these existing control points instead of a mathematical sampling plan. A typical scenario would be overhead power line monitoring where control points 109 could be sampled at transmission tower/electricity pole locations.

In a step S5, a respective sub-grid 111 is placed at each control point 109, the orientation of the respective sub-grid being determined by the local orientation of the GL at this control point 109. In particular, the subgrids 111 can be defined as regular linear grids arranged in a direction perpendicular to the trajectory of the GL according to the georeferenced frame of reference of the imagery. The detected spectral variability and visual complexity of the environment of the GL to be monitored is controlled by the grid parameters (number of cells, width and height of each cell) of the subgrids 111 and the number of control points 109.

In an exemplary embodiment, these parameters are received via a user interface. They can therefore be selected manually based on expert knowledge in a specific area. In another example implementation, these parameters can be optimized through machine learning and cross-validation methods, i.e. a method is trained with different sets of parameters and the results are compared with known reference data (supervised Learn). The parameter set that yields the best accuracy can then be chosen as the final model. In a further exemplary embodiment, multiple sets of parameters are combined and used simultaneously for a specific prediction method. In this way, at each control point 109, an image stack with multiple resolutions is obtained. A typical example might be roadside surveillance, where a high-resolution grid is able to detect specific road marking patterns, while lower-resolution grids capture vegetation features alongside the road.

In a process that has the further steps S6 and S7, the image stack is now transformed geometrically into a local internal reference system of the GL. For this purpose, in step S6, in each sub-grid 111, image features from the source images 112 are aggregated using a suitable aggregation function 114 over the image area in which the source images 112 overlap with the respective sub-grid 111, to either a single value or a data structure for each grid cell 113 of the respective sublattice 111 to obtain.

Typical suitable aggregation functions 114 are (a) averaging the pixel intensities of all pixels of the respective source image 112 that are located entirely within a cell 113 of the sublattice 111, (b) averaging the pixel intensities of all pixels of the respective source image 112 that are located with the cell 113 overlap modified by the percentage of space they contribute to the cell, (c) weighted averaging of pixel intensities, with pixels in the center of the cell 113 receiving higher weights than pixels located at the edge, (d) average, minimum and maximum intensities, (e) intensities representing a certain percentile of the intensity distribution within the cell 113, e.g. to account for outliers, (f) range, standard deviation, variance or any other higher order function representing the characteristics of the intensity distribution within the cell 113 detected, (g) histograms of intensities, (h) histograms of orientations or others image features. The latter two are special cases since they increase the dimensionality of the output space (e.g. street space) but allow for more detailed characterization. While this is a non-exhaustive list of possible feature extraction methods, others might be advantageous for specific use cases.

The sublattices 111 may be anisotropic, i.e., the width and height of the grid cells may not be equal, to reflect that certain features of transport lines vary greatly with lateral distance from the center of the transport line, but remain largely the same along the transport line. In addition, the sub-grids 111 may not be equidistant, i.e. the width and height of the grid cells may vary, for example to map more smoothing when further away from a transport line. Other variations between the cells are also possible.

6 11 specifically illustrates a set 600 of different exemplary embodiments of sublattices 111. Specifically, FIG 6(a) and (b) examples of elements of a first category (“regular”) of the sublattices 610 or 620, the individual cells 615, 625 being arranged in a regular order, such as a one-dimensional matrix arrangement 610 or a two-dimensional matrix arrangement 620, in each case all Cells 615, 625 have the same shape, size and orientation and the same distance (centre-centre) to their respective neighboring cells. 6c FIG. 12 illustrates, by way of example, a second category (“anisotropic”) sublattice 630 characterized in that the shape and size of the various cells 635 can vary. In the case of the rectangular cells 635 in particular, their width and height can vary over the entirety of the cells in the sublattice. the 6d and 6e 12 show, by way of example, the sublattices 640 and 650 of a third category (“non-gapless”), which is characterized in that at least some of the cells of the sublattice are spaced apart. In 6d the cells are arranged equidistantly, while in 6e the distances between adjacent cells can vary (“non-equidistant”). Other embodiments of sublattices 111 can be achieved by implementing two or more of the above examples simultaneously (unless they contradict each other). For example, the two-dimensional aspect of array 620 can be mixed with any of embodiments 630, 640, and 650 to define corresponding two-dimensional versions of the cell arrays according to respective embodiments 630, 640, and 650, respectively.

up again 1 Referring to, in subsequent step S7, the obtained values for each cell 113 in each sub-grid 111 are placed in a regular grid 115 (eg straightened street space if the GL corresponds to a street) so that a geometrically transformed one for each plane straightened image of GL and its surroundings, which can be used for further processing and visualization purposes. Specifically, the regular grid 115 defines the local internal reference system of GL, and the positions of the individual cells of the grid 115 can be viewed as coordinates of this local internal reference system.

Method 100 also has an evaluation step S8, in which the stack of transformed, straightened images of the GL and its surroundings is evaluated using a classification scheme in order to obtain evaluation data that classify one or more properties of the GL or its surroundings according to one or more Properties - eg physical, chemical or biological properties - represent GL.

3 illustrates exemplary evaluation data resulting from performance of the method of the present invention, eg, the method of FIG 1 or 5 , and refer to several different classifications in relation to a GL in the form of a road. In the example, the classification relates, among other things, to a type of road surface, a slope of the road, a risk of flooding, an avalanche risk, an overgrowth index (which indicates a level of vegetation that is overgrowing the road, cf. exemplary evaluation data 116 in 1 ), a road surface flatness, an air temperature, and a road surface temperature along the road. In 3 In the RGB image at the top, the spatial extent of the GL, ie the contiguous road segment of interest, extends into 3 from left to right, and the various rows of data shown below are arranged accordingly.

Hence the procedure of 1 (the same applies to the procedure of 5 , discussed below) configured to provide a normalized image stack that has one or more image planes in relation to the geographic lines (GL) of interest such that these GLs always appear straight and left to right along the horizontal center line of the normalized image. Other subgrid arrangements are also covered, eg arranging the subgrids row by row so that the road becomes a column in the image. Those skilled in the art will consider the state of the art when selecting an appropriate memory layout, CPU architecture and file system, eg storing each image plane in a single file can cause significant overhead for certain file systems. The above method can be implemented in software or with dedicated hardware, eg semiconductor chips.

4 FIG. 4 shows an image compression method 400 that can be used particularly in connection with one of the methods 100. FIG 1 or 500 off 5 may be used to compress image data representing an original image plane 410. In particular, the method 400 can be used for GL-focused compression of satellite images.

In this scenario, aerial or satellite images covering an ROI are processed in an irrelevance-reduction-based approach that focuses on those picture elements (pixels) of the original image plane 410 that represent the ROI that contribute to a GL of interest belongs. In the present example, the GL of interest has been selected as a particular transport line within a transport line network 411, eg road network, represented in image 410. The compression process of 4 however, does not depend on the selection of a specific GL with the transport line network 411, but rather relates to the entire ROI and thus to the entire transport line network 411 and its surroundings.

In the present example, each GL (transport line) from the original image 410 is densely sampled using the transport line straightening method mentioned above at several resolution levels, e.g. the GL itself can be stored at native resolution 430, the immediate surroundings at half the native resolution 440, etc. Thus, each satellite image is represented by a low-resolution base image and a series of higher-resolution GL-related images. Specifically, in this example, the pixels that represent the transport line network 411 itself are mapped using the technique mentioned above (cf. 1 ) is densely sampled including sub-grids, and in addition the spatial resolution of the surrounding pixels is varied depending on the lateral and longitudinal distances of these surrounding pixels with respect to a reference pixel, eg a pixel 431 that is part of a transport line network. For example, the transport line pixels 431 can be stored at the native resolution of the original image 410, the pixels 441 representing the immediate vicinity of the transport line network 411 can be subsampled at a reduced sampling rate, e.g. at half the native resolution, and even more distant pixels 421 can be sampled at an even lower resolution, and so on. Each of the different resolutions may correspond to an associated sub-sampling plane that is sampled from the original image 410 at the respective resolution. Finally, a very low resolution base image 420 is added to the stack of sub-sampling planes.

In this way, several different approaches can be applied to collectively represent a resampled version of the original image 410 . In particular, the orientation of the pixel grid of the base image 420 may correspond to that of the original image 410 (rather than the path of the correspond to GL). The native resolution and subsampling windows of each subsampling level 420, 430, 440 and the total number of subsampling levels 420, 430, 440 represent the parameters that control the degree of irrelevance reduction by this compression method 400.

The resulting data structure effectively compresses the original data 410, which typically contains large amounts of irrelevant information, in terms of the application focus, which focuses only on the areas of the original data 410 that contain GLs, by successively reducing the information density the further the Data points are distant from these GL-centered areas of interest.

In order to reconstruct a single compressed, straightened, multi-resolution image 460 from the various individual planes 420, 430 and 440, the low-resolution base image 420 is overlaid with all GL-focused sub-sampling planes 430 and 440, which have increasing native resolutions, to reconstruct an intermediate image 450, that has high detail only in the areas of the GLs and their immediate surroundings, while only coarse detail is preserved in the areas of low GL density.

Then the desired compressed, straightened, multi-resolution image 460 is reconstructed by combining each of the superimposed 420, 430, 440 planes of the intermediate image 450 to determine a single pixel value for each pixel of the image 460 and map it onto the pixel grid, also resulting in a straightening is reached. In particular, the orientation of the pixel grid of image 460 may be the same as that of base image 420. Some areas of intermediate image 450 may be covered by pixels from different planes 420, 430, or 440. In this case, an application-specific combination function is to be applied, e.g., the median, mean, or weighted average of the contributing pixels of sub-sampling planes 420, 430, and 440 to determine a single pixel value for each pixel of image 460.

This scenario enables very high compression ratios, allowing the storage of initially large amounts of data on memory-constrained systems, enabling the transmission of such data over channels with limited bandwidth such as cellular networks, and reducing the memory footprint of machine learning-oriented algorithms , which makes it possible to use it on memory-constrained systems. Especially in geographical regions, eg rural regions, with a low density of GLs, eg of transport line networks, this enables very high compression rates, so that the storage of such images on a device or their transmission via cellular networks becomes possible without the essential information about the transport line network will be lost.

The methods and processes described above can be used in particular in connection with machine learning or pattern recognition frameworks. An example of this is in 5 1, which illustrates an embodiment of a model-based method 500 according to the present invention, the method comprising in particular: (i) a model building phase for optimizing a parameterized model of the method 100 out 1 and (ii) a model application phase for applying the model as an instantiation of the method 100 to actually perform a determination of evaluation data representing a classification of one or more properties of the GL or its environment according to one or more constitutions of a GL.

Referring now to the modeling phase of the method 500, in a first step 505, images are captured and stored in a database or file system (“image database”). The images can e.g. B. similar to the image stack 200 from 2 be as described above.

In a further step 510, GL geometries are acquired from open or proprietary data sources, eg electronic map data, of an ROI of interest in which the GL is located and stored in a database or comparable system (“GL database”). This step 510 may optionally include augmenting the acquired GL features with other sensory inputs or predictions, eg, simulation results, related to particular GL segments. For example, where the GL is a road, such other sensory inputs or predictions may refer to, but are not limited to, sensor data representing (a) vibration, orientation, speed, acceleration, or any other vehicle parameter used by one or more vehicle-based sensors and can provide insight into certain transport line conditions, (b) measurements from roadside weather sensors for temperature, precipitation and air pressure measurement, (c) frequency of use of the transport line (d) maintenance status of the transport line or time since last Maintenance interval (e) Age and condition of the surface (f) Climate zone, climate profiles, expected temperatures and precipitation (g) vulnerability to unexpected events such as accidents, natural disasters, floods, landslides, avalanches, storm damage, tornadoes, hurricanes, typhoons; h) electromagnetic fields, e.g. cellular network coverage (such data can be generated, e.g. by sensory assessment of the network coverage, e.g. with test devices in the field, or by simulation).

The term "attribute" as used herein refers to a specific measurable condition of a GL or GL segment. A non-limiting set of examples of different characteristics, such as the type of road surface, a slope of the GL (e.g. road or railway line), etc. are given in 3 shown as discussed above.

In a further step 515, the straightening parameters for a subsequent image straightening/normalization step 520 are defined either (i) by receiving corresponding parameter data via an interface, e.g. a user interface, or (ii) by systematic evaluation. In the latter case (ii), a number of parameter sets are used to predict a specific target feature (i.e. a specific type of state of the GL), and based on a comparison with known reference image data, which is independently collected and/or verified correct information for describe this feature, the most suitable set of parameters is finally selected. Such a parameter set can in particular have parameters that relate to the number of pixels, their size, arrangement and orientation. The straightening step 520 creates a set of image patches stored in a file system or database (“patch database”), and may specifically include applying the compression method 400 of FIG. 400 to at least one of the image planes of the images captured in step 505 .

The term "image patch" or "patch" for short as used herein specifically refers to a straightened image of a continuous portion of a related GL. Specifically, the result 115 of step S7 is from 1 an image patch, or it can also be a set of two or more separate image patches, where each image page relates to a particular continuous portion of a considered GL. If, in addition to the imagery, sensor data is also used to predict the target feature, the concept of an "image patch" can be extended to a "smoothed GL datagram" which, in addition to the one or more image patches, also contains one or more layers of sensor data that cover the same GL or GL segment. For simplicity, the following steps are only described in terms of image patches, while the same concepts can be applied to straightened GL datagrams as well.

Depending on the desired application, in a step 525 a target feature to be mapped or to be predicted is selected. In a further step 530, a pattern recognition method is then used to derive a model. It selects a subset of the patches previously stored in the patch database for training and validates the results against a maintained subset. The model parameters describe a function that maps any image patch to the target feature. The model parameters can, for example, form parameters, e.g. After the model parameters have been determined in step 530, the established model represented thereby is ready for use.

Reference is now made to the model application phase of the method 500, where this phase can begin by first defining a geographic region of interest (ROI) in a step 540, acquiring the corresponding images in a further step 545 and in the GL Database to find the data for a specific desired GL or GL segment within the ROI, e.g. a specific transport line segment. All information is thus available to perform the image straightening with the same parameter set (“straightening parameters”) in a further step 555 as in step 520 of the model creation phase. The target feature is then predicted in a further step 560 using the relevant obtained one or more image patches and the stored model parameters. The result in the form of corresponding evaluation data can then be output in a step 565, e.g. for visualization or as input for optional further processing steps, e.g. for deriving control signals for controlling a device based thereon, such as a dynamic traffic sign or a barrier.

• In einer Implementierung wird für jede GL oder jedes GL-Segment über einen bestimmten Zeitraum ein Wetterprofil bestimmt. Sie könnte (a) eine Bildebene einer Live-Wettervorhersage verwenden, um abzubilden, wie viel Regen, Schnee oder Wind in den nächsten Minuten, Stunden oder Tagen erwartet wird, (b) eine Schätzung auf der Grundlage historischer Daten abgeben, wie das Wetter in einer bestimmten Zeit des Jahres sein wird, d.h. sie könnte saisonale Wetterphänomene entlang einer GL, z.B. einer Straße, abbilden. Auf diese Weise würden zur Routenplanung und Navigation straßensegmentspezifische Wetterrisikoinformationen zur Verfügung gestellt werden, um Gefahrensituationen zu vermeiden.

The invention can be used for a number of different applications. For example and without limitation, it can be used in connection with the following applications and related implementations of the invention (cf. 3 ):

• In one implementation, a weather profile is determined for each GL or GL segment over a specified time period. It could be (a) an image layer of a live weather forecast use to show how much rain, snow or wind is expected over the next few minutes, hours or days, (b) give an estimate, based on historical data, of what the weather will be like at a particular time of year, ie it might be map seasonal weather phenomena along a GL, e.g. a road. In this way, road segment-specific weather risk information would be made available for route planning and navigation in order to avoid hazardous situations.

Another implementation would make it possible to assess the maintenance needs for certain transport route sections. It has been shown that the spectral information of road surfaces such as asphalt or concrete varies significantly depending on age and state of degradation, i.e. maintenance measures could be triggered if a certain threshold value is exceeded.

Another implementation would take into account the location and severity of disasters. Very detailed information on the locations of vehicle accidents is available in some countries. This allows an accident risk level to be assigned to each road segment or image patch, and would allow predicting whether other regions are prone to the same risks, e.g. an avenue near the road may represent a higher risk of fatal accidents than treeless roads. The same procedure is applicable to cases of natural disasters. Almost every disaster that claims lives is recorded by the local authorities. Utilizing weather information, elevation data, and disaster locations in the image stack would make it possible to point out other transport lines that might be vulnerable to the same type of disaster. Thus, the present invention could even support the planning of new transportation routes by estimating the risk of disasters along the planned route.

In another implementation, the method could be used to assess the expected range of motion along or across a transport line. Motion would be defined as one or more vectors for each patch and could describe motion information such as wind, traffic, or water flow.

While at least one exemplary embodiment of the present invention has been described above, it should be appreciated that a large number of variations exist thereto.

Furthermore, it is understood that the exemplary embodiments described are merely illustrative of non-limiting examples of how the present invention may be practiced and that the devices and methods described herein are not intended to limit the scope, application, or configuration. Rather, the foregoing description provides those skilled in the art with constructions for implementing at least one exemplary embodiment of the invention, it being understood that various changes in the functionality and arrangement of elements of the exemplary embodiment may be made without departing from the scope of the appended claims and their legal equivalents to deviate from the defined object.

reference list

GL

geographic line, e.g. a transport line, like a road

100

exemplary embodiment of a method for determining one or more characteristics of a geographic line, GL, or its vicinity

101

drone

102

plane

103

satellite

104

Earth, especially the surface of the earth

105

Region of Interest, ROI

106

image stack

107

Georeferenced reference geometry of GL, e.g. road network from electronic map

108

combined image stack aligned to the reference geometry of the GL

109

checkpoint

110

continuous GL segment, identical to the GL in the present example

111

sub-grid

112

Image characteristics (e.g. pixel values) of georeferenced source images from the image stack

113

grid cell

114

aggregation function

115

straightened image with regular grid

116

evaluation data

200

combined image stack aligned to the reference geometry of the GL

210

RGB image plane

220

Elevation/topology image plane

230

vegetation image layer

240

Image plane to represent the surface type

250

evaporation image plane

260

Precipitation forecast image layer

300

exemplary output of possible evaluation results of the procedure 1 or 5

400

compression method

410

Original image, i.e. original image plane

411

GL network, e.g. transport line network

412

Snow free ground

413

Forest

414

snowy ground

420

Low resolution base image

421

Picture element (pixel) of a base image with low resolution

430

native, high-resolution image of GL

421

Picture element (pixel) of a base image with high resolution

440

Reduced resolution, e.g. half resolution, image of the immediate GL environment

441

Picture element (pixel) of an image with reduced resolution

450

Multi-resolution intermediate image resulting from the overlay of images 420, 430 and 440

460

compressed straightened multi-resolution image

500

Method for creating and applying a parameterized model of a method for determining one or more characteristics of a geographic line, GL, or its surroundings

505-565

Process steps of the process 500

600

various exemplary embodiments of sublattices

610

regular, one-dimensional sublattice

615

Cells of sublattice 610

620

regular, two-dimensional sublattice

625

Cells of sublattice 620

630

anisotropic one-dimensional sublattice

635

Cells of sublattice 630

640

non-gapless, equidistantly spaced one-dimensional sublattice

645

Cells of sublattice 640

650

non-gap, non-equidistantly spaced one-dimensional sublattice

655

Cells of sublattice 650

Claims (19)

A method (100) for determining one or more characteristics of a geographic line, GL, or its surroundings, the method (100) comprising: acquiring (S1, S2) georeferenced earth observation data (S1, S2) containing one or more spatially overlapping represent image planes (106) covering a particular region of interest, ROI (105), of the earth's surface from an air or space perspective, the ROI (105) containing the GL; geometrically transforming (S4 to S7) the georeferenced observation data, at least in part, into a local internal frame of reference (115) of the GL within the ROI (105) to obtain a mapping of the georeferenced observation data to corresponding coordinates within the local internal frame of reference of the GL; and evaluation (S8) of the imaged earth observation data, as represented in the local internal reference system of GL, according to a classification scheme in order to obtain evaluation data therefrom which classify one or more properties represent properties of GL or its environment according to one or more properties of GL. Method (100) according to claim 1 wherein the geometric transformation of the georeferenced observation data, at least in part, into the local internal frame of reference of the GL comprises: defining a plurality of control points within the ROI (105) at different positions along the GL; determining for each control point a respective local cell grid covering a limited local spatial neighborhood of that control point, the resulting set of local cell grids defining the local internal frame of reference of the GL; and geometrically transforming the observation data, at least in part, into the local internal frame of reference of GL to obtain a mapping of the observation data onto the set of local cell grids of the control points. Method (100) according to claim 2 , further comprising: concatenating the individual control point cell grids to form a data representation of a straightened overall cell grid defining the local internal reference system of the GL, wherein the individual control point cell grids are arranged in a rectified, straightened manner in an order that corresponds to the order of the control points along the GL. Method (100) according to claim 3 , in which the evaluation of the respective earth observation data is carried out on the basis of the representation of the earth observation data as they are mapped on the straightened total cell grid. Method (100) according to any one of claims 2 until 4 , wherein at least one of the local cell lattices is defined as having one or more of the following properties: - anisotropic arrangement of at least a subset of the cells within the local cell lattice; - the widths or heights or distances of the cells vary in at least a subset of the cells within the local cell grid; - an orientation of at least a subset of the cells within the local cell grid is aligned with the local orientation of the GL according to its georeferenced reference geometry at the control point corresponding to the local cell grid. Method (100) one of claims 2 until 5 , wherein the georeferenced earth observation data represents one or more pixel values per pixel of an original pixel grid covering the at least one image plane; and geometrically transforming the georeferenced observation data comprises determining at least one cell value for each cell of a local cell grid based on an aggregation of a subset of the pixel values corresponding to an area of the ROI (105) covered by the respective cell of the local cell grid will. Method (100) according to claim 6 , wherein aggregating the subset of pixel values comprises one or a combination of at least two of the following operations: a) averaging the pixel values of all pixels located entirely within the corresponding cell of the local cell grid; b) averaging the pixel values of all pixels overlapping the area of the ROI (105) covered by the corresponding cell of the local cell grid, each pixel value being weighted according to the extent of its overlap with that cell of the local cell grid; c) weighted averaging of the pixel values, where pixels closer to the center of the cell of the local cell grid are given higher weights than pixels further from the center of the cell; d) determining the median, minimum and/or maximum of the pixel values among the pixel values covered by the corresponding cell of the local cell grid; e) determining the pixel values representing a specific percentile of the distribution of pixel values within the corresponding cell of the local cell grid; f) determining one or more of a range, standard deviation, variance, or other higher-order function that captures features of the distribution of pixel values within the cell of the local cell grid; g) determining a histogram that represents the distribution of the pixel values within the cell of the local cell grid. Method (100) according to any one of claims 2 until 7 , wherein at least one of the local cell lattices is defined by a set of lattice parameters; and the method (100) further comprises determining at least one of the lattice parameters based on a process involving machine learning in conjunction with cross-validating intermediate results of the evaluation data obtained during the machine learning process with reference parameter data representing known correct classifications of the one or more concerned Properties of GL or its represent environment according to one or more characteristics. Method (100) according to one of the preceding claims, wherein the acquisition of the georeferenced earth observation data comprises one or more of the following steps: - acquiring georeferenced earth observation data representing one or more of the image planes (210, 220), each of these image planes (210, 220) representing images which cover the ROI (105) in a respective different spectral range of the electromagnetic spectrum; - acquisition of georeferenced earth observation data representing one or more data planes (230 to 260) as further image planes, each of these data planes (230 to 260) respectively containing sensory measurements or predictions of one or more local conditions of the GL or its surroundings within the ROI (105) represents; - acquiring georeferenced earth observation data representing one or more pre-processed image planes resulting from a combination of two or more of these planes (210, 220) or data planes (230 to 260). Method (100) according to claim 9 wherein the data planes (230 to 260) representing the respective sensory measurements of one or more local conditions of the GL or its surroundings within the ROI (105) represent one or more of the following conditions: - precipitation; - Intensity or wavelength of local radiation; - temperature; - air pressure; - Wind strength or direction; - if the GL is a water-based geographical line at least in a portion of the ROI (105): speed and direction of the water flow; - if at least in a portion of the ROI (105) the GL is a land-based geographic line accessible to a land vehicle: vibration, orientation, speed, acceleration or any other vehicle parameter measured by one or more vehicle-based sensors and insight into certain characteristics of the transport line; - if the GL is a transport line: the frequency of use of the transport line, the maintenance status of the transport line or the time since the last maintenance interval, an age or a condition of the surface of the transport line; - Climate zone, climate profiles, expected temperatures or precipitation: - Vulnerability to unexpected events such as accidents, natural disasters, floods, landslides, avalanches, storm damage, tornadoes, hurricanes, typhoons; - electromagnetic fields. Method (100) according to one of the preceding claims, further comprising aligning (S3) the respective geometry of the GL in the one or more image planes (210-260) represented by the earth observation data with a georeferenced reference geometry (107) of the GL in order to to form a combined image stack (200), wherein at least two of the image planes (210 to 260) are integrated into the stack (200), each of these image planes (210 to 260) representing a respective digital image in a given respective spatial resolution, wherein the spatial resolutions of at least two of these image planes (210 to 260) differ from one another, at least in an overlapping area, resulting in a combined image stack (106) with multiple resolutions. The method (100) of any preceding claim, further comprising applying a data compression process (400) to reduce the spatial resolution of a given image plane (410), the data compression process (400) comprising: creating two or more subplanes (420, 430, 440) of the image plane (410) by resampling at least one region of the image plane at a spatial resolution lower than its original spatial resolution, while maintaining or resampling the spatial resolution of at least one other region of the image plane (410) such that when the sub-planes (420, 430, 440) are overlapped to collectively define a combined multi-resolution representation (450; 460) of the image plane (410), the average resolution of the combined multi-resolution representation is lower than the original resolution the image plane (410). Method (100) according to claim 12 , wherein the lower spatial resolution at which at least a region of the image plane (410) is resampled is a function of the region's distance from the GL. Method (100) according to claim 12 or 13 , wherein overlapping the sub-levels (420, 430, 440) comprises aggregating the values of spatially overlapping pixels from different sub-levels (420, 430, 440) according to a predetermined aggregation method to obtain a pixel value of a pixel included in the resulting combined Multi-resolution representation (450; 460) representing a spatial region corresponding to the overlapping pixels of the different sub-planes. Method (100) according to any one of Claims 12 until 14 , in which the pixels of a first of the sub-planes (430, 440) are locally aligned to the respective local orientation of the GL at or in the vicinity of the respective pixel, while the pixels of a second of the sub-planes (420) are aligned to a regular orientation before compression of the pixels in the given image plane (410) correspond. Method (100) according to Claim 14 or 15 , wherein the overlapping and aggregating the values of the spatially overlapping pixels of different subplanes (420, 430, 440) comprises generating the combined multi-resolution representation (460) of the image plane such that the orientation of the resulting pixels of this representation is the same for all those pixels. The method (100) of any preceding claim, further comprising performing one or more of the following operations based on the evaluation data: determining a navigation or information output of a vehicle navigation system; controlling a dynamic traffic sign; setting one or more configurable technical parameters of a vehicle or other transportation system; initiating a maintenance or inspection process related to GL; Predicting a future condition of the geographic line or its surroundings. A data processing system configured to perform the method (100) of any Claims 1 until 17 executes A computer program or computer program product that contains instructions that, when executed on one or more processors of a system, Claim 18 are executed, causing the system to perform the method (100) according to any one of Claims 1 until 17 to execute.

Images