DE102020113193A1 - Method and system for processing sensor data for transmission to a central unit - Google Patents

Abstract

The present disclosure relates to a method for processing sensor data for transmission to a central unit, comprising providing sensor data by vehicle-internal sensors of a vehicle; deriving abstracted information from the sensor data; and sending, by the vehicle, the abstracted information to a central unit.

Description

The present disclosure relates to a method for processing sensor data for transmission to a central unit, a storage medium for carrying out the method, a system for processing sensor data for transmission to a central unit and a vehicle with such a system. The present disclosure relates in particular to an architectural concept for managing vehicle data streams.

State of the art

The networking of modern vehicles with the Internet, for example, is becoming increasingly important. For example, data collected by vehicle-internal sensors can be used to enable personalization in the vehicle that is tailored to the user. In particular, vehicle functions can be personalized, for example with regard to an interior ambience, driving properties, communication functions, navigation functions, etc.

The personalization can take place by means of the creation of models, such as, for example, user models. The models can be created and optionally also updated using the data from the vehicle's internal sensors and machine learning. The creation and / or updating of the models can take place in a central unit, and in particular in a backend of a vehicle manufacturer. To do this, the data from the vehicle's internal sensors must be transmitted to the backend.

However, the in-vehicle sensors produce large amounts of data. The amount of data for a single vehicle can be in the order of 25 gigabytes or more per hour. A transmission of such amounts of data from a vehicle to the backend is difficult and often even impossible with today's communication technologies. In addition, the data in their raw form are only of limited applicability. Depending on the application and the quality of the data (e.g. structures, formats, metadata, etc.), data preparation is time-consuming and resource-intensive.

Disclosure of the invention

It is an object of the present disclosure to provide a method for processing sensor data for transmission to a central unit, a storage medium for carrying out the method, a system for processing sensor data for transmission to a central unit and a vehicle with such a system which reduce the amount of data that is communicated to a central unit. Furthermore, it is an object of the present disclosure to reduce computation resources in the central unit and / or to make the data processing in the central unit more independent of the received data.

This problem is solved by the subject matter of the independent claims. Advantageous refinements are given in the subclaims.

According to an independent aspect of the present disclosure, a method for processing sensor data for transmission to a central unit is specified. The method comprises providing sensor data by vehicle-internal sensors of a vehicle; deriving abstracted information from the sensor data; and sending, by the vehicle, the abstracted information to a central unit.

According to the invention, abstracted information is determined and sent to the central unit. In other words, it is not the complete raw sensor data that is sent to the central unit, but information derived from the raw sensor data. For example, if a vehicle accelerates rapidly over a short period of time, this fact can be communicated to the central unit from start to finish instead of sending the corresponding exact speed to the central unit. The same applies to constant speed or braking.

In particular, knowledge can be expressed in the form of basic events. The basic events can be generated with an analytical description of the signals or sensor data over time. Once the events are generated, they can be converted into semantic annotations to get the results in a standardized form. In this way, the amount of data that is communicated to a central unit can be reduced. In addition, fewer computing resources are required in the central unit. A certain independence from the underlying system is also guaranteed.

The sensor data (or data streams) preferably comprise, or are, continuous sensor data (or data streams) and / or categorical sensor data (or data streams). Continuous data are numbers that can have any value in a range (e.g. speed). In contrast, categorical data takes values from a predefined set of categories (e.g. number of aisles).

The abstracted information is preferably derived based on a time series of sensor data. The time series can for example be contained in a time window or base window.

Deriving abstracted information from the sensor data preferably includes extracting at least one feature, in particular a statistical feature, from the sensor data in order to generate at least one event. At least one feature can in particular be extracted from continuous sensor data (or data streams) in order to categorize the continuous sensor data (or data streams). By using statistical features and thus a combination of individual points from time series raw data to stable and well-described abstract information, a stable input with less information can be provided without too much information being lost.

The method preferably further comprises determining a semantic annotation for the at least one event. With this, knowledge can be expressed in the form of basic events. For example, the annotation can be formalized with ontological metadata about the dynamic operation of the vehicle.

The method preferably further comprises inputting the abstracted information into a machine learning (ML) module. In particular, the abstracted information can be used to train models. The abstracted information provides a stable input with less information, so that reliable training can still take place. In some embodiments, the machine learning (ML) module can be implemented in the central unit.

ML models are only one exemplary way of using the abstracted information. Semantic knowledge extraction can also be implemented, especially if the data is already described in this format.

The derivation of abstracted information from the sensor data is preferably carried out by the vehicle and / or by means of edge computing. This means that no large volumes of data need to be transmitted to the central unit, since the abstracted information is generated directly in the vehicle and / or in the vicinity of the vehicle.

According to a further independent aspect of the present disclosure, a software (SW) program is specified. The SW program can be set up to be executed on one or more processors, and thereby to execute the method described in this document for processing sensor data for transmission to a central unit.

According to a further independent aspect of the present disclosure, a storage medium is specified. The storage medium can comprise a SW program which is set up to be executed on one or more processors and thereby to carry out the method described in this document for processing sensor data for transmission to a central unit.

According to a further independent aspect of the present disclosure, a system for processing sensor data for transmission to a central unit is specified. The system comprises one or more processors which are set up to carry out the method described in this document for processing sensor data for transmission to a central unit.

The one or more processors are set up in particular to derive abstracted information from sensor data that are provided by vehicle-internal sensors of a vehicle; and to transmit the abstracted information to a central unit.

In particular, the system is set up to carry out what is described in this document for processing sensor data for transmission to a central unit.

According to a further aspect of the present disclosure, a vehicle, in particular a motor vehicle, is specified. The vehicle comprises the system for processing sensor data for transmission to a central unit according to the embodiments of the present disclosure.

The term vehicle includes cars, trucks, buses, mobile homes, motorcycles, etc. that are used to transport people, goods, etc. In particular, the term includes motor vehicles for passenger transport.

Figure list

• 1 ein Flussdiagram eines Verfahrens zum Verarbeiten von Sensordaten zur Übermittlung an eine zentrale Einheit gemäß Ausführungsformen der vorliegenden Offenbarung,
• 2 schematisch ein System zum Verarbeiten von Sensordaten zur Übermittlung an eine zentrale Einheit gemäß Ausführungsformen der vorliegenden Offenbarung,
• 3 eine Übersichtsdarstellung einer Verarbeitung von Sensordaten gemäß Ausführungsformen der vorliegenden Offenbarung, und
• 4 eine initiale Verarbeitung einen Datenstroms gemäß Ausführungsformen der vorliegenden Offenbarung.

Exemplary embodiments of the disclosure are shown in the figures and are described in more detail below. Show it:

• 1 a flow diagram of a method for processing sensor data for transmission to a central unit according to embodiments of the present disclosure,
• 2 schematically a system for processing sensor data for transmission to a central unit according to embodiments of the present disclosure,
• 3 an overview illustration of a processing of sensor data according to embodiments of the present disclosure, and
• 4th initial processing of a data stream in accordance with embodiments of the present disclosure.

Embodiments of the Disclosure

Unless otherwise noted, the same reference symbols are used below for elements that are the same and have the same effect.

1 schematically shows a flow diagram of a method 100 for processing sensor data for transmission to a central unit according to embodiments of the present disclosure. The procedure 100 can be implemented by appropriate software that can be executed by one or more processors (e.g. a CPU).

The procedure 100 includes in the block 110 a provision of sensor data by in-vehicle sensors of a vehicle; in the block 120 deriving abstracted information from the sensor data; and in the block 130 sending, by the vehicle, the abstracted information to a central unit.

The creation of vehicle context for personalized features can be done through machine learning models that are strictly dependent on a specific signal input. The creation and maintenance of these models is usually associated with a high level of computing resources and data. In order to validate and improve the model over time, it is also necessary to synchronize the data with the backend. The same is true for applications related to vehicle fleets.

According to the embodiments of the present disclosure, the communicated data and / or the computing resources in the backend can be reduced and can be independent of the source signal. In addition, the abstraction or description of the data can take place in a standardized manner in order to ensure use across a vehicle fleet.

2 FIG. 3 schematically shows a system for processing sensor data for transmission to a central unit in accordance with embodiments of the present disclosure.

The vehicle is exemplary 10 , the central unit 20th and a mobile device 30th shown.

The mobile device 30th can be used in the context of edge computing to generate the abstracted information from the sensor data. The use of the mobile device 30th when generating the abstracted information is optional and only shown here as an example. In some embodiments, different and / or additional edge computing resources may be used. In still other embodiments, the abstracted information can be generated from the sensor data exclusively by the vehicle or by elements / resources installed in the vehicle (eg control devices, etc.).

A mobile terminal device is a device that is able to operate in a mobile network via local networks or Local Area Networks (LANs), such as wireless LAN (WiFi / WLAN), or via wide area networks (WANs) such as Global System for Mobile Communication (GSM), General Package Radio Service (GPRS), Enhanced Data Rates for Global Evolution (EDGE), Universal Mobile Telecommunications System (UMTS), High Speed Downlink / Uplink Packet Access (HSDPA, HSUPA), Long-Term Evolution (LTE), or World Wide Interoperability for Microwave Access (WIMAX) to communicate wirelessly. Communication via other current or future communication technologies, e.g. 5G mobile radio systems, is possible.

The term mobile device includes in particular smartphones, but also other mobile telephones or cell phones, personal digital assistants (PDAs), tablet PCs, notebooks, smart watches and all current and future electronic devices that are equipped with a communication capability.

In some embodiments, the vehicle can 10 comprise a communication module, for example with a SIM unit. The communication module can be used for communication with the mobile terminal 30th and / or the central unit 20th be set up.

In an exemplary embodiment, the mobile terminal can 30th via a first communication link 1 with the vehicle 10 and via a second communication link 2 with the central unit 20th communicate. The vehicle 10 can be via a third communication link 3 with the central unit 30th communicate.

The first communication link 1 between the mobile device 30th and the vehicle 10 can for example be done using a mobile network or near field technology such as Bluetooth. The mobile network can be, for example, an LTE network or 5G network.

The second communication link 2 between the mobile device 30th and the central unit 20th can be done via a mobile network, such as an LTE network or 5G network.

The third communication link 3 between the vehicle 10 and the central unit 20th can be done via a mobile network, such as an LTE network or 5G network.

Some of the above communication links between the vehicle 10 , the mobile device 30th and the central unit 20th however, they are not limited to the examples mentioned and other suitable types of direct or indirect communication connections, which direct or indirect communication between the vehicle, can be used 10 , the mobile device 30th and the central unit 20th enable. For example, communication links can be used for the first 1 a local network such as wireless LAN (WiFi / WLAN) can be used.

3 FIG. 11 shows an overview of a processing of sensor data according to embodiments of the present disclosure. In particular shows 3 Steps in the semantification of vehicle data streams. A continuous data type is categorized and semantic annotation is added to the categorized data.

Due to the enormous variety of vehicle data, the present disclosure provides a data architecture with a generic and application-related agnostic approach for handling data streams. In particular, with the embodiments of the present disclosure, knowledge can be expressed in the form of basic events. One possibility for this is to generate such basic events through an analytical description of the signals over time. Once the events have been generated, the events can be converted into semantic annotations in order to obtain the results in a standardized form (e.g. graphical data model and ontologies).

The data streams are either continuous or categorical. Continuous data are numbers that can have any value in a range (e.g. speed). In contrast, categorical data takes values from a predefined set of categories (e.g. number of aisles).

The approach explained above will now be explained in detail below.

Data stream

Since countless use cases can be derived from the variety of vehicle data, the processing of the sensor data begins with treating the data streams independently of one another. For this purpose, a partial memory approach with time windows that can be configured depending on the application can be used.

4th shows an initial processing of independent data streams. Exact statistical features are continuously obtained from a base window B. Sums thereof are used as input for a further window W.

In particular, an incoming data stream X is processed through a basic time window B, which can contain up to b elements. When the base time window B is full, it provides a summary of the features S. Another window W collects up to w such summaries and performs a certain interpretation thereof in order to trigger events and add annotations. The window W thus contains summarized information from the data stream X.

Feature extraction

• Kontinuierliche Daten: In jedem Basisfenster B kann ein Mittelwert erhalten werden, der zur Berechnung eines Trends verendet wird, um Zusammenfassungen S zu konstruieren. In jedem Fenster W werden diese Zusammenfassungen erfasst, und es kann eine Standardabweichung berechnet werden. Aus diesen Merkmalen ist eine symbolische aggregierte Näherung (Symbolic Aggregate Approximation, SAX) möglich. Schließlich wird ein Feature-Vektor erzeugt.

On the basis of the data processing explained in the previous section, features are extracted that contain the interpretation of the data stream. To do this, the following aspects are defined:

• Continuous data: In each base window B, an average value can be obtained which is used to calculate a trend in order to construct summaries S. In each window W these summaries are recorded and a standard deviation can be calculated. A symbolic aggregate approximation (SAX) is possible from these features. Finally, a feature vector is created.

Categorical data: Here the value can be described over time. Therefore the consecutive instances of a value can be counted. In an event-driven system, categorical data is only updated when a new value is available. In this case, the time that has elapsed until the change occurred is recorded, with the occurrence of the previous value, the time of the change and the new value to be discribed. If the system is not event driven, the same value will appear repeatedly at a regular sampling rate until it changes. For this type of data stream, a basic window made up of two elements can be defined, with changes in value being monitored.

Event based knowledge

In the automotive industry, many software architectures follow an event-driven approach. The knowledge is therefore preferably expressed in the form of basic events that are generated by an analytical description of the signals or sensor data over time.

In detail:

a) Generation of events

The architecture according to the embodiments of the present disclosure preferably generates events from categorical data. For example, the feature extraction method discussed in the previous paragraph can be used to convert continuous data streams into categorical data streams. For example, instead of the raw sensor data, there are sequences of trends. This means that a simple logic can be used for categorical data with a finite state machine (FSM). Possible events are of the type "Change", "Descriptive" and "Observation".

A categorical data stream is always in one of two possible states: "Changing" or "Grouping".

The "Changing" status is always reached when the current value differs from its previous value. In this state, two types of events are generated, namely “Change” and “Descriptive”. A "Change" event indicates that the value of the data stream has changed at a certain point in time. A "Descriptive" event, on the other hand, describes the previous value of the data stream and its duration.

In contrast to this, a “grouping” state is reached when the incoming element is the same as the previous element. In this case, "observation" events can be generated after a certain number of instances in order to keep the system up to date if necessary.

b) Semantic annotation

As soon as the data has been interpreted, the semantics can be expressed explicitly (e.g. with RDF or another graph data model). For this purpose, the annotation can be formalized with ontological metadata about the dynamic operation of the vehicle. For example, the ontologies VSSo and IoTStream can be used for this purpose. Both use the SOSA ontology to define sensor observations and actuators.

c) Expressing knowledge:

Knowledge can be expressed in terms of events resulting from categorical data streams. Rules can be used for this.

Below are some examples of this rule-based approach. If the “Observation” event of the “SPEED” data stream is an “INCREASE” in its trend value, this may imply that the vehicle is performing an “ACCELERATION”. In a further example, it can be defined that if the duration of a “Descriptive” event of the “RISE” type lasts more than a certain number of time steps, this is “LONG”, otherwise it is “SHORT”. Similarly, a “change” event in the number of gears from 5th gear to 1st gear could imply “BAD DRIVING PRACTICE”, which could be of interest for further analysis.

It should be understood that some insights can be subjective and their conditions can change over time. Indeed, the embodiments of the present disclosure allow knowledge to be dynamic rather than hard-coded. Therefore, changes in knowledge do not affect data processing or feature extraction methods.

In summary, with the embodiments of the present disclosure, abstracted information is determined and sent to the central unit. In other words, it is not the complete raw sensor data that is sent to the central unit, but information derived from the raw sensor data. For example, if a vehicle accelerates rapidly over a short period of time, this fact can be communicated to the central unit from start to finish instead of sending the corresponding exact speed to the central unit. The same applies to constant speed or braking.

In particular, knowledge can be expressed in the form of basic events. The basic events can be generated with an analytical description of the signals or sensor data over time. Once the events are generated, they can be converted into semantic annotations to get the results in a standardized form. In this way, the amount of data that is communicated to a central unit can be reduced. In addition, fewer computing resources are required in the central unit. A certain independence from the underlying system is also guaranteed.

Although the invention has been illustrated and explained in more detail by preferred exemplary embodiments, the invention is not restricted by the disclosed examples and other variations can be derived therefrom by the person skilled in the art without departing from the scope of protection of the invention. It is therefore clear that there is a multitude of possible variations. It is also clear that the embodiments mentioned by way of example really only represent examples that are not to be interpreted in any way as a limitation, for example, of the scope of protection, the possible applications or the configuration of the invention. Rather, the preceding description and the description of the figures enable the person skilled in the art to specifically implement the exemplary embodiments, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without the To leave the scope of protection, which is defined by the claims and their legal equivalents, such as further explanations in the description.

Claims (10)

A method (100) for processing sensor data for transmission to a central unit, comprising: Providing (110) sensor data by in-vehicle sensors of a vehicle; Deriving (120) abstracted information from the sensor data; and Sending (120), by the vehicle, the abstracted information to a central unit. The method (100) according to Claim 1 , wherein abstracted information is derived based on a time series of sensor data. The method (100) according to Claim 1 or 2 wherein deriving abstracted information comprises: generating at least one event from categorical sensor data; and / or extracting at least one feature, in particular a statistical feature, from continuous sensor data in order to generate at least one event. The method (100) according to Claim 3 , further comprising: determining a semantic annotation for the at least one event. The method (100) according to one of the Claims 1 until 4th , wherein the sensor data comprise continuous sensor data and / or categorical sensor data. The method (100) according to one of the Claims 1 until 5 , further comprising: inputting the abstracted information into a machine learning module. The method (100) according to one of the Claims 1 until 6th , the deriving of abstracted information from the sensor data takes place by the vehicle and / or by means of edge computing. A storage medium, comprising a software program which is set up to be executed on one or more processors, and thereby the method (100) according to one of the Claims 1 until 7th to execute. System for processing sensor data for transmission to a central unit, comprising one or more processors which are set up to carry out the method (100) according to one of the Claims 1 until 7th to execute. Vehicle (10), in particular motor vehicle, comprising the system according to Claim 9 .

Images