EP4379681A1

EP4379681A1 - Method for detecting, predicting and preventing occurrence of failures and continuity control of vehicles, system performing said method and the device used in said system

Info

Publication number: EP4379681A1
Application number: EP22210696.5A
Authority: EP
Inventors: Grzegorz Wiekiera
Original assignee: Prosperita Sp Z OO
Current assignee: Prosperita Sp Z OO
Priority date: 2022-11-30
Filing date: 2022-11-30
Publication date: 2024-06-05

Abstract

The present invention discloses a diagnostic device plugged into vehicle's OBD port, including a main microcontroller; a communication microcontroller OBD/UART; a module for determining device location; a wireless communication module; a non-volatile memory, enabling writing and reading, preferably in the form of memory card; wherein the main microcontroller implements the following stages of the algorithm: recurring analysis of supply voltage, where after exceeding certain threshold diagnostic device is woken up from sleep mode; activation of modules determining device location and wireless communication in order to ensure connectivity with remote diagnostic system and ensure data on localization and current date and time; downloading current PID identifiers from the CAN bus; storing data in the non-volatile memory and transmitting them to remote diagnostic system with the use of wireless communication module; analysis continuation - PID identifiers analysis is continued if the supply voltage exceeds certain threshold, otherwise it goes back to sleep mode. Furthermore the invention discloses a method implemented by the device, and the use of both device and the method for detecting, predicting and preventing the occurrence of malfunction and controlling vehicle performance continuity, preferably in trucks.

Description

Field of the inventnion

Disclosed system and device relate in general to systems assisting in identification of vehicle operating status, particularly in trucks.

Background of the invention

The OBD (On-Board Diagnostics) system is a system installed in motor vehicles, which enables vehicle's self-diagnostic and allows conducting diagnostics during car service check-up. This system constitutes the array of sensors and logic circuits, which control the performance of car elements, including the engine, electronic systems, immobilizer etc.
The OBD system records errors in the form of codes, the set of which is called DTC (Diagnostic Trouble Codes). The overview of codes that have occurred in a given vehicle may be read with a screen reader, which is plugged to the OBD diagnostic socket. The system in the current OBD-II version is a fixed and mandatory specification element of all passenger vehicles and trucks that were manufactured after 2004 in European Union.
The OBD system also includes PID identifiers, which have been specified in SAE J1979. Those codes enable reading various operating parameters of the vehicle, and sending inquiries with the use of CAN bus in order to download certain parameters or conduct tests.
Information carried by PID identifiers is elusive. Individual modules connected to CAN network repeatedly send information to CAN network about the parameters of their performance. For example, the engine controller constantly and repeatedly within a second sends information regarding the engine speed (rpm), cooling liquid temperature or oil pressure. These information is not stored in any car module, and when they are not read (therefore not saved) they will be permanently lost. After parameters are used by other modules, they are being replaced by subsequent, more current values. It means, that without malfunction occurring at a given moment it is impossible to restore engine operating parameters, even if they had occurred a few seconds earlier.

The state of art

Method and device using intelligent sensors for automotive vehicle diagnostics are known from patent specification PCT/US2020/045622 . The patent discloses different embodiments of method and device for diagnosing the vehicle's condition with the use of SMART cognitive sensors. SMART sensors have at least certain computing power directly connected with the sensor. The SMART sensor, which contains artificial intelligence (Al) core, is directly connected to a sensor in the SMART sensor. The sensor may be an audio sensor, such as a high-quality microphone capable of registering sounds coming from the vehicle. Such sounds may be linked to one or several vehicle elements. AI core analyses sounds coming from the vehicle with the purpose of determining the operational condition of the vehicle. Communication module enables transferring the results to remote, interested agents.
Method of determining flawed vehicles, where flawed vehicles comprise the subset of vehicles, and vehicles are divided into many vehicle types is known from patent description PCT/EP2022/052918 , and the said method includes: providing the value of expected number of messages predefined to be sent to each vehicle type, establishing the real value of predefined messages sent to each vehicle type, establishing deviation value for each vehicle, where each deviation value is representative for the difference between real value and expected value, establishing the flawed subset of flawed vehicles depending on the deviation value, where real value of the flawed subset differs from expected value. The invention also relates to the device, computer program and computer readable data carrier.
The current state of art provides sensors which may be helpful in vehicle diagnostics, but which are not equipped with central server (diagnostic system based on cloud technologies) that would enable data aggregation from multiple vehicles and effective, centralized management of possible malfunctions and states of malfunction occurrence.

Summary of the invention

The main element of the method of the invention is diagnostic device, plugged into a standardized vehicle OBD port. Detailed operation and operational scope of this module, understood as the part of diagnostic system, is described below.
Diagnostic device, which every vehicle connected to the diagnostic system is equipped with, is used for periodical reading of the available data from control units connected to the vehicle's internal CAN network. Data is collected in an active way - through control units polling on individual PID identifiers. Simultaneously, the vehicle location is determined with the use of embedded GPS receiver module.
Data collected by the diagnostic device is aggregated in the diagnostic system, which uses artificial intelligence algorithms to asses vehicle's technical condition and notify the user about the danger of future malfunction.

Brief overview of figures

In order to better understand diagnostic system and diagnostic device, the figures have been enclosed, where:

Fig. 1 depicts a control flow diagram of individual performance stages for the device according to the invention;
Fig. 2 depicts communication scheme for the online and offline solution;
Fig. 3 depicts a diagram of advanced stage diagnostic system
Fig. 4 depicts a sequence diagram for the learning of inference process
Fig. 5 depicts exemplary implementation of the diagnostic device according to the invention, where fig. 5a depicts the device in a casing, and fig. 5b its interior.

Examples

The invention is enclosed in appended claims. Examples below shall be treated as certain, not limiting invention implementation possibilities.

Example 1

Diagnostic device according to the invention includes the following modules:

the main microcontroller
communication microcontroller OBD/UART;
GPS module;
GSM module;
memory card, i.e. micro SD card.

The main microcontroller implements individual stages of the algorithm depicted in fig. 1, which includes the following modules:

1. recurring analysis of supply voltage - in the event of exceeding certain threshold diagnostic device is woken up from sleep mode;
2. once woken up, the device activates GPS and GSM modules, which provide connectivity, localization data and current date and time information;
3. after reaching the connection, diagnostic device downloads current PID identifiers from the CAN bus;
4. next, collected data is saved on a memory card (micro SD card) and transmitted on diagnostic system server with the use of GSM module.
5. if the supply voltage exceeds certain threshold, the analysis of PID identifiers carries on, and when the supply voltage decreases (which means engine shutdown), diagnostic device returns to sleep mode.

Diagnostic device according to the invention was closed in a miniature casing with dimensions of 70x50x25mm. End user receives a fully configured device, the only action to be taken is to plug it into the vehicle's OBD port supervised by the system. Three LEDs located on the casing inform about the operation status:

white LED - informs that the device is supplied with power
blue LED - informs about active communication in CAN network
green LED - informs about communication with the use of GSM network.

Device is adapted to operate both in trucks (supply voltage 24V) and passenger cars (supply voltage 12V), however in order to function properly, it requires configuration adjustment that enables initiating data acquisition at engine start and turning it off at engine shut-off. Decision about waking up the reader and initiating data acquisition is made on the basis of voltage measurement on the power line - during operation of car alternator there is a voltage increase by measurable value, above the nominal value (12/24V) or idle value.
Communication between the device and server is carried out with the help of TCP/IP protocol, with the use of http or https POST method. Data collected from the vehicle is transmitted together with system-unique device identifier. Connection transmission with the use of TCP/IP protocol ensures that all collected data will be sent to the aggregating system. Downloading data at specific frequency may be relevant for the usability of collected data, which classifies this system fraction as the so-called firm real-time. Transmitting data to the server with maximum delay is not critical, however useful considering vehicle tracking functions, which classifies system as soft real-time.
Diagnostic device's internal construction consists of printed circuit boards. Preferably, device according to the invention has been divided into three PCBs, fulfilling separate functions:

Motherboard - implementing the function of protecting power and communication circuits, supplying circuits, main microcontroller with pre-loaded specialist software, programming/debugging connectors, CAN bus communication service;
Communication board - implementing the GSM communicative function, dedicated supply for GSM and micro SD card connector;
GPS navigation board - implementing function of dedicated GSM receiver together with peripheral equipment.

As depicted in fig. 2, diagnostic device can implement communication both online and offline. It means that data collected in the offline mode can be transmitted to the diagnostic system after being connected to the GSM network, enabling TCP/IP connectivity.
This solution has been introduced due to the possibility of limiting transmission medium GSM, which the diagnostic device uses. That is why diagnostic circuit has developed the frame queueing mechanism by adding frame counter and GPS date, if available (depending on the GPS module). On the server side, being a constituent part of the diagnostic system, the aggregating data point has been expanded by subsequent methods - end points. Server application calculates date on the basis of frame number and frequency defined for each device and, in the event of low accuracy, algorithm is able to correct frame date also by data from the GPS frame date, as long as it is available and correct. Such manner of processing outstanding data, queued on the SD card, results in accuracy increase of the entire system.
Device implemented in this manner has the capacity to power the artificial intelligence system with data necessary for running processes, especially predictive and prescriptive processes.

Example 2

Another component of the diagnostic system according to the invention is the cloud, which collects data coming from diagnostic devices and performs adequate normalization steps, and also processes normalized data for detecting anomalies, especially with the use of artificial intelligence techniques, so that in the end, due to internet application, the result of specialized analyses can be presented in an understandable way to the end user. The scope of system implementations and the description of individual components have been depicted in the fig. 3.
Fig. 3 depicts the following modules of the diagnostic system:

1. Data aggregation module enabling reception of raw data sent from diagnostic devices and storing them in the unprocessed form in the preliminary data base. For this purpose, the embodiment uses Microsoft Internet Information Services (IIS) http server (as to launch the application in NET technology) and Microsoft SQL Server data base.
2. Data integration module, consolidating data from several diagnostic devices according to the invention read by the device to the common structure, constituting data source for the data normalizing processor. In the embodiment this module has been implemented and launched as Microsoft Windows.
3. Data normalization module processes (translates) raw, structured data to form understandable both for vehicle mechanics experts and artificial intelligence and at the same time can be delivered to the artificial intelligence module. Complete data is stored in the main system database, shared by other modules according to this invention.
4. CpmDb database module. Microsoft SQL Server database was used in the embodiment.
5. Artificial intelligence module, where the dana analysis is carried out. In this module prediction is conducted and they are stored on previously listed CmpDb database module.
6. Mobile module, which constitutes an application made for mobile platform, i.e. Android platform, designed with React Native technology for reporting faults and malfunctions by the driver.
7. Management-service-expert module, which is the main part of diagnostic system. This module consists of WebApi, which has been created on the base of NET platform and client application designed with ReactJs technology. This module enables managing data, service operations and supports experts in evaluating results, it also enables the preview of vehicles' condition and localization, if available.

CpmDB database module contains all information related to the inference process, including processed raw data under the OBD2 standard (exemplary PID identifiers within the OBD 01 Service can be found on https://en.wikipedia.org/wiki/OBD-II_PIDs#Service_01-_Show_current_data). In this module ancient vehicle performance parameters read from the CAN bus are stored. Additionally, CpmDB database module stores data:

on carried out service works (i.e. oil change);
read DTC codes (stored, pending, permanent)
carried out diagnostic works;
marked events, such as detected fault, error (DTC), prediction, prescription.

Marked event is an event assigned to a given vehicle and to the approximate location where it took place, if relevant.
CpmDB database module is used in particular by data normalization module and artificial intelligence module. By collecting raw data from the device, data normalization module stores processed PID identifiers to this base. Next, artificial intelligence module conducts operation on the collected data set, carrying out advanced operations in order to identify prediction and prescription.
It is carried out in the following way:

1. Determining by clustering the adequate model for a given vehicle. The necessity of conducting this vehicle-adjusted stage derives from the fact that some vehicle makes don't comply 100% with the OBD2 standard. This is why individual models have to differ from one another and further steps cannot always be based on the same data set.
2. Next, data are run through machine learning previously learned model, which detects anomalies. The purpose of this stage is discontinuation of processing data, which don't provide any additional information apart from standard information and are deprived of diagnostic value. Therefore, simplifying the data filtration mechanism is possible.
3. If, after the 2 stage, there is a real-value data set, data are run through rule-based system. On the basis of data collected by an expert, i.e. vehicle mechanics expert, the system verifies if a given data sample constitutes the sufficient indication for detecting and classifying a malfunction.

During system operation all data is collected from the OBD Service 01, as well as from the remaining services with the exception of service 04. Additionally, data is collected from the user, i.e. data on conducted service works. In some instances, localization data is also collected. In one implementation, on the basis of API of OpenStreetMap and technical data, information about the quality of the road or altitude above sea level is stored, which emphasizes certain groups of malfunction classes.
Client application, being part of the system, has been designed on the basis of responsive user interface design, hence is suitable for operating on all devices - such as desktop computer, laptop, tablet, smartphone. Additionally, purposeful division of access application into API and client application results in possibility to access or integrate with any given system, which is able to communicate with API. Such module construction will in the future allow any implementations of this interface.
Fig. 4 depicts sequence diagram for the learning process of inference process.
The set of algorithms has been developed for the inference process and it was based on techniques such as clustering, machine learning and rule-based system. Therefore, the average efficiency of 97% has been achieved for 100 malfunction classes. Learning process of the system, where the artificial intelligence techniques are used, includes the following stages:

1. Adjusting device to the diagnostic cluster, with the aim of selecting the best machine learning model in further steps.
2. Verifying sample in the context of possible anomalies through machine learning.
3. Verifying random samples by a vehicle diagnostic expert in the context of proper functionality of the model.
4. Accepting or rejecting new features for the model.

Artificial intelligence training datasets have been developed on the basis of data actually collected from device tests as part of prototyping subsequent system versions. Additionally, data has been reinforced and selected in a way to emphasize the learning results through accumulated knowledge and information from vehicle mechanics experts.
At later stages, samples marked as potentially displaying signs of a forthcoming malfunction are transmitted to the rule-based subsystem, which on the basis of synthesized expert knowledge (by a knowledge engineer) recognizes if a given sample can be classified under specific malfunction class. If the outcome is valid, the next step is to process this information by the remaining informatics systems with the aim of presenting the result of prediction and/or prescription to the end user, who by definition is not technically skilled to manage such a diagnostic problem.

Example 3

Inference carried out by the artificial intelligence module is based not only on the analysis of errors reported by individual modules, but also on specific operational parameters.
Due to application of the expert system with artificial intelligence it is possible to verify whether the malfunction state actually occurs. Inference is carried out from different starting points, only some of them are directly connected to the DTC, and in such an event system verifies if certain parameters actually exceed reference range and prediction or prescription occurs.
Average values, different for each vehicle cluster.
Further examples show some basic inference models with rules. Those rules may differ for each cluster, as filtering models also are different.

Example 3a

Based on readings from CAN bus, it is possible to conduct prediction of DPF (diesel particulate filter) state.
Conditions:

Emergence of a temporary DTC P2458 (pending)
Is PID value 122 > 50 mBar?
- ∘ If yes, PID 122 > 50 mBar: diesel particulate filter is clogged
- ∘ If not: filter operates properly.

Example 3b

Based on readings from CAN bus, it is possible to conduct prescription of a turbocharger.
Conditions:

Collecting 30 PID 111 and PID 12 sequence readings
Collecting 30 PID 11 and PID 12 sequence readings
Calculating Bv absolute deviation:
- ∘ If Bv for PID12 in the range 1000-1500 > 3% * PID 111, or
- ∘ If PID12 in the range 1900-2300 > 4% * PID 111
If yes: turbocharging module malfunction is approaching
If not: the turbocharging module operates properly

Above conditions result from the fact that after processing PID 111 provides a set of information on requested boost pressure, while PID 11 on actual boost pressure.

Example 3c

Based on readings from CAN bus, it is possible to conduct prediction of DPF malfunction.
Conditions:

PID 12 in the range 1200-1800
PID 90 > 17%
PID 124 > 700°C or PID 122 > 60 mBar
- ∘ If yes: DPF filer failed (has been clogged)
- ∘ If not: filter operates properly.

Example 3d

By analogy, in the event of increased exhaust gas temperature (EGT) maintaining for a longer period of time, the expert system may signalize the higher risk of head gasket malfunction and advice the user to visit service station in order to conduct in-depth diagnostics and determine the cause.

Example 4

The device according to the invention implements the method of detecting, predicting and preventing the occurrence of malfunction and controlling vehicle performance continuity, which includes the following stages:

1. aggregation of raw data raw data sent from diagnostic device according to the invention and storing these data in the unprocessed form in the preliminary data base.
2. consolidating data from at least one diagnostic device according to the invention, read by the device to the common structure, constituting data source for the data normalizing processor;
3. processing raw, structured data to the form understandable both for vehicle mechanics experts and artificial intelligence and at the same time the form can be analyzed by the artificial intelligence module.
4. collecting
1. a. information on inference process, including the input processed data under the OBD2 standard, as well as ancient vehicle performance parameters read from the CAN bus, and
2. b. information on carried out service works;
3. c. read DTC codes (stored, pending, permanent)
4. d. carried out diagnostic works
5. e. events, such as detected fault, error (DTC), prediction, prescription.
5. data analysis with the use of artificial intelligence, including conducting prediction and storing data in the database;
6. ensuring interaction with mobile application for reporting faults and malfunctions by the user;
7. managing devices according to the invention and sharing vehicle condition and localization with the user, also supporting experts in analyzing and evaluating results.

The above method is implemented by the system according to the invention.

Claims

Diagnostic device plugged into vehicle's OBD port, including:
• a main microcontroller;

• a communication microcontroller OBD/UART;

• a module for determining device location;

• a wireless communication module;

• a non-volatile memory, enabling writing and reading, preferably in the form of memory card;
wherein the main microcontroller implements the following stages of the algorithm:
1. recurring analysis of supply voltage, where after exceeding certain threshold diagnostic device is woken up from sleep mode;

2. activation of modules determining device location and wireless communication in order to ensure connectivity with remote diagnostic system and ensure data on localization and current date and time;

3. downloading current PID identifiers from the CAN bus;

4. storing data in the non-volatile memory and transmitting them to remote diagnostic system with the use of wireless communication module;

5. analysis continuation - PID identifiers analysis is continued (stage 3-4) if the supply voltage exceeds certain threshold, otherwise it goes back to sleep mode.
Diagnostic device according to claim 1, characterized in that the wireless communication module uses GSM network.
Diagnostic device according to claims 1-2, characterized in that the module for determining device location used GPS technology.
Diagnostic device according to claims 1-3, characterized in that on stage 4 implemented by the main microcontroller, in the event of inability to immediately transmit data to remote diagnostic system with the use of wireless communication module, the data will be transmitted immediately after restoring wireless communication.
Diagnostic system for detecting, predicting and preventing vehicle malfunction, including the following modules:
1. data aggregation module, receiving raw data sent by the diagnostic device according to claims 1-4 and storing them in the unprocessed form in the preliminary data base;

2. data integration module, consolidating data from at least one diagnostic device according to claims 1-4, read by the device to the common structure, constituting data source for the data normalizing processor;

3. data normalization module, processing raw, structured data to form understandable both for vehicle mechanics experts and artificial intelligence, which and at the same time can be delivered to the artificial intelligence module.

4. data base module collecting at least:
a. information related to the inference process, including processed raw data under the OBD2 standard, as well as ancient vehicle performance read from the CAN bus, and

b. information on carried out service works;

c. read DTC codes (stored, pending, permanent)

d. carried out diagnostic works

e. events, such as detected fault, error (DTC), prediction, prescription.

5. artificial intelligence module, where the data analysis is carried out, including prediction, and they are stored in the database module.

6. mobile module, including the application made for mobile platform designed technology for reporting faults and malfunctions by the user;

7. management-service-expert module enabling the management of diagnostic devices, the preview of vehicles' condition and localization, if available, service works and supporting experts in analyzing and evaluating results
Method of detecting, predicting and preventing the occurrence of malfunction and controlling vehicle performance continuity, which includes the following steps:
1. aggregation of raw data, sent by diagnostic devices according to claims 1-4 and storing these data in the unprocessed form in the preliminary database;

2. consolidating data from several diagnostic devices according to the claims 1-4 read by the device to the common structure, constituting data source for the data normalizing processor;

3. processing raw, structured data to form understandable both for vehicle mechanics experts and artificial intelligence and this form can be analyzed by artificial intelligence module;

4. collecting
a. information related to the inference process, including processed raw data under the OBD2 standard, as well as ancient vehicle performance read from the CAN bus, and

b. information on carried out service works;

c. read DTC codes (stored, pending, permanent)

d. carried out diagnostic works

e. events, such as detected fault, error (DTC), prediction, prescription.

5. data analysis with the use of artificial intelligence, including conducting prediction and storing data in the database;

6. ensuring interaction with mobile application for reporting faults and malfunctions by the user;

7. managing devices according to claims 1-4 and sharing vehicle condition and localization with the user, also supporting experts in analysing and evaluating results.
The use of the device according claims 1-4 for detecting, predicting and preventing the occurrence of malfunction and controlling vehicle performance continuity, preferably in trucks.
The use of the system according claim 5 for detecting, predicting and preventing the occurrence of malfunction and controlling vehicle performance continuity, preferably in trucks.