CN114679417A - In-vehicle data distribution method, system, device and readable storage medium - Google Patents

Abstract

The present application relates to the technical field of in-vehicle data processing, and discloses a method, system, device and readable storage medium for distributing in-vehicle data. The method includes: acquiring on-board data returned by at least one on-board system; buffering the on-board data in a message queue; acquiring at least part of the on-board data from the front end of the message queue, and distributing to the corresponding vehicle management system according to the type of at least part of the on-board data platform. Through the above method, the persistence of the on-board data can be ensured, the data amount of the on-board data received by the vehicle management platform can be reduced, and the processing efficiency of the on-board data by the vehicle management platform can be improved.

Description

In-vehicle data distribution method, system, device and readable storage medium

technical field

The present application relates to the technical field of in-vehicle data processing, and in particular, to a method, system, device and readable storage medium for distributing in-vehicle data.

Background technique

At present, with the rapid development of the Internet and the rapid popularization of 5G, the era of the Internet of Everything is coming. Vehicles can send back various data collected by sensors in real time through the network. All departments of the company need these data for their analysis and iterative evolution of algorithms. In order to process these data quickly and accurately analyze the data, each department of the company has developed corresponding platforms, but the on-board data required by these management platforms are not the same. It is very time-consuming and the processing efficiency is relatively low.

For example, the management platforms of different departments have different requirements for data, such as the vehicle status monitoring platform, which requires vehicle online status, automatic driving mileage of each vehicle, vehicle abnormal reporting information, vehicle order information, etc., which requires higher real-time vehicle status. , the data needs to be sent to it in real time; the hardware research and development platform needs to know the hardware status information of the vehicle; the route planning platform needs to know the location information of the vehicle; the algorithm platform needs the historical data of the vehicle and so on.

SUMMARY OF THE INVENTION

The present application provides a method, system, device and readable storage medium for distributing vehicle data, which can ensure the persistence of vehicle data, reduce the amount of data received by the vehicle management platform, and improve the processing efficiency of vehicle data by the vehicle management platform.

In order to solve the above technical problems, a technical solution adopted in the present application is to provide a method for distributing on-board data, the method comprising: acquiring on-board data returned by at least one on-board system; buffering the on-board data in a message queue; The front end of the queue acquires at least part of the vehicle-mounted data, and distributes it to the corresponding vehicle management platform according to the type of at least part of the vehicle-mounted data.

Wherein, before the on-board data is cached in the message queue, it includes: encapsulating the on-board data according to a preset field; and caching the on-board data in the message queue includes: buffering the encapsulated on-board data in the message queue.

Wherein, encapsulating the in-vehicle data according to the preset fields includes: encapsulating the in-vehicle data according to the context field, the event type field, the metadata field, the priority field and the payload field.

Wherein, acquiring at least part of the on-board data from the front end of the message queue includes: monitoring the on-board data of the message queue, and when monitoring the on-board data, acquiring at least part of the on-board data from the front end of the message queue.

Wherein, before the on-board data is cached in the message queue, it includes: encapsulating the on-board data according to the event type field; acquiring at least part of the on-board data from the front end of the message queue, and distributing to the corresponding vehicle management system according to the type of at least part of the on-board data The platform includes: determining whether the event type of the target on-board data in at least part of the on-board data matches the preset event type of the vehicle management platform; if it matches, data encapsulates the target on-board data again, and distributes it to the corresponding vehicle management platform; If it does not match, the target vehicle data is discarded.

Wherein, performing data encapsulation on the target vehicle-mounted data again and distributing it to the corresponding vehicle management platform includes: filtering at least part of the encapsulated vehicle-mounted data; and distributing at least part of the filtered vehicle-mounted data to the corresponding vehicle management platform.

Wherein, distributing the filtered at least part of the vehicle-mounted data to the corresponding vehicle management platform includes: judging whether the data format of at least part of the vehicle-mounted data meets the requirements; if not, converting the data format of at least part of the vehicle-mounted data into the data of the vehicle management platform After the format is completed, it is distributed to the vehicle management platform.

Wherein, the method further includes: determining a publisher and a publisher status corresponding to the vehicle management platform; if the publisher status is in an active state, publishing at least part of the vehicle-mounted data to the vehicle management platform through the publisher.

Wherein, the method further includes: when at least part of the vehicle-mounted data is successfully published to the vehicle management platform, resetting the publisher state, and sending feedback information to the message queue.

The method further includes: when at least part of the vehicle-mounted data fails to be released to the vehicle management platform, counting the number of consecutive failures by the publisher, and performing fuse processing based on the number of consecutive failures.

Wherein, the method further includes: establishing a list of items to be processed according to at least part of the vehicle-mounted data; checking whether there is a list of items to be processed for consumers who need to be restored; if so, consuming from the starting position of the list of items to be processed; Unconsumed location consumption of the entry list.

In order to solve the above technical problems, another technical solution adopted in the present application is to provide an on-board data distribution system, the on-board data distribution system includes: a data interface module for acquiring on-board data returned by at least one on-board system; a message queue; The module, connected to the data interface module, is used to cache the on-board data by using the message queue; the data distribution module, connected to the message queue module, is used to obtain at least part of the on-board data from the front end of the message queue, and according to the type of at least part of the on-board data, distribute it to the corresponding data. vehicle management platform.

Wherein, the data distribution system further includes: a remote configuration module for performing remote system configuration on the data distribution system.

In order to solve the above-mentioned technical problem, another technical solution adopted in the present application is: to provide a vehicle-mounted data distribution device, the distribution device includes a processor and a memory coupled to the processor; wherein, the memory is used for storing computer programs, and the processor uses to execute the computer program to implement the method provided by the above technical solution.

In order to solve the above-mentioned technical problems, another technical solution adopted in the present application is to provide a computer-readable storage medium, which is used for storing a computer program, and when the computer program is executed by the processor, it is used to realize such as: The method provided by the above technical solution.

The beneficial effects of the embodiments of the present application are: different from the prior art, the method, system, device and readable storage medium for distributing on-board data provided by the present application include: acquiring on-board data returned by at least one on-board system; The in-vehicle data is cached in the message queue; at least part of the in-vehicle data is obtained from the front end of the message queue, and distributed to the corresponding vehicle management platform according to the type of at least part of the in-vehicle data. Through the above method, on the one hand, the use of message queues to cache on-board data can ensure the persistence of on-board data. On the other hand, according to the type of at least part of the on-board data, it is distributed to the corresponding vehicle management platform, which can reduce the number of on-board data received by the vehicle management platform. It can improve the processing efficiency of vehicle data on the vehicle management platform.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort. in:

1 is a schematic flowchart of an embodiment of a method for distributing on-board data provided by the present application;

2 is a schematic diagram of an application scenario of a method for distributing on-board data provided by the present application;

3 is a schematic flowchart of another embodiment of a method for distributing on-board data provided by the present application;

4 is a schematic flowchart of another embodiment of a method for distributing on-board data provided by the present application;

5 is a schematic flowchart of an embodiment of step 45 provided by the present application;

6 is a schematic flowchart of an embodiment of step 452 provided by the present application;

7 is a schematic flowchart of another embodiment of a method for distributing on-board data provided by the present application;

8 is a schematic flowchart of another embodiment of a method for distributing on-board data provided by the present application;

9 is a schematic structural diagram of an embodiment of a vehicle-mounted data distribution system provided by the present application;

10 is a schematic diagram of an application scenario of the vehicle-mounted data distribution system provided by the present application;

Fig. 11 is the workflow schematic diagram of the data interface module provided by this application;

Fig. 12 is the workflow schematic diagram of the data distribution module provided by this application;

Fig. 13 is another workflow schematic diagram of the data distribution module provided by the present application;

Fig. 14 is another workflow schematic diagram of the data distribution module provided by the present application;

Fig. 15 is another workflow schematic diagram of the data distribution module provided by the present application;

Fig. 16 is another workflow schematic diagram of the data distribution module provided by the present application;

Fig. 17 is another workflow schematic diagram of the data distribution module provided by the present application;

18 is a schematic structural diagram of another embodiment of a vehicle-mounted data distribution system provided by the present application;

19 is a schematic structural diagram of an embodiment of a vehicle-mounted data distribution device provided by the present application;

FIG. 20 is a schematic structural diagram of an embodiment of a computer-readable storage medium provided by the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, the drawings only show some but not all the structures related to the present application. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

Referring to FIG. 1 , FIG. 1 is a schematic flowchart of an embodiment of a method for distributing vehicle data provided by the present application. The method includes:

Step 11: Acquire vehicle-mounted data returned by at least one vehicle-mounted system.

In some embodiments, in-vehicle data returned by at least one in-vehicle system may be acquired by means of wireless transmission. For example, multiple data interfaces are set, and each data interface is distributed hierarchically. Each data interface is used to obtain in-vehicle data returned by an in-vehicle system.

In some embodiments, in-vehicle data returned by at least one in-vehicle system may be acquired according to a preset period.

In some embodiments, in-vehicle data returned by at least one in-vehicle system can be received in real time.

The vehicle-mounted data may include vehicle online status, driving mileage, vehicle abnormality data, vehicle order information, vehicle hardware status information, vehicle driving location information, vehicle historical data, and the like.

Step 12: Cache the vehicle data in the message queue.

In some embodiments, the message queue may be a redis stream message queue. The stream message queue provides message persistence and master-replicated functions, allowing any client to access data at any time, remember the access location of each client, and ensure that data is not lost. Its writing speed is very fast, and it is very suitable for the middleware of this real-time distribution system.

In some embodiments, the message queue may also be at least one of ZeroMQ, RabbitMQ, ActiveMQ, and Kafka/Jafka.

Step 13: Acquire at least part of the vehicle-mounted data from the front end of the message queue, and distribute it to the corresponding vehicle management platform according to the type of at least part of the vehicle-mounted data.

The types of at least part of the vehicle-mounted data can be classified into vehicle online status, driving mileage, vehicle abnormal data, vehicle order information, vehicle hardware status information, vehicle location information, vehicle history data, and the like.

In some embodiments, the in-vehicle data of the message queue is monitored, and when the in-vehicle data is monitored, at least part of the in-vehicle data is obtained from the front end of the message queue, and distributed to the corresponding vehicle management platform according to the type of at least part of the in-vehicle data.

In an application scenario, it is described with reference to Figure 2:

Through step 11, the in-vehicle data returned by the in-vehicle system A, the in-vehicle system B, and the in-vehicle system C are acquired in real time.

After acquiring the in-vehicle data returned by the in-vehicle system A, the in-vehicle system B, and the in-vehicle system C, the in-vehicle data is sent to the message queue for buffering.

After the on-board data is cached in the message queue, at least part of the on-board data is obtained from the front end of the message queue, and distributed to the corresponding vehicle management platform D, vehicle management platform E, and vehicle management platform F according to the type of at least part of the on-board data.

For example, there are 100 pieces of vehicle data in the message queue. Acquire 20 pieces of vehicle-mounted data from the front end of the message queue, and distribute them to the corresponding vehicle management platform D, vehicle management platform E or vehicle management platform F according to the types of the 20 vehicle-mounted data. After the distribution is successful, it is fed back to the message queue module 20 . The message queue module 20 clears the 20 pieces of on-board data. The data distribution module 30 obtains 20 pieces of in-vehicle data in sequence from the remaining 80 pieces of in-vehicle data in the message queue, and distributes them to the corresponding vehicle management platform D, vehicle management platform E, and vehicle management platform according to the type of the 20 pieces of in-vehicle data. platform F. After the distribution is successful, it is fed back to the message queue module 20 . Distribute all the on-board data in the message queue in sequence.

For example, vehicle management platform D needs on-board data corresponding to the online status of vehicles, vehicle management platform E needs on-board data corresponding to driving mileage, and vehicle management platform F needs on-board data corresponding to abnormal vehicle data. Then, the data distribution module 30 distributes to the corresponding vehicle management platform D, vehicle management platform E and vehicle management platform F according to the type of at least part of the vehicle-mounted data.

In this embodiment, the in-vehicle data returned by at least one in-vehicle system is acquired; the in-vehicle data is cached in a message queue; at least part of the in-vehicle data is acquired from the front end of the message queue, and according to the type of at least part of the in-vehicle data, distributed to corresponding On the one hand, the function of caching on-board data using message queue can ensure the persistence of on-board data; The platform receives the data volume of in-vehicle data, and improves the processing efficiency of in-vehicle data by the vehicle management platform.

Referring to FIG. 3 , FIG. 3 is a schematic flowchart of another embodiment of the method for distributing on-board data provided by the present application. The method includes:

Step 31: Acquire vehicle-mounted data returned by at least one vehicle-mounted system.

Step 32: Encapsulate the vehicle data according to the preset fields.

Among them, the preset field can be set in advance. For example, preset fields can be configured in the configuration file in advance. Thus, the packaging requirements are preset in the configuration file.

In some embodiments, onboard data may be encapsulated in a context field, event type field, metadata field, priority field, and payload field.

Among them, the context field context can contain context information, which is very important for link tracking and cross-platform link tracking and troubleshooting. For example, the context field can be set to "dict={}".

The event type field event_type has the same value as the path parameter topic, which is very important for subsequent distribution. For example, the event type field can be set to string type.

The metadata field metadata is used to record the information of some in-vehicle data, fill in as needed. The metadata field can be set to "dict".

The priority field priority is sample by default. Generally, do not modify it. It is very important to use the notification processing for subsequent distribution. The priority field can be set to "sample".

The payload field payload is used to store the message body. The payload field can be set to "dict".

Step 33: Cache the packaged vehicle data in the message queue.

Step 34: Acquire at least part of the vehicle-mounted data from the front end of the message queue, and distribute to the corresponding vehicle management platform according to the type of at least part of the vehicle-mounted data.

In step 34, the event type of the in-vehicle data may be further used for judgment to determine whether the in-vehicle data conforms to the target vehicle management platform. If so, it is then distributed to the corresponding vehicle management platform according to the type of at least part of the vehicle-mounted data.

In this embodiment, the on-board data is encapsulated according to preset fields, which can facilitate subsequent data distribution, and the message queue is used to cache on-board data, which can ensure the persistence of on-board data. On the other hand, according to the type of at least part of the on-board data , distributed to the corresponding vehicle management platform, which can reduce the amount of data received by the vehicle management platform and improve the processing efficiency of the vehicle management platform for the vehicle data.

Referring to FIG. 4 , FIG. 4 is a schematic flowchart of another embodiment of the method for distributing on-board data provided by the present application. The method includes:

Step 41: Acquire vehicle-mounted data returned by at least one vehicle-mounted system.

Step 42: Encapsulate the in-vehicle data according to the event type field.

Wherein, in step 42, encapsulation of various fields may also be performed according to any of the foregoing embodiments.

Step 43: Cache the encapsulated vehicle data in the message queue.

Step 44: Determine whether the event type of the target vehicle data in at least part of the vehicle data matches the preset event type of the vehicle management platform.

If it matches, go to step 45, if not, go to step 46.

Step 45: Perform data encapsulation on the target vehicle data again, and distribute to the corresponding vehicle management platform.

Because in the message queue, the data type of the vehicle data is string type, so it is necessary to encapsulate the data again before distribution to encapsulate the data type required by the vehicle management platform.

In some embodiments, referring to FIG. 5, step 45 may be the following flow:

Step 451: Perform a filtering operation on at least part of the encapsulated vehicle-mounted data.

In step 451, filtering is performed according to the type of at least part of the vehicle-mounted data. For example, if the vehicle management platform needs the on-board data of the online status of the vehicle, other on-board data will be filtered out, and the on-board data of the online status of the vehicle will be retained.

Step 452: Distribute at least part of the filtered vehicle-mounted data to the corresponding vehicle management platform.

In some embodiments, referring to FIG. 6, step 452 may be the following flow:

Step 61: Determine whether the data format of at least part of the vehicle-mounted data meets the requirements.

It can be understood that different vehicle management platforms have different data formats. Therefore, it is determined whether the data formats of at least part of the vehicle-mounted data meet the requirements.

If yes, it is directly distributed to the vehicle management platform; if not, step 62 is executed.

Step 62: After converting the data format of at least part of the vehicle data into the data format of the vehicle management platform, distribute to the vehicle management platform.

Step 46: Discard the target vehicle data.

It can be understood that discarding the target on-board data is not to distribute to one of the vehicle management platforms, but the target on-board data may be distributed to other vehicle management platforms.

In this embodiment, the on-board data is firstly encapsulated according to the event type field, which can facilitate subsequent data distribution to be matched according to the event type, so as to preliminarily determine the on-board data of the corresponding vehicle management platform. According to the data requirements of the on-board management platform, further, through the data format conversion, it is convenient for the on-board data to be released to the vehicle management platform smoothly. Based on this, the use of message queues to cache in-vehicle data can ensure the persistence of in-vehicle data, and at least part of the in-vehicle data is distributed to the corresponding vehicle management platform, which can reduce the amount of data received by the vehicle management platform. Data processing efficiency.

Referring to FIG. 7 , FIG. 7 is a schematic flowchart of another embodiment of the method for distributing on-board data provided by the present application. The method includes:

Step 71: Determine the publisher and publisher status corresponding to the vehicle management platform.

The publisher can be understood as a URL (Uniform Resoure Locator, Uniform Resource Locator) of the vehicle management platform. Through this URL, the status of the corresponding vehicle management platform, that is, the publisher status, can be determined.

For example, the publisher status is detected by pinging the URL.

Step 72: If the publisher state is in the active state, publish at least part of the vehicle data to the vehicle management platform through the publisher.

If the publisher status is active, it means that the vehicle management platform communicates normally, and data can be published to it.

In some embodiments, when at least part of the in-vehicle data is successfully published to the vehicle management platform, the publisher status is reset, and feedback information is sent to the message queue.

It can be understood that the status of the publisher is reset after successful publishing, which can facilitate real-time detection of the status of the vehicle management platform, that is, the status of the publisher.

For example, if the previous publisher status was inactive, but this time the publisher status is active, when at least part of the vehicle data is successfully published to the vehicle management platform, reset the publisher status to make the publisher status active .

In some embodiments, when at least part of the in-vehicle data fails to be published to the vehicle management platform, the number of consecutive failures by the publisher is counted, and the fuse processing is performed based on the number of consecutive failures.

It can be understood that if the release fails, it means that the current vehicle management platform is in an abnormal stage. If the number of consecutive failures exceeds the threshold, it means that the vehicle management platform has been in the abnormal stage for a long time, and the data cannot be released to it in a short period of time. Publish data to the vehicle management platform to save system resources.

Wherein, a fusing time can be set, and after the fusing time arrives, the publisher status is determined again.

In this embodiment, according to the state of the publisher, it is decided whether to publish at least part of the vehicle-mounted data to the vehicle management platform, so as to save system resources and avoid the problem of repeated publishing and failure.

Referring to FIG. 8 , FIG. 8 is a schematic flowchart of another embodiment of the method for distributing vehicle data provided by the present application. The method includes:

Step 81: Acquire vehicle-mounted data returned by at least one vehicle-mounted system.

Step 82: Cache the on-board data in the message queue.

Steps 81 to 82 have the same or similar technical solutions as any of the above-mentioned embodiments, and will not be repeated here.

Step 83: Acquire at least part of the vehicle-mounted data from the front end of the message queue, and create a to-be-processed item list according to the at least part of the vehicle-mounted data.

Step 84: Check if there is a pending entry list for the consumer that needs to be restored.

The consumer can understand the process of data distribution. When a consumer is normal, the vehicle data in the item list to be processed will be distributed to the corresponding vehicle management platform according to the distribution method in any of the above embodiments.

If the current consumer dies, that is, the process is interrupted, another process needs to be activated again, that is, a new consumer is created to continue data distribution.

In step 84, if it is checked that there is a pending entry list for a consumer that needs to be restored, step 85 is performed. If it is checked that there is no pending item list for the consumer that needs to be restored, step 86 is performed. If it does not exist, it means that the consumer is normal.

Step 85: Consume from the beginning of the list of pending items.

The pattern of consumption in the list of pending entries is to start processing from the beginning and clear each entry as it is processed. Therefore, when the list of pending items needs to be restored, it needs to be consumed from the starting position.

Step 86: Consume from the unconsumed position of the pending item list.

Step 87: Distribute to the corresponding vehicle management platform according to the type of at least part of the vehicle-mounted data.

In this embodiment, the consumer is monitored, and when the consumer dies, a new consumer is established, and the vehicle-mounted data in the item list to be processed by the new consumer is used for processing, so as to improve the processing efficiency of the vehicle-mounted data.

Referring to FIG. 9 , FIG. 9 is a schematic structural diagram of an embodiment of a vehicle-mounted data distribution system provided by the present application. The in-vehicle data distribution system 100 includes: a data interface module 10 , a message queue module 20 and a data distribution module 30 .

Wherein, the data interface module 10 is used for acquiring the in-vehicle data returned by at least one in-vehicle system.

In some embodiments, the data interface module 10 may have multiple sub-data interfaces, and each sub-data interface is distributed hierarchically. Each sub-data interface is used to obtain the in-vehicle data returned by an in-vehicle system.

In some embodiments, the data interface module 10 may acquire on-board data returned by at least one on-board system according to a preset period.

In some embodiments, the data interface module 10 can receive in-vehicle data returned by at least one in-vehicle system in real time.

The vehicle-mounted data may include vehicle online status, driving mileage, vehicle abnormality data, vehicle order information, vehicle hardware status information, vehicle driving location information, vehicle historical data, and the like.

The message queue module 20 is connected to the data interface module 10, and is used for buffering the on-board data by using the message queue.

In some embodiments, the message queue in the message queue module 20 may be a redis stream message queue. The stream message queue provides message persistence and master-replicated functions, allowing any client to access data at any time, remember the access location of each client, and ensure that data is not lost. Its writing speed is very fast, and it is very suitable for the middleware of this real-time distribution system.

In some embodiments, the message queue in the message queue module 20 may also be at least one of ZeroMQ, RabbitMQ, ActiveMQ and Kafka/Jafka.

The data distribution module 30 is connected to the message queue module 20 for acquiring at least part of the vehicle data from the front end of the message queue, and distributing to the corresponding vehicle management platform according to the type of at least part of the vehicle data.

In an application scenario, it is described with reference to Figure 10:

The data interface module 10 can acquire the in-vehicle data returned by the in-vehicle system A, the in-vehicle system B, and the in-vehicle system C in real time.

After acquiring the in-vehicle data returned by the in-vehicle system A, the in-vehicle system B, and the in-vehicle system C, the data interface module 10 sends the in-vehicle data to the message queue module 20 .

The message queue module 20 uses the message queue to buffer vehicle data.

After the on-board data is cached in the message queue module 20, the data distribution module 30 obtains at least part of the on-board data from the front end of the message queue, and distributes at least part of the on-board data to the corresponding vehicle management platform D, vehicle management platform E, Vehicle management platform F.

For example, there are 100 pieces of vehicle data in the message queue. The data distribution module 30 obtains 20 pieces of vehicle-mounted data from the front end of the message queue, and distributes them to the corresponding vehicle management platform D, vehicle management platform E or vehicle management platform F according to the types of the 20 vehicle-mounted data. After the distribution is successful, it is fed back to the message queue module 20 . The message queue module 20 clears the 20 pieces of on-board data. The data distribution module 30 obtains 20 pieces of in-vehicle data in sequence from the remaining 80 pieces of in-vehicle data in the message queue, and distributes them to the corresponding vehicle management platform D, vehicle management platform E, and vehicle management platform according to the type of the 20 pieces of in-vehicle data. platform F. After the distribution is successful, it is fed back to the message queue module 20 . Distribute all the on-board data in the message queue in sequence.

For example, vehicle management platform D needs on-board data corresponding to the online status of vehicles, vehicle management platform E needs on-board data corresponding to driving mileage, and vehicle management platform F needs on-board data corresponding to abnormal vehicle data. Then, the data distribution module 30 distributes to the corresponding vehicle management platform D, vehicle management platform E and vehicle management platform F according to the type of at least part of the vehicle-mounted data.

In this embodiment, the in-vehicle data distribution system includes: a data interface module for acquiring in-vehicle data returned by at least one in-vehicle system; a message queue module, connected to the data interface module, for caching the in-vehicle data by using the message queue; data distribution The module is connected to the message queue module, and is used for acquiring at least part of the vehicle-mounted data from the front end of the message queue, and distributing to the corresponding vehicle management platform according to the type of at least part of the vehicle-mounted data. In the above way, the data interface module and the data distribution module are separated by the message queue module. On the one hand, the message queue module provides the function of caching on-board data, which can ensure the persistence of on-board data. On the other hand, the separated data interface module and data distribution module The module can avoid the situation that the data interface module fails to send the vehicle data due to the restart of the logic change of the data distribution module, which leads to the loss of the vehicle data, thereby ensuring the security of the vehicle data and greatly reducing the maintenance difficulty of the vehicle data distribution system. On the other hand, according to the Distributing at least some types of vehicle-mounted data to the corresponding vehicle management platform can reduce the amount of data received by the vehicle management platform and improve the processing efficiency of vehicle-mounted data by the vehicle management platform.

In some embodiments, the data interface module 10 is further configured to encapsulate the in-vehicle data according to preset configuration parameters; the message queue module 20 is further configured to cache the encapsulated in-vehicle data by using a message queue.

In some embodiments, parameter configuration may be performed in a configuration file corresponding to the data interface module 10 in advance. For example, the data type corresponding to the data interface module 10 and the event type that needs to be encapsulated for the on-board data. Thus, the packaging requirements are preset in the configuration file. After receiving the on-board data, the data interface module 10 encapsulates the on-board data according to preset configuration parameters. Then, the packaged vehicle data is cached in the message queue of the message queue module.

In an application scenario, with reference to FIG. 11 , the workflow of the data interface module 10 is described:

The data source sends the data to the dispatcher-api service, and the dispatcher-api service encapsulates it and writes it to the redis message queue. Among them, dispatcher-api is equivalent to the data interface module 10 . The data data is equivalent to the above-mentioned in-vehicle data.

The dispatcher-api service obtains the corresponding config file in the nacos configuration, encapsulates the on-board data according to the configuration information in the config file, and then caches the encapsulated on-board data in the redis message queue.

The configuration information in the config file can include context, event_type, metadata, priority, and payload fields.

context: This field contains context information, which is very important for link tracking and cross-platform link tracking and troubleshooting.

event_type: The event type, consistent with the value of the path parameter topic, is very important for the subsequent dispatcher-server service pipeline filtering.

metadata: This field records some information about this message, fill in as needed.

priority: The priority field defaults to sample. Generally, do not modify it. It is very important to use the notification processing for the dispatcher-server service.

Payload: The message body is stored here.

In some embodiments, the configuration information used by the dispatcher-api service is obtained from the nacos service, while monitoring the nacos configuration. If the nacos configuration is manually updated, the dispatcher-api service will synchronize the configuration and dynamically update the service.

In this embodiment, by encapsulating the in-vehicle data, the in-vehicle data of different formats or types are unified, which is convenient to improve the subsequent distribution efficiency.

In some embodiments, the data distribution module 30 is further configured to monitor the in-vehicle data of the message queue, and obtain at least part of the in-vehicle data from the front end of the message queue when the in-vehicle data is monitored.

In some embodiments, the data interface module 10 is further configured to encapsulate the in-vehicle data according to the event type field; the data distribution module 30 is further configured to determine whether the event type of the target in-vehicle data in at least a part of the in-vehicle data is consistent with the pre-defined event type of the vehicle management platform. It is assumed that the event type matches; if it matches, the target vehicle data is encapsulated again and distributed to the corresponding vehicle management platform; if it does not match, the target vehicle data is discarded.

The data distribution module 30 is further configured to perform a filtering operation on at least part of the encapsulated vehicle-mounted data; and distribute the filtered at least part of the vehicle-mounted data to the corresponding vehicle management platform.

The data distribution module 30 is also used to determine whether the data format of at least part of the vehicle data meets the requirements; if not, convert the data format of at least part of the vehicle data into the data format of the vehicle management platform and distribute to the vehicle management platform.

In an application scenario, it is described with reference to Figure 12:

The dispatcher-server service listens to the redis queue, reads the data from the redis queue, and then sends the data to the corresponding pipeline, and filters it through the event filter event filter. After filtering, the data is encapsulated. After the encapsulation is completed Filter the data according to the customized filter conditions, and then select the publisher configured by the user to publish if the conditions are met. The dispatcher-server service is equivalent to the data distribution module 30 .

Among them, the publisher can be a management platform such as kafka, es, http, test, etc. The customized filter condition can be determined according to the publisher, for example, the data required by the management platform corresponding to the publisher, if the vehicle operation log is required, set this as the filter condition.

Among them, the dispatcher-server service configuration problem is also obtained from nacos, monitors the nacos configuration, and can hot update the configuration.

In some embodiments, described in conjunction with Figure 13:

The data distribution module 30 monitors the on-board data of the message queue, and when monitoring the on-board data, takes out the on-board data encapsulated in the message queue in batches, and uses the thread pool to process the batch of on-board data in parallel. The thread processing process is to parse the message first, Traverse all endpoints to determine whether the event_type matches. If it matches, select the corresponding priority function to process the vehicle data, and then encapsulate the vehicle data to determine whether the filter conditions are met. If not, discard it directly. If it is satisfied, determine whether format conversion is required (here is for Adapting to different vehicle management platforms has different requirements for data formats), format conversion if necessary, and direct release if not needed.

In some embodiments, the publishing process is described in conjunction with FIG. 14 :

The publishing process first determines whether the publisher is activated. Only activated publishers can publish messages. If they are not activated, you need to check the status of the publisher to see if the fuse waiting time exceeds the dynamically set fuse time. If it exceeds, activate the publisher and reset it. The duration of the circuit breaker is exponentially increased, but the upper limit is 5 minutes. After the publisher finishes sending the message, it determines whether the publication is successful. If it is successful, the publisher status is reset. Otherwise, the number of failures is counted, and if it exceeds 5 failures, the fuse processing is performed.

Among them, the pipeline configuration uses the yaml file. The yaml file can define sources, sinks and transfers, where sources and sinks must be defined.

Each source must define name, meters and sinks. The data information of the resource is defined in the meters. For example, sentry-source will filter out all the data whose topic is sentry, and send it to kafka through the publisher of kafka_sink. For another example, hardware-source needs to filter finer data, which can be defined in filters, and two publishers are defined in the data source sink, and both require data format conversion. The format converter can be defined in transfers. Among them, the URL corresponding to the publisher also needs to be defined.

Add filter conditions as needed, and send the filtered data through the publisher. If the data format required by the target platform is different, you can define transfer as needed, and so on. Vehicle monitoring platform, hardware R&D platform, positioning module, perception module and PNC module You can define filtered data and corresponding format converters as needed to complete the distribution of the desired data.

In some embodiments, the data distribution module 30 is further configured to check whether there is a pending item list of the consumer that needs to be restored, if so, consume from the beginning of the pending item list, if not, from the unconsumed item list of the pending item list Location consumption, where the list of items to be processed corresponds to part of the vehicle data in the message queue.

In some embodiments, described in conjunction with Figures 15-17:

The redis driver developed by the data distribution module 30 of this system mainly has three threads, which are the thread for maintaining the heartbeat of consumers, as shown in Figure 15; the thread for data recovery, as shown in Figure 16; the normal consumption thread, as shown in Figure 17 .

In the normal consumption process, first check whether there is a PEL (PendingEntries List, pending entry list) of the dead consumer that needs to be restored. If there is, consume from the starting position of the PEL. If not, consume from the latest position of the PEL. . Each time a batch of in-vehicle data is read from PEL, the size of the batch can be controlled by parameters. After getting this batch of in-vehicle data, it starts to distribute the in-vehicle data. The thread pool is used to distribute the in-vehicle data, which is executed concurrently to speed up consumption. After the distribution message thread executes the callback function ack, it confirms to PEL that the on-board data processing is complete, and PEL removes it from the list.

If the consumer is dead, it will go through the recovery process, and the unprocessed PEL of the consumer will be allocated to other consumers to continue consumption. If the dead consumer loses the heartbeat timeout, the consumer will be cleaned up, and another consumer will be regenerated for data distribution.

Referring to FIG. 18 , FIG. 18 is a schematic structural diagram of another embodiment of a vehicle-mounted data distribution system provided by the present application. The in-vehicle data distribution system 100 includes: a data interface module 10 , a message queue module 20 , a data distribution module 30 and a remote configuration module 40 .

The remote configuration module 40 is used for remote system configuration of the data distribution system 100 .

In some embodiments, the remote configuration module 40 may exist in the form of a web page. The user performs system configuration of the data distribution system 100 by logging into the web page. In this way, there is no need for the user to directly operate the data distribution system 100 , the impact on the data distribution system 100 is reduced, and the convenience of system configuration is improved. Wherein, the remote configuration module 40 may adopt nacos configuration. Then monitor the update of the nacos configuration, and remotely control the service configuration update, etc.

To sum up, any of the above-mentioned embodiments provided in this application can solve the problem of distributing multi-vehicle data source data to different platforms, in a many-to-many relationship, and in real-time transmission, transparent to users, and supporting a variety of different types of service interfaces. It needs to be configured remotely at will, and it will take effect immediately after the configuration, and the expansion is convenient and fast, and so on.

The data distribution system 100 adopts a separate structure of the data interface module 10 and the data distribution module 30. The data interface module 10 can monitor the multi-node deployment of input and output flow information according to the message queue, and adopts dynamic routing at the same time. Identify different data sources for easy maintenance. The data distribution module 30 can monitor the multi-node deployment of input and output flow information according to the message queue, which may change frequently according to business requirements. The use of a separate buffer structure with the message queue module 20 can avoid the failure of the data source to send data due to the change of the data distribution module 30 and restarting the service. The situation that leads to data loss can improve data security and reduce the difficulty of platform maintenance.

In some embodiments, the data distribution system 100 can use the newly added stream message queue of redis v5.0 and above, which provides message persistence and master-replicated functions, allowing any client to access data at any time, And remember the access location of each client, and also ensure that data is not lost. Its writing speed is very fast, and it is very suitable for the middleware of this real-time distribution system.

At the same time, the data distribution system 100 also adds traffic statistics information such as input and output, combined with Prometheus collection and grafana visual display, to monitor the data distribution system 100 in real time.

Referring to FIG. 19 , FIG. 19 is a schematic structural diagram of an embodiment of an in-vehicle data distribution apparatus provided by the present application. The distribution device 190 includes a processor 191 and a memory 192 coupled to the processor 191; wherein, the memory 192 is used to store computer programs, and the processor 191 is used to execute the computer programs to implement the following methods:

Acquire on-board data returned by at least one on-board system; cache the on-board data in a message queue; obtain at least part of the on-board data from the front end of the message queue, and distribute to the corresponding vehicle management platform according to the type of at least part of the on-board data.

It can be understood that the processor 191 is further configured to execute a computer program to implement the method of any of the above embodiments.

Referring to FIG. 20, FIG. 20 is a schematic structural diagram of an embodiment of a computer-readable storage medium provided by the present application. The computer-readable storage medium 200 is used to store a computer program 201, and when the computer program 201 is executed by the processor, is used to implement the following methods:

Acquire on-board data returned by at least one on-board system; cache the on-board data in a message queue; obtain at least part of the on-board data from the front end of the message queue, and distribute to the corresponding vehicle management platform according to the type of at least part of the on-board data.

It can be understood that, when the computer program 201 is executed by the processor, it is also used to implement the method of any of the foregoing embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed method and device may be implemented in other manners. For example, the device implementations described above are only illustrative. For example, the division of the circuits or units is only a logical function division. In actual implementation, there may be other divisions. For example, multiple units or components may be Incorporation may either be integrated into another system, or some features may be omitted, or not implemented.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this implementation manner.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The above description is only an embodiment of the present application, and is not intended to limit the scope of the patent of the present application. Any equivalent structure or equivalent process transformation made according to the contents of the description and drawings of the present application, or directly or indirectly applied to other related technologies Fields are similarly included within the scope of patent protection of this application.

Claims (14)

1. A method for distributing vehicle-mounted data, the method comprising:

acquiring vehicle-mounted data returned by at least one vehicle-mounted system;

caching the vehicle-mounted data in a message queue;

and acquiring at least part of vehicle-mounted data from the front end of the message queue, and distributing the vehicle-mounted data to a corresponding vehicle management platform according to the type of the at least part of vehicle-mounted data.

2. The method of claim 1, wherein prior to buffering the in-vehicle data in a message queue, the method comprises:

packaging the vehicle-mounted data according to a preset field;

the caching of the vehicle-mounted data in a message queue comprises the following steps:

and buffering the packaged vehicle-mounted data in the message queue.

3. The method according to claim 2, wherein the encapsulating the vehicle data according to the preset field comprises:

And encapsulating the vehicle-mounted data according to a context field, an event type field, a metadata field, a priority field and a payload field.

4. The method of claim 1, wherein the obtaining at least a portion of the vehicle data from the front end of the message queue comprises:

and monitoring the vehicle-mounted data of the message queue, and acquiring at least part of the vehicle-mounted data from the front end of the message queue when the vehicle-mounted data is monitored.

5. The method of claim 1, wherein prior to buffering the vehicle data in a message queue, comprising:

packaging the vehicle-mounted data according to the event type field;

the step of acquiring at least part of vehicle-mounted data from the front end of the message queue and distributing the vehicle-mounted data to the corresponding vehicle management platform according to the type of the at least part of vehicle-mounted data comprises the following steps:

determining whether the event type of target vehicle-mounted data in the at least part of vehicle-mounted data is matched with a preset event type of the vehicle management platform;

if the matching is carried out, carrying out data packaging on the target vehicle-mounted data again, and distributing the data to the corresponding vehicle management platform;

And if not, discarding the target vehicle-mounted data.

6. The method according to claim 5, wherein the step of data packaging the target vehicle-mounted data again and distributing the target vehicle-mounted data to the corresponding vehicle management platform comprises the following steps:

filtering the at least part of the packaged vehicle-mounted data;

and distributing the at least part of the filtered vehicle-mounted data to the corresponding vehicle management platform.

7. The method of claim 6, wherein the distributing the at least partial filtered in-vehicle data to the corresponding vehicle management platform comprises:

judging whether the data format of the at least part of the vehicle-mounted data meets the requirement or not;

if not, the data format of at least part of the vehicle-mounted data is converted into the data format of the vehicle management platform, and then the data format is distributed to the vehicle management platform.

8. The method of claim 1, further comprising:

determining a publisher and a publisher state corresponding to the vehicle management platform;

and if the publisher state is in an activated state, publishing the at least part of the vehicle-mounted data to the vehicle management platform through the publisher.

9. The method of claim 8, further comprising:

and when at least part of the vehicle-mounted data is successfully published to the vehicle management platform, resetting the publisher state and sending feedback information to the message queue.

10. The method of claim 8, further comprising:

and when the at least part of vehicle-mounted data is failed to be published to the vehicle management platform, counting the continuous failure times of the publisher, and performing fusing processing based on the continuous failure times.

11. The method of claim 1, further comprising:

establishing a list of items to be processed according to at least part of the vehicle-mounted data;

checking whether a to-be-processed item list of the consumer needing to be recovered exists;

if yes, consuming from the starting position of the list of the items to be processed;

if not, consuming from the unconsumed location of the pending entry list.

12. An in-vehicle data distribution system characterized by comprising:

the data interface module is used for acquiring vehicle-mounted data returned by at least one vehicle-mounted system;

the message queue module is connected with the data interface module and used for caching the vehicle-mounted data by using a message queue;

And the data distribution module is connected with the message queue module and used for acquiring at least part of vehicle-mounted data from the front end of the message queue and distributing the vehicle-mounted data to the corresponding vehicle management platform according to the type of the at least part of vehicle-mounted data.

13. The distribution device of the vehicle-mounted data is characterized by comprising a processor and a memory coupled with the processor; wherein the memory is adapted to store a computer program and the processor is adapted to execute the computer program to implement the method according to any of claims 1-11.

14. A computer-readable storage medium, characterized in that the computer-readable storage medium is used to store a computer program which, when being executed by a processor, is used to carry out the method according to any one of claims 1-11.

Images