CN117194329A - Vehicle driving data storage access method, device and equipment - Google Patents
Vehicle driving data storage access method, device and equipment Download PDFInfo
- Publication number
- CN117194329A CN117194329A CN202311060058.0A CN202311060058A CN117194329A CN 117194329 A CN117194329 A CN 117194329A CN 202311060058 A CN202311060058 A CN 202311060058A CN 117194329 A CN117194329 A CN 117194329A
- Authority
- CN
- China
- Prior art keywords
- data
- vehicle
- index
- theme
- stored
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013500 data storage Methods 0.000 title claims abstract description 41
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000004590 computer program Methods 0.000 claims description 7
- 230000007774 longterm Effects 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 13
- 238000004458 analytical method Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 229910052799 carbon Inorganic materials 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000000547 structure data Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a vehicle driving data storage access method, device and equipment, and relates to the technical field of computers. The method comprises the steps of firstly constructing a theme file which corresponds to each vehicle data theme and comprises a theme index, a plurality of first vehicle indexes and data records, and then storing the vehicle data in a chained mode through the cooperation of the first vehicle indexes and the subsequent pointers in the data records when the vehicle data uploaded by the vehicle are received. And when the stored data meets the preset merging condition, merging the stored data into a sequential storage structure by a chain storage structure. When a data reading request carrying query information is received, the non-combined data is subjected to chained reading, and the combined data is sequentially read. According to the application, the near-term data is stored in the virtual chain type data structure, and the long-term data is stored in the sequential structure, so that the read speed of the long-term data and the near-term data is consistent, and the access speed of the archived data is improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method, an apparatus, and a device for storing and accessing vehicle driving data.
Background
At present, along with the guidance of the standard of carbon emission, the vehicle needs to meet the standard supervision requirement, so that the vehicle needs to upload the data signal quantity acquired by the electronic element to the supervision platform in real time for the reservation supervision. In high frequency vehicle travel data uploading, the flow rate received by the supervisory platform per second is up to tens of megabits, and for platform developers, important attention is paid to the stability and subsequent usability of the data. And after a period of time when the platform receives a large amount of data, the data needs to be archived, and after archiving, if a supervision department needs to call the specified data of a certain period of time of a certain vehicle, it is a difficult task.
In the prior art, aiming at high-frequency reporting of vehicle driving data, for data from a few days to a few months, a general processing means of a platform is to build a relational data storage cluster, such as a pg cluster, and to perform structured split storage on the data so as to facilitate data analysis operation of a foreground service on the vehicle. With the expansion of data volume, the traditional relational data storage mode cannot support the development of business, and a general data background can periodically archive and upload data which is outdated for several months into a large data cluster for backup, and at the moment, the archived data can be deleted from the relational cluster. In the case that the foreground service needs to archive data, the foreground service usually needs to archive large-scale data in a large data cluster to read and filter out the needed data, and in this case, if the service similar to the generation of the report form of the historical operation condition is very difficult to meet, a few days are required, and a few months are required to carry out full-disk scanning and filtering on the historical data.
In summary, for the current vehicle running data storage access, the access to the archived vehicle running data is difficult, and the access efficiency is low.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a vehicle travel data storage access method, apparatus, and device.
The technical scheme adopted in the specification is as follows:
the specification provides a vehicle running data storage access method, which comprises the following steps:
configuring a plurality of theme files according to the data theme of the vehicle driving data, wherein the theme files comprise theme indexes, a plurality of first vehicle indexes and data records, and the data records at least comprise subsequent indexes;
when vehicle running data uploaded by a vehicle is received, a subject index is acquired from a subject file corresponding to the uploaded data, the vehicle running data is stored to an address pointed by the subject index, a subsequent index in a data record in the address pointed by the corresponding first vehicle index is acquired according to a vehicle identification code, the subsequent index is pointed to a storage address of the uploaded data, and the subject index and the corresponding first vehicle index are updated;
when the stored data meet preset merging conditions, the stored data of each vehicle in each theme file are read in a chained mode according to the first vehicle index and the subsequent index in the data record, and the stored data are stored and integrated in a sequential writing mode to generate a second vehicle index;
when a data reading request carrying query information is received, a corresponding first vehicle index is acquired according to the query information, chain reading is performed according to the first vehicle index and a subsequent index in the data record, and/or sequential reading is performed according to a second vehicle index acquired by the query information.
Optionally, the first vehicle index specifically includes:
a head index pointing to a storage address of data initially uploaded by a vehicle in a corresponding subject file, and
a tail index pointing to the storage address of the most recently uploaded data in the corresponding subject file for the vehicle.
Optionally, the updating the theme index and the corresponding first vehicle index specifically includes:
self-increasing offset is carried out on the topic index;
and leading the tail index in the corresponding first vehicle index to point to the storage address of the current uploading data.
Optionally, the method further comprises:
for each theme file, when the vehicle driving data of the data theme uploaded by the vehicle is received for the first time, the subsequent index in the data record corresponding to the uploaded data points to a null value, the data record corresponding to the uploaded data is stored to the address pointed by the theme index,
and constructing a first vehicle index corresponding to the vehicle identification code, enabling a head index and a tail index in the corresponding first vehicle index to point to a storage address of the current uploaded data, and performing self-increasing offset on the subject index.
Optionally, the preset merging conditions specifically include:
the stored data is stored for a period of time longer than a preset merge period, or,
the sum of the data records of the stored data is greater than a preset data amount threshold.
Optionally, the query information at least comprises a vehicle identification code, a query theme and a query time;
acquiring a corresponding first vehicle index according to the query information, and performing chain reading according to the first vehicle index and a subsequent index in the data record, and/or performing sequential reading according to a second vehicle index acquired by the query information, wherein the method specifically comprises the following steps:
according to the query time, determining one type of data which does not meet the combination condition and is not combined from the data to be read, and determining two types of data which meet the combination condition and are combined;
aiming at one type of data, acquiring a corresponding first vehicle index according to a query subject and a vehicle identification code, and performing chain reading according to the first vehicle index and a subsequent index in a data record;
and aiming at the class II data, acquiring a corresponding second vehicle index according to the query subject and the vehicle identification code, and sequentially reading according to the second vehicle index.
Optionally, the acquiring the corresponding first vehicle index according to the query subject and the vehicle identification code, and performing chain reading according to the first vehicle index and a subsequent index in the data record specifically includes:
determining a target theme file corresponding to one type of data according to the query theme;
acquiring a corresponding first vehicle index from the target theme file according to the vehicle identification code;
determining a target storage address of the first uploading data in the target subject file in one type of data according to the head index in the first vehicle index, and reading a data record of the first uploading data from the target storage address;
and determining the storage address of the data uploaded again in the target subject file in one type of data according to the subsequent index in the data record of the first uploaded data, reading, and repeating the chained reading through a plurality of rounds.
Optionally, the data record specifically includes:
vehicle identification code, data upload time, data value length, data value, and subsequent index.
The present specification provides a vehicle running data storage access apparatus including:
the configuration module is used for configuring a plurality of theme files according to the data theme of the vehicle driving data, wherein the theme files comprise theme indexes, a plurality of first vehicle indexes and data records, and the data records at least comprise subsequent indexes;
the storage module is used for acquiring a theme index from a theme file corresponding to the uploaded data when the vehicle running data uploaded by the vehicle is received, storing the vehicle running data to an address pointed by the theme index, acquiring a subsequent index in a data record in the address pointed by the corresponding first vehicle index according to the vehicle identification code, enabling the subsequent index to point to a storage address of the uploaded data, and updating the theme index and the corresponding first vehicle index;
the merging module is used for reading the stored data to be merged in a chained mode according to the first vehicle index and the subsequent index in the data record aiming at the stored data of each vehicle in each theme file when the stored data meet the preset merging condition, and storing and integrating the stored data in a sequential writing mode to generate a second vehicle index;
and the reading module is used for acquiring the corresponding first vehicle index according to the query information when receiving the data reading request carrying the query information, performing chain reading according to the first vehicle index and the subsequent index in the data record, and/or performing sequential reading according to the second vehicle index acquired by the query information.
The present specification provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the above-described vehicle running data storage access method.
The present specification provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the above method of accessing vehicle travel data storage when executing the program.
The above-mentioned at least one technical scheme that this specification adopted can reach following beneficial effect:
firstly, constructing a theme file which corresponds to each vehicle driving data theme and comprises a theme index, a plurality of first vehicle indexes and data records, then when vehicle driving data uploaded by a vehicle is received, storing the data to an address pointed by the theme index in the corresponding theme file, acquiring a subsequent index in the data records in the address pointed by the corresponding first vehicle index according to a vehicle identification code, leading the subsequent index to point to a storage address of the uploaded data, and updating the theme index and the corresponding first vehicle index.
When the stored data meet preset merging conditions, the stored data of each vehicle in each theme file are read in a chained mode according to the first vehicle index and the subsequent index in the data record, and the stored data are stored and integrated in a sequential writing mode to generate a second vehicle index.
When a data reading request carrying query information is received, a corresponding first vehicle index is acquired according to the query information, chain reading is performed according to the first vehicle index and a subsequent index in the data record, and/or sequential reading is performed according to a second vehicle index acquired by the query information.
According to the application, the recent data is stored in the virtual chain type data structure, so that the data can be sequentially read according to the virtual chain type data structure and stored in the sequential structure when the data is archived, the extraction speed of the archived long-term data and the recent data is consistent, and the access speed and the access efficiency of the archived data are improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1 is a schematic diagram illustrating a comparison of an overall architecture provided in the present specification;
FIG. 2 is a schematic flow chart of a method for accessing vehicle driving data storage provided in the present specification;
FIG. 3 is a schematic diagram of a write engine architecture provided herein;
FIG. 4 is a schematic diagram of a merging engine architecture provided in the present specification;
FIG. 5 is a schematic diagram of a read engine architecture provided herein;
FIG. 6 is a schematic diagram of a vehicle travel data storage access device provided herein;
fig. 7 is a schematic diagram of a computer device for implementing a method for accessing vehicle driving data storage provided in the present specification.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the present specification more apparent, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments of the present specification and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art without the exercise of inventive faculty, are intended to be within the scope of the application, based on the embodiments in the specification.
At present, along with expansion of various data in society, more and more scenes need to call relevant data of a bicycle at historical moment for analysis (for example, analysis of thermal track of a few months and analysis of traffic flow of a limited area), at present, because on the premise that more network vehicles exist and data are reported at high frequency, a relational data cluster can only meet the requirement of storing for a short time, data which are more than a few months old can be backed up to a large data cluster, but after traditional data archiving, it is very difficult to call data of a certain bicycle for a designated time period when the application needs to call the data of all data or the data of a block archived on a designated date, and the pressure on server resources is high and the time cost is high.
Aiming at the characteristics of the bicycle data, the application provides a data collection and storage solution meeting the follow-up service development. Firstly, data of a single vehicle can be stored in a centralized and orderly chain structure according to a data dimension, then the data of the vehicle is archived according to a data dimension of a year or month, and the storage is integrated into a sequential structure storage. Therefore, the data can be rapidly positioned and extracted through the appointed inquiry identifier (generally referred to as the vehicle unique identifier VIN) +the appointed inquiry time, the large-scale data full-disc scanning and the resource consumption are avoided, and the subsequent rapid development of analysis business is well supported.
The application provides a vehicle driving data storage access method, which can be said to be a vehicle driving data storage engine, in particular to a data storage engine which is suitable for collecting data of vehicles according to data subjects, centralizing and gathering the data carrying reporting time through subject classification and time sequencing, and providing rapid reading and writing for the outside. The storage engine solves the problems caused by the traditional high-frequency vehicle driving data reporting relation type database and the data backup cluster mode.
Fig. 1 is a schematic diagram illustrating the overall architecture of the present disclosure. The left side in fig. 1 is a schematic diagram of the whole architecture at present, and the right side is a whole architecture of the storage access method provided by the application, as shown in fig. 1, a set of integrated writing, inquiring and archiving algorithm engine integrated service is constructed on a storage server by a storage engine in the application, and the whole architecture is composed of an inquiring engine, a writing engine, a merging engine and an archiving data record.
The following describes in detail the technical solutions provided by the embodiments of the present application with reference to the accompanying drawings.
Fig. 2 is a flow chart of a method for storing and accessing vehicle driving data in the present specification, specifically including the following steps:
s101: and configuring a plurality of theme files according to the data theme of the vehicle driving data, wherein the theme files comprise theme indexes, a plurality of first vehicle indexes and data records, and the data records at least comprise subsequent indexes.
In general, a vehicle is generally provided with various data collecting sensors to collect various vehicle driving data during driving of the vehicle, so in one or more embodiments of the present disclosure, a storage server of a service platform may be connected to a storage engine in the present disclosure, and a plurality of theme files are preconfigured according to data theme of the vehicle driving data.
The data topics may be classified according to types of vehicle driving data, for example, vehicle speed signal data corresponds to a vehicle speed topic, and tire pressure signal data corresponds to a tire pressure topic. The specification is not limited in particular as to how this is categorized. The server may configure, for each data topic, a topic file corresponding to the data topic, where the topic file includes a topic index, a number of first vehicle indexes, and a data record.
Further, in one or more embodiments of the present disclosure, the first vehicle index may include a head index pointing to a storage address of the vehicle that was first uploaded with data in the corresponding subject file, and a tail index pointing to a storage address of the vehicle that was last uploaded with data in the corresponding subject file. The topic index points to the next storage unit of the storage units corresponding to the most recently stored data record in the topic file.
Still further, in one or more embodiments of the present description, the data record may include a vehicle identification code, a data upload time, a data value length, a data value, and a subsequent index. The vehicle identification code can occupy four bytes, the data uploading time occupies four bytes, the data value length occupies 2 bytes, the data value is determined according to the value of the data value length, and the four bytes are indexed subsequently.
The storage server mentioned in the present specification may be a storage server provided in a service platform, or a device such as a desktop, a notebook, or the like capable of executing the aspects of the present specification. For convenience of explanation, only the server is used as the execution subject.
S102: when vehicle running data uploaded by a vehicle is received, a subject index is acquired from a subject file corresponding to the uploaded data, the vehicle running data is stored to an address pointed by the subject index, a subsequent index in a data record in the address pointed by the corresponding first vehicle index is acquired according to a vehicle identification code, the subsequent index is pointed to a storage address of the uploaded data, and the subject index and the corresponding first vehicle index are updated.
Corresponding to the plurality of theme files configured as above, when the server receives the vehicle running data uploaded by the vehicle, the data can be split according to the data theme, such as vehicle speed, tire pressure and the like, and then the data carrying the theme information is sent to the storage engine, so that the writing engine in the storage engine can acquire the theme index from the corresponding theme file according to the theme information.
The topic index points to the next storage unit of the storage units corresponding to the most recently stored data records in the topic file, and then the server can store the vehicle driving data to the address pointed by the topic index.
If the theme file is the vehicle driving data of the data theme uploaded by the vehicle for the first time, the server can enable the subsequent index in the data record corresponding to the uploaded data to point to the null value, construct a first vehicle index corresponding to the vehicle identification code, enable the head index and the tail index in the corresponding first vehicle index to point to the storage address of the current uploaded data, and automatically increase and offset the theme index.
If the theme file is the vehicle driving data of the data theme uploaded by the vehicle, the server can enable the subsequent index in the data record corresponding to the uploaded data to point to the null value, acquire the subsequent index in the data record in the address pointed by the tail index in the corresponding first vehicle index according to the vehicle identification code, enable the subsequent index to point to the storage address of the uploaded data, enable the tail index in the corresponding first vehicle index to point to the storage address of the current uploaded data, and enable the theme index to perform self-increasing offset.
By the above, the corresponding data of each vehicle in each theme can be stored in a virtual chain data structure, as shown in fig. 3.
FIG. 3 is a schematic diagram of a write engine architecture according to the present disclosure. As can be seen from fig. 3, from bottom to top in fig. 3, pointers and data records form a virtual chain data structure, where a topic write pointer cache, that is, a topic index, is illustrated by taking a topic-1 write pointer as an example, where the topic-1 write pointer always points to a data record available storage unit corresponding to a topic-1 topic, and after receiving reported vehicle running data, a server may directly store the vehicle running data in an address pointed by the corresponding topic index, for example, a data topic of the vehicle running data is topic-1, and then the vehicle running data may be directly stored in an address pointed by the topic-1 write pointer.
As can be seen from fig. 3, taking the topic-1 as an example for illustration, the first vehicle index corresponding to the topic-1 includes frame-1, frame-2, frame-3, and frame-N, and the numerals after each first vehicle index represent the addresses pointed by the tail index in the first vehicle index, which is of course only a brief example. Corresponding to frame-1:1 in FIG. 3, the vehicle travel data storage address representing the last reported topic-1 data topic for the vehicle of frame-1 is "1". The other similar matters are not described in detail herein.
In the subject data block, each data record can include a next offset portion, which is a subsequent index in the data record, and is illustrated by taking "frame 1+next offset+length+message" as an example, where the next offset refers to a storage address of vehicle driving data of the next reported topic-1 data subject after the vehicle of frame-1 reports "frame 1+next offset+length+message" of the topic-1 data subject.
By the method, the storage data of each vehicle in each theme can be stored in a virtual chain type data structure according to the first vehicle index and the subsequent index in the data record.
In summary, each topic is composed of an index file+topic data block file, where the index file includes a topic index, a first vehicle index, and a topic block number. The working mechanism of the writing engine is that when the latest theme signal is reported, the latest theme information can be additionally written into the corresponding theme file for sequential writing, the position of the last reported signal of the current vehicle, which is written into the corresponding theme block, is obtained through the vehicle identification code, and after the position index is obtained, the address pointed by the subsequent index in the data record in the position index can be modified to be the address written in.
Wherein the server may convert the data into a binary data format in sequence for storage. The topic index and the first vehicle index may be created by day by default to avoid excessive index resulting in poor performance of subsequent queries. The topic data block size may default to java memory map maximum length 2G, exceeding 2G data size, the storage engine generates the next data block, and modifies topic block number increment in the topic index file, each topic data block may include multiple data records.
From the above, the vehicle does not involve the disc addressing process in the process of writing the subject data block, the signal data is sequentially written into the latest position of the disc pointer, the last data writing position is accurately obtained through the index file, the subsequent index is written, the latest writing data position is filled, and a virtual chain type data structure constructed by a single vehicle unique number is built inside.
S103: when the stored data meet preset merging conditions, the stored data of each vehicle in each theme file are read in a chained mode according to the first vehicle index and the subsequent index in the data record, and the stored data are stored and integrated in a sequential writing mode to generate a second vehicle index.
During the data storage process, the server can monitor the state of the stored data, and when the stored data meets the preset merging condition, the server can file the stored data of each vehicle in each theme file.
The core idea of the merging engine is to trigger a merging mechanism according to the storage time of stored data, the size of a data block (the data record quantity of stored data) and other condition thresholds, and to generate an image file of an uncombined block for preventing influence on user service in the merging process of the uncombined block, and to clean the original uncombined block after merging. The merging block is not only the accumulation of data, but also the sequential merging of bicycle data according to the information such as the appointed time dimension, the theme and the like, and in the actual service, the merging block is triggered to continue to merge large blocks according to the service configuration for the vehicle information data exceeding the service period, so that the influence of small files on a server is prevented, as shown in fig. 4.
Fig. 4 is a schematic diagram of a merging engine architecture in the present specification. As can be seen from fig. 4, the server may merge the unmixed blocks by the cooperation of the write engine and the read engine, resulting in a merged block. The data storage format is changed from a chained structure to a sequential structure.
When the stored data meets the preset merging condition, the server can read the stored data to be merged in a chained mode according to the first vehicle index and the subsequent index in the data record aiming at the stored data of each vehicle in each theme file, and store and integrate the stored data in a sequential writing mode to generate a second vehicle index, and the first vehicle index can be deleted.
Specifically, the inside of the merging engine can monitor the writing of the corresponding data blocks of each theme, and when the corresponding data blocks are written up, the background copies the corresponding data mirror images of the data blocks, so that the current reading service is prevented from being influenced. And then, the merging thread can read the virtual chained data according to the read-write flow and re-write the virtual chained data in sequence, merge a plurality of data images into a large data block, and finally, delete the data images after the plurality of image data are read, and mark the data read blocks as sequence data blocks.
S104: when a data reading request carrying query information is received, a corresponding first vehicle index is acquired according to the query information, chain reading is performed according to the first vehicle index and a subsequent index in the data record, and/or sequential reading is performed according to a second vehicle index acquired by the query information.
When the server receives a data reading request carrying query information, the server can call a reading engine to read data, wherein the query information can at least comprise a vehicle identification code, a query theme and a query time.
The server may determine, from the data to be read, one type of data that does not satisfy the merge condition and that has not been merged, and determine, from the data to be read, one type of data that satisfies the merge condition and that has been merged.
And then, aiming at one type of data, acquiring a corresponding first vehicle index according to the query subject and the vehicle identification code, and performing chain reading according to the first vehicle index and a subsequent index in the data record.
And aiming at the class II data, acquiring a corresponding second vehicle index according to the query subject and the vehicle identification code, and sequentially reading according to the second vehicle index.
Specifically, for one type of data, the server may determine, according to the query topic, a target topic file corresponding to the one type of data, then obtain, according to the vehicle identification code, a corresponding first vehicle index from the target topic file, then determine, according to a head index in the first vehicle index, a target storage address of the first uploaded data in the target topic file in the one type of data, and read a data record of the first uploaded data from the target storage address, and finally determine, according to a subsequent index in the data record of the first uploaded data, a storage address of the second uploaded data in the target topic file in the one type of data, and read the second uploaded data, and repeatedly perform chained reading through multiple rounds.
Fig. 5 is a schematic diagram of a read engine architecture in the present specification, as can be seen from fig. 5, a read engine can query a frame number (vehicle identification code), a query subject and a query time according to a vehicle to which a service is transferred, a read algorithm locates a starting offset of the frame number of a specified day according to a query condition, then transfers the data to a read thread, internally judges whether to read an uncombined block or a merged block or two types of data blocks to be acquired, if the data block is read, the internal read structure is a chained structure, thereby reducing the waste of read resources and the storage efficiency of invalid data, if the data block is read, the data is re-archived and stored intensively by the merging engine, the internal read structure is continuously stored, the read performance of the structural data is only dependent on the performance of a magnetic disk, and the queried data is summarized by the merged thread and finally returned to the service.
Taking the index file as an example of day creation, the data is stored and archived, the use scene is a single data theme+internal single vehicle chain type storage structure constructed based on a writing engine for inquiring random events according to the vehicle identification code, inquiring time and data theme, and the combined sequential storage structure. The reading engine can split the received query conditions according to the days, a plurality of query threads are built in the reading engine according to the split query information of the days, and two data file formats are related according to an internal merging strategy, wherein one is a bicycle virtual chain type storage format and the other is a bicycle sequential storage format.
If the reading algorithm judges that the storage structure corresponding to the split date is in a bicycle chain type storage format, the internal part firstly reads the theme index file generated according to the writing engine to acquire the first writing index address of the bicycle in the same day, the reading engine acquires the address, directly reads the corresponding data theme block file, and the read data is recorded as a vehicle identification code, reporting time, a data value length, a data value and a subsequent index. Then, the next data record can be sequentially acquired according to the subsequent index in the data records, and then the step is repeated to sequentially acquire the complete virtual chain structure data.
If the reading algorithm judges that the sequence storage format is adopted, the second vehicle index file generated according to the writing engine is firstly read in the interior, the reading engine acquires the address, the bicycle interval data are sequentially acquired, and finally the returned chain data and the sequential data can be subjected to the merging algorithm to carry out integral sequencing in time sequence in the interior and return to the user for use.
Based on the vehicle driving data storage access method shown in fig. 2, a theme file which corresponds to each vehicle driving data theme and comprises a theme index, a plurality of first vehicle indexes and data records is constructed, then when the vehicle driving data uploaded by a vehicle is received, the data is stored to an address pointed by the theme index in the corresponding theme file, a subsequent index in the data records in the address pointed by the corresponding first vehicle index is obtained according to the vehicle identification code, the subsequent index is pointed to a storage address of the uploaded data, and the theme index and the corresponding first vehicle index are updated. When the stored data meet preset merging conditions, the stored data of each vehicle in each theme file are read in a chained mode according to the first vehicle index and the subsequent index in the data record, and the stored data are stored and integrated in a sequential writing mode to generate a second vehicle index. When a data reading request carrying query information is received, a corresponding first vehicle index is acquired according to the query information, chain reading is performed according to the first vehicle index and a subsequent index in the data record, and/or sequential reading is performed according to a second vehicle index acquired by the query information.
According to the application, the recent data is stored in the virtual chain type data structure, so that the data can be sequentially read according to the virtual chain type data structure and stored in the sequential structure when the data is archived, the extraction speed of the archived long-term data and the recent data is consistent, and the access speed and the access efficiency of the archived data are improved.
When the vehicle running data storage access method provided in the present specification is applied, the steps may be executed in a different order from the order of the steps shown in fig. 2, and the specific order of execution of the steps may be determined as needed, which is not limited in the present specification.
The above provides a vehicle running data storage access method for one or more embodiments of the present disclosure, and based on the same concept, the present disclosure further provides a corresponding vehicle running data storage access device, as shown in fig. 6.
Fig. 6 is a schematic diagram of a vehicle driving data storage access device provided in the present specification, including:
the configuration module 201 is configured to configure a plurality of theme files according to a data theme of the vehicle driving data, where the theme files include a theme index, a plurality of first vehicle indexes, and a data record, and the data record includes at least a subsequent index;
the storage module 202 is configured to, when receiving vehicle running data uploaded by a vehicle, obtain a topic index from a topic file corresponding to the uploaded data, store the vehicle running data to an address pointed by the topic index, and obtain a subsequent index in a data record in the address pointed by the corresponding first vehicle index according to a vehicle identification code, so that the subsequent index points to a storage address of the uploaded data, and update the topic index and the corresponding first vehicle index;
the merging module 203 is configured to, when the stored data meets a preset merging condition, read the stored data to be merged in a chained manner according to the first vehicle index and a subsequent index in the data record for the stored data of each vehicle in each theme file, and store and integrate the stored data in a sequential writing manner to generate a second vehicle index;
the reading module 204 is configured to, when receiving a data reading request carrying query information, obtain a corresponding first vehicle index according to the query information, perform chain reading according to the first vehicle index and a subsequent index in the data record, and/or perform sequential reading according to a second vehicle index obtained by the query information.
Optionally, the first vehicle index includes: a head index pointing to a storage address of the data that the vehicle has first uploaded in the corresponding subject file, and a tail index pointing to a storage address of the data that the vehicle has recently uploaded in the corresponding subject file.
Optionally, the storage module 202 performs self-increasing offset on the subject index, so that the tail index in the corresponding first vehicle index points to the storage address of the current uploaded data.
Optionally, for each theme file, when the vehicle running data of the data theme uploaded by the vehicle is received for the first time, the storage module 202 directs the subsequent index in the data record corresponding to the uploaded data to the null value, stores the data record corresponding to the uploaded data to the address pointed by the theme index, constructs the first vehicle index corresponding to the vehicle identification code, directs the head index and the tail index in the corresponding first vehicle index to the storage address of the current uploaded data, and performs self-increasing offset on the theme index.
Optionally, the preset combining condition includes: the storage time period of the stored data is longer than the preset merging time period, or the sum of the data records of the stored data is larger than the preset data quantity threshold value.
Optionally, the query information at least includes a vehicle identification code, a query topic, and a query time.
The reading module 204 determines, according to the query time, a type of data that does not satisfy the merging condition and is not merged from the data to be read, and determines a type of data that satisfies the merging condition and is merged, for the type of data, acquires a corresponding first vehicle index according to the query subject and the vehicle identification code, and performs chain reading according to the first vehicle index and a subsequent index in the data record, and for the type of data, acquires a corresponding second vehicle index according to the query subject and the vehicle identification code, and sequentially reads according to the second vehicle index.
Optionally, the reading module 204 determines, according to the query subject, a target subject file corresponding to the first type of data, obtains a corresponding first vehicle index from the target subject file according to the vehicle identification code, determines, according to a head index in the first vehicle index, a target storage address of the first type of data in the target subject file, where the first type of data is uploaded, reads a data record of the first type of data from the target storage address, determines, according to a subsequent index in the data record of the first type of data, a storage address of the first type of data in the second type of data, where the second type of data is uploaded again, reads the second type of data, and performs chain reading through multiple rounds of repetition.
Optionally, the data record includes: vehicle identification code, data upload time, data value length, data value, and subsequent index.
The specific limitation regarding the vehicle travel data storage access means may be referred to the limitation regarding the vehicle travel data storage access method hereinabove, and will not be described in detail herein. The respective modules in the above-described vehicle running data storage access apparatus may be implemented in whole or in part by software, hardware, and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
The present specification also provides a computer-readable storage medium storing a computer program operable to execute the vehicle travel data storage access method provided in fig. 2 described above.
The present specification also provides a schematic structural diagram of the computer device shown in fig. 7, where, as shown in fig. 7, the computer device includes a processor, an internal bus, a network interface, a memory, and a nonvolatile memory, and may include hardware required by other services. The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to realize the method for accessing the vehicle running data storage provided by the fig. 2.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, or the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static RandomAccess Memory, SRAM) or dynamic random access memory (Dynamic RandomAccess Memory, DRAM), and the like.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
Claims (10)
1. A vehicle travel data storage access method, characterized by comprising:
configuring a plurality of theme files according to the data theme of the vehicle driving data, wherein the theme files comprise theme indexes, a plurality of first vehicle indexes and data records, and the data records at least comprise subsequent indexes;
when vehicle running data uploaded by a vehicle is received, a subject index is acquired from a subject file corresponding to the uploaded data, the vehicle running data is stored to an address pointed by the subject index, a subsequent index in a data record in the address pointed by the corresponding first vehicle index is acquired according to a vehicle identification code, the subsequent index is pointed to a storage address of the uploaded data, and the subject index and the corresponding first vehicle index are updated;
when the stored data meet preset merging conditions, the stored data of each vehicle in each theme file are read in a chained mode according to the first vehicle index and the subsequent index in the data record, and the stored data are stored and integrated in a sequential writing mode to generate a second vehicle index;
when a data reading request carrying query information is received, a corresponding first vehicle index is acquired according to the query information, chain reading is performed according to the first vehicle index and a subsequent index in the data record, and/or sequential reading is performed according to a second vehicle index acquired by the query information.
2. The vehicle travel data storage access method of claim 1, wherein the first vehicle index specifically includes:
a head index pointing to a storage address of data initially uploaded by a vehicle in a corresponding subject file, and
a tail index pointing to the storage address of the most recently uploaded data in the corresponding subject file for the vehicle.
3. The vehicle travel data storage access method of claim 2, wherein updating the subject index and the corresponding first vehicle index specifically comprises:
self-increasing offset is carried out on the topic index;
and leading the tail index in the corresponding first vehicle index to point to the storage address of the current uploading data.
4. The vehicle travel data storage access method according to claim 2, characterized by further comprising:
for each theme file, when the vehicle driving data of the data theme uploaded by the vehicle is received for the first time, the subsequent index in the data record corresponding to the uploaded data points to a null value, the data record corresponding to the uploaded data is stored to the address pointed by the theme index,
and constructing a first vehicle index corresponding to the vehicle identification code, enabling a head index and a tail index in the corresponding first vehicle index to point to a storage address of the current uploaded data, and performing self-increasing offset on the subject index.
5. The vehicle travel data storage access method according to claim 1, wherein the preset merge condition specifically includes:
the stored data is stored for a period of time longer than a preset merge period, or,
the sum of the data records of the stored data is greater than a preset data amount threshold.
6. The vehicle travel data storage access method according to claim 1, wherein the inquiry information includes at least a vehicle identification code, an inquiry topic, an inquiry time;
acquiring a corresponding first vehicle index according to the query information, and performing chain reading according to the first vehicle index and a subsequent index in the data record, and/or performing sequential reading according to a second vehicle index acquired by the query information, wherein the method specifically comprises the following steps:
according to the query time, determining one type of data which does not meet the combination condition and is not combined from the data to be read, and determining two types of data which meet the combination condition and are combined;
aiming at one type of data, acquiring a corresponding first vehicle index according to a query subject and a vehicle identification code, and performing chain reading according to the first vehicle index and a subsequent index in a data record;
and aiming at the class II data, acquiring a corresponding second vehicle index according to the query subject and the vehicle identification code, and sequentially reading according to the second vehicle index.
7. The method for accessing the vehicle running data storage according to claim 6, wherein the acquiring the corresponding first vehicle index according to the query subject and the vehicle identification code, and the chain reading according to the first vehicle index and the subsequent index in the data record, specifically comprises:
determining a target theme file corresponding to one type of data according to the query theme;
acquiring a corresponding first vehicle index from the target theme file according to the vehicle identification code;
determining a target storage address of the first uploading data in the target subject file in one type of data according to the head index in the first vehicle index, and reading a data record of the first uploading data from the target storage address;
and determining the storage address of the data uploaded again in the target subject file in one type of data according to the subsequent index in the data record of the first uploaded data, reading, and repeating the chained reading through a plurality of rounds.
8. The vehicle travel data storage access method of claim 1, wherein the data record specifically comprises:
vehicle identification code, data upload time, data value length, data value, and subsequent index.
9. A vehicle travel data storage access device characterized by comprising:
the configuration module is used for configuring a plurality of theme files according to the data theme of the vehicle driving data, wherein the theme files comprise theme indexes, a plurality of first vehicle indexes and data records, and the data records at least comprise subsequent indexes;
the storage module is used for acquiring a theme index from a theme file corresponding to the uploaded data when the vehicle running data uploaded by the vehicle is received, storing the vehicle running data to an address pointed by the theme index, acquiring a subsequent index in a data record in the address pointed by the corresponding first vehicle index according to the vehicle identification code, enabling the subsequent index to point to a storage address of the uploaded data, and updating the theme index and the corresponding first vehicle index;
the merging module is used for reading the stored data to be merged in a chained mode according to the first vehicle index and the subsequent index in the data record aiming at the stored data of each vehicle in each theme file when the stored data meet the preset merging condition, and storing and integrating the stored data in a sequential writing mode to generate a second vehicle index;
and the reading module is used for acquiring the corresponding first vehicle index according to the query information when receiving the data reading request carrying the query information, performing chain reading according to the first vehicle index and the subsequent index in the data record, and/or performing sequential reading according to the second vehicle index acquired by the query information.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the vehicle travel data storage access method of any one of the preceding claims 1 to 8 when the program is executed.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202311060058.0A CN117194329A (en) | 2023-08-22 | 2023-08-22 | Vehicle driving data storage access method, device and equipment |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202311060058.0A CN117194329A (en) | 2023-08-22 | 2023-08-22 | Vehicle driving data storage access method, device and equipment |
Publications (1)
Publication Number | Publication Date |
---|---|
CN117194329A true CN117194329A (en) | 2023-12-08 |
Family
ID=89002620
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202311060058.0A Pending CN117194329A (en) | 2023-08-22 | 2023-08-22 | Vehicle driving data storage access method, device and equipment |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN117194329A (en) |
-
2023
- 2023-08-22 CN CN202311060058.0A patent/CN117194329A/en active Pending
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN106951375B (en) | Method and device for deleting snapshot volume in storage system | |
CN104462563B (en) | A kind of file memory method and system | |
US6182082B1 (en) | Method and system for managing object-oriented database | |
CN108984686B (en) | Distributed file system indexing method and device based on log merging | |
CN112131140B (en) | SSD-based key value separation storage method supporting efficient storage space management | |
CN113568582B (en) | Data management method, device and storage equipment | |
CN113515487A (en) | Directory query method, computing device and distributed file system | |
CN110287201A (en) | Data access method, device, equipment and storage medium | |
CN111639072A (en) | Data storage method and system in Internet of vehicles scene and readable storage medium | |
CN111125171A (en) | Monitoring data access method, device, equipment and readable storage medium | |
CN112558873A (en) | Embedded audio and video data storage, retrieval and deletion method and device and memory | |
CN113448946B (en) | Data migration method and device and electronic equipment | |
CN114153395B (en) | Object storage data life cycle management method, device and equipment | |
US10769105B2 (en) | Modifying Lucene index file | |
CN109189724B (en) | Method and device for improving audio and video data storage efficiency of video monitoring system | |
CN111752941B (en) | Data storage and access method and device, server and storage medium | |
CN116955278A (en) | Aggregation access method and device for distributed file system snapshot and computer equipment | |
JP3896077B2 (en) | Computer system and file management method | |
CN117194329A (en) | Vehicle driving data storage access method, device and equipment | |
CN110866068A (en) | Announcement data storage method and device based on HDFS | |
JP4127461B2 (en) | Backup system and method in disk shared file system | |
CN115858471A (en) | Service data change recording method, device, computer equipment and medium | |
CN115328859A (en) | Metadata access method, device, equipment and medium | |
CN114185849A (en) | File operation method, file operation system, electronic device and storage medium | |
CN116028572A (en) | Communication service data processing method, device and computer storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication |