CN115277843B - Method and system for merging frequency division data of vehicle network - Google Patents
Method and system for merging frequency division data of vehicle network Download PDFInfo
- Publication number
- CN115277843B CN115277843B CN202210772930.3A CN202210772930A CN115277843B CN 115277843 B CN115277843 B CN 115277843B CN 202210772930 A CN202210772930 A CN 202210772930A CN 115277843 B CN115277843 B CN 115277843B
- Authority
- CN
- China
- Prior art keywords
- data
- merging
- period
- frequency
- batch
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000012163 sequencing technique Methods 0.000 claims abstract description 11
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 238000007596 consolidation process Methods 0.000 claims 1
- 238000004590 computer program Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000001174 ascending effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
- G06F16/244—Grouping and aggregation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Medical Informatics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a method and a system for merging frequency division data of a vehicle network, and relates to the technical field of vehicle networks; sequencing the data based on the data generation time, taking out the data in batches, and dividing each batch of the data taken out according to the period by taking the data merging reference frequency as the period; the data with earliest time generated in each period is used as merging reference data, and the data in the period are merged into the merging reference data to generate a merging result; the data type in each period, the original data of each frequency data corresponding to the latest data, and the combined number of each frequency data are recorded. The invention can effectively reduce the problem of data accuracy caused by data loss.
Description
Technical Field
The invention relates to the technical field of vehicle networks, in particular to a method and a system for merging frequency division data of a vehicle network.
Background
With the rapid development of 5G (fifth generation mobile communication technology) mobile internet technology, most of vehicles are loaded with an in-vehicle network, a vehicle-mounted mobile internet and an inter-vehicle network, and a large amount of information is collected, processed and shared through the vehicles, so that the connection between the vehicles and people, the connection between the vehicles and roads and the connection between the vehicles and cloud platforms are realized, and intelligent dynamic information service, intelligent vehicle control and intelligent traffic management are realized.
In the internet of vehicles service, the vehicle terminal continuously uploads vehicle data of the vehicle in any state, including GPS (Global Positioning System ) data, CAN (Controller Area Network, controller area network) data, tbox (internet of vehicles system) data, and the like. The data of various types can be uploaded according to different Hertz frequencies according to different vehicle conditions, different equipment, different use environments and the like. For uploaded data, in a specific analysis process, data with different types and different uploading frequencies are usually required to be put together to analyze and display the data at the same time, so that the data needs to be combined first. However, the difference of the mobile network signals can cause delayed uploading of data and loss of partial data with small probability under different environments of the vehicle, so that a better vehicle data merging strategy is needed currently.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a method and a system for merging frequency division data of a vehicle network, which can effectively reduce the problem of data accuracy caused by data loss.
In order to achieve the above purpose, the invention provides a method for merging frequency division data of a vehicle network, which specifically comprises the following steps:
determining a data type range and a time range of data to be combined, and determining a data combination reference frequency;
sequencing the data based on the data generation time, taking out the data in batches, and dividing each batch of the data taken out according to the period by taking the data merging reference frequency as the period;
the data with earliest time generated in each period is used as merging reference data, and the data in the period are merged into the merging reference data to generate a merging result;
the data type in each period, the original data of each frequency data corresponding to the latest data, and the combined number of each frequency data are recorded.
On the basis of the technical scheme, the data with the earliest time generated in each period is used as the merging reference data, the data in the period is merged into the merging reference data, and a merging result is generated, wherein when the data in a new period is merged:
judging whether data missing exists in the current period of merging or not:
if yes, according to the type and the frequency of the missing data, finding corresponding data from the original data of the latest data corresponding to each frequency data recorded in the previous period, and merging the found data into the current period when the time difference of the data meets a time threshold;
if not, the processing is not performed.
On the basis of the technical proposal, the method comprises the following steps,
when the mttTime is more than or equal to |labstDataTime-rmdTime| is satisfied, the time difference of the data satisfies a time threshold value, otherwise, the data does not satisfy the time threshold value;
where mttTime denotes a time threshold, latestDataTime denotes data found from the latest data of each frequency data recorded in the previous cycle, rmdTime denotes the generation time of the merged reference data in the current cycle.
On the basis of the technical scheme, when corresponding data is not found from the original data of the latest data corresponding to each frequency data recorded in the previous period, or the time difference of the found data does not meet the time threshold, the data missing part of the current period for merging is set to be empty.
On the basis of the technical proposal, the method comprises the following steps,
after the fetched batch of data is combined according to the period, marking the combination result of the last period of the batch of data as latestMr;
when the new batch of data is taken out for merging, judging whether the first data of the current batch of data and the latestMr of the previous batch of data are located in the same period, if so, taking the latestMr as merging reference data of the first period in the current batch of data, and if not, taking the data with earliest time generated in the first period in the current batch of data as merging reference data.
On the basis of the technical proposal, the method comprises the following steps,
when T is less than or equal to |head DataTime-latestMrTime| is met, the first data of the current batch of data and latestMr of the previous batch of data are located in the same period, otherwise, the first data of the current batch of data and latestMr of the previous batch of data are not located in the same period;
the headDataTime represents the generation time of the first piece of data of the current batch of data, the latestMrTime represents the merging result time of the last period of the last batch of data, T represents time, the calculation mode is t=1/f×1000, and f represents the frequency of the data.
On the basis of the technical scheme, the data are sequenced based on the data generation time, and the data are taken out in batches, specifically:
and sequencing the data based on the data generation time, and then taking out the data in batches according to the sequence of the data generation time, wherein the data quantity of each batch is the same.
On the basis of the technical scheme, after determining the data type range and the time range of the data to be combined, the method further comprises the following steps:
and inquiring the data total amount in the range from the data warehouse according to the determined range, judging whether the data total amount is 0, if so, indicating that the data in the range is not met, notifying and reminding, and if not, recording the data total amount.
On the basis of the technical proposal, the method comprises the following steps,
after all batches of data are combined, outputting the combined result to a data warehouse, and inquiring the data combining progress in real time in the data combining process;
the calculation formula of the data merging progress is as follows:
A＝mergeNum/totalNum*100％
wherein A represents the data merging progress, mergeNum represents the merged data quantity, and totalNum represents the total data quantity.
The invention provides a system for merging frequency division data of a vehicle network, which comprises the following components:
the determining module is used for determining a data type range and a time range of data to be combined and determining a data combination reference frequency;
the dividing module is used for sequencing the data based on the data generation time, taking out the data in batches, and dividing each batch of the taken-out data according to the period by taking the data merging reference frequency as the period;
the merging module is used for taking the data with the earliest generation time in each period as merging reference data, merging the data in the period into the merging reference data and generating a merging result;
a recording module for recording the data type in each period, the latest data of each frequency data, and the combined number of each frequency data.
Compared with the prior art, the invention has the advantages that: based on the vehicle data uploaded by the vehicle-mounted terminal, each piece of data comprises the characteristics of the corresponding type, the uploading frequency and the generating time, the merging reference frequency is determined firstly, the data involved in merging are merged into the reference frequency data according to the merging reference frequency, and the data in the same time range can be merged as much as possible.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a method for merging frequency division data of a vehicle network according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a method for merging frequency division data of a vehicle network, which is based on the characteristics of vehicle data uploaded by a vehicle-mounted terminal, wherein each piece of data comprises a corresponding type, uploading frequency and generation time. The embodiment of the invention correspondingly provides a system for merging the frequency division data of the vehicle network.
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments.
Referring to fig. 1, an embodiment of the present invention provides a method for merging frequency division data of a vehicle network, which specifically includes the following steps:
s1: determining a data type range and a time range of data to be combined, and determining a data combination reference frequency;
it should be noted that, the vehicle network frequency division data merging method of the present invention may be performed by using a relevant merging device. In the actual application process, the merging device is embedded into a task submitted by a user and supports automatic plug, the merging device automatically analyzes according to task parameters, a data type range and a time range of data to be merged are generated, and meanwhile, the data merging reference frequency is determined. After determining the data type range and the time range of the data to be combined, inquiring the data warehouse to obtain the total data in the range according to the determined range, judging whether the total data is 0, if so, indicating that the data in the range is not met, notifying and reminding, and if not, recording the total data.
S2: sequencing the data based on the data generation time, taking out the data in batches, and dividing each batch of the data taken out according to the period by taking the data merging reference frequency as the period;
in the embodiment of the invention, the data are sequenced based on the generation time of the data, and the data are taken out in batches, specifically: sequencing the data based on the data generation time, and then taking out the data in batches according to the sequence of the data generation time, wherein the data quantity of each batch is the same, and the specific quantity is dynamically set according to the equipment hardware configuration.
S3: the data with earliest time generated in each period is used as merging reference data, and the data in the period are merged into the merging reference data to generate a merging result;
s4: the data type in each period, the original data of each frequency data corresponding to the latest data, and the combined number of each frequency data are recorded.
For example, when the first batch of data is merged, the data with the earliest time in the batch of data is taken as merging reference data, then the data within a range of one period is merged into the merging reference data from the generation time of the merging reference data by taking the data merging reference frequency as a period, and a merging result is generated, then the data type, the latest data of each frequency data and the merging quantity of each frequency data in the period are recorded, and then the merging of the first period is ended, the merging of the next period is started, the data with the earliest time in the next period is taken as merging reference data, and so on. In the course of data merging, the same data will overwrite the old value with the latest value.
The data combination formula is:
mr＝rmd+md
where mr represents the merging result, rmd represents the merging reference data, and md represents the data in the period.
In the embodiment of the invention, the data with the earliest time generated in each period is used as the merging reference data, the data in the period is merged into the merging reference data, and a merging result is generated, wherein when the data in a new period is merged:
judging whether data missing exists in the current period of merging or not:
if yes, according to the type and the frequency of the missing data, finding corresponding data from the original data of the latest data corresponding to each frequency data recorded in the previous period, and merging the found data into the current period when the time difference of the data meets a time threshold;
if not, the processing is not performed.
Namely, when data combination of a new period is carried out, data with earliest time generated in the period (namely, the first piece of data of the period) is taken as combination reference data, then the data in the period is combined into the combination reference data, a combination result is generated, if partial data is not found in the combination process, corresponding data is found from the latest data of each frequency data recorded in the previous period according to the type and the frequency of the missing data, and when the time difference of the data meets a time threshold value, the found data is combined into the current period.
In the embodiment of the invention, when the mttTime is more than or equal to |labetDataTime-rmdTime| is satisfied, the time difference of the data satisfies the time threshold value, otherwise, the time difference does not satisfy the time threshold value;
where mttTime denotes a time threshold, latestDataTime denotes data found from the latest data of each frequency data recorded in the previous cycle, rmdTime denotes the generation time of the merged reference data in the current cycle. .
In the embodiment of the invention, when corresponding data is not found from the original data of the latest data corresponding to each frequency data recorded in the previous period, or the time difference of the found data does not meet the time threshold, the data missing part of the current period for merging is set to be empty.
In the embodiment of the invention, after the fetched batch of data is combined according to the period, the combination result of the last period of the batch of data is marked as latestMr;
when the new batch of data is taken out for merging, judging whether the first data of the current batch of data and the latestMr of the previous batch of data are located in the same period, if so, taking the latestMr as merging reference data of the first period in the current batch of data, and if not, taking the data with earliest time generated in the first period in the current batch of data as merging reference data.
According to the data merging logic, when all the data of each batch are merged until the last period in the current batch, the merging result latestMr needs to be recorded, so that in the process of acquiring the data of a specific batch, the data is carried out according to the number of the data, and possibly, part of the data in the current period is also in the data of the next batch, so that the latestMr needs to be recorded.
Before the data of the new batch is merged, whether the first data of the current batch and the latestMr of the previous batch are located in the same period is needed to be judged, judging logic is carried out according to the time difference of the two data, if yes, the latestMr is used as merging reference data of the first period in the current batch, and if not, the data with earliest time generated in the first period in the current batch is used as merging reference data.
In the embodiment of the invention, when T is less than or equal to |head DataTime-latestMrTime| is satisfied, the first data of the current batch of data and latestMr of the previous batch of data are positioned in the same period, otherwise, the first data of the current batch of data and latestMr of the previous batch of data are not positioned in the same period;
the headDataTime represents the generation time of the first piece of data of the current batch of data, the latestMrTime represents the merging result time of the last period of the last batch of data, T represents time, the calculation mode is t=1/f×1000, f represents the frequency of the data, and the unit is millisecond.
In the embodiment of the invention, after all batches of data are combined, the combined result is output to the data warehouse, and the data combining progress can be queried in real time in the data combining process;
the calculation formula of the data merging progress is as follows:
A＝mergeNum/totalNum*100％
wherein A represents the data merging progress, mergeNum represents the merged data quantity, and totalNum represents the total data quantity.
That is, when all the batches of data are combined, the combined result can be output to a common component, such as a data warehouse kafka, hdfs, hive, mysql, es, redis, mongodb, according to a specified output mode.
In the invention, the data merging reference frequency rmf is dynamically acquired according to related requirements, the uploading frequency of each piece of data, the merging time threshold mtt, the generating time of each piece of data, the merging reference data rmd, the merged data md and the merging result mr; there may be multiple pieces of combined data depending on different frequencies and different signal types. In view of the fact that the data merging logic of three or more frequencies and signal types is identical to the data merging logic of two frequencies and signal types, taking two data merging as an example, assuming that two data participating in merging are respectively GPS data and CAN data, uploading frequencies are respectively 10HZ and 1HZ, the splicing sequence is to determine a data merging reference frequency rmf first, then merging the merging data onto the reference merging frequency data to obtain a merging result mr, various data delay losses are considered during the merging process, and the uploading frequency is not strictly the problem of uploading according to the required specified frequency.
The method for merging frequency division data of a vehicle network according to the present invention will be specifically described with reference to the following example.
S101: the user submits the task, the user may submit the task into a specialized merge device, and the merge device may be embedded into other services. In view of the fact that the data merging logic of three or more frequencies and signal types is identical to the logic of data merging of two frequencies and signal types, the example is presented by taking two data types as an example, assuming that the merged data designated by the user are GPS data gpsData and CAN data, the uploading frequency is 10Hz and 1Hz respectively, and the determined data merging reference frequency is 1Hz.
S102: the merging device merges gpsData and canData data according to dynamic analysis, and takes out the data from the data warehouse in batches according to time ascending sequence according to the standard of ten thousands of data of each batch according to the analyzed submission;
s103: the gpsData data and canData data for each lot are merged, and then the merged result is output to a place designated by the user.
For the data merging process, specifically:
s201: the merging device creates a merging period with the time of 1 second according to the data merging reference frequency of 1Hz as a reference;
s202: the method comprises the steps of merging data of a first period in first batch data, finding out the first piece of data of the batch as merging reference data, merging gpsData and canData in the period into the merging reference data, and generating a merging result. Recording the latest original data of gpsData and canData in the first period as the replacement when the next period is combined with the missing period;
s203: and merging the data of the next period, taking the first piece of data in the period as merging reference data, and merging gpsData and canData in the period into merging reference data to generate a merging result. After the period is combined, the type and frequency of the combined data are compared with the combination requirement, if the data are found to be missing, the corresponding data are found in the original data corresponding to the gpsData and canData data which are latest in the previous period, the corresponding data are compared with the current combination result time, if the time is smaller than or equal to the time threshold, the data are combined into the combination reference data, otherwise, the type signal is subjected to the emptying treatment.
S204: judging whether all the data of the current batch are combined, if not, turning to S203, and if yes, turning to S205;
s205: the data of the current batch is completed, and the merging result of the last period of the batch of data is marked as latestMr and is used for judging when the first period of the next batch is merged;
s206: when merging the first period data of the next batch, judging whether the first data of the current batch and the latestMr of the previous batch are positioned in the same period, if so, taking the latestMr as merging reference data of the first period in the current batch, and if not, taking the data with earliest time generated in the first period in the current batch as merging reference data;
s207: and judging whether all the batch data are combined, if not, turning to S203, and if so, outputting a combined result to a place designated by a user.
For the inquiry of the data merging progress, the specific steps are as follows:
s301: a user submits a data merging progress query request, wherein the data merging progress query request comprises a batch ID to be queried;
s302: inquiring to obtain the total data amount and the combined data amount of the batch according to the batch ID to be inquired;
s303: and calculating according to the mergeNum/total num 100%, obtaining the data merging progress, and outputting the result to the user.
The system for merging the frequency division data of the vehicle network is based on the vehicle data uploaded by the vehicle-mounted terminal, each piece of data comprises the characteristics of the corresponding type, the uploading frequency and the generating time, the merging reference frequency is firstly determined, the data involved in merging are merged into the reference frequency data according to the merging reference frequency, and the data in the same time range are merged into the reference frequency data, so that each piece of data can be merged as much as possible. Meanwhile, the merging progress can be queried in real time, the method is simple and clear, different types of data with different frequencies can be provided, and the method is high in feasibility and reliability and wide in application range and meets the analysis requirement at the same moment.
In a possible implementation manner, the embodiment of the present invention further provides a readable storage medium, where the readable storage medium is located in a PLC (Programmable Logic Controller ) controller, and a computer program is stored on the readable storage medium, where the computer program is executed by a processor to implement the following steps of a method for merging vehicle network frequency division data:
determining a data type range and a time range of data to be combined, and determining a data combination reference frequency;
sequencing the data based on the data generation time, taking out the data in batches, and dividing each batch of the data taken out according to the period by taking the data merging reference frequency as the period;
the data with earliest time generated in each period is used as merging reference data, and the data in the period are merged into the merging reference data to generate a merging result;
the data type in each period, the original data of each frequency data corresponding to the latest data, and the combined number of each frequency data are recorded.
The storage media may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present invention may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The embodiment of the invention provides a system for merging frequency division data of a vehicle network, which comprises a determining module, a dividing module, a merging module and a recording module.
The determining module is used for determining a data type range and a time range of data to be combined and determining a data combination reference frequency; the dividing module is used for sequencing the data based on the data generation time, taking out the data in batches, and dividing each batch of the data taken out according to the period by taking the data merging reference frequency as the period; the merging module is used for taking the data with earliest time generated in each period as merging reference data, merging the data in the period into the merging reference data and generating a merging result; the recording module is used for recording the data type in each period, the original data of the latest data corresponding to each frequency data and the combination quantity of each frequency data.
The foregoing is merely a specific embodiment of the application to enable one skilled in the art to understand or practice the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Claims (9)
1. The method for merging the frequency division data of the vehicle network is characterized by comprising the following steps of:
determining a data type range and a time range of data to be combined, and determining a data combination reference frequency;
sequencing the data based on the data generation time, taking out the data in batches, and dividing each batch of the data taken out according to the period by taking the data merging reference frequency as the period;
the data with earliest time generated in each period is used as merging reference data, and the data in the period are merged into the merging reference data to generate a merging result;
recording the data type in each period, the original data of the latest data corresponding to each frequency data and the merging quantity of each frequency data;
the data generated in each period is used as merging reference data, the data in the period is merged into the merging reference data, and a merging result is generated, wherein when the data in a new period is merged:
judging whether data missing exists in the current period of merging or not:
if yes, according to the type and the frequency of the missing data, finding corresponding data from the original data of the latest data corresponding to each frequency data recorded in the previous period, and merging the found data into the current period when the time difference of the data meets a time threshold;
if not, the processing is not performed.
2. A method of merging fractional data on a vehicle network as claimed in claim 1, wherein:
when the mttTime is more than or equal to |labstDataTime-rmdTime| is satisfied, the time difference of the data satisfies a time threshold value, otherwise, the data does not satisfy the time threshold value;
where mttTime denotes a time threshold, latestDataTime denotes data found from the latest data of each frequency data recorded in the previous cycle, rmdTime denotes the generation time of the merged reference data in the current cycle.
3. A method of merging fractional data on a vehicle network as claimed in claim 1, wherein: when the corresponding data is not found from the original data of the latest data corresponding to each frequency data recorded in the previous period, or the time difference of the found data does not meet the time threshold, the data missing part of the current period for merging is set to be empty.
4. A method of merging fractional data on a vehicle network as claimed in claim 1, wherein:
after the fetched batch of data is combined according to the period, marking the combination result of the last period of the batch of data as latestMr;
when the new batch of data is taken out for merging, judging whether the first data of the current batch of data and the latestMr of the previous batch of data are located in the same period, if so, taking the latestMr as merging reference data of the first period in the current batch of data, and if not, taking the data with earliest time generated in the first period in the current batch of data as merging reference data.
5. A method of merging fractional data on a vehicle network as defined in claim 4, wherein:
when T is less than or equal to |head DataTime-latestMrTime| is met, the first data of the current batch of data and latestMr of the previous batch of data are located in the same period, otherwise, the first data of the current batch of data and latestMr of the previous batch of data are not located in the same period;
the headDataTime represents the generation time of the first piece of data of the current batch of data, the latestMrTime represents the merging result time of the last period of the last batch of data, T represents time, the calculation mode is t=1/f×1000, and f represents the frequency of the data.
6. The method for merging frequency division data of a vehicle network according to claim 1, wherein the data are ordered based on the time of data generation, and the data are fetched in batches, specifically:
and sequencing the data based on the data generation time, and then taking out the data in batches according to the sequence of the data generation time, wherein the data quantity of each batch is the same.
7. The method for merging frequency-divided data of a vehicle network as claimed in claim 1, further comprising, after determining a data type range and a time range of the data to be merged:
and inquiring the data total amount in the range from the data warehouse according to the determined range, judging whether the data total amount is 0, if so, indicating that the data in the range is not met, notifying and reminding, and if not, recording the data total amount.
8. A method of merging fractional data on a vehicle network as defined in claim 7, wherein:
after all batches of data are combined, outputting the combined result to a data warehouse, and inquiring the data combining progress in real time in the data combining process;
the calculation formula of the data merging progress is as follows:
A＝mergeNum/totalNum*100％
wherein A represents the data merging progress, mergeNum represents the merged data quantity, and totalNum represents the total data quantity.
9. A system for vehicle network crossover data consolidation, comprising:
the determining module is used for determining a data type range and a time range of data to be combined and determining a data combination reference frequency;
the dividing module is used for sequencing the data based on the data generation time, taking out the data in batches, and dividing each batch of the taken-out data according to the period by taking the data merging reference frequency as the period;
the merging module is used for taking the data with the earliest generation time in each period as merging reference data, merging the data in the period into the merging reference data and generating a merging result;
a recording module for recording a data type in each period, latest data of each frequency data, and a combination number of each frequency data;
wherein the data generated in the earliest time in each period is used as merging reference data, the data in the period is merged into the merging reference data and a merging result is generated, wherein,
when data merging of a new cycle is performed:
judging whether data missing exists in the current period of merging or not:
if yes, according to the type and the frequency of the missing data, finding corresponding data from the original data of the latest data corresponding to each frequency data recorded in the previous period, and merging the found data into the current period when the time difference of the data meets a time threshold;
if not, the processing is not performed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210772930.3A CN115277843B (en) | 2022-06-30 | 2022-06-30 | Method and system for merging frequency division data of vehicle network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210772930.3A CN115277843B (en) | 2022-06-30 | 2022-06-30 | Method and system for merging frequency division data of vehicle network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115277843A CN115277843A (en) | 2022-11-01 |
| CN115277843B true CN115277843B (en) | 2024-01-26 |
Family
ID=83762344
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210772930.3A Active CN115277843B (en) | 2022-06-30 | 2022-06-30 | Method and system for merging frequency division data of vehicle network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115277843B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112087500A (en) * | 2020-08-27 | 2020-12-15 | 东风小康汽车有限公司重庆分公司 | Remote vehicle data acquisition method and system |
| CN112749204A (en) * | 2019-10-31 | 2021-05-04 | 北京沃东天骏信息技术有限公司 | Method and device for reading data |
| CN112804760A (en) * | 2021-04-14 | 2021-05-14 | 网络通信与安全紫金山实验室 | Method, system, device and medium for scheduling of a periodic data stream |
| CN113177024A (en) * | 2021-06-29 | 2021-07-27 | 南京烽火星空通信发展有限公司 | Data global merging method under mass data scene |
| CN114296944A (en) * | 2022-01-04 | 2022-04-08 | 腾讯科技（深圳）有限公司 | Data processing method, data processing device, electronic device, program product, and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11341165B2 (en) * | 2020-09-15 | 2022-05-24 | Nice Ltd. | Staging tables for data ingestion of micro-batched data records |
-
2022
- 2022-06-30 CN CN202210772930.3A patent/CN115277843B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112749204A (en) * | 2019-10-31 | 2021-05-04 | 北京沃东天骏信息技术有限公司 | Method and device for reading data |
| CN112087500A (en) * | 2020-08-27 | 2020-12-15 | 东风小康汽车有限公司重庆分公司 | Remote vehicle data acquisition method and system |
| CN112804760A (en) * | 2021-04-14 | 2021-05-14 | 网络通信与安全紫金山实验室 | Method, system, device and medium for scheduling of a periodic data stream |
| CN113177024A (en) * | 2021-06-29 | 2021-07-27 | 南京烽火星空通信发展有限公司 | Data global merging method under mass data scene |
| CN114296944A (en) * | 2022-01-04 | 2022-04-08 | 腾讯科技（深圳）有限公司 | Data processing method, data processing device, electronic device, program product, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115277843A (en) | 2022-11-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020181879A1 (en) | Vehicle-related data processing method and system | |
| CN110689804B (en) | Method and apparatus for outputting information | |
| CN112630799B (en) | Method and apparatus for outputting information | |
| CN113537584A (en) | Inspection task conflict processing method and device, electronic equipment and storage medium | |
| CN112590813A (en) | Method, apparatus, electronic device, and medium for generating information of autonomous vehicle | |
| CN108985819A (en) | Driver's portrait method, system and equipment | |
| CN113868350A (en) | Parking lot map processing method, vehicle and equipment | |
| CN115761702A (en) | Vehicle track generation method and device, electronic equipment and computer readable medium | |
| CN115240157A (en) | Road scene data persistence method, device, equipment and computer readable medium | |
| WO2024192998A1 (en) | Method and apparatus for determining target position, and system and storage medium | |
| CN108711025B (en) | Car rental vehicle inventory query method and device, electronic equipment and storage medium | |
| CN106648839B (en) | Data processing method and device | |
| CN112622923B (en) | Method and device for controlling a vehicle | |
| CN115277843B (en) | Method and system for merging frequency division data of vehicle network | |
| CN113743456B (en) | Scene positioning method and system based on unsupervised learning | |
| CN113761306A (en) | Vehicle-end data processing method and device | |
| CN116442787A (en) | An electric vehicle energy consumption warning method, device, medium and equipment | |
| CN113450794B (en) | Navigation broadcasting detection method and device, electronic equipment and medium | |
| CN115061386A (en) | Intelligent driving automatic simulation test system and related equipment | |
| CN113008246B (en) | Map matching method and device | |
| CN114118440A (en) | Model iteration method, model iteration device, electronic equipment and computer readable storage medium | |
| CN114861321A (en) | Problem scene extraction method, device, equipment and medium for traffic flow simulation | |
| CN112729319A (en) | Automatic data acquisition and analysis system and method | |
| CN117261930B (en) | Early warning method and device for fatigue driving | |
| CN116013106B (en) | Smart parking management method, smart parking management system, smart parking management terminal and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |