CN112181906A - Test data synchronization method, device and system based on combined data acquisition instrument - Google Patents
Test data synchronization method, device and system based on combined data acquisition instrument Download PDFInfo
- Publication number
- CN112181906A CN112181906A CN201910587443.8A CN201910587443A CN112181906A CN 112181906 A CN112181906 A CN 112181906A CN 201910587443 A CN201910587443 A CN 201910587443A CN 112181906 A CN112181906 A CN 112181906A
- Authority
- CN
- China
- Prior art keywords
- data
- processed
- parameter
- time
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 41
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000012545 processing Methods 0.000 claims abstract description 38
- 239000011159 matrix material Substances 0.000 claims description 25
- 238000007865 diluting Methods 0.000 claims description 8
- 230000001360 synchronised effect Effects 0.000 abstract description 10
- 238000005070 sampling Methods 0.000 description 10
- 230000001133 acceleration Effects 0.000 description 8
- 238000010790 dilution Methods 0.000 description 6
- 239000012895 dilution Substances 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 239000000284 extract Substances 0.000 description 5
- 230000006872 improvement Effects 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000013519 translation Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 3
- 239000000243 solution Substances 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 239000000446 fuel Substances 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/178—Techniques for file synchronisation in file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a test data synchronization method based on a combined data acquisition instrument, which comprises the following steps: s1, acquiring a data file acquired by the data acquisition instrument, wherein the data file is provided with time information and comprises a data file containing a first parameter and a data file containing a second parameter; s2, extracting and merging the data files containing the first parameters in the target time period according to the first parameters and the time information to generate data to be processed, and extracting and merging the data files containing the second parameters in the target time period according to the second parameters and the time information to generate reference data; and S3, performing time synchronization processing on the data to be processed and the reference data by taking the time of the reference data as a reference. The invention also discloses a time synchronization device and a test data synchronization system based on the combined data acquisition instrument. By adopting the invention, the data files can be classified, merged and synchronously processed, the frequency and time of the data files can be quickly synchronized, and the utilization rate of the data acquisition instrument can be improved.
Description
Technical Field
The invention relates to the technical field of time synchronization, in particular to a test data synchronization method and a time synchronization device based on a combined data acquisition instrument and a test data synchronization system based on the combined data acquisition instrument.
Background
In the process of a finished automobile test, a large number of signals may need to be collected for some special working conditions. Because the physical characteristics of different signals are different, the data acquisition instrument using a single model cannot acquire all signals, the data acquisition instruments using a plurality of different models and types are often required to be combined for acquisition, and the problem of asynchronous data signal time often occurs easily in the process of manually operating the data acquisition instrument. On the premise, the data sampling frequency of the signals is difficult to ensure consistency, and the problem of time lag caused by inconsistent starting time of the instrument is difficult to eliminate in the data acquisition process, so that great difficulty is generated in the post-splicing processing of the data.
In order to solve the above problems, in the prior art, a single data acquisition instrument with a higher sampling channel and a wider signal acquisition performance adaptation is often used for data synchronization processing, or a manual processing mode is used for time synchronization processing of multiple items of data, so that the test cost is increased; meanwhile, due to the addition of the testing instruments, more tools are required to be added in the testing process, the space in the vehicle is occupied, and conditions required by the whole vehicle test are increased.
For example, chinese patent CN201610232484.1 discloses a method and an apparatus for synchronizing sampling point time in a car navigation system, which uses a first sampling device to collect a first signal point, and then sends a synchronization signal to a second sampling device to start collecting a second signal point, so as to achieve the synchronization effect. However, the patent adopts a communication method between instruments, synchronous signals between the instruments are at risk of being lost, meanwhile, the patent needs a specific instrument to set a program, once a problem occurs in field test, the program needs to be adjusted immediately, the real-time requirement of the whole vehicle test is not met, and the method is convenient without an off-line data processing mode.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a test data synchronization method, a test data synchronization system and a time synchronization device based on a combined data acquisition instrument, which can classify, combine and synchronize data files, realize the rapid synchronization of the frequency and time of the data files and improve the utilization rate of the data acquisition instrument.
In order to solve the technical problem, the invention provides a test data synchronization method based on a combined data acquisition instrument, which comprises the following steps:
s1, acquiring a data file acquired by the data acquisition instrument, wherein the data file is provided with time information and comprises a data file containing a first parameter and a data file containing a second parameter;
s2, extracting and merging the data files containing the first parameters in the target time period according to the first parameters and the time information to generate data to be processed, and extracting and merging the data files containing the second parameters in the target time period according to the second parameters and the time information to generate reference data;
s3, taking the time of the reference data as the reference to perform time synchronization processing on the data to be processed and the reference data
As a modification of the above, the step S2 includes: extracting a data file containing the first parameter, updating the file name of the data file according to the first parameter and the time information, and merging the data files containing the first parameter in the target time period according to the file name to generate data to be processed; and extracting the data file containing the second parameter, updating the file name of the data file according to the second parameter and the time information, and merging the data files containing the second parameter in the target time period according to the file name to generate the datum data.
As a modification of the above, the step S3 includes: calculating lag time of the data to be processed according to the cross correlation between the data to be processed and the reference data; diluting the data to be processed according to the time information, wherein partial data are missing after the data to be processed is diluted; and moving the diluted data to be processed to be aligned with the reference data according to the lag time, and performing interpolation processing on missing data in the diluted data to be processed so as to synchronize the interpolated data to be processed with the reference data.
As an improvement of the above solution, the step of calculating the lag time of the data to be processed according to the cross-correlation between the data to be processed and the reference data includes: constructing a data matrix according to the data to be processed and the reference data, wherein a first column in the data matrix is the difference between adjacent data to be processed in the data to be processed, and a second column is the difference between adjacent reference data in the reference data; calculating a cross-correlation value between the data to be processed and reference data according to the data matrix; and calculating the lag time of the data to be processed according to the cross-correlation value.
Correspondingly, the invention also provides a time synchronization device, which comprises: the acquisition module is used for acquiring a data file acquired by the data acquisition instrument, wherein the data file has time information and comprises a data file containing a first parameter and a data file containing a second parameter; the generating module is used for extracting and combining the data files containing the first parameters in the target time period according to the first parameters and the time information so as to generate data to be processed, and extracting and combining the data files containing the second parameters in the target time period according to the second parameters and the time information so as to generate reference data; and the synchronization module is used for carrying out time synchronization processing on the data to be processed and the reference data by taking the time of the reference data as a reference.
As an improvement of the above scheme, the generating module includes: the to-be-processed data generation unit is used for extracting the data file containing the first parameter, updating the file name of the data file according to the first parameter and the time information, and merging the data files containing the first parameter in the target time period according to the file name to generate to-be-processed data; and the reference data generating unit is used for extracting the data file containing the second parameter, updating the file name of the data file according to the second parameter and the time information, and merging the data files containing the second parameter in the target time period according to the file name to generate reference data.
As an improvement of the above solution, the synchronization module includes: the calculation unit is used for calculating the lag time of the data to be processed according to the cross correlation between the data to be processed and the reference data; the diluting unit is used for diluting the data to be processed according to time information, and part of the data is lost after the data to be processed is diluted; and the synchronization unit is used for moving the diluted data to be processed to be aligned with the reference data according to the lag time and performing interpolation processing on missing data in the diluted data to be processed so as to synchronize the interpolated data to be processed with the reference data.
As an improvement of the above, the calculation unit includes: the matrix subunit is used for constructing a data matrix according to the data to be processed and the reference data, wherein a first column in the data matrix is a difference between adjacent data to be processed in the data to be processed, and a second column is a difference between adjacent reference data in the reference data; the cross correlation subunit is used for calculating a cross correlation value between the data to be processed and the reference data according to the data matrix; and the lag subunit is used for calculating the lag time of the data to be processed according to the cross-correlation value.
Correspondingly, the invention also provides a test data synchronization system based on the combined data acquisition instrument, which comprises: the data acquisition instrument is used for acquiring data files; and the time synchronization device is used for carrying out synchronization processing on the data files.
The invention can use data acquisition instruments with different sampling frequencies to carry out multi-channel data acquisition on the same test working condition at different time starting points, then input data files output by the data acquisition instruments to the time synchronization device, the time synchronization device classifies and combines the input data files through a unique mode to form two types of data (reference data and data to be processed) based on different parameters, and then the time synchronization device carries out synchronization processing on the two types of data to realize the rapid synchronization of the frequency and the time of the data files. Specifically, the invention has the following beneficial effects:
1. the utilization rate of the data acquisition equipment is improved, and the cost for using a high-precision acquisition instrument is saved;
2. and merging the data files according to time and fields to form mutually independent data to be processed and reference data, thereby realizing the high-efficiency preprocessing of the data files.
Drawings
FIG. 1 is a flow chart of a first embodiment of a test data synchronization method based on a combined data acquisition instrument according to the present invention;
FIG. 2 is a flow chart of a test data synchronization method based on a combined data acquisition instrument according to a second embodiment of the present invention;
FIG. 3 is a flow chart of a third embodiment of the test data synchronization method based on a combined data acquisition instrument according to the present invention;
FIG. 4 is a schematic structural diagram of a test data synchronization system based on a combined data acquisition instrument according to the present invention;
FIG. 5 is a schematic diagram of the structure of the time synchronizer of the present invention;
FIG. 6 is a schematic structural diagram of a generating module in the time synchronizer according to the present invention;
FIG. 7 is a schematic structural diagram of a synchronization module in the time synchronization apparatus according to the present invention;
fig. 8 is another structural diagram of the synchronization module in the time synchronization apparatus according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings.
As shown in fig. 1, fig. 1 is a flowchart of a test data synchronization method based on a combined data acquisition instrument according to a first embodiment of the present invention, which includes:
and S101, acquiring a data file acquired by the data acquisition instrument.
In the test, data acquisition instruments with different sampling frequencies are needed to be used, multichannel data acquisition is carried out on the same test working condition at different time starting points, and a plurality of data files acquired by the data acquisition instruments are output to a time synchronization device. The data files are data files with time information and including a data file containing a first parameter and a second parameter (namely the data files are data files with time information and including a data file containing the first parameter and data files with time information and including a data file containing the second parameter).
S102, extracting and combining data files containing the first parameter in a target time period according to the first parameter and the time information to generate data to be processed; and extracting and combining the data files containing the second parameters in the target time period according to the second parameters and the time information to generate reference data.
Different data files contain different data fields, in the test, a certain data field can be set as a first parameter or a second parameter according to the actual situation, the data files containing the first parameter are extracted, and then merging processing is carried out according to time information to generate data to be processed; and meanwhile, extracting a data file containing a second parameter, and merging according to the time information to generate reference data.
Specifically, the first parameter and the second parameter may be, but are not limited to, a vehicle speed data field, a GPS data field, an acceleration data field, a water temperature data field, or a fuel consumption data field. In general, the low frequency data field may be set to a first parameter and the high frequency data field may be set to a second parameter, where the first parameter and the second parameter have the same auxiliary signal data. For example, when vehicle speed synchronization needs to be achieved, a low-frequency vehicle speed data field can be set as a first parameter, and a high-frequency vehicle speed data field can be set as a second parameter; when acceleration synchronization needs to be achieved, the low-frequency acceleration data field can be set as a first parameter, and the high-frequency acceleration data field can be set as a second parameter.
It should be noted that the number of times of recording of the high-frequency data field is ten times or more of that of the other data fields in the same period of time. In the vehicle-mounted general-purpose recorded data, a GPS signal is generally used as a high-frequency data comparison field, and a signal having a frequency close to that of the GPS signal can be considered as a high-frequency data field.
And S103, performing time synchronization processing on the data to be processed and the reference data by taking the time of the reference data as a reference.
After the time synchronizer acquires the data files, the data files are classified, merged and synchronously processed, and the frequency and time of the data files are quickly synchronized.
Therefore, the test data synchronization method based on the combined data acquisition instrument can effectively solve the problems of different signal frequencies, different time and incapability of splicing caused by using different types of data acquisition instruments to combine and acquire data in the whole vehicle test, and improves the utilization rate of the data acquisition instrument.
As shown in fig. 2, fig. 2 is a flowchart of a test data synchronization method based on a combined data acquisition instrument according to a second embodiment of the present invention, which includes:
s201, acquiring a data file acquired by a data acquisition instrument; the data file is provided with time information and comprises a data file containing a first parameter and a data file containing a second parameter.
S202, extracting the data file containing the first parameter, updating the file name of the data file according to the first parameter and the time information, and merging the data files containing the first parameter in the target time period according to the file name to generate data to be processed.
And after acquiring the data files acquired by the data acquisition instrument, the time synchronizer opens the data files one by one, judges whether the data files have the first parameters or not, and if so, indicates that the data files have the first parameters. And after the judgment of all the data files is finished, merging the data files containing the first parameter in the target time period to generate the data to be processed. Wherein the target time period may be one day, one hour, or 30 minutes; preferably, the target time period is at least 30 minutes, but not limited thereto, and may be preset by a tester according to actual circumstances.
Specifically, the present invention first extracts the first parameter of the data file with the first parameter to form a new data file named by the first parameter and the time information, that is, the file name format of the new data file is "first parameter + time information", for example: 60_2019-2-15-09-23, 67_ 2019-2-15-09-25. Then, scanning the file names of the data files one by one, and merging the data files containing the first parameter in the target time period according to the time information in the file names to form data to be processed, for example: merging data files containing the first parameters on the day of 2019-2-15 to form to-be-processed data, wherein the file name of the to-be-processed data is as follows: 2019-2-15.
S203, extracting the data file containing the second parameter, updating the file name of the data file according to the second parameter and the time information, and merging the data files containing the second parameter in the target time period according to the file name to generate the reference data.
And after acquiring the data files acquired by the data acquisition instrument, the time synchronizer opens the data files one by one, judges whether the data files have second parameters or not, and if so, indicates that the data files have the second parameters. And after the judgment of all the data files is finished, merging the data files containing the second parameter in the target time period to generate the data to be processed. Wherein the target time period may be one day, one hour, or 30 minutes; preferably, the target time period is at least 30 minutes, but not limited thereto, and may be preset by a tester according to actual circumstances.
Specifically, the present invention first extracts the second parameter of the data file with the second parameter to form a new data file named with the second parameter and the time information, that is, the file name format of the new data file is "second parameter + time information", for example: 60_2019-2-15-09-23 and 70_ 2019-2-15-09-24. Then, scanning the file names of the data files one by one, and merging the data files containing the second parameter in the target time period according to the time information in the file names to form data to be processed, for example: merging data files containing the second parameter on the day of 2019-2-15 to form reference data, wherein the file name of the reference data is as follows: 2019-2-15.
It should be noted that there is no inevitable order between step S202 and step S203, and step S203 may be performed simultaneously with step S202.
And S204, performing time synchronization processing on the data to be processed and the reference data by taking the time of the reference data as a reference.
After step S202 and step S203 are completed, the time synchronization device performs synchronization processing on the data to be processed and the reference data, so as to realize rapid frequency and time synchronization of the data files.
Therefore, the data files are combined according to the fields (the first parameter and the second parameter) and the time (the time information) to form two groups of data (to-be-processed data and reference data) which are independent from each other, so that the data files are preprocessed; meanwhile, the frequency and time of the data files are quickly synchronized by synchronously processing the two groups of data, and the utilization rate of the data acquisition instrument is improved.
As shown in fig. 3, fig. 3 is a flowchart of a third embodiment of the test data synchronization method based on a combined data acquisition instrument, including:
s301, acquiring a data file acquired by the data acquisition instrument, wherein the data file has time information and comprises a data file containing a first parameter and a data file containing a second parameter.
S302, extracting the data file containing the first parameter, updating the file name of the data file according to the first parameter and the time information, and merging the data files containing the first parameter in the target time period according to the file name to generate data to be processed.
And S303, extracting the data file containing the second parameter, updating the file name of the data file according to the second parameter and the time information, and merging the data files containing the second parameter in the target time period according to the file name to generate the reference data.
It should be noted that there is no inevitable order between step S302 and step S303, and step S303 may be performed at the same time as step S302.
S304, calculating the lag time of the data to be processed according to the cross correlation between the data to be processed and the reference data.
After the step S302 and the step S303 are completed, the time synchronization device performs synchronization processing on the data to be processed and the reference data, so as to realize rapid frequency and time synchronization of the data files.
Specifically, the step S304 includes:
(1) and constructing a data matrix according to the data to be processed and the reference data, wherein the first column in the data matrix is the difference between the adjacent data to be processed in the data to be processed, and the second column is the difference between the adjacent reference data in the reference data.
(2) And calculating a cross-correlation value between the data to be processed and the reference data according to the data matrix.
(3) And calculating the lag time of the data to be processed according to the cross-correlation value.
S305, performing dilution processing on the data to be processed according to the time information, wherein partial data are missing after the data to be processed is diluted.
The time synchronization device reads the data to be processed and the reference data on the same date, and dilutes the first parameter in the data to be processed according to the time information. The dilution processing, that is, the frequency reduction processing, may be performed by extracting data 10000 times per minute proportionally to 1000 times per minute.
S306, the diluted data to be processed is moved to be aligned with the reference data according to the lag time, and interpolation processing is carried out on missing data in the diluted data to be processed so that the interpolated data to be processed is synchronous with the reference data.
Specifically, the steps include, according to 306:
(1) and aligning the diluted data to be processed with the reference data. And after the lag time is obtained through calculation, synchronous operation is carried out according to the time difference, and the first parameter translation in the data to be processed is aligned with the second parameter in the reference data.
(2) And performing interpolation processing on the diluted data to be processed. And carrying out interpolation operation on missing data in the data to be processed so as to obtain data which is time-synchronous with the reference data.
Therefore, the invention realizes the targeted classification of the data files by a unique extraction and combination mode, and has strong flexibility and high accuracy.
Therefore, the invention can use the data acquisition instruments with different sampling frequencies to perform multi-channel data acquisition on the same test working condition at different time starting points, and then input the data files output by the data acquisition instruments to the time synchronization device, and the time synchronization device performs synchronous processing on the input data files through unique classification, combination and synchronization modes, thereby realizing the rapid synchronization of the frequency and the time of the data files.
The invention is described in further detail below with specific data fields:
step 3, scanning file names of the data files in the folders one by one, and combining the data files in the same day according to time information in the file names to form data to be processed A1, A2.;
step 4, opening the data files in the folder one by one, and judging whether the data files have GPS speed data fields (second parameters);
step 5, extracting the GPS speed data field and the time information from the data file B with the GPS speed data field to form a new data file GPSpdd _ Date named by the GPS speed data field and the time information;
step 6, scanning the file names of the data files in the folders one by one, and combining the data files in the same day according to the time information in the file names to form a reference file B1, B2.;
step 7, reading the files A1 and B1 on the same date, and diluting the vehicle speed data field in the file A1 to be processed according to time;
step 8, combining the vehicle speed data field in the file A1 to be processed with the GPS speed data field in the reference file B1 to form a data matrix;
step 9, calculating the cross-correlation value of the data in the data matrix so as to obtain lag time;
step 10, after the lag time is obtained through calculation, synchronous operation is carried out according to a time difference value, and the vehicle speed data field translation in the file A1 to be processed is aligned with the GPS speed data field in the reference file B1;
and 11, carrying out interpolation operation on the data missing in the file A1 to be processed, thereby obtaining data time-synchronized with the reference file B1.
Referring to fig. 4, fig. 4 shows a specific structure of the test data synchronization system 100 based on the combined data acquisition instrument of the present invention, which includes:
and the data acquisition instrument 1 is used for acquiring data files.
And the time synchronizer 2 is used for carrying out synchronization processing on the data files.
The invention uses the data acquisition instruments 1 with different sampling frequencies to carry out multi-channel data acquisition to the same test working condition at different time starting points, and outputs a plurality of data files acquired by the data acquisition instruments 1 to a time synchronizer; meanwhile, the time synchronizer 2 merges the data files according to the parameters (the first parameter and the second parameter) and the time information to generate data to be processed and reference data, and then performs time synchronization on the data to be processed and the reference data.
As shown in fig. 5, the time synchronizer 2 includes an obtaining module 21, a generating module 22, and a synchronizing module 23, specifically:
the acquiring module 21 is configured to acquire a data file acquired by the data acquisition instrument 1, where the data file includes a data file with time information and includes a first parameter and a second parameter.
The generating module 22 is configured to extract and combine the data files containing the first parameter in the target time period according to the first parameter and the time information to generate to-be-processed data, and extract and combine the data files containing the second parameter in the target time period according to the second parameter and the time information to generate the reference data.
It should be noted that different data files contain different data fields, in a test, a certain data field can be set as a first parameter or a second parameter according to an actual situation, and the data file containing the first parameter is extracted and then merged according to time information to generate data to be processed; and meanwhile, extracting a data file containing a second parameter, and merging according to the time information to generate reference data. Specifically, the first parameter and the second parameter may be, but are not limited to, a vehicle speed data field, a GPS data field, an acceleration data field, a water temperature data field, or a fuel consumption data field. In general, the low frequency data field may be set to a first parameter and the high frequency data field may be set to a second parameter, where the first parameter and the second parameter have the same auxiliary signal data. For example, when vehicle speed synchronization needs to be achieved, a low-frequency vehicle speed data field can be set as a first parameter, and a high-frequency vehicle speed data field can be set as a second parameter; when acceleration synchronization needs to be achieved, the low-frequency acceleration data field can be set as a first parameter, and the high-frequency acceleration data field can be set as a second parameter.
And the synchronization module 23 is configured to perform time synchronization processing on the data to be processed and the reference data by using the time of the reference data as a reference. After the time synchronizer acquires the data files, the data files are classified, merged and synchronously processed, and the frequency and time of the data files are quickly synchronized.
Therefore, the test data synchronization method based on the combined data acquisition instrument can effectively solve the problems of different signal frequencies, different time and incapability of splicing caused by using different types of data acquisition instruments to combine and acquire data in the whole vehicle test, and improves the utilization rate of the data acquisition instrument.
As shown in fig. 6, the generating module 22 includes a to-be-processed data generating unit 221 and a reference data generating unit 222, specifically:
the to-be-processed data generating unit 221 extracts the data file containing the first parameter, updates the file name of the data file according to the first parameter and the time information, and merges the data files containing the first parameter in the target time period according to the file name to generate to-be-processed data. The data to be processed generating unit 221 opens the data files one by one after acquiring the data files acquired by the data acquisition instrument, and judges whether the data files have the first parameters, and if so, the data files have the first parameters. And after the judgment of all the data files is finished, merging the data files containing the first parameter in the target time period to generate the data to be processed. Specifically, the to-be-processed data generating unit 221 first extracts the first parameter from the data file with the first parameter to form a new data file named by the first parameter and the time information, that is, the file name format of the new data file is "first parameter + time information", for example: 60_2019-2-15-09-23, 67_ 2019-2-15-09-25. Then, scanning the file names of the data files one by one, and merging the data files containing the first parameter in the target time period according to the time information in the file names to form data to be processed, for example: merging data files containing the first parameters on the day of 2019-2-15 to form to-be-processed data, wherein the file name of the to-be-processed data is as follows: 2019-2-15.
The reference data generating unit 222 is configured to extract a data file containing the second parameter, update a file name of the data file according to the second parameter and the time information, and perform merging processing on the data files containing the second parameter in the target time period according to the file name to generate reference data. The reference data generating unit 222 opens the data files one by one after acquiring the data files acquired by the data acquisition instrument, and determines whether the data files have the second parameters, and if so, indicates that the data files have the second parameters. And after the judgment of all the data files is finished, merging the data files containing the second parameter in the target time period to generate the data to be processed. Specifically, the reference data generating unit 222 first performs second parameter extraction on the data file with the second parameter to form a new data file named by the second parameter and the time information, that is, the file name format of the new data file is "second parameter + time information", for example: 60_2019-2-15-09-23 and 70_ 2019-2-15-09-24. Then, scanning the file names of the data files one by one, and merging the data files containing the second parameter in the target time period according to the time information in the file names to form data to be processed, for example: merging data files containing the second parameter on the day of 2019-2-15 to form reference data, wherein the file name of the reference data is as follows: 2019-2-15.
Wherein the target time period may be one day, one hour or 30 minutes. Preferably, the target time period is at least 30 minutes, but not limited thereto, and may be preset by a tester according to actual circumstances.
Therefore, by the combined action of the to-be-processed data generation unit 221 and the reference data generation unit 222, the data file can be classified into two sets of data (to-be-processed data and reference data) independent of each other, thereby realizing the preprocessing of the data file.
Therefore, the to-be-processed data generation unit 221 and the reference data generation unit 222 realize the targeted classification of the data files through a unique extraction and combination mode, and have strong flexibility and high accuracy.
As shown in fig. 7, the synchronization module 23 includes:
a calculating unit 321, configured to calculate a lag time of the data to be processed according to a cross-correlation between the data to be processed and reference data.
The diluting unit 232 is configured to dilute the data to be processed according to the time information, where the data to be processed lacks part of the data after dilution. The dilution unit 232 reads the data to be processed and the reference data on the same date, and performs dilution processing on the first parameter in the data to be processed according to the time information. The dilution processing, that is, the frequency reduction processing, may be performed by extracting data 10000 times per minute proportionally to 1000 times per minute.
A synchronization unit 233, configured to shift the diluted to-be-processed data to be aligned with the reference data according to the lag time, and perform interpolation processing on missing data in the diluted to-be-processed data to synchronize the interpolated to-be-processed data with the reference data. Specifically, the synchronization unit 233 aligns the diluted data to be processed with the reference data, that is, after calculating the lag time, the synchronization operation may be performed according to the time difference, and the alignment subunit 2331 aligns the first parameter translation in the data to be processed with the second parameter translation in the reference data; then, interpolation processing is performed on the diluted data to be processed, that is, interpolation operation is performed on missing data in the data to be processed, so that data time-synchronized with the reference data is obtained.
Therefore, the time synchronization between two groups of data can be accurately and quickly realized through the synchronization module, the flexibility is strong, and the accuracy is high.
As shown in fig. 8, the calculation unit 231 includes:
the matrix subunit 2311 constructs a data matrix according to the data to be processed and the reference data, where a first column in the data matrix is a difference between adjacent data to be processed in the data to be processed, and a second column is a difference between adjacent reference data in the reference data. The matrix subunit 2321 combines the first parameter in the data to be processed with the second parameter in the reference data to form a data matrix.
A cross correlation subunit 2312, configured to calculate a cross correlation value between the data to be processed and the reference data according to the data matrix.
A lag subunit 2313, configured to calculate a lag time of the to-be-processed data according to the cross-correlation value.
From the above, the invention can use the data acquisition instruments with different sampling frequencies to perform multi-channel data acquisition on the same test condition at different time starting points, then input the data files output by each data acquisition instrument to the time synchronizer, the time synchronizer classifies and combines the input data files in a unique mode to form two types of data (reference data and data to be processed) based on different parameters, and then the time synchronizer performs synchronous processing on the two types of data to realize the rapid synchronization of the frequency and the time of the data files. Specifically, the invention has the following beneficial effects:
1. the cost of using a high-precision acquisition instrument is saved;
2. merging the data files according to time and fields to realize the high-efficiency pretreatment of the data files;
3. and the synchronization operation of the data files is realized by calculating the lag time and carrying out merging interpolation processing on the two groups of data.
The foregoing is a preferred embodiment of the present invention, and it should be noted that it would be apparent to those skilled in the art that various modifications and enhancements can be made without departing from the principles of the invention, and such modifications and enhancements are also considered to be within the scope of the invention.
Claims (9)
1. A test data synchronization method based on a combined data acquisition instrument is characterized by comprising the following steps:
s1, acquiring a data file acquired by the data acquisition instrument, wherein the data file is provided with time information and comprises a data file containing a first parameter and a data file containing a second parameter;
s2, extracting and merging the data files containing the first parameters in the target time period according to the first parameters and the time information to generate data to be processed, and extracting and merging the data files containing the second parameters in the target time period according to the second parameters and the time information to generate reference data;
and S3, performing time synchronization processing on the data to be processed and the reference data by taking the time of the reference data as a reference.
2. The method for synchronizing test data based on a combined data acquisition instrument according to claim 1, wherein the step S2 comprises:
extracting a data file containing the first parameter, updating the file name of the data file according to the first parameter and the time information, and merging the data files containing the first parameter in the target time period according to the file name to generate data to be processed;
and extracting the data file containing the second parameter, updating the file name of the data file according to the second parameter and the time information, and merging the data files containing the second parameter in the target time period according to the file name to generate the datum data.
3. The method for synchronizing test data based on a combined data acquisition instrument according to claim 1, wherein the step S3 comprises:
calculating lag time of the data to be processed according to the cross correlation between the data to be processed and the reference data;
diluting the data to be processed according to the time information, wherein partial data are missing after the data to be processed is diluted;
and moving the diluted data to be processed to be aligned with the reference data according to the lag time, and performing interpolation processing on missing data in the diluted data to be processed so as to synchronize the interpolated data to be processed with the reference data.
4. The method of claim 3, wherein the step of calculating the lag time of the data to be processed based on the cross-correlation between the data to be processed and the reference data comprises:
constructing a data matrix according to the data to be processed and the reference data, wherein a first column in the data matrix is the difference between adjacent data to be processed in the data to be processed, and a second column is the difference between adjacent reference data in the reference data;
calculating a cross-correlation value between the data to be processed and reference data according to the data matrix;
and calculating the lag time of the data to be processed according to the cross-correlation value.
5. A time synchronization apparatus, comprising:
the acquisition module is used for acquiring a data file acquired by the data acquisition instrument, wherein the data file has time information and comprises a data file containing a first parameter and a data file containing a second parameter;
the generating module is used for extracting and combining the data files containing the first parameters in the target time period according to the first parameters and the time information so as to generate data to be processed, and extracting and combining the data files containing the second parameters in the target time period according to the second parameters and the time information so as to generate reference data;
and the synchronization module is used for carrying out time synchronization processing on the data to be processed and the reference data by taking the time of the reference data as a reference.
6. The time synchronization module of claim 5, wherein the generation module comprises:
the to-be-processed data generation unit is used for extracting the data file containing the first parameter, updating the file name of the data file according to the first parameter and the time information, and merging the data files containing the first parameter in the target time period according to the file name to generate to-be-processed data;
and the reference data generating unit is used for extracting the data file containing the second parameter, updating the file name of the data file according to the second parameter and the time information, and merging the data files containing the second parameter in the target time period according to the file name to generate reference data.
7. The time synchronization module of claim 5, wherein the synchronization module comprises:
the calculation unit is used for calculating the lag time of the data to be processed according to the cross correlation between the data to be processed and the reference data;
the diluting unit is used for diluting the data to be processed according to time information, and part of the data is lost after the data to be processed is diluted;
and the synchronization unit is used for moving the diluted data to be processed to be aligned with the reference data according to the lag time and performing interpolation processing on missing data in the diluted data to be processed so as to synchronize the interpolated data to be processed with the reference data.
8. The time synchronization module of claim 7, wherein the computing unit comprises:
the matrix subunit is used for constructing a data matrix according to the data to be processed and the reference data, wherein a first column in the data matrix is a difference between adjacent data to be processed in the data to be processed, and a second column is a difference between adjacent reference data in the reference data;
the cross correlation subunit is used for calculating a cross correlation value between the data to be processed and the reference data according to the data matrix;
and the lag subunit is used for calculating the lag time of the data to be processed according to the cross-correlation value.
9. The utility model provides a test data synchronization system based on combination data acquisition appearance which characterized in that includes:
the data acquisition instrument is used for acquiring data files;
a time synchronising device as claimed in any one of claims 5 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910587443.8A CN112181906A (en) | 2019-07-02 | 2019-07-02 | Test data synchronization method, device and system based on combined data acquisition instrument |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910587443.8A CN112181906A (en) | 2019-07-02 | 2019-07-02 | Test data synchronization method, device and system based on combined data acquisition instrument |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112181906A true CN112181906A (en) | 2021-01-05 |
Family
ID=73914794
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910587443.8A Pending CN112181906A (en) | 2019-07-02 | 2019-07-02 | Test data synchronization method, device and system based on combined data acquisition instrument |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112181906A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114548058A (en) * | 2022-02-16 | 2022-05-27 | 东风商用车有限公司 | Intelligent driving data analysis and synchronization method based on ROS |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104142624A (en) * | 2014-08-15 | 2014-11-12 | 武汉大学 | Time synchronization method and system based on waveform matching |
| CN106484803A (en) * | 2016-09-22 | 2017-03-08 | 北京润科通用技术有限公司 | A kind of data analysing method and system |
| CN106547643A (en) * | 2015-09-21 | 2017-03-29 | 阿里巴巴集团控股有限公司 | The restoration methods and device of abnormal data |
| JP2018157363A (en) * | 2017-03-17 | 2018-10-04 | ヤマハ株式会社 | Data synchronization method |
-
2019
- 2019-07-02 CN CN201910587443.8A patent/CN112181906A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104142624A (en) * | 2014-08-15 | 2014-11-12 | 武汉大学 | Time synchronization method and system based on waveform matching |
| CN106547643A (en) * | 2015-09-21 | 2017-03-29 | 阿里巴巴集团控股有限公司 | The restoration methods and device of abnormal data |
| CN106484803A (en) * | 2016-09-22 | 2017-03-08 | 北京润科通用技术有限公司 | A kind of data analysing method and system |
| JP2018157363A (en) * | 2017-03-17 | 2018-10-04 | ヤマハ株式会社 | Data synchronization method |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114548058A (en) * | 2022-02-16 | 2022-05-27 | 东风商用车有限公司 | Intelligent driving data analysis and synchronization method based on ROS |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106406174B (en) | A kind of multimode multichannel collecting synchronization system and working method | |
| CN102122997B (en) | Method, device and terminal for detecting long term evolution (LTE) master synchronizing signal | |
| CN109936420B (en) | Method and apparatus for synchronizing data collected by multiple sensors | |
| CN103605023A (en) | Method and device for measuring merging unit time characteristics | |
| CN104517606A (en) | Method and device for recognizing and testing speech | |
| CN105989846B (en) | Multichannel voice signal synchronization method and device | |
| CN102594356B (en) | Realization method of synchronous collection | |
| ATE529763T1 (en) | PROCESSING OF SATELLITE NAVIGATION SYSTEM SIGNALS | |
| CN102571236B (en) | Method and system for measuring maximum time interval error | |
| CN106407119A (en) | Browser compatibility testing method based on automatic testing | |
| CN112181906A (en) | Test data synchronization method, device and system based on combined data acquisition instrument | |
| CN113766215A (en) | Airborne passenger cabin passenger broadcasting synchronous testing method and system | |
| CN104537059A (en) | Object ranking method and device | |
| CN114745514B (en) | Video signal expansion method, video signal expansion device, computer equipment and storage medium | |
| CN104297543A (en) | Hybrid oscilloscope with channel synchronization function | |
| CN113460061B (en) | Driving behavior analysis method and system | |
| CN112882069B (en) | Single-flow multi-index test evaluation method and system for satellite navigation terminal | |
| CN108663555A (en) | A kind of the waveform searching method and digital oscilloscope of digital oscilloscope | |
| CN116989973A (en) | Temporary impact type high-speed wind tunnel continuous variable attack angle pressure measurement test data processing method | |
| CN104977978A (en) | Acquiring and processing method for non-synchronizing signals of system clock | |
| CN110446103A (en) | A kind of phonotape and videotape test method, device and storage medium | |
| CN103199879A (en) | Digital receiver signal detection method | |
| CN107330317A (en) | A kind of terminal unlock method and terminal | |
| CN112362088A (en) | Synchronous acquisition method and system for multi-grating data | |
| CN104793031A (en) | Oscilloscope display zooming method and oscilloscope display zooming device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |