CN110245114B - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN110245114B CN110245114B CN201910549757.9A CN201910549757A CN110245114B CN 110245114 B CN110245114 B CN 110245114B CN 201910549757 A CN201910549757 A CN 201910549757A CN 110245114 B CN110245114 B CN 110245114B
- Authority
- CN
- China
- Prior art keywords
- message
- data
- waveform data
- group
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/148—File search processing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
The invention discloses a data processing method and a device, which search a target TDMS file from a technical data management stream TDMS file based on message playback instruction information; reading all message data stored in a target message data group; acquiring a channel name of message data selected by a user in a target message data group, and obtaining a message sequence number corresponding to the message data selected by the user by using the acquired channel name and a target message sequence number group index; and reading all waveform data under the channel name corresponding to the message serial number obtained by the index from the target waveform data group, and analyzing and then playing back and displaying the read waveform data. According to the technical scheme of the embodiment of the invention, all waveform data are not required to be read during waveform playback, other operations can be responded simultaneously, the waveform data is reselected after the user performs misoperation, the system is not blocked or crashed, the waiting time of the waveform data played back by the user is shortened, and the user experience is improved.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a data processing method and device.
Background
With the progress of science, various waveform data are generated in experiments in many fields, and because the information carried by the waveform data has important value, the waveform data are stored and played back when needed. In the process of replaying waveform data, source code data for storing the waveform data are required to be read firstly, the source code data are generally stored as a bin format file or a txt format file, a section of source codes with a certain length are sequentially intercepted from the source code data, the intercepted source codes are read again, then a message data packet is analyzed from the read source code data, the message data packet is displayed through a display, and finally, a section of source codes with a certain length is intercepted again according to the message data packet selected by a client in sequence to find the waveform data packet corresponding to the message data packet.
However, because the data size of the waveform data stored in the system is large and the operation capability of the computer is limited, when the waveform data is played back, the computer needs to read the stored data at full speed and parse the message data packet, and the computer cannot respond to other operations.
Disclosure of Invention
The embodiment of the invention provides a data processing method and device, which can partially or completely solve the problems mentioned in the background technology and improve the user experience.
In order to achieve the purpose, the invention provides the following technical scheme:
the first aspect of the embodiments of the present invention discloses a data processing method, which includes:
searching a target TDMS file from technical data management stream TDMS files based on the message playback instruction information, wherein each TDMS file comprises: the message data comprises a message sequence number group in which message sequence numbers are stored, a message data group in which message data are stored, and a waveform data group in which waveform data are stored; the message sequence number, the message data and the waveform data stored in each TDMS file are derived from the same bus message in the bus data; in each TDMS file, the message serial numbers in the message serial number group simultaneously correspond to the channel names of the message data group and the channel names of the waveform data group;
reading all message data stored in a target message data group; the target message data group is a message data group included in the target TDMS file;
acquiring a channel name of the message data selected by the user in the target message data group, and obtaining a message sequence number corresponding to the message data selected by the user by using the acquired channel name and the target message sequence number group index; the target message sequence number group is a message sequence number group included in the target TDMS file;
reading all waveform data under the channel name corresponding to the message serial number obtained by indexing from the target waveform data group, and analyzing and then playing back and displaying the read waveform data; and the target waveform data group is a waveform data group included in the target TDMS file.
Optionally, all TDMS files are obtained by pre-storing bus data, and the pre-storing process of all TDMS files includes:
acquiring bus data, and unpacking the bus data to obtain a message serial number, message data and waveform data of each bus message;
combining the message serial number and the message data of each bus message into a one-dimensional character string array and adding the one-dimensional character string array into a first storage queue, and combining the message serial number and the waveform data of each bus message into a one-dimensional U32 array and adding the one-dimensional U32 array into a second storage queue;
reading each one-dimensional character string array in the first storage queue, and indexing a message serial number in each one-dimensional character string array;
storing the message serial numbers in the message serial number group in the TDMS file;
storing the message data in the one-dimensional character string array in a message data group in a TDMS file, naming a channel name of each message data in the message data group according to a message serial number led out by a cable, wherein each message data and the channel name of the message data belong to the same bus message;
reading each one-dimensional U32 array in the second storage queue, and indexing a message sequence number in each one-dimensional U32 array;
storing the waveform data in the one-dimensional U32 array in a waveform data group in a TDMS file, naming the channel name of each waveform data in the waveform data group according to the message serial number led out by a cable, wherein each waveform data and the channel name of the waveform data belong to the same bus message;
inquiring whether the data volume of the waveform data stored in the TDMS file exceeds a preset value or not;
if yes, establishing new TDMS file storage bus data.
Optionally, the pre-storing process of all TDMS files further includes:
when storing the message serial number indexed out, the message data in the one-dimensional character string array, and the waveform data in the one-dimensional U32 array, the storing is performed once every time N message serial numbers, N message data, and N waveform data are accumulated.
Optionally, reading all message data stored in the target message data group, including:
acquiring the target message sequence number group, and reading all message sequence numbers in the target message sequence number group;
acquiring all message data corresponding to all message sequence numbers in the target message sequence number group from the target message data group;
and paging all the acquired message data, and displaying all the acquired message data on a display interface.
Optionally, reading all waveform data under the channel name corresponding to the message sequence number obtained by indexing from the target waveform data group, and analyzing and then playing back and displaying the read waveform data, including:
determining a channel name corresponding to the message sequence number obtained by indexing;
acquiring all waveform data under the determined channel name from the target waveform data group;
analyzing all the acquired waveform data, and displaying the waveform obtained after analyzing the waveform data on a display interface.
A second aspect of the embodiments of the present invention discloses a data processing apparatus, including:
an obtaining unit, configured to search a target TDMS file from technical data management stream TDMS files based on the message playback instruction information, where each TDMS file includes: the message data comprises a message sequence number group in which message sequence numbers are stored, a message data group in which message data are stored, and a waveform data group in which waveform data are stored; the message sequence number, the message data and the waveform data stored in each TDMS file are derived from the same bus message in the bus data; in each TDMS file, the message serial numbers in the message serial number group simultaneously correspond to the channel names of the message data group and the channel names of the waveform data group;
a reading unit, configured to read all message data stored in the target message data group; the target message data group is a message data group included in the target TDMS file;
the index unit is used for acquiring the channel name of the message data selected by the user in the target message data group and obtaining the message sequence number corresponding to the message data selected by the user by using the acquired channel name and the target message sequence number group index; the target message sequence number group is a message sequence number group included in the target TDMS file;
the display unit is used for reading all waveform data under the channel name corresponding to the message serial number obtained by the index from the target waveform data group, analyzing the read waveform data and then playing back and displaying the analyzed waveform data; and the target waveform data group is a waveform data group included in the target TDMS file.
Optionally, the method further includes: a pre-storing unit;
the pre-storing unit includes:
the unpacking subunit is used for acquiring the bus data and unpacking the bus data to obtain the message serial number, the message data and the waveform data of each bus message;
the combination subunit is used for combining the message serial number and the message data of each bus message into a one-dimensional character string array and adding the one-dimensional character string array into a first storage queue, and combining the message serial number and the waveform data of each bus message into a one-dimensional U32 array and adding the one-dimensional U32 array into a second storage queue;
the first indexing subunit is used for reading each one-dimensional character string array in the first storage queue and indexing a message serial number in each one-dimensional character string array;
the first storage subunit is used for storing the message serial numbers led out by the index into a message serial number group in the TDMS file;
the second storage subunit is used for storing the message data in the one-dimensional character string array into a message data group in the TDMS file, naming the channel name of each message data in the message data group according to the message serial number led out by a cable, and enabling each message data and the channel name of the message data to belong to the same bus message;
the second indexing subunit is used for reading each one-dimensional U32 array in the second storage queue and indexing a message sequence number in each one-dimensional U32 array;
the third storage subunit is used for storing the waveform data in the one-dimensional U32 array into a waveform data group in the TDMS file, naming the channel name of each waveform data in the waveform data group according to the message sequence number led out by a cable, and enabling each waveform data and the channel name of the waveform data to belong to the same bus message;
the query new subunit is used for querying whether the data volume of the waveform data stored in the TDMS file exceeds a preset value; if yes, establishing new TDMS file storage bus data.
Optionally, the pre-storing unit further includes: an accumulation subunit;
and the accumulation subunit is used for storing the indexed message sequence numbers, the message data in the one-dimensional character string array and the waveform data in the one-dimensional U32 array once when accumulating the N message sequence numbers, the N message data and the N waveform data.
Optionally, the reading unit includes:
a reading subunit, configured to obtain the target message sequence number group, and read all message sequence numbers in the target message sequence number group;
an obtaining subunit, configured to obtain, in the target message data group, all message data corresponding to all message sequence numbers in the target message sequence number group;
and the first display subunit is used for performing paging processing on all the acquired message data and displaying all the acquired message data on a display interface.
Optionally, the display unit includes:
the determining subunit is used for determining the channel name corresponding to the message sequence number obtained by the indexing;
an obtaining subunit, configured to obtain all waveform units under the determined channel name from the target waveform data group;
and the second display subunit is used for analyzing all the acquired waveform data and displaying the waveform obtained after analyzing the waveform data on the display interface.
According to the technical scheme, the embodiment of the invention discloses a data processing method and a device, which are used for searching a target TDMS file from a technical data management stream TDMS file based on message playback instruction information; reading all message data stored in a target message data group; the target message data group is a message data group included in the target TDMS file; acquiring a channel name of the message data selected by the user in the target message data group, and obtaining a message sequence number corresponding to the message data selected by the user by using the acquired channel name and the target message sequence number group index; and reading all waveform data under the channel name corresponding to the message serial number obtained by the index from the target waveform data group, and analyzing and then playing back and displaying the read waveform data. According to the technical scheme of the embodiment of the invention, when a user needs to play back waveform data, all message data stored in a target message data group under a target TDMS file corresponding to message playback instruction information can be directly read, after the user selects the message data, a message serial number corresponding to the message data selected by the user can be obtained by using a channel name of the selected message data and a target message serial number group index, the waveform data is obtained from the target waveform data group according to the message serial number obtained by the index, and finally the read waveform data is played back and displayed. Whole process need not to read whole waveform data, so system resource utilization rate is low, and on the one hand the computer can respond to other operations simultaneously, after the user maloperation, also only need reselect waveform data can, can not cause system card to die or system collapse, and on the other hand need not to read whole waveform data repeatedly also shorten user playback waveform data latency greatly to user experience effect has been improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a schematic flow chart of a data processing method according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating another data processing method according to an embodiment of the present invention;
FIG. 3 is a flow chart illustrating another data processing method according to an embodiment of the present invention;
FIG. 4 is a flow chart illustrating another data processing method according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a pre-storing unit in a data processing apparatus according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a reading unit in a data processing apparatus according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a display unit in a data processing apparatus according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the embodiments of the present invention, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The embodiment of the invention discloses a data processing method and a data processing device, and aims to solve the problem of poor user experience effect caused by the fact that when stored bus data is played back, the data volume of the stored waveform data is large, the stored source code data of all the waveform data needs to be read twice and intercepted according to a certain length sequence, and then the waveform data can be played back, so that almost all system resources need to be used when the waveform data is played back, a user is easy to cause system jamming or system crash when in misoperation, and the user needs to wait for a long time. In order to solve the above problem, an embodiment of the present invention discloses a data processing apparatus, which can be carried on an upper computer, a data analyzer, and other devices capable of implementing data processing.
An embodiment of the present invention provides a data processing method, and referring to fig. 1, the method at least includes the following steps:
step S101: and searching a target TDMS file from the technical data management stream TDMS file based on the message playback instruction information.
In step S101, each TDMS (Technical Data Management Streaming) file includes: the message data comprises a message sequence number group in which message sequence numbers are stored, a message data group in which message data are stored, and a waveform data group in which waveform data are stored; the message sequence number, the message data and the waveform data stored in each TDMS file are derived from the same bus message in the bus data; in each TDMS file, the message sequence numbers in the message sequence number group simultaneously correspond to the channel name of the message data group and the channel name of the waveform data group.
And the user selects a TDMS file corresponding to the waveform data to be played back from all the stored TDMS files, and after the user selection is finished, message playback instruction information is generated and used for representing the playback path of the TDMS file selected by the user, and the target TDMS file is the TDMS file selected by the user. In practical applications, the number of TDMS files selected by the user at a time may be different, such as 1, 2, 3, 4, 5, and so on.
The TDMS file records the channel names in a binary manner, that is, the channel names of the message data group and the channel names of the waveform data group are recorded in a binary manner.
It should be noted that, because the bus data is composed of a plurality of bus messages, and after each bus message is unpacked, the message sequence number, the message data, and the waveform data of each bus message may be obtained, and in the plurality of bus messages, the message data and the waveform data may be the same, but the message sequence number of each bus message is unique. When the bus message is stored in the TDMS file, since tag information such as a group name, a channel name, and the like can be set in the TDMS file, the message data and the waveform data can be stored in the manner of "message sequence number + message data" and "message sequence number + waveform data". Specifically, the message data and the waveform data are combined respectively through the unique message serial number to obtain combined message data and combined waveform data, and then the combined message data and the combined waveform data are stored.
In order to facilitate reading of the stored waveform data at a later stage, when the combined message data and the combined waveform data are stored, the message sequence numbers are respectively stored as index groups, and when the waveform data need to be acquired, the message data corresponding to the message sequence numbers need to be read through the index message sequence numbers, and then the waveform data corresponding to the message sequence numbers are read through the message data index message sequence numbers selected by a user.
The specific storing process of the combined message data is to index the message sequence number of the combined message data, use the message sequence number as the data of the message sequence number group and the channel name of the message data group at the same time, and store the message data in the TDMS file as the data under the channel name corresponding to the message data group. Wherein, here correspond and mean: in the message data group, the message sequence number used by the channel name of any message data and the message data are derived from the same bus message.
The specific storage process of the combined waveform data is to index the message serial number of the combined waveform data, take the message serial number in the message serial number group as the channel name of the waveform data group, and store the waveform data in the TDMS file as the data in the channel name corresponding to the waveform data group. Wherein, here correspond and mean: in the waveform data group, the message sequence number used by the channel name of any waveform data and the waveform data itself are derived from the same bus message.
Through the storage process of the combined message data and the combined waveform data, the message sequence number, the message data and the waveform data stored in each TDMS file are ensured to be derived from the same bus message in the bus data. Thus, the message sequence numbers in the message sequence number group, the message data in the message data group, and the waveform data in the waveform data group in each TDMS file are identical in number and are in one-to-one correspondence.
Step S102: and reading all message data stored in the target message data group.
It should be noted that the target message data group is a message data group included in the target TDMS file. Therefore, after the target TDMS file is found, the target message data group can be found in the target TDMS file.
Since the message data is stored in the target message data group, all the message data stored in the target message data group can be read by executing step S102.
Step S103: and acquiring the channel name of the message data selected by the user in the target message data group, and acquiring the message sequence number corresponding to the message data selected by the user by using the acquired channel name and the target message sequence number group index.
In step S103, the target message sequence number group is a message sequence number group included in the target TDMS file.
It should be noted that, in each TDMS file, the message sequence number in the message sequence number group corresponds to the channel name of the message data group and the channel name of the waveform data group at the same time, and the message data is data in the target message data group, so that the channel name of the message data selected by the user in the target message data group is obtained, and the message sequence number corresponding to the message data selected by the user can be obtained by using the obtained channel name and the target message sequence group index.
Step S104: and reading all waveform data under the channel name corresponding to the message serial number obtained by the index from the target waveform data group, and analyzing and then playing back and displaying the read waveform data.
In step S104, the target waveform data group is a waveform data group included in the target TDMS file.
It should be noted that, in each TDMS file, the message sequence number in the message sequence number group corresponds to the channel name of the message data group and the channel name of the waveform data group at the same time, so that the waveform data under the channel name corresponding to the message sequence number can be read in the waveform data group through the message sequence number. Since the target waveform data is stored in the target waveform data group, all the waveform data in the channel name corresponding to the message sequence number indexed in step S103 can be acquired from the target waveform data group. And then playback display is performed after the read waveform data is parsed.
The embodiment of the invention searches a target TDMS file from technical data management stream TDMS files based on the message playback instruction information; reading all message data stored in a target message data group; acquiring a channel name of message data selected by a user in a target message data group, and obtaining a message sequence number corresponding to the message data selected by the user by using the acquired channel name and a target message sequence number group index; and reading all waveform data under the channel name corresponding to the message serial number obtained by the index from the target waveform data group, and analyzing and then playing back and displaying the read waveform data. According to the technical scheme of the embodiment of the invention, when a user needs to play back waveform data, all message data stored in a target message data group under a target TDMS file corresponding to message playback instruction information can be directly read, after the user selects the message data, a message serial number corresponding to the message data selected by the user can be obtained by using a channel name of the selected message data and a target message serial number group index, the waveform data is obtained from the target waveform data group according to the message serial number obtained by the index, and finally the read waveform data is played back and displayed. Whole process need not to read whole waveform data, so system resource utilization rate is low, and on the one hand the computer can respond to other operations simultaneously, after the user maloperation, also only need reselect waveform data can, can not cause system card to die or system collapse, and on the other hand need not to read whole waveform data repeatedly also shorten user playback waveform data latency greatly to user experience effect has been improved.
As shown in fig. 2, in the data processing method disclosed in the above embodiment, all TDMS files are obtained by pre-storing bus data, and the process of pre-storing all TDMS files includes the following steps:
step S201: and acquiring bus data, and unpacking the bus data to obtain the message serial number, the message data and the waveform data of each bus message.
It should be noted that unpacking bus data is a conventional technique. Since the bus data is composed of a plurality of messages, each bus message includes a message sequence number, message data, and waveform data. Therefore, after the unpacking processing is carried out on the bus data, the message sequence number, the message data and the waveform data of each bus message can be obtained. Each message sequence number is unique among the plurality of bus messages.
Step S202: and combining the message serial number and the message data of each bus message into a one-dimensional character string array and adding the one-dimensional character string array into a first storage queue, and combining the message serial number and the waveform data of each bus message into a one-dimensional U32 array and adding the one-dimensional U32 array into a second storage queue.
It should be noted that the one-dimensional string array and the one-dimensional U32 array are both data types, the one-dimensional string array represents sequential combinations of a plurality of strings, and the one-dimensional U32 array represents sequential combinations of 32-bit unsigned data.
And the message serial number and the waveform data of each bus message are combined into a one-dimensional U32 array to be used as combined waveform data.
In the step, the message serial number and the message data of each bus message are combined into a one-dimensional character string array and added into a first storage queue, and the message serial number and the waveform data of each bus message are combined into a one-dimensional U32 array and added into a second storage queue for storage.
Step S203: and reading each one-dimensional character string array in the first storage queue, and indexing a message sequence number in each one-dimensional character string array.
It should be noted that, the message sequence number and the message data are combined into a one-dimensional string array in the first storage queue, and the entire one-dimensional string array is stored when the one-dimensional string array is stored, so that each one-dimensional string array in the first storage queue needs to be read first, and during reading, the message sequence number is indexed out from each one-dimensional string array.
Step S204: and storing the message sequence numbers indexed out in a message sequence number group in the TDMS file.
It should be noted that, when reading the message data in the later period, it is necessary to use the message number in the message number group to read the waveform data under the channel name corresponding to the waveform data group, so that, during storage, the message number retrieved in step S203 should be stored in the message number group in the TDMS file.
Step S205: and storing the message data in the one-dimensional character string array in a message data group in a TDMS file, naming the channel name of each message data in the message data group according to the message sequence number led out by a cable, wherein each message data and the channel name of the message data belong to the same bus message.
It should be noted that, in order to ensure that the waveform data that the user needs to play back can be found according to the message data selected by the user at the later stage, it is necessary to store the message data in the message data group in the TDMS file, name the channel name of each message data in the message data group according to the message sequence number retrieved in step S203, and each message data and its channel name belong to the same bus message.
Step S206: each one-dimensional U32 array in the second storage queue is read and a message sequence number is indexed in each one-dimensional U32 array.
It should be noted that, the message sequence number and the waveform data are added in the second storage queue as a one-dimensional U32 array, and when the one-dimensional U32 array is stored, it is necessary to read each one-dimensional U32 array in the second storage queue and index a message sequence number in each one-dimensional U32 array.
Step S207: waveform data in the one-dimensional U32 array is stored in a waveform data group in a TDMS file, a channel name of each waveform data in the waveform data group is named according to a message sequence number led out by a cable, and each waveform data and the channel name of the waveform data belong to the same bus message.
It should be noted that, in each TDMS file, the message sequence number in the message sequence number group corresponds to the channel name of the message data group and the channel name of the waveform data group at the same time, and when the waveform data is read, the waveform data under the corresponding channel name in the waveform data group is read according to the message sequence number corresponding to the channel name of the message data selected by the user in the message data group, so when the waveform data in the one-dimensional U32 array is stored in the waveform data group in the TDMS file, the channel name of each waveform data in the waveform data group is named according to the message sequence number retrieved in step S206, so that each waveform data itself and the waveform data channel name belong to the same bus message.
Step S208: and inquiring whether the data volume of the waveform data stored in the TDMS file exceeds a preset value, if not, executing step S209, and if so, executing step S210.
When storing the combined waveform data, the waveform data in the one-dimensional U32 array is stored in the waveform data group, and since the waveform data of the waveform data group has a large data size, if the waveform data of the waveform data group is stored in one TDMS file, the TDMS file is too large, and when playing back the waveform data in the TDMS file, the waiting time for playing back the waveform data is long due to the large data size, which is inconvenient for reading. Therefore, in order to make the file size suitable and facilitate reading, the amount of waveform data stored in each TDMS file can be controlled. Specifically, when storing the waveform data, it may be first queried whether the amount of waveform data in the TDMS file exceeds a preset value. In practical application, the preset value is set by taking convenient reading as a standard.
Step S209: the waveform data is stored in a TDMS file.
Step S210: and establishing new TDMS file storage bus data.
It should be noted that, because the amount of waveform data stored in each TDMS file is limited, and the waveform data needs to be stored completely to form the integrity of the waveform data, it is necessary to establish bus data stored in a new TDMS file, so as to ensure the integrity of the waveform data, and when the waveform data is played back, it is avoided that a user waits for a long time due to an excessively large amount of waveform data.
Optionally, the pre-storing process of all TDMS files may further include:
when storing the message serial number indexed out, the message data in the one-dimensional character string array, and the waveform data in the one-dimensional U32 array, the storing is performed once every time N message serial numbers, N message data, and N waveform data are accumulated. In practical application, the value of N can be flexibly set according to practical requirements, for example, the value can be set to 100, 200, and the like.
It should be noted that, after accumulating the N message sequence numbers, the N message data, and the N waveform data, the storage is performed once, which can improve the storage efficiency of the TDMS file.
According to the data processing method disclosed by the embodiment of the invention, the waveform data and the message data in the bus data are combined through the message serial numbers and then are respectively stored, and the message serial numbers are used as the channel names of the combined waveform data and the combined message data during storage, so that the storage efficiency of the bus data is improved.
Based on the data processing method disclosed in the foregoing embodiment, a specific implementation process of reading all message data stored in the target message data group in step S102 shown in fig. 1, as shown in fig. 3, includes the following steps:
step S301: and acquiring a target message sequence number group, and reading all message sequence numbers in the target message sequence number group.
It should be noted that, because the TDMS file includes the message sequence number group, after the target TDMS file is queried, the target message sequence number group can be found in the target TDMS file. And the target message sequence number group comprises a plurality of message sequence numbers, so that all message sequence numbers in the target message sequence number group can be read after the target message sequence number group is obtained.
Step S302: and acquiring all message data corresponding to all message sequence numbers in the target message sequence number group from the target message data group.
It should be noted that, in each TDMS file, the message sequence numbers in the message sequence number group correspond to the channel name of the message data group and the channel name of the waveform data group at the same time, and when storing the message data, the message sequence numbers are indexed and stored, so that all the message data under the channel name corresponding to each message sequence number can be found in the target message data group through all the message sequence numbers in the target message sequence number group.
Step S303: and paging all the acquired message data, and displaying all the acquired message data on a display interface.
It should be noted that the paging process is to sort all the message data acquired in step S302, and display all the sorted message data through a page, and on the display interface, the position information displayed on the page of each message data is unique, that is, one display position corresponds to one message data.
The embodiment of the invention reads all message sequence numbers in the target message sequence number group by acquiring the target message sequence number group; acquiring all message data corresponding to all message sequence numbers in the target message sequence number group from the target message data group; and paging all the acquired message data, and displaying all the acquired message data on a display interface. Through the data processing method disclosed above, all message data stored in the target message data group are acquired according to the playback path selected by the user, and finally displayed on the display interface.
Based on the data processing method disclosed in the foregoing embodiment, a specific implementation procedure shown in fig. 1, in which a message sequence number obtained by reading an index in a target waveform data group in step S104 corresponds to all waveform data under a channel name, and the read waveform data is analyzed and then played back for display, as shown in fig. 4, includes the following steps:
step S401: and determining the channel name corresponding to the message sequence number obtained by indexing.
It should be noted that, in each TDMS file, the message sequence numbers in the message sequence number group are simultaneously corresponding to the channel name of the message data group and the channel name of the waveform data group, so that waveform data needs to be obtained, and after the message sequence numbers are obtained, the channel name corresponding to the message sequence number in the target message sequence number group, that is, the channel name corresponding to the message sequence number, should be determined.
Step S402: and acquiring all waveform data under the determined channel name from the target waveform data group.
In each TDMS file, the message sequence numbers in the message sequence number group simultaneously correspond to the channel names of the message data group and the channel names of the waveform data group, so that the channel names of the waveform data group can be determined according to the channel names corresponding to the message sequence numbers. When storing the waveform data, the waveform data is stored under the channel name of the waveform data group, so that all the waveform data under the channel name can be acquired through the channel name of the target waveform data group.
Step S403: analyzing all the acquired waveform data, and displaying the waveform obtained after analyzing the waveform data on a display interface.
It should be noted that, the waveform data is stored in a txt format file or a bin format file in a source code manner, so that the waveform data needs to be analyzed to obtain a waveform, and the waveform is presented to a user.
The embodiment of the invention determines the channel name corresponding to the message sequence number obtained by indexing; acquiring all waveform data under the determined channel name from the target waveform data group; analyzing all the acquired waveform data, and displaying the waveform obtained after analyzing the waveform data on a display interface. According to the data processing method disclosed by the above, all waveform data under the channel name are determined from the target waveform data group according to the channel name corresponding to the index obtained message serial number, and the waveform data are analyzed and displayed. When the waveform data is played back by a user, the waiting time of the data played back by the user is shortened, and therefore the user experience effect is improved.
Corresponding to the data processing method provided by the embodiment of the present invention, an embodiment of the present invention further provides a corresponding data processing apparatus, as shown in fig. 5, which is a schematic structural diagram of a data processing apparatus disclosed by the embodiment of the present invention, where the data processing apparatus includes:
an obtaining unit 501, configured to search a target TDMS file from technical data management stream TDMS files based on the message playback instruction information, where each TDMS file includes: the message data comprises a message sequence number group in which message sequence numbers are stored, a message data group in which message data are stored, and a waveform data group in which waveform data are stored; the message sequence number, the message data and the waveform data stored in each TDMS file are derived from the same bus message in the bus data; in each TDMS file, the message serial numbers in the message serial number group simultaneously correspond to the channel names of the message data group and the channel names of the waveform data group;
a reading unit 502, configured to read all message data stored in the target message data group; the target message data group is a message data group included in the target TDMS file;
an indexing unit 503, configured to acquire a channel name of the message data selected by the user in the target message data group, and obtain a message sequence corresponding to the message data selected by the user by using the acquired channel name and the target message sequence group index; the target message sequence number group is a message sequence number group included in the target TDMS file;
a display unit 504, configured to read all waveform data under the channel name corresponding to the message sequence number obtained by indexing from the target waveform data group, and analyze and playback-display the read waveform data; and the target waveform data group is a waveform data group included in the target TDMS file.
Optionally, as shown in fig. 6, the data processing apparatus further includes: a pre-storing unit;
the pre-storing unit comprises:
the unpacking subunit 601 is a unpacking unit and is used for acquiring bus data and unpacking the bus data to obtain a message sequence number, message data and waveform data of each bus message;
a combining subunit 602, configured to combine the message sequence number and the message data of each bus message into a one-dimensional string array and add the one-dimensional string array to a first storage queue, and combine the message sequence number and the waveform data of each bus message into a one-dimensional U32 array and add the one-dimensional U32 array to a second storage queue;
a first indexing subunit 603, configured to read each one-dimensional string array in the first storage queue, and index a message sequence number in each one-dimensional string array;
a first storing subunit 604, configured to create the message sequence number group by using the retrieved target message sequence number;
a second storage subunit 605, configured to store the message data in the one-dimensional character string array in a message data group in the TDMS file, and name a channel name of each message data in the message data group according to a message sequence number extracted by a cable, where each message data and the channel name of the message data belong to the same bus message;
a second indexing subunit 606, configured to read each one-dimensional U32 array in the second storage queue, and index a message sequence number in each one-dimensional U32 array;
a third storage subunit 607, configured to store the waveform data in the one-dimensional U32 array in a waveform data group in the TDMS file, and name a channel name of each waveform data in the waveform data group according to a message sequence number indexed by a cable, where each waveform data and its channel name belong to the same bus message;
a query new subunit 608, configured to query whether the data size of the waveform data stored in the TDMS file exceeds a preset value; if yes, establishing new TDMS file storage bus data.
Optionally, in the data processing apparatus, the pre-storing unit further includes: an accumulation subunit;
and the accumulation subunit is used for storing the indexed message sequence numbers, the message data in the one-dimensional character string array and the waveform data in the one-dimensional U32 array once when accumulating the N message sequence numbers, the N message data and the N waveform data.
Optionally, as shown in fig. 7, the reading unit 502 includes:
a reading subunit 701, configured to obtain the target message sequence number group, and read all message sequence numbers in the target message sequence number group;
an obtaining subunit 702, configured to obtain, in the target message data group, all message data corresponding to all message sequence numbers in the target message sequence number group;
the first display subunit 703 is configured to perform paging processing on all acquired message data, and display all acquired message data on a display interface.
Optionally, as shown in fig. 8, the display unit 504 includes:
a determining subunit 801, configured to determine a channel name corresponding to the message sequence number obtained by indexing;
an obtaining subunit 802, configured to obtain all waveform data under the determined channel name from the target waveform data group;
and a second display subunit 803, configured to analyze all the acquired waveform data, and display a waveform obtained after analyzing the waveform data on a display interface.
For a specific execution process and an execution principle of each module unit in the data processing apparatus disclosed in the above embodiment of the present invention, reference may be made to corresponding parts related to data processing in the data processing method disclosed in the above embodiment of the present invention, and details are not described here.
According to the embodiment of the invention, the acquisition unit searches a target TDMS file from technical data management stream TDMS files based on the message playback instruction information; the reading unit reads all message data stored in the target message data group; the indexing unit acquires a channel name of the message data selected by the user in the target message data group, and obtains a message sequence number corresponding to the message data selected by the user by using the acquired channel name and the target message sequence number group index; and the display unit reads all waveform data under the channel name corresponding to the message serial number obtained by indexing from the target waveform data group, and plays back and displays the read waveform data after analyzing the read waveform data. According to the technical scheme of the embodiment of the invention, when a user needs to play back waveform data, all message data stored in a target message data group under a target TDMS file corresponding to message playback instruction information can be directly read, after the user selects the message data, a message serial number corresponding to the message data selected by the user can be obtained by using a channel name of the selected message data and a target message serial number group index, the waveform data is obtained from the target waveform data group according to the message serial number obtained by the index, and finally the read waveform data is played back and displayed. Whole process need not to read whole waveform data, so system resource utilization rate is low, and on the one hand the computer can respond to other operations simultaneously, after the user maloperation, also only need reselect waveform data can, can not cause system card to die or system collapse, and on the other hand need not to read whole waveform data repeatedly also shorten user playback waveform data latency greatly to user experience effect has been improved.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus or apparatus embodiments, since they are substantially similar to the method embodiments, they are described relatively simply, and reference may be made to some descriptions of the method embodiments for related points. The above-described apparatuses and apparatus embodiments are merely illustrative, wherein the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method of data processing, the method comprising:
searching a target TDMS file from technical data management stream TDMS files based on the message playback instruction information, wherein each TDMS file comprises: the message data comprises a message sequence number group in which message sequence numbers are stored, a message data group in which message data are stored, and a waveform data group in which waveform data are stored; the message sequence number, the message data and the waveform data stored in each TDMS file are derived from the same bus message in the bus data; in each TDMS file, the message serial numbers in the message serial number group simultaneously correspond to the channel names of the message data group and the channel names of the waveform data group; wherein the message data, the message sequence number and the waveform data contained in one of the TDMS files are different from each other;
reading all message data stored in a target message data group, wherein the target message data group is a message data group included in the target TDMS file;
acquiring a channel name of the message data selected by the user in the target message data group, and acquiring the message sequence number of the target message sequence group corresponding to the message data selected by the user by using the acquired channel name; the target message sequence number group is a message sequence number group included in the target TDMS file;
reading all waveform data under the channel name corresponding to the message serial number obtained by indexing from the target waveform data group, and analyzing and then playing back and displaying the read waveform data; and the target waveform data group is a waveform data group included in the target TDMS file.
2. The method of claim 1, wherein all TDMS files are obtained by pre-storing bus data, and the pre-storing process of all TDMS files comprises:
acquiring bus data, and unpacking the bus data to obtain a message serial number, message data and waveform data of each bus message;
combining the message serial number and the message data of each bus message into a one-dimensional character string array and adding the one-dimensional character string array into a first storage queue, and combining the message serial number and the waveform data of each bus message into a one-dimensional U32 array and adding the one-dimensional U32 array into a second storage queue;
reading each one-dimensional character string array in the first storage queue, and indexing a message serial number in each one-dimensional character string array;
storing the message serial numbers in the message serial number group in the TDMS file;
storing the message data in the one-dimensional character string array in a message data group in a TDMS file, naming a channel name of each message data in the message data group according to a message serial number led out by a cable, wherein each message data and the channel name of the message data belong to the same bus message;
reading each one-dimensional U32 array in the second storage queue, and indexing a message sequence number in each one-dimensional U32 array;
storing the waveform data in the one-dimensional U32 array in a waveform data group in a TDMS file, naming the channel name of each waveform data in the waveform data group according to the message serial number led out by a cable, wherein each waveform data and the channel name of the waveform data belong to the same bus message;
inquiring whether the data volume of the waveform data stored in the TDMS file exceeds a preset value or not;
if yes, establishing new TDMS file storage bus data.
3. The method of claim 2, wherein the pre-storing procedure of all TDMS files further comprises:
when storing the message serial number indexed out, the message data in the one-dimensional character string array, and the waveform data in the one-dimensional U32 array, the storing is performed once every time N message serial numbers, N message data, and N waveform data are accumulated.
4. The method according to any one of claims 1-3, wherein reading all message data stored in a target message data group comprises:
acquiring the target message sequence number group, and reading all message sequence numbers in the target message sequence number group;
acquiring all message data corresponding to all message sequence numbers in the target message sequence number group from the target message data group;
and paging all the acquired message data, and displaying all the acquired message data on a display interface.
5. The method according to claim 1, wherein the step of reading the index from the target waveform data group to obtain the message sequence number corresponding to all waveform data under the channel name, and analyzing and then playing back and displaying the read waveform data comprises:
determining a channel name corresponding to the message sequence number obtained by indexing;
acquiring all waveform data under the determined channel name from the target waveform data group;
analyzing all the acquired waveform data, and displaying the waveform obtained after analyzing the waveform data on a display interface.
6. A data processing apparatus, comprising:
an obtaining unit, configured to search a target TDMS file from technical data management stream TDMS files based on the message playback instruction information, where each TDMS file includes: the message data comprises a message sequence number group in which message sequence numbers are stored, a message data group in which message data are stored, and a waveform data group in which waveform data are stored; the message sequence number, the message data and the waveform data stored in each TDMS file are derived from the same bus message in the bus data; in each TDMS file, the message serial numbers in the message serial number group simultaneously correspond to the channel names of the message data group and the channel names of the waveform data group; wherein the message data, the message sequence number and the waveform data contained in one of the TDMS files are different from each other;
a reading unit, configured to read all message data stored in the target message data group; the target message data group is a message data group included in the target TDMS file;
the index unit is used for acquiring the channel name of the message data selected by the user in the target message data group and acquiring the message sequence number of the target message sequence group corresponding to the message data selected by the user by using the acquired channel name; the target message sequence number group is a message sequence number group included in the target TDMS file;
the display unit is used for reading all waveform data under the channel name corresponding to the message serial number obtained by the index from the target waveform data group, analyzing the read waveform data and then playing back and displaying the analyzed waveform data; and the target waveform data group is a waveform data group included in the target TDMS file.
7. The apparatus of claim 6, further comprising: a pre-storing unit;
the pre-storing unit includes:
the unpacking subunit is used for acquiring the bus data and unpacking the bus data to obtain the message serial number, the message data and the waveform data of each bus message;
the combination subunit is used for combining the message serial number and the message data of each bus message into a one-dimensional character string array and adding the one-dimensional character string array into a first storage queue, and combining the message serial number and the waveform data of each bus message into a one-dimensional U32 array and adding the one-dimensional U32 array into a second storage queue;
the first indexing subunit is used for reading each one-dimensional character string array in the first storage queue and indexing a message serial number in each one-dimensional character string array;
the first storage subunit is used for storing the message serial numbers led out by the index into a message serial number group in the TDMS file;
the second storage subunit is used for storing the message data in the one-dimensional character string array into a message data group in the TDMS file, naming the channel name of each message data in the message data group according to the message serial number led out by a cable, and enabling each message data and the channel name of the message data to belong to the same bus message;
the second indexing subunit is used for reading each one-dimensional U32 array in the second storage queue and indexing a message sequence number in each one-dimensional U32 array;
the third storage subunit is used for storing the waveform data in the one-dimensional U32 array into a waveform data group in the TDMS file, naming the channel name of each waveform data in the waveform data group according to the message sequence number led out by a cable, and enabling each waveform data and the channel name of the waveform data to belong to the same bus message;
the query new subunit is used for querying whether the data volume of the waveform data stored in the TDMS file exceeds a preset value; if yes, establishing new TDMS file storage bus data.
8. The apparatus of claim 7, wherein the pre-storing unit further comprises: an accumulation subunit;
and the accumulation subunit is used for storing the indexed message sequence numbers, the message data in the one-dimensional character string array and the waveform data in the one-dimensional U32 array once when accumulating the N message sequence numbers, the N message data and the N waveform data.
9. The apparatus according to any one of claims 6-8, wherein the reading unit comprises:
a reading subunit, configured to obtain the target message sequence number group, and read all message sequence numbers in the target message sequence number group;
an obtaining subunit, configured to obtain, in the target message data group, all message data corresponding to all message sequence numbers in the target message sequence number group;
and the first display subunit is used for performing paging processing on all the acquired message data and displaying all the acquired message data on a display interface.
10. The apparatus of claim 6, wherein the display unit comprises:
the determining subunit is used for determining the channel name corresponding to the message sequence number obtained by the indexing;
an obtaining subunit, configured to obtain all waveform units under the determined channel name from the target waveform data group;
and the second display subunit is used for analyzing all the acquired waveform data and displaying the waveform obtained after analyzing the waveform data on the display interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910549757.9A CN110245114B (en) | 2019-06-24 | 2019-06-24 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910549757.9A CN110245114B (en) | 2019-06-24 | 2019-06-24 | Data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110245114A CN110245114A (en) | 2019-09-17 |
| CN110245114B true CN110245114B (en) | 2021-10-15 |
Family
ID=67889062
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910549757.9A Active CN110245114B (en) | 2019-06-24 | 2019-06-24 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110245114B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114648006B (en) * | 2022-03-09 | 2024-06-07 | 昆明理工大学 | Data processing method and device based on time alignment |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8098690B2 (en) * | 2008-03-18 | 2012-01-17 | Transwitch Corporation | System and method for transferring high-definition multimedia signals over four twisted-pairs |
| CN102520269A (en) * | 2011-11-23 | 2012-06-27 | 上海市电力公司 | Data collecting and analyzing device for energy storage system and collecting and analyzing method thereof |
| CN103699699B (en) * | 2014-01-16 | 2016-09-28 | 常州工学院 | One browses decoding CDR file method |
-
2019
- 2019-06-24 CN CN201910549757.9A patent/CN110245114B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110245114A (en) | 2019-09-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107885873B (en) | Method and apparatus for outputting information | |
| CN103150397B (en) | A kind of data directory creation method, data retrieval method and system | |
| CN103353821A (en) | System and method for managing desktop application program icons of mobile terminal | |
| CN104104971A (en) | Video file processing method and system | |
| CN109857901B (en) | Information display method and device, and method and device for information search | |
| KR101682659B1 (en) | Method for customized news alarm based on keyword and management server for news search for the same | |
| US20240054157A1 (en) | Song recommendation method and apparatus, electronic device, and storage medium | |
| CN105718578A (en) | Short link generation method and device | |
| RU2568276C2 (en) | Method of extracting useful content from mobile application setup files for further computer data processing, particularly search | |
| WO2017107679A1 (en) | Historical information display method and apparatus | |
| CN103237136A (en) | Mobile terminal and searching method of subject headings of mobile terminal | |
| CN110245114B (en) | Data processing method and device | |
| CN107562372B (en) | Electronic equipment, data management method thereof and computer readable storage medium | |
| CN105991583A (en) | Game application recommendation method, application server, terminal and system | |
| CN112231317A (en) | Data query method and device, electronic device and storage medium | |
| CN105760436B (en) | The processing method and processing device of audio data | |
| CN111538645A (en) | Data visualization method and related equipment | |
| CN107193922B (en) | A kind of method and device of information processing | |
| CN110287338B (en) | Industry hotspot determination method, device, equipment and medium | |
| CN112749127A (en) | Data providing method and system for model training | |
| JP2004185214A (en) | File management device | |
| CN112671878A (en) | Block chain information subscription method, device, server and storage medium | |
| CN109376174B (en) | Method and device for selecting database | |
| CN109558468B (en) | Resource processing method, device, equipment and storage medium | |
| WO2017076027A1 (en) | Wallpaper processing method and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |