CN107992624A - Parse method, apparatus, storage medium and the terminal device of serialized data - Google Patents
Parse method, apparatus, storage medium and the terminal device of serialized data Download PDFInfo
- Publication number
- CN107992624A CN107992624A CN201711407433.9A CN201711407433A CN107992624A CN 107992624 A CN107992624 A CN 107992624A CN 201711407433 A CN201711407433 A CN 201711407433A CN 107992624 A CN107992624 A CN 107992624A
- Authority
- CN
- China
- Prior art keywords
- data
- function
- serializing
- text
- serialized
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Fuzzy Systems (AREA)
- Stored Programmes (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention proposes a kind of method, apparatus, storage medium and terminal device for parsing serialized data, wherein, the described method includes：The unserializing request of response parsing serializing text, the serializing for obtaining the serializing text obtain function；Wherein, the serializing, which obtains function, includes data structure and the mapping relations of Data Identification, and the data structure is used to define the data structure of the serialized data corresponding to the Data Identification of the data structure mapping；Function is obtained by the serializing, obtains the data structure of the Data Identification mapping of the serialized data of pending unserializing operation；And the data structure according to acquisition, unserializing operation is performed to the serialized data of the pending unserializing operation.Using the present invention, it is possible to increase parse the efficiency of serialized data.
Description
Technical field
The present invention relates to field of communication technology, more particularly to a kind of method for down loading, device, storage for parsing serialized data
Medium and terminal device.
Background technology
Protobuf be that Google increases income it is a be similar to Json, XML data exchange format, interior data is pure two
System form, independent of language and platform, has simple, and data volume is small, it is quick the advantages that.Based on Protobuf formatted files
Have the advantages that above-mentioned, daily record for logging system operating condition generally carries out storage or transmission, letter using Protobuf forms
Claim PB daily records.Protobuf formatted files are the serializing numbers that string of binary characters is formed, it is unreadable file, thus
, it is necessary to by the PB daily records unserializing of serializing into the readable data type of text data etc when reading PB daily records.
It is daily to the reading of PB daily records or the mode of inquiry, generally by the first number for the access information for describing the PB daily records
According to obtain the Proto configuration files of PB daily records, daily record analytics engine reads task in each performed to PB daily record datas
In, dynamic directly acquires the class compiling information of Proto configuration files, and utilize sequence to Proto configuration files into serializing
The Proto configuration files changed carry out unserializing to PB daily record datas, so as to get the tool in the data after unserializing
Hold in vivo, complete the reading task.Its this programme advantage is versatility height, and the degree of coupling is low, it is not necessary to Proto configuration files
Digital independent can be completed by carrying out compiling in advance.But due to needing dynamic to serialize configuration file, cause to PB
The process that daily record carries out unserializing is time-consuming long, inefficiency.
The content of the invention
The embodiment of the present invention provides a kind of method, system, device, storage medium and terminal for parsing serialized data and sets
It is standby, improve the efficiency of parsing serialized data.
In a first aspect, an embodiment of the present invention provides it is a kind of parse serialized data method, the described method includes：
The unserializing request of response parsing serializing text, the serializing for obtaining the serializing text obtain function；
Wherein, the serializing, which obtains function, includes data structure and the mapping relations of Data Identification, and the data structure is used to define
The data structure of serialized data corresponding to the Data Identification of the data structure mapping；
Function is obtained by the serializing, the Data Identification for obtaining the serialized data of pending unserializing operation reflects
The data structure penetrated；And
According to the data structure of acquisition, unserializing behaviour is performed to the serialized data of the pending unserializing operation
Make.
With reference to first aspect, the present invention is obtaining the serializing text in the first embodiment of first aspect
Serializing obtain function before, further include：
Obtain the configuration file of the serializing text；Wherein, the configuration file is used to define the serializing text
The data structure of middle serialized data；
According to the configuration file, the acquisition function for serializing text is generated；And
To the acquisition function performing sequenceization operation, formation sequenceization obtains function.
With reference to first aspect, the present invention is described according to the configuration file in second of embodiment of first aspect,
The acquisition function of the serializing text is generated, including：
Extraction is right for the data structure institute for defining serialized data in the serializing text from the configuration file
The parent Data Identification answered is as the function name for obtaining function；
Compile the configuration file, generation configuration source code；And
According to the configuration source code and the function name for obtaining function, the acquisition letter of the configuration generation serializing text
Number；Wherein, the source code that the acquisition function includes being used for the data structure of defined nucleotide sequence data in the configuration source code is with being somebody's turn to do
Mapping relations between the source code of the Data Identification of serialized data
With reference to first aspect, the present invention is described that the acquisition function is held in the third embodiment of first aspect
Row serializing operation, including：
Call and be used for the serializing instrument that performing sequenceization operates and generates the serializing text；And
The serializing instrument is controlled, to the acquisition function performing sequenceization operation.
Second aspect, the embodiment of the present invention provide a kind of device for parsing serialized data, including：
Function acquisition module, the unserializing request of serializing text is parsed for responding, obtains the serializing text
Serializing obtain function；Wherein, the serializing, which obtains function, includes data structure and the mapping relations of Data Identification, described
Data structure is used to define the data structure of the serialized data corresponding to the Data Identification of the data structure mapping；
Data structure acquisition module, for the Data Identification of the serialized data operated according to pending unserializing, leads to
Cross the serializing and obtain function, obtain the Data Identification mapped of the serialized data of the pending unserializing operation
Data structure；And
Inverted sequence operation module, for the data structure according to acquisition, to the sequence of the pending unserializing operation
Change data and perform unserializing operation.
With reference to second aspect, in the first embodiment of second aspect, described device further includes the present invention：
Configuration file acquisition module, for before obtaining function in the serializing for obtaining the serializing text, obtaining institute
State the configuration file of serializing text；Wherein, the configuration file is used to define serialized data in the serializing text
Data structure；
Function generation module, for according to the configuration file, generating the acquisition function for serializing text；And
Sequence of function module, for obtaining function to the acquisition function performing sequenceization operation, formation sequenceization.
With reference to second aspect, the present invention is in second of embodiment of second aspect, the function generation module, bag
Include：
Function name definition unit, serializes for extracting to be used to define in the serializing text from the configuration file
Parent Data Identification corresponding to the data structure of data is as the function name for obtaining function；
Source code compilation unit, for compiling the configuration file, generation configuration source code；And
Function dispensing unit, for according to the configuration source code and the function name for obtaining function, configuring described in generation
Serialize the acquisition function of text；Wherein, the function that obtains includes being used for defined nucleotide sequence data in the configuration source code
Mapping relations between the source code of the Data Identification of the source code of data structure and the serialized data.
With reference to second aspect, the present invention is in the third embodiment of second aspect, the sequence of function module bag
Include：
Instrument call unit, for calling the sequence chemical industry for being used for performing sequenceization and operating and generating the serializing text
Tool；And
Execution unit is serialized, for controlling the serializing instrument, to the acquisition function performing sequenceization operation.
The function of described device can also be performed corresponding software and be realized by hardware realization by hardware.It is described
Hardware or software include the one or more and corresponding module of above-mentioned function.
In a possible design, parsing the structure of the device of serialized data includes processor and memory, institute
State memory be used for store support the device of parsing serialized data to perform the side of parsing serialized data in above-mentioned first aspect
The program of method, the processor are configurable for performing the program stored in the memory.The parsing serialized data
Device can also include communication interface, for parsing the device of serialized data and other equipment or communication.
The third aspect, an embodiment of the present invention provides a kind of computer-readable recording medium, is serialized for storing and resolving
Computer software instructions used in the device of data, it includes being used to perform the side for parsing serialized data in above-mentioned first aspect
Method is the program involved by the device of parsing serialized data.
Any one technical solution in above-mentioned technical proposal has the following advantages that or beneficial effect：
The embodiment of the present invention obtains the serializing text when response parses the unserializing request of serializing text
Serializing obtains function；Include data structure and the mapping relations of Data Identification since the serializing obtains function, and it is described
Data structure is used to define the data structure of the serialized data corresponding to the Data Identification of the data structure mapping, thus,
When the serialized data for treating performing sequence is inquired about, function is directly obtained by the serializing, is obtained pending
The data structure of the Data Identification mapping of the serialized data of unserializing operation；And the data structure according to acquisition, to institute
The serialized data for stating pending unserializing operation performs unserializing operation, and then can inquire about acquisition and visit serialized data
Content, should compared to being compiled to serializing this configuration file in one serialized data of every inquiry and obtaining description
The data structure of serialized data to carry out the serialized data mode of unserializing, greatly improves efficiency.
Above-mentioned general introduction is merely to illustrate that the purpose of book, it is not intended to is limited in any way.Except foregoing description
Schematical aspect, outside embodiment and feature, it is further by reference to attached drawing and the following detailed description, the present invention
Aspect, embodiment and feature would is that what is be readily apparent that.
Brief description of the drawings
In the accompanying drawings, unless specified otherwise herein, otherwise represent the same or similar through the identical reference numeral of multiple attached drawings
Component or element.What these attached drawings were not necessarily to scale.It should be understood that these attached drawings depict only according to the present invention
Some disclosed embodiments, and should not serve to limit the scope of the present invention.
Fig. 1 is the flow diagram of one embodiment of the method for parsing serialized data provided by the invention；
Fig. 2 is one that the serializing of the method for parsing serialized data provided by the invention obtains the generating process of function
The flow diagram of embodiment；
Fig. 3 is one embodiment of the generating process of the acquisition function of the method for parsing serialized data provided by the invention
Flow diagram；
Fig. 4 is the flow diagram of another embodiment of the method for parsing serialized data provided by the invention；
Fig. 5 is the structure diagram of one embodiment of the device of parsing serialized data provided by the invention；
Fig. 6 is the structure diagram of terminal device provided in an embodiment of the present invention.
Embodiment
Hereinafter, some exemplary embodiments are simply just described.As one skilled in the art will recognize that
Like that, without departing from the spirit or scope of the present invention, described embodiment can be changed by various different modes.
Therefore, attached drawing and description are considered essentially illustrative rather than restrictive.
Referring to Fig. 1, an embodiment of the present invention provides a kind of method for parsing serialized data, the method can be by inverted sequence
Row chemical industry tool performs, and also can call unserializing instrument by processor and control unserializing instrument to perform corresponding operation, or
Person is directly performed by processor, and the unserializing instrument can be an engine, such as：Wing- query engines, the inverted sequence
Row chemical industry tool is also substantially a kind of compiler, and then the resolving of the embodiment of the present invention is in fact it is also assumed that be to sequence
Change the process that data are compiled, in turn, the process of formation sequence data is also believed to a kind of compilation process.Below will
Technical solution provided in an embodiment of the present invention is elaborated with reference to Fig. 1：
Method provided in an embodiment of the present invention includes step S100 to step S300, specific as follows：
Step S100, the unserializing request of response parsing serializing text, obtains the serializing of the serializing text
Obtain function；Wherein, the serializing, which obtains function, includes data structure and the mapping relations of Data Identification, the data structure
For defining the data structure of the serialized data corresponding to the Data Identification of the data structure mapping.
In embodiments of the present invention, serializing text refers to the data or object that structuring is recorded with string of binary characters,
And the hybrid language data standard that the present invention is developed with Google companies, Google Protocol Buffer are (referred to as
Protobuf exemplified by), the serializing file of the embodiment of the present invention is described.Protobuf is a kind of structural data of portable and effective
Storage format, can be used for, by the data or object serialization of structuring, serializing in other words.Since its data volume is small, transmission
The characteristics of quick, for record the various running softwares of internal system system log generally using the form carry out storage and
Transmission, abbreviation PB daily records.Usually, PB daily records are stored in metadata, and the metadata describes the storage of the PB daily records
Position, configuration file information, it is anti-ask the information such as authority, thus, in embodiments of the present invention, can be obtained by the metadata
Take serializing text.
In embodiments of the present invention, the serializing acquisition function of the serializing text can be according to the serializing text
Configuration file generation, can also be according to directly being generated according to the metadata.
In embodiments of the present invention, when response parsing it is described serializing text unserializing request when, can inquire about whether
The serializing that there are the serializing text obtains function；If so, the serializing for then directly acquiring the serializing text obtains
Take function；If nothing, need to obtain configuration file from metadata, generated according to configuration file.Thus, in order to avoid inquiring about not
Serializing to the serializing text obtains function, can obtain function in the serializing for obtaining the serializing text
Before, or before the unserializing request of the parsing serializing text is responded, generate in advance and store serializing acquisition
Function.Such a mode with respect to existing scheme to serializing text carry out unserializing during, directly to serializing text
This configuration file, which performs to compile and obtain in real time, serializes the data structure of text to perform the anti-side of solution of the dynamic of unserializing
Formula, the embodiment of the present invention shift to an earlier date formation sequenceization according to configuration file and obtain function, which obtains function due to including
The mapping relations of data structure and Data Identification, that is, belong to the data structure of just-ahead-of-time compilation, and no on-the-flier compiler goes out data structure, i.e.,
Scheme provided in an embodiment of the present invention is a kind of anti-mode solved of static state, and can be by dynamic reflective mechanism, i.e., dynamically
Called data structure in function is obtained from serializing carry out unserializing operation according to the needs of unserializing.As shown in Fig. 2, this
The process that the formation sequenceization that inventive embodiments provide obtains function includes step S110 to step S130：
Step S110, obtains the configuration file of the serializing text；Wherein, the configuration file is used to define the sequence
The data structure of serialized data in rowization text；Specifically, the serializing text can be obtained from the metadatabase
Configuration file.
The embodiment of the present invention is by taking protobuf as an example, and the serializing text is protobuf format texts, and it includes have
Multiple serialized datas, then the configuration file is then to define each in the serializing text and the serializing text
The data structure of serialized data be how, the suffix of the configuration file of protobuf format texts is generally .proto, letter
Claim proto files, this document is similar to java or the data definition of C language.In the term of protobuf, serialize in text
A serialized data be referred to as Message.Wherein, file sheet and serialized data, serialized data and sequence are serialized
It can be before nested type to change data.
Step S120, according to the configuration file, generates the acquisition function for serializing text.
In embodiments of the present invention, due to the composition of configuration file be describe Message data structure be how, because
And an acquisition function can be generated according to the mark of Message and the mapping relations of the data structure of Message.
Referring to Fig. 3, the generating process provided in an embodiment of the present invention for obtaining function, is specifically segmented into following steps
S121 to step S123 is generated：
Step S121, data of the extraction for defining serialized data in the serializing text from the configuration file
Parent Data Identification corresponding to structure is as the function name for obtaining function.Since the data structure of the serializing text is embedding
Set form, the mark using the message of top are function name of the parent Data Identification as acquisition function, can conveniently be corresponded to not
Same serializing text selection serializes acquisition function accordingly.Usually, the mark of the message of the top with it is described
The Text Flag for serializing text is identical.Certainly, it is also not necessarily limited to using the parent Data Identification as function is obtained, can basis
Compiling custom is correspondingly set.
Step S122, compiles the configuration file, generation configuration source code.The form of preferably described configuration source code is C++,
Or other compiled formats, such as C, JAVA etc..In embodiments of the present invention, since the function of acquisition is to be based on configuration file
Generation, and since original configuration file is can not to directly generate to obtain function, it is necessary to which configuration file is passed through compiler
Source code is compiled into, can just be generated according to the configuration source code.
Step S123, according to the configuration source code and the function name for obtaining function, the configuration generation serializing text
This acquisition function；Wherein, since configuration file or configuration source code are the files for the data structure for defining serialized data,
Thus some serialized data can be inquired about from this document by setting a function data structure be how, then it is described
The source code for obtaining function includes at least the source code and the sequence for the data structure for being used for defined nucleotide sequence data in the configuration source code
Mapping relations between the source code of the Data Identification of rowization data.Wherein, call the acquisition function to be inquired about, be to call
The source code for obtaining function is inquired about.
Step S130, to the acquisition function performing sequenceization operation, formation sequenceization obtains function.
In embodiments of the present invention, since the serialized data of pending unserializing operation is serialization format, because
And need the source code for obtaining function being also converted into serialization format, it is performed in can just being operated with unserializing.That is, sequence is passed through
Rowization obtain function can be with any sequence data in serializing text described in unserializing.
Preferably, the specific implementation process of the step S130 of the embodiment of the present invention, Ke Yiwei：
Call and be used for the serializing instrument that performing sequenceization operates and generates the serializing text；And
The serializing instrument is controlled, to the acquisition function performing sequenceization operation.
In the present invention is implemented, the acquisition function is carried out using the primary serializing instrument into the serializing file
Serializing, can make it that the data structure that serializing obtains function is consistent with the data structure of the serializing file.When
So, serializing operation can also be carried out by other serializing instruments, it is only necessary to reach the number that serializing obtains function
It is consistent with the data structure of the serializing file according to structure.
After obtaining serializing and obtaining function, task of antitone sequence operation can be performed to serializing text be cut into N
A unserializing task, performing the process of each unserializing task can call serializing acquisition function as follows to perform
Step S200 and step S300, and as it is foregoing it is described, the embodiment of the present invention is the static anti-mode solved, performs antitone sequence parallel
The process of change is woth no need to compile out data structure, compared to the anti-efficiency for solving, can greatly improving unserializing of dynamic.Tradition is quiet
State is counter to solve mode only by compiled data structure input unserializing instrument, is so only capable of by common reading acquisition side
Formula reads data structure, and in contrast, the embodiment of the present invention is to read data structure by function, that is, reflects acquisition machine
System, dynamically can obtain called data structure in function from serializing according to the needs of unserializing and carry out unserializing behaviour
Make, further improve the analyzing efficiency of unserializing.
Step S200, function is obtained by the serializing, obtains the serialized data of pending unserializing operation
The data structure of Data Identification mapping；And
Step S300, according to the data structure of acquisition, performs the serialized data of the pending unserializing operation
Unserializing operates.
The process of step S200 and step S300 of the present invention is generically described, i.e., using serializing in embodiments of the present invention
Obtain function and extract a key, remove to parse corresponding serializing number in the serializing text using the key of this extraction
According to.
The skill of the parsing serialized data of static anti-solution provided in an embodiment of the present invention and reflection securing mechanism is respectively adopted
The technical solution that art scheme and the prior art provide carries out the experiment of parsing serialized data, finds using the embodiment of the present invention
The efficiency that technical solution parses compressed file or uncompressed file is more efficient than using the technical solution of the prior art,
Wherein, there can be the efficiency of 13 times of lifting for uncompressed file, the efficiency that there can be 2 times for compressed file.
Referring to Fig. 4, the embodiment of the present invention using it is described serializing text as PB daily records, using processor as executive agent,
Using Wing query engines as unserializing instrument is made, exemplified by PB compilers are as serializing instrument, the embodiment of the present invention is illustrated
Specific implementation process：
Step 51, the configuration file of PB daily records is obtained from metadata, configuration file is compiled into by lattice by PB compilers
Formula is the configuration source code of C++.
Step 52, according to the main message identification names of the content obtaining configuration file of configuration file, marked with main message
Name is known as the function name for obtaining function.
Step 53, according to the configuration source code, function will be obtained and be configured to executable acquisition function, it is inquired about for Wing
The identifiable bc file formats of engine.Wherein, when keying in main message identification names when function is obtained, configuration source can be obtained
Code in the main associated data structure of message identification names.
Step 54, executable acquisition function is imported into Wing query engines, the rear end optimization module of Wing query engines
Call PB compilers to serialize the executable acquisition function, obtain serializing and obtain function.
Step 55, by serialize obtain function import Wing query engines map ends, control map ends according to obtain function,
Obtain data structure corresponding with the identification name belonging to the serialized data of PB daily records.
Step 56, map ends are controlled to carry out unserializing to the serialized data according to the data structure.
Referring to Fig. 5, the embodiment of the present invention provides a kind of device for parsing serialized data, it is possible to achieve previous embodiment
Any flow and method, specifically includes：
Function acquisition module 10, the unserializing request of serializing text is parsed for responding, and obtains the serializing text
This serializing obtains function；Wherein, the serializing, which obtains function, includes data structure and the mapping relations of Data Identification, institute
State the data structure for the serialized data that data structure is used to define corresponding to the Data Identification of the data structure mapping；
Data structure acquisition module 20, for the Data Identification of the serialized data operated according to pending unserializing,
Function is obtained by the serializing, the Data Identification for obtaining the serialized data of the pending unserializing operation is mapped
Data structure；And
Inverted sequence operation module 30, for the data structure according to acquisition, to the sequence of the pending unserializing operation
Rowization data perform unserializing operation.
With reference to aforementioned embodiments, it is further improved, described device further includes：
Configuration file acquisition module, for before obtaining function in the serializing for obtaining the serializing text, obtaining institute
State the configuration file of serializing text；Wherein, the configuration file is used to define serialized data in the serializing text
Data structure；
Function generation module, for according to the configuration file, generating the acquisition function for serializing text；And
Sequence of function module, for obtaining function to the acquisition function performing sequenceization operation, formation sequenceization.
With reference to aforementioned embodiments, it is further improved, the function generation module, including：
Function name definition unit, serializes for extracting to be used to define in the serializing text from the configuration file
Parent Data Identification corresponding to the data structure of data is as the function name for obtaining function；
Source code compilation unit, for compiling the configuration file, generation configuration source code；And
Function dispensing unit, for according to the configuration source code and the function name for obtaining function, configuring described in generation
Serialize the acquisition function of text；Wherein, the function that obtains includes being used for defined nucleotide sequence data in the configuration source code
Mapping relations between the source code of the Data Identification of the source code of data structure and the serialized data.
With reference to aforementioned embodiments, it is further improved, the sequence of function module includes：
Instrument call unit, for calling the sequence chemical industry for being used for performing sequenceization and operating and generating the serializing text
Tool；And
Execution unit is serialized, for controlling the serializing instrument, to the acquisition function performing sequenceization operation.
The function of described device can also be performed corresponding software and be realized by hardware realization by hardware.It is described
Hardware or software include the one or more and corresponding module of above-mentioned function.
The embodiment of the present invention also provides a kind of terminal device, as shown in fig. 6, the equipment includes：Memory 21 and processor
22,21 memory of memory contains the computer program that can be run on processor 22.Processor 22 performs the computer program
The method of parsing serialized data in Shi Shixian above-described embodiments.The quantity of memory 21 and processor 22 can be one or
It is multiple.
The equipment further includes：
Communication interface 23, for the communication between memory 21 and processor 22.
Memory 21 may include high-speed RAM memory, it is also possible to further include nonvolatile memory (non-volatile
Memory), a for example, at least magnetic disk storage.
If memory 21, processor 22 and the independent realization of communication interface 23, memory 21, processor 22 and communication connect
Mouth 23 can be connected with each other by bus and complete mutual communication.The bus can be industry standard architecture
(ISA, Industry Standard Architecture) bus, external equipment interconnection (PCI, Peripheral
Component) bus or extended industry-standard architecture (EISA, Extended Industry Standard
Component) bus etc..The bus can be divided into address bus, data/address bus, controlling bus etc..For ease of representing, Fig. 6
In only represented with a thick line, it is not intended that an only bus or a type of bus.
Optionally, in specific implementation, if memory 21, processor 22 and communication interface 23 are integrated in chip piece
On, then memory 21, processor 22 and communication interface 23 can complete mutual communication by internal interface.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or the spy for combining the embodiment or example description
Point is contained at least one embodiment of the present invention or example.Moreover, particular features, structures, materials, or characteristics described
It may be combined in any suitable manner in any one or more of the embodiments or examples.In addition, without conflicting with each other, this
The technical staff in field can be by the different embodiments or example described in this specification and different embodiments or exemplary spy
Sign is combined and combines.
In addition, term " first ", " second " are only used for description purpose, and it is not intended that instruction or hint relative importance
Or the implicit quantity for indicating indicated technical characteristic.Thus, " first " is defined, the feature of " second " can be expressed or hidden
Include at least one this feature containing ground.In the description of the present invention, " multiple " are meant that two or more, unless otherwise
It is clearly specific to limit.
Any process or method described otherwise above description in flow chart or herein is construed as, and represents to include
Module, fragment or the portion of the code of the executable instruction of one or more the step of being used for realization specific logical function or process
Point, and the scope of the preferred embodiment of the present invention includes other realization, wherein can not press shown or discuss suitable
Sequence, including according to involved function by it is basic at the same time in the way of or in the opposite order, carry out perform function, this should be of the invention
Embodiment person of ordinary skill in the field understood.
Expression or logic and/or step described otherwise above herein in flow charts, for example, being considered use
In the order list for the executable instruction for realizing logic function, may be embodied in any computer-readable medium, for
Instruction execution system, device or equipment (such as computer based system including the system of processor or other can be held from instruction
The system of row system, device or equipment instruction fetch and execute instruction) use, or combine these instruction execution systems, device or set
It is standby and use.For the purpose of this specification, " computer-readable medium " can any can be included, store, communicate, propagate or pass
Defeated program is for instruction execution system, device or equipment or the dress used with reference to these instruction execution systems, device or equipment
Put.
Computer-readable medium described in the embodiment of the present invention can be that computer-readable signal media or computer can
Read storage medium either the two any combination.The more specifically example of computer-readable recording medium is at least (non-poor
Property list to the greatest extent) including following：Electrical connection section (electronic device) with one or more wiring, portable computer diskette box (magnetic
Device), random access memory (RAM), read-only storage (ROM), erasable edit read-only storage (EPROM or flash
Memory), fiber device, and portable read-only storage (CDROM).In addition, computer-readable recording medium even can be with
It is the paper or other suitable media that can print described program on it, because can be for example by being carried out to paper or other media
Optical scanner, is then handled described electronically to obtain into edlin, interpretation or if necessary with other suitable methods
Program, is then stored in computer storage.
In embodiments of the present invention, computer-readable signal media can be included in a base band or as a carrier wave part
The data-signal of propagation, wherein carrying computer-readable program code.The data-signal of this propagation can use a variety of
Form, includes but not limited to electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media is also
Can be any computer-readable medium beyond computer-readable recording medium, which can send, pass
Either transmission is broadcast for instruction execution system, input method or device use or program in connection.Computer can
The program code for reading to include on medium can be transmitted with any appropriate medium, be included but not limited to：Wirelessly, electric wire, optical cable, penetrate
Frequently (Radio Frequency, RF) etc., or above-mentioned any appropriate combination.
It should be appreciated that each several part of the present invention can be realized with hardware, software, firmware or combinations thereof.Above-mentioned
In embodiment, software that multiple steps or method can be performed in memory and by suitable instruction execution system with storage
Or firmware is realized.If, and in another embodiment, can be with well known in the art for example, realized with hardware
Any one of row technology or their combination are realized：With the logic gates for realizing logic function to data-signal
Discrete logic, have suitable combinational logic gate circuit application-specific integrated circuit, programmable gate array (PGA), scene
Programmable gate array (FPGA) etc..
Those skilled in the art are appreciated that to realize all or part of step that above-described embodiment method carries
Suddenly it is that relevant hardware can be instructed to complete by program, the program can be stored in a kind of computer-readable storage medium
In matter, the program upon execution, including one or a combination set of the step of embodiment of the method.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing module, can also
That unit is individually physically present, can also two or more units be integrated in a module.Above-mentioned integrated mould
Block can both be realized in the form of hardware, can also be realized in the form of software function module.The integrated module is such as
Fruit is realized in the form of software function module and as independent production marketing or in use, can also be stored in a computer
In readable storage medium storing program for executing.The storage medium can be read-only storage, disk or CD etc..
The above description is merely a specific embodiment, but protection scope of the present invention is not limited thereto, any
Those familiar with the art the invention discloses technical scope in, its various change or replacement can be readily occurred in,
These should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention should be with the guarantor of the claim
Protect subject to scope.
Claims (10)
- A kind of 1. method for parsing serialized data, it is characterised in that including：The unserializing request of response parsing serializing text, the serializing for obtaining the serializing text obtain function；Wherein, The serializing, which obtains function, includes data structure and the mapping relations of Data Identification, and the data structure is used to define the number According to the data structure of the serialized data corresponding to the Data Identification of structure mapping；Function is obtained by the serializing, obtains the Data Identification mapping of the serialized data of pending unserializing operation Data structure；AndAccording to the data structure of acquisition, unserializing operation is performed to the serialized data of the pending unserializing operation.
- 2. the method for parsing serialized data as claimed in claim 1, it is characterised in that obtaining the serializing text Before serializing obtains function, further include：Obtain the configuration file of the serializing text；Wherein, the configuration file is used to define sequence in the serializing text The data structure of rowization data；According to the configuration file, the acquisition function for serializing text is generated；AndTo the acquisition function performing sequenceization operation, formation sequenceization obtains function.
- 3. the method for parsing serialized data as claimed in claim 2, it is characterised in that it is described according to the configuration file, The acquisition function of the serializing text is generated, including：Extraction is used to define in the serializing text corresponding to the data structure of serialized data from the configuration file Parent Data Identification is as the function name for obtaining function；Compile the configuration file, generation configuration source code；AndAccording to the configuration source code and the function name for obtaining function, the acquisition function of the configuration generation serializing text； Wherein, the source code and the sequence for obtaining function and including being used for the data structure of defined nucleotide sequence data in the configuration source code Change the mapping relations between the source code of the Data Identification of data.
- 4. the method for the parsing serialized data as claimed in claim 2, it is characterised in that described to be held to the acquisition function Row serializing operation, including：Call and be used for the serializing instrument that performing sequenceization operates and generates the serializing text；AndThe serializing instrument is controlled, to the acquisition function performing sequenceization operation.
- A kind of 5. device for parsing serialized data, it is characterised in that including：Function acquisition module, the unserializing request of serializing text is parsed for responding, and obtains the sequence of the serializing text Rowization obtain function；Wherein, the serializing, which obtains function, includes data structure and the mapping relations of Data Identification, the data Structure is used to define the data structure of the serialized data corresponding to the Data Identification of the data structure mapping；Data structure acquisition module, for the Data Identification of the serialized data operated according to pending unserializing, passes through institute State serializing and obtain function, obtain the Data Identification mapped data of the serialized data of the pending unserializing operation Structure；AndInverted sequence operation module, for the data structure according to acquisition, to the serializing number of the pending unserializing operation Operated according to unserializing is performed.
- 6. the device of parsing serialized data as claimed in claim 5, it is characterised in that described device further includes：Configuration file acquisition module, for before obtaining function in the serializing for obtaining the serializing text, obtaining the sequence The configuration file of rowization text；Wherein, the configuration file is used for the data for defining serialized data in the serializing text Structure；Function generation module, for according to the configuration file, generating the acquisition function for serializing text；AndSequence of function module, for obtaining function to the acquisition function performing sequenceization operation, formation sequenceization.
- 7. the device of parsing serialized data as claimed in claim 6, it is characterised in that the function generation module, including：Function name definition unit, is used to define serialized data in the serializing text for extracting from the configuration file Data structure corresponding to parent Data Identification as obtain function function name；Source code compilation unit, for compiling the configuration file, generation configuration source code；AndFunction dispensing unit, for generating the sequence according to the configuration source code and the function name for obtaining function, configuration Change the acquisition function of text；Wherein, the function that obtains includes being used for the data of defined nucleotide sequence data in the configuration source code Mapping relations between the source code of the Data Identification of the source code of structure and the serialized data.
- 8. the device of the parsing serialized data as claimed in claim 6, it is characterised in that the sequence of function module bag Include：Instrument call unit, for calling the serializing instrument for being used for performing sequenceization and operating and generating the serializing text； AndExecution unit is serialized, for controlling the serializing instrument, to the acquisition function performing sequenceization operation.
- 9. a kind of terminal device for parsing serialized data, it is characterised in that the terminal device includes：One or more processors；Storage device, for storing one or more programs；When one or more of programs are performed by one or more of processors so that one or more of processors The method for realizing the parsing serialized data as described in any in claim 1-4.
- 10. a kind of computer-readable recording medium, it is stored with computer program, it is characterised in that the program is held by processor The method that the parsing serialized data as described in any in claim 1-4 is realized during row.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711407433.9A CN107992624B (en) | 2017-12-22 | 2017-12-22 | Method, device, storage medium and terminal equipment for analyzing serialized data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711407433.9A CN107992624B (en) | 2017-12-22 | 2017-12-22 | Method, device, storage medium and terminal equipment for analyzing serialized data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107992624A true CN107992624A (en) | 2018-05-04 |
| CN107992624B CN107992624B (en) | 2020-08-04 |
Family
ID=62041565
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711407433.9A Active CN107992624B (en) | 2017-12-22 | 2017-12-22 | Method, device, storage medium and terminal equipment for analyzing serialized data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107992624B (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109117209A (en) * | 2018-07-23 | 2019-01-01 | 广州多益网络股份有限公司 | Serializing and unserializing method and device |
| CN109699023A (en) * | 2018-12-12 | 2019-04-30 | 中国铁道科学研究院集团有限公司通信信号研究所 | The automatic test data encapsulation of high-speed rail signalling arrangement and processing method |
| CN109753048A (en) * | 2018-12-12 | 2019-05-14 | 中国铁道科学研究院集团有限公司通信信号研究所 | The automatic Test driver automotive engine system of high-speed rail signalling arrangement |
| CN110377289A (en) * | 2019-07-01 | 2019-10-25 | 北京字节跳动网络技术有限公司 | A kind of data analysis method, device, medium and electronic equipment |
| CN110515893A (en) * | 2019-07-26 | 2019-11-29 | 济南浪潮数据技术有限公司 | Date storage method, device, equipment and computer readable storage medium |
| CN110688530A (en) * | 2019-08-19 | 2020-01-14 | 天津开心生活科技有限公司 | Json data processing method and device |
| CN110990411A (en) * | 2019-12-18 | 2020-04-10 | 上海米哈游网络科技股份有限公司 | Data structure generation method and device and calling method and device |
| CN111931200A (en) * | 2020-07-13 | 2020-11-13 | 车智互联（北京）科技有限公司 | Data serialization method, mobile terminal and readable storage medium |
| WO2021097785A1 (en) * | 2019-11-22 | 2021-05-27 | 云图技术有限公司 | Data serialization and data deserialization methods and apparatuses, and computer device |
| CN112947938A (en) * | 2021-03-23 | 2021-06-11 | 广州趣丸网络科技有限公司 | File conversion method and device, electronic equipment and storage medium |
| CN113746842A (en) * | 2021-09-03 | 2021-12-03 | 唯品会(广州)软件有限公司 | Message sending method based on Protobuf protocol dynamic analysis |
| CN114331416A (en) * | 2021-12-30 | 2022-04-12 | 深圳市兆珑科技有限公司 | Data processing method and device, electronic equipment and storage medium |
| WO2024109441A1 (en) * | 2022-11-21 | 2024-05-30 | 飞天诚信科技股份有限公司 | Method and system for parsing pb format circulation data |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070220138A1 (en) * | 2006-03-20 | 2007-09-20 | Microsoft Corporation | Extensible alert types |
| CN103036961A (en) * | 2012-12-07 | 2013-04-10 | 蓝盾信息安全技术股份有限公司 | Distributed collection and storage method of journal |
| CN103389992A (en) * | 2012-05-09 | 2013-11-13 | 北京百度网讯科技有限公司 | Structured data storage method and device |
| CN104199879A (en) * | 2014-08-21 | 2014-12-10 | 广州华多网络科技有限公司 | Data processing method and device |
| CN105677805A (en) * | 2015-12-31 | 2016-06-15 | 北京奇艺世纪科技有限公司 | Data storing and reading method and device using protobuf |
| CN105760534A (en) * | 2016-03-10 | 2016-07-13 | 上海晶赞科技发展有限公司 | User-defined serializable data structure, hadoop cluster, server and application method thereof |
| CN105787128A (en) * | 2016-03-29 | 2016-07-20 | 四川秘无痕信息安全技术有限责任公司 | Method for recovering Java serialized file data |
-
2017
- 2017-12-22 CN CN201711407433.9A patent/CN107992624B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070220138A1 (en) * | 2006-03-20 | 2007-09-20 | Microsoft Corporation | Extensible alert types |
| CN103389992A (en) * | 2012-05-09 | 2013-11-13 | 北京百度网讯科技有限公司 | Structured data storage method and device |
| CN103036961A (en) * | 2012-12-07 | 2013-04-10 | 蓝盾信息安全技术股份有限公司 | Distributed collection and storage method of journal |
| CN104199879A (en) * | 2014-08-21 | 2014-12-10 | 广州华多网络科技有限公司 | Data processing method and device |
| CN105677805A (en) * | 2015-12-31 | 2016-06-15 | 北京奇艺世纪科技有限公司 | Data storing and reading method and device using protobuf |
| CN105760534A (en) * | 2016-03-10 | 2016-07-13 | 上海晶赞科技发展有限公司 | User-defined serializable data structure, hadoop cluster, server and application method thereof |
| CN105787128A (en) * | 2016-03-29 | 2016-07-20 | 四川秘无痕信息安全技术有限责任公司 | Method for recovering Java serialized file data |
Non-Patent Citations (2)
| Title |
|---|
| 李纪欣 等: "Google Protobuf在Linux Socket通讯中的应用", 《电脑开发与应用》 * |
| 王斌斌: "基于Protobuf的RPC系统的设计与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109117209B (en) * | 2018-07-23 | 2021-10-01 | 广州多益网络股份有限公司 | Serialization and deserialization method and device |
| CN109117209A (en) * | 2018-07-23 | 2019-01-01 | 广州多益网络股份有限公司 | Serializing and unserializing method and device |
| CN109699023A (en) * | 2018-12-12 | 2019-04-30 | 中国铁道科学研究院集团有限公司通信信号研究所 | The automatic test data encapsulation of high-speed rail signalling arrangement and processing method |
| CN109753048A (en) * | 2018-12-12 | 2019-05-14 | 中国铁道科学研究院集团有限公司通信信号研究所 | The automatic Test driver automotive engine system of high-speed rail signalling arrangement |
| CN109699023B (en) * | 2018-12-12 | 2022-02-22 | 中国铁道科学研究院集团有限公司通信信号研究所 | Automatic test data packaging and processing method for high-speed rail signal equipment |
| CN110377289A (en) * | 2019-07-01 | 2019-10-25 | 北京字节跳动网络技术有限公司 | A kind of data analysis method, device, medium and electronic equipment |
| CN110515893B (en) * | 2019-07-26 | 2022-12-09 | 济南浪潮数据技术有限公司 | Data storage method, device, equipment and computer readable storage medium |
| CN110515893A (en) * | 2019-07-26 | 2019-11-29 | 济南浪潮数据技术有限公司 | Date storage method, device, equipment and computer readable storage medium |
| CN110688530A (en) * | 2019-08-19 | 2020-01-14 | 天津开心生活科技有限公司 | Json data processing method and device |
| CN110688530B (en) * | 2019-08-19 | 2022-04-26 | 天津开心生活科技有限公司 | Json data processing method and device |
| WO2021097785A1 (en) * | 2019-11-22 | 2021-05-27 | 云图技术有限公司 | Data serialization and data deserialization methods and apparatuses, and computer device |
| CN110990411B (en) * | 2019-12-18 | 2024-02-06 | 上海米哈游网络科技股份有限公司 | Data structure generation method and device, and calling method and device |
| CN110990411A (en) * | 2019-12-18 | 2020-04-10 | 上海米哈游网络科技股份有限公司 | Data structure generation method and device and calling method and device |
| CN111931200A (en) * | 2020-07-13 | 2020-11-13 | 车智互联（北京）科技有限公司 | Data serialization method, mobile terminal and readable storage medium |
| CN111931200B (en) * | 2020-07-13 | 2024-02-23 | 车智互联（北京）科技有限公司 | Data serialization method, mobile terminal and readable storage medium |
| CN112947938A (en) * | 2021-03-23 | 2021-06-11 | 广州趣丸网络科技有限公司 | File conversion method and device, electronic equipment and storage medium |
| CN112947938B (en) * | 2021-03-23 | 2024-04-30 | 广州趣丸网络科技有限公司 | File conversion method and device, electronic equipment and storage medium |
| CN113746842A (en) * | 2021-09-03 | 2021-12-03 | 唯品会(广州)软件有限公司 | Message sending method based on Protobuf protocol dynamic analysis |
| CN114331416A (en) * | 2021-12-30 | 2022-04-12 | 深圳市兆珑科技有限公司 | Data processing method and device, electronic equipment and storage medium |
| WO2024109441A1 (en) * | 2022-11-21 | 2024-05-30 | 飞天诚信科技股份有限公司 | Method and system for parsing pb format circulation data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107992624B (en) | 2020-08-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107992624A (en) | Parse method, apparatus, storage medium and the terminal device of serialized data | |
| US7739696B2 (en) | Message translation systems and methods | |
| US6754884B1 (en) | Programming language extensions for processing XML objects and related applications | |
| WO2018210096A1 (en) | Rule engine-based rule configuration method, terminal and device, and storage medium | |
| JP3199013B2 (en) | Language processing method, language processing apparatus, and storage medium storing language processing program | |
| CN113312108B (en) | SWIFT message verification method and device, electronic equipment and storage medium | |
| CN101208690B (en) | Translating expressions in computing environment | |
| CN103853535A (en) | Method and device for modifying middleware | |
| CN107797823A (en) | Business Rule Management method, apparatus, storage medium and computer equipment | |
| US8918356B2 (en) | Method, apparatus and machine-readable medium for transforming, converting and processing messages between multiple systems | |
| CN110262784A (en) | A kind of cloud notes implementation method and device | |
| US20080065664A1 (en) | Computer-implemented method, tool, and program product for more efficiently utilizing java resource bundles | |
| CN111240772A (en) | Data processing method and device based on block chain and storage medium | |
| US7506068B2 (en) | Method, apparatus and system for transforming, converting and processing messages between multiple systems | |
| CN116107578A (en) | Method, device, electronic equipment and storage medium for executing intelligent contract | |
| CN109815455A (en) | Item file treating method and apparatus | |
| US7769896B2 (en) | Method, apparatus and system for dispatching messages within a system | |
| CN115310127A (en) | Data desensitization method and device | |
| EP1183596B1 (en) | Generating optimized computer data field conversion routines | |
| KR100772181B1 (en) | Method and System for cooperation scheme of the development tools based on Extensible Markup LanguageXML Schema | |
| CN117492747A (en) | Compiler back-end multiplexing method and device, compiler and storage medium | |
| AU2013203830B2 (en) | Translating expressions in a computing environment | |
| CN109725907A (en) | Installation method, device, equipment and the computer-readable medium of preset application | |
| CN111126021A (en) | Method for displaying report data in binary stream format in heterogeneous architecture | |
| AU2011204906B2 (en) | Translating XML with multiple namespace extensions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |