CN106775696A - Data processing method and device - Google Patents

Data processing method and device Download PDF

Info

Publication number
CN106775696A
CN106775696A CN201611124614.6A CN201611124614A CN106775696A CN 106775696 A CN106775696 A CN 106775696A CN 201611124614 A CN201611124614 A CN 201611124614A CN 106775696 A CN106775696 A CN 106775696A
Authority
CN
China
Prior art keywords
data
attribute
serialized
serialized data
relevant information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201611124614.6A
Other languages
Chinese (zh)
Inventor
安宁
刘斌
刘建
刘晨光
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Lanseen Workshop (beijing) Technology Co Ltd
Original Assignee
Lanseen Workshop (beijing) Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Lanseen Workshop (beijing) Technology Co Ltd filed Critical Lanseen Workshop (beijing) Technology Co Ltd
Priority to CN201611124614.6A priority Critical patent/CN106775696A/en
Publication of CN106775696A publication Critical patent/CN106775696A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/448Execution paradigms, e.g. implementations of programming paradigms
    • G06F9/4488Object-oriented
    • G06F9/449Object-oriented method invocation or resolution

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a kind of data processing method and device.The method includes:First ray data are obtained, First ray data are parsed, obtain the corresponding object of First ray data;The attribute data of object is obtained and analyzed, the relevant information of attribute data is determined;Corresponding second serialized data of relevant information generation object based on attribute data;Corresponding second serialized data of storage object.Technical scheme provided in an embodiment of the present invention is based on corresponding second serialized data of relevant information generation object of attribute data, when subsequently to call the object, the second serialized data that can be stored with direct access, the second serialized data is quickly and accurately parsed using the relevant information of attribute data, analyzing efficiency is there may be when being parsed to First ray data so as to solve not high, time-consuming more long, and the corresponding object of First ray data is applied using frequently, the problem for causing the overall performance of application not high.

Description

Data processing method and device
Technical field
The present invention relates to technical field of internet application, particularly a kind of data processing method and device.
Background technology
Serializing (Serialization) is to be converted to the attribute data (or status information) of object to store or pass The process of defeated form.During serializing, the attribute data of object is written to interim or persistence with serialized data and is deposited Storage area.After, can be read by from memory block, parsing or the unserializing serialized data, re-create the object.
At present, common serializing and unserializing agreement have XML (Extensible Markup Language, can expand Exhibition markup language), JSON (JavaScript Object Notation, JavaScript object representation), Protobuf (Protocol Buffers, it is a kind of DDL of Google's exploitation, can be by structuring number similar to XML According to serializing, can be used for the aspects such as data storage, communication protocol), Thrift (it be facebook company exploitation a software frame Frame so that the system of different language exploitation can be communicated by the framework) and Avro (it is that Data Serialization is System) etc..By taking JSON as an example, JSON originates from weak type language JavaScript, and its generation comes from one kind and is referred to as The concept of " Associative array (Associate array) ", its essence be exactly using " Attribute-value (attribute- Value) " mode carry out description object.Following example is the result after being serialized to certain object using JSON agreements, { " userid":1,"name":"messi","address":[{"city":" Beijing ", " postcode ":"10000"," street":" wangjingdonglu " }] }, the serialized data is being parsed or during unserializing, it is necessary to one by one Character is parsed backward, judge be ":" character such as (colon) or ", " (comma), and then come extract the object attribute and Property value, such analyzing efficiency is not high, time-consuming more long, and the interface of the application for obtaining the object can be caused interim card occur, influences Consumer's Experience.
The content of the invention
In view of the above problems, it is proposed that the present invention so as to provide one kind overcome above mentioned problem or at least in part solve on State the data processing method and corresponding device of problem.
According to an aspect of of the present present invention, there is provided a kind of data processing method, including:
First ray data are obtained, the First ray data are parsed, obtain the First ray number According to corresponding object;
The attribute data of the object is obtained and analyzed, the relevant information of the attribute data is determined;
Relevant information based on the attribute data generates corresponding second serialized data of the object;
Store corresponding second serialized data of the object.
Alternatively, after corresponding second serialized data of the object is stored, methods described also includes:
When the request for locally calling the object is received, the corresponding second serializing number of the object of storage is obtained According to;
Second serialized data is parsed using the relevant information of the attribute data, the object is obtained, and provide To called side.
Alternatively, after corresponding second serialized data of the object is stored, methods described also includes:
When the request of object described in external call is received, the corresponding second serializing number of the object of storage is obtained According to;
Corresponding second serialized data of the object for obtaining is supplied to called side, by the called side using described The relevant information of attribute data parses second serialized data, obtains the object.
Alternatively, the attribute data includes attribute and property value, and the relevant information of the attribute data includes attribute The length of length and property value.
Alternatively, the relevant information based on the attribute data generates corresponding second serialized data of the object, bag Include:
The length of the attribute based on the object and the length of property value, generate the corresponding second serializing number of the object According to, wherein, second serialized data includes the length and the length of property value of the attribute of the object.
Alternatively, the mark that the mark and object for also starting including object in second serialized data terminate.
Alternatively, the type of the object is also included in second serialized data.
Alternatively, the type of the object includes following one of any:Integer, byte type, short, long, single essence Degree floating type, double-precision floating point type, character type, Boolean type, date, set, array.
Alternatively, corresponding second serialized data of the object is generated in the relevant information based on the attribute data When, methods described also includes:
For the attribute data of the object sets up index;
The attribute data of the object in second serialized data is represented with the index value set up.
Alternatively, the acquisition First ray data, including:
Receive from outside First ray data;Or
First ray data are obtained from local storage.
Alternatively, second serialized data is character type data.
Alternatively, when serialized data of the First ray data for json forms, to the First ray Data are parsed, including:
The First ray data are parsed using fastjson components.
According to another aspect of the present invention, a kind of data processing equipment is additionally provided, including:
Parsing module, is suitable to obtain First ray data, and the First ray data are parsed, and obtains described The corresponding object of First ray data;
Determining module, is suitable to obtain and analyze the attribute data of the object, determines the relevant information of the attribute data;
Generation module, is suitable to the relevant information based on the attribute data and generates the corresponding second serializing number of the object According to;
Memory module, is suitable to store corresponding second serialized data of the object.
Alternatively, described device also includes:
First processing module, is suitable to after the memory module stores corresponding second serialized data of the object, When the request for locally calling the object is received, corresponding second serialized data of the object of storage is obtained;
Second serialized data is parsed using the relevant information of the attribute data, the object is obtained, and provide To called side.
Alternatively, described device also includes:
Second processing module, is suitable to after the memory module stores corresponding second serialized data of the object, When the request of object described in external call is received, corresponding second serialized data of the object of storage is obtained;
Corresponding second serialized data of the object for obtaining is supplied to called side, by the called side using described The relevant information of attribute data parses second serialized data, obtains the object.
Alternatively, the attribute data includes attribute and property value, and the relevant information of the attribute data includes attribute The length of length and property value.
Alternatively, the generation module is further adapted for:
The length of the attribute based on the object and the length of property value, generate the corresponding second serializing number of the object According to, wherein, second serialized data includes the length and the length of property value of the attribute of the object.
Alternatively, the mark that the mark and object for also starting including object in second serialized data terminate.
Alternatively, the type of the object is also included in second serialized data.
Alternatively, the type of the object includes following one of any:Integer, byte type, short, long, single essence Degree floating type, double-precision floating point type, character type, Boolean type, date, set, array.
Alternatively, the generation module is further adapted for:
It is described when the relevant information based on the attribute data generates corresponding second serialized data of the object The attribute data of object sets up index;
The attribute data of the object in second serialized data is represented with the index value set up.
Alternatively, the parsing module is further adapted for:
Receive from outside First ray data;Or
First ray data are obtained from local storage.
Alternatively, second serialized data is character type data.
Alternatively, the parsing module is further adapted for:
When serialized data of the First ray data for json forms, using fastjson components to described the One serialized data is parsed.
In embodiments of the present invention, after to First ray data parse and obtaining its corresponding object, obtain simultaneously The attribute data of the object is analyzed, the relevant information of attribute data is determined;And then the relevant information generation based on attribute data should Corresponding second serialized data of object, and store.Because the second serialized data is that the relevant information based on attribute data is given birth to Into, when subsequently to call the object, the second serialized data that can be stored with direct access, using the correlation of attribute data Information is quickly and accurately parsed to the second serialized data, when being parsed to First ray data so as to solve There may be analyzing efficiency not high, time-consuming more long, and the corresponding object of First ray data is applied using frequently, causing should Overall performance problem not high.Also, the second serialized data of embodiment of the present invention generation does not have the category of lost objects Property data, it still can realize entering between the second serialized data and First ray data row format conversion, to meet Different storage demands.
Described above is only the general introduction of technical solution of the present invention, in order to better understand technological means of the invention, And can be practiced according to the content of specification, and in order to allow the above and other objects of the present invention, feature and advantage can Become apparent, below especially exemplified by specific embodiment of the invention.
According to the accompanying drawings to the detailed description of the specific embodiment of the invention, those skilled in the art will be brighter Of the invention above-mentioned and other purposes, advantages and features.
Brief description of the drawings
By reading the detailed description of hereafter preferred embodiment, various other advantages and benefit is common for this area Technical staff will be clear understanding.Accompanying drawing is only used for showing the purpose of preferred embodiment, and is not considered as to the present invention Limitation.And in whole accompanying drawing, identical part is denoted by the same reference numerals.In the accompanying drawings:
Fig. 1 shows the flow chart of data processing method according to an embodiment of the invention;
Fig. 2 shows a kind of schematic diagram of the second serialized data form according to an embodiment of the invention;
Fig. 3 shows another schematic diagram of the second serialized data form according to an embodiment of the invention;
Fig. 4 show the relation schematic diagram of two kinds of storage formats conversion according to an embodiment of the invention;
Fig. 5 A show the flow chart of data processing method according to another embodiment of the present invention;
Fig. 5 B show the schematic diagram of the second serialized data using the method generation shown in Fig. 5 A;
Fig. 6 shows the structural representation of data processing equipment according to an embodiment of the invention;And
Fig. 7 shows the structural representation of data processing equipment according to another embodiment of the present invention.
Specific embodiment
The exemplary embodiment of the disclosure is more fully described below with reference to accompanying drawings.Although showing the disclosure in accompanying drawing Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here Limited.Conversely, there is provided these embodiments are able to be best understood from the disclosure, and can be by the scope of the present disclosure Complete conveys to those skilled in the art.
In order to solve the above technical problems, the embodiment of the invention provides a kind of data processing method.Fig. 1 is shown according to this Invent the flow chart of the data processing method of an embodiment.As shown in figure 1, the method at least comprises the following steps S102 to step S108。
Step S102, obtains First ray data, and First ray data are parsed, and obtains First ray number According to corresponding object.
Step S104, obtains and analyzes the attribute data of object, determines the relevant information of attribute data.
Step S106, corresponding second serialized data of relevant information generation object based on attribute data.
Step S108, corresponding second serialized data of storage object.
In embodiments of the present invention, after to First ray data parse and obtaining its corresponding object, obtain simultaneously The attribute data of the object is analyzed, the relevant information of attribute data is determined;And then the relevant information generation based on attribute data should Corresponding second serialized data of object, and store.Because the second serialized data is that the relevant information based on attribute data is given birth to Into, when subsequently to call the object, the second serialized data that can be stored with direct access, using the correlation of attribute data Information is quickly and accurately parsed to the second serialized data, when being parsed to First ray data so as to solve There may be analyzing efficiency not high, time-consuming more long, and the corresponding object of First ray data is applied using frequently, causing should Overall performance problem not high.Also, the second serialized data of embodiment of the present invention generation does not have the category of lost objects Property data, it still can realize entering between the second serialized data and First ray data row format conversion, to meet Different storage demands.
Called the turn in machine word, pair as if objective world present in the entity such as people, thing, object in computer logic Mapping.As introduced above, object in storage or transmission, it is necessary to serialized to it first, during serializing, by object Attribute data interim or persistent storage area is written to serialized data.After, can be read by from memory block, solved Analysis or the unserializing serialized data, re-create the object.
The First ray data obtained in above step S102 can be from outside transmission, or from this Ground memory storage.For example, when any applications client in terminal (such as PC, smart mobile phone) obtains the first sequence During rowization data, the First ray data can be from outside (such as server end, other-end) transmission, it is also possible to It is from terminal local memory storage.Again for example, when server obtain First ray data when, the First ray number According to can be deposited from outside (such as terminal, other server ends) transmission, or from server local storage Storage.Further, First ray data can be carried out using serializing agreement (such as XML, JSON) common at present What serializing was obtained.
In alternative embodiment of the invention, if First ray data are the serialized data of json forms, i.e. use Serializing agreement JSON is serialized and obtained to corresponding object.For example, certain object, sequence is carried out using JSON to it After change, First ray data are obtained for { " name ":"value"}.Again for example, the tissue data of certain company include office worker's Essential information (such as name, address), then can be using office worker as total object, using the essential information of office worker as each height pair As it is { " userid " that each subobject is carried out serializing the serialized data for obtaining:1,"name":"messi"," address":[{"city":" Beijing ", " postcode ":"10000","street":" wangjingdonglu " }] }, the example There are multiple subobjects in son, each subobject includes respective attribute data, such as Property Name and property value.
Fastjson components are the perfect JSON storehouses of a performance function write by Java language, and it uses original creation Algorithm, arrives ultimate attainment by the speed lifting of syntactic analysis.Therefore, it is possible to using fastjson components object is serialized come To the First ray data of json forms, and then First ray data are parsed using fastjson components, obtained The corresponding object of First ray data.Here, the reflex mechanism of fastjson can be utilized in serializing, word is filled automatically Section and construction object, greatly reduce application layer code amount.
In alternative embodiment of the invention, if First ray data are the serialized data of xml forms, i.e. use Serializing agreement XML is serialized and obtained to corresponding object.Still by taking above-mentioned example as an example, sequence is carried out to office worker's object The serialized data that rowization are obtained is as follows:
<Xml version=" 1.0 " encoding=" utf-8 ">
<clerk>
<userid>1</userid>
<name>messi</name>
<address>
<city>Beijing</city>
<postcode>10000</postcode>
<street>wangjingdonglu</street>
</address>
</clerk>
The First ray data of two kinds of forms (i.e. json and xml forms) are described above, may be used also in actual applications To be the First ray data of the corresponding format obtained using other sequences agreement (such as Protobuf or Thrift), This is no longer repeated one by one.
After step S102 obtains the corresponding object of First ray data, step S104 is obtained and is analyzed the category of object Property data determine the relevant information of attribute data, and attribute data here can include Property Name and property value.With above-mentioned As a example by enumerating, the Property Name in office worker in the attribute data of each subobject can be userid, name, address, City, postcode and street, its corresponding property value be respectively 1, messi, Beijing, 10000 and wangjingdonglu.Further, the relevant information of attribute data can be including the type of attribute data, length etc., such as The relevant information of attribute data can be the length of Property Name and the length of property value.
Corresponding second serialized data of relevant information generation object based on attribute data in above step S106, this Inventive embodiments provide a kind of optional scheme, i.e. the length of object-based attribute and the length of property value, generate object Corresponding second serialized data, wherein, the second serialized data includes the length of the attribute of object and the length of property value. So, subsequently when the request for calling the object is received, the second serialized data of the object can be obtained, and based on object The length of attribute and the length of property value the second serialized data is quickly and accurately parsed, obtain the object, and nothing Character is parsed need backward one by one as parsing First ray data (such as the First ray data of json forms), can To improve analyzing efficiency, the performance of called side is improved, and then improve Consumer's Experience.
In alternative embodiment of the invention, the mark and object that can also start including object in the second serialized data The mark of end, with the further efficiency and accuracy for improving parsing.
In alternative embodiment of the invention, the type of object can also be included in the second serialized data, to improve solution The accuracy of analysis.Here, the type of object can include integer, byte type, short, long, single-precision floating point type, double essences Degree floating type, character type, Boolean type, date, set, array, etc., the invention is not restricted to this.Example, office worker are enumerated as with above-mentioned The type of object can be character type or array.
In alternative embodiment of the invention, the relevant information generation object based on attribute data is corresponding in step s 106 The second serialized data when, can also be that the attribute data of object sets up index, and then will be right in the second serialized data The attribute data of elephant is represented with the index value set up, and such that it is able to save memory space, and read-write amount is also reduced therewith, parsing Speed is further enhanced.
In alternative embodiment of the invention, the second serialized data of generation can be character type data, i.e., using word Symbol type char [] data storage, rather than byte type byte [] data storage, eliminates the character string encoding and decoding time, and performance is obtained To further raising.
As previously mentioned, because the second serialized data is what the relevant information based on attribute data was generated, subsequently When calling the object, the second serialized data that can be stored with direct access, using the relevant information of attribute data to the second sequence Rowization data are quickly and accurately parsed, and parsing is there may be when being parsed to First ray data so as to solve It is inefficient, time-consuming more long, and the corresponding object of First ray data is applied using frequently, causes the overall performance of application Problem not high.Therefore, after corresponding second serialized data of above step S108 storage objects, the present invention can be exchanged Processed with the request of object, be introduced for the local or outside request for calling object separately below.
Situation one, for the local request for calling object.That is, when receive it is local call the request of object when, obtain Corresponding second serialized data of object of storage, and then using relevant information second serialized data of parsing of attribute data, The object is obtained, and is supplied to called side.Here attribute data can include Property Name and property value, the phase of attribute data Pass information can be bag in the length of Property Name and the length of property value, and corresponding second serialized data of object of storage Include the length of the attribute of object and the length of property value.Further, can also start including object in the second serialized data Mark, the mark that object terminates and object type (can as integer, byte type, short, long, single precision float Point-type, double-precision floating point type, character type, Boolean type, date, set, array etc.).
A kind of schematic diagram of the form of the second serialized data is illustrated in figure 2, second serialized data might as well be claimed Form is proto forms, wherein, the mark that 1 expression object starts, the mark that 2 expression objects terminate, 3 represent the type of object, The length of 4 Property Names for representing object, 5 represent the length of object's property value, and " key " represents Property Name, " value " table Show property value, object described herein can also be each subobject under a total object.It should be noted that shown in Fig. 2 Form in the order of each field only enumerate, do not limit the invention, in actual applications can be according to appointing Meaning order of the field layout format.
Can also be that the attribute data of object sets up rope to save memory space in alternative embodiment of the invention Draw, and then the attribute data of the object in the second serialized data is represented with the index value set up.In actual applications, pass through Analysis, there may be a large amount of character strings for repeating in key/value key-value pairs, especially array, if only stored once, will save Save a large amount of memory spaces.Also, due to the reduction of memory space, read-write amount is also reduced therewith, and resolution speed is further carried It is high.
Another schematic diagram of the form of the second serialized data is illustrated in figure 3, in figure 3, what 1 expression object started Mark, the mark that 2 expression objects terminate, 3 represent the type of object, and 4 represent the length of the Property Name of object, and 5 represent object Property value length, 6 represent Property Names, 7 represent property values.I.e., it is possible to represent attribute number respectively using index 6 and 7 " key " and " value " in, such that it is able to save substantial amounts of memory space.It should be noted that object described herein Can be each subobject under a total object, and the order of each field in form shown in Fig. 3 is only enumerated, and Do not limit the invention, in actual applications can be according to arbitrary fields sequential placement form.
Situation two, for the outside request for calling object.That is, when the request of external call object is received, obtain Corresponding second serialized data of object of storage, and then corresponding second serialized data of the object of acquisition is supplied to calls Side, the second serialized data is parsed by called side using the relevant information of attribute data, obtains the object.Here attribute data Property Name and property value can be included, the relevant information of attribute data can be the length of Property Name and the length of property value Degree, and corresponding second serialized data of object of storage includes the length of the attribute of object and the length of property value.Enter one Step ground, the type of the mark, the mark that object terminates and object that can also start including object in the second serialized data (can With such as integer, byte type, short, long, single-precision floating point type, double-precision floating point type, character type, Boolean type, date, collection Conjunction, array etc.).
In alternative embodiment of the invention, can also realize between First ray data and the second serialized data Enter row format conversion, to meet different storage demands.It is illustrated in figure 4 two kinds of storage format (First rays of json forms Change the second serialized data of data and proto forms) relation schematic diagram of conversion is in fig. 4, right in the branch of the left side Jave object reflections, carry out object analysis, syntactic analysis and json morphological analyses, obtain the serialized data of json forms, The jave objects can be re-created by the serialized data of the json forms of the unserializing from memory block later;On the right side In the branch of side, to jave object reflections, object analysis, syntactic analysis and proto morphological analyses are carried out, obtain proto forms Serialized data, this can be re-created by the serialized data of the proto forms of the unserializing from memory block later Jave objects.As shown in Figure 4, proto components employ separable design in morphology layer, can support json forms, also can Proto forms are supported, and also other forms can be expanded.Because syntax format is communicated, support json and proto in morphology layer Face quickly mutually turns.
The numerous embodiments of the links of embodiment illustrated in fig. 1 are enumerated above, below by a specific embodiment The implementation process of data processing method of the invention is discussed in detail.In this embodiment, the instant messaging in mobile phone end Using (such as blue letter), the data that many of which is obtained from server end are all that directly with json forms, (that is, server end will After object sequence turns to the serialized data of json forms, be transferred to the instant messaging application of smart mobile phone) storage local, than Such as many organized lists, structuring configuration information, in order that programming is convenient.But some json Documents Comparisons are big, the parsing time is more long, It is not high using the overall performance for very frequently, causing to apply again.Using proto forms (such as Fig. 2 or figure provided in an embodiment of the present invention 3) after storing, the performance of application is greatly improved.
Fig. 5 A show the flow chart of data processing method according to another embodiment of the present invention.As shown in Figure 5A, the party Method at least comprises the following steps S502 to step S514.
Step S502, obtains the First ray data of json forms.
In this step, First ray data can be transmitted from external server end, or from this Ground memory storage (that is, the serialized data of json form of the reception from the transmission of external server end, and store to this Ground).
For example, certain company realizes that Enterprise Mobile work exchanges collaboration, indigo plant letter and server end with inside and outside using blue letter Interact, the First ray data of json forms can be obtained from server end, data are such as organized, including office worker Essential information, such as { " userid ":1,"name":"messi","address":[{"city":" Beijing ", " postcode":"10000","street":"wangjingdonglu"}]}。
First ray data are parsed by step S504, obtain the corresponding object of First ray data.
In this step, First ray data can be parsed using fastjson components, obtains First ray Change the corresponding object of data.
Step S506, obtain and analyze the attribute data of object, determines the length and property value of the Property Name of object Length.
In this step, attribute data can include Property Name and property value.Example is enumerated as with above-mentioned, office worker's object Property Name in attribute data can be userid, name, address, city, postcode and street, its correspondence Property value be respectively 1, messi, Beijing, 10000 and wangjingdonglu.Further, the related letter of attribute data Breath can including the type of attribute data, length etc., the relevant information of such as attribute data can be Property Name length with The length of property value.
Step S508, the length of object-based Property Name and the length of property value, the corresponding proto lattice of generation object Second serialized data of formula, wherein, the second serialized data includes the length of the attribute of object and the length of property value.
In this step, the mark that the mark and object that can also start including object in the second serialized data terminate, With the further efficiency and accuracy for improving parsing.Additionally, the type of object can also be included in the second serialized data, to carry The accuracy of height parsing.Here, the type of object can include integer, byte type, short, long, single-precision floating point type, Double-precision floating point type, character type, Boolean type, date, set, array, etc., the invention is not restricted to this.Specifically, with Fig. 2 institutes As a example by the proto forms for showing, the second sequence of the proto forms of the First ray data genaration of the above-mentioned json forms enumerated Change data as shown in Figure 5 B.Under total object first proto form of subobject is described in Fig. 5 B, and (object starts, object Type, key length, value length, key, value, object terminate), the description of subobject below is by that analogy.So, exist When parsing the second serialized data, proto forms that can be directly according to fixed length carry out data parsing, without character one by one It is compared to extract attribute and property value, the efficiency of data parsing can be improved.In embodiments of the present invention, shown in Fig. 5 B Second serialized data of proto forms can preserve into file, and file content is as follows:1361userid12/ 1345namemessi2/1342city Beijing 2/1385postcode100002 ....
It is right in the length generation of the length and property value of object-based Property Name in alternative embodiment of the invention Can also be that the Property Name and property value of object set up index during as corresponding second serialized data, and then by the second sequence The Property Name and property value of the object in rowization data represent that the storage such that it is able to save data is empty with the index value set up Between, and read-write amount also reduces therewith, and resolution speed is further enhanced.For example, can be each of total object in figure 5b The Property Name and property value of individual subobject set up index, can represent that userid, n represent that name, m are represented with index of reference value u Messi, c represent that city, b represent Beijing, and p represents that postcode, y represent 10000, replace specific using corresponding index value Property Name and property value, it is possible to reduce the memory space of data.Certainly, enumerate herein and be only illustrative, not to this Invention is limited.In actual applications, can be indexed with reference to correlation technique (such as dictionary), when meeting search index Efficiency.
In alternative embodiment of the invention, the second serialized data of generation can be character type data, i.e., using word Symbol type char [] data storage, rather than byte type byte [] data storage, eliminates the character string encoding and decoding time, and performance is obtained To further raising.
Step S510, the second serialized data of the corresponding proto forms of storage object.
In this step, serialized using proto, be converted directly into proto forms from lexical analyzer aspect and store Local.
Step S512, when receive it is local call the request of object when, obtain the corresponding proto forms of object of storage Second serialized data.
Step S514, the second serialized data of proto forms is parsed using the relevant information of attribute data, obtains this pair As, and it is supplied to called side.
In this step, it is mark that the relevant information of attribute data can start including object mark, object terminate, right The type of the length of the Property Name of elephant, the length of object's property value and object.
In alternative embodiment of the invention, if obtaining the First ray data of json forms from local storage, In order to mutually compatible with the json forms for storing before, from memory read out First ray data after, it is necessary to recognize form. Proto forms have format header (as shown in Figure 2 or Figure 3), can recognise that to be proto forms.
In the embodiment of the present invention, using customized proto forms, it includes the mark that object starts, object and terminates The type of mark, the length of the Property Name of object, the length of object's property value and object, it is quick, accurate to realize The purpose of the second serialized data of ground parsing proto forms, so as to the First ray data solved to json forms are entered , it is necessary to character is parsed one by one backward during row parsing, judge be ":" character such as (colon) or ", " (comma), and then To extract the attribute and property value of the object, such analyzing efficiency is not high, time-consuming more long, can cause to obtain the application of the object Interface there is the problem of interim card, influence Consumer's Experience.
Based on the data processing method that each embodiment above is provided, based on same inventive concept, the embodiment of the present invention is also There is provided a kind of data processing equipment.
Fig. 6 shows the structural representation of data processing equipment according to an embodiment of the invention.As shown in fig. 6, should Device can at least include parsing module 610, determining module 620, generation module 630 and memory module 640.
Now introduce the company between each composition of the data processing equipment of the embodiment of the present invention or the function and each several part of device Connect relation:
Parsing module 610, is suitable to obtain First ray data, and the First ray data are parsed, and obtains The corresponding object of the First ray data;
Determining module 620, is coupled with parsing module 610, is suitable to obtain and analyze the attribute data of the object, it is determined that The relevant information of the attribute data;
Generation module 630, is coupled with determining module 620, is suitable to the relevant information generation institute based on the attribute data State corresponding second serialized data of object;
Memory module 640, is coupled with generation module 630, is suitable to store corresponding second serialized data of the object.
In an embodiment of the present invention, as shown in fig. 7, the device of figure 6 above displaying can also include:
First processing module 710, is coupled with memory module 640, is suitable to store the object correspondence in memory module 640 The second serialized data after, when the request for locally calling the object is received, obtain storage the object correspondence The second serialized data;
Second serialized data is parsed using the relevant information of the attribute data, the object is obtained, and provide To called side.
In an embodiment of the present invention, as shown in fig. 7, the device of figure 6 above displaying can also include:
Second processing module 720, is coupled with memory module 640, is suitable to store the object correspondence in memory module 640 The second serialized data after, when the request of object described in external call is received, obtain storage the object correspondence The second serialized data;
Corresponding second serialized data of the object for obtaining is supplied to called side, by the called side using described The relevant information of attribute data parses second serialized data, obtains the object.
In an embodiment of the present invention, the attribute data includes attribute and property value, the related letter of the attribute data Breath includes the length of the length of attribute and property value.
In an embodiment of the present invention, above-mentioned generation module 630 is further adapted for:
The length of the attribute based on the object and the length of property value, generate the corresponding second serializing number of the object According to, wherein, second serialized data includes the length and the length of property value of the attribute of the object.
In an embodiment of the present invention, the mark and object for also starting including object in second serialized data terminate Mark.
In an embodiment of the present invention, the type of the object is also included in second serialized data.
In an embodiment of the present invention, the type of the object includes following one of any:Integer, byte type, short, Long, single-precision floating point type, double-precision floating point type, character type, Boolean type, date, set, array.
In an embodiment of the present invention, above-mentioned generation module 630 is further adapted for:
It is described when the relevant information based on the attribute data generates corresponding second serialized data of the object The attribute data of object sets up index;
The attribute data of the object in second serialized data is represented with the index value set up.
In an embodiment of the present invention, above-mentioned parsing module 610 is further adapted for:
Receive from outside First ray data;Or
First ray data are obtained from local storage.
In an embodiment of the present invention, second serialized data is character type data.
In an embodiment of the present invention, above-mentioned parsing module 610 is further adapted for:
When serialized data of the First ray data for json forms, using fastjson components to described the One serialized data is parsed.
According to the combination of above-mentioned any one preferred embodiment or multiple preferred embodiments, the embodiment of the present invention can reach Following beneficial effect:
In embodiments of the present invention, after to First ray data parse and obtaining its corresponding object, obtain simultaneously The attribute data of the object is analyzed, the relevant information of attribute data is determined;And then the relevant information generation based on attribute data should Corresponding second serialized data of object, and store.Because the second serialized data is that the relevant information based on attribute data is given birth to Into, when subsequently to call the object, the second serialized data that can be stored with direct access, using the correlation of attribute data Information is quickly and accurately parsed to the second serialized data, when being parsed to First ray data so as to solve There may be analyzing efficiency not high, time-consuming more long, and the corresponding object of First ray data is applied using frequently, causing should Overall performance problem not high.Also, the second serialized data of embodiment of the present invention generation does not have the category of lost objects Property data, it still can realize entering between the second serialized data and First ray data row format conversion, to meet Different storage demands.
In specification mentioned herein, numerous specific details are set forth.It is to be appreciated, however, that implementation of the invention Example can be put into practice in the case of without these details.In some instances, known method, structure is not been shown in detail And technology, so as not to obscure the understanding of this description.
Similarly, it will be appreciated that in order to simplify one or more that the disclosure and helping understands in each inventive aspect, exist Above to the description of exemplary embodiment of the invention in, each feature of the invention is grouped together into single implementation sometimes In example, figure or descriptions thereof.However, the method for the disclosure should be construed to reflect following intention:I.e. required guarantor The application claims of shield features more more than the feature being expressly recited in each claim.More precisely, such as following Claims reflect as, inventive aspect is all features less than single embodiment disclosed above.Therefore, Thus the claims for following specific embodiment are expressly incorporated in the specific embodiment, and wherein each claim is in itself All as separate embodiments of the invention.
Those skilled in the art are appreciated that can be carried out adaptively to the module in the equipment in embodiment Change and they are arranged in one or more equipment different from the embodiment.Can be the module or list in embodiment Unit or component be combined into a module or unit or component, and can be divided into addition multiple submodule or subelement or Sub-component.In addition at least some in such feature and/or process or unit exclude each other, can use any Combine to all features disclosed in this specification (including adjoint claim, summary and accompanying drawing) and so disclosed appoint Where all processes or unit of method or equipment are combined.Unless expressly stated otherwise, this specification (including adjoint power Profit is required, summary and accompanying drawing) disclosed in each feature can the alternative features of or similar purpose identical, equivalent by offer carry out generation Replace.
Although additionally, it will be appreciated by those of skill in the art that some embodiments described herein include other embodiments In included some features rather than further feature, but the combination of the feature of different embodiments means in of the invention Within the scope of and form different embodiments.For example, in detail in the claims, embodiment required for protection it is one of any Mode can use in any combination.
All parts embodiment of the invention can be realized with hardware, or be run with one or more processor Software module realize, or with combinations thereof realize.It will be understood by those of skill in the art that can use in practice Microprocessor or digital signal processor (DSP) come realize in data processing equipment according to embodiments of the present invention some or The some or all functions of person's whole part.The present invention is also implemented as perform method as described herein one Point or whole equipment or program of device (for example, computer program and computer program product).It is such to realize this hair Bright program can be stored on a computer-readable medium, or can have the form of one or more signal.It is such Signal can be downloaded from internet website and obtained, or be provided on carrier signal, or be provided in any other form.
It should be noted that above-described embodiment the present invention will be described rather than limiting the invention, and ability Field technique personnel can design alternative embodiment without departing from the scope of the appended claims.In the claims, Any reference symbol being located between bracket should not be configured to limitations on claims.Word "comprising" is not excluded the presence of not Element listed in the claims or step.Word "a" or "an" before element is not excluded the presence of as multiple Element.The present invention can come real by means of the hardware for including some different elements and by means of properly programmed computer It is existing.If in the unit claim for listing equipment for drying, several in these devices can be by same hardware branch To embody.The use of word first, second, and third does not indicate that any order.These words can be explained and run after fame Claim.
So far, although those skilled in the art will appreciate that detailed herein have shown and described multiple of the invention and show Example property embodiment, but, without departing from the spirit and scope of the present invention, still can be direct according to present disclosure It is determined that or deriving many other variations or modifications for meeting the principle of the invention.Therefore, the scope of the present invention is understood that and recognizes It is set to and covers all these other variations or modifications.
The one side of the embodiment of the present invention, there is provided A1, a kind of data processing method, including:
First ray data are obtained, the First ray data are parsed, obtain the First ray number According to corresponding object;
The attribute data of the object is obtained and analyzed, the relevant information of the attribute data is determined;
Relevant information based on the attribute data generates corresponding second serialized data of the object;
Store corresponding second serialized data of the object.
A2, the method according to A1, wherein, it is described after corresponding second serialized data of the object is stored Method also includes:
When the request for locally calling the object is received, the corresponding second serializing number of the object of storage is obtained According to;
Second serialized data is parsed using the relevant information of the attribute data, the object is obtained, and provide To called side.
A3, the method according to A1, wherein, it is described after corresponding second serialized data of the object is stored Method also includes:
When the request of object described in external call is received, the corresponding second serializing number of the object of storage is obtained According to;
Corresponding second serialized data of the object for obtaining is supplied to called side, by the called side using described The relevant information of attribute data parses second serialized data, obtains the object.
A4, the method according to any one of A1-A3, wherein, the attribute data includes attribute and property value, described The relevant information of attribute data includes the length of attribute and the length of property value.
A5, the method according to A4, wherein, it is corresponding that the relevant information based on the attribute data generates the object Second serialized data, including:
The length of the attribute based on the object and the length of property value, generate the corresponding second serializing number of the object According to, wherein, second serialized data includes the length and the length of property value of the attribute of the object.
A6, the method according to A5, wherein, the mark that also starts including object in second serialized data and right As the mark for terminating.
A7, the method according to A5 or A6, wherein, the class of the object is also included in second serialized data Type.
A8, the method according to A7, wherein, the type of the object includes following one of any:Integer, byte type, Short, long, single-precision floating point type, double-precision floating point type, character type, Boolean type, date, set, array.
A9, the method according to any one of A1-A8, wherein, in the relevant information generation based on the attribute data During corresponding second serialized data of the object, methods described also includes:
For the attribute data of the object sets up index;
The attribute data of the object in second serialized data is represented with the index value set up.
A10, the method according to any one of A1-A9, wherein, the acquisition First ray data, including:
Receive from outside First ray data;Or
First ray data are obtained from local storage.
A11, the method according to any one of A1-A10, wherein, second serialized data is character type data.
A12, the method according to any one of A1-A11, wherein, when the First ray data are json forms Serialized data when, the First ray data are parsed, including:
The First ray data are parsed using fastjson components.
The another aspect of the embodiment of the present invention, additionally provides B13, a kind of data processing equipment, including:
Parsing module, is suitable to obtain First ray data, and the First ray data are parsed, and obtains described The corresponding object of First ray data;
Determining module, is suitable to obtain and analyze the attribute data of the object, determines the relevant information of the attribute data;
Generation module, is suitable to the relevant information based on the attribute data and generates the corresponding second serializing number of the object According to;
Memory module, is suitable to store corresponding second serialized data of the object.
B14, the device according to B13, wherein, also include:
First processing module, is suitable to after the memory module stores corresponding second serialized data of the object, When the request for locally calling the object is received, corresponding second serialized data of the object of storage is obtained;
Second serialized data is parsed using the relevant information of the attribute data, the object is obtained, and provide To called side.
B15, the device according to B13, wherein, also include:
Second processing module, is suitable to after the memory module stores corresponding second serialized data of the object, When the request of object described in external call is received, corresponding second serialized data of the object of storage is obtained;
Corresponding second serialized data of the object for obtaining is supplied to called side, by the called side using described The relevant information of attribute data parses second serialized data, obtains the object.
B16, the device according to any one of B13-B15, wherein, the attribute data includes attribute and property value, The relevant information of the attribute data includes the length of attribute and the length of property value.
B17, the device according to B16, wherein, the generation module is further adapted for:
The length of the attribute based on the object and the length of property value, generate the corresponding second serializing number of the object According to, wherein, second serialized data includes the length and the length of property value of the attribute of the object.
B18, the device according to B17, wherein, the mark that also starts including object in second serialized data and The mark that object terminates.
B19, the device according to B17 or B18, wherein, the object is also included in second serialized data Type.
B20, the device according to B19, wherein, the type of the object includes following one of any:Integer, byte Type, short, long, single-precision floating point type, double-precision floating point type, character type, Boolean type, date, set, array.
B21, the device according to any one of B13-20, wherein, the generation module is further adapted for:
It is described when the relevant information based on the attribute data generates corresponding second serialized data of the object The attribute data of object sets up index;
The attribute data of the object in second serialized data is represented with the index value set up.
B22, the device according to any one of B13-B21, wherein, the parsing module is further adapted for:
Receive from outside First ray data;Or
First ray data are obtained from local storage.
B23, the device according to any one of B13-B22, wherein, second serialized data is character type number According to.
B24, the device according to any one of B13-B23, wherein, the parsing module is further adapted for:
When serialized data of the First ray data for json forms, using fastjson components to described the One serialized data is parsed.

Claims (10)

1. a kind of data processing method, including:
First ray data are obtained, the First ray data are parsed, obtain the First ray data pair The object answered;
The attribute data of the object is obtained and analyzed, the relevant information of the attribute data is determined;
Relevant information based on the attribute data generates corresponding second serialized data of the object;
Store corresponding second serialized data of the object.
2. method according to claim 1, wherein, after corresponding second serialized data of the object is stored, institute Stating method also includes:
When the request for locally calling the object is received, corresponding second serialized data of the object of storage is obtained;
Second serialized data is parsed using the relevant information of the attribute data, the object is obtained, and be supplied to tune With side.
3. method according to claim 1, wherein, after corresponding second serialized data of the object is stored, institute Stating method also includes:
When the request of object described in external call is received, corresponding second serialized data of the object of storage is obtained;
Corresponding second serialized data of the object for obtaining is supplied to called side, the attribute is utilized by the called side The relevant information of data parses second serialized data, obtains the object.
4. the method according to any one of claim 1-3, wherein, the attribute data includes attribute and property value, institute Stating the relevant information of attribute data includes the length of attribute and the length of property value.
5. method according to claim 4, wherein, the relevant information based on the attribute data generates the object correspondence The second serialized data, including:
The length of the attribute based on the object and the length of property value, generate corresponding second serialized data of the object, Wherein, second serialized data includes the length and the length of property value of the attribute of the object.
6. method according to claim 5, wherein, the mark that also starts including object in second serialized data and The mark that object terminates.
7. the method according to claim 5 or 6, wherein, the also class including the object in second serialized data Type.
8. method according to claim 7, wherein, the type of the object includes following one of any:Integer, byte Type, short, long, single-precision floating point type, double-precision floating point type, character type, Boolean type, date, set, array.
9. the method according to any one of claim 1-8, wherein, in the relevant information generation based on the attribute data During corresponding second serialized data of the object, methods described also includes:
For the attribute data of the object sets up index;
The attribute data of the object in second serialized data is represented with the index value set up.
10. a kind of data processing equipment, including:
Parsing module, is suitable to obtain First ray data, and the First ray data are parsed, and obtains described first The corresponding object of serialized data;
Determining module, is suitable to obtain and analyze the attribute data of the object, determines the relevant information of the attribute data;
Generation module, is suitable to the relevant information based on the attribute data and generates corresponding second serialized data of the object;
Memory module, is suitable to store corresponding second serialized data of the object.
CN201611124614.6A 2016-12-08 2016-12-08 Data processing method and device Pending CN106775696A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611124614.6A CN106775696A (en) 2016-12-08 2016-12-08 Data processing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611124614.6A CN106775696A (en) 2016-12-08 2016-12-08 Data processing method and device

Publications (1)

Publication Number Publication Date
CN106775696A true CN106775696A (en) 2017-05-31

Family

ID=58881900

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611124614.6A Pending CN106775696A (en) 2016-12-08 2016-12-08 Data processing method and device

Country Status (1)

Country Link
CN (1) CN106775696A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107301208A (en) * 2017-06-02 2017-10-27 北京奇虎科技有限公司 A kind of tables of data treating method and apparatus
CN108540477A (en) * 2018-04-13 2018-09-14 腾讯科技(成都)有限公司 The method and relevant device of data communication
CN109901822A (en) * 2019-01-08 2019-06-18 平安科技(深圳)有限公司 The mutual shifting method of object, device and storage medium, server
CN109902274A (en) * 2019-02-22 2019-06-18 今天东翼科技有限公司 A kind of method and system converting json character string to thrift binary stream
CN112783581A (en) * 2021-01-21 2021-05-11 上海哔哩哔哩科技有限公司 Data processing method and device
CN114281429A (en) * 2021-12-31 2022-04-05 上海匠岩智能科技有限公司 JSON data analysis method based on array variables
WO2023124425A1 (en) * 2021-12-30 2023-07-06 深圳市兆珑科技有限公司 Data processing method and apparatus, electronic device, and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102981884A (en) * 2012-11-22 2013-03-20 用友软件股份有限公司 Serializing device and serializing method
CN103970737A (en) * 2013-01-24 2014-08-06 腾讯科技(深圳)有限公司 Data constitution method and device
CN105468606A (en) * 2014-08-25 2016-04-06 小米科技有限责任公司 Webpage saving method and device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102981884A (en) * 2012-11-22 2013-03-20 用友软件股份有限公司 Serializing device and serializing method
CN103970737A (en) * 2013-01-24 2014-08-06 腾讯科技(深圳)有限公司 Data constitution method and device
CN105468606A (en) * 2014-08-25 2016-04-06 小米科技有限责任公司 Webpage saving method and device

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107301208A (en) * 2017-06-02 2017-10-27 北京奇虎科技有限公司 A kind of tables of data treating method and apparatus
CN108540477A (en) * 2018-04-13 2018-09-14 腾讯科技(成都)有限公司 The method and relevant device of data communication
CN109901822A (en) * 2019-01-08 2019-06-18 平安科技(深圳)有限公司 The mutual shifting method of object, device and storage medium, server
CN109901822B (en) * 2019-01-08 2022-08-12 平安科技(深圳)有限公司 Object conversion method and device, storage medium and server
CN109902274A (en) * 2019-02-22 2019-06-18 今天东翼科技有限公司 A kind of method and system converting json character string to thrift binary stream
CN112783581A (en) * 2021-01-21 2021-05-11 上海哔哩哔哩科技有限公司 Data processing method and device
WO2023124425A1 (en) * 2021-12-30 2023-07-06 深圳市兆珑科技有限公司 Data processing method and apparatus, electronic device, and storage medium
CN114281429A (en) * 2021-12-31 2022-04-05 上海匠岩智能科技有限公司 JSON data analysis method based on array variables

Similar Documents

Publication Publication Date Title
CN106775696A (en) Data processing method and device
US11157560B2 (en) System and method for managing graph data
CN105760534B (en) Data structure, hadoop cluster, server and its application method of customized serializability
US8418142B2 (en) Architecture for data validation
CN107015948B (en) Log information formatting method and system
US9043757B2 (en) Identifying differences between source codes of different versions of a software when each source code is organized using incorporated files
TW201246071A (en) Unrolling quantifications to control in-degree and/or out degree of automation
JP2003510674A (en) How to convert 2D data to standard representation
CN109558128A (en) Json data analysis method, device and computer readable storage medium
CN106648569B (en) Target serialization realization method and device
CN111443901B (en) Java reflection-based service expansion method and device
CN112346775B (en) Index data general processing method, electronic device and storage medium
CN111782211B (en) Dynamic generation method and device for parameter page based on Vue component
CN110109983B (en) Method and device for operating Redis database
CN113238740A (en) Code generation method, code generation device, storage medium, and electronic apparatus
US20170277732A1 (en) Interpreting hl7 segment hierarchy dynamically
CN109669678A (en) Template engine integration method, device, electronic equipment and storage medium
CN105868270A (en) Android object storing framework based on reflection mechanism
CN108959294B (en) Method and device for accessing search engine
CN113127776A (en) Breadcrumb path generation method and device and terminal equipment
CN111124541A (en) Configuration file generation method, device, equipment and medium
US10482171B2 (en) Digital form optimization
Nolan et al. Javascript object notation
CN115495421A (en) Data formatting method, device, electronic equipment and medium
CN114968917A (en) Method and device for rapidly importing file data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: 100041 A-0337, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing.

Applicant after: Blue letter mobile (Beijing) Technology Co., Ltd.

Address before: 100041 A-0337, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing.

Applicant before: Lanseen workshop (Beijing) Technology Co. Ltd.

CB02 Change of applicant information
CB03 Change of inventor or designer information

Inventor after: An Ning

Inventor before: An Ning

Inventor before: Liu Bin

Inventor before: Liu Jian

Inventor before: Liu Chenguang

CB03 Change of inventor or designer information
RJ01 Rejection of invention patent application after publication

Application publication date: 20170531

RJ01 Rejection of invention patent application after publication