WO2017059798A1 - 序列化与反序列化的方法、装置、系统以及电子设备 - Google Patents

序列化与反序列化的方法、装置、系统以及电子设备 Download PDF

Info

Publication number
WO2017059798A1
WO2017059798A1 PCT/CN2016/101448 CN2016101448W WO2017059798A1 WO 2017059798 A1 WO2017059798 A1 WO 2017059798A1 CN 2016101448 W CN2016101448 W CN 2016101448W WO 2017059798 A1 WO2017059798 A1 WO 2017059798A1
Authority
WO
WIPO (PCT)
Prior art keywords
attribute
data object
description file
metadata
description
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/101448
Other languages
English (en)
French (fr)
Inventor
董石鸣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Publication of WO2017059798A1 publication Critical patent/WO2017059798A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/903Querying

Definitions

  • the present invention relates to the field of computer communication technologies, and in particular, to a method for serializing data objects, a device for serializing data objects, and an electronic device; a method for deserializing a data stream, and a pair of data A device for stream deserialization and an electronic device; a serialization and deserialization system.
  • the object serialization technique is a technique for converting object data into another form of persistent data, that is, a process of converting state information of an object into a form that can be stored or transmitted.
  • an object writes its current state to a temporary or persistent store.
  • the object can be recreated by reading or deserializing the state of the object from the store.
  • Protobuf is a data exchange format that serializes existing information into binary streams according to different types according to different types, for saving information or data transmission. Upon reading the binary information or receiving the binary information, the serialized binary stream is deserialized to obtain the original information or data.
  • Protobuf can reduce the metadata content in the serialization result during serialization, for compatibility, a small amount of metadata (attribute type, attribute number) is retained, and in the case of deserialization, in addition to obtaining a binary stream, It is also necessary to know the type used in serialization and to perform corresponding deserialization according to the type.
  • the present application provides a method for serializing a data object, a device for serializing a data object, an electronic device, a method for deserializing a data stream, and a device for deserializing a data stream, An electronic device and a serialization and deserialization system to solve the problem of low serialization compression rate in the prior art.
  • the application provides a method for serializing data objects, and the method for serializing data objects includes:
  • the read attribute values are written to the result data stream of the data object in the order of the attribute description sequence number, and the result data stream is used as a serialization result of the data object.
  • the attribute description further includes: an attribute type.
  • the method includes:
  • the corresponding attribute is given a predetermined default value for the attribute description.
  • the metadata description file of the data object to be serialized is:
  • the method before the step of writing the read attribute value into the result data stream of the data object in the order of the attribute description sequence number, the method includes:
  • the method before the step of acquiring the metadata description file of the data object to be serialized, the method further includes:
  • the modifying operation includes: adding, deleting, or changing an attribute of the data object to be serialized;
  • a new version of the metadata description file is created according to the modification operation, and the attribute description in the metadata description file is changed according to the modification operation based on the attribute description in the original version metadata description file.
  • the metadata description file is created according to the modifying operation, and the attribute description in the metadata description file is performed according to the attribute description in the original version metadata description file according to the modification operation.
  • the steps to change include:
  • the metadata description file in which the version number is recorded is saved in the metadata repository.
  • the method includes:
  • the read attribute values are compressed.
  • the present application further provides an apparatus for serializing data objects, where the apparatus for serializing data objects includes: a description file acquisition unit, an attribute operation class acquisition unit, an attribute reading unit, and a serialization write unit. ;
  • the description file obtaining unit is configured to acquire a metadata description file of the data object to be serialized, where the metadata description file includes an attribute description identified by a serial number and an attribute operation class name corresponding to an attribute in the data object;
  • the attribute operation class obtaining unit is configured to obtain an attribute operation class name corresponding to each attribute according to the attribute operation class name provided by each attribute description in the metadata description file;
  • the attribute reading unit is configured to read, according to a reading method provided by the attribute operation class corresponding to each attribute, an attribute value corresponding to each attribute description in the data object to be serialized;
  • the serialization writing unit is configured to write the read attribute values into the result data stream of the data object in the order of the attribute description sequence number, and the result data stream is used as a serialization result of the data object.
  • the metadata description file of the data object to be serialized acquired by the description file acquiring unit further includes an attribute type.
  • the attribute reading unit further includes: an attribute type reading subunit, a default attribute operation class obtaining subunit, and a default value reading subunit;
  • the attribute type reading subunit is configured to read an attribute value corresponding to each attribute description in the data object to be serialized, and obtain an attribute description attribute that cannot be read because the attribute operation class of the corresponding attribute cannot be obtained.
  • the default attribute operation class acquisition sub-unit is configured to retrieve, according to the attribute type, a default attribute operation class corresponding to the attribute type;
  • the default value reading subunit is configured to assign a predetermined default value to the corresponding attribute of the attribute description according to the default attribute operation class.
  • the description file acquiring unit is configured to: search, according to the name of the data object to be serialized, the to-be-sequence corresponding to the data object to be serialized and having the largest version number value from the metadata repository. Metadata description file for the data object.
  • the serialized writing unit further includes: a version number writing subunit;
  • the version number is written into the subunit, and is configured to read the data object to be serialized before the attribute value to be read is written into the result data stream of the data object in the order of the attribute description sequence number.
  • the metadata describes the version number of the file and writes it to the head of the resulting data stream of the data object to be serialized.
  • the device for serializing the data object further includes: a modification operation receiving unit, an operation class creation unit, and a description file creation unit;
  • the modifying operation receiving unit is configured to receive, before the acquiring the metadata description file of the data object to be serialized, a modification operation on the attribute of the data object to be serialized; the modifying operation includes: Adding, deleting, or changing the attributes of the serialized data object;
  • the operation class creation unit is configured to create an attribute operation class for the newly added attribute of the data object to be serialized based on the modification operation of the data object to be serialized;
  • a description file creation unit configured to create a new version of the metadata description file according to the modification operation, where the attribute description in the metadata description file is based on the attribute description in the original version metadata description file, according to the modification operation The change was made.
  • the device for serializing the data object further includes: a traversal unit, a value obtaining unit, a version number generating unit, and a saving unit;
  • the traversing unit is configured to create a new version of the metadata description file according to the modification operation, where the attribute description in the metadata description file is based on the attribute description in the original version metadata description file, according to the modification After the operation is changed, the metadata description file set for the data object to be serialized is traversed in the metadata repository storing the metadata description file according to the name of the data object to be serialized;
  • the value obtaining unit is configured to acquire, during the traversal process, a maximum value of a version number of the metadata description file of the data object to be serialized in the metadata repository;
  • the version number generating unit is configured to add the value by one, and use the added value as the version number of the metadata version file of the new version;
  • the saving unit is configured to save the metadata description file with the version number recorded in the metadata repository.
  • the device for serializing the data object further includes: an attribute value compression unit;
  • the attribute value compression unit is configured to read the attribute value after the attribute value corresponding to each attribute description in the data object to be serialized according to the reading method provided by the attribute operation class corresponding to each attribute Compress.
  • an embodiment of the present application further provides an electronic device, where the electronic device includes:
  • a memory for storing a serialization program
  • the program when being read and executed by the processor, performing an operation of: acquiring a metadata description file of a data object to be serialized, where the metadata description file includes a serial number identifier
  • the attribute description and the attribute operation class name corresponding to the attribute in the data object; the attribute operation class name provided according to each attribute description in the metadata description file, and the attribute operation class corresponding to each attribute are obtained; according to the corresponding attribute
  • the reading method provided by the attribute operation class reads the attribute value corresponding to each attribute description in the data object to be serialized; and writes the read attribute value into the result data of the data object in the order of the attribute description serial number. Flow, the resulting data stream as a serialization result of the data object.
  • the present application also provides a method for deserializing a data stream, and the method for deserializing the data stream includes:
  • the metadata description file includes an attribute description identified by a serial number and an attribute operation class name corresponding to an attribute in the data object;
  • the attribute values in the result data stream are sequentially read in the order of the attribute description sequence number
  • the read attribute values are written into the attributes of the local data object corresponding to the respective attribute descriptions according to the write method provided by the attribute operation class described by each attribute.
  • the attribute description further includes: an attribute type.
  • the method further includes:
  • the default attribute operation class corresponding to the attribute type is obtained according to the attribute type described by the attribute;
  • the data in the result data stream read is discarded.
  • the obtaining the metadata description file of the local data object includes:
  • a metadata description file of the local data object having the same name as the local data object and having the same version number as the first value in the result data stream is obtained.
  • the method includes:
  • the attribute values read from the resulting data stream are decompressed.
  • the present application further provides an apparatus for deserializing a data stream, where the apparatus for deserializing the data stream includes: a data stream obtaining unit, a description file acquiring unit, an attribute operation class acquiring unit, and an attribute value reading. The unit and the attribute value are written into the unit;
  • the data stream obtaining unit is configured to obtain a result data stream
  • the description file obtaining unit is configured to acquire a metadata description file of the local data object, where the metadata description file includes an attribute description identified by a serial number and an attribute operation class name corresponding to an attribute in the data object;
  • the attribute operation class obtaining unit is configured to obtain an attribute operation class name corresponding to each attribute according to the attribute operation class name provided by each attribute description in the metadata description file;
  • the attribute value reading unit is configured to sequentially read the attribute values in the result data stream according to the sequence of the attribute description sequence according to the reading method provided by the attribute operation class corresponding to each attribute;
  • the attribute value writing unit is configured to write the read attribute value into the attribute of the local data object corresponding to each attribute description according to the writing method provided by the attribute operation class described by each attribute .
  • the description file obtaining unit, the obtained metadata description file of the local data object further includes an attribute type.
  • the attribute value reading unit includes: an attribute type obtaining subunit, a data reading subunit, and a discarding subunit;
  • the attribute type obtaining subunit is configured to sequentially read the attribute values in the result data stream according to the sequence of the attribute description serial number in the writing method provided by the attribute operation class according to each attribute description.
  • the default attribute operation class corresponding to the attribute type is obtained according to the attribute type described by the attribute;
  • the data reading subunit is configured to acquire a reading method provided by the default attribute operation class, and read data in the result data stream according to an order of the attribute description sequence number;
  • the discarding a single sub-unit is configured to discard the data in the read result data stream.
  • the description file obtaining unit includes: a local data object name obtaining subunit, a result data stream parsing subunit, a description file traversing subunit, and a description file acquiring subunit;
  • the local data object name obtaining subunit is configured to acquire a name of the local data object
  • the result data stream parsing subunit is configured to parse the result data stream to obtain a first value in the result data stream;
  • the description file obtaining subunit configured to acquire metadata of the local data object having the same name as the local data object and having the same version number as the first value in the result data stream during the traversal process Description file.
  • the device for deserializing the data stream further includes: an attribute value decompression unit;
  • the attribute value decompressing unit is configured to sequentially read the attribute values in the result data stream according to the order of the attribute description sequence in the reading method provided by the attribute operation class corresponding to each attribute, and then From the stated As a result, the attribute values read out in the data stream are decompressed.
  • an embodiment of the present application further provides an electronic device, where the electronic device includes:
  • a memory for storing a deserialization program, the program, when being read and executed by the processor, performing an operation of: acquiring a result data stream, acquiring a metadata description file of the local data object, the metadata description
  • the file includes an attribute description identified by a sequence number and an attribute operation class name corresponding to the attribute in the data object, and an attribute operation class name corresponding to each attribute description in the metadata description file is obtained, and an attribute operation class corresponding to each attribute is obtained,
  • the attribute values in the result data stream are sequentially read in the order of the attribute description sequence number, and the writing method provided by the attribute operation class according to each attribute description is The read attribute values are written into the attributes of the local data object corresponding to the respective attribute descriptions.
  • serialization and deserialization system comprising:
  • a device for serializing a data object according to any of the preceding claims and the device for deserializing the data stream according to any of the preceding claims.
  • the method and apparatus for serializing a data object obtains a metadata description file of a data object to be serialized, and the metadata description file includes an attribute description identified by a serial number, and each The attribute description includes the attribute operation class name of the corresponding data object attribute; according to the attribute operation class name, the attribute operation class of the corresponding attribute can be obtained, and the reading method provided by the attribute operation class corresponding to each attribute may be Reading the attribute values corresponding to the respective attribute descriptions in the data objects to be serialized, and writing the read attribute values into the result data stream of the data object in the order of the attribute description sequence numbers, where the result data stream is used as the The serialization result of the data object.
  • the method and apparatus for deserializing a data stream obtains a metadata description file of the local data object by acquiring a result data stream, where the metadata description file includes an attribute description identified by a serial number and corresponding to the description
  • the attribute operation class name of the attribute in the data object according to the attribute operation class name provided by each attribute description in the metadata description file, obtaining the attribute operation class corresponding to each attribute, and reading according to the attribute operation class corresponding to each attribute
  • the method sequentially reads the attribute values in the result data stream according to the sequence of the attribute description sequence numbers, and writes the read attribute values into each attribute according to the writing method provided by the attribute operation class described by each attribute. Describe the properties of the corresponding local data object.
  • the above technical solution solves the compatibility problem of serialization and deserialization between systems after changing the attributes of the object, It also saves storage space and transmission traffic when saving information and data transmission by completely splitting the metadata of the object and the value of the object, and reduces the processing overhead of the system when deserializing.
  • FIG. 1 shows a flow chart of a method of serializing data objects provided in accordance with an embodiment of the present application
  • FIG. 2 shows a flowchart of creating a metadata description file provided in accordance with an embodiment of the present application
  • FIG. 3 illustrates a flow chart of saving a metadata description file provided in accordance with an embodiment of the present application
  • FIG. 4 shows a flow chart of a method of reading default values at serialization provided in accordance with an embodiment of the present application.
  • FIG. 5 shows a schematic diagram of an apparatus for serializing data objects provided in accordance with an embodiment of the present application
  • FIG. 6 shows a schematic diagram of an electronic device provided in accordance with an embodiment of the present application.
  • FIG. 7 shows a flow diagram of a method of deserializing a data stream, provided in accordance with an embodiment of the present application
  • FIG. 8 is a flowchart showing a metadata description file for acquiring the local data object according to an embodiment of the present application.
  • FIG. 9 is a flowchart of a method for discarding a default value at the time of deserialization according to an embodiment of the present application.
  • FIG. 10 shows a schematic diagram of an apparatus for deserializing a data stream, provided in accordance with an embodiment of the present application
  • FIG. 11 shows a schematic diagram of an electronic device provided in accordance with an embodiment of the present application.
  • FIG. 12 shows a schematic diagram of a serialization and deserialization system provided in accordance with an embodiment of the present application.
  • Embodiments of the present application provide a method for serializing a data object, a device for serializing a data object, a method for deserializing a data stream, a device for deserializing a data stream, and a device A serialization and deserialization system, the embodiment of the present application simultaneously provides an electronic device. Detailed description will be made one by one in the following embodiments.
  • the technical solution of the present application is directed to data transmission or storage between systems in a distributed system environment, but the application field thereof is not limited to a distributed system, and in an existing system or application, It is possible to perform serialization and deserialization methods, and the methods provided by the present application can be used, and corresponding beneficial effects can be obtained.
  • the object serialization technique is a technique for converting object data into another form of persistent data, that is, a process of converting state information of an object into a form that can be stored or transmitted.
  • an object writes its current state to a temporary or persistent store, which can be recreated by reading or deserializing the state of the object from the store.
  • serialization methods and deserialization methods are widely used in data transmission and application between networks. It is usually necessary to convert information into data streams for data transmission and application.
  • Embodiments of the present application provide a method of serializing data objects.
  • the method for serializing data objects is as follows:
  • FIG. 1 shows a flowchart of a method for serializing data objects according to an embodiment of the present application.
  • the method for serializing data objects includes:
  • Step S101 Obtain a metadata description file of a data object to be serialized, where the metadata description file includes an attribute description identified by a sequence number and an attribute operation class name corresponding to an attribute in the data object.
  • the obtaining a metadata description file of the data object to be serialized may be implemented by searching for a corresponding sequence from the metadata repository according to the name of the data object to be serialized.
  • the metadata warehouse is a database for storing metadata and metadata related information (such as a metadata description file), and the metadata warehouse may generally adopt a relational database.
  • the method of serializing the data object is to understand the structural composition of the data according to the description semantics of the metadata description file. It can be understood that, when the metadata description file of the data object to be serialized is obtained in step S101, the to-serialization corresponding to the data object to be serialized and having the largest version number value is searched from the metadata repository.
  • the metadata description file of the data object, that is, the metadata version description file obtained in this step is the latest version.
  • the metadata description file includes a plurality of attribute descriptions corresponding to attributes in the data object, each attribute description is identified by a sequence number, and includes an attribute operation class name corresponding to the attribute in the data object.
  • each attribute description in the metadata description file is described. Also includes: the attribute type described for the attribute of the strip.
  • the metadata description file can be as follows:
  • the numbers "1" and "2" are the sequence numbers of each attribute description
  • the UserIdFieldDef is the attribute operation class name corresponding to the attribute UserId in the data object
  • the AgeFieldDef is the attribute operation class name corresponding to the attribute Age in the data object
  • VARINT is the attribute type described by this attribute.
  • the metadata description file is data describing other data, or structural data. Specifically, the metadata description file is data describing an object such as an information resource or data.
  • the metadata description file can be shared.
  • the development and utilization environment of metadata description files is often a changing distributed environment. Since the metadata description file is also data, it can be stored and retrieved in the metadata repository by a similar data method, which is convenient for sharing and use in a distributed environment.
  • the data object to be serialized is (the attribute value is not shown):
  • a metadata description file of the data object to be serialized corresponding to the data object User to be serialized and having the highest version number value, for example, from the metadata repository, for example :
  • the data object described in the metadata description file described above does not exactly match the example User of the aforementioned data object. This is because the metadata description file is the latest version and is determined according to the upgraded version of User.
  • the data object is a mapping of entities such as people, things, objects, etc. existing in the objective world in computer logic.
  • the attributes thereof include a book number, an author, a publisher, a comment, and the like, and after serializing the book, a binary data stream corresponding to the book is generated, the binary data stream having attributes with the book.
  • Corresponding attribute values thereby representing entities such as people, things, objects, etc. that exist in the objective world through computer language.
  • the data object can be any entity, thing, object, etc. that exists in the objective world.
  • the data object in this example is the user, that is, User.
  • the object includes at least one attribute.
  • the name of the attribute is called the attribute name.
  • the value corresponding to the attribute is called the attribute value.
  • the attribute value can be an integer type such as an integer type, a character type, a floating point type, or a string type, for example, an example of the above data object.
  • the attribute userId is an integer
  • the attribute name is a string type
  • the attribute age is an integer.
  • the metadata description file needs to be defined or created, and specifically includes steps S100-1 to S100-3, which will be further described below with reference to FIG. 2 .
  • FIG. 2 shows a flowchart of creating a metadata description file according to an embodiment of the present application.
  • Step S100-1 Receive a modification operation on an attribute of the data object to be serialized; the modifying operation includes: adding, deleting, or changing an attribute of the data object to be serialized.
  • the receiving a modification operation on an attribute of the data object to be serialized includes: receiving an addition, deletion, or change of an attribute in the data object to be serialized. For example, add the attribute address of type string to the data object User, delete the age attribute of the data object User, or modify the name of the userId attribute in the data object to id.
  • Step S100-2 Create an attribute operation class for the newly added attribute of the data object to be serialized based on the modification operation of the data object to be serialized.
  • This step creates an attribute operation class for the newly added attribute according to the modification operation of the attribute of the data object to be serialized received in step S100-1. For example, if the attribute address of type string is added in step S100-1, an attribute operation class named AddressFieldDef is created for the attribute address.
  • the AddressFieldDef class encapsulates the getter and setter methods.
  • the attribute operation class of the corresponding attribute may not be deleted; if the modification operation is to modify the attribute, the original attribute is deleted, and the modified attribute is created, and the modified attribute is modified.
  • the properties of the corresponding property action class are created.
  • Step S100-3 Create a new version of the metadata description file according to the modification operation, and the attribute description in the metadata description file is changed according to the modification operation based on the attribute description in the original version metadata description file.
  • the new version of the metadata description file is created according to the modification operation, and the attribute description in the metadata description file is performed according to the modification operation on the basis of the attribute description in the original version metadata description file.
  • the change may be implemented by acquiring the attribute of the data object to be serialized after receiving the modification operation, generating the sequence number of the attribute description in the order defined by the attribute in the data object, and acquiring the data corresponding to the serialization to be serialized.
  • the attribute of the attribute in the object manipulates the class name, and obtains the attribute type of the attribute in the data object to be serialized, and writes the attribute operation class name and the attribute type into each attribute description in sequence according to the order defined by the attribute in the data object.
  • all the generated attribute descriptions are used as the metadata version file of the new version.
  • Steps S100-4 to S100-7 are included, which will be further described below in conjunction with FIG. 3.
  • FIG. 3 shows a flowchart of saving a metadata description file according to an embodiment of the present application.
  • Step S100-4 traversing a metadata description file set for the data object to be serialized in a metadata repository in which the metadata description file is stored according to the name of the data object to be serialized.
  • the name of the data object to be serialized is User
  • the metadata description file set for the data object User is traversed in the metadata repository in which the metadata description file is stored.
  • Step S100-5 in the traversal process, acquiring the data object to be serialized in the metadata repository
  • the metadata describes the version number of the file with the largest value.
  • step S100-6 the value is incremented by one, and the added value is used as the version number of the metadata version file of the new version.
  • the added value is mapped to the new version of the metadata description file created in step S100-3.
  • Step S100-7 storing the metadata description file with the version number recorded in the metadata repository.
  • step S100-1 to step S100-7 may be created in advance, or may be configured on site when performing the method of serializing data objects, which is not limited.
  • Step S103 Obtain an attribute operation class name corresponding to each attribute description in the metadata description file, and obtain an attribute operation class corresponding to each attribute.
  • the attribute operation class name provided by each attribute description in the metadata description file is obtained, and the attribute operation class corresponding to each attribute is obtained, which may be implemented in the following manner: according to the metadata description file Each attribute describes the provided attribute action class name, retrieves and retrieves the same attribute action class as the attribute action class name provided by each attribute description.
  • the data object to be serialized is (the attribute value is not shown):
  • the metadata description file of the data object to be serialized is:
  • the class name including UserIdFieldDef, NameFieldDef, PhoneFieldDef, and AgeFieldDef, retrieves and retrieves the same attribute operation class as the attribute operation class name provided by each attribute description.
  • the obtained attribute operation class includes: UserIdFieldDef, NameFieldDef, and AgeFieldDef. .
  • Step S105 Read the attribute values corresponding to the respective attribute descriptions in the data objects to be serialized according to the reading method provided by the attribute operation class corresponding to each attribute.
  • the reading operation method provided by the attribute operation class of each attribute description in the attribute operation class corresponding to each attribute is obtained, and the data object to be serialized is respectively corresponding to each attribute description.
  • Property value the reading operation method provided by the attribute operation class of each attribute description in the attribute operation class corresponding to each attribute.
  • the getter method encapsulated with the same attribute operation class name provided by each attribute description is called, and the attribute value of the corresponding attribute of the data object to be serialized is read.
  • the technical solution of the embodiment provides a
  • the read attribute values are compressed prior to writing the result data stream of the data object.
  • the attribute value can be compressed by using out.writeVarint(), or by using more popular LZ and series variants (LZW, LZR, LZX) compression algorithms.
  • step S103 when the attribute operation class name provided by each attribute description in the metadata description file is obtained, and the attribute operation class corresponding to each attribute is obtained, the following two cases occur:
  • the attribute operation class with the same number of the attribute operation class name provided by each attribute description in the metadata description file can be obtained in the code library, and the data object and metadata to be serialized are described.
  • the version of the description file matches, that is, the data object to be serialized is compatible with the metadata description file version, so compatibility problems do not occur.
  • the number and name of the attribute operation classes in the code base are the same as the attribute operation classes described in the respective attribute descriptions in the metadata description file;
  • the number of attribute operation classes in the code library is greater than the attribute operation class described by each attribute description in the metadata description file, but the code library includes all attribute operation classes described by each attribute description in the metadata description file,
  • the attribute operation class corresponding to the attribute operation class name in the metadata description file is loaded according to the metadata description file, and the attribute operation class not used in the code library is not used, so A compatibility issue will occur.
  • the technical solution of the present embodiment provides a preferred embodiment.
  • the method specifically includes the steps S106-1 to S106-3.
  • Figure 4 is further illustrated.
  • FIG. 4 shows a flowchart of a method for reading default values at the time of serialization according to an embodiment of the present application.
  • Step S106-1 Acquire an attribute type of the attribute description that cannot be read because the attribute operation class of the corresponding attribute cannot be obtained.
  • the obtaining the attribute type of the attribute description that cannot be read because the attribute operation class of the corresponding attribute cannot be obtained may be implemented as follows: from the metadata description file, determining that the attribute value cannot be read because the corresponding attribute operation class cannot be obtained The attribute description, that is, the attribute operation class described by the attribute description does not exist, and the attribute type described by the attribute is obtained.
  • the attribute value cannot be read because the corresponding attribute operation class cannot be obtained, because the code library is not updated or other reasons, and the attribute operation class is not stored. Since the code base is generally stored locally on various terminals, it is likely that it is not updated in time according to the latest version; the metadata description file used is the latest update version.
  • Step S106-2 retrieve a default attribute operation class corresponding to the attribute type.
  • the attribute type includes: an integer type, a character type, a floating point type, a string type, and the like.
  • the obtained property type is VARINT, and the default property operation class of the type DefaultVarintFieldDef is retrieved.
  • Step S106-3 according to the default attribute operation class, assign a predetermined default value to the attribute corresponding to the attribute description.
  • the predetermined default value may adopt different default values according to different attribute types.
  • the default value of the integer type is 0.
  • the data objects to be serialized are:
  • the specific attribute value of the data object is stored in the corresponding database table, for example, a specific data object User is:
  • the User table given above is only schematic.
  • the attribute data in the object User is usually more.
  • only a set of data is given to illustrate the processing of this step. .
  • the metadata description file of the obtained data object to be serialized is:
  • the obtained attribute operation class includes: UserIdFieldDef, NameFieldDef, and AgeFieldDef, wherein the attribute type in the third attribute description is obtained, because the attribute operation class with the same name as PhoneFieldDef is not acquired according to the third attribute description, that is, Get the VARINT type, retrieve the default property operation class DefaultVarintFieldDef of the corresponding type VARINT by obtaining the VARINT type, obtain the property operation class DefaultVarintFieldDef, and execute the getter method in UserIdFieldDef according to the sequence number of the attribute description in the metadata description file.
  • Step S107 the read attribute values are written into the result data stream of the data object in the order of the attribute description sequence number, and the result data stream is used as the serialization result of the data object.
  • the version number of the metadata description file of the data object to be serialized is first obtained, and the value of the version number is written into the position of the first byte of the result data stream, and the sequence is written.
  • the pointers are correspondingly moved backward, and the attribute values read in step S105 are written into the result data stream of the data object in the order of the attribute description sequence numbers.
  • serialized result data stream may be binary data, but is not limited to binary data.
  • a method for serializing a data object provided by the embodiment of the present application, by acquiring a metadata description file, serializing the data object according to the metadata description file, so that the generated result data stream does not include an addition except the data.
  • Data saves storage space and transmission traffic when saving information and data transmission, and can reduce system processing overhead when deserializing.
  • a method for serializing a data object is provided.
  • the present application further provides an apparatus for serializing a data object. Since the embodiment of the device is substantially similar to the embodiment of the method, the description is relatively simple, and the relevant portions can be referred to the description of the method embodiment.
  • the device embodiments described below are merely illustrative.
  • the device for serializing the data object is implemented as follows:
  • FIG. 5 shows a schematic diagram of an apparatus for serializing data objects according to an embodiment of the present application.
  • the device for serializing the data object includes: a description file obtaining unit 501, an attribute operation class obtaining unit 503, an attribute reading unit 505, and a serialization writing unit 507;
  • the description file obtaining unit 501 is configured to acquire a metadata description file of a data object to be serialized, where the metadata description file includes an attribute description identified by a serial number and an attribute operation class name corresponding to an attribute in the data object;
  • the attribute operation class obtaining unit 503 is configured to obtain an attribute operation class name corresponding to each attribute according to each attribute description in the metadata description file, and obtain an attribute operation class corresponding to each attribute;
  • the attribute reading unit 505 is configured to read, according to a reading method provided by the attribute operation class corresponding to each attribute, an attribute value corresponding to each attribute description in the data object to be serialized;
  • the serialization writing unit 507 is configured to write the read attribute values into the result data stream of the data object in the order of the attribute description sequence number, and the result data stream is used as a serialization result of the data object.
  • the metadata description file of the data object to be serialized acquired by the description file acquiring unit 501 further includes an attribute type.
  • the attribute reading unit 505 further includes: an attribute type reading subunit, a default attribute operation class obtaining subunit, and a default value reading subunit;
  • the attribute type reading subunit is configured to read an attribute value corresponding to each attribute description in the data object to be serialized, and obtain an attribute description attribute that cannot be read because the attribute operation class of the corresponding attribute cannot be obtained.
  • the default attribute operation class acquisition sub-unit is configured to retrieve, according to the attribute type, a default attribute operation class corresponding to the attribute type;
  • the default value reading subunit is configured to assign a predetermined default value to the corresponding attribute of the attribute description according to the default attribute operation class.
  • the description file obtaining unit 501 is specifically configured to use, according to the name of the data object to be serialized, Finding, from the metadata repository, a metadata description file of the data object to be serialized corresponding to the data object to be serialized and having the highest version number value.
  • serialization writing unit 507 further includes: a version number writing subunit;
  • the version number is written into the subunit, and is configured to read the data object to be serialized before the attribute value to be read is written into the result data stream of the data object in the order of the attribute description sequence number.
  • the metadata describes the version number of the file and writes it to the head of the resulting data stream of the data object to be serialized.
  • the device for serializing the data object further includes: a modification operation receiving unit, an operation class creation unit, and a description file creation unit;
  • the modifying operation receiving unit is configured to receive, before the acquiring the metadata description file of the data object to be serialized, a modification operation on the attribute of the data object to be serialized; the modifying operation includes: Adding, deleting, or changing the attributes of the serialized data object;
  • the operation class creation unit is configured to create an attribute operation class for the newly added attribute of the data object to be serialized based on the modification operation of the data object to be serialized;
  • a description file creation unit configured to create a new version of the metadata description file according to the modification operation, where the attribute description in the metadata description file is based on the attribute description in the original version metadata description file, according to the modification operation Changed
  • the device for serializing the data object further includes: a traversal unit, a value obtaining unit, a version number generating unit, and a saving unit;
  • the traversing unit is configured to create a new version of the metadata description file according to the modification operation, where the attribute description in the metadata description file is based on the attribute description in the original version metadata description file, according to the modification After the operation is changed, the metadata description file set for the data object to be serialized is traversed in the metadata repository storing the metadata description file according to the name of the data object to be serialized;
  • the value obtaining unit is configured to acquire, during the traversal process, a maximum value of a version number of the metadata description file of the data object to be serialized in the metadata repository;
  • the version number generating unit is configured to add the value by one, and use the added value as the version number of the metadata version file of the new version;
  • the saving unit is configured to save the metadata description file with the version number recorded in the metadata repository.
  • the device for serializing the data object further includes: an attribute value compression unit;
  • the attribute value compression unit is configured to read according to a reading method provided by an attribute operation class corresponding to each attribute After the attribute values corresponding to the respective attribute descriptions in the data objects to be serialized are respectively, the read attribute values are compressed.
  • a method for serializing data objects and a device for serializing data objects are provided.
  • the present application further provides an electronic device; the electronic device implementation is as follows:
  • FIG. 6 shows a schematic diagram of an electronic device provided in accordance with an embodiment of the present application.
  • the electronic device includes: a display 601; a processor 603; a memory 605;
  • the memory 605 is configured to store a serialization program, where the program, when being read and executed by the processor, performs an operation of: acquiring a metadata description file of a data object to be serialized, where the metadata description file includes The attribute operation class identified by the serial number and the attribute operation class name corresponding to the attribute in the data object; the attribute operation class name provided according to each attribute description in the metadata description file, and the attribute operation class corresponding to each attribute are obtained; The reading method provided by the attribute operation class of each attribute reads the attribute value corresponding to each attribute description in the data object to be serialized; and writes the read attribute value into the data object according to the order of the attribute description serial number The resulting data stream, the resulting data stream being the serialized result of the data object.
  • a method for serializing a data object a device for serializing a data object, and an electronic device are provided. Further, the present application further provides a deserialization of the data stream. Method; the method for deserializing the data stream is implemented as follows:
  • FIG. 7 illustrates a flow diagram of a method of deserializing a data stream, in accordance with an embodiment of the present application.
  • the method for deserializing a data stream includes:
  • Step S701 obtaining a result data stream.
  • the obtaining the result data stream includes: acquiring the result data stream by network transmission or obtaining the compressed result data stream from the local.
  • result data stream may be binary data, but is not limited to binary data.
  • Step S703 Obtain a metadata description file of the local data object, where the metadata description file includes an attribute description identified by a sequence number and an attribute operation class name corresponding to an attribute in the data object.
  • the acquiring a metadata description file of the local data object where the metadata description file includes an attribute description identified by a serial number and an attribute operation class name corresponding to an attribute in the data object, specifically including steps S703-1 to S703-4, which will be further described below in conjunction with FIG.
  • FIG. 8 shows a flowchart of acquiring a metadata description file of the local data object according to an embodiment of the present application.
  • Step S703-1 acquiring a name of the local data object.
  • the name of the local data object is User
  • the name of the local data object obtained is User
  • Step S703-2 parsing the result data stream to obtain a first value in the result data stream.
  • the value obtained is 6.
  • Step S703-3 traversing the metadata description file set for the local data object in a metadata repository in which the metadata description file is stored according to the name of the local data object.
  • a metadata description file set for the local data object User is traversed in a metadata repository in which a metadata description file is stored.
  • Step S703-4 in the traversal process, acquiring a metadata description file of the local data object having the same name as the local data object and having the same version number as the first value in the result data stream.
  • the traversal process if there are six metadata description files set for the data object User in the metadata repository, the version number is 1-6, and the version number is obtained in step S603-4. 6 metadata description file.
  • each of the metadata description files The attribute description also includes: the attribute type corresponding to the attribute description.
  • the present application provides a specific example to facilitate the method for deserializing the data stream:
  • the local data object to be written to the attribute value is:
  • the metadata description file of the obtained local data object is:
  • the metadata used during serialization and deserialization is used.
  • the description files are the same, ie the metadata description files are compatible.
  • Step S705 Acquire an attribute operation class name provided by each attribute description in the metadata description file, and obtain an attribute operation class corresponding to each attribute.
  • the attribute operation class name provided by each attribute description in the metadata description file is obtained, and the attribute operation class corresponding to each attribute is obtained, which may be implemented in the following manner: according to the metadata description file Each attribute describes the provided attribute action class name, retrieves and retrieves the same attribute action class as the attribute action class name provided by each attribute description.
  • the local data object to be written to the attribute value is:
  • the metadata description file of the local data object is:
  • the attribute operation class name provided by each attribute description including UserIdFieldDef, NameFieldDef, PhoneFieldDef, and AgeFieldDef, and retrieve and obtain the same attribute operation as the attribute operation class name provided by each attribute description.
  • the obtained property action classes include: UserIdFieldDef, NameFieldDef, and PhoneFieldDef.
  • Step S707 according to the reading method provided by the attribute operation class corresponding to each attribute, sequentially reading the attribute values in the result data stream in the order of the attribute description sequence number.
  • step S705 the reading operation method provided by the attribute operation class of each attribute description in the attribute operation class corresponding to each attribute is obtained, and the attribute value in the result data stream is sequentially described according to the attribute description number. The order is read.
  • the technical solution of the embodiment provides a preferred embodiment.
  • the result data stream is read from the result data stream.
  • the extracted attribute values are decompressed.
  • the in.readVarint() method of the attribute operation class described by each attribute is called, the attribute value of the corresponding attribute of the serialized data object is read, and the read attribute value is decompressed.
  • step S705 when the attribute operation class name provided by each attribute description in the metadata description file is obtained, and the attribute operation class corresponding to each attribute is obtained, the following two cases occur:
  • the attribute operation class with the same number of the attribute operation class name provided by each attribute description in the metadata description file can be obtained in the code library, and the local data object and the metadata description file are described.
  • the number and name of the attribute operation classes in the code base are the same as the attribute operation classes described in the respective attribute descriptions in the metadata description file;
  • the number of attribute operation classes in the code base is greater than the attribute operation class described in each attribute description file in the metadata description file, but the attribute operation class in the code library includes attributes described by each attribute description in the metadata description file.
  • the operation class but since the attribute operation class corresponding to the attribute operation class name in the metadata description file is loaded according to the metadata description file at the time of serialization, a compatibility problem does not occur.
  • the technical solution of the present embodiment provides a preferred implementation manner.
  • the method specifically includes steps S707-1 to S707-3. This will be further described below in conjunction with FIG.
  • FIG. 9 shows a flowchart of a method for discarding default values during deserialization according to an embodiment of the present application.
  • step S707-1 for the attribute description of the attribute operation class, the default attribute operation class corresponding to the attribute type is obtained according to the attribute type described by the attribute.
  • obtaining the default attribute operation class corresponding to the attribute type according to the attribute type described by the attribute may be implemented as follows: from the metadata description file, determining that the attribute cannot be obtained The attribute description of the operation class, that is, the attribute operation class described by the attribute description does not exist, the attribute corresponding to the attribute operation class does not exist in the local data object, and the attribute type described by the attribute is obtained, and according to the attribute The attribute type described, and obtain the default attribute operation class corresponding to the attribute type.
  • the attribute type includes: an integer type, a character type, a floating point type, a string type, and the like.
  • the obtained attribute type is VARINT, and the default attribute operation class of type DefaultVarintFieldDef is obtained.
  • step S707-2 the reading method provided by the default attribute operation class is obtained, and the data in the result data stream is read in the order of the attribute description sequence number.
  • Step S707-3 discarding the data in the read result data stream.
  • the discarding the data in the read result data stream refers to: after the attribute value in the result data stream is read out, no writing is performed, that is, the data is not to be The attribute value is written to the local data object.
  • the deserialization pointer is correspondingly moved to the position of the next attribute value.
  • the local data object to be written to the attribute value is:
  • the metadata description file of the local data object is:
  • the obtained attribute operation class includes: UserIdFieldDef, NameFieldDef, and PhoneFieldDef, wherein the attribute type in the fourth attribute description is obtained, because the attribute operation class having the same name as the AgeFieldDef is not acquired according to the fourth attribute description, that is, : Get the VARINT type, retrieve the default property operation class DefaultVarintFieldDef of the corresponding type VARINT by obtaining the VARINT type, obtain the property operation class DefaultVarintFieldDef, and execute the in. in the UserIdFieldDef according to the sequence number of the attribute description in the metadata description file.
  • the readVarint() method reads the data 1 at the corresponding position of the result data stream, and decompresses the data, moves the deserialized pointer accordingly to the position of the next attribute value, and executes in.readVarint() in the NameFieldDef.
  • the method reads the data aaa of the corresponding position of the result data stream, decompresses the data, moves the deserialized pointer to the position of the next attribute value, and executes the in.readVarint() method in the PhoneFieldDef.
  • the result data stream corresponds to the data of the location 0, and the number Decompresses the in.readVarint deserialization pointer position corresponding to the next attribute value, and performing the DefaultVarintFieldDef () method reads, discarding 20 the attribute value attribute value after decompression.
  • Step S709 Write the read attribute values into the attributes of the local data objects corresponding to the respective attribute descriptions according to the writing method provided by the attribute operation class described by each attribute.
  • the attribute value in the result data stream read by the in.readVarint() method in step S707 is decompressed, and then the attribute value is written according to the attribute description number by a setter method. Corresponding to the properties of the local data object.
  • the user.setUserId(in.readVarint()) in the UserIdFieldDef is executed to write the attribute value read from the result data stream into the attribute of the local data object corresponding to the userId.
  • a method for deserializing a data stream is provided.
  • the present application further provides an apparatus for deserializing a data stream. Since the embodiment of the device is substantially similar to the embodiment of the method, the description is relatively simple, and the relevant portions can be referred to the description of the method embodiment.
  • the device embodiments described below are merely illustrative.
  • the apparatus for deserializing the data stream is implemented as follows:
  • FIG. 10 shows a schematic diagram of an apparatus for deserializing a data stream according to an embodiment of the present application.
  • the device for deserializing the data stream includes: a data stream obtaining unit 1001, a description file acquiring unit 1003, an attribute operation class obtaining unit 1005, an attribute value reading unit 1007, and an attribute value writing unit 1009;
  • the data stream obtaining unit 1001 is configured to acquire a result data stream
  • the description file obtaining unit 1003 is configured to acquire a metadata description file of the local data object, where the metadata description file includes an attribute description identified by a serial number and an attribute operation class name corresponding to an attribute in the data object;
  • the attribute operation class obtaining unit 1005 is configured to obtain, according to the attribute operation class name provided by each attribute description in the metadata description file, an attribute operation class corresponding to each attribute;
  • the attribute value reading unit 1007 is configured to sequentially read the attribute values in the result data stream according to the sequence of the attribute description sequence according to the reading method provided by the attribute operation class corresponding to each attribute;
  • the attribute value writing unit 1009 is configured to write the read attribute value into the attribute of the local data object corresponding to each attribute description according to the writing method provided by the attribute operation class described by each attribute. in.
  • the description file obtaining unit 1003 the obtained metadata description file of the local data object further includes an attribute type.
  • the attribute value reading unit 1007 includes: an attribute type acquiring subunit, and a data reading subunit. And discarding subunits;
  • the attribute type obtaining subunit is configured to sequentially read the attribute values in the result data stream according to the sequence of the attribute description serial number in the writing method provided by the attribute operation class according to each attribute description.
  • the default attribute operation class corresponding to the attribute type is obtained according to the attribute type described by the attribute;
  • the data reading subunit is configured to acquire a reading method provided by the default attribute operation class, and read data in the result data stream according to an order of the attribute description sequence number;
  • the discarding a single sub-unit is configured to discard the data in the read result data stream.
  • the description file obtaining unit 1003 includes: a local data object name obtaining subunit, a result data stream parsing subunit, a description file traversing subunit, and a description file acquiring subunit;
  • the local data object name obtaining subunit is configured to acquire a name of the local data object
  • the result data stream parsing subunit is configured to parse the result data stream to obtain a first value in the result data stream;
  • the description file obtaining subunit configured to acquire metadata of the local data object having the same name as the local data object and having the same version number as the first value in the result data stream during the traversal process Description file.
  • the device for deserializing the data stream further includes: an attribute value decompression unit;
  • the attribute value decompressing unit is configured to sequentially read the attribute values in the result data stream according to the order of the attribute description sequence in the reading method provided by the attribute operation class corresponding to each attribute, and then The attribute values read from the result data stream are decompressed.
  • a method for serializing a data object a device for serializing a data object, an electronic device, a method for deserializing a data stream, and a pair of data streams are provided
  • the apparatus for deserialization in addition, the present application also provides yet another electronic device; the electronic device implementation is as follows:
  • FIG. 11 shows a schematic diagram of an electronic device provided in accordance with an embodiment of the present application.
  • the electronic device includes: a display 1101; a processor 1103; a memory 1105;
  • the memory 1105 is configured to store a deserialization program, where the program, when being read and executed by the processor, performs an operation of: acquiring a result data stream, acquiring a metadata description file of the local data object,
  • the metadata description file includes an attribute description identified by a sequence number and an attribute operation class name corresponding to the attribute in the data object, the root According to the attribute operation class name provided by each attribute description in the metadata description file, the attribute operation class corresponding to each attribute is obtained, and the read data provided by the attribute operation class corresponding to each attribute is sequentially used in the result data stream.
  • the attribute values are read in the order of the attribute description sequence numbers, and the read attribute values are written into the local data objects corresponding to the respective attribute descriptions according to the writing method provided by the attribute operation class described by each attribute. In the attribute.
  • a method for serializing a data object a device for serializing a data object, an electronic device, a method for deserializing a data stream, and a pair of data streams are provided.
  • Deserializing device and yet another electronic device in addition, the present application also provides a serialization and deserialization system; the serialization and deserialization system implementation is as follows:
  • FIG. 12 a schematic diagram of a serialization and deserialization system provided in accordance with an embodiment of the present application is shown.
  • the serialization and deserialization system includes: a device 1201 for serializing a data object and a device 1203 for deserializing the data stream;
  • the device 1201 for serializing the data object is configured to obtain a metadata description file, obtain an attribute operation class corresponding to each attribute according to the metadata description file, and read each attribute and each attribute of the data object to be serialized. Describe the corresponding attribute value, and write the read attribute value to the result data stream of the data object in the order of the attribute description sequence number;
  • the device 1203 for deserializing the data stream is configured to acquire a result data stream, obtain a metadata description file, obtain an attribute operation class corresponding to each attribute according to the metadata description file, and sequentially perform the attribute data stream in the result data stream.
  • the attribute values are read in the order of the attribute description sequence numbers, and the read attribute values are written in the attributes of the local data objects corresponding to the respective attribute descriptions.
  • a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
  • processors CPUs
  • input/output interfaces network interfaces
  • memory volatile and non-volatile memory
  • the memory may include non-persistent memory, random access memory (RAM), and/or non-volatile memory in a computer readable medium, such as read only memory (ROM) or flash memory.
  • RAM random access memory
  • ROM read only memory
  • Memory is an example of a computer readable medium.
  • Computer readable media including both permanent and non-persistent, removable and non-removable media may be implemented by any method or technology.
  • the information can be computer readable instructions, data structures, modules of programs, or other data.
  • Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory. (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disk read only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, Magnetic box magnetic A tape, magnetic tape storage or other magnetic storage device or any other non-transportable medium can be used to store information that can be accessed by a computing device.
  • computer readable media does not include non-transitory computer readable media, such as modulated data signals and carrier waves.
  • embodiments of the present application can be provided as a method, system, or computer program product.
  • the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment in combination of software and hardware.
  • the application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

一种对数据对象序列化的方法、装置及电子设备、一种对数据流反序列化的方法、装置及电子设备以及一种序列化与反序列化系统。其中所述对数据对象序列化的方法,包括:获取待序列化的数据对象的元数据描述文件(S101);根据元数据描述文件中的属性描述提供的属性操作类名称,获取对应的属性操作类(S103);根据对应的属性操作类提供的读取方法,读取待序列化的数据对象中的属性值(S105);将读取到的属性值按照属性描述序号的顺序写入数据对象的结果数据流(S107)。采用上述方法解决了改变对象的属性后系统之间的兼容性问题,还将对象的元数据和对象的值进行拆分,在保存和传输时节约了存储空间和传输流量,并且在反序列化时降低系统的处理开销。

Description

序列化与反序列化的方法、装置、系统以及电子设备
本申请要求2015年10月10日递交的申请号为201510650688.2发明名称为“序列化与反序列化的方法、装置、系统以及电子设备”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机通信技术领域,具体涉及一种对数据对象序列化的方法、一种对数据对象序列化的装置以及一种电子设备;一种对数据流反序列化的方法、一种对数据流反序列化的装置以及一种电子设备;一种序列化与反序列化系统。
背景技术
对象序列化(Serialization)技术,是将对象数据转换成另一种持久化数据形式的技术,即将对象的状态信息转换为可以存储或传输的形式的过程。在序列化期间,对象将其当前状态写入到临时或持久性存储区。为了使得存储和传输后的数据能够再次变回内存中的对象,这样的转换必须是可逆的。可以通过从存储区中读取或反序列化对象的状态,重新创建该对象。
在网络间数据传输和应用中,通常需要将信息转换成数据流,实现数据传输和应用。但是,在现有的序列化与反序列化方案下,如果对对象中的属性进行增减,执行反序列化的一侧若没有将对象中的属性进行同步,就会引起反序列化失败的问题。增加、减少或修改了对象中的属性,将原来序列化后的文件或数据进行反序列化时,会产生错误或序列化失败。这就是对象序列化的兼容问题。
目前,在现有技术下通常采用google的Protobuf(Protocol Buffers)数据交换格式实现序列化和反序列化。Protobuf是一种数据交换格式,将现有的信息根据不同的类型,按照相应的编码方式序列化为二进制流,用于保存信息或者数据传输。在读取该二进制信息或者接收该二进制信息时,对经序列化的二进制流进行反序列化,获取原始信息或数据。Protobuf在序列化时虽然能减少序列化结果中的元数据内容,但是为了兼容性,还是保留了少量的元数据(属性类型,属性序号),并且在反序列化时,除了获取二进制流外,还需要获知序列化时采用的类型,进而根据类型进行相应的反序列化。
由此可见,在现有技术下,由于序列化时还保留了元数据,使序列化的压缩率低,造 成在保存信息或者数据传输时浪费空间和流量;并且在反序列化时,获取对象序列化时采用的序列化类型的过程复杂,增加处理开销。
发明内容
本申请提供一种对数据对象序列化的方法、一种对数据对象序列化的装置、一种电子设备、一种对数据流反序列化的方法、一种对数据流反序列化的装置、一种电子设备以及一种序列化与反序列化系统,以解决现有技术中序列化压缩率低的问题。
本申请提供了一种对数据对象序列化的方法,所述对数据对象序列化的方法包括:
获取待序列化的数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称;
根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类;
根据对应各个属性的属性操作类提供的读取方法,读取待序列化的数据对象中分别与各个属性描述对应的属性值;
将读取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流,所述结果数据流作为所述数据对象的序列化结果。
可选的,所述属性描述,还包括:属性类型。
可选的,在所述读取待序列化的数据对象中分别与各个属性描述对应的属性值的步骤之后,包括:
获取由于无法获取对应属性的属性操作类而无法读取的属性描述的属性类型;
根据所述属性类型,检索获得对应所述属性类型的缺省属性操作类;
根据所述缺省属性操作类,为该属性描述对应的属性赋予预先确定的缺省值。
可选的,所述获取待序列化的数据对象的元数据描述文件是:
根据所述待序列化的数据对象的名称,从元数据仓库中查找对应所述待序列化的数据对象且版本号数值最大的所述待序列化的数据对象的元数据描述文件。
可选的,在所述将读取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流的步骤之前,包括:
读取所述待序列化的数据对象的元数据描述文件的版本号,并将其写入到所述待序列化的数据对象的结果数据流的头部。
可选的,在所述获取待序列化的数据对象的元数据描述文件的步骤之前,还包括:
接收对所述待序列化的数据对象的属性的修改操作;所述修改操作包括:对所述待序列化的数据对象的属性进行增加、删除或者变更;
基于对所述待序列化的数据对象的修改操作为所述待序列化的数据对象的新增属性创建属性操作类;
根据所述修改操作创建新版本的元数据描述文件,该元数据描述文件中的属性描述在原版本元数据描述文件中的属性描述的基础上根据所述修改操作进行了变更。
可选的,在所述根据所述修改操作创建新版本的元数据描述文件,该元数据描述文件中的属性描述在原版本元数据描述文件中的属性描述的基础上根据所述修改操作进行了变更的步骤之后,包括:
根据所述待序列化的数据对象的名称,在存有元数据描述文件的元数据仓库中遍历针对所述待序列化的数据对象设置的元数据描述文件;
在遍历过程中,获取在所述元数据仓库中的所述待序列化的数据对象的元数据描述文件的版本号最大的数值;
将所述数值加一,并将加一后的所述数值作为新版本的元数据描述文件的版本号;
将所述记录有版本号的所述元数据描述文件保存在所述元数据仓库中。
可选的,在所述根据对应各个属性的属性操作类提供的读取方法,读取待序列化的数据对象中分别与各个属性描述对应的属性值的步骤之后,包括:
将读取到的属性值进行压缩。
相应的,本申请还提供了一种对数据对象序列化的装置,所述对数据对象序列化的装置包括:描述文件获取单元、属性操作类获取单元、属性读取单元以及序列化写入单元;
所述描述文件获取单元,用于获取待序列化的数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称;
所述属性操作类获取单元,用于根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类;
所述属性读取单元,用于根据对应各个属性的属性操作类提供的读取方法,读取待序列化的数据对象中分别与各个属性描述对应的属性值;
所述序列化写入单元,用于将读取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流,所述结果数据流作为所述数据对象的序列化结果。
可选的,所述描述文件获取单元获取的所述待序列化的数据对象的元数据描述文件还包括属性类型。
可选的,所述属性读取单元,还包括:属性类型读取子单元、缺省属性操作类获取子单元以及缺省值读取子单元;
所述属性类型读取子单元,用于读取待序列化的数据对象中分别与各个属性描述对应的属性值之后,获取由于无法获取对应属性的属性操作类而无法读取的属性描述的属性类型;
所述缺省属性操作类获取子单元,用于根据所述属性类型,检索获得对应所述属性类型的缺省属性操作类;
所述缺省值读取子单元,用于根据所述缺省属性操作类,为该属性描述对应的属性赋予预先确定的缺省值。
可选的,所述描述文件获取单元具体用于根据所述待序列化的数据对象的名称,从元数据仓库中查找对应所述待序列化的数据对象且版本号数值最大的所述待序列化的数据对象的元数据描述文件。
可选的,所述序列化写入单元,还包括:版本号写入子单元;
所述版本号写入子单元,用于在所述将读取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流之前,读取所述待序列化的数据对象的元数据描述文件的版本号,并将其写入到所述待序列化的数据对象的结果数据流的头部。
可选的,所述对数据对象序列化的装置,还包括:修改操作接收单元、操作类创建单元以及描述文件创建单元;
所述修改操作接收单元,用于在所述获取待序列化的数据对象的元数据描述文件之前,接收对所述待序列化的数据对象的属性的修改操作;所述修改操作包括:对所述待序列化的数据对象的属性进行增加、删除或者变更;
所述操作类创建单元,用于基于对所述待序列化的数据对象的修改操作为所述待序列化的数据对象的新增属性创建属性操作类;
所述描述文件创建单元,用于根据所述修改操作创建新版本的元数据描述文件,该元数据描述文件中的属性描述在原版本元数据描述文件中的属性描述的基础上根据所述修改操作进行了变更。
可选的,所述对数据对象序列化的装置,还包括:遍历单元、数值获取单元、版本号生成单元以及保存单元;
所述遍历单元,用于在所述根据所述修改操作创建新版本的元数据描述文件,该元数据描述文件中的属性描述在原版本元数据描述文件中的属性描述的基础上根据所述修改操作进行了变更之后,根据所述待序列化的数据对象的名称,在存有元数据描述文件的元数据仓库中遍历针对所述待序列化的数据对象设置的元数据描述文件;
所述数值获取单元,用于在遍历过程中,获取在所述元数据仓库中的所述待序列化的数据对象的元数据描述文件的版本号最大的数值;
所述版本号生成单元,用于将所述数值加一,并将加一后的所述数值作为新版本的元数据描述文件的版本号;
所述保存单元,用于将所述记录有版本号的所述元数据描述文件保存在所述元数据仓库中。
可选的,所述对数据对象序列化的装置,还包括:属性值压缩单元;
所述属性值压缩单元,用于根据对应各个属性的属性操作类提供的读取方法,读取待序列化的数据对象中分别与各个属性描述对应的属性值之后,将读取到的属性值进行压缩。
此外,本申请实施例还提供了一种电子设备,所述电子设备包括:
显示器;
处理器;
存储器,用于存储序列化程序,所述程序在被所述处理器读取执行时,执行如下操作:获取待序列化的数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称;根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类;根据对应各个属性的属性操作类提供的读取方法,读取待序列化的数据对象中分别与各个属性描述对应的属性值;将读取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流,所述结果数据流作为所述数据对象的序列化结果。
此外,本申请还提供了一种对数据流反序列化的方法,所述对数据流反序列化的方法包括:
获取结果数据流;
获取所述本地数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称;
根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个 属性的属性操作类;
根据对应各个属性的属性操作类提供的读取方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取;
根据各个属性描述的所述属性操作类提供的写入方法,将读取到的属性值写入与各个属性描述相对应的所述本地数据对象的属性中。
可选的,所述属性描述,还包括:属性类型。
可选的,在所述根据各个属性描述的所述属性操作类提供的写入方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取的步骤之中,还包括:
对于无法获得属性操作类的属性描述,根据该属性描述的属性类型,获取对应所述属性类型的缺省属性操作类;
获取所述缺省属性操作类提供的读取方法,按照属性描述序号的顺序读取所述结果数据流中的数据;
将读取的所述结果数据流中的数据丢弃。
可选的,所述获取所述本地数据对象的元数据描述文件包括:
获取所述本地数据对象的名称;
解析所述结果数据流,获取所述结果数据流中的第一个值;
根据所述本地数据对象的名称,在存有元数据描述文件的元数据仓库中遍历针对所述本地数据对象设置的元数据描述文件;
在遍历过程中,获取与所述本地数据对象的名称相同且版本号与所述结果数据流中的第一个值相同的所述本地数据对象的元数据描述文件。
可选的,在所述根据对应各个属性的属性操作类提供的读取方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取的步骤之后,包括:
将从所述结果数据流中读取出的属性值进行解压缩。
相应的,本申请还提供了一种对数据流反序列化的装置,所述对数据流反序列化的装置包括:数据流获取单元、描述文件获取单元、属性操作类获取单元、属性值读取单元以及属性值写入单元;
所述数据流获取单元,用于获取结果数据流;
所述描述文件获取单元,用于获取所述本地数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称;
所述属性操作类获取单元,用于根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类;
所述属性值读取单元,用于根据对应各个属性的属性操作类提供的读取方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取;
所述属性值写入单元,用于根据各个属性描述的所述属性操作类提供的写入方法,将读取到的属性值写入与各个属性描述相对应的所述本地数据对象的属性中。
可选的,所述描述文件获取单元,获取的所述本地数据对象的元数据描述文件还包括属性类型。
可选的,所述属性值读取单元包括:属性类型获取子单元、数据读取子单元以及丢弃子单元;
所述属性类型获取子单元,用于在所述根据各个属性描述的所述属性操作类提供的写入方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取之中,对于无法获得属性操作类的属性描述,根据该属性描述的属性类型,获取对应所述属性类型的缺省属性操作类;
所述数据读取子单元,用于获取所述缺省属性操作类提供的读取方法,按照属性描述序号的顺序读取所述结果数据流中的数据;
所述丢弃单子元,用于将读取的所述结果数据流中的数据丢弃。
可选的,所述描述文件获取单元,包括:本地数据对象名称获取子单元、结果数据流解析子单元、描述文件遍历子单元以及描述文件获取子单元;
所述本地数据对象名称获取子单元,用于获取所述本地数据对象的名称;
所述结果数据流解析子单元,用于解析所述结果数据流,获取所述结果数据流中的第一个值;
所述描述文件遍历子单元,用于根据所述本地数据对象的名称,在存有元数据描述文件的元数据仓库中遍历针对所述本地数据对象设置的元数据描述文件;
所述描述文件获取子单元,用于在遍历过程中,获取与所述本地数据对象的名称相同且版本号与所述结果数据流中的第一个值相同的所述本地数据对象的元数据描述文件。
可选的,所述对数据流反序列化的装置,还包括:属性值解压缩单元;
所述属性值解压缩单元,用于在所述根据对应各个属性的属性操作类提供的读取方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取的之后,将从所述 结果数据流中读取出的属性值进行解压缩。
此外,本申请实施例还提供了一种电子设备,所述电子设备包括:
显示器;
处理器;
存储器,用于存储反序列化程序,所述程序在被所述处理器读取执行时,执行如下操作:获取结果数据流,获取所述本地数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称,根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类,根据对应各个属性的属性操作类提供的读取方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取,根据各个属性描述的所述属性操作类提供的写入方法,将读取到的属性值写入与各个属性描述相对应的所述本地数据对象的属性中。
此外,本申请还提供了一种序列化与反序列化系统,所述序列化与反序列化系统包括:
根据上述任一项所述的对数据对象序列化的装置;以及任一项所述的对数据流反序列化的装置。
与现有技术相比,本申请提供的对数据对象序列化的方法以及装置,通过获取待序列化的数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述,各个属性描述中包括所对应的所述数据对象属性的属性操作类名称;根据所述属性操作类名称,能够获取对应属性的属性操作类,根据对应各个属性的属性操作类提供的读取方法,可以读取待序列化的数据对象中分别与各个属性描述对应的属性值,将读取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流,所述结果数据流作为所述数据对象的序列化结果。
本申请提供的对数据流反序列化的方法以及装置,通过获取结果数据流,获取所述本地数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称,根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类,根据对应各个属性的属性操作类提供的读取方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取,根据各个属性描述的所述属性操作类提供的写入方法,将读取到的属性值写入与各个属性描述相对应的所述本地数据对象的属性中。
上述技术方案解决了改变对象的属性后系统之间序列化和反序列化的兼容性问题, 还通过将对象的元数据和对象的值进行彻底的拆分,在保存信息和数据传输时节约了存储空间和传输流量,并且在反序列化时降低系统的处理开销。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请中记载的一些实施例,对于本领域普通技术人员来讲,还可以根据这些附图获得其他的附图。
图1示出了根据本申请的实施例提供的对数据对象序列化的方法的流程图;
图2示出了根据本申请的实施例提供的创建元数据描述文件的流程图;
图3示出了根据本申请的实施例提供的保存元数据描述文件的流程图;
图4示出了根据本申请的实施例提供的序列化时缺省值的读取方法的流程图。
图5示出了根据本申请的实施例提供的对数据对象序列化的装置的示意图;
图6示出了根据本申请的实施例提供的电子设备的示意图;
图7示出了根据本申请的实施例提供的对数据流反序列化的方法的流程图;
图8示出了根据本申请的实施例提供的获取所述本地数据对象的元数据描述文件的流程图;
图9示出了根据本申请的实施例提供的反序列化时缺省值的丢弃方法的流程图;
图10示出了根据本申请的实施例提供的对数据流反序列化的装置的示意图;
图11示出了根据本申请的实施例提供的电子设备的示意图;
图12示出了根据本申请的实施例提供的序列化与反序列化系统的示意图。
具体实施方式
为了能够更清楚地理解本申请的上述目的、特征和优点,下面结合附图和具体实施方式对本申请进行进一步的详细描述。需要说明的是,在不冲突的情况下,本申请的实施例及实施例中的特征可以相互组合。
在下面的描述中阐述了很多具体细节以便于充分理解本申请。但是,本申请能够以很多不同于在此描述的其它方式来实施,本领域技术人员可以在不违背本申请内涵的情况下做类似推广,因此,本申请不受下面公开的具体实施的限制。
本申请的实施例提供了一种对数据对象序列化的方法、一种对数据对象序列化的装置、一种对数据流反序列化的方法、一种对数据流反序列化的装置以及一种序列化与反序列化系统,本申请的实施例同时提供了一种电子设备。在下面的实施例中逐一进行详细说明。
本申请的技术方案是针对分布式系统环境下,系统之间进行数据传递或存储而提出的,但是,其应用领域并不局限于分布式系统中,在现有的系统或应用程序中,只要是可以执行序列化与反序列化方法,都可以采用本申请提供的方法,并获得相应的有益效果。
在详细描述本实施例的具体步骤之前,先对本技术方案涉及的序列化方法和反序列化方法作简要说明。
对象序列化(Serialization)技术,是将对象数据转换成另一种持久化数据形式的技术,即将对象的状态信息转换为可以存储或传输的形式的过程。在序列化期间,对象将其当前状态写入到临时或持久性存储区,可以通过从存储区中读取或反序列化对象的状态,重新创建该对象。为了使得存储和传输后的数据能够再次变回内存中的对象,这样的转换必须是可逆的。序列化方法和反序列化方法被广泛使用在网络间数据传输和应用中,通常需要将信息转换成数据流,实现数据传输和应用。但是在数据传输和存储时,如果对对象中的属性进行增减,执行反序列化的一侧若没有将对象中的属性进行同步,就会引起反序列化失败的问题。增加、减少或修改了对象中的属性,将原来序列化后的文件或数据进行反序列化时,会产生错误或序列化失败。这就是对象序列化的兼容问题。通常对象的所有字段都会被序列化,这意味着数据会被表示为序列化数据。类似地,反序列化从序列化的表示形式中提取数据,并直接设置对象状态。
本申请的实施例提供了一种对数据对象序列化的方法。所述对数据对象序列化的方法实施例如下:
请参考图1,其示出了根据本申请的实施例提供的对数据对象序列化的方法的流程图。
所述对数据对象序列化的方法包括:
步骤S101,获取待序列化的数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称。
在本实施例中,所述获取待序列化的数据对象的元数据描述文件,可以采用如下方式实现:根据所述待序列化的数据对象的名称,从元数据仓库中查找对应所述待序列化 的数据对象且版本号数值最大的所述待序列化的数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称。
需要说明的是,所述元数据仓库是用于存储元数据以及与元数据相关信息(如元数据描述文件)的数据库,所述元数据仓库一般可以采用关系型数据库。
在本实施例中,所述对数据对象序列化的方法是根据元数据描述文件的描述语义去理解数据的结构构成。可以理解的,由于在步骤S101中获取待序列化的数据对象的元数据描述文件时,是从元数据仓库中查找对应所述待序列化的数据对象且版本号数值最大的所述待序列化的数据对象的元数据描述文件,即本步骤中获取到的都是最新版本的元数据描述文件。
具体的,所述元数据描述文件中包括若干条对应于数据对象中属性的属性描述,每条属性描述以序号标识,并包含对应所述数据对象中属性的属性操作类名称。
为了满足序列化时的元数据描述文件和数据对象中属性的兼容问题,本实施例的技术方案提供了一种优选实施方式,在优选方式下,所述元数据描述文件中的每条属性描述,还包括:对应该条属性描述的属性类型。
例如,元数据描述文件可以如下:
1:com.taobao.galaxy.io.UserIdFieldDef:VARINT
2:com.taobao.galaxy.io.AgeFieldDef:VARINT
其中,数字“1”和“2”是每条属性描述的序号,UserIdFieldDef是对应所述数据对象中属性UserId的属性操作类名称;AgeFieldDef是对应所述数据对象中属性Age的属性操作类名称;VARINT是本条属性描述的属性类型。
需要说明的是,元数据描述文件是描述其它数据的数据,或者说是结构数据。具体说,元数据描述文件是描述信息资源或数据等对象的数据。元数据描述文件一经建立便可共享使用。元数据描述文件的开发与利用环境往往是一个变化的分布式环境。由于元数据描述文件也是数据,因此可以用类似数据的方法在元数据仓库中进行存储和获取,方便在分布式环境下共享与使用。
下面针对所述对数据对象序列化的方法,本申请提供了一个具体例子以便对所述对数据对象序列化的方法进行说明:
待序列化的数据对象为(未示出其属性值):
Class User{
int userId;
string name;
int age;
}
根据所述待序列化的数据对象的名称User,从元数据仓库中查找对应所述待序列化的数据对象User且版本号数值最大的所述待序列化的数据对象的元数据描述文件,例如:获取到的待序列化的数据对象的元数据描述文件为:
1:com.taobao.test.io.UserIdFieldDef:VARINT
2:com.taobao.test.io.NameFieldDef:VARCHAR
3:com.taobao.test.io.PhoneFieldDef:VARINT
4:com.taobao.test.io.AgeFieldDef:VARINT
上述元数据描述文件所描述的数据对象与前述数据对象的例子User并不完全相符合,这是由于元数据描述文件是最新版本,是根据升级版本的User确定的。
需要说明的是,数据对象为客观世界中存在的人、事、物体等实体在计算机逻辑中的映射。例如,当对象为书籍,其具有的属性包括书籍编号、作者、出版社和备注等,将该书籍进行序列化后,生成与该书籍对应的二进制数据流,该二进制数据流具有与书籍的属性相对应的属性值,从而通过计算机语言表示客观世界中存在的人、事、物体等实体。数据对象可以是任意的客观世界中存在的人、事、物体等实体,本例子中的数据对象是用户,即User。对象包括至少一个属性,属性的名称称为属性名称,属性对应的值称为属性值,属性值可以为整型,字符型,浮点型,字符串型等基本类型,例如上述数据对象的例子中,属性userId为整型,属性name为字符串型,属性age为整型。
在本步骤S101获取待序列化的数据对象的元数据描述文件之前,还需定义或创建所述元数据描述文件,具体包括步骤S100-1至S100-3,下面结合图2作进一步说明。
请参考图2,其示出了根据本申请的实施例提供的创建元数据描述文件的流程图。
步骤S100-1,接收对所述待序列化的数据对象的属性的修改操作;所述修改操作包括:对所述待序列化的数据对象的属性进行增加、删除或者变更。
所述接收对所述待序列化的数据对象的属性的修改操作,包括:接收对所述待序列化的数据对象中的属性的增加、删除或者变更。例如,对数据对象User增加类型为string的属性address、将数据对象User中的age属性删除或是将数据对象中的userId属性的名称修改为id。
步骤S100-2,基于对所述待序列化的数据对象的修改操作为所述待序列化的数据对象的新增属性创建属性操作类。
本步骤根据步骤S100-1中接收的对所述待序列化的数据对象的属性的修改操作为新增属性创建属性操作类。例如,在步骤S100-1中增加了类型为string的属性address,则为属性address创建名称为AddressFieldDef的属性操作类。其中,所述AddressFieldDef类中封装了getter与setter方法。
需要说明的是,若所述修改操作为删除属性,可以不删除对应属性的属性操作类;若所述修改操作为修改属性,即删除原有属性后创建修改后的属性,并为该修改后的属性创建相应的属性操作类。
步骤S100-3,根据所述修改操作创建新版本的元数据描述文件,该元数据描述文件中的属性描述在原版本元数据描述文件中的属性描述的基础上根据所述修改操作进行了变更。
在本实施例中,所述根据所述修改操作创建新版本的元数据描述文件,该元数据描述文件中的属性描述在原版本元数据描述文件中的属性描述的基础上根据所述修改操作进行了变更,可以采用如下方式实现:获取接收修改操作后的所述待序列化的数据对象的属性,按照属性在数据对象中定义的顺序生成属性描述的序号,获取对应所述待序列化的数据对象中属性的属性操作类名称,并获取所述待序列化的数据对象中属性的属性类型,根据属性在数据对象中定义的顺序将属性操作类名称以及属性类型按顺序写入每条属性描述中,将生成的全部属性描述作为新版本的元数据描述文件。
在所述根据所述修改操作创建新版本的元数据描述文件,该元数据描述文件中的属性描述在原版本元数据描述文件中的属性描述的基础上根据所述修改操作进行了变更之后,为了使新版本的元数据描述文件与旧版本的元数据描述文件区分开,并且在分布式环境下能被共享使用,本实施例的技术方案提供了一种优选实施方式,在优选方式下,具体包括步骤S100-4至S100-7,下面结合图3作进一步说明。
请参考图3,其示出了根据本申请的实施例提供的保存元数据描述文件的流程图。
步骤S100-4,根据所述待序列化的数据对象的名称,在存有元数据描述文件的元数据仓库中遍历针对所述待序列化的数据对象设置的元数据描述文件。
例如,所述待序列化的数据对象的名称为User,在存有元数据描述文件的元数据仓库中遍历针对数据对象User设置的元数据描述文件。
步骤S100-5,在遍历过程中,获取在所述元数据仓库中的所述待序列化的数据对象 的元数据描述文件的版本号最大的数值。
例如,在遍历过程中,查找出在所述元数据仓库中的针对数据对象User设置的元数据描述文件共有6个,版本号为1-6,则在本步骤S100-5中获取的数值为6。
步骤S100-6,将所述数值加一,并将加一后的所述数值作为新版本的元数据描述文件的版本号。
在本实施例中,将加一后的数值与步骤S100-3中创建的新版本的元数据描述文件建立映射关系。
步骤S100-7,将所述记录有版本号的所述元数据描述文件保存在所述元数据仓库中。
需要说明的是,步骤S100-1至步骤S100-7这七个步骤,可以预先创建完成,也可以在执行对数据对象序列化的方法时候进行现场配置,对此不做限定。
步骤S103,根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类。
在本实施例中,所述根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类,可以采用如下方式实现:根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,检索并获取与各个属性描述提供的属性操作类名称相同的属性操作类。
下面仍沿用上述对数据对象序列化的方法的具体例子,对本步骤的处理过程作进一步说明。
待序列化的数据对象为(未示出其属性值):
Class User{
int userId;
string name;
int age;
}
待序列化的数据对象的元数据描述文件为:
1:com.taobao.test.io.UserIdFieldDef:VARINT
2:com.taobao.test.io.NameFieldDef:VARINT
3:com.taobao.test.io.PhoneFieldDef:VARINT
4:com.taobao.test.io.AgeFieldDef:VARINT
从所述待序列化的数据对象的元数据描述文件中,获取各个属性描述提供的属性操 作类名称,包括UserIdFieldDef、NameFieldDef、PhoneFieldDef以及AgeFieldDef,检索并获取与各个属性描述提供的属性操作类名称相同的属性操作类,在本例子中,获取到的属性操作类包括:UserIdFieldDef、NameFieldDef以及AgeFieldDef。
步骤S105,根据对应各个属性的属性操作类提供的读取方法,读取待序列化的数据对象中分别与各个属性描述对应的属性值。
在本实施例中,根据步骤S103中获取对应各个属性的属性操作类中,各个属性描述的所述属性操作类提供的读取方法,读取待序列化的数据对象中分别与各个属性描述对应的属性值。
在具体实施时,调用与各个属性描述提供的属性操作类名称相同的属性操作类封装的getter方法,读取待序列化的数据对象相应属性的属性值。
为了使读取到的待序列化的数据对象中的属性值,在写入所述数据对象的结果数据流时节省存储空间或传输时减小数据传输量,本实施例的技术方案提供了一种优选实施方式,在优选方式下,在写入所述数据对象的结果数据流前,将读取到的属性值进行压缩。
在具体实施时,对属性值进行压缩可以采用out.writeVarint()进行压缩,或者采用比如较流行的LZ及系列变种(LZW,LZR,LZX)压缩算法等。
需要说明的是,在步骤S103中,所述根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类时,会出现以下2种情况:
第一种情况,能在代码库中获取到与所述元数据描述文件中的各个属性描述提供的属性操作类名称且数目相同的属性操作类,说明所述待序列化的数据对象与元数据描述文件的版本匹配,即:所述待序列化的数据对象与所述元数据描述文件版本兼容,所以不会出现兼容性问题。
需要说明的是,在第一种情况中,有可能是代码库中的属性操作类的数目与名称均与所述元数据描述文件中各个属性描述描述的属性操作类相同;还有一种可能是,代码库中的属性操作类的数目大于所述元数据描述文件中各个属性描述描述的属性操作类,但是代码库中包含有所述元数据描述文件中各个属性描述描述的所有属性操作类,在序列化时,是按照所述元数据描述文件加载对应于所述元数据描述文件中的属性操作类名称的属性操作类,代码库中没有用到的属性操作类不会被使用,所以不会出现兼容性问题。
第二种情况,只能在代码库中获取到部分与所述元数据描述文件中的各个属性描述 提供的属性操作类名称相同的属性操作类,说明所述待序列化的数据对象与元数据描述文件版本不匹配,即:数据对象是老版本下生成的数据对象,而元数据描述文件是最新版本,在序列化时会出现兼容性问题。
为了解决序列化时的元数据描述文件和数据对象的兼容性问题,本实施例的技术方案提供了一种优选实施方式,在优选方式下,具体包括步骤S106-1至S106-3,下面结合图4作进一步说明。
请参考图4,其示出了根据本申请的实施例提供的序列化时缺省值的读取方法的流程图。
步骤S106-1,获取由于无法获取对应属性的属性操作类而无法读取的属性描述的属性类型。
所述获取由于无法获取对应属性的属性操作类而无法读取的属性描述的属性类型,可以采用如下方式实现:从元数据描述文件中,确定由于无法获取对应属性操作类而无法读取属性值的属性描述,即:该条属性描述描述的属性操作类不存在,获取该条属性描述的属性类型。
需要说明的是,所述由于无法获取对应属性操作类而无法读取属性值,是由于代码库没有更新或者其它原因而没有存储该属性操作类。由于代码库一般存放在各种终端本地,因此很可能没有及时根据最新版本更新;而所使用的元数据描述文件则均为最新更新版本。
步骤S106-2,根据所述属性类型,检索获得对应所述属性类型的缺省属性操作类。
在本实施例中,所述属性类型,包括:整型,字符型,浮点型,字符串型等类型。
例如,获取的所述属性类型为VARINT,检索类型为DefaultVarintFieldDef的缺省属性操作类。
步骤S106-3,根据所述缺省属性操作类,为该属性描述对应的属性赋予预先确定的缺省值。
加载检索出的所述缺省属性操作类,调用所述缺省属性操作类中封装的getter方法,读取预先赋予的缺省值。
所述预先确定的缺省值根据不同的属性类型可以采用不同的缺省值,例如,整型类型的缺省值为0。
下面仍沿用上述对数据对象序列化的方法的具体例子,对本步骤的处理过程作进一步说明。
待序列化的数据对象为:
Class User{
int userId;
string name;
int age;
}
该数据对象的具体属性值存放在对应的数据库表中,例如一个具体的数据对象User为:
userId name age
1 aaa 20
需要说明的是,上述给出的User表仅仅是示意性的,在具体的实施中,对象User中的属性数据通常会更多,在本例子中仅给出一组数据说明本步骤的处理过程。
获取到的待序列化的数据对象的元数据描述文件为:
1:com.taobao.test.io.UserIdFieldDef:VARINT
2:com.taobao.test.io.NameFieldDef:VARCHART
3:com.taobao.test.io.PhoneFieldDef:VARINT
4:com.taobao.test.io.AgeFieldDef:VARINT
在步骤S103中,获取到的属性操作类包括:UserIdFieldDef、NameFieldDef以及AgeFieldDef,其中由于第三条属性描述未获取到与PhoneFieldDef同名的属性操作类,所以获取第三条属性描述中的属性类型,即:获取VARINT类型,通过获取的VARINT类型,检索对应类型VARINT的缺省属性操作类DefaultVarintFieldDef,获取属性操作类DefaultVarintFieldDef后,按照元数据描述文件中属性描述的序号的顺序,分别执行UserIdFieldDef中的getter方法读取userId的属性值1、执行NameFieldDef中的getter方法读取name的属性值aaa、执行DefaultVarintFieldDef中的getter方法读取缺省值0以及执行AgeFieldDef中的getter方法读取age的属性值20。
以UserIdFieldDef和DefaultVarintFieldDef为例,详细代码示例如下所示:UserIdFieldDef:
public class UserIdFieldDef<Item>implements FieldDef<T>{
public void setter(User user,CodedInputStream in){
user.setUserId(in.readVarint());
}
public void getter(User user,CodedOutputStream out){
out.writeVarint(user.getUserId());
}
}
DefaultVarintFieldDef:
public class DefaultVarintFieldDef<Object>implements FieldDef<T>{
public void setter(Object o,CodedInputStream in){
in.readVarint();
}
public void getter(Object o,CodedOutputStream out){
out.writeVarint(0);
}
}
执行不同的属性操作类时,只需修改对应的参数即可,对此不再赘述。
步骤S107,将读取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流,所述结果数据流作为所述数据对象的序列化结果。
为了使结果数据流在之后执行反序列化时获取序列化时使用的元数据描述文件,保证兼容性问题,再将取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流的步骤之前,包括:
读取所述待序列化的数据对象的元数据描述文件的版本号,并将其写入到所述待序列化的数据对象的结果数据流的头部。
在具体实施时,先获取所述待序列化的数据对象的元数据描述文件的版本号,将所述版本号的的数值写入结果数据流的第一个字节的位置,写入后序列化指针相应后移,将步骤S105中读取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流。
需要说明的是,所述序列化的结果数据流可以是二进制数据,但不限于二进制数据。
通过本申请实施例提供的一种对数据对象序列化的方法,通过获取元数据描述文件,根据元数据描述文件,对数据对象进行序列化;使得生成的结果数据流中除了数据外不包含附加数据,在保存信息和数据传输时节约了存储空间和传输流量,并且在反序列化时可以降低系统的处理开销。
在上述的实施例中,提供了一种对数据对象序列化的方法,与上述对数据对象序列化的方法相对应的,本申请还提供了一种对数据对象序列化的装置。由于装置的实施例基本相似于方法的实施例,所以描述得比较简单,相关之处参见方法实施例的部分说明即可。下述描述的装置实施例仅仅是示意性的。所述对数据对象序列化的装置实施例如下:
请参考图5,其示出了根据本申请的实施例提供的对数据对象序列化的装置的示意图。
所述对数据对象序列化的装置,包括:描述文件获取单元501、属性操作类获取单元503、属性读取单元505以及序列化写入单元507;
所述描述文件获取单元501,用于获取待序列化的数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称;
所述属性操作类获取单元503,用于根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类;
所述属性读取单元505,用于根据对应各个属性的属性操作类提供的读取方法,读取待序列化的数据对象中分别与各个属性描述对应的属性值;
所述序列化写入单元507,用于将读取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流,所述结果数据流作为所述数据对象的序列化结果。
可选的,所述描述文件获取单元501获取的所述待序列化的数据对象的元数据描述文件还包括属性类型。
可选的,所述属性读取单元505,还包括:属性类型读取子单元、缺省属性操作类获取子单元以及缺省值读取子单元;
所述属性类型读取子单元,用于读取待序列化的数据对象中分别与各个属性描述对应的属性值之后,获取由于无法获取对应属性的属性操作类而无法读取的属性描述的属性类型;
所述缺省属性操作类获取子单元,用于根据所述属性类型,检索获得对应所述属性类型的缺省属性操作类;
所述缺省值读取子单元,用于根据所述缺省属性操作类,为该属性描述对应的属性赋予预先确定的缺省值。
可选的,所述描述文件获取单元501具体用于根据所述待序列化的数据对象的名称, 从元数据仓库中查找对应所述待序列化的数据对象且版本号数值最大的所述待序列化的数据对象的元数据描述文件。
可选的,所述序列化写入单元507,还包括:版本号写入子单元;
所述版本号写入子单元,用于在所述将读取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流之前,读取所述待序列化的数据对象的元数据描述文件的版本号,并将其写入到所述待序列化的数据对象的结果数据流的头部。
可选的,所述对数据对象序列化的装置,还包括:修改操作接收单元、操作类创建单元以及描述文件创建单元;
所述修改操作接收单元,用于在所述获取待序列化的数据对象的元数据描述文件之前,接收对所述待序列化的数据对象的属性的修改操作;所述修改操作包括:对所述待序列化的数据对象的属性进行增加、删除或者变更;
所述操作类创建单元,用于基于对所述待序列化的数据对象的修改操作为所述待序列化的数据对象的新增属性创建属性操作类;
所述描述文件创建单元,用于根据所述修改操作创建新版本的元数据描述文件,该元数据描述文件中的属性描述在原版本元数据描述文件中的属性描述的基础上根据所述修改操作进行了变更
可选的,所述对数据对象序列化的装置,还包括:遍历单元、数值获取单元、版本号生成单元以及保存单元;
所述遍历单元,用于在所述根据所述修改操作创建新版本的元数据描述文件,该元数据描述文件中的属性描述在原版本元数据描述文件中的属性描述的基础上根据所述修改操作进行了变更之后,根据所述待序列化的数据对象的名称,在存有元数据描述文件的元数据仓库中遍历针对所述待序列化的数据对象设置的元数据描述文件;
所述数值获取单元,用于在遍历过程中,获取在所述元数据仓库中的所述待序列化的数据对象的元数据描述文件的版本号最大的数值;
所述版本号生成单元,用于将所述数值加一,并将加一后的所述数值作为新版本的元数据描述文件的版本号;
所述保存单元,用于将所述记录有版本号的所述元数据描述文件保存在所述元数据仓库中。
可选的,所述对数据对象序列化的装置,还包括:属性值压缩单元;
所述属性值压缩单元,用于根据对应各个属性的属性操作类提供的读取方法,读取 待序列化的数据对象中分别与各个属性描述对应的属性值之后,将读取到的属性值进行压缩。
在上述的实施例中,提供了一种对数据对象序列化的方法以及一种对数据对象序列化的装置,此外,本申请还提供了一种电子设备;所述电子设备实施例如下:
请参考图6,其示出了根据本申请的实施例提供的电子设备的示意图。
所述电子设备,包括:显示器601;处理器603;存储器605;
所述存储器605,用于存储序列化程序,所述程序在被所述处理器读取执行时,执行如下操作:获取待序列化的数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称;根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类;根据对应各个属性的属性操作类提供的读取方法,读取待序列化的数据对象中分别与各个属性描述对应的属性值;将读取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流,所述结果数据流作为所述数据对象的序列化结果。
在上述的实施例中,提供了一种对数据对象序列化的方法、一种对数据对象序列化的装置以及一种电子设备,此外,本申请还提供了一种对数据流反序列化的方法;所述对数据流反序列化的方法实施例如下:
请参考图7,其示出了根据本申请的实施例提供的对数据流反序列化的方法的流程图。
所述对数据流反序列化的方法包括:
步骤S701,获取结果数据流。
在本实施例中,所述获取结果数据流,包括:通过网络传输获取所述结果数据流或是从本地获取压缩的所述结果数据流。
需要说明的是,所述结果数据流可以是二进制数据,但不限于二进制数据。
步骤S703,获取所述本地数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称。
在本实施例中,所述获取所述本地数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称,具体包括步骤S703-1至S703-4,下面结合图8作进一步说明。
请参考图8,其示出了根据本申请的实施例提供的获取所述本地数据对象的元数据描述文件的流程图。
步骤S703-1,获取所述本地数据对象的名称。
例如,所述本地数据对象的名称为User,则获取的所述本地数据对象的名称为User。
步骤S703-2,解析所述结果数据流,获取所述结果数据流中的第一个值。
解析所述结果数据流,获取所述结果数据流中的第一个字节。
例如,获取的数值为6。
步骤S703-3,根据所述本地数据对象的名称,在存有元数据描述文件的元数据仓库中遍历针对所述本地数据对象设置的元数据描述文件。
例如,根据所述本地数据对象User,在存有元数据描述文件的元数据仓库中遍历针对所述本地数据对象User设置的元数据描述文件。
步骤S703-4,在遍历过程中,获取与所述本地数据对象的名称相同且版本号与所述结果数据流中的第一个值相同的所述本地数据对象的元数据描述文件。
例如,在遍历过程中,查找出在所述元数据仓库中的针对数据对象User设置的元数据描述文件共有6个,版本号为1-6,则在本步骤S603-4中获取版本号为6的元数据描述文件。
为了满足反序列化时的元数据描述文件和本地数据对象中属性的兼容问题,本实施例的技术方案提供了一种优选实施方式,在优选方式下,所述元数据描述文件中的每条属性描述,还包括:对应该条属性描述的属性类型。
下面针对所述对数据流反序列化的方法,本申请提供了一个具体例子便于对所述对数据流反序列化的方法进行说明:
待写入属性值的本地数据对象为:
Class User{
int userId;
string name;
int phone;
}
根据所述本地数据对象User,在存有元数据描述文件的元数据仓库中遍历针对所述本地数据对象User设置的元数据描述文件,若所述结果数据流中的第一个数值为6,那么获取版本号为6的元数据描述文件。例如:获取到的所述本地数据对象的元数据描述文件为:
1:com.taobao.test.io.UserIdFieldDef:VARINT
2:com.taobao.test.io.NameFieldDef:VARCHART
3:com.taobao.test.io.PhoneFieldDef:VARINT
4:com.taobao.test.io.AgeFieldDef:VARINT
需要说明的是,由于所述本地数据对象在获取元数据描述文件时,是根据序列化时写入结果数据流头部的数值确定的,所以在序列化时与反序列化时使用的元数据描述文件是相同的,即元数据描述文件是兼容的。
步骤S705,根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类。
在本实施例中,所述根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类,可以采用如下方式实现:根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,检索并获取与各个属性描述提供的属性操作类名称相同的属性操作类。
下面仍沿用上述对数据流反序列化的方法的具体例子,对本步骤的处理过程作进一步说明。
待写入属性值的本地数据对象为:
Class User{
int userId;
String name;
int phone;
}
所述本地数据对象的元数据描述文件为:
1:com.taobao.test.io.UserIdFieldDef:VARINT
2:com.taobao.test.io.NameFieldDef:VARINT
3:com.taobao.test.io.PhoneFieldDef:VARINT
4:com.taobao.test.io.AgeFieldDef:VARINT
从所述本地数据对象的元数据描述文件中,获取各个属性描述提供的属性操作类名称,包括UserIdFieldDef、NameFieldDef、PhoneFieldDef以及AgeFieldDef,检索并获取与各个属性描述提供的属性操作类名称相同的属性操作类,在本例子中,获取到的属性操作类包括:UserIdFieldDef、NameFieldDef以及PhoneFieldDef。
步骤S707,根据对应各个属性的属性操作类提供的读取方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取。
在本实施例中,根据步骤S705中获取对应各个属性的属性操作类中,各个属性描述的所述属性操作类提供的读取方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取。
为了保证数据的完整性,本实施例的技术方案提供了一种优选实施方式,在优选方式下,在从所述结果数据流中读取出属性值后,将从所述结果数据流中读取出的属性值进行解压缩。
在具体实施时,调用各个属性描述的所述属性操作类的in.readVarint()方法,读取序列化的数据对象相应属性的属性值,并将读取到的属性值进行解压缩。
需要说明的是,在步骤S705中,所述根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类时,会出现以下2种情况:
第一种情况,能在代码库中获取到与所述元数据描述文件中的各个属性描述提供的属性操作类名称且数目相同的属性操作类,说明所述本地数据对象与元数据描述文件的版本匹配,即:所述本地数据对象与所述元数据描述文件版本兼容,所以不会出现兼容性问题。
需要说明的是,在第一种情况中,有可能是代码库中的属性操作类的数目与名称均与所述元数据描述文件中各个属性描述描述的属性操作类相同;还有一种可能是,代码库中的属性操作类的数目大于所述元数据描述文件中各个属性描述描述的属性操作类,但是代码库中的属性操作类均包含所述元数据描述文件中各个属性描述描述的属性操作类,但是由于在序列化时,是按照所述元数据描述文件加载对应于所述元数据描述文件中的属性操作类名称的属性操作类,所以不会出现兼容性问题。
第二种情况,只能在代码库中获取到部分与所述元数据描述文件中的各个属性描述提供的属性操作类名称相同的属性操作类,说明所述本地数据对象与元数据描述文件版本不匹配,即:元数据描述文件是较新版本,在序列化时会出现兼容性问题。
为了满足反序列化时的元数据描述文件和本地数据对象的兼容性问题,本实施例的技术方案提供了一种优选实施方式,在优选方式下,具体包括步骤S707-1至S707-3,下面结合图9作进一步说明。
请参考图9,其示出了根据本申请的实施例提供的反序列化时缺省值的丢弃方法的流程图。
步骤S707-1,对于无法获得属性操作类的属性描述,根据该属性描述的属性类型,获取对应所述属性类型的缺省属性操作类。
所述对于无法获得属性操作类的属性描述,根据该属性描述的属性类型,获取对应所述属性类型的缺省属性操作类,可以采用如下方式实现:从元数据描述文件中,确定无法获得属性操作类的属性描述,即:该条属性描述描述的属性操作类不存在,在所述本地数据对象中不存在对应该属性操作类的属性,获取该条属性描述的属性类型,并根据该属性描述的属性类型,获取对应所述属性类型的缺省属性操作类。
在本实施例中,所述属性类型,包括:整型,字符型,浮点型,字符串型等类型。
例如:获取的所述属性类型为VARINT,获取类型为DefaultVarintFieldDef的缺省属性操作类。
步骤S707-2,获取所述缺省属性操作类提供的读取方法,按照属性描述序号的顺序读取所述结果数据流中的数据。
加载获取的所述缺省属性操作类,调用所述缺省属性操作类中封装的setter方法,读取出所述结果数据流中的属性值。
步骤S707-3,将读取的所述结果数据流中的数据丢弃。
需要说明的是,所述将读取的所述结果数据流中的数据丢弃,具体是指:将所述结果数据流中的属性值读取出之后,不进行写入,即:不将该属性值写入所述本地数据对象中。
在具体实施时,在本步骤S707中,每从所述结果数据流中读取出一个属性值后,就将反序列化指针相应后移至下一个属性值的位置。
下面仍沿用上述对数据流反序列化的方法的具体例子,对本步骤的处理过程作进一步说明。
待写入属性值的本地数据对象为:
Class User{
int userId;
String name;
int phone;
}
所述本地数据对象的元数据描述文件为:
1:com.taobao.test.io.UserIdFieldDef:VARINT
2:com.taobao.test.io.NameFieldDef:VARINT
3:com.taobao.test.io.PhoneFieldDef:VARINT
4:com.taobao.test.io.AgeFieldDef:VARINT
在步骤S705中,获取到的属性操作类包括:UserIdFieldDef、NameFieldDef以及PhoneFieldDef,其中由于第四条属性描述未获取到与AgeFieldDef同名的属性操作类,所以获取第四条属性描述中的属性类型,即:获取VARINT类型,通过获取的VARINT类型,检索对应类型VARINT的缺省属性操作类DefaultVarintFieldDef,获取属性操作类DefaultVarintFieldDef后,按照元数据描述文件中属性描述的序号的顺序,分别执行UserIdFieldDef中的in.readVarint()方法读取所述结果数据流相应位置的数据1,并对该数据进行解压缩,将反序列化指针相应后移至下一个属性值的位置、执行NameFieldDef中的in.readVarint()方法读取所述结果数据流相应位置的数据aaa,并对该数据进行解压缩,将反序列化指针相应后移至下一个属性值的位置、执行PhoneFieldDef中的in.readVarint()方法读取所述结果数据流相应位置的数据0,并对该数据进行解压缩,将反序列化指针相应后移至下一个属性值的位置,以及执行DefaultVarintFieldDef中的in.readVarint()方法读取,对该属性值解压缩后丢弃属性值20。
以UserIdFieldDef和DefaultVarintFieldDef为例,详细代码示例如下所示:UserIdFieldDef:
public class UserIdFieldDef<Item>implements FieldDef<T>{
public void setter(User user,CodedInputStream in){
user.setUserId(in.readVarint());
}
public void getter(User user,CodedOutputStream out){
out.writeVarint(user.getUserId());
}
}
DefaultVarintFieldDef:
public class DefaultVarintFieldDef<Object>implements FieldDef<T>{
public void setter(Object o,CodedInputStream in){
in.readVarint();
}
public void getter(Object o,CodedOutputStream out){
out.writeVarint(0);
}
}
执行不同的属性操作类时,只需修改对应的参数即可,对此不再赘述。
步骤S709,根据各个属性描述的所述属性操作类提供的写入方法,将读取到的属性值写入与各个属性描述相对应的所述本地数据对象的属性中。
在具体实施时,将步骤S707中通过in.readVarint()方法读取的所述结果数据流中的属性值,对读取到的属性值进行解压缩后,按照属性描述序号通过setter方法写入对应的所述本地数据对象的属性中。
例如,执行UserIdFieldDef中的user.setUserId(in.readVarint())将从所述结果数据流中读取到的属性值写入与userId相对应的所述本地数据对象的属性中
在上述的实施例中,提供了一种对数据流反序列化的方法,与上述对数据流反序列化的方法相对应的,本申请还提供了一种对数据流反序列化的装置。由于装置的实施例基本相似于方法的实施例,所以描述得比较简单,相关之处参见方法实施例的部分说明即可。下述描述的装置实施例仅仅是示意性的。所述对数据流反序列化的装置实施例如下:
请参考图10,其示出了根据本申请的实施例提供的对数据流反序列化的装置的示意图。
所述对数据流反序列化的装置,包括:数据流获取单元1001、描述文件获取单元1003、属性操作类获取单元1005、属性值读取单元1007以及属性值写入单元1009;
所述数据流获取单元1001,用于获取结果数据流;
所述描述文件获取单元1003,用于获取所述本地数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称;
所述属性操作类获取单元1005,用于根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类;
所述属性值读取单元1007,用于根据对应各个属性的属性操作类提供的读取方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取;
所述属性值写入单元1009,用于根据各个属性描述的所述属性操作类提供的写入方法,将读取到的属性值写入与各个属性描述相对应的所述本地数据对象的属性中。
可选的,所述描述文件获取单元1003,获取的所述本地数据对象的元数据描述文件还包括属性类型。
可选的,所述属性值读取单元1007包括:属性类型获取子单元、数据读取子单元以 及丢弃子单元;
所述属性类型获取子单元,用于在所述根据各个属性描述的所述属性操作类提供的写入方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取之中,对于无法获得属性操作类的属性描述,根据该属性描述的属性类型,获取对应所述属性类型的缺省属性操作类;
所述数据读取子单元,用于获取所述缺省属性操作类提供的读取方法,按照属性描述序号的顺序读取所述结果数据流中的数据;
所述丢弃单子元,用于将读取的所述结果数据流中的数据丢弃。
可选的,所述描述文件获取单元1003,包括:本地数据对象名称获取子单元、结果数据流解析子单元、描述文件遍历子单元以及描述文件获取子单元;
所述本地数据对象名称获取子单元,用于获取所述本地数据对象的名称;
所述结果数据流解析子单元,用于解析所述结果数据流,获取所述结果数据流中的第一个值;
所述描述文件遍历子单元,用于根据所述本地数据对象的名称,在存有元数据描述文件的元数据仓库中遍历针对所述本地数据对象设置的元数据描述文件;
所述描述文件获取子单元,用于在遍历过程中,获取与所述本地数据对象的名称相同且版本号与所述结果数据流中的第一个值相同的所述本地数据对象的元数据描述文件。
可选的,所述对数据流反序列化的装置,还包括:属性值解压缩单元;
所述属性值解压缩单元,用于在所述根据对应各个属性的属性操作类提供的读取方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取的之后,将从所述结果数据流中读取出的属性值进行解压缩。
在上述的实施例中,提供了一种对数据对象序列化的方法、一种对数据对象序列化的装置、一种电子设备、一种对数据流反序列化的方法以及一种对数据流反序列化的装置,此外,本申请还提供了又一种电子设备;所述电子设备实施例如下:
请参考图11,其示出了根据本申请的实施例提供的电子设备的示意图。
所述电子设备,包括:显示器1101;处理器1103;存储器1105;
所述存储器1105,用于存储反序列化程序,所述程序在被所述处理器读取执行时,执行如下操作:获取结果数据流,获取所述本地数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称,根 据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类,根据对应各个属性的属性操作类提供的读取方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取,根据各个属性描述的所述属性操作类提供的写入方法,将读取到的属性值写入与各个属性描述相对应的所述本地数据对象的属性中。
在上述的实施例中,提供了一种对数据对象序列化的方法、一种对数据对象序列化的装置、一种电子设备、一种对数据流反序列化的方法、一种对数据流反序列化的装置以及又一种电子设备,此外,本申请还提供了一种序列化与反序列化系统;所述序列化与反序列化系统实施例如下:
请参考图12,示出了根据本申请的实施例提供的序列化与反序列化系统的示意图。
所述序列化与反序列化系统包括:对数据对象序列化的装置1201以及对数据流反序列化的装置1203;
其中,所述对数据对象序列化的装置1201,用于获取元数据描述文件,根据所述元数据描述文件获取对应各个属性的属性操作类,读取待序列化的数据对象中分别与各个属性描述对应的属性值,并将读取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流;
所述对数据流反序列化的装置1203,用于获取结果数据流,获取元数据描述文件,根据所述元数据描述文件获取对应各个属性的属性操作类,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取,并将读取到的属性值写入与各个属性描述相对应的所述本地数据对象的属性中。
在一个典型的配置中,计算设备包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。
内存可能包括计算机可读介质中的非永久性存储器,随机存取存储器(RAM)和/或非易失性内存等形式,如只读存储器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。
1、计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括,但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁 带,磁带磁磁盘存储或其他磁性存储设备或任何其他非传输介质,可用于存储可以被计算设备访问的信息。按照本文中的界定,计算机可读介质不包括非暂存电脑可读媒体(transitory media),如调制的数据信号和载波。
2、本领域技术人员应明白,本申请的实施例可提供为方法、系统或计算机程序产品。因此,本申请可采用完全硬件实施例、完全软件实施例或结合软件和硬件方面的实施例的形式。而且,本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本申请虽然以较佳实施例公开如上,但其并不是用来限定本申请,任何本领域技术人员在不脱离本申请的精神和范围内,都可以做出可能的变动和修改,因此本申请的保护范围应当以本申请权利要求所界定的范围为准。

Claims (29)

  1. 一种对数据对象序列化的方法,其特征在于,包括:
    获取待序列化的数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称;
    根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类;
    根据对应各个属性的属性操作类提供的读取方法,读取待序列化的数据对象中分别与各个属性描述对应的属性值;
    将读取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流,所述结果数据流作为所述数据对象的序列化结果。
  2. 根据权利要求1所述的对数据对象序列化的方法,其特征在于,所述属性描述,还包括:属性类型。
  3. 根据权利要求2所述的对数据对象序列化的方法,其特征在于,在所述读取待序列化的数据对象中分别与各个属性描述对应的属性值的步骤之后,包括:
    获取由于无法获取对应属性的属性操作类而无法读取的属性描述的属性类型;
    根据所述属性类型,检索获得对应所述属性类型的缺省属性操作类;
    根据所述缺省属性操作类,为该属性描述对应的属性赋予预先确定的缺省值。
  4. 根据权利要求1所述的对数据对象序列化的方法,其特征在于,所述获取待序列化的数据对象的元数据描述文件是:
    根据所述待序列化的数据对象的名称,从元数据仓库中查找对应所述待序列化的数据对象且版本号数值最大的所述待序列化的数据对象的元数据描述文件。
  5. 根据权利要求4所述的对数据对象序列化的方法,其特征在于,在所述将读取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流的步骤之前,包括:
    读取所述待序列化的数据对象的元数据描述文件的版本号,并将其写入到所述待序列化的数据对象的结果数据流的头部。
  6. 根据权利要求1所述的对数据对象序列化的方法,其特征在于,在所述获取待序列化的数据对象的元数据描述文件的步骤之前,还包括:
    接收对所述待序列化的数据对象的属性的修改操作;所述修改操作包括:对所述待序列化的数据对象的属性进行增加、删除或者变更;
    基于对所述待序列化的数据对象的修改操作为所述待序列化的数据对象的新增属性创建属性操作类;
    根据所述修改操作创建新版本的元数据描述文件,该元数据描述文件中的属性描述在原版本元数据描述文件中的属性描述的基础上根据所述修改操作进行了变更。
  7. 根据权利要求6所述的对数据对象序列化的方法,其特征在于,在所述根据所述修改操作创建新版本的元数据描述文件,该元数据描述文件中的属性描述在原版本元数据描述文件中的属性描述的基础上根据所述修改操作进行了变更的步骤之后,包括:
    根据所述待序列化的数据对象的名称,在存有元数据描述文件的元数据仓库中遍历针对所述待序列化的数据对象设置的元数据描述文件;
    在遍历过程中,获取在所述元数据仓库中的所述待序列化的数据对象的元数据描述文件的版本号最大的数值;
    将所述数值加一,并将加一后的所述数值作为新版本的元数据描述文件的版本号;
    将所述记录有版本号的所述元数据描述文件保存在所述元数据仓库中。
  8. 根据权利要求1所述的对数据对象序列化的方法,其特征在于,在所述根据对应各个属性的属性操作类提供的读取方法,读取待序列化的数据对象中分别与各个属性描述对应的属性值的步骤之后,包括:
    将读取到的属性值进行压缩。
  9. 一种对数据对象序列化的装置,其特征在于,包括:描述文件获取单元、属性操作类获取单元、属性读取单元以及序列化写入单元;
    所述描述文件获取单元,用于获取待序列化的数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称;
    所述属性操作类获取单元,用于根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类;
    所述属性读取单元,用于根据对应各个属性的属性操作类提供的读取方法,读取待序列化的数据对象中分别与各个属性描述对应的属性值;
    所述序列化写入单元,用于将读取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流,所述结果数据流作为所述数据对象的序列化结果。
  10. 根据权利要求9所述的对数据对象序列化的装置,其特征在于,所述描述文件获取单元获取的所述待序列化的数据对象的元数据描述文件还包括属性类型。
  11. 根据权利要求10所述的对数据对象序列化的装置,其特征在于,所述属性读取单元,还包括:属性类型读取子单元、缺省属性操作类获取子单元以及缺省值读取子单元;
    所述属性类型读取子单元,用于读取待序列化的数据对象中分别与各个属性描述对应的属性值之后,获取由于无法获取对应属性的属性操作类而无法读取的属性描述的属性类型;
    所述缺省属性操作类获取子单元,用于根据所述属性类型,检索获得对应所述属性类型的缺省属性操作类;
    所述缺省值读取子单元,用于根据所述缺省属性操作类,为该属性描述对应的属性赋予预先确定的缺省值。
  12. 根据权利要求9所述的对数据对象序列化的装置,其特征在于,所述描述文件获取单元具体用于根据所述待序列化的数据对象的名称,从元数据仓库中查找对应所述待序列化的数据对象且版本号数值最大的所述待序列化的数据对象的元数据描述文件。
  13. 根据权利要求12所述的对数据对象序列化的装置,其特征在于,所述序列化写入单元,还包括:版本号写入子单元;
    所述版本号写入子单元,用于在所述将读取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流之前,读取所述待序列化的数据对象的元数据描述文件的版本号,并将其写入到所述待序列化的数据对象的结果数据流的头部。
  14. 根据权利要求9所述的对数据对象序列化的装置,其特征在于,还包括:修改操作接收单元、操作类创建单元以及描述文件创建单元;
    所述修改操作接收单元,用于在所述获取待序列化的数据对象的元数据描述文件之前,接收对所述待序列化的数据对象的属性的修改操作;所述修改操作包括:对所述待序列化的数据对象的属性进行增加、删除或者变更;
    所述操作类创建单元,用于基于对所述待序列化的数据对象的修改操作为所述待序列化的数据对象的新增属性创建属性操作类;
    所述描述文件创建单元,用于根据所述修改操作创建新版本的元数据描述文件,该元数据描述文件中的属性描述在原版本元数据描述文件中的属性描述的基础上根据所述修改操作进行了变更。
  15. 根据权利要求14所述的对数据对象序列化的装置,其特征在于,还包括:遍 历单元、数值获取单元、版本号生成单元以及保存单元;
    所述遍历单元,用于在所述根据所述修改操作创建新版本的元数据描述文件,该元数据描述文件中的属性描述在原版本元数据描述文件中的属性描述的基础上根据所述修改操作进行了变更之后,根据所述待序列化的数据对象的名称,在存有元数据描述文件的元数据仓库中遍历针对所述待序列化的数据对象设置的元数据描述文件;
    所述数值获取单元,用于在遍历过程中,获取在所述元数据仓库中的所述待序列化的数据对象的元数据描述文件的版本号最大的数值;
    所述版本号生成单元,用于将所述数值加一,并将加一后的所述数值作为新版本的元数据描述文件的版本号;
    所述保存单元,用于将所述记录有版本号的所述元数据描述文件保存在所述元数据仓库中。
  16. 根据权利要求9所述的对数据对象序列化的装置,其特征在于,还包括:属性值压缩单元;
    所述属性值压缩单元,用于根据对应各个属性的属性操作类提供的读取方法,读取待序列化的数据对象中分别与各个属性描述对应的属性值之后,将读取到的属性值进行压缩。
  17. 一种电子设备,其特征在于,所述电子设备包括:
    显示器;
    处理器;
    存储器,用于存储序列化程序,所述程序在被所述处理器读取执行时,执行如下操作:获取待序列化的数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称;根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类;根据对应各个属性的属性操作类提供的读取方法,读取待序列化的数据对象中分别与各个属性描述对应的属性值;将读取到的属性值按照属性描述序号的顺序写入所述数据对象的结果数据流,所述结果数据流作为所述数据对象的序列化结果。
  18. 一种对数据流反序列化的方法,其特征在于,包括:
    获取结果数据流;
    获取所述本地数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称;
    根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类;
    根据对应各个属性的属性操作类提供的读取方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取;
    根据各个属性描述的所述属性操作类提供的写入方法,将读取到的属性值写入与各个属性描述相对应的所述本地数据对象的属性中。
  19. 根据权利要求18所述的对数据流反序列化的方法,其特征在于,所述属性描述,还包括:属性类型。
  20. 根据权利要求19所述的对数据流反序列化的方法,其特征在于,在所述根据各个属性描述的所述属性操作类提供的写入方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取的步骤之中,还包括:
    对于无法获得属性操作类的属性描述,根据该属性描述的属性类型,获取对应所述属性类型的缺省属性操作类;
    获取所述缺省属性操作类提供的读取方法,按照属性描述序号的顺序读取所述结果数据流中的数据;
    将读取的所述结果数据流中的数据丢弃。
  21. 根据权利要求18所述的对数据流反序列化的方法,其特征在于,所述获取所述本地数据对象的元数据描述文件包括:
    获取所述本地数据对象的名称;
    解析所述结果数据流,获取所述结果数据流中的第一个值;
    根据所述本地数据对象的名称,在存有元数据描述文件的元数据仓库中遍历针对所述本地数据对象设置的元数据描述文件;
    在遍历过程中,获取与所述本地数据对象的名称相同且版本号与所述结果数据流中的第一个值相同的所述本地数据对象的元数据描述文件。
  22. 根据权利要求18所述的对数据流反序列化的方法,其特征在于,在所述根据对应各个属性的属性操作类提供的读取方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取的步骤之后,包括:
    将从所述结果数据流中读取出的属性值进行解压缩。
  23. 一种对数据流反序列化的装置,其特征在于,包括:数据流获取单元、描述文件获取单元、属性操作类获取单元、属性值读取单元以及属性值写入单元;
    所述数据流获取单元,用于获取结果数据流;
    所述描述文件获取单元,用于获取所述本地数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称;
    所述属性操作类获取单元,用于根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类;
    所述属性值读取单元,用于根据对应各个属性的属性操作类提供的读取方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取;
    所述属性值写入单元,用于根据各个属性描述的所述属性操作类提供的写入方法,将读取到的属性值写入与各个属性描述相对应的所述本地数据对象的属性中。
  24. 根据权利要求23所述的对数据流反序列化的装置,其特征在于,所述描述文件获取单元,获取的所述本地数据对象的元数据描述文件还包括属性类型。
  25. 根据权利要求24所述的对数据流反序列化的装置,其特征在于,所述属性值读取单元包括:属性类型获取子单元、数据读取子单元以及丢弃子单元;
    所述属性类型获取子单元,用于在所述根据各个属性描述的所述属性操作类提供的写入方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取之中,对于无法获得属性操作类的属性描述,根据该属性描述的属性类型,获取对应所述属性类型的缺省属性操作类;
    所述数据读取子单元,用于获取所述缺省属性操作类提供的读取方法,按照属性描述序号的顺序读取所述结果数据流中的数据;
    所述丢弃单子元,用于将读取的所述结果数据流中的数据丢弃。
  26. 根据权利要求23所述的对数据流反序列化的装置,其特征在于,所述描述文件获取单元,包括:本地数据对象名称获取子单元、结果数据流解析子单元、描述文件遍历子单元以及描述文件获取子单元;
    所述本地数据对象名称获取子单元,用于获取所述本地数据对象的名称;
    所述结果数据流解析子单元,用于解析所述结果数据流,获取所述结果数据流中的第一个值;
    所述描述文件遍历子单元,用于根据所述本地数据对象的名称,在存有元数据描述文件的元数据仓库中遍历针对所述本地数据对象设置的元数据描述文件;
    所述描述文件获取子单元,用于在遍历过程中,获取与所述本地数据对象的名称相 同且版本号与所述结果数据流中的第一个值相同的所述本地数据对象的元数据描述文件。
  27. 根据权利要求23所述的对数据流反序列化的装置,其特征在于,还包括:属性值解压缩单元;
    所述属性值解压缩单元,用于在所述根据对应各个属性的属性操作类提供的读取方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取的之后,将从所述结果数据流中读取出的属性值进行解压缩。
  28. 一种电子设备,其特征在于,所述电子设备包括:
    显示器;
    处理器;
    存储器,用于存储反序列化程序,所述程序在被所述处理器读取执行时,执行如下操作:获取结果数据流,获取所述本地数据对象的元数据描述文件,所述元数据描述文件包含以序号标识的属性描述以及对应所述数据对象中属性的属性操作类名称,根据所述元数据描述文件中的各个属性描述提供的属性操作类名称,获取对应各个属性的属性操作类,根据对应各个属性的属性操作类提供的读取方法,依次将所述结果数据流中的属性值按照属性描述序号的顺序读取,根据各个属性描述的所述属性操作类提供的写入方法,将读取到的属性值写入与各个属性描述相对应的所述本地数据对象的属性中。
  29. 一种序列化与反序列化系统,其特征在于,所述系统包括:
    根据上述权利要求9至16中任一项所述的对数据对象序列化的装置;以及根据权利要求23至27中任一项所述的对数据流反序列化的装置。
PCT/CN2016/101448 2015-10-10 2016-10-08 序列化与反序列化的方法、装置、系统以及电子设备 Ceased WO2017059798A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510650688.2 2015-10-10
CN201510650688.2A CN106570018B (zh) 2015-10-10 2015-10-10 序列化与反序列化的方法、装置、系统以及电子设备

Publications (1)

Publication Number Publication Date
WO2017059798A1 true WO2017059798A1 (zh) 2017-04-13

Family

ID=58487676

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/101448 Ceased WO2017059798A1 (zh) 2015-10-10 2016-10-08 序列化与反序列化的方法、装置、系统以及电子设备

Country Status (2)

Country Link
CN (1) CN106570018B (zh)
WO (1) WO2017059798A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108733744A (zh) * 2018-04-02 2018-11-02 东方视界科技(北京)有限公司 数据存储、读取方法及装置
CN109359144A (zh) * 2018-08-24 2019-02-19 中国建设银行股份有限公司 数据存储方法及系统、装置和存储介质
CN114490654A (zh) * 2022-01-25 2022-05-13 重庆零壹空间航天科技有限公司 火箭测试等效器的数据处理方法、计算机设备和存储介质
CN116301666A (zh) * 2023-05-17 2023-06-23 杭州数云信息技术有限公司 Java对象序列化方法及其反序列化方法及装置、终端

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107273654B (zh) * 2017-04-27 2021-04-09 广州慧扬健康科技有限公司 电子病历表单的存储优化系统
CN107341262B (zh) * 2017-07-14 2020-01-21 上海达梦数据库有限公司 数据库中对象类型列的序列化、反序列化方法及装置
CN107451237B (zh) * 2017-07-26 2020-09-15 阿里巴巴集团控股有限公司 序列化与反序列化方法、装置及设备
CN107463668B (zh) * 2017-08-02 2019-03-26 湖南新航动力信息科技有限公司 序列化和反序列化的方法及装置、计算机设备及存储介质
CN109460304B (zh) * 2017-09-06 2022-06-03 阿里巴巴集团控股有限公司 一种跨应用的通知展示、通知内容封装方法和装置
CN109558251B (zh) * 2017-09-26 2021-04-30 北京京东尚科信息技术有限公司 页面结构信息修改的方法及终端
CN108153896B (zh) * 2018-01-08 2020-07-10 第四范式(北京)技术有限公司 针对输入数据、输出数据的处理方法及装置
CN109947739B (zh) * 2018-05-31 2021-10-15 新华三大数据技术有限公司 数据源管理方法及装置
CN110914799B (zh) 2018-06-30 2023-02-28 华为云计算技术有限公司 一种反序列化方法、装置以及计算设备
CN110895530A (zh) * 2018-09-11 2020-03-20 北京奇虎科技有限公司 数据存储方法、分区服务器及电子设备
CN109636884A (zh) * 2018-10-25 2019-04-16 阿里巴巴集团控股有限公司 动画处理方法、装置及设备
CN110515893B (zh) * 2019-07-26 2022-12-09 济南浪潮数据技术有限公司 数据存储方法、装置、设备及计算机可读存储介质
CN111563123B (zh) * 2020-05-07 2023-08-22 北京首汽智行科技有限公司 一种hive仓库元数据实时同步方法
CN113553033B (zh) * 2021-06-24 2022-04-19 厦门正航软件科技有限公司 对象序列化和反序列化方法、装置及计算机可读存储介质
CN115563956A (zh) * 2022-09-26 2023-01-03 重庆长安汽车股份有限公司 一种车载数据的解析方法、装置、设备及介质
CN115617875A (zh) * 2022-10-25 2023-01-17 广东美的智能科技有限公司 序列化和反序列化方法及装置、电子设备和存储介质
CN119806486B (zh) * 2024-12-02 2025-10-24 中交武汉智行国际工程咨询有限公司 基于虚幻引擎的对象数据序列化和反序列化系统及方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040064825A1 (en) * 2002-09-30 2004-04-01 Timothy Lim Method and system for object system interoperability
CN101477532A (zh) * 2008-12-23 2009-07-08 北京畅游天下网络技术有限公司 实现数据存储、读取的方法、装置及系统
CN104360865A (zh) * 2014-11-28 2015-02-18 中国建设银行股份有限公司 一种序列化方法、反序列化方法及相关设备
CN104850565A (zh) * 2014-02-19 2015-08-19 北京京东尚科信息技术有限公司 一种基于k-v存储系统的元数据管理方法
CN104899316A (zh) * 2015-06-17 2015-09-09 北京奇虎科技有限公司 游戏数据的序列化与反序列化处理方法及装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7191196B2 (en) * 2001-06-28 2007-03-13 International Business Machines Corporation Method and system for maintaining forward and backward compatibility in flattened object streams
CN102004633B (zh) * 2009-09-03 2013-04-24 阿里巴巴集团控股有限公司 一种处理控件属性的方法及装置
CN101661391B (zh) * 2009-09-24 2013-08-28 金蝶软件(中国)有限公司 一种对象序列化方法、对象反序列化方法、装置及系统
CN101794225A (zh) * 2010-02-26 2010-08-04 南京师范大学 一种基于gdom及其持久化的gml解析方法
CN102495757A (zh) * 2011-11-15 2012-06-13 深圳市融创天下科技股份有限公司 一种适用于java远程调用对象传输的优化方法、装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040064825A1 (en) * 2002-09-30 2004-04-01 Timothy Lim Method and system for object system interoperability
CN101477532A (zh) * 2008-12-23 2009-07-08 北京畅游天下网络技术有限公司 实现数据存储、读取的方法、装置及系统
CN104850565A (zh) * 2014-02-19 2015-08-19 北京京东尚科信息技术有限公司 一种基于k-v存储系统的元数据管理方法
CN104360865A (zh) * 2014-11-28 2015-02-18 中国建设银行股份有限公司 一种序列化方法、反序列化方法及相关设备
CN104899316A (zh) * 2015-06-17 2015-09-09 北京奇虎科技有限公司 游戏数据的序列化与反序列化处理方法及装置

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108733744A (zh) * 2018-04-02 2018-11-02 东方视界科技(北京)有限公司 数据存储、读取方法及装置
CN109359144A (zh) * 2018-08-24 2019-02-19 中国建设银行股份有限公司 数据存储方法及系统、装置和存储介质
CN114490654A (zh) * 2022-01-25 2022-05-13 重庆零壹空间航天科技有限公司 火箭测试等效器的数据处理方法、计算机设备和存储介质
CN116301666A (zh) * 2023-05-17 2023-06-23 杭州数云信息技术有限公司 Java对象序列化方法及其反序列化方法及装置、终端
CN116301666B (zh) * 2023-05-17 2023-10-10 杭州数云信息技术有限公司 Java对象序列化方法及其反序列化方法及装置、终端

Also Published As

Publication number Publication date
CN106570018A (zh) 2017-04-19
CN106570018B (zh) 2021-01-15

Similar Documents

Publication Publication Date Title
WO2017059798A1 (zh) 序列化与反序列化的方法、装置、系统以及电子设备
CN106933703B (zh) 一种数据库数据备份的方法、装置及电子设备
US9678969B2 (en) Metadata updating method and apparatus based on columnar storage in distributed file system, and host
JP5376696B2 (ja) ステートレスなプロトコルを介したドキュメントの同期
CN111324665B (zh) 一种日志回放方法及装置
CN109710185B (zh) 数据处理方法及装置
CN110347674A (zh) 一种基于业务标识的索引创建方法、装置及设备
CN110716965A (zh) 一种块链式账本中的查询方法、装置及设备
CN106980665A (zh) 数据字典实现方法、装置及数据字典管理系统
CN106708911A (zh) 一种云环境下数据文件同步的方法和装置
CN111198880A (zh) 一种基于redis的数据存储方法、装置及电子设备
CN106874329A (zh) 数据库表索引的实现方法和装置
CN102597995B (zh) 同步数据库和非数据库资源
CN113177095A (zh) 一种企业知识管理方法、系统、电子设备及存储介质
CN105404653A (zh) 一种全分布式文件索引及协作编辑机制的实现方法
US20070156778A1 (en) File indexer
CN120596435A (zh) 元数据写入、元数据读取方法和装置
CN114647630B (zh) 文件同步、信息生成方法、装置、计算机设备和存储介质
CN117891796A (zh) 一种适用于多读少写场景的hdfs海量小文件存储方法
CN112527911B (zh) 一种数据存储方法、装置、设备及介质
CN111444194B (zh) 一种块链式账本中索引的清除方法、装置及设备
CN118519964A (zh) 数据处理方法、装置、计算机程序产品、设备及存储介质
CN115964442A (zh) 一种区块链中的账户状态访问方法和区块链节点
CN114816247A (zh) 一种逻辑数据获取方法及装置
CN116069755A (zh) mapper文件处理方法及相关设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16853102

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16853102

Country of ref document: EP

Kind code of ref document: A1