CN106570018B

CN106570018B - Serialization and deserialization method, device and system and electronic equipment

Info

Publication number: CN106570018B
Application number: CN201510650688.2A
Authority: CN
Inventors: 董石鸣
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2015-10-10
Filing date: 2015-10-10
Publication date: 2021-01-15
Anticipated expiration: 2035-10-10
Also published as: CN106570018A; WO2017059798A1

Abstract

The application discloses a method, a device and electronic equipment for serializing data objects, a method, a device and electronic equipment for deserializing data streams, and a serialization and deserialization system. Wherein the method of serializing data objects comprises: acquiring a metadata description file of a data object to be serialized; acquiring a corresponding attribute operation class according to an attribute operation class name provided by attribute description in the metadata description file; reading the attribute value in the data object to be serialized according to the reading method provided by the corresponding attribute operation class; and writing the read attribute values into a result data stream of the data object according to the sequence of the attribute description sequence numbers. The method solves the compatibility problem between systems after the attributes of the objects are changed, splits the metadata of the objects and the values of the objects, saves storage space and transmission flow during storage and transmission, and reduces the processing overhead of the systems during deserialization.

Description

Serialization and deserialization method, device and system and electronic equipment

Technical Field

The present application relates to the field of computer communications technologies, and in particular, to a method, an apparatus, and an electronic device for serializing data objects; a method of deserializing a data stream, an apparatus for deserializing a data stream and an electronic device; a serialization and deserialization system.

Background

Object Serialization (Serialization) is a technique that converts object data into another form of persistent data, i.e., the process of converting the state information of an object into a form that can be stored or transmitted. During serialization, the object writes its current state to a temporary or persistent store. Such a conversion must be reversible in order to enable the stored and transferred data to be transformed back to the object in memory again. The object may be recreated by reading or deserializing the state of the object from storage.

In data transmission and application between networks, information is generally required to be converted into data streams to realize data transmission and application. However, in the conventional serialization and deserialization schemes, if the attribute in the object is increased or decreased, the deserialization side fails to synchronize the attribute in the object. The attributes in the object are added, reduced or modified, and errors or serialization failures may occur when deserializing the original serialized file or data. This is a compatibility problem for object serialization.

Currently, the prior art generally adopts the protocol buffer (data interchange) format of google to realize serialization and deserialization. The Protobuf is a data exchange format, and sequences the existing information into binary streams according to different types and corresponding coding modes, and is used for storing information or transmitting data. When the binary information is read or received, the serialized binary stream is deserialized to obtain the original information or data. Although Protobuf can reduce metadata content in a serialization result during serialization, a small amount of metadata (attribute types and attribute sequence numbers) is reserved for compatibility, and during deserialization, besides obtaining a binary stream, the type adopted during serialization needs to be known, and then corresponding deserialization is performed according to the type.

Therefore, in the prior art, because metadata is also reserved during serialization, the serialization compression rate is low, and space and flow are wasted during information storage or data transmission; in addition, in deserialization, the process of obtaining the serialization type used in object serialization is complex, and the processing overhead is increased.

Disclosure of Invention

The application provides a method for serializing a data object, a device for serializing a data object, an electronic device, a method for deserializing a data stream, a device for deserializing a data stream, an electronic device and a serialization and deserializing system, so as to solve the problem of low serialization compression rate in the prior art.

The present application provides a method of serializing data objects, the method of serializing data objects comprising:

acquiring a metadata description file of a data object to be serialized, wherein the metadata description file comprises attribute description marked by a sequence number and an attribute operation class name corresponding to an attribute in the data object;

obtaining attribute operation classes corresponding to the attributes according to the attribute operation class names provided by the attribute descriptions in the metadata description file;

reading attribute values respectively corresponding to the attribute descriptions in the data objects to be serialized according to the reading method provided by the attribute operation class corresponding to each attribute;

and writing the read attribute values into a result data stream of the data object according to the sequence of the attribute description sequence number, wherein the result data stream is used as a serialization result of the data object.

Optionally, the attribute description further includes: the attribute type.

Optionally, after the step of reading the attribute values corresponding to the attribute descriptions in the data object to be serialized, the method includes:

acquiring an attribute type of attribute description which cannot be read because the attribute operation class corresponding to the attribute cannot be acquired;

retrieving and obtaining a default attribute operation class corresponding to the attribute type according to the attribute type;

and according to the default attribute operation class, giving a predetermined default value to the attribute corresponding to the attribute description.

Optionally, the obtaining of the metadata description file of the data object to be serialized is:

and searching a metadata description file of the data object to be serialized, which corresponds to the data object to be serialized and has the maximum version number value, from a metadata warehouse according to the name of the data object to be serialized.

Optionally, before the step of writing the read attribute values into the result data stream of the data object according to the sequence of the attribute description sequence numbers, the method includes:

and reading the version number of the metadata description file of the data object to be serialized, and writing the version number into the head of a result data stream of the data object to be serialized.

Optionally, before the step of obtaining the metadata description file of the data object to be serialized, the method further includes:

receiving modification operation on the attribute of the data object to be serialized; the modifying operation includes: adding, deleting or changing the attribute of the data object to be serialized;

creating an attribute operation class for the newly added attribute of the data object to be serialized on the basis of the modification operation on the data object to be serialized;

and creating a new version of the metadata description file according to the modification operation, wherein the attribute description in the metadata description file is changed according to the modification operation on the basis of the attribute description in the original metadata description file.

Optionally, after the step of creating a new version of the metadata description file according to the modification operation, where the attribute description in the metadata description file is changed according to the modification operation on the basis of the attribute description in the original metadata description file, the method includes:

traversing a metadata description file set for the data object to be serialized in a metadata warehouse in which the metadata description file is stored according to the name of the data object to be serialized;

in the traversing process, acquiring a numerical value with the maximum version number of the metadata description file of the data object to be serialized in the metadata warehouse;

adding one to the numerical value, and using the numerical value after adding one as the version number of the metadata description file of the new version;

and saving the metadata description file recorded with the version number in the metadata warehouse.

Optionally, after the step of reading the attribute values corresponding to the attribute descriptions in the data object to be serialized according to the reading method provided by the attribute operation class corresponding to each attribute, the method includes:

and compressing the read attribute values.

Accordingly, the present application also provides an apparatus for serializing data objects, the apparatus for serializing data objects comprising: the device comprises a description file acquisition unit, an attribute operation class acquisition unit, an attribute reading unit and a serialization writing unit;

the description file acquisition unit is used for acquiring a metadata description file of a data object to be serialized, wherein the metadata description file comprises attribute description marked by a sequence number and an attribute operation class name corresponding to an attribute in the data object;

the attribute operation class acquisition unit is used for acquiring the attribute operation classes corresponding to the attributes according to the attribute operation class names provided by the attribute descriptions in the metadata description file;

the attribute reading unit is used for reading attribute values which respectively correspond to the attribute descriptions in the data object to be serialized according to the reading method provided by the attribute operation class corresponding to each attribute;

and the serialization writing unit is used for writing the read attribute values into a result data stream of the data object according to the sequence of the attribute description sequence number, and the result data stream is used as a serialization result of the data object.

Optionally, the metadata description file of the data object to be serialized, which is acquired by the description file acquiring unit, further includes an attribute type.

Optionally, the attribute reading unit further includes: attribute type reading subunit, default attribute operation class obtaining subunit and default value reading subunit;

the attribute type reading subunit is configured to, after reading attribute values corresponding to the attribute descriptions in the data object to be serialized, obtain an attribute type of the attribute description that cannot be read because the attribute operation class corresponding to the attribute cannot be obtained;

the default attribute operation class obtaining subunit is configured to, according to the attribute type, retrieve and obtain a default attribute operation class corresponding to the attribute type;

and the default value reading subunit is used for endowing the attribute corresponding to the attribute description with a predetermined default value according to the default attribute operation class.

Optionally, the description file obtaining unit is specifically configured to search, according to the name of the data object to be serialized, a metadata description file of the data object to be serialized, which corresponds to the data object to be serialized and has a largest version number value, from a metadata repository.

Optionally, the serialized write unit further includes: writing the version number into the subunit;

and the version number writing subunit is configured to, before the read attribute values are written into the result data stream of the data object according to the sequence of the attribute description sequence numbers, read the version number of the metadata description file of the data object to be serialized, and write the version number into the header of the result data stream of the data object to be serialized.

Optionally, the apparatus for serializing data objects further includes: the device comprises a modification operation receiving unit, an operation class creating unit and a description file creating unit;

the modification operation receiving unit is used for receiving the modification operation of the attribute of the data object to be serialized before the metadata description file of the data object to be serialized is acquired; the modifying operation includes: adding, deleting or changing the attribute of the data object to be serialized;

the operation class creating unit is used for creating an attribute operation class for the newly added attribute of the data object to be serialized on the basis of the modification operation on the data object to be serialized;

and the description file creating unit is used for creating a new version of the metadata description file according to the modification operation, and the attribute description in the metadata description file is changed according to the modification operation on the basis of the attribute description in the original metadata description file.

Optionally, the apparatus for serializing data objects further includes: the device comprises a traversing unit, a numerical value acquiring unit, a version number generating unit and a storing unit;

the traversal unit is configured to create a new version of the metadata description file according to the modification operation, where an attribute description in the metadata description file is changed according to the modification operation on the basis of an attribute description in an original metadata description file, and traverse, according to a name of the data object to be serialized, the metadata description file set for the data object to be serialized in a metadata repository where the metadata description file is stored;

the numerical value obtaining unit is configured to obtain, in a traversal process, a numerical value with a maximum version number of the metadata description file of the data object to be serialized in the metadata repository;

the version number generation unit is used for adding one to the numerical value and taking the numerical value after adding one as the version number of the metadata description file of the new version;

the saving unit is configured to save the metadata description file recorded with the version number in the metadata repository.

Optionally, the apparatus for serializing data objects further includes: an attribute value compression unit;

and the attribute value compression unit is used for compressing the read attribute values after reading the attribute values corresponding to the attribute descriptions in the data objects to be serialized according to the reading method provided by the attribute operation class corresponding to each attribute.

In addition, an embodiment of the present application further provides an electronic device, where the electronic device includes:

a display;

a processor;

the storage is used for storing a serialization program, and the program executes the following operations when being read and executed by the processor, wherein the metadata description file of the data object to be serialized is obtained, and comprises attribute description marked by a sequence number and attribute operation class names corresponding to the attributes in the data object; obtaining attribute operation classes corresponding to the attributes according to the attribute operation class names provided by the attribute descriptions in the metadata description file; reading attribute values respectively corresponding to the attribute descriptions in the data objects to be serialized according to the reading method provided by the attribute operation class corresponding to each attribute; and writing the read attribute values into a result data stream of the data object according to the sequence of the attribute description sequence number, wherein the result data stream is used as a serialization result of the data object.

In addition, the present application also provides a method for deserializing a data stream, including:

obtaining a result data stream;

acquiring a metadata description file of the local data object, wherein the metadata description file comprises attribute description identified by a sequence number and an attribute operation class name corresponding to an attribute in the data object;

according to the reading method provided by the attribute operation class corresponding to each attribute, sequentially reading the attribute values in the result data stream according to the sequence of the attribute description sequence numbers;

and writing the read attribute values into the attributes of the local data object corresponding to each attribute description according to the writing method provided by the attribute operation class of each attribute description.

Optionally, the attribute description further includes: the attribute type.

Optionally, in the step of sequentially reading the attribute values in the result data stream according to the sequence of the attribute description sequence numbers by the write method provided by the attribute operation class described according to each attribute, the method further includes:

for the attribute description of the attribute operation class which cannot be obtained, obtaining a default attribute operation class corresponding to the attribute type according to the attribute type of the attribute description;

acquiring a reading method provided by the default attribute operation class, and reading data in the result data stream according to the sequence of the attribute description sequence number;

and discarding the read data in the result data stream.

Optionally, the obtaining the metadata description file of the local data object includes:

acquiring the name of the local data object;

analyzing the result data stream to obtain a first value in the result data stream;

traversing a metadata description file set for the local data object in a metadata warehouse in which the metadata description file exists according to the name of the local data object;

and in the traversing process, acquiring the metadata description file of the local data object, wherein the name of the local data object is the same, and the version number of the metadata description file is the same as the first value in the result data stream.

Optionally, after the step of sequentially reading the attribute values in the result data stream according to the sequence of the attribute description sequence numbers according to the reading method provided by the attribute operation class corresponding to each attribute, the method includes:

and decompressing the attribute values read from the result data stream.

Correspondingly, the present application also provides an apparatus for deserializing a data stream, including: the device comprises a data stream acquisition unit, a description file acquisition unit, an attribute operation class acquisition unit, an attribute value reading unit and an attribute value writing unit;

the data stream acquiring unit is used for acquiring a result data stream;

the description file obtaining unit is configured to obtain a metadata description file of the local data object, where the metadata description file includes an attribute description identified by a sequence number and an attribute operation class name corresponding to an attribute in the data object;

the attribute value reading unit is used for sequentially reading the attribute values in the result data stream according to the reading method provided by the attribute operation class corresponding to each attribute and the sequence of the attribute description sequence number;

and the attribute value writing unit is used for writing the read attribute values into the attributes of the local data object corresponding to the attribute descriptions according to the writing method provided by the attribute operation class of each attribute description.

Optionally, the description file obtaining unit obtains the metadata description file of the local data object, where the metadata description file further includes an attribute type.

Optionally, the attribute value reading unit includes: the attribute type acquisition subunit, the data reading subunit and the discarding subunit;

the attribute type obtaining subunit is configured to, in the writing method provided by the attribute operation class according to each attribute description, sequentially read the attribute values in the result data stream according to the sequence of the attribute description sequence numbers, and for an attribute description of an attribute operation class that cannot be obtained, obtain, according to an attribute type of the attribute description, a default attribute operation class corresponding to the attribute type;

the data reading subunit is configured to obtain a reading method provided by the default attribute operation class, and read data in the result data stream according to an order of the attribute description sequence numbers;

and the discarding unit sub-element is used for discarding the data in the read result data stream.

Optionally, the description file obtaining unit includes: the system comprises a local data object name acquisition subunit, a result data stream analysis subunit, a description file traversal subunit and a description file acquisition subunit;

the local data object name acquiring subunit is configured to acquire a name of the local data object;

the result data stream analyzing subunit is configured to analyze the result data stream to obtain a first value in the result data stream;

the description file traversing subunit is configured to traverse, according to the name of the local data object, a metadata description file set for the local data object in a metadata repository where the metadata description file is stored;

the description file obtaining subunit is configured to, during traversal, obtain a metadata description file of the local data object, where the metadata description file is the same as the name of the local data object and has a version number that is the same as the first value in the result data stream.

Optionally, the apparatus for deserializing a data stream further includes: an attribute value decompression unit;

the attribute value decompressing unit is configured to decompress the attribute values read from the result data stream after sequentially reading the attribute values in the result data stream according to the order of the attribute description sequence numbers according to the reading method provided by the attribute operation class corresponding to each attribute.

a display;

a processor;

a memory for storing an deserialization program that, when read by the processor for execution, the following operations are performed: obtaining a resultant data stream, obtaining a metadata description file of the local data object, the metadata description file contains attribute descriptions identified by sequence numbers and attribute operation class names corresponding to the attributes in the data object, obtaining the attribute operation class corresponding to each attribute according to the attribute operation class name provided by each attribute description in the metadata description file, reading the attribute values in the result data stream in sequence according to the sequence of the attribute description sequence numbers according to the reading method provided by the attribute operation class corresponding to each attribute, and writing the read attribute values into the attributes of the local data object corresponding to each attribute description according to the writing method provided by the attribute operation class of each attribute description.

In addition, the present application also provides a serialization and deserialization system comprising:

means for serializing data objects as described in any preceding claim; and any of the means for deserializing the data stream.

Compared with the prior art, the method and the device for serializing the data objects have the advantages that the metadata description file of the data objects to be serialized is obtained, the metadata description file comprises attribute descriptions marked by serial numbers, and each attribute description comprises the attribute operation class name of the corresponding data object attribute; according to the attribute operation class name, the attribute operation class corresponding to the attribute can be obtained, according to the reading method provided by the attribute operation class corresponding to each attribute, the attribute values corresponding to each attribute description in the data object to be serialized can be read, the read attribute values are written into the result data stream of the data object according to the sequence of the attribute description sequence number, and the result data stream is used as the serialization result of the data object.

The method and the device for deserializing the data stream acquire a metadata description file of the local data object by acquiring a result data stream, wherein the metadata description file comprises attribute descriptions identified by sequence numbers and attribute operation class names corresponding to attributes in the data object, acquire the attribute operation classes corresponding to the attributes according to the attribute operation class names provided by the attribute descriptions in the metadata description file, sequentially read the attribute values in the result data stream according to the sequence of the attribute description sequence numbers according to the reading method provided by the attribute operation classes corresponding to the attributes, and write the read attribute values into the attributes of the local data object corresponding to the attribute descriptions according to the writing method provided by the attribute operation classes described by the attributes.

The technical scheme solves the compatibility problem of serialization and deserialization between systems after the attribute of the object is changed, saves the storage space and the transmission flow when information and data are stored and data are transmitted by thoroughly splitting the metadata of the object and the value of the object, and reduces the processing overhead of the system when the object is deserialized.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art according to the drawings.

FIG. 1 illustrates a flow diagram of a method of serializing data objects provided in accordance with an embodiment of the present application;

FIG. 2 illustrates a flow diagram provided in accordance with an embodiment of the present application for creating a metadata description file;

FIG. 3 illustrates a flow diagram for saving a metadata description file provided in accordance with an embodiment of the present application;

fig. 4 shows a flowchart of a method for reading default values upon serialization provided according to an embodiment of the present application.

FIG. 5 shows a schematic diagram of an apparatus for serializing data objects provided in accordance with an embodiment of the present application;

FIG. 6 shows a schematic view of an electronic device provided in accordance with an embodiment of the present application;

FIG. 7 illustrates a flow diagram of a method of deserializing a data stream provided in accordance with an embodiment of the present application;

FIG. 8 is a flowchart illustrating a method for obtaining a metadata description file of the local data object according to an embodiment of the present application;

FIG. 9 illustrates a flow chart of a default value discarding method upon deserialization provided in accordance with an embodiment of the present application;

FIG. 10 shows a schematic diagram of an apparatus for deserializing a data stream provided in accordance with an embodiment of the present application;

FIG. 11 shows a schematic view of an electronic device provided in accordance with an embodiment of the present application;

FIG. 12 shows a schematic diagram of a serialization and deserialization system provided in accordance with an embodiment of the present application.

Detailed Description

In order that the above objects, features and advantages of the present application can be more clearly understood, the present application will be described in further detail with reference to the accompanying drawings and detailed description. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of implementation in many different ways than those herein set forth and of similar import by those skilled in the art without departing from the spirit and scope of this application, and it is therefore not limited to the specific implementations disclosed below.

The embodiment of the application provides a method for serializing a data object, a device for serializing the data object, a method for deserializing a data stream, a device for deserializing the data stream and a serialization and deserialization system. Details are described in the following examples one by one.

The technical scheme of the application is provided for data transmission or storage between systems in a distributed system environment, but the application field is not limited to the distributed system, and the method provided by the application can be adopted in the existing system or application program as long as the method can execute serialization and deserialization, and the corresponding beneficial effects are obtained.

Before describing the specific steps of this embodiment in detail, the serialization method and the deserialization method according to this embodiment will be briefly described.

Object Serialization (Serialization) is a technique that converts object data into another form of persistent data, i.e., the process of converting the state information of an object into a form that can be stored or transmitted. During serialization, an object writes its current state to a temporary or persistent store, and the object may be recreated by reading or deserializing the state of the object from the store. Such a conversion must be reversible in order to enable the stored and transferred data to be transformed back to the object in memory again. The serialization method and the deserialization method are widely used in data transmission and application between networks, and generally information needs to be converted into data streams to realize the data transmission and application. However, in data transmission and storage, if the attribute in the object is increased or decreased, the deserializing side fails to synchronize the attributes in the object. The attributes in the object are added, reduced or modified, and errors or serialization failures may occur when deserializing the original serialized file or data. This is a compatibility problem for object serialization. Typically all fields of an object will be serialized, meaning that the data will be represented as serialized data. Similarly, deserialization extracts data from the serialized representation and sets the object state directly.

Embodiments of the present application provide a method of serializing data objects. An example of the method for serializing data objects is as follows:

referring to fig. 1, a flow chart of a method for serializing data objects provided in accordance with an embodiment of the present application is shown.

The method of serializing data objects comprises:

step S101, obtaining a metadata description file of a data object to be serialized, wherein the metadata description file comprises attribute description marked by a sequence number and an attribute operation class name corresponding to an attribute in the data object.

In this embodiment, the obtaining of the metadata description file of the data object to be serialized can be implemented by the following method: and searching a metadata description file of the data object to be serialized, which corresponds to the data object to be serialized and has the maximum version number value, from a metadata warehouse according to the name of the data object to be serialized, wherein the metadata description file comprises attribute description identified by a sequence number and an attribute operation class name corresponding to an attribute in the data object.

It should be noted that the metadata repository is a database for storing metadata and information related to the metadata (such as a metadata description file), and the metadata repository may generally adopt a relational database.

In this embodiment, the method for serializing data objects is to understand the structural composition of data according to the description semantics of the metadata description file. It can be understood that, when the metadata description file of the data object to be serialized is obtained in step S101, the metadata description file of the data object to be serialized, which corresponds to the data object to be serialized and has the largest version number value, is searched from the metadata repository, that is, the metadata description file obtained in this step is the latest version.

Specifically, the metadata description file includes a plurality of attribute descriptions corresponding to attributes in the data object, where each attribute description is identified by a sequence number and includes an attribute operation class name corresponding to an attribute in the data object.

In order to satisfy the problem of compatibility between attributes in a metadata description file and a data object during serialization, the technical solution of this embodiment provides a preferred implementation manner, where in the preferred implementation manner, each attribute description in the metadata description file further includes: the attribute type corresponding to the attribute description of the bar.

For example, the metadata description file may be as follows:

1:com.taobao.galaxy.io.UserIdFieldDef:VARINT

2:com.taobao.galaxy.io.AgeFieldDef:VARINT

wherein, the numbers "1" and "2" are the serial number of each attribute description, and UserIdFieldDef is the attribute operation class name corresponding to the attribute UserId in the data object; AgeFieldDef is the name of the attribute operation class corresponding to the attribute Age in the data object; VARINT is the attribute type described by this piece of attribute.

It should be noted that the metadata description file is data describing other data, or structural data. Specifically, the metadata description file is data describing an object such as an information resource or data. The metadata description file is shareable for use once created. The development and utilization environment of the metadata description file is often a varied distributed environment. Since the metadata description file is also data, the metadata description file can be stored and acquired in a metadata warehouse by a data-like method, and sharing and use in a distributed environment are facilitated.

In the following, a specific example is provided for the method of serializing data objects, so as to explain the method of serializing data objects:

the data objects to be serialized are (their attribute values are not shown):

according to the name User of the data object to be serialized, searching a metadata description file of the data object to be serialized, which corresponds to the data object User to be serialized and has the largest version number value, from a metadata warehouse, for example: the obtained metadata description file of the data object to be serialized is as follows:

1:com.taobao.test.io.UserIdFieldDef:VARINT

2:com.taobao.test.io.NameFieldDef:VARCHAR

3:com.taobao.test.io.PhoneFieldDef:VARINT

4:com.taobao.test.io.AgeFieldDef:VARINT

the data object described by the metadata description file does not completely correspond to the example User of the data object, since the metadata description file is the latest version and is determined according to the User of the upgraded version.

The data object is a mapping of entities such as people, things, and objects existing in the objective world in computer logic. For example, when the object is a book, the attribute of the book includes a book number, an author, a publisher, a remark, and the like, and the book is serialized, a binary data stream corresponding to the book is generated, and the binary data stream has an attribute value corresponding to the attribute of the book, and represents an entity such as a person, an event, or an object existing in the objective world in a computer language. The data object may be any entity, such as a person, thing, object, etc., that exists in the objective world, in this example the User, i.e., User. The object includes at least one attribute, a name of the attribute is referred to as an attribute name, a value corresponding to the attribute is referred to as an attribute value, and the attribute value may be an integer, a character, a floating point, a string, or other basic type.

Before the metadata description file of the data object to be serialized is acquired in step S101, the metadata description file needs to be defined or created, and specifically includes steps S100-1 to S100-3, which will be further described with reference to fig. 2.

Referring to fig. 2, a flowchart for creating a metadata description file according to an embodiment of the present application is shown.

S100-1, receiving modification operation on the attribute of the data object to be serialized; the modifying operation includes: and adding, deleting or changing the attribute of the data object to be serialized.

The receiving of the modification operation on the attribute of the data object to be serialized comprises: receiving addition, deletion or change of the attribute in the data object to be serialized. For example, add an attribute address with type string to the data object User, delete an age attribute in the data object User, or modify the name of a userId attribute in the data object to id.

And S100-2, creating an attribute operation class for the newly added attribute of the data object to be serialized on the basis of the modification operation on the data object to be serialized.

This step creates an attribute operation class for the added attribute according to the modification operation of the attribute of the data object to be serialized received in step S100-1. For example, if the attribute address of type string is added in step S100-1, an attribute operation class named address fielddef is created for the attribute address. Wherein, the addressFieldDef class encapsulates the getter and the setter method.

It should be noted that, if the modification operation is to delete an attribute, the attribute operation class corresponding to the attribute may not be deleted; if the modification operation is to modify the attribute, the modified attribute is created after the original attribute is deleted, and a corresponding attribute operation class is created for the modified attribute.

And step S100-3, creating a new version of the metadata description file according to the modification operation, wherein the attribute description in the metadata description file is changed according to the modification operation on the basis of the attribute description in the original metadata description file.

In this embodiment, the creating of the new version of the metadata description file according to the modification operation, where the attribute description in the metadata description file is changed according to the modification operation on the basis of the attribute description in the original metadata description file, may be implemented as follows: obtaining the attribute of the data object to be serialized after receiving the modification operation, generating a serial number of attribute description according to the sequence defined by the attribute in the data object, obtaining the attribute operation class name corresponding to the attribute in the data object to be serialized, obtaining the attribute type of the attribute in the data object to be serialized, writing the attribute operation class name and the attribute type into each attribute description according to the sequence defined by the attribute in the data object, and taking all the generated attribute descriptions as a metadata description file of a new version.

After the new version of the metadata description file is created according to the modification operation, and the property description in the metadata description file is changed according to the modification operation based on the property description in the original metadata description file, in order to distinguish the new version of the metadata description file from the old version of the metadata description file and enable the new version of the metadata description file to be shared and used in a distributed environment, the technical solution of this embodiment provides a preferred embodiment, and specifically includes steps S100-4 to S100-7 in the preferred embodiment, which is further described below with reference to fig. 3.

Referring to fig. 3, a flowchart for saving a metadata description file according to an embodiment of the present application is shown.

And S100-4, traversing the metadata description file set for the data object to be serialized in a metadata warehouse in which the metadata description file exists according to the name of the data object to be serialized.

For example, the name of the data object to be serialized is User, and a metadata description file set for the data object User is traversed in a metadata warehouse in which the metadata description file exists.

Step S100-5, in the traversal process, obtaining the numerical value with the maximum version number of the metadata description file of the data object to be serialized in the metadata warehouse.

For example, in the traversal process, if the number of the metadata description files set for the data object User in the metadata repository is found to be 6, and the version number is 1 to 6, the value obtained in step S100 to 5 is 6.

And S100-6, adding one to the numerical value, and taking the numerical value after the addition as the version number of the metadata description file of the new version.

In this embodiment, the incremented value is mapped to the new version of the metadata description file created in step S100-3.

And S100-7, storing the metadata description file recorded with the version number in the metadata warehouse.

It should be noted that, the seven steps from step S100-1 to step S100-7 may be created in advance, or may be configured on site when the method for serializing data objects is executed, which is not limited to this.

Step S103, according to the attribute operation class name provided by each attribute description in the metadata description file, acquiring the attribute operation class corresponding to each attribute.

In this embodiment, the obtaining of the attribute operation class corresponding to each attribute according to the attribute operation class name provided by each attribute description in the metadata description file may be implemented by adopting the following manner: and retrieving and acquiring the attribute operation class with the same name as the attribute operation class provided by each attribute description according to the attribute operation class name provided by each attribute description in the metadata description file.

The following still uses the above specific example of the method for serializing data objects to further explain the processing procedure of this step.

The data objects to be serialized are (their attribute values are not shown):

the metadata description file of the data object to be serialized is:

1:com.taobao.test.io.UserIdFieldDef:VARINT

2:com.taobao.test.io.NameFieldDef:VARINT

3:com.taobao.test.io.PhoneFieldDef:VARINT

4:com.taobao.test.io.AgeFieldDef:VARINT

obtaining attribute operation class names provided by each attribute description from the metadata description file of the data object to be serialized, wherein the attribute operation class names include useridfieldedf, namefieldef, PhoneFieldDef and agefieldef, and retrieving and obtaining attribute operation classes which are the same as the attribute operation class names provided by each attribute description, and in this example, the obtained attribute operation classes include: UserIdFieldDef, NameFieldDef, and AgeFieldDef.

Step S105, reading the attribute values corresponding to the attribute descriptions in the data object to be serialized according to the reading method provided by the attribute operation class corresponding to each attribute.

In this embodiment, according to the reading method provided by the attribute operation class corresponding to each attribute in the attribute operation classes corresponding to each attribute acquired in step S103, the attribute values corresponding to each attribute description in the data object to be serialized are read.

During specific implementation, a getter method of attribute operation class encapsulation with the same name as the attribute operation class provided by each attribute description is called, and the attribute values of the corresponding attributes of the data object to be serialized are read.

In order to save storage space or reduce data transmission amount when writing the result data stream of the data object for reading the attribute values in the data object to be serialized, the technical solution of this embodiment provides a preferred embodiment, and in the preferred embodiment, the read attribute values are compressed before writing the result data stream of the data object.

In particular implementations, the attribute values may be compressed using out Varint () or using, for example, the more popular LZ and series variant (LZW, LZR, LZX) compression algorithms.

It should be noted that, in step S103, when the attribute operation class corresponding to each attribute is acquired according to the attribute operation class name provided by each attribute description in the metadata description file, the following 2 situations may occur:

in the first case, the same number of attribute operation classes as the name of the attribute operation class provided by each attribute description in the metadata description file can be obtained in the code library, which indicates that the data object to be serialized matches with the version of the metadata description file, that is: the data object to be serialized is compatible with the metadata description file version, so that the compatibility problem cannot occur.

It should be noted that, in the first case, it is possible that the number and name of the attribute operation classes in the code library are the same as the attribute operation classes described by the respective attribute descriptions in the metadata description file; there is also a possibility that the number of attribute operation classes in the code library is greater than the attribute operation classes described by each attribute description in the metadata description file, but the code library contains all the attribute operation classes described by each attribute description in the metadata description file, and during serialization, the attribute operation classes corresponding to the names of the attribute operation classes in the metadata description file are loaded according to the metadata description file, and the attribute operation classes that are not used in the code library are not used, so that the compatibility problem does not occur.

In the second case, only a part of attribute operation classes with the same name as the attribute operation class provided by each attribute description in the metadata description file can be acquired in the code library, which indicates that the data object to be serialized is not matched with the metadata description file version, that is: the data objects are generated under the old version, and the metadata description file is the latest version, so that compatibility problems can occur during serialization.

In order to solve the problem of compatibility between the metadata description file and the data object during serialization, the technical solution of this embodiment provides a preferred embodiment, and specifically includes steps S106-1 to S106-3 in the preferred embodiment, which is further described below with reference to fig. 4.

Referring to fig. 4, a flowchart of a method for reading default values during serialization is provided according to an embodiment of the present application.

Step S106-1, the attribute type of the attribute description which cannot be read because the attribute operation class corresponding to the attribute cannot be obtained is obtained.

The obtaining of the attribute type of the attribute description that cannot be read because the attribute operation class corresponding to the attribute cannot be obtained may be implemented in the following manner: from the metadata description file, determining an attribute description for which the attribute value cannot be read because the corresponding attribute operation class cannot be obtained, that is: and the attribute operation class described by the attribute description does not exist, and the attribute type described by the attribute is obtained.

It should be noted that the reason why the attribute value cannot be read because the corresponding attribute operation class cannot be obtained is that the attribute operation class is not stored because the code library is not updated or for other reasons. Since the code base is generally stored locally in various terminals, it is likely that the code base is not updated in time according to the latest version; the used metadata description files are all latest update versions.

And S106-2, retrieving and obtaining a default attribute operation class corresponding to the attribute type according to the attribute type.

In this embodiment, the attribute types include: integer, character, floating point, string, etc.

For example, the type of the obtained attribute is VARINT, and the type of the retrieved default attribute operation class is DefaultVarintFieldDef.

And step S106-3, according to the default attribute operation class, giving a predetermined default value to the attribute corresponding to the attribute description.

And loading the retrieved default attribute operation class, calling a getter method packaged in the default attribute operation class, and reading a default value given in advance.

The predetermined default value may be different according to different attribute types, for example, the default value of the integer type is 0.

The data objects to be serialized are:

the specific attribute value of the data object is stored in a corresponding database table, for example, a specific data object User is:

userId	name	age
			1	aaa	20

it should be noted that the above-mentioned User table is only illustrative, and in a specific implementation, the attribute data in the object User is usually more, and only one set of data is given in this example to illustrate the processing procedure of this step.

The obtained metadata description file of the data object to be serialized is as follows:

1:com.taobao.test.io.UserIdFieldDef:VARINT

2:com.taobao.test.io.NameFieldDef:VARCHART

3:com.taobao.test.io.PhoneFieldDef:VARINT

4:com.taobao.test.io.AgeFieldDef:VARINT

in step S103, the acquired attribute operation class includes: UserIdFieldDef, NameFieldDef, and AgeFieldDef, wherein since the third attribute description does not obtain the attribute operation class with the same name as PhoneFieldDef, the attribute type in the third attribute description is obtained, that is: the method comprises the steps of obtaining a VARINT type, retrieving a default attribute operation class DefaultVarintFieldDef corresponding to the VARINT type through the obtained VARINT type, and after obtaining the attribute operation class DefaultVarintFieldDef, respectively executing a getter method in the UserIdFieldDef to read an attribute value 1 of a userId, executing a getter method in the NameFieldDef to read an attribute value aaa of a name, executing a getter method in the DefaultVarintFieldDef to read a default value 0, and executing a getter method in the AgeFieldDef to read an attribute value 20 of a age according to the sequence of the attribute description in the metadata description file.

Taking UserIdFieldDef and DefaultVarintFieldDef as examples, detailed code examples are shown below: UserIdFieldDef:

when different attribute operation classes are executed, only the corresponding parameters need to be modified, which is not described in detail herein.

And step S107, writing the read attribute values into a result data stream of the data object according to the sequence of the attribute description sequence number, wherein the result data stream is used as a serialization result of the data object.

In order to obtain the metadata description file used in serialization when the result data stream is subjected to deserialization later and ensure compatibility, the method comprises the following steps before the step of writing the obtained attribute values into the result data stream of the data object according to the sequence of the attribute description sequence numbers:

In specific implementation, the version number of the metadata description file of the data object to be serialized is obtained first, the numerical value of the version number is written into the position of the first byte of the result data stream, the serialization pointer is correspondingly moved backwards after writing, and the attribute values read in step S105 are written into the result data stream of the data object according to the sequence of the attribute description sequence numbers.

It should be noted that the serialized resultant data stream may be binary data, but is not limited to binary data.

According to the method for serializing the data objects, the metadata description file is obtained, and the data objects are serialized according to the metadata description file; the generated result data stream does not contain additional data except data, so that the storage space and the transmission flow are saved during information storage and data transmission, and the processing overhead of the system can be reduced during deserialization.

In the above embodiments, a method for serializing a data object is provided, and the present application also provides an apparatus for serializing a data object, corresponding to the above method for serializing a data object. Since the embodiments of the apparatus are substantially similar to the embodiments of the method, they have been described in a relatively simple manner, and reference may be made to the description of the embodiments of the method for relevant points. The device embodiments described below are merely illustrative. An embodiment of the apparatus for serializing data objects is as follows:

referring to fig. 5, a schematic diagram of an apparatus for serializing data objects according to an embodiment of the present application is shown.

The apparatus for serializing data objects, comprising: a description file acquisition unit 501, an attribute operation class acquisition unit 503, an attribute reading unit 505, and a serialization writing unit 507;

the description file obtaining unit 501 is configured to obtain a metadata description file of a data object to be serialized, where the metadata description file includes an attribute description identified by a sequence number and an attribute operation class name corresponding to an attribute in the data object;

the attribute operation class obtaining unit 503 is configured to obtain an attribute operation class corresponding to each attribute according to an attribute operation class name provided by each attribute description in the metadata description file;

the attribute reading unit 505 is configured to read attribute values corresponding to the attribute descriptions in the data object to be serialized according to the reading method provided by the attribute operation class corresponding to each attribute;

the serialization write-in unit 507 is configured to write the read attribute values into a result data stream of the data object according to the sequence of the attribute description sequence numbers, where the result data stream is used as a serialization result of the data object.

Optionally, the metadata description file of the data object to be serialized, acquired by the description file acquiring unit 501, further includes an attribute type.

Optionally, the attribute reading unit 505 further includes: attribute type reading subunit, default attribute operation class obtaining subunit and default value reading subunit;

Optionally, the description file obtaining unit 501 is specifically configured to search, according to the name of the data object to be serialized, a metadata description file of the data object to be serialized, which corresponds to the data object to be serialized and has the largest version number value, from a metadata repository.

Optionally, the serialized writing unit 507 further includes: writing the version number into the subunit;

the description file creating unit is used for creating a new version of the metadata description file according to the modification operation, and the attribute description in the metadata description file is changed according to the modification operation on the basis of the attribute description in the original metadata description file

In the above embodiments, a method for serializing data objects and an apparatus for serializing data objects are provided, and in addition, an electronic device is provided; the electronic equipment comprises the following embodiments:

referring to fig. 6, a schematic diagram of an electronic device provided according to an embodiment of the present application is shown.

The electronic device includes: a display 601; a processor 603; a memory 605;

the memory 605 is used for storing a serialization program, and when the program is read and executed by the processor, the program executes the following operations of obtaining a metadata description file of a data object to be serialized, wherein the metadata description file comprises attribute description marked by a sequence number and an attribute operation class name corresponding to an attribute in the data object; obtaining attribute operation classes corresponding to the attributes according to the attribute operation class names provided by the attribute descriptions in the metadata description file; reading attribute values respectively corresponding to the attribute descriptions in the data objects to be serialized according to the reading method provided by the attribute operation class corresponding to each attribute; and writing the read attribute values into a result data stream of the data object according to the sequence of the attribute description sequence number, wherein the result data stream is used as a serialization result of the data object.

In the above embodiments, a method for serializing a data object, an apparatus for serializing a data object, and an electronic device are provided, and in addition, a method for deserializing a data stream is provided; the embodiment of the method for deserializing the data stream is as follows:

referring to fig. 7, shown is a flow chart of a method of deserializing a data stream provided in accordance with an embodiment of the present application.

The method for deserializing a data stream includes:

step S701, a result data stream is acquired.

In this embodiment, the obtaining the result data stream includes: the resulting data stream is obtained by network transmission or compressed locally.

It should be noted that the result data stream may be binary data, but is not limited to binary data.

Step S703 is to obtain a metadata description file of the local data object, where the metadata description file includes attribute descriptions identified by sequence numbers and attribute operation class names corresponding to the attributes in the data object.

In this embodiment, the obtaining of the metadata description file of the local data object, where the metadata description file includes an attribute description identified by a sequence number and an attribute operation class name corresponding to an attribute in the data object, specifically includes steps S703-1 to S703-4, which are further described below with reference to fig. 8.

Referring to fig. 8, a flowchart for obtaining a metadata description file of the local data object is shown according to an embodiment of the present application.

Step S703-1, the name of the local data object is acquired.

For example, if the name of the local data object is User, the obtained name of the local data object is User.

Step S703-2, analyzing the result data stream to obtain a first value in the result data stream.

And analyzing the result data stream to obtain a first byte in the result data stream.

For example, the obtained value is 6.

Step S703-3, traversing the metadata description file set for the local data object in the metadata repository in which the metadata description file is stored, according to the name of the local data object.

For example, according to the local data object User, a metadata description file set for the local data object User is traversed in a metadata repository where the metadata description file exists.

Step S703-4, in the traversal process, obtaining the metadata description file of the local data object, which has the same name as the local data object and the same version number as the first value in the result data stream.

For example, in the traversal process, if it is found that there are 6 metadata description files set for the data object User in the metadata repository, and the version number is 1 to 6, the metadata description file with the version number of 6 is obtained in this step S603-4.

In order to satisfy the problem of compatibility between attributes in a metadata description file and a local data object during deserialization, the technical solution of this embodiment provides a preferred implementation manner, where in the preferred implementation manner, each attribute description in the metadata description file further includes: the attribute type corresponding to the attribute description of the bar.

With respect to the method for deserializing a data stream, a specific example is provided to facilitate the description of the method for deserializing a data stream:

the local data object to be written with the attribute value is:

according to the local data object User, traversing a metadata description file set for the local data object User in a metadata warehouse in which the metadata description file exists, and if a first numerical value in the result data stream is 6, acquiring the metadata description file with a version number of 6. For example: the obtained metadata description file of the local data object is as follows:

1:com.taobao.test.io.UserIdFieldDef:VARINT

2:com.taobao.test.io.NameFieldDef:VARCHART

3:com.taobao.test.io.PhoneFieldDef:VARINT

4:com.taobao.test.io.AgeFieldDef:VARINT

it should be noted that, since the local data object is determined according to the value written in the header of the result data stream during serialization when the metadata description file is obtained, the metadata description file used during serialization and deserialization is the same, that is, the metadata description file is compatible.

Step S705, obtaining the attribute operation class corresponding to each attribute according to the attribute operation class name provided by each attribute description in the metadata description file.

The following will further explain the processing procedure of this step along with the above-mentioned specific example of the method for deserializing the data stream.

The local data object to be written with the attribute value is:

the metadata description file of the local data object is:

1:com.taobao.test.io.UserIdFieldDef:VARINT

2:com.taobao.test.io.NameFieldDef:VARINT

3:com.taobao.test.io.PhoneFieldDef:VARINT

4:com.taobao.test.io.AgeFieldDef:VARINT

obtaining attribute operation class names provided by each attribute description from the metadata description file of the local data object, including useridfieldedf, namefieldef, PhoneFieldDef, and agefieldef, and retrieving and obtaining the attribute operation classes with the same attribute operation class names provided by each attribute description, where in this example, the obtained attribute operation classes include: UserIdFieldDef, NameFieldDef, and PhoneFieldDef.

Step S707, sequentially reading the attribute values in the result data stream according to the reading method provided by the attribute operation class corresponding to each attribute in the order of the attribute description sequence number.

In this embodiment, according to the reading method provided by the attribute operation class described by each attribute in the attribute operation classes corresponding to each attribute acquired in step S705, the attribute values in the result data stream are sequentially read according to the sequence of the attribute description sequence numbers.

In order to ensure the integrity of data, the technical solution of this embodiment provides a preferred implementation manner, and in the preferred implementation manner, after reading out the attribute value from the result data stream, the attribute value read out from the result data stream is decompressed.

In specific implementation, an in.readvarint () method of the attribute operation class described by each attribute is called, the attribute values of the corresponding attributes of the serialized data objects are read, and the read attribute values are decompressed.

It should be noted that, in step S705, when the attribute operation class corresponding to each attribute is acquired according to the attribute operation class name provided by each attribute description in the metadata description file, the following 2 cases may occur:

in the first case, the same number of attribute operation classes as the name of the attribute operation class provided by each attribute description in the metadata description file can be obtained in the code library, which indicates that the local data object matches with the version of the metadata description file, that is: the local data object is compatible with the metadata description file version, so that the compatibility problem cannot occur.

It should be noted that, in the first case, it is possible that the number and name of the attribute operation classes in the code library are the same as the attribute operation classes described by the respective attribute descriptions in the metadata description file; there is also a possibility that the number of attribute operation classes in the code library is greater than the attribute operation classes described by each attribute description in the metadata description file, but the attribute operation classes in the code library all contain the attribute operation classes described by each attribute description in the metadata description file, but since the attribute operation classes corresponding to the names of the attribute operation classes in the metadata description file are loaded according to the metadata description file during serialization, no compatibility problem occurs.

In the second case, only some attribute operation classes with the same name as the attribute operation class provided by each attribute description in the metadata description file can be obtained in the code library, which indicates that the local data object does not match the metadata description file version, that is: the metadata description file is a newer version, and compatibility problems can occur during serialization.

In order to satisfy the compatibility problem between the metadata description file and the local data object during deserialization, the technical solution of this embodiment provides a preferred embodiment, and specifically includes steps S707-1 to S707-3 in the preferred embodiment, which is further described below with reference to fig. 9.

Referring to fig. 9, a flowchart of a default value discarding method in deserialization provided according to an embodiment of the present application is shown.

And step S707-1, for the attribute description of the attribute operation class which cannot be obtained, according to the attribute type of the attribute description, obtaining a default attribute operation class corresponding to the attribute type.

For the attribute description of the unavailable attribute operation class, obtaining the default attribute operation class corresponding to the attribute type according to the attribute type of the attribute description, which can be realized by adopting the following mode: determining that the attribute description of the attribute operation class cannot be obtained from the metadata description file, namely: the attribute operation class described by the attribute description does not exist, the attribute corresponding to the attribute operation class does not exist in the local data object, the attribute type described by the attribute is obtained, and the default attribute operation class corresponding to the attribute type is obtained according to the attribute type described by the attribute.

For example: the acquired attribute type is VARINT, and the acquired type is a default attribute operation class of DefaultVarintFieldDef.

And step S707-2, acquiring the reading method provided by the default attribute operation class, and reading the data in the result data stream according to the sequence of the attribute description sequence numbers.

And loading the acquired default attribute operation class, calling a setter method packaged in the default attribute operation class, and reading the attribute value in the result data stream.

And step S707-3, discarding the data in the read result data stream.

It should be noted that, the discarding the data in the read result data stream specifically means: after reading out the attribute values in the result data stream, no writing is performed, that is: the attribute value is not written to the local data object.

In this step S707, each time an attribute value is read from the result data stream, the deserializing pointer is shifted back to the next attribute value.

The local data object to be written with the attribute value is:

the metadata description file of the local data object is:

1:com.taobao.test.io.UserIdFieldDef:VARINT

2:com.taobao.test.io.NameFieldDef:VARINT

3:com.taobao.test.io.PhoneFieldDef:VARINT

4:com.taobao.test.io.AgeFieldDef:VARINT

in step S705, the acquired attribute operation class includes: UserIdFieldDef, NameFieldDef, and PhoneFieldDef, wherein since the fourth attribute description does not obtain the attribute operation class with the same name as the yet AgeFieldDef, the attribute type in the fourth attribute description is obtained, that is: obtaining VARINT type, searching default attribute operation class DefaultVarintFieldDef corresponding to the type VARINT through the obtained VARINT type, respectively executing an in.readVarint () method in UserIdField Def to read data 1 at the corresponding position of the result data stream after obtaining the attribute operation class DefaultVarintFieldDef, decompressing the data, correspondingly moving an anti-serialization pointer to the position of the next attribute value, executing an in.readVarint () method in NameField Def to read data aaa at the corresponding position of the result data stream, decompressing the data, correspondingly moving the anti-serialization pointer to the position of the next attribute value, executing an in.readVarint () method in PhefFieldDef to read data 0 at the corresponding position of the result data stream, decompressing the data, and correspondingly moving the anti-serialization pointer to the position of the next attribute value, executing an in.readvadrinrat () method in PhefFieldDef to read the corresponding position of the result data stream, and executing an anti-serialization pointer to read data. The attribute value is decompressed and discarded 20.

Step S709, according to the writing method provided by the attribute operation class described in each attribute description, writing the read attribute value into the attribute of the local data object corresponding to each attribute description.

In specific implementation, after decompressing the attribute values in the result data stream read by the in.readvarint () method in step S707, the read attribute values are written into the attributes of the corresponding local data object by the setter method according to the attribute description sequence numbers.

For example, executing user.setuserid (in.readvarint ()) in UserIdFieldDef writes the attribute value read from the result data stream into the attribute of the local data object corresponding to userId

In the above embodiments, a method for deserializing a data stream is provided, and the present application also provides an apparatus for deserializing a data stream corresponding to the above method for deserializing a data stream. Since the embodiments of the apparatus are substantially similar to the embodiments of the method, they have been described in a relatively simple manner, and reference may be made to the description of the embodiments of the method for relevant points. The device embodiments described below are merely illustrative. An embodiment of the apparatus for deserializing a data stream is as follows:

referring to fig. 10, a schematic diagram of an apparatus for deserializing a data stream is provided according to an embodiment of the present application.

The apparatus for deserializing a data stream comprising: a data stream acquisition unit 1001, a description file acquisition unit 1003, an attribute operation class acquisition unit 1005, an attribute value reading unit 1007, and an attribute value writing unit 1009;

the data stream obtaining unit 1001 is configured to obtain a result data stream;

the description file obtaining unit 1003 is configured to obtain a metadata description file of the local data object, where the metadata description file includes an attribute description identified by a sequence number and an attribute operation class name corresponding to an attribute in the data object;

the attribute operation class obtaining unit 1005 is configured to obtain an attribute operation class corresponding to each attribute according to an attribute operation class name provided by each attribute description in the metadata description file;

the attribute value reading unit 1007 is configured to sequentially read the attribute values in the result data stream according to the reading method provided by the attribute operation class corresponding to each attribute, and according to the sequence of the attribute description sequence numbers;

the attribute value writing unit 1009 is configured to write the read attribute values into the attributes of the local data object corresponding to each attribute description according to the writing method provided by the attribute operation class described in each attribute description.

Optionally, the description file obtaining unit 1003 further obtains the metadata description file of the local data object, where the metadata description file further includes an attribute type.

Optionally, the attribute value reading unit 1007 includes: the attribute type acquisition subunit, the data reading subunit and the discarding subunit;

Optionally, the description file obtaining unit 1003 includes: the system comprises a local data object name acquisition subunit, a result data stream analysis subunit, a description file traversal subunit and a description file acquisition subunit;

In the above embodiments, a method for serializing a data object, an apparatus for serializing a data object, an electronic device, a method for deserializing a data stream, and an apparatus for deserializing a data stream are provided, and in addition, another electronic device is provided; the electronic equipment comprises the following embodiments:

referring to fig. 11, a schematic diagram of an electronic device provided according to an embodiment of the application is shown.

The electronic device includes: a display 1101; a processor 1103; a memory 1105;

the memory 1105 for storing an deserialization program that, when read and executed by the processor, the following operations are performed: obtaining a resultant data stream, obtaining a metadata description file of the local data object, the metadata description file contains attribute descriptions identified by sequence numbers and attribute operation class names corresponding to the attributes in the data object, obtaining the attribute operation class corresponding to each attribute according to the attribute operation class name provided by each attribute description in the metadata description file, reading the attribute values in the result data stream in sequence according to the sequence of the attribute description sequence numbers according to the reading method provided by the attribute operation class corresponding to each attribute, and writing the read attribute values into the attributes of the local data object corresponding to each attribute description according to the writing method provided by the attribute operation class of each attribute description.

In the above embodiments, a method for serializing a data object, an apparatus for serializing a data object, an electronic device, a method for deserializing a data stream, an apparatus for deserializing a data stream, and another electronic device are provided, and in addition, a serialization and deserialization system is also provided; the embodiments of the serialization and deserialization system are as follows:

referring to fig. 12, a schematic diagram of a serialization and deserialization system provided in accordance with an embodiment of the present application is shown.

The serialization and deserialization system comprises: means 1201 for serializing data objects and means 1203 for deserializing data streams;

the device 1201 for serializing data objects is configured to obtain a metadata description file, obtain attribute operation classes corresponding to respective attributes according to the metadata description file, read attribute values corresponding to respective attribute descriptions in the data objects to be serialized, and write the read attribute values into a result data stream of the data objects according to an order of attribute description sequence numbers;

the device 1203 for deserializing the data stream is configured to obtain a result data stream, obtain a metadata description file, obtain an attribute operation class corresponding to each attribute according to the metadata description file, sequentially read attribute values in the result data stream according to an order of attribute description sequence numbers, and write the read attribute values into attributes of the local data object corresponding to each attribute description.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

1. Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.

2. As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Although the present application has been described with reference to the preferred embodiments, it is not intended to limit the present application, and those skilled in the art can make variations and modifications without departing from the spirit and scope of the present application, therefore, the scope of the present application should be determined by the claims that follow.

Claims

1. A method of serializing a data object, comprising:

acquiring a metadata description file of a data object to be serialized, wherein the metadata description file comprises attribute descriptions marked by sequence numbers and attribute operation class names corresponding to attributes in the data object, and the attribute descriptions comprise the attribute operation class names;

reading attribute values respectively corresponding to the attribute descriptions in the data objects to be serialized according to the reading method provided by the attribute operation class corresponding to each attribute; and writing the read attribute values into a result data stream of the data object according to the sequence of the attribute description sequence number, wherein the result data stream is used as a serialization result of the data object.

2. The method of serializing data objects as recited in claim 1, wherein said attribute describes, further comprising: the attribute type.

3. The method of claim 2, wherein after the step of reading attribute values respectively corresponding to the attribute descriptions in the data object to be serialized, comprising:

4. The method of claim 1, wherein obtaining the metadata description file of the data object to be serialized is:

5. The method of claim 4, wherein before the step of writing the read attribute values into the result data stream of the data object in the order of the attribute description sequence number, the method comprises:

6. The method of claim 1, wherein prior to the step of obtaining a metadata description file of the data object to be serialized, further comprising:

7. The method of claim 6, wherein after the step of creating a new version of the metadata description file according to the modify operation, the property description in the metadata description file being changed according to the modify operation based on the property description in the original metadata description file, the method comprises:

and storing the new version of the metadata description file in the metadata warehouse.

8. The method according to claim 1, wherein after the step of reading the attribute values corresponding to the attribute descriptions in the data object to be serialized according to the reading method provided by the attribute operation class corresponding to each attribute, the method comprises:

and compressing the read attribute values.

9. An apparatus for serializing data objects, comprising: the device comprises a description file acquisition unit, an attribute operation class acquisition unit, an attribute reading unit and a serialization writing unit;

the description file obtaining unit is used for obtaining a metadata description file of a data object to be serialized, wherein the metadata description file comprises attribute descriptions marked by sequence numbers and attribute operation class names corresponding to attributes in the data object, and each attribute description comprises an attribute operation class name;

10. The apparatus for serializing data objects as claimed in claim 9, wherein the metadata description file of the data object to be serialized, which is obtained by the description file obtaining unit, further comprises an attribute type.

11. The apparatus for serializing data objects as recited in claim 10, wherein said attribute reading unit further comprises: attribute type reading subunit, default attribute operation class obtaining subunit and default value reading subunit;

12. The apparatus for serializing data objects as claimed in claim 9, wherein the description file acquiring unit is specifically configured to search the metadata description file of the data object to be serialized, which corresponds to the data object to be serialized and has the largest version number value, from a metadata repository according to the name of the data object to be serialized.

13. The apparatus for serializing data objects as recited in claim 12, wherein the serializing write unit further comprises: writing the version number into the subunit;

14. The apparatus for serializing data objects as recited in claim 9, further comprising: the device comprises a modification operation receiving unit, an operation class creating unit and a description file creating unit;

15. The apparatus for serializing data objects as recited in claim 14, further comprising: the device comprises a traversing unit, a numerical value acquiring unit, a version number generating unit and a storing unit;

and the saving unit is used for saving the metadata description file of the new version in the metadata warehouse.

16. The apparatus for serializing data objects as recited in claim 9, further comprising: an attribute value compression unit;

17. An electronic device, characterized in that the electronic device comprises:

a display;

a processor;

the storage is used for storing a serialization program, and the program executes the following operations when being read and executed by the processor, wherein the metadata description file of the data object to be serialized is obtained, and the metadata description file contains attribute descriptions marked by sequence numbers and attribute operation class names corresponding to the attributes in the data object, and each attribute description contains the attribute operation class name; obtaining attribute operation classes corresponding to the attributes according to the attribute operation class names provided by the attribute descriptions in the metadata description file; reading attribute values respectively corresponding to the attribute descriptions in the data objects to be serialized according to the reading method provided by the attribute operation class corresponding to each attribute; and writing the read attribute values into a result data stream of the data object according to the sequence of the attribute description sequence number, wherein the result data stream is used as a serialization result of the data object.

18. A method of deserializing a data stream, comprising:

obtaining a result data stream;

acquiring a metadata description file of a local data object, wherein the metadata description file comprises attribute descriptions identified by sequence numbers and attribute operation class names corresponding to attributes in the data object, and each attribute description comprises an attribute operation class name; obtaining attribute operation classes corresponding to the attributes according to the attribute operation class names provided by the attribute descriptions in the metadata description file;

19. The method of deserializing a data stream of claim 18, wherein the attribute describes, further comprising: the attribute type.

20. The method of claim 19, wherein the step of sequentially reading the attribute values in the resultant data stream according to the sequence of the attribute description sequence numbers in the reading method provided according to the attribute operation class corresponding to each attribute further comprises:

and discarding the read data in the result data stream.

21. The method of deserializing a data stream of claim 18, wherein the obtaining the metadata description file for the local data object comprises:

acquiring the name of the local data object;

22. The method according to claim 18, wherein after the step of sequentially reading the attribute values in the result data stream according to the sequence of the attribute description sequence numbers according to the reading method provided according to the attribute operation class corresponding to each attribute, the method comprises:

and decompressing the attribute values read from the result data stream.

23. An apparatus for deserializing a data stream, comprising: the device comprises a data stream acquisition unit, a description file acquisition unit, an attribute operation class acquisition unit, an attribute value reading unit and an attribute value writing unit;

the data stream acquiring unit is used for acquiring a result data stream;

the description file obtaining unit is used for obtaining a metadata description file of a local data object, wherein the metadata description file comprises attribute descriptions marked by sequence numbers and attribute operation class names corresponding to attributes in the data object, and each attribute description comprises an attribute operation class name; the attribute operation class acquisition unit is used for acquiring the attribute operation classes corresponding to the attributes according to the attribute operation class names provided by the attribute descriptions in the metadata description file;

24. The apparatus for deserializing a data stream of claim 23, wherein the description file obtaining unit obtains the metadata description file of the local data object further comprising an attribute type.

25. The apparatus for deserializing a data stream of claim 24, wherein the attribute value reading unit comprises: the attribute type acquisition subunit, the data reading subunit and the discarding subunit;

the attribute type obtaining subunit is configured to, in the reading method provided according to the attribute operation class corresponding to each attribute, sequentially read the attribute values in the result data stream according to the sequence of the attribute description sequence numbers, and obtain, for the attribute description of the attribute operation class that cannot be obtained, a default attribute operation class corresponding to the attribute type according to the attribute type of the attribute description;

and the discarding subunit is configured to discard the read data in the result data stream.

26. The apparatus for deserializing a data stream of claim 23, wherein the description file obtaining unit comprises: the system comprises a local data object name acquisition subunit, a result data stream analysis subunit, a description file traversal subunit and a description file acquisition subunit;

27. The apparatus for deserializing a data stream of claim 23, further comprising: an attribute value decompression unit;

28. An electronic device, characterized in that the electronic device comprises:

a display;

a processor;

a memory for storing an deserialization program that, when read by the processor for execution, the following operations are performed: obtaining a result data stream, obtaining a metadata description file of the local data object, the metadata description file contains attribute descriptions identified by sequence numbers and attribute operation class names contained by each of the attribute descriptions and corresponding to attributes in the data object, obtaining the attribute operation class corresponding to each attribute according to the attribute operation class name provided by each attribute description in the metadata description file, reading the attribute values in the result data stream in sequence according to the sequence of the attribute description sequence numbers according to the reading method provided by the attribute operation class corresponding to each attribute, and writing the read attribute values into the attributes of the local data object corresponding to each attribute description according to the writing method provided by the attribute operation class of each attribute description.

29. A serialization and deserialization system, wherein the system comprises:

apparatus for serializing data objects according to any one of claims 9 to 16; and an apparatus for deserializing a data stream according to any of claims 23-27.