CN102253980A - Data processing method and data processing system - Google Patents
Data processing method and data processing system Download PDFInfo
- Publication number
- CN102253980A CN102253980A CN2011101724666A CN201110172466A CN102253980A CN 102253980 A CN102253980 A CN 102253980A CN 2011101724666 A CN2011101724666 A CN 2011101724666A CN 201110172466 A CN201110172466 A CN 201110172466A CN 102253980 A CN102253980 A CN 102253980A
- Authority
- CN
- China
- Prior art keywords
- field
- data
- data recording
- client
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data processing method, which comprises the following steps of: acquiring a storage request of a data record from a client, wherein the data record comprises a plurality of fields, each field comprises metadata information and data contents, and the metadata data information comprises a field name, a field type and a field length; and storing the data record in a data record set. A data processing system comprises the client and a storage engine, wherein the client is used for sending the storage request of the data record; the data record comprises the fields having the metadata information and the data contents; the storage engine comprises a storage unit; and the storage unit is used for acquiring the storage request of the data record from the client and storing the data record into the data record set. By the technical scheme of the invention, the client is supported to store the data record of which structures of the fields are flexible and variable in the same data set without modification of definitions of list structures in the set, so the downtime and unavailability of the whole data set when the storage engine receives the data record having new fields can be avoided.
Description
Technical field
The present invention relates to data processing technique, particularly a kind of data processing method and data handling system.
Background technology
Along with the progress of science and technology, the data volume that is used for descriptor is also in continuous increase, and the scope that data relate to more and more widely, and it is more complicated that the relation between the data also becomes.In order to improve the work efficiency of data processing, the data treatment technology is also had higher requirement.
The main at present data processing technique of using at first is to deposit data in data record set, and for example database is operated the data in the database then.Data storing can be regarded the data recording that comprises field one by one as in database.Data record set generally is stored on the data storage engine (also being called for short storage engines), and client is communicated by letter with storage engines, visit data record set wherein.
What traditional relational database requirement deposited in is the fixed field data recording of the good field format of predefined, adds new field if desired and gives recording storage, just requires to change Field Definition, and it is unavailable that this can cause whole data record set to be shut down.
Generally speaking, mostly be the usage data record set together of a plurality of clients, that is to say, data record set need support a plurality of clients to share this data set of records ends, must satisfy simultaneously different clients again has the requirement of custom field when the storage data recording, and this access to the data recording that increases field newly should be transparent to other clients, do not make a difference.
If but by traditional mode the definition of data field storage to be managed by the mode of a definition of a tables of data, one of them client just may cause whole data record set unavailable to the interpolation or the modification of custom field.
The patent No. is the Chinese patent of ZL02148795.2, discloses the disposal route of object relationship in a kind of database, has simplified the management process that concerns between the object in the database, has reduced the workload of database maintenance.But how to be implemented under the situation that does not influence other client stores data recording, allow the field of client stores free schema, and alleviate the expense of the Network Transmission between storage engines and the client and the processing burden of client becomes one of present designer's problem demanding prompt solution.
Summary of the invention
The problem that the present invention solves provides a kind of data processing method and system, add self-defining field to support client, and this field is transparent to other clients, also can not cause the shutdown of storage engines unavailable.
For addressing the above problem, the invention provides a kind of data processing method, comprising:
From the storage request that client is obtained data recording, described data recording comprises a plurality of fields, and each field comprises metadata information and data content, and described metadata information comprises field name, field type and field length;
Described data recording is stored in the data record set.
Optionally, described data recording also comprises the sign of the described data recording of mark, and described data processing method also comprises the identification index of setting up described data recording.
Optionally, described data processing method also comprises:
Obtain the request of access of data recording from client, described request of access comprises the field name of the data recording that will visit;
Identification index according to described data recording is inquired about the described data recording that will visit from described data record set;
The field of the data recording that the field name of the data recording that foundation inquires and field length inquiry will be visited.
Optionally, the field of inquiring about the data recording that will visit according to the field name and the field length of the data recording that inquires comprises:
Compare the field name of field of the described data recording that inquires and the field name of the described data recording that will visit successively, if not matching then, the field length of the field of the current comparison of foundation jumps to next field, continue comparison, mate until the field name of the field of current comparison field name with the described data recording that will visit.
Optionally, described request of access is the request of reading, and described data processing method also comprises: the data content of the field of the data recording that visit that will inquire returns to described client.
Optionally, described request of access is for revising request, described modification request also comprises new data content, and described data processing method also comprises: the data content that the field type of the field of the data recording that will visit that foundation inquires is revised this field is new data content.
Optionally, the nested a plurality of fields of described data content, each field comprises the metadata information and the data content of this field.
For solving the problems of the technologies described above, the present invention also provides a kind of data handling system, comprising:
Client, in order to send the storage request of data recording, described data recording comprises a plurality of fields, and each field comprises metadata information and data content, and described metadata information comprises field name, field type and field length;
Storage engines comprises storage unit, and described storage unit is stored to described data recording in the data record set in order to obtain the storage request of described data recording from described client.
Optionally, described data recording also comprises the sign of the described data recording of mark, and described storage engines also comprises indexing units, and described indexing units is in order to set up the identification index of described data recording.
Optionally, described storage engines also comprises:
The record queries unit, in order to obtain the request of access of data recording from client, described request of access comprises the field name of the data recording that will visit, and the identification index of the data recording of the described indexing units foundation of foundation is inquired about the described data recording that will visit from described data record set;
The field query unit is in order to the field name of the data recording that inquires according to described record queries unit and the field of the data recording that the field length inquiry will be visited.
Optionally, described field query unit comprises:
Comparing unit is in order to the field name of the field of comparing the described data recording that inquires successively and the field name of the described data recording that will visit;
Control module, whether the comparison result of judging described comparing unit is for not matching, if then the field length of the field of the current comparison of foundation jumps to next field, and the control comparing unit continues comparison, mates until the field name of the field of the current comparison field name with the described data recording that will visit.
Optionally, described request of access is the request of reading, and described storage engines also comprises: feedback unit returns to described client in order to the data content of the field of the data recording that will visit that will inquire.
Optionally, described request of access is for revising request, described modification request also comprises new data content, and described storage engines also comprises: revising the unit, is new data content in order to the data content of revising this field according to the field type of the field of the data recording that will visit that inquires.
Optionally, the nested a plurality of fields of described data content, each field comprises the metadata information and the data content of this field.
Compared with prior art, the technical program has the following advantages:
Access metadata information in the data recording of each storage, metadata information has defined field name, field length and the field type of field contained in the whole data recording, this is by unified standard and can be understood by the data storage engine, avoids giving client oneself content format of data recording merely and handles waste and the unnecessary spending of being brought.
There is not the unified data field format definition metadata information of safeguarding of needs in each data record set, thereby yet just do not safeguard or revise the Field Definition metadata information of certain record and need cause the user of whole data recording collection all to shut down the problem of wait.
The metadata information that exists in this data recording can help storage engines to navigate to one of them or several fields on request fast, extracts or revise the operation of its content.
Comprise metadata information in the data of storage and be supported in and do not shut down ground storage format amended data recording in the storage engines by allowing; Accordingly, storage engines is supported the data structure of the free definition of data record of client, it is free object (Schema-Flexible) pattern, client as required the field in the self-defining data record, field data content with and metadata information, and send it to storage engines and preserve into data recording.
Description of drawings
Fig. 1 is the process flow diagram of data processing method provided by the invention;
Fig. 2 is the structural representation of a kind of embodiment of the data handling system that provides of the embodiment of the invention;
Fig. 3 is the structural representation of the another kind of embodiment of the data handling system that provides of the embodiment of the invention;
Fig. 4 is the structural representation of the data recording storage that provides of the embodiment of the invention;
Fig. 5 is the structural drawing of the data record set that provides of the embodiment of the invention;
Fig. 6 is first kind of embodiment of the data recording storage that provides of the embodiment of the invention;
Fig. 7 is second kind of embodiment of the data recording storage that provides of the embodiment of the invention;
Fig. 8 is the third embodiment of the data recording storage that provides of the embodiment of the invention;
Fig. 9 is the 4th a kind of embodiment of the data recording storage that provides of the embodiment of the invention;
Figure 10 is the amended data recording of data recording shown in Figure 8.
Embodiment
For above-mentioned purpose of the present invention, feature and advantage can more be become apparent, below in conjunction with present situation and the specific embodiment of the present invention of data processing work are described in detail at present.
Data are present in nature and the human society widely as a kind of message form.Be accompanied by the appearance of computer technology, rely on its excellent data processing performance, be applied in widely in the data processing business in each field.
What traditional relational database requirement deposited in is the data recording of the fixed field of the good field format of predefined, adds new field if desired and stores to data recording, just requires to change Field Definition, and it is unavailable that this will cause whole data record set to be shut down.Because a plurality of clients are used the set of same data recording, this not only needs to satisfy different clients has custom field when the storage data recording requirement, and this access to the data recording that increases field newly is transparent to client, do not make a difference.
In addition, when client needs field in the data query record, because the parsing need of work client of data recording is finished, so storage engines need be sent to client with the whole piece data recording, and this is in the processing burden of expense that has increased Network Transmission virtually and client.If when client needs more newer field, need upgrade the whole piece data recording, this not only expends time in and also expends the resource of client, causes a large amount of unnecessary spending.
The inventor finds: if the metadata information that description field data layout in the storage engines of a standard is not provided is to the data storage engine, and oneself go to solve the operation of the Context resolution of whole data recording being given each client, will cause following problem:
A) under most of situation, certain client only needs the field in partial data of access record or the information of several fields in fact, but if give client the parsing work of data layout, then certainly will all transmit a complete data recording at every turn and give client application, this can cause very a large amount of unnecessary network overheads and handle burden;
B) when certain client only needs to upgrade certain field in the data record, because have only client could understand data layout, to make use of momentum and necessary whole data recording is upgraded, this can cause the expense of a large amount of unnecessary operations.
Fig. 1 is the process flow diagram of data processing method provided by the invention, and described data processing method comprises:
S101, the storage request of obtaining data recording from client;
S102 is stored to described data recording in the data record set.
Described data recording comprises a plurality of fields, and each field comprises metadata information and data content, and described metadata information comprises field name, field type and field length.Described metadata information is also referred to as self-described information, and in order to field name, field type and the field length of describing each field, and a field has a metadata information.Described data recording also can only comprise a field, and this field comprises data content and metadata information.
All right nested field in the described data content, data content can be that simple numerical value, text can also be complicated fields.Nested field in the data content also has data content and metadata information.In addition, the data content of nested field can also nested again field in the data content, that is to say that data content also is the nested structure of multilayer.Can to be one also can be that a plurality of, nested numbers of plies can be a multilayer to the number of nested field in the data content, also can be individual layer.In actual design, nested in the data content can also be other structures.
Described data recording also comprises the sign of the described data recording of mark, and described data processing method also comprises the identification index of setting up described data recording.Described to be designated the overall situation unique, belongs to which client in order to distinguish data recording.Set up described identification index, described data recording is conveniently inquired about according to described expression index stores.
Described data processing method also comprises:
Obtain the request of access of data recording from client, described request of access comprises the field name of the data recording that will visit;
Identification index according to described data recording is inquired about the described data recording that will visit from described data record set;
The field of the data recording that the field name of the data recording that foundation inquires and field length inquiry will be visited, field in the described data recording has metadata information, and the field of inquiring about the data recording that will visit according to the field name and the field length of the data recording that inquires comprises:
Compare the field name of field of the described data recording that inquires and the field name of the described data recording that will visit successively, if not matching then, the field length of the field of the current comparison of foundation jumps to next field, continue comparison, mate until the field name of the field of current comparison field name with the described data recording that will visit.
General request of access comprises the request of reading and revises request that if described request of access is the request of reading, then described data processing method also comprises: the data content of the field of the data recording that visit that will inquire returns to described client.
If described request of access is for revising request, then described modification request also comprises new data content, and described data processing method also comprises: the data content that the field type of the field of the data recording that will visit that foundation inquires is revised this field is new data content.Because the data content of amended field may change the length of field or field name etc., so the metadata information of this field also can revise, and for example revises field length or field name etc.
Fig. 2 is the structural representation of a kind of embodiment of the data handling system that provides of the embodiment of the invention, and described data handling system comprises:
Client 200, in order to send the storage request of data recording, described data recording comprises a plurality of fields, and each field comprises metadata information and data content, and described metadata information comprises field name, field type and field length;
Common described data recording also comprises the sign of the described data recording of mark, and storage engines 100 also comprises indexing units 102, and indexing units 102 is in order to set up the identification index of described data recording.
Wherein the field query unit can comprise:
Comparing unit is in order to the field name of the field of comparing the described data recording that inquires successively and the field name of the described data recording that will visit;
Control module, whether the comparison result of judging described comparing unit is for not matching, if then the field length of the field of the current comparison of foundation jumps to next field, and the control comparing unit continues comparison, mates until the field name of the field of the current comparison field name with the described data recording that will visit.
Fig. 3 is the structural representation of the another kind of embodiment of the data handling system that provides of the embodiment of the invention, described data handling system comprises a plurality of clients, client 201, client 202, client 203 and client 204, the data record set in these client shared storage engines 100.
Fig. 4 is the structural representation of the data recording storage that provides of the embodiment of the invention, and wherein field name 11, type (field type) 22, length (field length) 33 have been formed metadata information, and a field has a metadata information.Value44 represents the data content of this field, the data content of field can comprise field, that is to say, the data content of field is not limited to simple numerical value, the field that can also be nested form comprises field, and the field that comprises in the data content also has metadata information.
When client is preserved data recording in request, provide data recording to be stored, described data recording comprises the data content of field, field and the metadata information of each field.Client is when sending described request, and described data recording can be carried in the described request according to structure shown in Figure 4, after storage engines receives described request, can preserve data recording according to structure shown in Figure 4 according to the request of client.
Student information and curriculum information with a school is example below, describes technical scheme provided by the invention in detail.
Fig. 5 is the structural drawing of the data record set that provides of the embodiment of the invention, need to preserve student information and curriculum information in the storage engines, the information that wherein comprises name, address, course achievement in the student information also comprises the information of street, city, state, postcode in the address information.
Storage engines only need be preserved 2 data set of records ends: student information data record set 1 and curriculum information data record set 2, wherein curriculum information data record set 2 can be a relational database.
For the student information data record set:
(1) every data recording comprises 3 fields, is respectively name, Address, score, and each field is carried self-described information (be also referred to as metadata information, comprise field name, type and length);
(2) nested 4 field: address, city, state, postalCode in the Address field, each field is carried self-described information; The data pattern freedom of Address field can increase field;
(3) nested 2 fields in the score field: a pointer field that is the sensing course writes down (generally is exactly the key value Key of curriculum information, and the pointer of course record is pointed in expression, in order to from the course database, to obtain the key assignments key of curriculum information), another is the achievement field of this this subject of student; The data pattern freedom of score field, promptly student can have the achievement of multi-door course, and every subject comprises 2 fields (course pointer field and course achievement field, each field is carried self-described information).
For the curriculum information data record set:
Every data recording comprises the name (being generally the key assignments key of curriculum information) and the corresponding data content thereof of course.
Storage engines and three clients have annexation, are the applicating example of each client below:
Fig. 6 is first kind of embodiment of the data recording storage that provides of the embodiment of the invention, describes in detail below in conjunction with Fig. 6:
Storage engines obtains the data of client 1 and preserves the request back:
(1) set up to indicate index, the sign of client 1 is joined in the described sign index.Data are recorded in to store in the storage engines and have a memory location, and described identification index has write down the mapping relations between this memory location and the sign, by the position that described identification index can find data recording to store very soon, accelerate the speed of data processing.Described identification index can be common Hash table;
Identification index is generally set up when for the first time storing data recording, and is follow-up when storing data recording again, only needs sign and corresponding memory location thereof are directly joined in the described sign index.
(2) write the name field, type string, length is 4, value (data content) is Jane;
(3) write the Address field, type string, because nested field in the Address field, the length of Address field is its nested field length summation, when writing the value of Address field, write nested 4 field: address, city, state, postalCode; The writing mode of these 4 fields is as (2);
(4) write the score field, the writing mode of score is shown in (2), and the field that is embedded among the score also writes shown in (2).
Adjustment on above-mentioned (1)-(4) step can be done in proper order according to actual conditions.
Fig. 7 is second kind of embodiment of the data recording storage that provides of the embodiment of the invention, describes in detail below in conjunction with Fig. 7:
Storage engines obtains the data of client 2 and preserves the request back:
(1) sign user2 and the corresponding memory location thereof with client 2 joins in the sign index;
(2) write the name field, type string, length is 4, value (data content) is Jack;
(3) write the Address field, type string, because nested field city in the Address field, the length of Address field is the length of field city, when writing the value of Address field, writes 1 nested field: city; Writing mode is as (2);
(4) write the score field, the writing mode of score is shown in (2), and the field that is embedded among the score also writes shown in (2).
Adjustment on above-mentioned (1)-(4) step can be done in proper order according to actual conditions.
Fig. 8 is the third embodiment of the data recording storage that provides of the embodiment of the invention, describes in detail below in conjunction with Fig. 8:
(1) sign user3 and the corresponding memory location thereof with client 3 joins in the sign index;
(2) write the name field, type string, length is 4, value (data content) is Tomy;
(3) write the Address field, type string, because it is nested except nested shared field in the Address field, also nested privately owned field, when writing the value of Address field, write nested 4 shared field: address, city, state, postalCode and 1 privately owned field province; Writing mode is as (2);
(4) write the score field, the writing mode of score is shown in (2), the field that is embedded among the score also writes shown in (2), because the adding of privately owned field province (supposition length is 3), field integral body after the Address field is moved 3 length spaces backward, and promptly the memory location of score field has been moved 3 length spaces backward.
Adjustment on above-mentioned (1)-(4) step can be done in proper order according to actual conditions.
Each client can define type, length and the data content of the field of storage freely according to the needs of oneself when the storage data recording.The length of the field of the data recording of each client can be identical with data content, also can be different.
Because the data pattern freedom (field length is any) of Address field, can in the Address field, increase custom field (as the data recording of storage client 3), also can in the Address field, only use the nested field of part (as the data recording of storage client 2), therefore support the data recording storage of random length (behind the form modifying).Comprise self-described information (metadata information) in the data of storage and be supported in and do not shut down ground storage format amended data recording in the storage engines by allowing like this.
Owing to support the data recording storage of random length; and the field of every data recording of student information data record set is all carried self-described information (metadata information); rather than whole student information data record set has only a descriptor; therefore when client 3 storage data recording; avoided causing the client 1 of shared student database and the shutdown of client 2 to wait for that client 1 and client 2 still can be operated its data recording because will make amendment to the descriptor of whole student information data record set (data recording of storage before can influencing).
What traditional relational database requirement deposited in is the fixed field data recording of the good field format of predefined; add new field if desired and give recording storage; just require to change Field Definition; it is unavailable that this can cause whole data record set to be shut down; technical solution of the present invention is not stored data recording Field Definition metadata description information by a kind of by the mode of a description of a data set; but in the data recording of each storage the data recording field metadata of the certain self-described of access; there is not the unified data field format definition metadata of safeguarding of needs in each data centralization, thereby does not yet just safeguard or revise the Field Definition metadata of certain record and need cause the user of whole data recording collection all to shut down the problem of wait.
This satisfies different clients has the requirement of custom field during data recording in storage, and this access to the data recording that increases field newly should be to be transparent to other clients, does not make a difference each other.
Fig. 9 is the 4th a kind of embodiment of the data recording storage that provides of the embodiment of the invention, describes in detail below in conjunction with Fig. 9:
Need to increase client 4 at present, that is to say, the number of client increases to four from original three, the mode that client 4 is preserved data recording is consistent with the preserving type of other clients, but comprise two groups of course achievement below the score field, in the course achievement field, have two groups of course achievement.
Because therefore the data pattern freedom of score field can increase by one group of pointer field (for_course) and achievement field (grade) in the score field, promptly support the data recording storage of random length;
Owing to support the data recording storage of random length, and the field of every data recording of student information data record set is all carried self-described information (metadata information), rather than whole student information data record set has only a descriptor, therefore when client 4 storage data recording, avoided and to have made amendment to the descriptor of whole student information data record set, can not influence the use of other clients.If revise then can influence before the data recording of storage, thereby cause the shutdown of other clients of the student information data record set shared to wait for that other clients still can be unavailable to its data recording.
If client is sent inquiry or is revised request, when for example client 3 needs inquiry or revises data, for example request of " in the inquiry student information data record set: the data content of name field is the score field of the data recording (being designated hereinafter simply as data recording Tomy) of Tomy ".
After storage engines obtains this request, according to sign ID-user3, inquire about this sign in the identification index that indexing units is set up, and obtain the memory location of the data recording of user3 correspondence, the record queries unit can find the data recording that will visit very easily like this;
The field query unit begins to search this data recording according to the field name that will visit that carries in the request.
Concrete, comparing unit is compared the field name of field of the described data recording that inquires and the field name of the described data recording that will visit successively; Control module judges that the name field name is not score, and then the field length according to the name in the data recording jumps to back one field, continues comparison, up to searching the field that belongs to client 3 fields score by name;
After finding the score field, the score field is the content (with the request content coupling of client 3) that client 3 will be inquired about, and the data content that feedback unit takes out the score field feeds back to client 3.If revise the data content of score field, be revised as 4.5 as grade with Biology, then revising the grade that revises Biology the unit has 4.0 to change 4.5 into.Figure 10 is the amended data recording of data recording shown in Figure 8.
Client identification can be determined the data recording that will visit fast, and the metadata information of each field (field name and field length) can help storage engines to find the field location that client need be inquired about fast.That is to say that the self-described information (metadata information) that exists in the data recording can help storage engines to navigate to one of them or several fields, the operation of inquiring about or revising on request fast.
Each client knows what the field that oneself needs is, how to preserve, and when carrying out data processing (for example data query record), only needs to send the field that needs inquiry to storage engines, thereby has alleviated the processing burden of storage engines;
The field of the data recording that storage engines only need need client feeds back to client, and need not transmit a complete data recording, has reduced unnecessary network overhead like this;
Client can directly be obtained the data content that needs, and does not need whole piece data are resolved the data content that obtains needs, has reduced the processing burden of client like this.That is to say, each field all has the information of self-described, the name of this field, the length of field and the basic data type of field have been write down, this is by unified standard and can be understood by the data storage engine, avoids giving application oneself content format of data recording merely and handles waste and the unnecessary spending of being brought.
Technical scheme of the present invention is not stored data recording Field Definition metadata description information by a kind of by the mode of a description of data set, but the data recording field metadata of the certain self-described of access reaches following target in the data recording of each storage:
A) information definition of self-described the name of contained record field, the length of field and the basic data type of field in the whole data recording, this is by unified standard and can be understood by the data storage engine, avoids giving application oneself content format of data recording merely and handles waste and the unnecessary spending of being brought:
B) there is not the unified data field format definition metadata of safeguarding of needs in each data centralization, thereby does not yet just safeguard or revise the Field Definition metadata of certain record and need cause the user of whole data recording collection all to shut down the problem of wait;
C) metadata information that exists in this data recording can help storage engines to navigate to one of them or several fields on request fast, extracts or revise the operation of its content.
Comprise self-described information in the data of storage and be supported in and do not shut down ground storage format amended data recording in the storage engines by allowing; Accordingly, storage engines is supported the data structure of the free definition of data record of client, it is free object (Schema-Flexible) pattern, the client data content of the field in the self-defining data record, field as required and sends it to storage engines and preserves into data recording with its metadata information.
Data processing method and disposal system are supported the data recording of client at same data set stored field structure flexibility and changeability in the technical program; and need not revise concentrated list structure definition, thereby can not cause the shutdown of storage engines whole data set when receiving the data recording of new field unavailable.Client is when visit data writes down simultaneously, the numerical value of certain or a plurality of specific fields in the request msg record separately, be forced to whole data read and be transferred to client and for want of uniform data structure and record field are variable, storage engines can be according to the field name of client-requested, and utilize this part contained in data recording metadata information and obtain in the record certain or a plurality of field in information directly return to client.
Though the present invention with preferred embodiment openly as above; but it is not to be used for limiting the present invention; any those skilled in the art without departing from the spirit and scope of the present invention; can utilize the method and the technology contents of above-mentioned announcement that technical solution of the present invention is made possible change and modification; therefore; every content that does not break away from technical solution of the present invention; to any simple modification, equivalent variations and modification that above embodiment did, all belong to the protection domain of technical solution of the present invention according to technical spirit of the present invention.
Claims (14)
1. a data processing method is characterized in that, comprising:
From the storage request that client is obtained data recording, described data recording comprises a plurality of fields, and each field comprises metadata information and data content, and described metadata information comprises field name, field type and field length;
Described data recording is stored in the data record set.
2. data processing method as claimed in claim 1 is characterized in that described data recording also comprises the sign of the described data recording of mark, and described data processing method also comprises the identification index of setting up described data recording.
3. data processing method as claimed in claim 2 is characterized in that, also comprises:
Obtain the request of access of data recording from client, described request of access comprises the field name of the data recording that will visit;
Identification index according to described data recording is inquired about the described data recording that will visit from described data record set;
The field of the data recording that the field name of the data recording that foundation inquires and field length inquiry will be visited.
4. data processing method as claimed in claim 3 is characterized in that, the field of inquiring about the data recording that will visit according to the field name and the field length of the data recording that inquires comprises:
Compare the field name of field of the described data recording that inquires and the field name of the described data recording that will visit successively, if not matching then, the field length of the field of the current comparison of foundation jumps to next field, continue comparison, mate until the field name of the field of current comparison field name with the described data recording that will visit.
5. data processing method as claimed in claim 3 is characterized in that described request of access is the request of reading, and described data processing method also comprises: the data content of the field of the data recording that visit that will inquire returns to described client.
6. data processing method as claimed in claim 3, it is characterized in that, described request of access is for revising request, described modification request also comprises new data content, and described data processing method also comprises: the data content that the field type of the field of the data recording that will visit that foundation inquires is revised this field is new data content.
7. as each described data processing method of claim 1-6, it is characterized in that, the nested a plurality of fields of described data content, each field comprises the metadata information and the data content of this field.
8. a data handling system is characterized in that, comprising:
Client, in order to send the storage request of data recording, described data recording comprises a plurality of fields, and each field comprises metadata information and data content, and described metadata information comprises field name, field type and field length;
Storage engines comprises storage unit, and described storage unit is stored to described data recording in the data record set in order to obtain the storage request of described data recording from described client.
9. data handling system as claimed in claim 8 is characterized in that described data recording also comprises the sign of the described data recording of mark, and described storage engines also comprises indexing units, and described indexing units is in order to set up the identification index of described data recording.
10. data handling system as claimed in claim 9 is characterized in that, described storage engines also comprises:
The record queries unit, in order to obtain the request of access of data recording from client, described request of access comprises the field name of the data recording that will visit, and the identification index of the data recording of the described indexing units foundation of foundation is inquired about the described data recording that will visit from described data record set;
The field query unit is in order to the field name of the data recording that inquires according to described record queries unit and the field of the data recording that the field length inquiry will be visited.
11. data handling system as claimed in claim 10 is characterized in that, described field query unit comprises:
Comparing unit is in order to the field name of the field of comparing the described data recording that inquires successively and the field name of the described data recording that will visit;
Control module, whether the comparison result of judging described comparing unit is for not matching, if then the field length of the field of the current comparison of foundation jumps to next field, and the control comparing unit continues comparison, mates until the field name of the field of the current comparison field name with the described data recording that will visit.
12. data handling system as claimed in claim 10, it is characterized in that, described request of access is the request of reading, and described storage engines also comprises: feedback unit returns to described client in order to the data content of the field of the data recording that will visit that will inquire.
13. data handling system as claimed in claim 10, it is characterized in that, described request of access is for revising request, described modification request also comprises new data content, described storage engines also comprises: revising the unit, is new data content in order to the data content of revising this field according to the field type of the field of the data recording that will visit that inquires.
14. as each described data handling system of claim 8-13, it is characterized in that, the nested a plurality of fields of described data content, each field comprises the metadata information and the data content of this field.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011101724666A CN102253980A (en) | 2011-06-23 | 2011-06-23 | Data processing method and data processing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011101724666A CN102253980A (en) | 2011-06-23 | 2011-06-23 | Data processing method and data processing system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102253980A true CN102253980A (en) | 2011-11-23 |
Family
ID=44981244
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011101724666A Pending CN102253980A (en) | 2011-06-23 | 2011-06-23 | Data processing method and data processing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102253980A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102609452A (en) * | 2012-01-11 | 2012-07-25 | 上海西本网络科技有限公司 | Data storage method and data storage device |
| CN103092916A (en) * | 2012-12-14 | 2013-05-08 | 华为技术有限公司 | Method and device for modifying data structure |
| CN103500206A (en) * | 2013-09-29 | 2014-01-08 | 北京华胜天成科技股份有限公司 | Storage method and device based on file storage data |
| CN103577483A (en) * | 2012-08-07 | 2014-02-12 | 腾讯科技（深圳）有限公司 | Data storage method, data storage system, data access method and data access system |
| CN104270432A (en) * | 2014-09-22 | 2015-01-07 | 苏州耐克斯特能源开采技术有限公司 | Real-time data service system and data interaction method based on drilling industry |
| CN105159691A (en) * | 2015-10-30 | 2015-12-16 | 北京奇虎科技有限公司 | Method and device for updating metadata |
| CN105809408A (en) * | 2014-12-30 | 2016-07-27 | 金蝶软件（中国）有限公司 | Data transmission method and data transmission device |
| CN106033582A (en) * | 2016-04-29 | 2016-10-19 | 苏州奖多多科技有限公司 | Data processing method and device |
| CN107864404A (en) * | 2017-11-20 | 2018-03-30 | 四川长虹电器股份有限公司 | The method for not falling data upgrading is realized in data of set top box storehouse |
| CN108228759A (en) * | 2017-12-22 | 2018-06-29 | 金蝶软件（中国）有限公司 | Storage processing method, device, computer equipment and the storage medium of record set |
| CN109144950A (en) * | 2018-07-20 | 2019-01-04 | 中国邮政储蓄银行股份有限公司 | The storage method and device of business datum |
| CN109446208A (en) * | 2018-09-03 | 2019-03-08 | 深圳壹账通智能科技有限公司 | A kind of date storage method, computer readable storage medium and server |
| CN112788077A (en) * | 2019-11-07 | 2021-05-11 | 上海哔哩哔哩科技有限公司 | Data acquisition method and device, computer equipment and computer-readable storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1735890A (en) * | 2003-10-23 | 2006-02-15 | 微软公司 | System and method for storing and retrieving a field of a user defined type outside of a database store in which the type is defined |
| CN101021876A (en) * | 2007-03-09 | 2007-08-22 | 华为技术有限公司 | Data management method, equipment and data bank system |
| CN101777057A (en) * | 2004-04-02 | 2010-07-14 | 易享信息技术（上海）有限公司 | Methods and systems for storing customer fields for multiple tenants in multi-tenant database system |
-
2011
- 2011-06-23 CN CN2011101724666A patent/CN102253980A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1735890A (en) * | 2003-10-23 | 2006-02-15 | 微软公司 | System and method for storing and retrieving a field of a user defined type outside of a database store in which the type is defined |
| CN101777057A (en) * | 2004-04-02 | 2010-07-14 | 易享信息技术（上海）有限公司 | Methods and systems for storing customer fields for multiple tenants in multi-tenant database system |
| CN101021876A (en) * | 2007-03-09 | 2007-08-22 | 华为技术有限公司 | Data management method, equipment and data bank system |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102609452B (en) * | 2012-01-11 | 2014-12-10 | 上海西本网络科技有限公司 | Data storage method and data storage device |
| CN102609452A (en) * | 2012-01-11 | 2012-07-25 | 上海西本网络科技有限公司 | Data storage method and data storage device |
| CN103577483A (en) * | 2012-08-07 | 2014-02-12 | 腾讯科技（深圳）有限公司 | Data storage method, data storage system, data access method and data access system |
| CN103577483B (en) * | 2012-08-07 | 2018-07-24 | 腾讯科技（深圳）有限公司 | The method and system of date storage method and system and data access |
| CN103092916A (en) * | 2012-12-14 | 2013-05-08 | 华为技术有限公司 | Method and device for modifying data structure |
| CN103092916B (en) * | 2012-12-14 | 2016-11-02 | 华为技术有限公司 | The method and apparatus of amendment data structure |
| CN103500206A (en) * | 2013-09-29 | 2014-01-08 | 北京华胜天成科技股份有限公司 | Storage method and device based on file storage data |
| CN104270432B (en) * | 2014-09-22 | 2018-07-17 | 苏州耐克斯特能源开采技术有限公司 | Based on drilling well industry Real-time Data Service system and data interactive method |
| CN104270432A (en) * | 2014-09-22 | 2015-01-07 | 苏州耐克斯特能源开采技术有限公司 | Real-time data service system and data interaction method based on drilling industry |
| CN105809408A (en) * | 2014-12-30 | 2016-07-27 | 金蝶软件（中国）有限公司 | Data transmission method and data transmission device |
| CN105159691B (en) * | 2015-10-30 | 2019-03-05 | 北京奇虎科技有限公司 | The method and device of more new metadata |
| CN105159691A (en) * | 2015-10-30 | 2015-12-16 | 北京奇虎科技有限公司 | Method and device for updating metadata |
| CN106033582A (en) * | 2016-04-29 | 2016-10-19 | 苏州奖多多科技有限公司 | Data processing method and device |
| CN107864404A (en) * | 2017-11-20 | 2018-03-30 | 四川长虹电器股份有限公司 | The method for not falling data upgrading is realized in data of set top box storehouse |
| CN108228759A (en) * | 2017-12-22 | 2018-06-29 | 金蝶软件（中国）有限公司 | Storage processing method, device, computer equipment and the storage medium of record set |
| CN108228759B (en) * | 2017-12-22 | 2021-07-27 | 金蝶软件（中国）有限公司 | Record set storage processing method and device, computer equipment and storage medium |
| CN109144950A (en) * | 2018-07-20 | 2019-01-04 | 中国邮政储蓄银行股份有限公司 | The storage method and device of business datum |
| CN109144950B (en) * | 2018-07-20 | 2022-02-15 | 中国邮政储蓄银行股份有限公司 | Service data storage method and device |
| CN109446208A (en) * | 2018-09-03 | 2019-03-08 | 深圳壹账通智能科技有限公司 | A kind of date storage method, computer readable storage medium and server |
| WO2020048054A1 (en) * | 2018-09-03 | 2020-03-12 | 深圳壹账通智能科技有限公司 | Data storage method, computer-readable storage medium, server, and apparatus |
| CN112788077A (en) * | 2019-11-07 | 2021-05-11 | 上海哔哩哔哩科技有限公司 | Data acquisition method and device, computer equipment and computer-readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102253980A (en) | Data processing method and data processing system | |
| CN107291948B (en) | Access method of distributed newSQL database | |
| CN101876983B (en) | Method for partitioning database and system thereof | |
| US7376658B1 (en) | Managing cross-store relationships to data objects | |
| AU2014240211B2 (en) | Background format optimization for enhanced sql-like queries in hadoop | |
| CN104850572B (en) | HBase non-primary key index construct and querying method and its system | |
| JP4839706B2 (en) | Index management method for database management system | |
| CN102750356B (en) | Construction and management method for secondary indexes of key value library | |
| EP3333726A1 (en) | Distributed database processing method and device | |
| CN101840400B (en) | Multilevel classification retrieval method and system | |
| CN106294565A (en) | A kind of data bank access method and system | |
| CN103064875A (en) | Distributed query method of spatial service data | |
| CN100594497C (en) | System for implementing network search caching and search method | |
| CN101727465A (en) | Methods for establishing and inquiring index of distributed column storage database, device and system thereof | |
| CN104123346A (en) | Structural data searching method | |
| CN104391908B (en) | Multiple key indexing means based on local sensitivity Hash on a kind of figure | |
| CN101546325A (en) | Grid heterogeneous data integrating method based on SOA | |
| CN102067116A (en) | Spatial querying in a data warehouse | |
| CN102915382A (en) | Method and device for carrying out data query on database based on indexes | |
| CN102760165B (en) | Full text retrieval method using bitmap index and device | |
| CN101789027A (en) | Metadata management method based on DBMS and metadata server | |
| CN106294374A (en) | The method of small documents merging and data query system | |
| CN103164408A (en) | Information storage and query method based on vertical search engine and device thereof | |
| Force et al. | Encouraging data citation and discovery with the Data Citation Index | |
| CN104731945A (en) | Full-text searching method and device based on HBase |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20111123 |