CN110569398A - data processing method and device, computer equipment and readable storage medium - Google Patents
data processing method and device, computer equipment and readable storage medium Download PDFInfo
- Publication number
- CN110569398A CN110569398A CN201910861161.2A CN201910861161A CN110569398A CN 110569398 A CN110569398 A CN 110569398A CN 201910861161 A CN201910861161 A CN 201910861161A CN 110569398 A CN110569398 A CN 110569398A
- Authority
- CN
- China
- Prior art keywords
- data
- written
- area
- data block
- main key
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
the embodiment of the application provides a data processing method and device, computer equipment and a readable storage medium. The data processing method comprises the steps of obtaining a combined main key index value of data to be written, judging whether an address index of the data to be written exists in a main key index area or a combined main key index area according to the combined main key index value, and if the address index of the data to be written exists in the main key index area or the combined main key index area, storing the data to be written into a position in a data block area pointed by the address index. Therefore, the data to be written can be conveniently stored in the corresponding position of the data block area.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data processing method, an apparatus, a computer device, and a readable storage medium.
background
At present, in the data processing, all related data are mainly stored in one data block, if a certain data in the data block is to be processed (for example, updated and deleted), the whole data block needs to be taken out, and the data processing can be realized through operations such as decompression, modification, repacking and the like.
In view of this, it is necessary for those skilled in the art to provide a convenient data processing scheme.
Disclosure of Invention
the embodiment of the application provides a data processing method, a data processing device, computer equipment and a readable storage medium.
The embodiment of the application can be realized as follows:
In a first aspect, an embodiment of the present application provides a data processing method, which is applied to a computer device, where a memory of the computer device includes a primary key index area, a joint primary key index area, and a data block area, and the method includes:
Acquiring a combined primary key index value of data to be written;
and judging whether the address index of the data to be written exists in the main key index area or the combined main key index area according to the combined main key index value, and if so, storing the data to be written into the position of the data block area pointed by the address index.
In an optional implementation manner, the step of determining whether the address index of the to-be-written data exists in the primary key index area or the joint primary key index area, and if so, storing the to-be-written data to a position in the data block area pointed by the address index includes:
judging whether the address index of the data to be written exists in the combined main key index area, if so, updating the data to be written to the position in the data block area pointed by the address index; if not, judging whether the address index of the data to be written exists in the main key index area, if so, writing the data to be written into the position in the data block area pointed by the address index, and updating the joint main key index area according to the joint main key index value.
In an optional embodiment, the step of updating the data to be written to the location in the data block area pointed to by the address index includes:
judging whether the data block of the data to be written is larger than the data block in the data block area pointed by the address index, if so, allocating a first memory block adaptive to the size of the data block of the data to be written from the data block area, and writing the data block of the data to be written into the first memory block; and if not, updating the data block of the data to be written to the position in the data block area pointed by the address index.
In an optional embodiment, the step of writing the data to be written to a location in the data block area pointed to by the address index includes:
Calculating the size of the data block of the data to be written, and allocating a second memory block adaptive to the size of the data block of the data to be written from the data block area according to the size of the data block of the data to be written; and writing the data block into which the data is to be written into the second memory block, where the second memory block is located at a position in the data block area pointed to by the address index.
in an alternative embodiment, the method further comprises: if the address index of the data to be written does not exist in the main key index area and the combined main key index area, creating a target main key and a target data block according to the combined main key index value and the data to be written, storing the target main key and the target data block to the position in the data block area pointed by the address index, and updating the main key index area and the combined main key index area.
In an alternative embodiment, creating a target primary key, storing the target primary key into a position in the data block area pointed by the address index, and updating the primary key index area, by the following steps:
Calculating the size of the main key head of the data to be written, and allocating a third memory block adapted to the size of the main key head of the data to be written from the data block area according to the size of the main key head of the data to be written, wherein the third memory block is located at a position in the data block area pointed by the address index;
Writing the main key head of the data to be written into the third memory block;
and updating the main key index area according to the combined main key index value.
In an alternative embodiment, creating a target data block, storing the target data block to a position in the data block area pointed by the address index, and updating the joint primary key index area, by:
Calculating the size of the data block of the data to be written, and allocating a fourth memory block adaptive to the size of the data block of the data to be written from the data block area according to the size of the data block of the data to be written; writing the data block into which the data is to be written into the fourth memory block, where the fourth memory block is located at a position in the data block area pointed to by the address index;
And updating the joint main key index area according to the joint main key index value.
In an optional implementation manner, the step of determining whether the address index of the data to be written exists in the primary key index area or the joint primary key index area includes:
acquiring a main key head index value and a data block index value of the data to be written according to the combined main key index value;
determining a corresponding first numerical value in the joint main key index area according to the data block index value, judging whether the first numerical value is empty, and if so, judging that the address index of the data to be written does not exist in the joint main key index area; if not, judging that the address index of the data to be written exists in the combined main key index area;
determining a corresponding second array in the main key index area according to the main key head index value, judging whether the second array is empty, and if so, judging that the address index of the data to be written does not exist in the main key index area; if not, judging that the address index of the data to be written exists in the main key index area.
In a second aspect, an embodiment of the present application provides a data processing method, which is applied to a computer device, where a memory of the computer device includes a primary key index area, a joint primary key index area, and a data block area, and the method includes:
Acquiring a combined main key index value of data to be read;
Acquiring data block information of the data to be read according to the combined main key index value of the data to be read and the combined main key index area;
Acquiring main key head information of the data to be read according to the data block information of the data to be read and the main key index area;
And reading the data to be read from the data block area according to the data block information and the main key head information of the data to be read.
in an optional implementation manner, the acquiring, according to the joint primary key index value of the data to be read and the joint primary key index area, data block information of the data to be read includes:
Acquiring data block index information of the data to be read according to the combined main key index value of the data to be read;
Acquiring the absolute address of the data block of the data to be read from the joint main key index area according to the data block index information of the data to be read;
And acquiring the data block information of the data to be read according to the absolute address of the data block of the data to be read.
In an optional implementation manner, the acquiring, according to the data block information of the data to be read and the primary key index area, primary key header information of the data to be read includes:
Acquiring the absolute address of the main key head of the data to be read from the main key index area according to the data block information of the data to be read; and acquiring the main key head information of the data to be read according to the absolute address of the main key head of the data to be read.
In a third aspect, an embodiment of the present application provides a data processing apparatus, which is applied to a computer device, where a memory of the computer device includes a primary key index area, a joint primary key index area, and a data block area, and the apparatus includes:
the obtaining module is used for obtaining a combined main key index value of data to be written;
and the judging module is used for judging whether the address index of the data to be written exists in the main key index area or the combined main key index area according to the combined main key index value, and if so, storing the data to be written into the position in the data block area pointed by the address index.
In a fourth aspect, an embodiment of the present application provides a data processing apparatus, which is applied to a computer device, where a memory of the computer device includes a primary key index area, a joint primary key index area, and a data block area, and the apparatus includes:
the acquisition module is used for acquiring a combined main key index value of data to be read;
The processing module is used for acquiring data block information of the data to be read according to the combined main key index value of the data to be read and the combined main key index area; acquiring main key head information of the data to be read according to the data block information of the data to be read and the main key index area;
And the reading module is used for reading the data to be read from the data block area according to the data block information and the main key head information of the data to be read.
In a fifth aspect, an embodiment of the present application provides a computer device, including a processor and a non-volatile memory storing computer instructions, where the computer instructions, when executed by the processor, cause the computer device to perform the data processing method described in any one of the foregoing embodiments.
In a sixth aspect, an embodiment of the present application provides a readable storage medium, where the readable storage medium includes a computer program, and the computer program controls, when executed, a computer device in the readable storage medium to perform the data processing method described in any one of the foregoing embodiments.
By adopting the data processing method, the data processing device, the computer equipment and the readable storage medium provided by the embodiment of the application, when data is processed, the joint main key index value of the data to be written is obtained firstly, and then whether the address index of the data to be written is stored in the main key index area or the joint main key index area in the memory of the computer equipment is judged according to the joint main key index value of the data to be written.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
fig. 1 is a schematic flowchart illustrating steps of a data processing method according to an embodiment of the present application;
FIG. 2 is a flowchart illustrating a sub-step of step S22 in FIG. 1;
FIG. 3 is a schematic block diagram of a data storage structure provided by an embodiment of the present application;
FIG. 4 is a flowchart illustrating a sub-step of step S222 in FIG. 2;
FIG. 5 is a flowchart illustrating the sub-steps of step S223 in FIG. 2;
Fig. 6 is a schematic flow chart illustrating another step of the data processing method according to the embodiment of the present application;
FIG. 7 is a flowchart illustrating another step of a data processing method according to an embodiment of the present application;
FIG. 8 is a flowchart illustrating another step of a data processing method according to an embodiment of the present application;
FIG. 9 is a flowchart illustrating the sub-steps of step S32 in FIG. 8;
FIG. 10 is a flowchart illustrating the sub-step of step S33 in FIG. 8;
fig. 11 is a schematic block diagram of a structure of a data processing apparatus according to an embodiment of the present application;
FIG. 12 is a schematic block diagram of another structure of a data processing apparatus according to an embodiment of the present application;
Fig. 13 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Icon: 10-memory; 11-primary key index region; 12-a joint primary key index region; 13-data block area; 14-data to be written; 141-primary key head; 142-a data block; 100-a computer device; 110-a data processing device; 1101-an obtaining module; 1102-a judgment module; 1201-an obtaining module; 1202-a processing module; 1203-a reading module; 111-a memory; 112-a processor; 113-communication unit.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.
thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
it should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
Furthermore, the appearances of the terms "first," "second," and the like, if any, are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
it should be noted that the features of the embodiments of the present application may be combined with each other without conflict.
example one
Referring to fig. 1, fig. 1 is a schematic block diagram illustrating a flow of steps of a data processing method according to an embodiment of the present application, where the data processing method is applied to a computer device, and a memory of the computer device includes a primary key index area, a joint primary key index area, and a data block area. It is understood that the computer device in the embodiment of the present application may be an electronic device, a server, or the like capable of performing data processing (e.g., performing data processing for a live broadcast platform). The data processing method includes step S21 and step S22.
and step S21, acquiring the combined primary key index value of the data to be written.
Step S22, according to the value of the associated primary key index, determining whether the address index of the data to be written exists in the primary key index area or the associated primary key index area, and if so, storing the data to be written in a position in the data block area pointed by the address index.
in this embodiment, after the associated primary key index value of the data to be written is obtained, whether the address index of the data to be written exists in the primary key index area or the associated primary key index area may be determined according to the associated primary key index value of the data to be written, and when the address index of the data to be written exists in the primary key index area or the associated primary key index area, the data to be written may be stored in the corresponding position of the data block area to which the address index points, so as to implement processing of a single data to be written.
in this embodiment, the joint primary key index area may be understood as a sub-index area of the primary key index area, address indexes are stored in the joint primary key index area and the primary key index area, respectively, and the data block area may be used to provide a memory block for storing data. The associated primary key index value of the data to be written may include an address index of the data to be written, and the address index of the data to be written included in the associated primary key index value of the data to be written is matched with the address indexes stored in the primary key index area and the associated primary key index area, so that the data to be written may be stored according to a matching result. If the address index of the data to be written exists in the main key index area or the combined main key index area, the data to be written can be stored in the corresponding position in the data block area pointed by the address index. Through the data processing operation, when the data to be written is written into the data block area, the address index of the data to be written is associated with the main key index area or the combined main key index area, so that when a user subsequently performs other operations such as modification, deletion or reading on the data to be written, the data can be quickly positioned and acquired according to the association relationship between the address index and the main key index area or the combined main key index area, and other related data cannot be influenced.
on the basis of the above, please refer to fig. 2 in combination, this embodiment provides an exemplary implementation of step S22.
step S22 may include sub-steps S221 through S223.
and a substep S221 of determining whether the address index of the data to be written exists in the associated primary key index region, if so, executing a substep S222, and if not, executing a substep S223.
And a substep S222, updating the data to be written to the position in the data block area pointed by the address index.
And a substep S223 of determining whether the address index of the data to be written exists in the primary key index region, and if so, writing the data to be written into the position in the data block region pointed by the address index.
In this embodiment, the joint primary key index area may be a sub-index area of the primary key index area, and if the address index of the data to be written exists in the joint primary key index area, a parent index area (i.e., the primary key index area) of the joint primary key index area inevitably exists. Therefore, in order to improve the efficiency of data processing, it may be determined whether the address index of the data to be written exists in the associated primary key index area, and if so, the step of updating the data to be written to the position in the data block area pointed by the address index may be performed.
In the case that the address index of the data to be written exists in the joint primary key index area, the data to be written may be updated to the location of the data block area pointed to by the address index (which may be understood as data existing at the location of the data block area pointed to by the address index is overwritten with the data to be written). Since the address index of the data to be written exists in the joint main key index area, and the joint main key index area can be attached to the corresponding main key index area, the address indexes stored in the main key index area and the joint main key index area do not need to be changed.
when the address index of the data to be written does not exist in the joint main key index area but exists in the main key index area, the step of writing the data to be written to the position in the data block area pointed to by the address index may be performed in view of that it has been determined that the address index of the data to be written is divided into the corresponding main key index areas in the joint main key index value of the data to be written. For example, an address index existing in the joint primary key index area may be newly created under an address index in the primary key index area, and the newly created address index existing in the joint primary key index area is determined according to a value of the joint primary key index of the data to be written, so that after the data to be written is written to a location in the data block area pointed by the address index, the joint primary key index area is updated according to the value of the joint primary key index of the data to be written.
On the basis of the foregoing, referring to fig. 3 in combination, in this embodiment, the data to be written 14 may be composed of a main key header 141 (main key header) and a data block 142(block), where the relationship between the main key header 141 and the data block 142 may correspond to the relationship between the main key index area 11 and the joint main key index area 12, respectively, that is, a location in the data block area 13 pointed by an address index in the main key index area 11 in the memory 10 stores the main key header 141, and a location in the data block area 13 pointed by an address index in the joint main key index area 12 stores the data block 142, where the data block 142 may include real data content of the data to be written 14. For example, in the live broadcasting industry, the primary key header 141 to be written with the data 14 may represent a directory of a main broadcasting "accompanying" business, while the data block 142 to be written with the data 14 may represent a plurality of game types specifically accompanied by the main broadcasting, such as "royal," "turret 2," and "absolutely required," etc., and the primary key header 141 to be written with the data 14 may also specifically represent a specific item in the main broadcasting "accompanying" business, such as "turret 2," at which time the data block 142 to be written with the data 14 may represent the order details of each user under the main broadcasting "turret 2" accompanying business.
on this basis, if the address index of the data to be written 14 does not exist in the primary key index area 11 and the joint primary key index area 12, a target primary key and a target data block are created according to the joint primary key index value and the data to be written 14, the target primary key and the target data block are stored to the position in the data block area 13 pointed by the address index, and the primary key index area 11 and the joint primary key index area 12 are updated.
when the address index of the data to be written 14 exists in neither the associated primary key index area 12 nor the primary key index area 11, an operation of creating a target primary key and a target data block can be performed. Here, the process of creating the target primary key may be understood as adding the address index of the primary key header 141 of the data 14 to be written to the primary key index area 11, and the process of creating the target data block may be understood as adding the address index of the data block 142 of the data 14 to be written to the joint primary key index area 12. Therefore, both the primary key index area 11 and the joint primary key index area 12 need to be updated. For example, the master key header 141 of the data 14 to be written may be used as data related to an item in a host "accompanying" service, and the data block 142 of the data 14 to be written may be order details of each user, in the case where an item "bastion night" is newly added and the item is not previously present, when a user issues an accompanying request for the "bastion night" item to the host for the first time, a new item (the master key header 141 of the data 14 to be written, i.e., a target master key) needs to be created in consideration that the "bastion night" item is not yet present in the host "accompanying" service, and then the order issued by the user (the data block 142 of the data 14 to be written, i.e., a target data block) is added.
in this embodiment, when the address index of the data to be written 14 exists in the associated primary key index area 12, since the original data block may exist in the position of the data block area 13 pointed by the address index of the data to be written 14, and if the data to be written 14 is larger than the original data block, the problem of data writing failure and the like may be caused, in order to ensure the reliability of data storage, a step of comparing the size of the data to be written 14 block with that of the original data block is proposed in this embodiment, please refer to fig. 4, where the step of determining the size of the data to be written 14 block and the size of the original data block may include substeps 2221 to substep S2223.
in the substep S2221, it is determined whether the data block 142 of the data to be written 14 is larger than the data block in the data block area 13 pointed by the address index, and if so, the substep S2222 is performed, and if not, the substep S2223 is performed.
in sub-step S2222, a first memory block with a size that is adaptive to the size of the data block 142 of the data to be written 14 is allocated from the data block region 13, and the data block 142 of the data to be written 14 is written into the first memory block.
In sub-step S2223, the data block 142 of the data to be written 14 is updated to the position in the data block area 13 pointed by the address index.
When the address index of the data to be written 14 exists in the joint primary key index area 12, the operation performed may be understood as covering the data block 142 in the data block area 13 pointed to by the address index of the data to be written 14 with the data to be written 14, at this time, the size of the data block to be written 14 and the size of the data block originally stored therein may be compared, when the data block 142 of the data to be written 14 is larger than the data block in the data block area 13 pointed to by the address index, it is indicated that the data to be written 14 cannot be loaded by the memory block in the data block area 13 pointed to by the address index, in order to ensure reliable storage of the data to be written 14, the data to be written 14 may be stored by dividing the first memory block in the data block area 13, which is adapted to the size of the data to be written 14, it should be understood that, at this time, the position in the data block area 13 pointed to by the address index of the data to be written 14 is the first, and the memory block originally used for storing the original data block returns to the data block area 13 for the next call. When the data block 142 of the data to be written 14 is not larger than the data block in the data block area 13 pointed by the address index, it indicates that the memory block in the data block area 13 pointed by the address index is sufficient to store the data to be written 14, and the data block in the data block area 13 pointed by the address index is directly overwritten with the data to be written 14 without dividing a new memory block from the data block area 13.
for example, when the data 14 to be written is the data related to the "turret 2" item in the "accompany" business performed by the host, the data block 142 of the data 14 to be written may be the details of the order of each user, when the user sends an order including the content "accompany for one hour", after the order is generated, the user wants to modify the order, such as adding a remark "must be divided up", after the modification, the size of the data block 142 of the data 14 to be written changes, and it is necessary to determine the size of the order after adding the remark and the size of the original order, because the modification operation is adding the remark, in such cases, the size of the order after adding the remark is larger than the size of the original order, a memory block adapted to the size of the order after adding the remark needs to be reallocated from the data block area 13 to store the order after adding the remark, and a memory block for storing the original order returns to delete the order into the data block area 13, for the next call.
When the address index of the data 14 to be written does not exist in the associated primary key index area 12 but exists in the primary key index area 11, it is necessary to add the address index of the data block 142 of the data 14 to be written in the associated primary key index area 12, and to achieve the above purpose, this embodiment provides an exemplary implementation scheme of step S223, and as shown in fig. 5, step S223 may include a sub-step S2231 and a sub-step S2232.
In the sub-step S2231, the size of the data block 142 of the data to be written 14 is calculated, and a second memory block adapted to the size of the data block 142 of the data to be written 14 is allocated from the data block area 13 according to the size of the data block 142 of the data to be written 14.
in the sub-step S2232, the data block 142 of the data 14 to be written is written into the second memory block, and the joint primary key index area 12 is updated according to the joint primary key index value.
In this embodiment, when the address index of the data to be written 14 does not exist in the joint main key index area 12 but exists in the main key index area 11, the size of the data block 142 of the data to be written 14 may be calculated first, then a second memory block with an adaptive size is obtained from the data block area 13 according to the calculated size of the data block 142 of the data to be written 14, and the data block 142 of the data to be written 14 is stored by using the second memory block, which may be understood as that an address index is newly added in the joint main key index area 12, and a position in the data block area 13 to which the address index of the data to be written 14 points is the second memory block. After the data block 142 to be written with the data 14 is stored in the second memory block, the associated primary key index area 12 needs to be updated. For example, the primary key head 141 of the data to be written 14 may be used as the associated data of the "turret 2" item in the anchor "accompany" business, and the data block 142 of the data to be written 14 may be the order details of the respective user. It can be understood that a user sends a new "turret 2" accompanied order to the anchor, and at this time, the item "turret 2" (the primary key head 141 to be written with data 14) is already existed originally, and only the order information (the data block 142 to be written with data 14) sent by the user needs to be added.
Referring to fig. 6, when the address index of the data 14 to be written does not exist in the associated primary key index area 12 or the primary key index area 11, the present embodiment provides an exemplary implementation scheme for creating a target primary key, storing the target primary key to a position in the data block area 13 pointed by the address index, and updating the primary key index area 11.
As shown in fig. 6, creating a target primary key, storing the target primary key into the location in the data block area 13 pointed by the address index, and updating the primary key index area 11 may be implemented by the following steps:
In the substep S231, the size of the primary key head 141 of the data to be written 14 is calculated, and a third memory block adapted to the size of the primary key head 141 of the data to be written 14 is allocated from the data block region 13 according to the size of the primary key head 141 of the data to be written 14.
in the substep S232, the primary key head 141 of the data 14 to be written is written into the third memory block.
and a substep S233, updating the primary key index area 11 according to the joint primary key index value.
In this embodiment, the length of the data 14 to be written may be obtained first, the length of the primary key head 141 of the data 14 to be written is obtained from the length of the data 14 to be written, then the size of the primary key head 141 of the data 14 to be written is obtained according to the length of the primary key head 141 of the data 14 to be written, a third memory block adapted to the size of the primary key head 141 of the data 14 to be written may be allocated from the data block area 13 according to the size of the primary key head 141 of the data 14 to be written, and the primary key head 141 of the data 14 to be written is written into the third memory block, so that the address index of the data 14 to be written may be added to the primary key index area 11, and the position of the address index in the data block area 13 pointed to by the primary key index area 11 is the third memory block.
referring to fig. 7, when the address index of the data to be written 14 exists in neither the joint primary key index area 12 nor the primary key index area 11, the present embodiment provides an exemplary implementation scheme of creating a target data block, storing the target data block to a position in the data block area 13 pointed by the address index, and updating the joint primary key index area 12.
As shown in fig. 7, creating a target data block, storing the target data block to the position in the data block area 13 pointed to by the address index, and updating the joint primary key index area 12 may be implemented through steps S234 to S236.
in the substep S234, the size of the data block 142 of the data to be written 14 is calculated, and a fourth memory block adapted to the size of the data block 142 of the data to be written 14 is allocated from the data block region 13 according to the size of the data block 142 of the data to be written 14.
In the substep S235, the data block 142 of the data 14 to be written is written into the fourth memory block.
And a substep S236, updating the joint primary key index region 12 according to the joint primary key index value.
In this embodiment, the length of the data 14 to be written may be obtained first, the length of the data block 142 of the data 14 to be written is obtained from the length of the data 14 to be written, then the size of the data block 142 of the data 14 to be written is obtained by calculating according to the length of the data block 142 of the data 14 to be written, a fourth memory block adapted to the data block is allocated from the data block area 13 according to the size of the data block 142 of the data 14 to be written, the data block 142 of the data 14 to be written is written into the fourth memory block, and then the address index of the data 14 to be written may be added to the joint main key index area 12, where the position of the address index in the data block area 13 pointed to in the joint main key index area 12 is the fourth memory block. In this embodiment, the address indexes of the data to be written 14 in the main key index area 11 and the joint main key index area 12 are obtained according to the joint main key index value of the data to be written 14, so that the address index of the data to be written 14 in the main key index area 11 and the address index of the data to be written 14 in the joint main key index area 12 in the joint main key index value have a relationship between a parent level and a child level.
In this embodiment, there are various ways to determine whether the address index of the to-be-written data 14 exists in the primary key index area 11 or the joint primary key index area 12, and this embodiment provides one of the exemplary determination schemes.
judging whether the address index of the data to be written 14 exists in the primary key index area 11 or the joint primary key index area 12 can be realized by the following modes: and acquiring the index value of the main key head and the index value of the data block of the data to be written 14 according to the combined index value of the main key. Determining a corresponding first numerical value in the joint main key index area 12 according to the data block index value, judging whether the first numerical value is empty, and if so, judging that the address index of the data to be written 14 does not exist in the joint main key index area 12; if not, the address index of the data to be written 14 is determined to exist in the joint primary key index area 12. Determining a corresponding second array in the primary key index area 11 according to the primary key head index value, judging whether the second array is empty, and if so, judging that the address index of the data 14 to be written does not exist in the primary key index area 11; if not, the address index of the data to be written 14 is determined to exist in the primary key index area 11.
in this embodiment, the joint primary key index value of the data to be written 14 may be obtained according to an index function corresponding to a specific scenario, for example, the joint primary key index value of the data to be written 14 may be a joint primary key hash (hash) value, and the joint primary key hash value of the data to be written 14 may be obtained by using a hash function "std:: hash < std:: string > { } (str)" (index function), where "str" is a combined character string of a primary key (primary key header 141) and a secondary key (data block 142) included in the data to be written 14, and the combined character string may be obtained according to a binary character string converted from the primary key header 141 and the data block 142 of the data to be written 14. Correspondingly, the joint primary key index value may include a primary key header hash value and a data block hash value of the data 14 to be written, and the memory 10 of the computer device may also be divided into a primary key hash index area, a joint primary key hash index area, and a data block area 13. In this case, it may be determined whether the address index of the data to be written 14 exists in the joint primary key hash index region, that is, whether an array determined in the joint primary key hash index region by the data block hash value of the data to be written 14 is empty is determined, and if the array is empty (that is, 0), it indicates that the address index of the data to be written 14 does not exist in the joint primary key hash index region. If not null (i.e., a value other than 0), the address index of the data 14 to be written may be considered to exist in the associated primary key hash index region, and the value represents the mapping of the chunk hash value to the chunk 142 address. Similarly, the determining whether the address index of the data to be written 14 exists in the primary key hash index area may be determining whether a value determined by the primary key header hash value of the data to be written 14 in the primary key hash index area is null, if so (i.e., 0), it indicates that the address index of the data to be written 14 does not exist in the primary key hash index area, and if not (i.e., a value not 0), it may be considered that the address index of the data to be written 14 exists in the primary key hash index area, and the value represents the mapping from the primary key header hash value to the primary key header address. By adopting the method, the writing position of the data to be written 14 can be accurately determined, the efficiency of subsequent data processing can be improved, and the problem that the operation processing cannot be carried out independently because all data are packed together is solved.
It should be understood that the above implementation manner is only an example, for example, in other implementation manners of this embodiment, in addition to the hash index, a B-tree index (tree index) or the like may be used to implement the processing of the data to be written 14.
example two
In this embodiment, a flow of data reading will be described. Specifically, the present embodiment provides a data processing method for reading data, please refer to fig. 8, in which the data reading process specifically includes steps S31 to S34.
step S31, a joint primary key index value of the data to be read is obtained.
Step S32, obtaining the data block information of the data to be read according to the associated primary key index value of the data to be read and the associated primary key index area 12.
Step S33, obtaining the main key header information of the data to be read according to the data block information of the data to be read and the main key index area 11.
step S34, reading the data to be read from the data block area 13 according to the data block information and the main header information of the data to be read.
in this embodiment, the step of executing data reading may be to first obtain a joint primary key index value of the data to be read, and then obtain data block information of the data to be read according to the joint primary key index value and the joint primary key index area 12, where the data block information may include related information about a primary key header address, and may obtain the primary key header information of the data to be read according to the data block information of the data to be read and the primary key index area 11, and after obtaining the data block information and the primary key header information, the data to be read may be read from the data block area 13.
on the basis, referring to fig. 9, the step of acquiring the data block information of the data to be read may include substeps S321 to S323.
And a substep S321, obtaining data block index information of the data to be read according to the associated primary key index value of the data to be read.
in the substep S322, the absolute address of the data block 142 of the data to be read is obtained from the associated primary key index area 12 according to the data block index information of the data to be read.
in the substep S323, the data block information of the data to be read is obtained according to the absolute address of the data block 142 of the data to be read.
In this embodiment, the associated primary key index value of the data to be read may include data block index information of the data to be read, and according to the data block index information of the data to be read, the relative address of the data block 142 of the data to be read may be found from the associated primary key index area 12, and according to the obtained relative address of the data block 142, the relative address may be further converted into an absolute address of the data block 142 of the data to be read, so that a data link table header tagListBlockHead (i.e., data block information) of the data to be read may be obtained, specifically, the data link table header may be represented as:
Further, referring to fig. 10, the step of acquiring the main header information of the data to be read includes substeps S331 and 332.
in the substep S331, the absolute address of the main key header 141 of the data to be read is obtained from the main key index area 11 according to the data block information of the data to be read.
And a substep S332, obtaining the main key header information of the data to be read according to the absolute address of the main key header 141 of the data to be read.
In this embodiment, the relative address (iMainKey) of the primary key header 141 of the data to be read can be obtained from the aforementioned data link table header tagListBlockHead (i.e. data block information), and is converted into an absolute address, and then the primary key header information can be obtained by jumping from the primary key index area 11 to the pointed position in the data block area 13 according to the absolute address of the primary key header 141 to be read:
After the data block information and the primary key header information are obtained, the data information of the data to be read can be obtained according to the cData [0] and the iDataLen, and the data to be read which is desired by the user can be obtained by unpacking the data information.
EXAMPLE III
Referring to fig. 11, in order to perform the corresponding steps in the above-described embodiment and various possible manners, an implementation manner of the data processing apparatus 110 is given below, where the data processing apparatus 110 includes:
an obtaining module 1101 is configured to obtain a joint primary key index value of the data 14 to be written.
A determining module 1102, configured to determine, according to the value of the associated primary key index, whether an address index of the to-be-written data 14 exists in the primary key index area 11 or the associated primary key index area 12, and if so, store the to-be-written data 14 to a position in the data block area 13 pointed by the address index.
In this embodiment, a data processing apparatus 110 may be provided, where the data processing apparatus 110 may be configured to implement an operation of writing data 14 to be written into a memory 10 of a computer device, and an implementation principle of the data processing apparatus 110 is the same as an implementation principle of the data processing method when processing the data 14 to be written, and is not described herein again.
example four
Referring to fig. 12, in order to perform the corresponding steps in the above-described embodiment and various possible manners, an implementation manner of the data processing apparatus 110 is given below, where the data processing apparatus 110 includes:
An obtaining module 1201, configured to obtain a combined primary key index value of data to be read.
A processing module 1202, configured to obtain data block information of the data to be read according to the associated primary key index value of the data to be read and the associated primary key index area 12; and acquiring the main key head information of the data to be read according to the data block information of the data to be read and the main key index area 11.
A reading module 1203, configured to read the data to be read from the data block area 13 according to the data block information and the main header information of the data to be read.
in this embodiment, a data processing apparatus 110 may be provided, where the data processing apparatus 110 may be configured to implement an operation of reading data to be read from a memory 10 of a computer device, and an implementation principle of the data processing apparatus 110 is the same as an implementation principle of the data processing method when processing the data to be read, and is not described herein again.
EXAMPLE five
Referring to fig. 13, fig. 13 is a schematic structural diagram of a computer device 100 according to an embodiment of the present disclosure. The embodiment of the present application provides a computer device 100, the computer device 100 is in communication connection with a terminal device, the computer device 100 includes a processor and a non-volatile memory storing computer instructions, and when the computer instructions are executed by the processor, the computer device 100 executes the aforementioned data processing method.
The computer device 100 comprises a data processing means 110, a memory 111, a processor 112 and a communication unit 113.
The memory 111, the processor 112 and the communication unit 113 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The data processing device 110 includes at least one software functional module which can be stored in the memory 111 in the form of software or firmware (firmware) or solidified in an Operating System (OS) of the computer apparatus 100. The processor 112 is used for executing executable modules stored in the memory 111, such as software functional modules and computer programs included in the data processing device 110.
The Memory 111 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like. The memory 111 is used to store programs or data.
EXAMPLE six
an embodiment of the present application provides a readable storage medium, where the readable storage medium includes a computer program, and the computer program controls, when running, the computer device 100 where the readable storage medium is located to execute the foregoing data processing method.
In summary, embodiments of the present application provide a data processing method, an apparatus, a computer device, and a readable storage medium, which determine whether an address index of data to be written exists in a primary key index area or a joint primary key index area according to a joint primary key index value of the data to be written, and store the data to be written in a position in a data block area to which the address index points according to a determination result. By the data processing method, the data processing device, the computer equipment and the readable storage medium, data are processed, and the problem of inconvenience in data processing caused by direct packaging of the data during storage is solved.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (15)
1. A data processing method is applied to computer equipment, wherein a memory of the computer equipment comprises a main key index area, a joint main key index area and a data block area, and the method comprises the following steps:
Acquiring a combined primary key index value of data to be written;
and judging whether the address index of the data to be written exists in the main key index area or the combined main key index area according to the combined main key index value, and if so, storing the data to be written into the position of the data block area pointed by the address index.
2. the method according to claim 1, wherein the step of determining whether the address index of the data to be written exists in the primary key index area or the joint primary key index area, and if so, storing the data to be written to a position in the data block area pointed to by the address index comprises:
Judging whether the address index of the data to be written exists in the combined main key index area, if so, updating the data to be written to the position in the data block area pointed by the address index; if not, judging whether the address index of the data to be written exists in the main key index area, if so, writing the data to be written into the position in the data block area pointed by the address index, and updating the joint main key index area according to the joint main key index value.
3. the method according to claim 2, wherein the step of updating the data to be written to the location in the data block area pointed to by the address index comprises:
Judging whether the data block of the data to be written is larger than the data block in the data block area pointed by the address index, if so, allocating a first memory block adaptive to the size of the data block of the data to be written from the data block area, and writing the data block of the data to be written into the first memory block; and if not, updating the data block of the data to be written to the position in the data block area pointed by the address index.
4. The method of claim 2, wherein the step of writing the data to be written to the location in the data block area pointed to by the address index comprises:
calculating the size of the data block of the data to be written, and allocating a second memory block adaptive to the size of the data block of the data to be written from the data block area according to the size of the data block of the data to be written; and writing the data block into which the data is to be written into the second memory block, where the second memory block is located at a position in the data block area pointed to by the address index.
5. the method of claim 1, further comprising: if the address index of the data to be written does not exist in the main key index area and the combined main key index area, creating a target main key and a target data block according to the combined main key index value and the data to be written, storing the target main key and the target data block to the position in the data block area pointed by the address index, and updating the main key index area and the combined main key index area.
6. The method of claim 5, wherein the steps of creating a target primary key, storing the target primary key in a location in the data block area pointed to by the address index, and updating the primary key index area are performed by:
calculating the size of the main key head of the data to be written, and allocating a third memory block adapted to the size of the main key head of the data to be written from the data block area according to the size of the main key head of the data to be written, wherein the third memory block is located at a position in the data block area pointed by the address index;
Writing the main key head of the data to be written into the third memory block;
And updating the main key index area according to the combined main key index value.
7. The method of claim 5, wherein creating a target data block, storing the target data block to a location in the data block area pointed to by the address index, and updating the joint primary key index area is performed by:
Calculating the size of the data block of the data to be written, and allocating a fourth memory block adaptive to the size of the data block of the data to be written from the data block area according to the size of the data block of the data to be written; writing the data block into which the data is to be written into the fourth memory block, where the fourth memory block is located at a position in the data block area pointed to by the address index;
And updating the joint main key index area according to the joint main key index value.
8. The method according to claim 5, wherein the step of determining whether the address index of the data to be written exists in the primary key index area or the joint primary key index area comprises:
Acquiring a main key head index value and a data block index value of the data to be written according to the combined main key index value;
Determining a corresponding first numerical value in the joint main key index area according to the data block index value, judging whether the first numerical value is empty, and if so, judging that the address index of the data to be written does not exist in the joint main key index area; if not, judging that the address index of the data to be written exists in the combined main key index area;
Determining a corresponding second array in the main key index area according to the main key head index value, judging whether the second array is empty, and if so, judging that the address index of the data to be written does not exist in the main key index area; if not, judging that the address index of the data to be written exists in the main key index area.
9. A data processing method is applied to computer equipment, wherein a memory of the computer equipment comprises a main key index area, a joint main key index area and a data block area, and the method comprises the following steps:
acquiring a combined main key index value of data to be read;
Acquiring data block information of the data to be read according to the combined main key index value of the data to be read and the combined main key index area;
Acquiring main key head information of the data to be read according to the data block information of the data to be read and the main key index area;
And reading the data to be read from the data block area according to the data block information and the main key head information of the data to be read.
10. the method according to claim 9, wherein the obtaining the data block information of the data to be read according to the joint primary key index value of the data to be read and the joint primary key index area comprises:
Acquiring data block index information of the data to be read according to the combined main key index value of the data to be read;
Acquiring the absolute address of the data block of the data to be read from the joint main key index area according to the data block index information of the data to be read;
And acquiring the data block information of the data to be read according to the absolute address of the data block of the data to be read.
11. the method according to claim 9, wherein the obtaining the primary key header information of the data to be read according to the data block information of the data to be read and the primary key index area comprises:
Acquiring the absolute address of the main key head of the data to be read from the main key index area according to the data block information of the data to be read;
and acquiring the main key head information of the data to be read according to the absolute address of the main key head of the data to be read.
12. A data processing apparatus, applied to a computer device, a memory of the computer device including a primary key index area, a joint primary key index area, and a data block area, the apparatus comprising:
the obtaining module is used for obtaining a combined main key index value of data to be written;
And the judging module is used for judging whether the address index of the data to be written exists in the main key index area or the combined main key index area according to the combined main key index value, and if so, storing the data to be written into the position in the data block area pointed by the address index.
13. a data processing apparatus, applied to a computer device, a memory of the computer device including a primary key index area, a joint primary key index area, and a data block area, the apparatus comprising:
The acquisition module is used for acquiring a combined main key index value of data to be read;
the processing module is used for acquiring data block information of the data to be read according to the combined main key index value of the data to be read and the combined main key index area; acquiring main key head information of the data to be read according to the data block information of the data to be read and the main key index area;
and the reading module is used for reading the data to be read from the data block area according to the data block information and the main key head information of the data to be read.
14. a computer device comprising a processor and a non-volatile memory having stored thereon computer instructions which, when executed by the processor, cause the computer device to perform the data processing method of any one of claims 1 to 11.
15. A readable storage medium, characterized in that the readable storage medium comprises a computer program which, when running, controls a computer device on which the readable storage medium is located to perform the data processing method of any one of claims 1-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910861161.2A CN110569398A (en) | 2019-09-09 | 2019-09-09 | data processing method and device, computer equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910861161.2A CN110569398A (en) | 2019-09-09 | 2019-09-09 | data processing method and device, computer equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110569398A true CN110569398A (en) | 2019-12-13 |
Family
ID=68779356
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910861161.2A Pending CN110569398A (en) | 2019-09-09 | 2019-09-09 | data processing method and device, computer equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110569398A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111352594A (en) * | 2020-03-12 | 2020-06-30 | 上海路虹电子科技有限公司 | Method and device for writing data and reading data in eFuse |
| CN112506918A (en) * | 2020-11-03 | 2021-03-16 | 深圳市宏电技术股份有限公司 | Data access method, terminal and computer readable storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101178733A (en) * | 2007-12-07 | 2008-05-14 | 彭海杰 | Index, search, storage and display control information systems for associated data |
| US20100083003A1 (en) * | 2008-09-26 | 2010-04-01 | Spackman Stephen P | Method and apparatus for non-redundant encrypted storage |
| CN102779180A (en) * | 2012-06-29 | 2012-11-14 | 华为技术有限公司 | Operation processing method of data storage system and data storage system |
| CN103186537A (en) * | 2011-12-27 | 2013-07-03 | 腾讯科技（北京）有限公司 | Data reading-writing method and device based on index sorting |
| CN103186542A (en) * | 2011-12-27 | 2013-07-03 | 腾讯科技（北京）有限公司 | Data querying method and device |
| CN103455631A (en) * | 2013-09-22 | 2013-12-18 | 广州中国科学院软件应用技术研究所 | Method, device and system for processing data |
| CN103473314A (en) * | 2013-09-11 | 2013-12-25 | 北京思特奇信息技术股份有限公司 | Key value pair storing method and device based on shared memory |
| CN104268272A (en) * | 2014-10-14 | 2015-01-07 | 青岛海尔软件有限公司 | Method and device for checking uniqueness of data |
| CN107273556A (en) * | 2017-08-23 | 2017-10-20 | 上海点融信息科技有限责任公司 | Block chain data index method and equipment |
| CN110188096A (en) * | 2019-04-18 | 2019-08-30 | 阿里巴巴集团控股有限公司 | A kind of index creation method, device and equipment of data record |
-
2019
- 2019-09-09 CN CN201910861161.2A patent/CN110569398A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101178733A (en) * | 2007-12-07 | 2008-05-14 | 彭海杰 | Index, search, storage and display control information systems for associated data |
| US20100083003A1 (en) * | 2008-09-26 | 2010-04-01 | Spackman Stephen P | Method and apparatus for non-redundant encrypted storage |
| CN103186537A (en) * | 2011-12-27 | 2013-07-03 | 腾讯科技（北京）有限公司 | Data reading-writing method and device based on index sorting |
| CN103186542A (en) * | 2011-12-27 | 2013-07-03 | 腾讯科技（北京）有限公司 | Data querying method and device |
| CN102779180A (en) * | 2012-06-29 | 2012-11-14 | 华为技术有限公司 | Operation processing method of data storage system and data storage system |
| CN103473314A (en) * | 2013-09-11 | 2013-12-25 | 北京思特奇信息技术股份有限公司 | Key value pair storing method and device based on shared memory |
| CN103455631A (en) * | 2013-09-22 | 2013-12-18 | 广州中国科学院软件应用技术研究所 | Method, device and system for processing data |
| CN104268272A (en) * | 2014-10-14 | 2015-01-07 | 青岛海尔软件有限公司 | Method and device for checking uniqueness of data |
| CN107273556A (en) * | 2017-08-23 | 2017-10-20 | 上海点融信息科技有限责任公司 | Block chain data index method and equipment |
| CN110188096A (en) * | 2019-04-18 | 2019-08-30 | 阿里巴巴集团控股有限公司 | A kind of index creation method, device and equipment of data record |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111352594A (en) * | 2020-03-12 | 2020-06-30 | 上海路虹电子科技有限公司 | Method and device for writing data and reading data in eFuse |
| CN111352594B (en) * | 2020-03-12 | 2023-06-20 | 湖州旻合科技有限公司 | Method and device for writing data and reading data in eFuse |
| CN112506918A (en) * | 2020-11-03 | 2021-03-16 | 深圳市宏电技术股份有限公司 | Data access method, terminal and computer readable storage medium |
| CN112506918B (en) * | 2020-11-03 | 2024-06-04 | 深圳市宏电技术股份有限公司 | Data access method, terminal and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7136748B2 (en) | Map data processing apparatus and center system | |
| CN108536745B (en) | Shell-based data table extraction method, terminal, equipment and storage medium | |
| US20140115571A1 (en) | Electronic device, non-transient readable medium and method thereof | |
| JP3686564B2 (en) | Database system, database replica generation method, and computer-readable recording medium recording database replica generation program | |
| CN107783770B (en) | Page configuration updating method, device, server and medium | |
| US10166467B2 (en) | Information processing device, data structure of game data, and recording medium | |
| CN110569398A (en) | data processing method and device, computer equipment and readable storage medium | |
| CN112181471A (en) | Differential upgrading method and device, storage medium and computer equipment | |
| CN113504918A (en) | Equipment tree configuration optimization method and device, computer equipment and storage medium | |
| US9971789B2 (en) | Selective disk volume cloning for virtual disk creation | |
| CN114756333A (en) | Mirror image processing method and device, electronic equipment and readable medium | |
| CN112463026A (en) | Method and apparatus for deduplication of supplemental data in a distributed object storage system | |
| US7548927B2 (en) | Abstracted metadata policy component and related architecture | |
| CN113986828A (en) | Method and device for storing mass files, electronic equipment and storage medium | |
| CN113688139A (en) | Object storage method, gateway, device and medium | |
| CN117112004A (en) | Differential data determining method, differential restoring device, differential data determining equipment, differential data restoring equipment and differential data medium | |
| CN116955278A (en) | Aggregation access method and device for distributed file system snapshot and computer equipment | |
| CN108052294B (en) | Modification writing method and modification writing system of distributed storage system | |
| US20120221612A1 (en) | Real-Time Data Transformation to Access Foreign Data Sources | |
| CN107908634B (en) | Cache control method of browser and mobile terminal | |
| CN110708355A (en) | File uploading method, system, device and readable storage medium | |
| CN112347046A (en) | Method for acquiring creation time of file in distributed system | |
| CN111143293A (en) | Metadata acquisition method, device, equipment and computer readable storage medium | |
| CN112650723B (en) | File sharing method, device, equipment and computer readable storage medium | |
| CN110543575A (en) | media file retrieval method and device of vehicle-mounted external storage equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191213 |
|
| RJ01 | Rejection of invention patent application after publication |