CN115994145B - Method and device for processing data - Google Patents
Method and device for processing data Download PDFInfo
- Publication number
- CN115994145B CN115994145B CN202310111816.0A CN202310111816A CN115994145B CN 115994145 B CN115994145 B CN 115994145B CN 202310111816 A CN202310111816 A CN 202310111816A CN 115994145 B CN115994145 B CN 115994145B
- Authority
- CN
- China
- Prior art keywords
- data
- index
- node
- index tree
- nodes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 115
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000004044 response Effects 0.000 claims abstract description 11
- 238000004590 computer program Methods 0.000 claims description 10
- 230000000903 blocking effect Effects 0.000 claims description 3
- 239000000758 substrate Substances 0.000 claims 1
- 230000000875 corresponding effect Effects 0.000 description 59
- 238000010586 diagram Methods 0.000 description 12
- 238000012360 testing method Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 11
- 230000008859 change Effects 0.000 description 10
- 230000006870 function Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a device for processing data, and relates to the technical field of big data. One embodiment of the method comprises the following steps: the method comprises the steps that in response to a data processing request, an index tree matched with an index field contained in data is determined from a plurality of preset index trees, wherein each index tree is stored in an external memory, and node information of each node in the index tree comprises own node information and associated nodes; positioning one or more target nodes corresponding to the field values to be processed, which are included in the data processing request, according to the field values included in each node in the index tree; and positioning the data in the data source through the target node and processing the positioned data. The embodiment of the invention improves the stability and efficiency of data processing and improves the user experience.
Description
Technical Field
The present invention relates to the field of big data technologies, and in particular, to a method and an apparatus for processing data.
Background
At present, big data technology is widely applied to internet application, and as the types of services contained in the internet application are more and more complex, the requirements on the complexity of data processing are also higher and higher.
In the prior art, data is directly inquired or processed from a data source based on SQL sentences, and when the data is accessed at high frequency, the performance loss is larger due to network delay; when data are placed in a local memory, the excessive data volume can possibly influence garbage collection of an application to cause performance degradation, and a commonly used key-value data access mode is difficult to meet diversified data access requirements of users, so that user experience is influenced.
Disclosure of Invention
In view of this, an embodiment of the present invention provides a method and an apparatus for processing data, capable of determining, in response to a data processing request, an index tree matching an index field included in data from a plurality of types of index trees set in advance, where each index tree is stored in an external memory, and node information of each node in the index tree includes own node information and associated nodes; positioning one or more target nodes corresponding to the field values to be processed, which are included in the data processing request, according to the field values included in each node in the index tree; and positioning the data in the data source through the target node and processing the positioned data. The embodiment of the invention improves the stability and efficiency of data processing and improves the user experience.
To achieve the above object, according to one aspect of the embodiments of the present invention, there is provided a method of processing data, including: in response to a data processing request, determining a first index tree matched with an index field from a plurality of preset index trees under the condition that the data processing request indicates the index field, wherein each type of index tree comprises nodes including field values of the same index field; the first index tree is stored in a memory space acquired from an external memory, node information of each node contained in the first index tree is stored in a storage block in the memory space, wherein the node information contains self node information and associated nodes with association relation with the nodes in a data structure of the first index tree; positioning one or more target nodes corresponding to the field values to be processed, which are included in the data processing request, according to the field values included in each node in the first index tree; and positioning data in a data source by using the one or more searched target nodes, and processing the positioned data.
Optionally, the method for processing data includes: determining a user-defined index type, wherein the index type comprises one or more index fields; acquiring field values of one or more index fields included in the index type from the data source; and constructing an index tree by using the acquired field values of one or more index fields.
Optionally, the method for processing data further includes: acquiring a primary key of data from the data source; the constructing an index tree by using the acquired field values of one or more index fields includes: and constructing an index tree by using the acquired field values of one or more index fields and the primary key values of the primary keys as nodes of the index tree.
Optionally, the constructing an index tree further includes: acquiring a memory space for storing the index tree from an external memory; for each node contained in the index tree, performing: acquiring a storage block for storing the node from the memory space; and storing node information of the nodes by using the storage block, wherein the node information comprises self node information and associated nodes with association relation with the nodes in a data structure of the index tree.
Optionally, the method for processing data further includes: receiving a piece of newly-added data, wherein a primary key value contained in the newly-added data does not exist in the data source; for the newly added data, the following operations are executed: adding the newly added data to the data source; analyzing one or more index fields from the newly added data, and correspondingly sending the newly added data to a message queue; and acquiring the newly added data from the message queue by using each second index tree associated with the data source, and adding nodes corresponding to the newly added data in the second index tree, wherein each second index tree performs sorting based on the newly added nodes.
Optionally, the method for processing data further includes:
receiving a piece of newly-added data, wherein a primary key value contained in the newly-added data exists in the data source; for the newly added data, the following operations are executed: replacing original data matched with the primary key value in the data source with the newly added data; analyzing one or more index fields from the newly added data, and sending the replaced original data and the newly added data to a message queue; and acquiring the replaced original data and the newly added data from the message queue by using each second index tree associated with the data source, deleting the node corresponding to the original data in the second index tree, and adding the node corresponding to the newly added data, wherein each second index tree respectively performs sorting based on the node corresponding to the deleted original data and the newly added node.
Optionally, the method for processing data further includes: recording the times of updating data in the data source to a message queue by using a first counter, and accumulating and recording the times of finishing updating nodes in the second index tree according to the message queue by using a second counter; and judging whether the product of the first counter and the total number of the index fields is equal to the second counter or not, and if so, determining that each index tree corresponding to the data source is in a ready state.
Optionally, the method for processing data further includes: and under the condition that the product of the first counter and the total number of the index fields is not equal to the second counter, blocking the data query request aiming at the newly added data until the record value of the second counter is equal to the record value of the first counter.
Optionally, said locating data in the data source using the one or more target nodes found includes: and acquiring a node value of the target node, analyzing a key value corresponding to a primary key from the node value, and inquiring from the data source according to the key value to obtain the data inquired by the data processing request.
To achieve the above object, according to a second aspect of an embodiment of the present invention, there is provided an apparatus for processing data, comprising: determining an index tree module, a positioning node module and a data processing module; wherein,,
the index tree determining module is used for determining a first index tree matched with the index field from a plurality of preset index trees when the data processing request indicates the index field in response to the data processing request, wherein nodes included in each type of index tree comprise field values of the same index field or the same index field combination; the first index tree is stored in a memory space acquired from an external memory, node information of each node contained in the first index tree is stored in a storage block in the memory space, wherein the node information contains self node information and associated nodes with association relation with the nodes in a data structure of the first index tree;
the positioning node module is configured to position, according to a field value included in each node in the first index tree, one or more target nodes corresponding to a field value to be processed included in the data processing request;
And the data processing module is used for positioning the data in the data source by using the one or more searched target nodes and processing the positioned data.
Optionally, the data processing device is configured to determine a user-defined index type, where the index type includes one or more index fields; acquiring field values of one or more index fields included in the index type from the data source; and constructing an index tree by using the acquired field values of one or more index fields.
Optionally, the device for processing data is further configured to obtain a primary key of data from the data source; the constructing an index tree by using the acquired field values of one or more index fields includes: and constructing an index tree by using the acquired field values of one or more index fields and the primary key values of the primary keys as nodes of the index tree.
Optionally, the data processing device is further configured to obtain a memory space storing the index tree in an off-heap memory; for each node contained in the index tree, performing: acquiring a storage block for storing the node from the memory space; and storing node information of the nodes by using the storage block, wherein the node information comprises self node information and associated nodes with association relation with the nodes in a data structure of the index tree.
Optionally, the device for processing data is further configured to receive a piece of new data, where a primary key value included in the new data is not present in the data source; for the newly added data, the following operations are executed: adding the newly added data to the data source; analyzing one or more index fields from the newly added data, and correspondingly sending the newly added data to a message queue; and acquiring the newly added data from the message queue by using each second index tree associated with the data source, and adding nodes corresponding to the newly added data in the second index tree, wherein each second index tree performs sorting based on the newly added nodes.
Optionally, the device for processing data is further configured to receive a piece of new data, where a primary key value included in the new data already exists in the data source; for the newly added data, the following operations are executed: replacing original data matched with the primary key value in the data source with the newly added data; analyzing one or more index fields from the newly added data, and sending the replaced original data and the newly added data to a message queue; and acquiring the replaced original data and the newly added data from the message queue by using each second index tree associated with the data source, deleting the node corresponding to the original data in the second index tree, and adding the node corresponding to the newly added data, wherein each second index tree is used for respectively executing sorting based on the node corresponding to the deleted original data and the newly added node.
Optionally, the data processing device is further configured to record, with a first counter, the number of times of updating data in the data source to a message queue, and record, with a second counter, the number of times that each of the second index trees completes updating nodes in the second index tree according to the message queue; and judging whether the product of the first counter and the total number of the index fields is equal to the second counter or not, and if so, determining that each index tree corresponding to the data source is in a ready state.
Optionally, the data processing device is configured to block the data query request for the new data until the record value of the second counter is equal to the record value of the first counter, where the product of the first counter and the total number of index fields is not equal to the second counter.
Optionally, the means for processing data is configured to locate data in a data source by using the one or more target nodes that are found, including: and acquiring a node value of the target node, analyzing a key value corresponding to a primary key from the node value, and inquiring from the data source according to the key value to obtain the data inquired by the data processing request.
In order to achieve the above object, according to a third aspect of an embodiment of the present invention, there is provided an electronic device that processes data, comprising: one or more processors; and a storage means for storing one or more programs that, when executed by the one or more processors, cause the one or more processors to implement the method of any of the methods of processing data described above.
To achieve the above object, according to a fourth aspect of embodiments of the present invention, there is provided a computer-readable medium having stored thereon a computer program, characterized in that the program, when executed by a processor, implements a method as described in any one of the above methods of processing data.
One embodiment of the above invention has the following advantages or benefits: the method comprises the steps that in response to a data processing request, an index tree matched with an index field contained in data is determined from a plurality of preset index trees, wherein each index tree is stored in an external memory, and node information of each node in the index tree comprises own node information and associated nodes; positioning one or more target nodes corresponding to the field values to be processed, which are included in the data processing request, according to the field values included in each node in the index tree; and locating data in the data source through the target node and processing the located data. The embodiment of the invention improves the stability and the efficiency of data processing and improves the user experience.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a flow chart of a method of processing data according to an embodiment of the present invention;
FIG. 2 is a flow chart of processing newly added data according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an index tree according to one embodiment of the present invention;
FIG. 4 is a schematic diagram of an apparatus for processing data according to an embodiment of the present invention;
FIG. 5 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
fig. 6 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
As shown in fig. 1, an embodiment of the present invention provides a method for processing data, which may include the following steps:
step S101: in response to a data processing request, determining a first index tree matched with an index field from a plurality of preset index trees under the condition that the data processing request indicates the index field, wherein each type of index tree comprises nodes including field values of the same index field.
Specifically, according to the application scenario of the user to the data, an index may be set for one or more fields contained in the data: for example, the data object is a Test, and the data structure thereof is as follows, and includes fields such as field1, field2, field3, field4, etc.;
public class Test{
String field1,
String field2,
String field3,
String field4,}
illustratively, the specific data corresponding to Test is: test { "1", "2", "3", "4" }, test { "A", "B", "C", "D" }, test { "Q", "W", "E", "R" }, etc.; it will be appreciated that the structure of Test is merely an example, and in a practical application scenario, the number of fields is large, and the order of magnitude of data is also large. The invention does not limit the data structure, the data quantity and the data content of the data to be processed.
Further, multiple indexes can be created based on fields according to user requirements so as to improve the query efficiency of the data by utilizing the indexes, wherein the multiple indexes are mutually independent; each index type may contain a number of fields: for example: field1 as an index field is one index type, two index fields of field3 and field4 are combined into another index type, two index fields of field3 and field1 are combined into another index type, etc.; further, each index type is correspondingly provided with an index tree of a corresponding type; i.e. preset with a plurality of types of index tree; when a data request is processed, determining a first index tree matched with an index field from a plurality of preset index trees according to the index field indicated by the data request; for example: the index field indicated by the data processing request is "field1", and then a first index tree matching "field1" is determined from among the preset multiple types of index trees.
In one embodiment of the present invention, in a case where it is determined that the data processing request indicates an index field, determining a first index tree matching the index field from among a plurality of types of index trees set in advance; the method for determining that the data processing request indicates the index field can determine whether one or more fields contained in the query request are index fields by querying a data dictionary of a data source, or directly indicate the one or more fields to be index fields by utilizing an identifier related to the index in the data query request; further similarly, in the case that the query by the primary key is indicated in the data processing request, one or more primary keys are resolved from the data processing request, and the data is directly queried and processed from the data source by the primary key and the data query condition contained in the data processing request; the method comprises the steps of directly inquiring a piece of matched data by using a primary key, and inquiring one or more pieces of data by using an index field.
Further, for one index tree, the index fields to which the field values of the index fields included in each node of the index tree belong are identical, and it is understood that, in the case that one index field is a combination of a plurality of fields, the field values corresponding to the index fields of the category are formed by splicing the field values of the plurality of fields. Taking a plurality of Test data as an example, for example, field1 as an index field corresponds to one index tree 1, each node of the index tree 1 contains a field value corresponding to field1, for example, 3 nodes correspondingly contain node values respectively "1", "a" and "W", that is, field values respectively of field1 fields; for example, for a plurality of Test data, field3 and field4 are combined as index fields to correspond to one index tree 2, and each node of the index tree 2 contains a combination of index values corresponding to field3 and field4, for example, 3 nodes correspondingly contain node values of "3-4", "C-D", "Y-Z", and the like, respectively; that is, each type of index tree includes nodes that include field values that include the same index field.
It can be seen that, for the type of index field contained in one data source (e.g., data table, hash table, etc.), a plurality of index trees may be correspondingly provided to store field values corresponding to different kinds of index fields through different types of index trees. Preferably, the data structure of the index tree is a red black tree. The red-black tree is a self-balancing binary search tree, and the balance of the binary search tree is kept through specific operation when the insertion and deletion operation of the data nodes is carried out, so that higher data query performance is obtained.
One or more index types corresponding to the data source are set by a user according to the application scene; wherein the index type may contain one or more index fields, such as: field1 as an index field is one index type, and two index fields of field3 and field4 are combined into another index type; further, acquiring a primary key of data from the data source; the constructing an index tree by using the acquired field values of one or more index fields includes: and constructing an index tree by using the acquired field values of one or more index fields and the primary key values of the primary keys as nodes of the index tree.
Further, the index type may contain one or more primary key fields in addition to the index field; by flexibly setting the index type, the flexibility and the efficiency of processing data are improved. Further, under the condition that the index type further comprises a main key, a main key value of the main key is obtained, and an index tree is constructed by using the obtained field values of one or more index fields and the main key value of the main key.
In an embodiment of the present invention, the method of constructing the index tree may be performed dynamically, for example: based on an empty index tree component, the data of the data source is changed along with the processing and other operations of each piece of newly added data, and the index tree is correspondingly changed and updated. The construction of the index tree may also be based on traversing data already present in the data source. After the construction, data processing such as new data and changed data is correspondingly performed.
Taking a Test data object as an example, assuming that a primary key field is a joint primary key formed by field1 and field2, and an index field is field3, constructing a node value of a node of an index tree as '3-1_2' for one data Test { '1', '2', '3', '4' }; it is understood that the node value may be obtained by concatenating the field value of the index field and the primary key value of the primary key, where the method of concatenation may be by way of one or more symbols, and the "-", "_" symbols used in embodiments of the present invention are merely examples. Similarly, an index tree diagram as shown in fig. 3 is constructed from the acquired pieces of data. It will be appreciated that the user may change the index type, and in the event that a user-defined index type change is determined, the index tree corresponding to the index type is correspondingly changed.
Further, applying memory space for the index tree from the off-heap memory to store the index tree; the off-heap memory corresponds to the in-heap memory, and the off-heap memory is a memory which is allocated outside the heap of the Java virtual machine for the memory object; namely, acquiring a memory space for storing the index tree from an external memory; in the project of developing applications by using Java, non-empty objects are distributed to Java virtual machines and are generally put into in-heap memory, and the Java virtual machines can regularly recycle garbage; for the in-pile memory, the influence of garbage collection every time is positively correlated with the size of the in-pile memory; if a large amount of data is stored in the memory, the garbage collection strategy of the Java virtual machine works abnormally to cause the abrupt attenuation of the application performance; the method comprises the steps that relative to the in-heap memory, the out-heap memory is a memory except a Java virtual machine, and is managed by an operating system and an application program; compared with the in-heap memory, the out-heap memory has the advantages of reducing garbage collection of the Java virtual machine and accelerating data copying. The embodiment of the invention can carry out memory management on one or more index trees by utilizing the off-heap memory aiming at Java application, so that the application runs with stable processing efficiency. The stability of processing data is improved, and the efficiency of processing data (data inquiry, data change and the like) is further improved.
It may be understood that, in the embodiment of the present invention, each index tree (including the first index tree and the second index tree) is stored in the external memory, and when the index tree is stored, the overall structure of the index tree needs to be saved, so that, first, a continuous memory space is applied for the index tree from the external memory, then, a plurality of storage blocks are divided in this memory space, each storage block is used to store node information of a node of the index tree, and when each node of the index tree is stored to the storage block, in addition to storing a value of an index type corresponding to the node, a parent node, a left and right sibling node, a current node color (red or black, etc.), a node value length, a node value (which is a combination of the index value and a primary key value, etc.), a tree data structure of the index tree is stored by storing own node information and associated nodes (parent node, left and right sibling node) of each node, that is, the node information includes an associated node having an association relationship with the node in the data structure of the index tree. Wherein the node value length is related to the length of the node value, i.e. the node value length is variable, and thus the size of the memory block of the memory node may also be variable. That is, the constructing an index tree further includes: acquiring a memory space for storing the index tree from an external memory; for each node contained in the index tree, performing: acquiring a storage block for storing the node from the memory space; and storing node information of the nodes by using the storage block, wherein the node information comprises self node information and associated nodes with association relation with the nodes in a data structure of the index tree. That is, the first index tree is stored in a memory space acquired from an off-heap memory, node information of each node included in the first index tree is stored in a storage block in the memory space, where the node information includes own node information and an associated node having an association relationship with the node in a data structure of the first index tree.
Step S102: and positioning one or more target nodes corresponding to the field values to be processed, which are included in the data processing request, according to the field values included in each node in the first index tree.
Specifically, when data query is performed, a target node matching the data query condition is searched from the corresponding first index tree according to a field, a field value and the like contained in the query condition contained in the received data request.
Taking the schematic diagram of fig. 3 as an example, assuming that the field value of the index contained in the field value to be processed in the data query request is "1", according to the field values of the indexes contained in each node of the index tree, according to the sequence of traversing the nodes by the red-black tree, three matched target nodes "1-1_0", "1-1_2" and "1-1_3" are found out; that is, one or more target nodes corresponding to the field values to be processed included in the data processing request are located; it can be understood that according to the order of the red and black trees, when the nodes are traversed, under the condition that the index value of the node is judged to be greater than 1, whether the traversal is finished or not can be determined, namely, all the nodes of the index tree do not need to be traversed, so that the efficiency of positioning data is improved.
Step S103: and positioning data in a data source by using the one or more searched target nodes, and processing the positioned data.
Specifically, after the target node is positioned, the data in the data source is further positioned according to the target node; still referring to the example of step S102, assuming that the node values of the three target nodes are "1-1_0", "1-1_2" and "1-1_3", if the data query request includes "1_2" as the primary key value of the primary key of the data to be queried, the "1_0", "1-1_2" and "1_1_3" as the primary key value of the primary key are resolved from the node values of the three target nodes, and further, the corresponding data is obtained from the data source (e.g. hash table) according to the primary key value of the primary key, so as to obtain the data queried by the data processing request. It will be appreciated that the data stored in the hash table is all the fields (including index field, primary key field, and other fields other than index field, primary key field, etc.) corresponding to the data source; that is, the locating data in the data source using the one or more target nodes found includes: and acquiring a node value of the target node, analyzing a key value corresponding to a primary key from the node value, and inquiring from a data source (such as a hash table) according to the key value to obtain the data inquired by the data processing request.
Therefore, the data storage, query and other operations of the embodiment of the invention all occur in the memory outside the heap, thereby achieving the beneficial effect of optimal execution efficiency of single operation.
Further, in the case of a data change, if the changed data is associated with a node of the index tree, the index tree node is correspondingly changed; that is, when the type of the data processing request indicates that the data is changed, the method includes: positioning the data to be changed from the hash table according to the primary key of the changed data, and executing data change; and judging whether the field value included in the target node of the change data is changed, and if so, changing the target node. When the target node in the index tree is changed, the original node is deleted, and the node corresponding to the updated data is updated by a new node adding method, so that the changed node can be automatically adjusted in position by the red-black tree model to achieve the balance of the red-black tree structure.
Further, processing the positioned data; the operation on the data includes: modifying data, deleting data, etc.; further, one or more pieces of data obtained through positioning can be subjected to statistical analysis and other processing to realize data query and data visualization.
As shown in fig. 2, an embodiment of the present invention provides a process for processing newly added data, where the process may include the following steps:
step S201: receiving a new piece of added data; and judging whether the primary key value contained in the newly added data exists in the data source. If not, step S202 is performed, otherwise step S203 is performed.
In one embodiment of the invention, the received data is considered "newly added data" for the case where an index tree, or data source, is built for the dynamically received data.
Step S202: adding the newly added data to the data source; analyzing one or more index fields from the newly added data, and correspondingly sending the newly added data to a message queue; and acquiring the newly added data from the message queue by using each second index tree associated with the data source, and adding nodes corresponding to the newly added data in the second index tree, wherein each second index tree performs sorting based on the newly added nodes.
Specifically, when receiving a piece of newly added data and judging that the main key value of the newly added data does not exist in the data source, firstly storing the newly added data into a hash table (such as ConcurrentHashMap or HashMap, etc.), and further correspondingly sending the newly added data to a message queue.
Further, the new data is obtained from the message queue based on the message associated with the new data by using each second index tree associated with the data source, and nodes corresponding to the new data are added in the second index tree. That is, in the case where the data source corresponds to an index tree having a plurality of types (i.e., a plurality of second index trees), each of the second index trees needs to add the newly added data as its own node correspondingly for the newly added data. And each of the second index trees performs ordering based on the newly added nodes.
Taking the index tree diagram shown in fig. 3 as an example, assume that the index tree shown in fig. 3 is a second index tree to which newly added data is matched, and for example, the newly added data is Test { "1", "2", "9", "0" }; the index tree is newly added with a node, and the node value of the added node corresponding to the newly added data is '9-1_2'; it can be understood that the storage block is used for storing node information corresponding to the newly added node; the description of storing the node information corresponding to the newly added node by using the storage block is identical to the description of step S101, and will not be repeated here.
Step S203: replacing original data matched with the primary key value in the data source with the newly added data; analyzing one or more index fields from the newly added data, and sending the replaced original data and the newly added data to a message queue; and acquiring the replaced original data and the newly added data from the message queue by using each second index tree associated with the data source, deleting the node corresponding to the original data in the second index tree, and adding the node corresponding to the newly added data, wherein each second index tree is used for respectively executing sorting based on the node corresponding to the deleted original data and the newly added node.
Specifically, when a piece of newly added data is received and it is judged that a primary key value of the newly added data already exists in a data source, the newly added data is firstly replaced in original data matched with the primary key value in the data source.
Further, one or more index fields are parsed from the newly added data, and the replaced original data and the newly added data are sent to a message queue.
Further, each second index tree associated with a data source is utilized to acquire the replaced original data and the newly added data from the message queue, the node corresponding to the original data is deleted in the second index tree, and the node corresponding to the newly added data is added. And each of the second index trees performs ordering based on the newly added nodes. And each second index tree performs sorting based on deleting the node corresponding to the original data and the newly added node.
It can be seen that when one or more pieces of new added data are received, the new added data need to be added to the corresponding index tree according to the index type while being stored to the data source (hash table). In general, the efficiency of adding nodes corresponding to newly added data of an index tree is lower than the speed of adding the newly added data to a hash table, and operations on the hash table can support multiple concurrencies, while operations on the index tree do not support concurrency, so that the index tree is asynchronously updated according to the embodiment of the invention.
Preferably, in one embodiment of the present invention, a first counter is used to record the number of times of updating data in the data source to a message queue, and a second counter is used to record the number of times that each second index tree completes updating nodes in the second index tree according to the message queue in an accumulating manner; and judging whether the product of the first counter and the total number of the index fields is equal to the second counter or not, and if so, determining that each index tree corresponding to the data source is in a ready state.
Specifically, each time the newly added data includes an index field, an index change message needs to be sent to the message queue, and the first counter is incremented by 1. When the new data is changed data, an index deletion is sent, the new index inserts two pieces of information into the message queue, and the first counter is increased twice. It will be appreciated that if the data source does not have a corresponding index tree, i.e. no index is established in the data source (which only requires Key-Value Key queries to handle the business requirements of data processing), no index change information will be sent and the first counter will not change. After each second index tree acquires index change information, maintenance of the index of the second index tree is performed, and 1 is added to the second counter after the maintenance is completed. Only when the first counter=the second counter indicates the total number of indexes (also the number of total index trees), the index query function is provided on behalf of the index being ready. And judging whether the product of the first counter and the total number of the index fields is equal to the second counter or not, and if so, determining that each index tree corresponding to the data source is in a ready state. Otherwise, under the condition that the product of the first counter and the total number of the index fields is not equal to the second counter, blocking the data query request aiming at the newly added data until the record value of the second counter is equal to the record value of the first counter.
Therefore, by adding the first counter and the second counter, the function of counting the lock is realized, and the situation that when the index is not updated, the data is equivalent to the hash table, but the data cannot be found according to the index tree, so that the result of the query data provided for the data query request is wrong is prevented. By setting the counter, the consistency and reliability of the data are improved.
Further, after the end of node addition of the index tree is judged, resources occupied by the processing index are released. The resources occupied by the indexes can be threads, and each index type is provided with a corresponding thread.
Therefore, for various types of operations (adding, modifying, updating, deleting, etc.) of the data, it is necessary to first perform corresponding processing in the data source, then parse one or more index fields contained in the data, and send the information of the field Guan Suoyin to the message queue, at this time, a processing result may be returned to the application, and further, each index tree corresponding to the data source may asynchronously acquire the information corresponding to the index field from the message queue, so as to complete updating of the node of the index tree itself. For newly added data, if a data primary key of the newly added data exists in the data total source, when the newly added data is inserted into the data source, the original data is replaced by the newly added data, and at the moment, two pieces of information are sent to a message queue: covered data information, newly added data information. Each second index tree corresponding to the data source processes the information one by one, deletes the index node corresponding to the information of the covered data (replaced data) from the index tree, and finishes the ordering of the self nodes; inserting new index nodes for the newly added data information, and completing the ordering of the nodes. If the data main key of the newly added data does not exist in the data total source, the newly added data is directly added to the data source, and then the information of the newly added data is sent to each second index tree corresponding to the data source. Each second index tree can newly add index nodes according to the data information and complete the ordering of the nodes. If the data is deleted, deleting the data from the data source, and then deleting the node corresponding to the deleted data in the index tree of each second index tree.
As shown in fig. 4, an embodiment of the present invention provides an apparatus 400 for processing data, including: a determine index tree module 401, a locate node module 402, and a process data module 403; wherein,,
the index tree determining module 401 is configured to determine, in response to a data processing request, a first index tree matching with an index field from a plurality of preset index trees when determining that the data processing request indicates the index field, where nodes included in each type of index tree include field values of the same index field or the same index field combination; the first index tree is stored in a memory space acquired from an external memory, node information of each node contained in the first index tree is stored in a storage block in the memory space, wherein the node information contains self node information and associated nodes with association relation with the nodes in a data structure of the first index tree;
the positioning node module 402 is configured to position, according to field values included in each node in the first index tree, one or more target nodes corresponding to field values to be processed included in the data processing request;
The data processing module 403 is configured to locate data in a data source by using the one or more found target nodes, and process the located data.
The embodiment of the invention also provides an electronic device for processing data, which comprises: one or more processors; and a storage device for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the method provided by any of the embodiments described above.
The embodiment of the invention also provides a computer readable medium, on which a computer program is stored, which when executed by a processor implements the method provided by any of the above embodiments.
Fig. 5 illustrates an exemplary system architecture 500 to which a method of processing data or an apparatus of processing data of an embodiment of the present invention may be applied.
As shown in fig. 5, the system architecture 500 may include terminal devices 501, 502, 503, a network 504, and a server 505. The network 504 is used as a medium to provide communication links between the terminal devices 501, 502, 503 and the server 505. The network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 505 via the network 504 using the terminal devices 501, 502, 503 to receive or send messages or the like. Various client applications such as an electronic mall client application, a web browser application, a search class application, an instant messaging tool, a mailbox client, and the like may be installed on the terminal devices 501, 502, 503.
The terminal devices 501, 502, 503 may be a variety of electronic devices having a display screen and supporting a variety of client applications, including but not limited to smartphones, tablet computers, laptop and desktop computers, and the like.
The server 505 may be a server providing various services, such as a background management server providing support for client applications used by the user with the terminal devices 501, 502, 503. The background management server can process the received data processing request and feed back the result obtained by the data processing request to the terminal equipment.
It should be noted that, the method for processing data provided by the embodiment of the present invention is generally performed by the server 505, and accordingly, the device for processing data is generally disposed in the server 505.
It should be understood that the number of terminal devices, networks and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 6, there is illustrated a schematic diagram of a computer system 600 suitable for use in implementing an embodiment of the present invention. The terminal device shown in fig. 6 is only an example, and should not impose any limitation on the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, mouse, etc.; an output portion 607 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The drive 610 is also connected to the I/O interface 605 as needed. Removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on drive 610 so that a computer program read therefrom is installed as needed into storage section 608.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 609, and/or installed from the removable medium 611. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 601.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules and/or units involved in the embodiments of the present invention may be implemented in software, or may be implemented in hardware. The described modules and/or units may also be provided in a processor, e.g., may be described as: a processor includes a determine index tree module, a locate node module, and a process data module. The names of these modules do not in any way constitute a limitation of the module itself, for example, a process data module may also be described as a "module for locating data in a data source using one or more of the target nodes found".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include: in response to a data processing request, determining a first index tree matched with an index field from a plurality of preset index trees under the condition that the data processing request indicates the index field, wherein each type of index tree comprises nodes including field values of the same index field; the first index tree is stored in a memory space acquired from an external memory, node information of each node contained in the first index tree is stored in a storage block in the memory space, wherein the node information contains self node information and associated nodes with association relation with the nodes in a data structure of the first index tree; positioning one or more target nodes corresponding to the field values to be processed, which are included in the data processing request, according to the field values included in each node in the first index tree; and positioning data in a data source by using the one or more searched target nodes, and processing the positioned data.
According to the embodiment of the invention, in response to a data processing request, an index tree matched with an index field contained in data can be determined from a plurality of preset index trees, wherein each index tree is stored in an external memory, and node information of each node in the index tree comprises own node information and associated nodes; positioning one or more target nodes corresponding to the field values to be processed, which are included in the data processing request, according to the field values included in each node in the index tree; and locating data in the data source through the target node and processing the located data. The embodiment of the invention improves the data processing efficiency and the user experience.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. A method of processing data, comprising:
In response to a data processing request, determining a first index tree matched with an index field from a plurality of preset index trees under the condition that the data processing request indicates the index field, wherein each type of index tree comprises nodes including field values of the same index field;
the first index tree is stored in a memory space acquired from an external memory, node information of each node contained in the first index tree is stored in a storage block in the memory space, wherein the node information contains self node information and associated nodes with association relation with the nodes in a data structure of the first index tree;
positioning one or more target nodes corresponding to the field values to be processed, which are included in the data processing request, according to the field values included in each node in the first index tree;
positioning data in a data source by using one or more searched target nodes, and processing the positioned data;
the method further comprises the steps of: determining a user-defined index type, wherein the index type comprises one or more index fields; acquiring field values of one or more index fields included in the index type from the data source; constructing an index tree by using the acquired field values of one or more index fields; wherein, the constructing the index tree further comprises: acquiring a memory space for storing the index tree from an external memory; for each node contained in the index tree, performing: acquiring a storage block for storing the node from the memory space; and storing node information of the nodes by using the storage block, wherein the node information comprises self node information and associated nodes with association relation with the nodes in a data structure of the index tree.
2. The method of claim 1, wherein the step of determining the position of the substrate comprises,
further comprises: acquiring a primary key of data from the data source;
the constructing an index tree by using the acquired field values of one or more index fields includes: and constructing an index tree by using the acquired field values of one or more index fields and the primary key values of the primary keys as nodes of the index tree.
3. The method as recited in claim 1, further comprising:
receiving a piece of newly-added data, wherein a primary key value contained in the newly-added data does not exist in the data source;
for the newly added data, the following operations are executed:
adding the newly added data to the data source;
analyzing one or more index fields from the newly added data, and correspondingly sending the newly added data to a message queue;
and acquiring the newly added data from the message queue by using each second index tree associated with the data source, and adding nodes corresponding to the newly added data in the second index tree, wherein each second index tree performs sorting based on the newly added nodes.
4. The method as recited in claim 1, further comprising:
Receiving a piece of newly-added data, wherein a primary key value contained in the newly-added data exists in the data source;
for the newly added data, the following operations are executed:
replacing original data matched with the primary key value in the data source with the newly added data;
analyzing one or more index fields from the newly added data, and sending the replaced original data and the newly added data to a message queue;
and acquiring the replaced original data and the newly added data from the message queue by using each second index tree associated with the data source, deleting the node corresponding to the original data in the second index tree, and adding the node corresponding to the newly added data, wherein each second index tree respectively performs sorting based on the node corresponding to the deleted original data and the newly added node.
5. The method according to claim 3 or 4, wherein,
further comprises: the number of updates to the message queue in the data source is recorded using a first counter,
accumulating and recording the times of finishing updating the nodes in the second index tree according to the message queue by using a second counter;
And judging whether the product of the first counter and the total number of the index fields is equal to the second counter or not, and if so, determining that each index tree corresponding to the data source is in a ready state.
6. The method as recited in claim 5, further comprising:
and under the condition that the product of the first counter and the total number of the index fields is not equal to the second counter, blocking the data query request aiming at the newly added data until the record value of the second counter is equal to the record value of the first counter.
7. The method of claim 1, wherein locating data in a data source using the one or more target nodes found comprises:
and acquiring a node value of the target node, analyzing a key value corresponding to a primary key from the node value, and inquiring from the data source according to the key value to obtain the data inquired by the data processing request.
8. An apparatus for processing data, comprising: determining an index tree module, a positioning node module and a data processing module; wherein,,
the index tree determining module is used for determining a first index tree matched with the index field from a plurality of preset index trees when the data processing request indicates the index field in response to the data processing request, wherein nodes included in each type of index tree comprise field values of the same index field or the same index field combination; the first index tree is stored in a memory space acquired from an external memory, node information of each node contained in the first index tree is stored in a storage block in the memory space, wherein the node information contains self node information and associated nodes with association relation with the nodes in a data structure of the first index tree;
The positioning node module is configured to position, according to a field value included in each node in the first index tree, one or more target nodes corresponding to a field value to be processed included in the data processing request;
the data processing module is used for positioning data in a data source by using one or more searched target nodes and processing the positioned data;
the determining index tree module further includes: determining a user-defined index type, wherein the index type comprises one or more index fields; acquiring field values of one or more index fields included in the index type from the data source; constructing an index tree by using the acquired field values of one or more index fields; wherein, the constructing the index tree further comprises: acquiring a memory space for storing the index tree from an external memory; for each node contained in the index tree, performing: acquiring a storage block for storing the node from the memory space; and storing node information of the nodes by using the storage block, wherein the node information comprises self node information and associated nodes with association relation with the nodes in a data structure of the index tree.
9. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs,
when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-7.
10. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310111816.0A CN115994145B (en) | 2023-02-09 | 2023-02-09 | Method and device for processing data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310111816.0A CN115994145B (en) | 2023-02-09 | 2023-02-09 | Method and device for processing data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115994145A CN115994145A (en) | 2023-04-21 |
| CN115994145B true CN115994145B (en) | 2023-08-22 |
Family
ID=85991931
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310111816.0A Active CN115994145B (en) | 2023-02-09 | 2023-02-09 | Method and device for processing data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115994145B (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101315628A (en) * | 2007-06-01 | 2008-12-03 | 华为技术有限公司 | Internal memory database system and method and device for implementing internal memory data base |
| CN103339624A (en) * | 2010-12-14 | 2013-10-02 | 加利福尼亚大学董事会 | High efficiency prefix search algorithm supporting interactive, fuzzy search on geographical structured data |
| CN107533518A (en) * | 2015-01-20 | 2018-01-02 | 乌尔特拉塔有限责任公司 | Distributed index for fault tolerant object memory construction |
| CN108885604A (en) * | 2015-12-08 | 2018-11-23 | 乌尔特拉塔有限责任公司 | Memory construction software implement scheme |
| CN110019211A (en) * | 2017-11-27 | 2019-07-16 | 北京京东尚科信息技术有限公司 | The methods, devices and systems of association index |
| CN111061680A (en) * | 2018-10-15 | 2020-04-24 | 北京京东尚科信息技术有限公司 | Data retrieval method and device |
| CN112883125A (en) * | 2021-04-28 | 2021-06-01 | 北京奇岱松科技有限公司 | Entity data processing method, device, equipment and storage medium |
| CN113297204A (en) * | 2020-07-15 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Index generation method and device |
| CN113312539A (en) * | 2021-06-10 | 2021-08-27 | 北京百度网讯科技有限公司 | Method, device, equipment and medium for providing retrieval service |
| US11210596B1 (en) * | 2020-11-06 | 2021-12-28 | issuerPixel Inc. a Nevada C. Corp | Self-building hierarchically indexed multimedia database |
| CN114461746A (en) * | 2021-12-31 | 2022-05-10 | 安徽科大讯飞医疗信息技术有限公司武汉分公司 | Method for searching data and search engine system |
| CN114579561A (en) * | 2020-12-02 | 2022-06-03 | 北京金山云网络技术有限公司 | Data processing method and device, and storage medium |
| CN115510058A (en) * | 2022-09-22 | 2022-12-23 | 彩讯科技股份有限公司 | Address book information searching method, device, equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101266358B1 (en) * | 2008-12-22 | 2013-05-22 | 한국전자통신연구원 | A distributed index system based on multi-length signature files and method thereof |
-
2023
- 2023-02-09 CN CN202310111816.0A patent/CN115994145B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101315628A (en) * | 2007-06-01 | 2008-12-03 | 华为技术有限公司 | Internal memory database system and method and device for implementing internal memory data base |
| CN103339624A (en) * | 2010-12-14 | 2013-10-02 | 加利福尼亚大学董事会 | High efficiency prefix search algorithm supporting interactive, fuzzy search on geographical structured data |
| CN107533518A (en) * | 2015-01-20 | 2018-01-02 | 乌尔特拉塔有限责任公司 | Distributed index for fault tolerant object memory construction |
| CN108885604A (en) * | 2015-12-08 | 2018-11-23 | 乌尔特拉塔有限责任公司 | Memory construction software implement scheme |
| CN110019211A (en) * | 2017-11-27 | 2019-07-16 | 北京京东尚科信息技术有限公司 | The methods, devices and systems of association index |
| CN111061680A (en) * | 2018-10-15 | 2020-04-24 | 北京京东尚科信息技术有限公司 | Data retrieval method and device |
| CN113297204A (en) * | 2020-07-15 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Index generation method and device |
| US11210596B1 (en) * | 2020-11-06 | 2021-12-28 | issuerPixel Inc. a Nevada C. Corp | Self-building hierarchically indexed multimedia database |
| CN114579561A (en) * | 2020-12-02 | 2022-06-03 | 北京金山云网络技术有限公司 | Data processing method and device, and storage medium |
| CN112883125A (en) * | 2021-04-28 | 2021-06-01 | 北京奇岱松科技有限公司 | Entity data processing method, device, equipment and storage medium |
| CN113312539A (en) * | 2021-06-10 | 2021-08-27 | 北京百度网讯科技有限公司 | Method, device, equipment and medium for providing retrieval service |
| CN114461746A (en) * | 2021-12-31 | 2022-05-10 | 安徽科大讯飞医疗信息技术有限公司武汉分公司 | Method for searching data and search engine system |
| CN115510058A (en) * | 2022-09-22 | 2022-12-23 | 彩讯科技股份有限公司 | Address book information searching method, device, equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 面向应用集成的内存数据索引算法的实现;张金等;《计算机工程与科学》;第71-74页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115994145A (en) | 2023-04-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110019211A (en) | The methods, devices and systems of association index | |
| CN112445626B (en) | Data processing method and device based on message middleware | |
| CN111597148B (en) | Distributed metadata management method for distributed file system | |
| CN111061680A (en) | Data retrieval method and device | |
| CN110807028B (en) | Method, apparatus and computer program product for managing a storage system | |
| CN111241189B (en) | Method and device for synchronizing data | |
| CN111381820A (en) | Method and device for automatically generating API based on GUI | |
| CN112579695A (en) | Data synchronization method and device | |
| CN113760948A (en) | Data query method and device | |
| CN110909022A (en) | Data query method and device | |
| CN111858586A (en) | Data processing method and device | |
| CN108959294B (en) | Method and device for accessing search engine | |
| CN113704245A (en) | Database main key generation method, sub-table positioning method and device | |
| CN111753019B (en) | Data partitioning method and device applied to data warehouse | |
| CN111427899A (en) | Method, device, equipment and computer readable medium for storing file | |
| CN112182138A (en) | Catalog making method and device | |
| CN112100168A (en) | Method and device for determining data association relationship | |
| CN110851343A (en) | Test method and device based on decision tree | |
| CN111159207B (en) | Information processing method and device | |
| CN113760969A (en) | Data query method and device based on elastic search | |
| CN115994145B (en) | Method and device for processing data | |
| CN108733668B (en) | Method and device for querying data | |
| CN110109919B (en) | Method and device for determining logic information | |
| CN110858199A (en) | Document data distributed computing method and device | |
| CN113722113A (en) | Traffic statistic method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |