CN110222064A - Data processing method, device, electronic equipment and readable storage medium storing program for executing - Google Patents
Data processing method, device, electronic equipment and readable storage medium storing program for executing Download PDFInfo
- Publication number
- CN110222064A CN110222064A CN201910530224.6A CN201910530224A CN110222064A CN 110222064 A CN110222064 A CN 110222064A CN 201910530224 A CN201910530224 A CN 201910530224A CN 110222064 A CN110222064 A CN 110222064A
- Authority
- CN
- China
- Prior art keywords
- data
- data processing
- mantissa
- field
- routing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 34
- 238000012545 processing Methods 0.000 claims abstract description 128
- 230000006870 function Effects 0.000 claims description 12
- 238000004891 communication Methods 0.000 claims description 9
- 238000013507 mapping Methods 0.000 abstract description 25
- 230000004044 response Effects 0.000 abstract description 6
- 238000005516 engineering process Methods 0.000 abstract description 4
- 238000000034 method Methods 0.000 description 17
- 238000010586 diagram Methods 0.000 description 13
- 230000003287 optical effect Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- 230000000977 initiatory effect Effects 0.000 description 3
- 238000000638 solvent extraction Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 239000003999 initiator Substances 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
In the data processing method that the embodiment of the present disclosure provides, device, in electronic equipment and readable storage medium storing program for executing, data processing request is received first, the data processing request includes the Data Identification of requested date, the first processing then is carried out to the preset field of the Data Identification, obtain the routing mantissa of the requested date, divide library total amount according to the routing mantissa and database after again, determine that point library in the target point library for being stored with the requested date identifies, it is finally identified according to point library in the target point library and the data processing request is forwarded to the target point library.Data are carried out using mapping relations in compared with the existing technology to divide for the acquisition in library, and the response time for data processing request can be effectively reduced using aforesaid way, reduce the operating cost of storage equipment, improve the flexibility of query processing.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of data processing, and in particular, to a data processing method and apparatus, an electronic device, and a readable storage medium.
Background
With the development of online service technology, the amount of data in the database is increasingly large, and the processing efficiency of data processing such as data query, storage, reading and the like in the database is reduced.
In the prior art, in order to improve the efficiency of data processing and avoid the excessive operating pressure borne by the database, a database-based method is generally adopted to store, query, read and the like data. Specifically, the mapping relation database is arranged to store mapping relations between the routing keys of the data and the database numbers. The mapping relation database responds to the query request of each data, and feeds back the data sub-database number of the stored data to the upstream by using the mapping relation, so that the processing of storing, querying, reading and the like of the data is realized.
However, in such a processing method, firstly, an additional storage device needs to be occupied to store the mapping relationship, and secondly, the mapping relationship database needs to respond to a large number of data processing requests and perform corresponding processing, so that the response efficiency is low.
Disclosure of Invention
In order to solve the above problem, embodiments of the present disclosure provide a data processing method and apparatus, an electronic device, and a readable storage medium.
In a first aspect, an embodiment of the present disclosure provides a data processing method, including:
receiving a data processing request, wherein the data processing request comprises a data identifier of requested data;
performing first processing on a preset field of the data identifier to obtain a routing mantissa of the requested data;
determining the sub-database identification of the target sub-database storing the requested data according to the routing mantissa and the total sub-database amount of the database;
and forwarding the data processing request to the target sub-library according to the sub-library identification of the target sub-library.
In a second aspect, an embodiment of the present disclosure provides a data processing apparatus, including:
the communication unit is used for receiving a data processing request, and the data processing request comprises a data identifier of requested data;
the first processing unit is used for carrying out first processing on the preset field of the data identifier to obtain the routing mantissa of the requested data;
the second processing unit is used for determining the sub-database identification of the target sub-database storing the requested data according to the routing mantissa and the total sub-database amount of the database;
the communication unit is further configured to forward the data processing request to the target sub-library according to the sub-library identifier of the target sub-library.
In a third aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the data processing method of any one of the preceding claims.
In a fourth aspect, the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored, and when a processor executes the computer-executable instructions, the data processing method according to any one of the preceding claims is implemented.
In the data processing method, the data processing apparatus, the electronic device, and the readable storage medium provided in this embodiment, a data processing request is first received, where the data processing request includes a data identifier of requested data, then a preset field of the data identifier is first processed to obtain a routing mantissa of the requested data, then a branch base identifier of a target branch base in which the requested data is stored is determined according to the routing mantissa and a total branch base amount of a database, and finally the data processing request is forwarded to the target branch base according to the branch base identifier of the target branch base. Compared with the prior art that the mapping relation is used for acquiring the data sub-databases, the method can effectively reduce the response time to the data processing request, reduce the operation cost of the storage device and improve the flexibility of query processing.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present disclosure, and for those skilled in the art, other drawings can be obtained according to the drawings without inventive exercise.
FIG. 1 is a diagram of a network architecture based on the prior art;
FIG. 2 is a schematic diagram of a network architecture upon which the present disclosure is based;
FIG. 3 is a schematic flow chart diagram of a data processing method provided by the present disclosure;
fig. 4 is a schematic structural diagram of a data processing method apparatus provided in the present disclosure;
fig. 5 is a schematic diagram of a hardware structure of an electronic device provided in the present disclosure.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are some, but not all embodiments of the present disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
With the development of online service technology, the amount of data in the database is increasingly large, and the processing efficiency of data processing such as data query, storage, reading and the like in the database is reduced.
In the prior art, in order to improve the efficiency of data processing and avoid the excessive operating pressure borne by the database, a database-based method is generally adopted to store, query, read and the like data. Fig. 1 is a schematic diagram of a network architecture based on the prior art, and as shown in fig. 1, the prior data processing is generally implemented based on route mapping, and the architecture includes, but is not limited to, a request initiator 1, a mapping relation database 2, and a data sub-database 3.
The request initiating terminal 1 may specifically be a user terminal, a server of an application service, and other hardware entity devices or server clusters that need to perform data interaction with the database. The user terminal includes but is not limited to a smart phone, a platform computer, a desktop computer, etc.; the servers of the application service include, but are not limited to, game servers, search engine servers, advertisement platform servers, and the like.
The mapping relation database 2 and the data sub-database 3 may be specifically hardware device-based storage units, which may be cloud storage units or local storage units; in addition, the two devices may belong to the same hardware device or different hardware devices. The mapping relation database 2 stores the mapping relation between the routing key of each data and the database number, wherein the routing key refers to a character segment or a character string including a data identity and a data storage address. When storing data into the data sub-database 3, the data sub-database 3 feeds back a corresponding data sub-database number to the mapping relation database 2, wherein the data sub-database number is used for indicating the position of the data sub-database 3 where the data is stored. The mapping relation database 2 divides the routing key storing each data and the database number where the data is located into banks in order to perform data processing such as query, storage, and reading on the data. Therefore, when the request initiator 1 initiates a data processing request, the mapping relation database 2 will respond to the request, obtain the corresponding data sub-database number of the data by using mapping relation query, and forward the request to the corresponding data sub-database 3 for processing.
For example, for mysql database, for the convenience of scene application, data query and other processes are generally adopted based on data id. Generally, when data is stored, the data id can be mapped to a data sub-library number db _ x, and an optical system between the data id and the data sub-library number db _ x is stored by using a mapping relation map _ db in a mapping relation database.
However, in such a processing manner, extra storage devices are required to be occupied to store the mapping relationship; secondly, as the mapping relation database needs to respond to a large amount of data processing requests and perform corresponding processing, namely, before data processing is performed each time, the map _ db needs to be inquired in the mapping relation database 2, the response time is increased; finally, the method can only query according to the data id, cannot query by using other conditions, and has low query pertinence.
The embodiment of the present disclosure provides a data processing method to solve the above problem.
Fig. 2 is a schematic diagram of a network architecture on which the present disclosure is based, and as shown in fig. 2, unlike the prior art, the network architecture on which the present disclosure is based only includes a request initiating terminal 1 and a data sub-base 3, and the data sub-base 3 is provided with a data processing method apparatus described below in the present disclosure, which can execute the data processing method provided by the present disclosure.
Referring to fig. 3, fig. 3 is a schematic flow chart of a data processing method provided in the present disclosure. The method of the embodiment can be applied to a server based on which a data processing method device is based, and the data processing method comprises the following steps:
step 101, receiving a data processing request, wherein the data processing request comprises a data identifier of requested data.
In the present embodiment, the data processing request includes, but is not limited to, various types of processing for data, such as query, reading, updating, deleting, and the like for data. Specifically, the data processing request may be initiated by the request originator 1 shown in fig. 1. The data processing request comprises the requested content and also comprises a data identifier of the requested data, wherein the data identifier generally comprises a unique character string and is used for representing the identity, the service information and the like of the requested data.
And 102, performing first processing on a preset field of the data identifier to obtain a routing mantissa of the requested data.
Subsequently, a first process may be performed on a preset field in the data identification of the requested data to obtain a routing mantissa.
In one embodiment of the disclosure, the first processing is different based on the difference of the preset fields, wherein the routing mantissa is related to the database sublibrary and has a specific relationship with the sublibrary identifier of the database. That is, once the routing mantissa corresponding to the requested data is obtained, the sub-base identifier of the database where the requested data is located is also determined.
For example, when the predetermined field of the data identifier is a service field, such as route _ key, the service field route _ key is used to indicate the service requirement of the requested data. The service requirement can have different contents according to the scene based on the present disclosure, for example, when the scene is data processing based on an advertisement platform, the service requirement can specifically represent advertisement type, advertiser id, advertisement duration, and the like; when the scene is data processing based on a game platform, the service requirement may specifically represent a game type, a user type, a game operation type, and the like.
At this time, the first processing may be hash processing, and the hash processing is performed on the service field to obtain the route mantissa tail. Specifically, a hash function can be used to perform hash operation on the service field route _ key, and the operation result is subjected to remainder according to a preset first parameter, so as to obtain the routing tail number tail.
That is, the routing mantissa tail, which is hash (route _ key)% c, is obtained based on the following formula; wherein, the hash (route _ key) is used for representing that the hash function is used for carrying out the hash operation on the service field route _ key,% c is used for representing that the operation result of the hash operation is divided by c and is subjected to remainder, and the remainder is the routing mantissa tail. For example, when route _ key ═ x, c ═ 100, if hash (x) ═ 111, tail ═ 11. Wherein c is a positive integer not less than the total amount n of the sub-pools, and c is required to be a multiple of 10.
Or,
for example, if the preset field of the data identifier is a data identity field, the first processing is a remainder operation, that is, a remainder is performed on the data identity field according to the second parameter to obtain the routing mantissa.
It should be noted that the data identity field is generally used to distinguish data, and is generally generated by the database itself, and the field itself does not carry any information. In the embodiment of the disclosure, in order to facilitate fast acquisition of the route mantissa of the requested data in the data processing request, the data identity field is randomly generated by the identity field generator and is obtained by splicing according to a preset splicing rule. That is to say, the present disclosure gives more content meaning to the data identity field, specifically, in view of the uniqueness of the data identity field, when generating the data identity field of the data, firstly, an identity field generator may randomly generate a unique field g _ id, and then, the tail of the data is spliced to the i data identity field new _ id behind the g _ id to form an i data identity field new _ id of the data, that is, new _ id ═ g _ id } { tail }; if tail is 99 and g _ id is 1000, then new _ id is 100099.
Therefore, the data identity field obtained in this way can facilitate rapid processing of the data. That is to say, in the embodiment of the present disclosure, the data identity field new _ id in the data identifier may be obtained, and the second parameter c' is used to perform complementation on the data identity field new _ id, so as to obtain the routing mantissa tail.
That is, the routing mantissa tail, tail ═ new _ id% c', may be obtained based on the following formula; where% c is used to represent dividing new _ id by c' and the remainder is the routing mantissa tail. For example, when new _ id is 100099 and c' is 100, tail is 99. Wherein c 'is a positive integer not less than the total amount n of the sub-pools, and c' is a multiple of 10, which may be the same as or different from c.
The two modes can be used in parallel or alternatively, and the obtained routing mantissas are consistent for the same data no matter which mode is adopted.
And 103, determining the sub-database identification of the target sub-database storing the requested data according to the routing mantissa and the total sub-database amount of the database.
As described above, when the database stores data, which data sub-bank the data is stored into is determined based on the routing mantissa tail and the sub-bank total amount n. Thus, by means of this relationship, the sub-library identity shrard of the target sub-library db _ shrard storing the requested data can be determined.
For example, the sub-pool identifier shrard may be determined based on the following formula, where the shrard is used to identify the sub-pool identifier shrard of the target sub-pool db _ shrard, where% n is used to represent the routing mantissa tail divided by the total number n of sub-pools and the remainder is the sub-pool identifier shrard. For example, when tail is 11 and n is 10, the share is 1. Wherein the total amount n of the sub-libraries is a positive integer; the shredders are positive integers and the numeric area of the shredders is [0, n-1 ].
And 104, forwarding the data processing request to the target sub-library according to the sub-library identification of the target sub-library.
Similar to the prior art, after the sub-library identifier shard of the target sub-library is obtained, the data processing request can be directly forwarded to the target sub-library db _ shard, so that the target sub-library db _ shard can process the corresponding data processing request and correspondingly process the requested data.
The data processing method provided in this embodiment first receives a data processing request, where the data processing request includes a data identifier of requested data, then performs first processing on a preset field of the data identifier to obtain a routing mantissa of the requested data, then determines a sub-library identifier of a target sub-library in which the requested data is stored according to the routing mantissa and a total sub-library amount of a database, and finally forwards the data processing request to the target sub-library according to the sub-library identifier of the target sub-library.
Compared with the prior art that the mapping relation is used for acquiring the data sub-databases, the method does not need additional storage equipment, namely a mapping relation database for storing the mapping relation db _ map, and effectively reduces the storage cost; secondly, after receiving the data processing request of the request initiating end, the processing steps can be directly executed aiming at the data processing request without processing operations such as request, inquiry and the like of db _ map, the response time of the data processing request is shortened, and the response efficiency is improved; finally, in the embodiment of the present disclosure, the route mantissa and the sub-database identifier may be determined according to the predetermined field, and the predetermined field may specifically include the service field with the service requirement, so that the user may initiate a processing request of service data, such as query, to the database according to a specific service requirement, and the data query has a high pertinence, and is convenient for the user to use.
Corresponding to the data processing method of the above embodiment, fig. 4 is a schematic structural diagram of a data processing method apparatus provided by the present disclosure. For ease of illustration, only portions that are relevant to embodiments of the present disclosure are shown. Referring to fig. 4, the data processing method apparatus includes: a communication unit 10, a first processing unit 20 and a second processing unit 30.
The communication unit 10 is configured to receive a data processing request, where the data processing request includes a data identifier of requested data;
a first processing unit 20, configured to perform first processing on a preset field of the data identifier, to obtain a route mantissa of the requested data;
the second processing unit 30 is configured to determine, according to the routing mantissa and a total database partitioning amount of the database, a library partitioning identifier of a target library partitioning in which the requested data is stored;
the communication unit 10 is further configured to forward the data processing request to the target sub-library according to the sub-library identifier of the target sub-library.
In an embodiment of the present disclosure, the preset field of the data identifier is a service field, and the service field is used for representing a service requirement of the requested data; the first processing unit 20 is specifically configured to perform hash processing on the service field to obtain the route mantissa.
In an optional embodiment provided by the present disclosure, the first processing unit 20 is specifically configured to perform an operation on the service field by using a hash function, and obtain the routing mantissa by performing a remainder operation on an operation result according to a preset first parameter.
In an optional embodiment provided by the present disclosure, the preset field of the data identifier is a data identity field; the first processing unit 20 is specifically configured to perform complementation on the data identity field according to the second parameter to obtain the routing mantissa.
In an optional embodiment provided by the present disclosure, the data identity field is randomly generated by an identity field generator and is obtained by splicing according to a preset splicing rule.
In an optional embodiment provided by the present disclosure, the second processing unit 30 is specifically configured to utilize the total number of the sublibraries to perform remainder on the routing mantissa, so as to obtain a sublibrary identifier of the target sublibrary.
In the optional embodiment provided in this disclosure, the present embodiment may be used to implement the technical solution of the method embodiment, which implements similar principles and technical effects, and this embodiment is not described herein again.
Referring to fig. 5, fig. 5 is a schematic diagram of a hardware structure of an electronic device provided in the present disclosure, where the present disclosure provides an electronic device, including:
one or more processors 42;
a storage device 41 for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement a method as in any preceding claim.
Specifically, as shown in fig. 5, the electronic device may include a processor 42 (e.g., a central processing unit, a graphic processor, etc.), which may perform various suitable actions and processes according to a program stored in a storage device 41 including a Read Only Memory (ROM) or a program loaded from the storage device 41 into a Random Access Memory (RAM). In the storage device 41, various programs and data necessary for the operation of the electronic apparatus are also stored.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program containing program code for performing the method illustrated by the flow chart.
It should be noted that the computer readable storage medium of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable storage medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer-readable storage medium may be included in the electronic device; or may exist separately without being assembled into the electronic device.
The computer-readable storage medium carries one or more programs which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of Network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of an element does not in some cases constitute a limitation on the element itself.
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
In a first aspect, according to one or more embodiments of the present disclosure, there is provided a data processing method, including:
receiving a data processing request, wherein the data processing request comprises a data identifier of requested data;
performing first processing on a preset field of the data identifier to obtain a routing mantissa of the requested data;
determining the sub-database identification of the target sub-database storing the requested data according to the routing mantissa and the total sub-database amount of the database;
and forwarding the data processing request to the target sub-library according to the sub-library identification of the target sub-library.
According to one or more embodiments of the present disclosure, the preset field of the data identifier is a service field, and the service field is used for representing a service requirement of the requested data; correspondingly, the performing the first processing on the preset field of the data identifier to obtain the route mantissa of the requested data includes:
and carrying out hash processing on the service field to obtain the routing mantissa.
According to one or more embodiments of the present disclosure, the performing hash processing on the service field to obtain the route mantissa includes: utilizing a hash function to operate the service field; and obtaining the routing mantissa by carrying out complementation on the operation result according to a preset first parameter.
According to one or more embodiments of the present disclosure, the preset field of the data identifier is a data identity field; correspondingly, the performing the first processing on the preset field of the data identifier to obtain the route mantissa of the requested data includes: and performing complementation on the data identity field according to the second parameter to obtain the routing mantissa.
According to one or more embodiments of the present disclosure, the data identity field is randomly generated by an identity field generator and is obtained by splicing according to a preset splicing rule.
According to one or more embodiments of the present disclosure, the determining, according to the routing mantissa and a total database split amount, a split identifier of a target split stored with the requested data includes:
and utilizing the total number of the sub-libraries to carry out complementation on the routing mantissas to obtain the sub-library identification of the target sub-library.
In a second aspect, according to one or more embodiments of the present disclosure, there is provided a data processing apparatus comprising:
the communication unit is used for receiving a data processing request, and the data processing request comprises a data identifier of requested data;
the first processing unit is used for carrying out first processing on the preset field of the data identifier to obtain the routing mantissa of the requested data;
the second processing unit is used for determining the sub-database identification of the target sub-database storing the requested data according to the routing mantissa and the total sub-database amount of the database;
the communication unit is further configured to forward the data processing request to the target sub-library according to the sub-library identifier of the target sub-library.
According to one or more embodiments of the present disclosure, the preset field of the data identifier is a service field, and the service field is used for representing a service requirement of the requested data; the first processing unit is specifically configured to perform hash processing on the service field to obtain the route mantissa.
According to one or more embodiments of the present disclosure, the first processing unit is specifically configured to perform an operation on the service field by using a hash function, and obtain the routing mantissa by performing a remainder operation on an operation result according to a preset first parameter.
According to one or more embodiments of the present disclosure, the preset field of the data identifier is a data identity field; the first processing unit is specifically configured to perform complementation on the data identity field according to the second parameter to obtain the routing mantissa.
According to one or more embodiments of the present disclosure, the data identity field is randomly generated by an identity field generator and is obtained by splicing according to a preset splicing rule.
According to one or more embodiments of the present disclosure, the second processing unit is specifically configured to utilize the total number of the sublibraries to perform remainder on the routing mantissa, so as to obtain a sublibrary identifier of the target sublibrary.
In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device including: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the data processing method as described above in relation to the first aspect and the various possible references to the first aspect.
In a fourth aspect, according to one or more embodiments of the present disclosure, there is provided a computer-readable storage medium having stored therein computer-executable instructions that, when executed by a processor, implement the data processing method as described above in the first aspect and various possible references to the first aspect.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.
Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
Claims (14)
1. A data processing method, comprising:
receiving a data processing request, wherein the data processing request comprises a data identifier of requested data;
performing first processing on a preset field of the data identifier to obtain a routing mantissa of the requested data;
determining the sub-database identification of the target sub-database storing the requested data according to the routing mantissa and the total sub-database amount of the database;
and forwarding the data processing request to the target sub-library according to the sub-library identification of the target sub-library.
2. The data processing method according to claim 1, wherein the preset field of the data identifier is a service field, and the service field is used for indicating service requirements of the requested data;
correspondingly, the performing the first processing on the preset field of the data identifier to obtain the route mantissa of the requested data includes:
and carrying out hash processing on the service field to obtain the routing mantissa.
3. The data processing method of claim 2, wherein the hashing the traffic field to obtain the routing mantissa comprises:
utilizing a hash function to operate the service field;
and obtaining the routing mantissa by carrying out complementation on the operation result according to a preset first parameter.
4. The data processing method according to claim 1, wherein the preset field of the data identifier is a data identity field;
correspondingly, the performing the first processing on the preset field of the data identifier to obtain the route mantissa of the requested data includes:
and performing complementation on the data identity field according to the second parameter to obtain the routing mantissa.
5. The data processing method of claim 4, wherein the data identity field is randomly generated by an identity field generator and spliced according to a preset splicing rule.
6. The data processing method according to any one of claims 1 to 5, wherein the determining, according to the routing mantissa and a total database split amount, a split identifier of a target split bank in which the requested data is stored comprises:
and utilizing the total number of the sub-libraries to carry out complementation on the routing mantissas to obtain the sub-library identification of the target sub-library.
7. A data processing apparatus, comprising:
the communication unit is used for receiving a data processing request, and the data processing request comprises a data identifier of requested data;
the first processing unit is used for carrying out first processing on the preset field of the data identifier to obtain the routing mantissa of the requested data;
the second processing unit is used for determining the sub-database identification of the target sub-database storing the requested data according to the routing mantissa and the total sub-database amount of the database;
the communication unit is further configured to forward the data processing request to the target sub-library according to the sub-library identifier of the target sub-library.
8. The data processing apparatus according to claim 7, wherein the preset field of the data identifier is a service field, and the service field is used for indicating a service requirement of the requested data;
the first processing unit is specifically configured to perform hash processing on the service field to obtain the route mantissa.
9. The data processing apparatus according to claim 8, wherein the first processing unit is specifically configured to perform an operation on the service field by using a hash function, and obtain the routing mantissa by performing a remainder operation on an operation result according to a preset first parameter.
10. The data processing apparatus according to claim 7, wherein the preset field of the data identifier is a data identity field;
the first processing unit is specifically configured to perform complementation on the data identity field according to the second parameter to obtain the routing mantissa.
11. The data processing apparatus of claim 10, wherein the data identity field is randomly generated by an identity field generator and is obtained by concatenation according to a preset concatenation rule.
12. The data processing method according to any one of claims 7 to 11, wherein the second processing unit is specifically configured to obtain the segment identifier of the target segment by using the total segment count to perform remainder on the route mantissa.
13. An electronic device, comprising: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the data processing method of any of claims 1 to 6.
14. A computer-readable storage medium having stored thereon computer-executable instructions which, when executed by a processor, implement a data processing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910530224.6A CN110222064A (en) | 2019-06-19 | 2019-06-19 | Data processing method, device, electronic equipment and readable storage medium storing program for executing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910530224.6A CN110222064A (en) | 2019-06-19 | 2019-06-19 | Data processing method, device, electronic equipment and readable storage medium storing program for executing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110222064A true CN110222064A (en) | 2019-09-10 |
Family
ID=67817697
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910530224.6A Pending CN110222064A (en) | 2019-06-19 | 2019-06-19 | Data processing method, device, electronic equipment and readable storage medium storing program for executing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110222064A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110851427A (en) * | 2019-11-19 | 2020-02-28 | 泰康保险集团股份有限公司 | Database capacity changing method and device |
| CN112579623A (en) * | 2019-09-29 | 2021-03-30 | 北京国双科技有限公司 | Method, device, storage medium and equipment for storing data |
| CN112597222A (en) * | 2020-12-16 | 2021-04-02 | 中国建设银行股份有限公司 | Financial database connection processing method, device, equipment and storage medium |
| CN113779604A (en) * | 2021-09-13 | 2021-12-10 | 网易（杭州）网络有限公司 | Business service implementation method, device, equipment and storage medium based on block chain |
| CN114490742A (en) * | 2020-11-12 | 2022-05-13 | 中移物联网有限公司 | Data query method and device and electronic equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103124215A (en) * | 2013-01-25 | 2013-05-29 | 匡创公司 | Self-certifying method with time marks |
| CN103853718A (en) * | 2012-11-28 | 2014-06-11 | 纽海信息技术（上海）有限公司 | Fragmentation database access method and database system |
| US20150312155A1 (en) * | 2014-04-25 | 2015-10-29 | Telefonaktiebolaget L M Ericsson (Publ) | System and method for efectuating packet distribution among servers in a network |
| CN107292636A (en) * | 2017-08-17 | 2017-10-24 | 上海焕云网络技术有限公司 | A kind of commodity counterfeit prevention checking system based on natural biological information |
| CN108009883A (en) * | 2017-11-30 | 2018-05-08 | 泰康保险集团股份有限公司 | Method and device for order processing |
| CN108897767A (en) * | 2018-05-24 | 2018-11-27 | 努比亚技术有限公司 | A kind of data query method and system |
| CN109857723A (en) * | 2019-01-31 | 2019-06-07 | 深圳市迷你玩科技有限公司 | Dynamic date migration method and relevant device based on extendible capacity data-base cluster |
-
2019
- 2019-06-19 CN CN201910530224.6A patent/CN110222064A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103853718A (en) * | 2012-11-28 | 2014-06-11 | 纽海信息技术（上海）有限公司 | Fragmentation database access method and database system |
| CN103124215A (en) * | 2013-01-25 | 2013-05-29 | 匡创公司 | Self-certifying method with time marks |
| US20150312155A1 (en) * | 2014-04-25 | 2015-10-29 | Telefonaktiebolaget L M Ericsson (Publ) | System and method for efectuating packet distribution among servers in a network |
| CN107292636A (en) * | 2017-08-17 | 2017-10-24 | 上海焕云网络技术有限公司 | A kind of commodity counterfeit prevention checking system based on natural biological information |
| CN108009883A (en) * | 2017-11-30 | 2018-05-08 | 泰康保险集团股份有限公司 | Method and device for order processing |
| CN108897767A (en) * | 2018-05-24 | 2018-11-27 | 努比亚技术有限公司 | A kind of data query method and system |
| CN109857723A (en) * | 2019-01-31 | 2019-06-07 | 深圳市迷你玩科技有限公司 | Dynamic date migration method and relevant device based on extendible capacity data-base cluster |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112579623A (en) * | 2019-09-29 | 2021-03-30 | 北京国双科技有限公司 | Method, device, storage medium and equipment for storing data |
| CN110851427A (en) * | 2019-11-19 | 2020-02-28 | 泰康保险集团股份有限公司 | Database capacity changing method and device |
| CN114490742A (en) * | 2020-11-12 | 2022-05-13 | 中移物联网有限公司 | Data query method and device and electronic equipment |
| CN112597222A (en) * | 2020-12-16 | 2021-04-02 | 中国建设银行股份有限公司 | Financial database connection processing method, device, equipment and storage medium |
| CN113779604A (en) * | 2021-09-13 | 2021-12-10 | 网易（杭州）网络有限公司 | Business service implementation method, device, equipment and storage medium based on block chain |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110222064A (en) | Data processing method, device, electronic equipment and readable storage medium storing program for executing | |
| US10585915B2 (en) | Database sharding | |
| CN109656923B (en) | Data processing method and device, electronic equipment and storage medium | |
| CN107704202B (en) | Method and device for quickly reading and writing data | |
| CN108255958A (en) | Data query method, apparatus and storage medium | |
| CN110175174A (en) | A kind of data query method, apparatus, equipment and storage medium | |
| CN112597126A (en) | Data migration method and device | |
| CN108399046B (en) | File operation request processing method and device | |
| CN111401684A (en) | Task processing method and device | |
| CN117130663B (en) | Instruction reading method, L2 instruction cache, electronic equipment and storage medium | |
| CN114281663A (en) | Test processing method, test processing device, electronic equipment and storage medium | |
| CN111694992A (en) | Data processing method and device | |
| CN110020040B (en) | Method, device and system for querying data | |
| CN114116247A (en) | Redis-based message processing method, device, system, server and medium | |
| CN113704222A (en) | Method and device for processing service request | |
| CN112784596A (en) | Method and device for identifying sensitive words | |
| CN113326038B (en) | Method, apparatus, device, storage medium and program product for providing service | |
| CN109308299B (en) | Method and apparatus for searching information | |
| CN115391605A (en) | Data query method, device, equipment, computer readable medium and program product | |
| CN113256301A (en) | Data shielding method, device, server and medium | |
| CN110647623B (en) | Method and device for updating information | |
| CN113419792A (en) | Event processing method and device, terminal equipment and storage medium | |
| CN113157695A (en) | Data processing method and device, readable medium and electronic equipment | |
| CN113032820A (en) | File storage method, access method, device, equipment and storage medium | |
| CN113840020B (en) | Positioning method and device based on IP address information, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190910 |
|
| RJ01 | Rejection of invention patent application after publication |