CN108733477A - The method, apparatus and equipment of data clusterization processing - Google Patents
The method, apparatus and equipment of data clusterization processing Download PDFInfo
- Publication number
- CN108733477A CN108733477A CN201710263207.1A CN201710263207A CN108733477A CN 108733477 A CN108733477 A CN 108733477A CN 201710263207 A CN201710263207 A CN 201710263207A CN 108733477 A CN108733477 A CN 108733477A
- Authority
- CN
- China
- Prior art keywords
- data
- data source
- application process
- locking
- catalogue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/12—Accounting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/12—Accounting
- G06Q40/125—Finance or payroll
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
Abstract
The embodiment of the invention discloses the method, apparatus and equipment of a kind of processing of data clusterization.This method includes：The access request for receiving data source determines the corresponding locking type of data source based on the data characteristic of data source；Locking type is distributed lock, according to the application process of the distributed server of the storage information allocation processing data source of data source and processing data source；Whether the locking catalogue detection application process according to distributed server is currently idle；Application process current idle is handled data source by application process.The embodiment of the present invention can be reduced in accounting data management, adjustment, the complexity of program publication etc..
Description
Technical field
The present invention relates to method, apparatus and equipment that field of communication technology more particularly to a kind of data clusterization are handled.
Background technology
Accounting data service system BDS is the highly available system that multiple computers are combined into cluster composition,
Data storage is provided for business operation support system BOSS internal accountings, account and its related system to support, realizes charging core
The system stored with unification in the data set of the heart.
The clustering deployment that accounting data service system has been carried out certain applications is supported by multi-process preempt-mode
The clustering deployment for handling the application of message, file and data storage, specifically include sorting, two batches, online charging, make up accounts etc. into
Journey.
For key business, such as bill storage, hire charge calculate, letter control, since its processing speed requires high process simultaneously
Multi-process is not supported to seize processing, so these key business processes do not carry out clustering transformation, but temporarily by blade
Two-shipper mode is handled.But this key business process is handled temporarily by blade two-shipper mode, in accounting data pipe
Reason, adjustment, program publication etc. both increase great complexity.
Invention content
An embodiment of the present invention provides the method, apparatus and equipment of a kind of processing of data clusterization, can reduce in account
Data management, adjustment, the complexity of program publication etc..
In a first aspect, an embodiment of the present invention provides a kind of methods of data clusterization processing, including：
The access request for receiving data source determines the corresponding locking type of data source based on the data characteristic of data source；
Locking type be distributed lock, according to the distributed server of the storage information allocation processing data source of data source with
Handle the application process of data source；
Whether the locking catalogue detection application process according to distributed server is currently idle；
Application process current idle is handled the data source by the application process.
Second aspect, an embodiment of the present invention provides a kind of devices of data clusterization processing, including：
Determination unit, the access request for receiving data source determine data source pair based on the data characteristic of data source
The locking type answered；
Allocation unit is distributed lock for locking type, according to the storage information allocation processing data source of data source
The application process of distributed server and processing data source；
Process detection unit, it is currently whether idle for the locking catalogue detection application process according to distributed server；
Creating unit is used for application process current idle, is handled the data source by the application process.
The third aspect, an embodiment of the present invention provides a kind of equipment of data clusterization processing, including：
Memory, processor, communication interface and bus；
Memory, processor and communication interface are connected by bus and complete mutual communication；
Memory is for storing program code；
Processor is corresponding with executable program code to run by reading the executable program code stored in memory
Program, the method for the method for executing a kind of processing of data clusterization, data clusterization processing includes：
The access request for receiving data source determines the corresponding locking type of data source based on the data characteristic of data source；
Locking type be distributed lock, according to the distributed server of the storage information allocation processing data source of data source with
Handle the application process of data source；
Whether the locking catalogue detection application process according to distributed server is currently idle；
Application process current idle is handled the data source by the application process.
An embodiment of the present invention provides the method, apparatus and equipment of a kind of processing of data clusterization, in the embodiment of the present invention,
After receiving the access request of data source, the corresponding locking type of data source is determined based on the data characteristic of data source；It locks
Type is distributed lock, according to the distributed server of the storage information allocation processing data source of data source and handles data source
Application process；Whether the locking catalogue detection application process according to distributed server is currently idle；Application process current idle,
Data source is handled by application process.In the embodiment of the present invention, data source pair is determined based on the data characteristic of data source
The locking type answered distributes multiple step format server and application process if locking type is distributed lock to this row data source
It handles this data source, realizes and clustering deployment is carried out to these data sources not by way of multi-process preempt-mode, in turn
It can also realize that clustering is disposed not by way of multi-process preempt-mode to key business process, reduce in accounting data
Management, adjustment, the complexity of program publication etc..
Description of the drawings
In order to illustrate the technical solution of the embodiments of the present invention more clearly, will make below to required in the embodiment of the present invention
Attached drawing is briefly described, it should be apparent that, drawings described below is only some embodiments of the present invention, for
For those of ordinary skill in the art, without creative efforts, other are can also be obtained according to these attached drawings
Attached drawing.
Fig. 1 is the schematic flow chart of the method for the data clusterization processing provided according to one embodiment of the invention；
Fig. 2 is the schematic flow chart of the method for the data clusterization processing provided according to further embodiment of this invention；
Fig. 3 is the schematic block diagram of the device of the data clusterization processing provided according to one embodiment of the invention；
Fig. 4 is the schematic block diagram of the device of the data clusterization processing provided according to further embodiment of this invention；
Fig. 5 is the schematic block diagram of the equipment of the data clusterization processing provided according to one embodiment of the invention.
Specific implementation mode
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
The every other embodiment obtained without creative efforts, shall fall within the protection scope of the present invention.
It should be noted that in the absence of conflict, the features in the embodiments and the embodiments of the present application can phase
Mutually combination.The application is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
The embodiment of the present invention can be used for the scene of data clusterization processing.For example, in accounting data service system, it is right
In key business, such as bill storage, hire charge calculate, letter control is handled using distributed lock.In accounting data service system,
Application layer is handled the accounting data for being stored in data Layer by data link layer, data link layer to data source at
Reason.Wherein, after receiving the access request of data source, data source is distinguished by the way of institute first, is suitable for more
File lock or data lock etc. may be used to handle in the data source that process seizes processing, and seizes place for not being suitable for multi-process
Distributed lock may be used to handle, i.e., by the way of the embodiment of the present invention in the data source of reason.It so can be by accounting data
Clustering deployment is carried out, is reduced in accounting data management, adjustment, the complexity of program publication etc..
Fig. 1 is the schematic flow chart of the method for data clusterization processing according to an embodiment of the invention.Such as Fig. 1 institutes
Show, this approach includes the following steps.
110, the access request of data source is received, the corresponding locking class of data source is determined based on the data characteristic of data source
Type.
Wherein, the access request of data source includes the data characteristic of data source, then is asked in the access for receiving data source
After asking, the corresponding locking type of data source is determined based on the data characteristic of data source.The data characteristic of data source can be according to tool
Body scene setting.For example, in accounting data, the first data characteristic is arranged in the data source that suitable multi-process is seized to processing, will
Processing is seized in unsuitable multi-process, needs the data source that mode through the embodiment of the present invention is handled that the second data characteristic is arranged,
To after receiving the access request of data source, you can determine the corresponding locking class of data source based on the data characteristic of data source
Type：Data characteristic is the first data characteristic, it is determined that the corresponding locking type of data source is file lock or data lock etc.；Data are special
Property be the second data characteristic, it is determined that corresponding the lockings type of data source be distributed lock.For another example the value of setting data characteristic
For on or off, the value of data characteristic is to determine that the corresponding locking type of data source is distributed lock when opening；The value of data characteristic is
Guan Shi determines that the corresponding locking type of data source is file lock or data lock etc..
It should be noted that in the embodiment of the present invention, data source may include subqueue and/subtask.When data source is
When subqueue, the value of data characteristic can be subqueue.lock.type；When data source is subtask, the value of data characteristic
Can be cluster.task.lock.type.
120, locking type is distributed lock, according to the Distributed Services of the storage information allocation processing data source of data source
The application process of device and processing data source.
Wherein, it is distributed lock to lock type, illustrates that this data source is not suitable for multi-process and seizes processing, but uses and divide
Cloth locks corresponding processing mode.In this step, the distributed server for data source allocation processing data source and processing first
The application process of data source.
130, whether the locking catalogue detection application process according to distributed server is currently idle.
Wherein, after being assigned with application process for data source, it can detect and answer according to the locking catalogue of distributed server
It is currently whether idle with process, that is, detect whether application process is handling other tasks.
140, application process current idle is handled data source by application process.
Wherein, in application process current idle, illustrate that it can currently be handled data source, so by application into
Journey handles data source.Application process carries out processing to data source：Data are created on distributed server
The corresponding locking catalogue in source makes application process lock data source, and then handles data source.
In the embodiment of the present invention, the corresponding locking type of data source is determined based on the data characteristic of data source, if locked
Type is distributed lock, then distributes multiple step format server and application process to handle this data source to this row data source, realize not
Clustering deployment is carried out to these data sources by way of multi-process preempt-mode, and then can be obstructed to key business process
Crossing the mode of multi-process preempt-mode can also realize that clustering is disposed, and reduce in accounting data management, adjustment, the side such as program publication
The complexity in face.
It is understood that in embodiments of the present invention, step 130 can also be executed specifically as following steps.
131, the locking catalogue of distributed server includes child node, determines that application process is not currently idle.132, distribution
The locking catalogue of formula server does not include child node, and interim child node is created in the case where locking catalogue, and interim child node is not to lock
The child node of serial number minimum under catalogue, determines that application process is not currently idle, and delete interim child node.133, Distributed Services
The locking catalogue of device does not include child node, and interim child node is created in the case where locking catalogue, and interim child node is to lock sequence under catalogue
Number minimum child node, determines application process current idle.
Wherein, in embodiments of the present invention, application can be determined by locking the state of catalogue in distributed server
Whether process is idle.Application process needs first to lock it before handling data source, it is stated that this data source has had
Application process processing, reminds other processes that need not handle again this data source.Application process locks data source
When, child node can be created under the locking catalogue of distributed server, be saved so the locking catalogue of distributed server includes son
Point illustrates that application process has locked other data sources and handled, that is, determines that application process is not currently idle.Distributed server
Locking catalogue include child node when, interim child node can be created in the case where locking catalogue, then further according to interim child node
Whether it is to lock the child node of serial number minimum under catalogue to determine whether application process is idle.Interim child node is to lock under catalogue
The child node of serial number minimum, determines application process current idle；Interim child node is not to lock the minimum son section of serial number under catalogue
Point determines that application process is not currently idle, and deletes interim child node.
In the embodiment of the present invention, application process is determined by locking the state of catalogue lower word node in distributed server
It is whether idle, in the application process free time, application program is made to handle data source, the lock for reducing application process waits for, and carries
High-performance.
It should be noted that in embodiments of the present invention, determining that application process is not currently idle in above process
Afterwards, it can also be that data source distributes application process again, determine whether the application process redistributed is empty by the above process
Spare time is handled data source by idle application process until determining to distribute the application process free time for data source.
As an optional embodiment, Fig. 2 is the method handled according to the data clusterization of further embodiment of this invention
Schematic flow chart.Step 110-140 and the basic phases of step 110-140 in embodiment illustrated in fig. 1 in embodiment illustrated in fig. 2
Together, as shown in Fig. 2, after step 140, this method further includes following steps.
150, whether detection application process there is exception.
160, there is exception in application process, is unlocked to data source.
Wherein, application process is during processing task, it is possible that the abnormal conditions such as failure, go out in application process
When now abnormal, in order to avoid the task in its processing is interrupted, so needing in application process failure, release is needed to locate
The data source of reason, i.e., be unlocked data source, in order to which data source can be handled by other application process.
It should be noted that in the embodiment of the present invention, when being unlocked to data source, it can delete and create data source correspondence
Locking catalogue, i.e., when application process occurs abnormal, the corresponding locking catalogue of data source is automatically deleted unlock.
It is understood that in embodiments of the present invention, in step 120 according to the storage information distribution of the data source at
Managing the distributed server of the data source can also specifically execute as following steps：It is looked into according to the memory node of storage data source
Ask the corresponding task identification of memory node；Task based access control identifies and preset attribute mark determines distributed server；According to process
Identify the application process that processing data source is determined from the process list of distributed server.
Wherein, in the embodiment of the present invention, physical database can be pre-established, physical database include cluster definition table,
Cluster attribute table, host node define table, cluster task defines the allocation lists such as table, can be with store tasks mark, cluster number, collection
The information such as group's attribute-bit, process title, process identification (PID).After determining the memory node of data source, host node definition is searched
The task identification field of table searches cluster attribute according to task identification and defines the distribution that attribute-bit in table is preset attribute mark
Formula server, it is possible thereby to determine the distributed server of processing data source.Then in the process list of distributed server
Select application process as the application process of allocation processing data source according to process identification (PID).
The mode of the application process of processing data source is determined from the process list of distributed server according to process identification (PID)
Can be：Application process of the process of process identification (PID) minimum as allocation processing data source is selected in process list.
It should be noted that can divide distributed server in the embodiment of the present invention, such as distribution is taken
Business device is divided according to data source affiliated area is gone out, that is, the distributed server for belonging to certain region divided, which is only handled, belongs to this
The data source in region, so can be to avoid the unbalanced problem of data source distribution pending in distributed server.The present invention
After the distributed server for determining processing data source in embodiment, the association interconnected between the network of distributed server can be obtained
IP address and interface PORT are discussed, is connected to be established with distributed server.
Specifically, preset attribute mark may include lock_server, if in task based access control mark and preset attribute mark
When determining distributed server, inquires a plurality of attribute-bit and recorded for lock_server, it at this time can be according to ATTR_ORDER
Then sequence separates and is spliced together, the symbol for dividing splicing includes comma.
It should be noted that when data source is subqueue, the locking catalogue that is created on distributed server can be with
For：/ queue/ data store name/items/item- subqueue serial numbers；When data source is subtask, in distributed server
The locking catalogue of upper establishment is：/ job/ cluster tasks number/items/item- clusters subtask number.
The embodiment of the present invention realizes that whole clusterings of data source in account application are handled by introducing distributed lock.This
Inventive embodiments are not necessarily based on islion file system lock and oracle database lock, that is, realize answering for data clusterization processing
With the application level extended capability and high availability of raising accounting data service system.In addition, in the prior art, two-shipper mode
The business process of deployment needs to expend many times progress Two-channel switchings when Single Point of Faliure occurs for process, and the present invention is implemented
Clustering deployment way in example, when Single Point of Faliure occurs for process, other nodes can take over the business process of failure, pole
Big reduces human cost, improves work efficiency.And the waiting of the lock between different processes can be reduced, performance is improved.
Fig. 3 shows the schematic block diagram of the device 200 of the data clusterization processing provided according to one embodiment of the invention.
As shown in figure 3, the device 200 includes：
Determination unit 201, the access request for receiving data source determine data source based on the data characteristic of data source
Corresponding locking type；
Allocation unit 202 is distributed lock for locking type, according to the storage information allocation processing data source of data source
Distributed server and processing data source application process；
Process detection unit 203, it is currently whether empty for the locking catalogue detection application process according to distributed server
It is not busy；
Processing unit 204 is used for application process current idle, is handled data source by application process.
In the embodiment of the present invention, the corresponding locking type of data source is determined based on the data characteristic of data source, if locked
Type is distributed lock, then distributes multiple step format server and application process to handle this data source to this row data source, realize not
Clustering deployment is carried out to these data sources by way of multi-process preempt-mode, and then can be obstructed to key business process
Crossing the mode of multi-process preempt-mode can also realize that clustering is disposed, and reduce in accounting data management, adjustment, the side such as program publication
The complexity in face.
It is understood that process detection unit 203 is specifically used for：
The locking catalogue of distributed server includes child node, determines that application process is not currently idle；
The locking catalogue of distributed server does not include child node, and interim child node is created in the case where locking catalogue, interim son
Node is not the child node for locking serial number minimum under catalogue, determines that application process is not currently idle, and delete interim child node；
The locking catalogue of distributed server does not include child node, and interim child node is created in the case where locking catalogue, interim son
Node is the child node for locking serial number minimum under catalogue, determines application process current idle.
In above process, it is to determine application process by locking the state of catalogue lower word node in distributed server
The no free time makes application program handle data source in the application process free time, and the lock for reducing application process waits for, and improves
Performance.
Fig. 4 shows the schematic block diagram of the device 200 handled according to the data clusterization of further embodiment of this invention.
It is understood that as shown in figure 4, the device 200 can also include：
Abnormality detecting unit 205, for detecting whether application process exception occurs；
There is exception for application process, is unlocked to data source in unlocking unit 206.
In the above process, when being unlocked to data source, it can delete and create the corresponding locking catalogue of data source, i.e., ought answer
When occurring abnormal with process, the corresponding locking catalogue of data source is automatically deleted unlock.
It is understood that allocation unit 202 is specifically used for：
According to the corresponding task identification of memory node query node of storage data source；
Task based access control identifies and preset attribute mark determines distributed server；
Determined from the process list of the distributed server according to process identification (PID) handle the application of the data source into
Journey.
It is understood that data characteristic includes data characteristic switch value；
Data characteristic switch value is preset value, and locking type is distributed lock.
The device 200 of data clusterization processing according to the ... of the embodiment of the present invention can correspond to number according to the ... of the embodiment of the present invention
Executive agent in the method handled according to clustering, and the modules in the device 200 of data clusterization processing is above-mentioned
With other operation and/or functions respectively in order to realize the corresponding flow of each method in Fig. 1 to Fig. 2, for sake of simplicity, herein not
It repeats again.
Fig. 5 is the schematic block diagram of the equipment 300 of data clusterization processing according to an embodiment of the invention.Such as Fig. 5 institutes
Show, equipment 300 includes processor 301, memory 302 and communication interface 303, and memory 302 is for storing executable program
Code, processor 301 are run and executable program code pair by reading the executable program code stored in memory 302
The program answered, communication interface 303 is used for and external device communication, and equipment 300 can also include bus 304, and bus 304 is for connecting
Processor 301, memory 302 and communication interface 303 are connect, processor 301, memory 302 and communication interface 303 is made to pass through bus
304 are in communication with each other.
Specifically, processor 301 is additionally operable to execute a kind of method of data clusterization processing；Wherein, at data cluster
The method of reason includes：
The access request for receiving data source determines the corresponding locking type of data source based on the data characteristic of data source；
Locking type be distributed lock, according to the distributed server of the storage information allocation processing data source of data source with
Handle the application process of data source；
Whether the locking catalogue detection application process according to distributed server is currently idle；
Application process current idle is handled data source by application process.
The equipment 300 of data clusterization processing according to the ... of the embodiment of the present invention can correspond to number according to the ... of the embodiment of the present invention
Executive agent in the method handled according to clustering, and the modules in the equipment 300 of data clusterization processing is above-mentioned
With other operation and/or functions respectively in order to realize the corresponding flow of each method in Fig. 1 to Fig. 2, for sake of simplicity, herein not
It repeats again.
Those of ordinary skill in the art may realize that lists described in conjunction with the examples disclosed in the embodiments of the present disclosure
Member and algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware
With the interchangeability of software, each exemplary composition and step are generally described according to function in the above description.This
A little functions are implemented in hardware or software actually, depend on the specific application and design constraint of technical solution.Specially
Industry technical staff can use different methods to achieve the described function each specific application, but this realization is not
It is considered as beyond the scope of this invention.
It is apparent to those skilled in the art that for convenience of description and succinctly, foregoing description is
The specific work process of system, device and unit, can refer to corresponding processes in the foregoing method embodiment, details are not described herein.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can readily occur in various equivalent modifications or replace
It changes, these modifications or substitutions should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention should be with right
It is required that protection domain subject to.
Claims (11)
1. a kind of method of data clusterization processing, which is characterized in that including：
The access request of data source is received, the data characteristic in source determines the corresponding locking class of the data source based on the data
Type；
The locking type is distributed lock, the distribution of data source described in the storage information allocation processing according to the data source
Server and the application process for handling the data source；
Whether the locking catalogue according to the distributed server detects the application process currently idle；
The application process current idle is handled the data source by the application process.
2. the method for data clusterization processing according to claim 1, which is characterized in that described according to the Distributed Services
Whether the locking catalogue detection application process of device is currently idle, including：
The locking catalogue of the distributed server includes child node, determines that the application process is not currently idle；
The locking catalogue of the distributed server does not include child node, and interim child node, institute are created under the locking catalogue
It is the child node of serial number minimum under the locking catalogue to state interim child node not, determines that the application process is not currently idle, and
Delete the interim child node；
The locking catalogue of the distributed server does not include child node, and interim child node, institute are created under the locking catalogue
The child node that interim child node is serial number minimum under the locking catalogue is stated, determines the application process current idle.
3. the method for data clusterization processing according to claim 1, which is characterized in that described in the distributed server
It is upper create the corresponding locking catalogue of the data source after, further include：
Detect whether the application process exception occurs；
There is exception in the application process, is unlocked to the data source.
4. the method for data clusterization processing according to claim 1, which is characterized in that the depositing according to the data source
It stores up the distributed server of data source described in information allocation processing and handles the application process of the data source, including：
The corresponding task identification of the memory node is inquired according to the memory node for storing the data source；
The distributed server is determined based on the task identification and preset attribute mark；
The application process for handling the data source is determined from the process list of the distributed server according to process identification (PID).
5. the method for data clusterization processing according to claim 1, which is characterized in that the data characteristic includes data spy
Property switch value；
The data characteristic switch value is preset value, and the locking type is distributed lock.
6. a kind of device of data clusterization processing, which is characterized in that including：
Determination unit, the access request for receiving data source, based on the data the data characteristic in source determine the data
The corresponding locking type in source；
Allocation unit is used for the locking type for distributed lock, described in the storage information allocation processing according to the data source
The distributed server of data source and the application process for handling the data source；
Process detection unit, it is currently whether empty for detecting the application process according to the locking catalogue of the distributed server
It is not busy；
Processing unit is used for the application process current idle, is handled the data source by the application process.
7. the device of data clusterization processing according to claim 6, which is characterized in that the process detection unit is specifically used
In：
The locking catalogue of the distributed server includes child node, determines that the application process is not currently idle；
The locking catalogue of the distributed server does not include child node, and interim child node, institute are created under the locking catalogue
It is the child node of serial number minimum under the locking catalogue to state interim child node not, determines that the application process is not currently idle, and
Delete the interim child node；
The locking catalogue of the distributed server does not include child node, and interim child node, institute are created under the locking catalogue
The child node that interim child node is serial number minimum under the locking catalogue is stated, determines the application process current idle.
8. the device of data clusterization processing according to claim 6, which is characterized in that further include：
Abnormality detecting unit, for detecting whether the application process exception occurs；
There is exception for the application process, is unlocked to the data source in unlocking unit.
9. the device of data clusterization processing according to claim 6, which is characterized in that the allocation unit is specifically used for：
The corresponding task identification of the node is inquired according to the memory node for storing the data source；
The distributed server is determined based on the task identification and preset attribute mark；
The application process for handling the data source is determined from the process list of the distributed server according to process identification (PID).
10. the device of data clusterization processing according to claim 9, which is characterized in that the data characteristic includes data
Characteristic switch value；
The data characteristic switch value is preset value, and the locking type is distributed lock.
11. a kind of equipment of data clusterization processing, which is characterized in that including：
Memory, processor, communication interface and bus；
Memory, processor and communication interface are connected by bus and complete mutual communication；
Memory is for storing program code；
Processor runs journey corresponding with executable program code by reading the executable program code stored in memory
Sequence, for executing a kind of method of data clusterization processing, the method for the data clusterization processing includes：
The access request of data source is received, the data characteristic in source determines the corresponding locking class of the data source based on the data
Type；
The locking type is distributed lock, the distribution of data source described in the storage information allocation processing according to the data source
Server and the application process for handling the data source；
Whether the locking catalogue according to the distributed server detects the application process currently idle；
The application process current idle is handled the data source by the application process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710263207.1A CN108733477B (en) | 2017-04-20 | 2017-04-20 | Method, device and equipment for data clustering processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710263207.1A CN108733477B (en) | 2017-04-20 | 2017-04-20 | Method, device and equipment for data clustering processing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108733477A true CN108733477A (en) | 2018-11-02 |
| CN108733477B CN108733477B (en) | 2021-04-23 |
Family
ID=63933315
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710263207.1A Active CN108733477B (en) | 2017-04-20 | 2017-04-20 | Method, device and equipment for data clustering processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108733477B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109491928A (en) * | 2018-11-05 | 2019-03-19 | 深圳乐信软件技术有限公司 | Buffer control method, device, terminal and storage medium |
| CN110362401A (en) * | 2019-06-20 | 2019-10-22 | 深圳壹账通智能科技有限公司 | Data run the member host in batch method, apparatus, storage medium and cluster |
| CN110795241A (en) * | 2019-10-18 | 2020-02-14 | 北京并行科技股份有限公司 | Job scheduling management method, scheduling center and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101013381A (en) * | 2007-01-26 | 2007-08-08 | 华中科技大学 | Distributed lock based on object memory system |
| CN103686224A (en) * | 2013-12-26 | 2014-03-26 | 乐视网信息技术（北京）股份有限公司 | Method and system for transcoding task obtaining on basis of distributed locks |
| CN104536834A (en) * | 2014-11-26 | 2015-04-22 | 华为技术有限公司 | Method for authorizing lock permission and distributed lock manager |
| US20160092488A1 (en) * | 2014-09-26 | 2016-03-31 | Futurewei Technologies, Inc. | Concurrency control in a shared storage architecture supporting on-page implicit locks |
| CN106326014A (en) * | 2016-08-24 | 2017-01-11 | 杭州费尔斯通科技有限公司 | Resource access method and device |
-
2017
- 2017-04-20 CN CN201710263207.1A patent/CN108733477B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101013381A (en) * | 2007-01-26 | 2007-08-08 | 华中科技大学 | Distributed lock based on object memory system |
| CN103686224A (en) * | 2013-12-26 | 2014-03-26 | 乐视网信息技术（北京）股份有限公司 | Method and system for transcoding task obtaining on basis of distributed locks |
| US20160092488A1 (en) * | 2014-09-26 | 2016-03-31 | Futurewei Technologies, Inc. | Concurrency control in a shared storage architecture supporting on-page implicit locks |
| CN104536834A (en) * | 2014-11-26 | 2015-04-22 | 华为技术有限公司 | Method for authorizing lock permission and distributed lock manager |
| CN106326014A (en) * | 2016-08-24 | 2017-01-11 | 杭州费尔斯通科技有限公司 | Resource access method and device |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109491928A (en) * | 2018-11-05 | 2019-03-19 | 深圳乐信软件技术有限公司 | Buffer control method, device, terminal and storage medium |
| CN110362401A (en) * | 2019-06-20 | 2019-10-22 | 深圳壹账通智能科技有限公司 | Data run the member host in batch method, apparatus, storage medium and cluster |
| CN110795241A (en) * | 2019-10-18 | 2020-02-14 | 北京并行科技股份有限公司 | Job scheduling management method, scheduling center and system |
| CN110795241B (en) * | 2019-10-18 | 2022-07-19 | 北京并行科技股份有限公司 | Job scheduling management method, scheduling center and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108733477B (en) | 2021-04-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109814998A (en) | A kind of method and device of multi-process task schedule | |
| US7870230B2 (en) | Policy-based cluster quorum determination | |
| US5924097A (en) | Balanced input/output task management for use in multiprocessor transaction processing system | |
| US6889253B2 (en) | Cluster resource action in clustered computer system incorporation prepare operation | |
| US20070094396A1 (en) | Server pool management method | |
| CN104272259B (en) | System and method for supporting the route based on version in transaction middleware machine environment | |
| WO2010121218A2 (en) | Policy-based storage structure distribution | |
| CN112039970B (en) | Distributed business lock service method, server, system and storage medium | |
| CN108733477A (en) | The method, apparatus and equipment of data clusterization processing | |
| US10768974B2 (en) | Specifying an order of a plurality of resources in a transaction according to distance | |
| CN106155775A (en) | Message treatment method, equipment and system | |
| CN104793981B (en) | A kind of online snapshot management method and device of cluster virtual machine | |
| US20040117792A1 (en) | System and method for terminating processes in a distributed computing system | |
| CN112333065A (en) | Automatic opening method and device for IPRAN Ethernet private line | |
| JP4356018B2 (en) | Asynchronous messaging over storage area networks | |
| US20080320563A1 (en) | System and program product for associating event categorization and routing with security authorization roles | |
| KR19990043986A (en) | Business take over system | |
| JPH0981409A (en) | Mutual hot standby system selecting system | |
| CN108829709A (en) | Distributed database management method, apparatus, storage medium and processor | |
| CN109033134A (en) | A kind of message treatment method, device, storage medium and computer equipment | |
| CN111343101B (en) | Server current limiting method, device, electronic equipment and readable storage medium | |
| CN114049005A (en) | Workflow task allocation and control method and device and electronic equipment | |
| CN107528724A (en) | A kind of optimized treatment method and device of node cluster | |
| WO2003003244A1 (en) | Method of rapidly eliminating different information in databases | |
| CN112685142A (en) | Distributed data processing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |