CN111444214A - Method and device for processing large-scale data and industrial monitoring memory database - Google Patents

Method and device for processing large-scale data and industrial monitoring memory database Download PDF

Info

Publication number
CN111444214A
CN111444214A CN202010247316.6A CN202010247316A CN111444214A CN 111444214 A CN111444214 A CN 111444214A CN 202010247316 A CN202010247316 A CN 202010247316A CN 111444214 A CN111444214 A CN 111444214A
Authority
CN
China
Prior art keywords
record
recording
read
address corresponding
lock
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010247316.6A
Other languages
Chinese (zh)
Inventor
王志鹏
刘伟东
盛志冰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hisense TransTech Co Ltd
Original Assignee
Hisense TransTech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hisense TransTech Co Ltd filed Critical Hisense TransTech Co Ltd
Priority to CN202010247316.6A priority Critical patent/CN111444214A/en
Publication of CN111444214A publication Critical patent/CN111444214A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2308—Concurrency control
    • G06F16/2336—Pessimistic concurrency control approaches, e.g. locking or multiple versions without time stamps
    • G06F16/2343—Locking methods, e.g. distributed locking or locking implementation details
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2282—Tablespace storage structures; Management thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a method and a device for processing large-scale data and an industrial monitoring memory database, which comprises the following steps: the method comprises the steps of obtaining a read-write instruction, wherein the read-write instruction comprises a record ID, determining a record address corresponding to the record ID according to the record ID and the corresponding relation between the record ID and the record address, and calling a process to perform read-write operation on a record on the record address corresponding to the record ID, so that after the read-write instruction is obtained, the record address corresponding to the record ID is directly located through the record ID according to the record ID and the corresponding relation between the record ID and the record address, and the record searching efficiency is improved.

Description

Method and device for processing large-scale data and industrial monitoring memory database
Technical Field
The invention relates to the field of data processing, in particular to a method and a device for processing large-scale data and an industrial monitoring memory database.
Background
With the increasingly wide application of industrial monitoring systems in the industries such as rail transit, electric power systems and the like, the development of the industries puts higher requirements on the scale and the real-time performance of the systems, the scale of the systems rises from hundreds of thousands of points to millions of points, some systems already put requirements on the scale of tens of millions of points, and the capacity of collection and processing points of the scale has very high requirements on the performance of a memory database.
However, in the prior art, when the memory database is read and written, data positioning is performed based on the index, and the positioning data has poor efficiency.
Disclosure of Invention
The embodiment of the invention provides a method and a device for processing large-scale data and an industrial monitoring memory database, which are used for directly positioning a recording address of a data record, improving the efficiency of searching the record and preventing concurrency conflict during reading and writing the record.
In a first aspect, an embodiment of the present invention provides a method for processing large-scale data and an industrial monitoring memory database, including:
acquiring a read-write instruction, wherein the read-write instruction comprises a record ID;
determining a recording address corresponding to the recording ID according to the recording ID and the corresponding relation between the recording ID and the recording address;
and calling the process to perform read-write operation on the record address corresponding to the record ID.
According to the technical scheme, after the read-write instruction is obtained, the recording address corresponding to the recording ID can be directly positioned through the recording ID according to the recording ID and the recording address corresponding to the recording ID, a data positioning searching process based on indexes is not used, recording searching time is shortened, and recording searching efficiency is improved.
Optionally, the invoking process performs a read-write operation on the record address corresponding to the record ID, including:
determining the delay processing time of the current period of the process;
and calling the process to perform read-write operation on the record address corresponding to the record ID according to the delay processing time.
In the technical scheme, different starting times of the process are set according to the delay processing time of each period of the process, and then the record on the record address corresponding to the record ID is read and written according to the different starting times of the process, so that the concurrent conflict of the record during reading and writing is prevented.
Optionally, the invoking process performs a read-write operation on the record address corresponding to the record ID, including:
calling the process to determine whether the mark lock of the recording address corresponding to the recording ID is in a locked state, if not, performing read-write operation on the recording address corresponding to the recording ID, setting the mark lock of the recording address corresponding to the recording ID to be in the locked state, and after the read-write operation is completed, unlocking the mark lock of the recording address corresponding to the recording ID, which is in the locked state; otherwise, re-determining whether the mark lock of the recording address corresponding to the recording ID is in a locked state at intervals of preset time, and if the re-determined times exceed the preset times, returning a message that the read-write operation is unsuccessful.
In the above technical solution, before the process reads and writes the record address corresponding to the record ID, if it is determined that the flag lock of the record address corresponding to the record ID is in the locked state, the process is not allowed to perform the read and write operation, thereby preventing a concurrence conflict from occurring when the process reads and writes the record, and if it is determined that the flag lock of the record address corresponding to the record ID is not in the locked state, the process reads and writes the record address corresponding to the record ID, and sets the flag lock of the record address corresponding to the record ID to the locked state during the read and write operation, thereby preventing a concurrence conflict from occurring when the process reads and writes the record.
Optionally, the mark lock is a library mark lock, a table mark lock, or a record mark lock.
According to the technical scheme, the mark lock can be arranged in multiple granularities of the memory database, and the mark lock is arranged in different granularities, so that concurrency conflict can be prevented when the process reads and writes records, and the read-write recording efficiency is improved.
Optionally, determining a recording address corresponding to the recording ID according to formula (1);
Pr＝Pz+(IDr-IDz)*L………………………………(1)
wherein, PrTo record the address of r, PzFor tables in-memory databasesInitial address, IDrTo record a record ID, ID of rzIs the initial record ID of the in-memory database table, r is a natural number, z is a natural number, L is the byte length of the record.
In the technical scheme, according to the relation between the record ID and the recorded address, the recorded address can be directly searched through the record ID, and the record searching efficiency is improved.
Optionally, the method further includes:
and judging whether the record is in a deadlock state, and if so, clearing the zone bit of the mark lock.
In the technical scheme, the flag bit of the recording flag lock in the deadlock state is cleared, the execution process is reset, and the read-write recording is prevented from being abnormal.
In a second aspect, an embodiment of the present invention provides an apparatus for processing large-scale data and an industrial monitoring memory database, including:
the acquisition module is used for acquiring a read-write instruction, and the read-write instruction comprises a record ID;
the processing module is used for determining a recording address corresponding to the recording ID according to the recording ID and the corresponding relation between the recording ID and the recording address;
and calling the process to perform read-write operation on the record address corresponding to the record ID.
Optionally, the processing module is specifically configured to:
determining the delay processing time of the current period of the process;
and calling the process to perform read-write operation on the record address corresponding to the record ID according to the delay processing time.
Optionally, the processing module is specifically configured to:
calling the process to determine whether the mark lock of the recording address corresponding to the recording ID is in a locked state, if not, performing read-write operation on the recording address corresponding to the recording ID, setting the mark lock of the recording address corresponding to the recording ID to be in the locked state, and after the read-write operation is completed, unlocking the mark lock of the recording address corresponding to the recording ID, which is in the locked state; otherwise, re-determining whether the mark lock of the recording address corresponding to the recording ID is in a locked state at intervals of preset time, and if the re-determined times exceed the preset times, returning a message that the read-write operation is unsuccessful.
Optionally, the mark lock is a library mark lock, a table mark lock, or a record mark lock.
Optionally, the processing module is specifically configured to:
determining a recording address corresponding to the recording ID according to a formula (1);
Pr＝Pz+(IDr-IDz)*L…………………………………………(1)
wherein, PrTo record the address of r, PzFor initial addresses, IDs, of tables in the in-memory databaserTo record a record ID, ID of rzIs the initial record ID of the in-memory database table, r is a natural number, z is a natural number, L is the byte length of the record.
Optionally, the processing module is further configured to:
and judging whether the record is in a deadlock state, and if so, clearing the zone bit of the mark lock.
In a third aspect, an embodiment of the present invention further provides a computing device, including:
a memory for storing program instructions;
and the processor is used for calling the program instructions stored in the memory and executing the method for processing the large-scale data and the industrial monitoring memory database according to the obtained program.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, where computer-executable instructions are stored, and the computer-executable instructions are configured to enable a computer to execute the method for processing the large-scale data and the industrial monitoring in-memory database.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a system architecture diagram according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating a method for large-scale data and industrial monitoring of in-memory database processing according to an embodiment of the present invention;
FIG. 3 is a flow chart illustrating a method for large-scale data and industrial monitoring of in-memory database processing according to an embodiment of the present invention;
FIG. 4 is a flow chart illustrating a method for large-scale data and industrial monitoring of in-memory database processing according to an embodiment of the present invention;
FIG. 5 is a flow chart illustrating a method for large-scale data and industrial monitoring of in-memory database processing according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an apparatus for processing large-scale data and industrial monitoring in-memory databases according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 illustrates an exemplary system architecture to which embodiments of the present invention are applicable, which includes a server 100, where the server 100 may include a processor 110, a communication interface 120, and a memory 130.
The communication interface 120 is used for transmitting processes and data.
The processor 110 is a control center of the server 100, connects various parts of the entire server 100 using various interfaces and routes, performs various functions of the server 100 and processes data by operating or executing software programs and/or modules stored in the memory 130 and calling data stored in the memory 130. Alternatively, processor 110 may include one or more processing units.
The memory 130 may be used to store software programs and modules, and the processor 110 executes various functional applications and data processing by operating the software programs and modules stored in the memory 130. The memory 130 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to a business process, and the like. Further, the memory 130 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
It should be noted that the structure shown in fig. 1 is only an example, and the embodiment of the present invention is not limited thereto.
Based on the above description, fig. 2 exemplarily illustrates a flow of a method for processing a large-scale data and industrial monitoring in-memory database according to an embodiment of the present invention, where the flow can be performed by an apparatus for processing a large-scale data and industrial monitoring in-memory database.
As shown in fig. 2, the process specifically includes:
step 201, a read-write instruction is obtained.
In the embodiment of the present invention, the read/write command includes a record ID, and a record needs to be created before the record ID is acquired, specifically, a table is created in the memory database, and the table includes: creating basic attributes such as a table structure, a maximum record number and a reserved record number, for example, the maximum record number is Nm, which refers to the maximum record number carried by the table, the reserved record number is Nr, which refers to the reserved record number of the table, and is used for expanding the maximum record number, creating records in the table according to an array storage structure, and then the total record number in the table is N-Nm + Nr.
The record refers to a row in a table created by the in-memory database, and is used for storing data, for example, storing information related to a certain client on a record in a client mailing list.
Step 202, determining a recording address corresponding to the recording ID according to the recording ID and the corresponding relation between the recording ID and the recording address.
According to the embodiment of the invention, a recording address corresponding to the recording ID is determined according to a formula (1);
Pr＝Pz+(IDr-IDz)*L…………………………………………………(1)
wherein, PrTo record the address of r, PzFor initial addresses, IDs, of tables in the in-memory databaserTo record a record ID, ID of rzIs the initial record ID of the in-memory database table, r is a natural number, z is a natural number, L is the byte length of the record.
According to the corresponding relationship between the record ID and the record address, the record ID in the read-write command and the formula (1) can be directly positioned to the record address corresponding to the record ID, for example, the initial ID of the table is defaulted to 0, and then the initial address P of the table iszIs P0The record ID of the table is also regarded as the start ID of the table and is ID0Then IDrR in (1), i.e., the record ID of the record r is ID, is 0, 1rAccording to ID0，IDrLength of bytes recorded L, initial address of the table P0Determining the address P of record r in combination with equation (1)r。
It should be noted that the initial ID of the table may be set to any value through an interface provided by the in-memory database, and the application program modifies the initial ID of the table according to the range of the record ID of the table.
According to the calculation, the complicated hash operation and hash table searching process can be avoided, and the record searching efficiency is improved.
Step 203, invoking the process to perform read-write operation on the record address corresponding to the record ID.
According to the embodiment of the invention, after the record is found according to the read-write instruction and the record address corresponding to the record ID, the process can be called in the following two ways to perform read-write operation on the found record, so that the concurrency conflict generated during the read-write of the record is prevented.
Mode 1
And determining the delay processing time of the current period of the process, and calling the process to perform read-write operation on the record address corresponding to the record ID according to the delay processing time.
The process is enabled to perform read-write operation at different initial times by setting the delay processing time of different periods of the process, and the read-write operation of the process is separated in time, so that concurrency conflict during reading and writing is prevented, wherein the delay processing time of different periods of the process can be embodied by a time control lock, for example, the time control lock is set in the process, and the time control lock comprises the information in the following table 1.
TABLE 1
Attribute name Type (B)
Time control lock zone bit int1
Time-controlled lock operation cycle uint4
Time-delay processing time of time-controlled lock uint4
Time control lock locking process read-write time uint4
Wherein int1 is a 1-byte integer data type, and uint4 is a 4-byte unsigned integer data type, the following steps are described in detail with reference to specific embodiments, and the following procedures are algorithmic pseudo code.
Locking by a time control lock:
Figure BDA0002434291190000081
unlocking a time control lock:
Function TimeUnlock(DBTimeLock lock
lock.bLocked＝false；
End
according to the above algorithm pseudo code, the process is caused to perform the read/write operation shown in fig. 3 by the time control lock, as shown in fig. 3, 300 is the time control lock operation cycle time of the process 1, the process 2 and the process 3, 311 is the time control lock delay processing time of the process 1, 312 is the time control lock process read/write time of the process 1, 321 is the time control lock delay processing time of the process 2, 322 is the time control lock process read/write time of the process 2, 331 is the time control lock delay processing time of the process 3, 332 is the time control lock process read/write time of the process 3, after the time control lock process read/write time 312 of the process 1 is finished, the time control lock process read/write time of the process 2 is started, the process 2 is allowed to perform read/write recording, and so on, the Application scenario of the time lock is a periodic execution process, and an API (Application program Interface) Interface provided by a memory database is used, the read-write record is separated in time, so that the process is executed in time error, and concurrent read-write during the read-write record is prevented.
Mode 2
The calling process determines whether the mark lock of the recording address corresponding to the recording ID is in a locked state, if not, the reading and writing operation is carried out on the recording address corresponding to the recording ID, the mark lock of the recording address corresponding to the recording ID is set to be in the locked state, and after the reading and writing operation is completed, the mark lock of the recording address corresponding to the recording ID in the locked state is unlocked; otherwise, re-determining whether the mark lock of the recording address corresponding to the recording ID is in a locked state at intervals of preset time, and if the re-determined times exceed the preset times, returning a message that the read-write operation is unsuccessful.
After the recording address corresponding to the recording ID is obtained, calling a process to determine whether the mark lock of the recording address corresponding to the recording ID is in a locking state, performing read-write operation on the recording address corresponding to the recording ID, setting the mark lock of the recording address corresponding to the recording ID to be in the locking state, then after the read-write operation is completed, setting the mark lock of the recording address corresponding to the recording ID to be in an unlocking state, otherwise, re-determining whether the mark lock of the recording address corresponding to the recording ID is in the locking state or not through preset time, and if the re-determination times exceed the preset times, returning a message that the process read-write operation is unsuccessful.
It should be noted that the flag lock is a library flag lock, a table flag lock, or a record flag lock.
The granularity of the flag lock may be divided into three levels, including library flag lock, table flag lock, or record flag lock, and when a process is invoked, it is determined whether the flag lock is in a locked state for the flag lock of the same level of granularity, where the granularity is a memory database term, and is specifically the information in table 2 below.
TABLE 2
Attribute name Type (B)
Marker lock zone bit int2
Sign lock locking or unlocking process uint4
Time of locking sign lock uint8
Wherein int2 is a 2-byte integer data type, and uint8 is an 8-byte unsigned integer data type, and the following program is an algorithmic pseudo code, and the flag setting process is interpreted according to the following algorithmic pseudo code.
The sign locking algorithm:
Figure BDA0002434291190000101
sign lock unlocking algorithm:
Figure BDA0002434291190000102
the algorithm pseudo code is locking or unlocking operation completed by using an atomic plus or minus 1 interface related to an operating system.
In order to better explain the above technical solution, the process of locking or unlocking the above-mentioned sign lock will be described in the following specific examples.
Example 1
As shown in fig. 4, fig. 4 is a process of locking a mark lock, and the specific process includes:
step 401, determining whether the flag lock is in a locked state, if so, executing step 402, otherwise, executing step 405.
And judging whether the mark lock is in the locking state according to the mark lock zone bit in the table 2, for example, when the mark lock zone bit is '1', determining that the mark lock is in the locking state.
Step 402, the state of the flag lock is determined again after a preset time.
If the current process determines that the flag lock is in the locked state, the state of the flag lock is determined again after a preset time, for example, if the flag lock of the record of the current process on the record address corresponding to the record ID determined to be read and written is in the locked state, it is considered that the record may be being read and written by other processes, and after the preset time, the state of the flag lock is determined again, so that the waiting time for reading and writing the record is reduced by the current process.
Step 403, determine whether the number of times of determining the flag bit is greater than 3, if yes, execute step 404, otherwise return to step 401.
And judging whether the number of times that whether the flag lock is in the locking state or not is determined again by the current process is more than 3 times (preset number of times).
In step 404, if 403 is true, the current process fails to execute.
If the number of times that the flag lock of the record on the record address corresponding to the record ID is determined to be in the locked state by the current process is greater than 3 times, it is determined that the current process fails to execute, and a result of the previous process fails to execute is returned.
Step 405, locking.
When the calling process determines that the flag lock of the record on the record address corresponding to the record ID is not in the locked state, the read/write operation is performed on the record address corresponding to the record ID, and the flag lock of the record on the record address corresponding to the record ID is set to be in the locked state, for example, the record is locked according to the flag lock locking time in table 2.
Step 406, return the result.
And returning a result which marks that the lock is set to be in a locking state success or failure.
Example 2
As shown in fig. 5, fig. 5 is a flow of unlocking the sign lock, and the specific flow includes:
step 501, judging whether the flag lock is in a locked state, if so, executing step 502, otherwise, executing step 503.
According to the flag lock flag bit in table 2, it is determined whether the flag lock is in the locked state, for example, when the flag lock flag bit is "0", it is determined that the flag lock is in the unlocked state.
And 502, unlocking.
After the current process finishes the read-write operation of the record on the record address corresponding to the record ID, if the flag lock is judged to be in the locked state, the flag lock of the record on the record address corresponding to the record ID is unlocked, so that the flag lock of the record on the record address corresponding to the record ID is in the unlocked state, for example, after the current process finishes, the flag lock flag bit is judged to be '0', and the flag lock is determined to be successfully unlocked.
Step 503, return the result.
And returning a result of successful or failed setting of the flag lock, for example, after the process finishes the read-write operation on the record address corresponding to the record ID, judging that the flag lock is not in the locking state, and if the process is considered to execute the read-write operation, setting the flag lock of the record address corresponding to the record ID to be in the locking state and failing to set, and returning a result of failed setting of the flag lock.
In the process of executing the method for processing the large-scale data and the industrial monitoring memory database, whether the record is in a deadlock state is judged, if so, the zone bit of the flag lock is cleared, for example, if SignUnlock is not executed due to abnormal exit of the process, the zone bit of the flag lock is cleared, namely reset, and the initial state is recovered.
It should be noted that a process is a running activity of a program in a computer on a certain data set, is a basic unit for resource allocation and scheduling of a system, and is a basis of an operating system structure, and in a computer structure designed by a thread, a process is a container of the thread, that is, in the embodiment of the present invention, it is also applicable that the thread performs read-write operations.
According to the embodiment of the invention, after the record ID in the read-write instruction is obtained, the record address corresponding to the record ID is determined according to the record ID and the corresponding relation between the record ID and the record address, the delay processing time of the current period of the process is determined according to the time control lock, the process is called to carry out read-write operation on the record address corresponding to the record ID according to the delay processing time, or the process read-write operation is controlled according to the state of the mark lock, and the zero clearing function of the mark bit is set, so that the record is prevented from being in a deadlock state, the efficiency of searching the record is improved, and the occurrence of concurrency conflict during the read-write record is prevented.
Based on the same technical concept, fig. 6 exemplarily shows a structure of an apparatus for large-scale data and industrial monitoring memory database processing, which can perform a method for large-scale data and industrial monitoring memory database processing according to an embodiment of the present invention.
As shown in fig. 6, the apparatus specifically includes:
an obtaining module 601, configured to obtain a read-write instruction, where the read-write instruction includes a record ID;
a processing module 602, configured to determine, according to the record ID and the corresponding relationship between the record ID and the record address, a record address corresponding to the record ID;
and calling the process to perform read-write operation on the record address corresponding to the record ID.
Optionally, the processing module 602 is specifically configured to:
determining the delay processing time of the current period of the process;
and calling the process to perform read-write operation on the record address corresponding to the record ID according to the delay processing time.
Optionally, the processing module 602 is specifically configured to:
calling the process to determine whether the mark lock of the recording address corresponding to the recording ID is in a locked state, if not, performing read-write operation on the recording address corresponding to the recording ID, setting the mark lock of the recording address corresponding to the recording ID to be in the locked state, and after the read-write operation is completed, unlocking the mark lock of the recording address corresponding to the recording ID, which is in the locked state; otherwise, re-determining whether the mark lock of the recording address corresponding to the recording ID is in a locked state at intervals of preset time, and if the re-determined times exceed the preset times, returning a message that the read-write operation is unsuccessful.
Optionally, the mark lock is a library mark lock, a table mark lock, or a record mark lock.
Optionally, the processing module 602 is specifically configured to:
determining a recording address corresponding to the recording ID according to a formula (1);
Pr＝Pz+(IDr-IDz)*L………………………………………………(1)
wherein, PrTo record the address of r, PzFor initial addresses, IDs, of tables in the in-memory databaserTo record a record ID, ID of rzIs the initial record ID of the in-memory database table, r is a natural number, z is a natural number, L is the byte length of the record.
Optionally, the processing module 602 is further configured to:
and judging whether the record is in a deadlock state, and if so, clearing the zone bit of the mark lock.
Based on the same technical concept, an embodiment of the present invention further provides a computing device, including:
a memory for storing program instructions;
and the processor is used for calling the program instructions stored in the memory and executing the method for processing the large-scale data and the industrial monitoring memory database according to the obtained program.
Based on the same technical concept, the embodiment of the invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used for enabling a computer to execute the method for processing the large-scale data and the industrial monitoring memory database.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims (10)

1. A method for large-scale data and industrial monitoring memory database processing is characterized by comprising the following steps:
acquiring a read-write instruction, wherein the read-write instruction comprises a record ID;
determining a recording address corresponding to the recording ID according to the recording ID and the corresponding relation between the recording ID and the recording address;
and calling the process to perform read-write operation on the record address corresponding to the record ID.
2. The method of claim 1, wherein the invoking process performs a read-write operation on the record at the record address corresponding to the record ID, comprising:
determining the delay processing time of the current period of the process;
and calling the process to perform read-write operation on the record address corresponding to the record ID according to the delay processing time.
3. The method of claim 1, wherein the invoking process performs a read-write operation on the record at the record address corresponding to the record ID, comprising:
calling the process to determine whether the mark lock of the recording address corresponding to the recording ID is in a locked state, if not, performing read-write operation on the recording address corresponding to the recording ID, setting the mark lock of the recording address corresponding to the recording ID to be in the locked state, and after the read-write operation is completed, unlocking the mark lock of the recording address corresponding to the recording ID, which is in the locked state; otherwise, re-determining whether the mark lock of the recording address corresponding to the recording ID is in a locked state at intervals of preset time, and if the re-determined times exceed the preset times, returning a message that the read-write operation is unsuccessful.
4. The method of claim 3, wherein the flag lock is a library flag lock or a table flag lock or a record flag lock.
5. The method of claim 1, wherein a recording address corresponding to the recording ID is determined according to formula (1);
Pr＝Pz+(IDr-IDz)*L...............................(1)
wherein, PrTo record the address of r, PzFor initial addresses, IDs, of tables in the in-memory databaserTo record a record ID, ID of rzIs the initial record ID of the in-memory database table, r is a natural number, z is a natural number, L is the byte length of the record.
6. The method of claim 1, wherein the method further comprises:
and judging whether the record is in a deadlock state, and if so, clearing the zone bit of the mark lock.
7. An apparatus for large scale data and industrial monitoring of in-memory database processing, comprising:
the acquisition module is used for acquiring a read-write instruction, and the read-write instruction comprises a record ID;
the processing module is used for determining a recording address corresponding to the recording ID according to the recording ID and the corresponding relation between the recording ID and the recording address;
and calling the process to perform read-write operation on the record address corresponding to the record ID.
8. The apparatus of claim 7, wherein the processing module is specifically configured to:
determining the delay processing time of the current period of the process;
and calling the process to perform read-write operation on the record address corresponding to the record ID according to the delay processing time.
9. A computing device, comprising:
a memory for storing program instructions;
a processor for calling program instructions stored in said memory to execute the method of any one of claims 1 to 6 in accordance with the obtained program.
10. A computer-readable storage medium having computer-executable instructions stored thereon for causing a computer to perform the method of any one of claims 1 to 6.
CN202010247316.6A 2020-03-31 2020-03-31 Method and device for processing large-scale data and industrial monitoring memory database Pending CN111444214A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010247316.6A CN111444214A (en) 2020-03-31 2020-03-31 Method and device for processing large-scale data and industrial monitoring memory database

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010247316.6A CN111444214A (en) 2020-03-31 2020-03-31 Method and device for processing large-scale data and industrial monitoring memory database

Publications (1)

Publication Number Publication Date
CN111444214A true CN111444214A (en) 2020-07-24

Family

ID=71649455

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010247316.6A Pending CN111444214A (en) 2020-03-31 2020-03-31 Method and device for processing large-scale data and industrial monitoring memory database

Country Status (1)

Country Link
CN (1) CN111444214A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112231131A (en) * 2020-09-28 2021-01-15 北京金山云网络技术有限公司 Method, device and equipment for realizing database lock and readable storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101706815A (en) * 2009-12-03 2010-05-12 北京和利时系统工程有限公司 Database equipment and system
CN102651006A (en) * 2011-02-25 2012-08-29 上海网环信息科技有限公司 Database table record locking method and device

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101706815A (en) * 2009-12-03 2010-05-12 北京和利时系统工程有限公司 Database equipment and system
CN102651006A (en) * 2011-02-25 2012-08-29 上海网环信息科技有限公司 Database table record locking method and device

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112231131A (en) * 2020-09-28 2021-01-15 北京金山云网络技术有限公司 Method, device and equipment for realizing database lock and readable storage medium
CN112231131B (en) * 2020-09-28 2024-05-28 北京金山云网络技术有限公司 Method, device and equipment for realizing database lock and readable storage medium

Similar Documents

Publication Publication Date Title
EP0457473B1 (en) Method for accessing shared data
US8832159B2 (en) Systems and methods for asynchronous schema changes
CN105955804B (en) A kind of method and apparatus for handling distributed transaction
WO2021027956A1 (en) Blockchain system-based transaction processing method and device
JP2010539609A (en) Parallel nested transactions in transactional memory
CN101308471B (en) Method and device for data restoration
CN104937556A (en) Recovering pages of database
CN107665219B (en) Log management method and device
TWI730690B (en) Method and device for simultaneously executing transactions in block chain, computer readable storage medium and computing equipment
CN106897342B (en) Data verification method and equipment
US6944635B2 (en) Method for file deletion and recovery against system failures in database management system
US20160162366A1 (en) A method for a logging process in a data storage system
CN110675255A (en) Method and apparatus for concurrently executing transactions in a blockchain
WO2017028705A1 (en) Computer system task backup method, system state recovery method and devices thereof
US11983168B2 (en) Block verification method, apparatus and device
CN109726264B (en) Method, apparatus, device and medium for index information update
Padhye et al. Scalable transaction management with snapshot isolation on cloud data management systems
CN111444214A (en) Method and device for processing large-scale data and industrial monitoring memory database
CN113505114A (en) Multi-version concurrency control method of database and database system
CN111597193B (en) Tree data locking and unlocking method
CN110706108B (en) Method and apparatus for concurrently executing transactions in a blockchain
KR20120082176A (en) Data processing method of database management system and system thereof
US5497487A (en) Merge, commit recovery protocol for real-time database management systems
US5524239A (en) Replay recovery protocol for real-time database management systems
JPS62245348A (en) Method and device for updating data base

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200724

RJ01 Rejection of invention patent application after publication