CN111813783A - Data processing method, data processing device, computer equipment and storage medium - Google Patents
Data processing method, data processing device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN111813783A CN111813783A CN202010730402.2A CN202010730402A CN111813783A CN 111813783 A CN111813783 A CN 111813783A CN 202010730402 A CN202010730402 A CN 202010730402A CN 111813783 A CN111813783 A CN 111813783A
- Authority
- CN
- China
- Prior art keywords
- data
- power data
- power
- memory space
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012545 processing Methods 0.000 title claims abstract description 144
- 238000003672 processing method Methods 0.000 title abstract description 18
- 238000000034 method Methods 0.000 claims abstract description 89
- 230000001960 triggered effect Effects 0.000 claims abstract description 20
- 230000008569 process Effects 0.000 claims description 51
- 238000004590 computer program Methods 0.000 claims description 14
- 238000012986 modification Methods 0.000 claims description 8
- 230000004048 modification Effects 0.000 claims description 8
- 238000012217 deletion Methods 0.000 claims description 4
- 230000037430 deletion Effects 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 14
- 238000004422 calculation algorithm Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 238000013507 mapping Methods 0.000 description 4
- FFBHFFJDDLITSX-UHFFFAOYSA-N benzyl N-[2-hydroxy-4-(3-oxomorpholin-4-yl)phenyl]carbamate Chemical compound OC1=C(NC(=O)OCC2=CC=CC=C2)C=CC(=C1)N1CCOCC1=O FFBHFFJDDLITSX-UHFFFAOYSA-N 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000005192 partition Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007717 exclusion Effects 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- Software Systems (AREA)
- Human Resources & Organizations (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- General Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a data processing method, a data processing device, computer equipment and a storage medium. After a data processing instruction triggered by a user in a system is acquired, a data identifier of power data to be processed is acquired from a private first memory space corresponding to the user, a target data table corresponding to the power data to be processed is determined from a file system of a second memory space according to the data identifier, wherein the second memory space is a public memory space used for storing the power data, and finally the power data in the target data table are processed according to processing mode information in the data processing instruction. Compared with the traditional method for processing the electric power data by utilizing the open source memory bank and the like, the method and the device have the advantages that the electric power data are quickly, simply and conveniently processed through the data table and the memory space separating mode, and the electric power data processing efficiency is improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a mapping-based file processing method and apparatus, a computer device, and a storage medium.
Background
The electric power is one of important energy sources for guaranteeing normal operation of life of people, along with the development of the electric power industry, electric power equipment in an electric power system is gradually increased, for example, data items and density of power utilization information acquisition of users are gradually increased, in the operation process of the electric power system, electric power data are generally required to be processed, at present, a processing mode of the electric power data is generally processed in a database, for example, sqlite and the like, however, as the data of the electric power data are continuously produced, the data volume is large, the current database lacks professional service support, and the performance cannot meet the requirements.
Therefore, the current data processing method has the defect of low processing efficiency.
Disclosure of Invention
In view of the above, it is necessary to provide a data processing method, an apparatus, a computer device, and a storage medium capable of improving data processing efficiency.
A method of data processing, the method comprising:
responding to a data processing instruction triggered by a user in a system, and acquiring a data identifier of power data to be processed from a first memory space; the data processing instruction comprises processing mode information aiming at the power data to be processed; the first memory space is a private memory space corresponding to the user;
determining a target data table corresponding to the to-be-processed power data from a file system according to the data identification; the file system is arranged in a second memory space and comprises a plurality of data tables; the second memory space is a public memory space for storing the power data;
and processing the power data in the target data table according to the processing mode information.
In one embodiment, the first memory space includes an identification table, the data identifier is stored in the identification table, and the identification table includes at least one data identifier;
before the data processing instruction triggered by the user is used for acquiring the data identifier of the power data to be processed from the first memory space corresponding to the user, the method further includes:
detecting whether a data identifier corresponding to the to-be-processed power data exists in the identifier table;
if not, a data identifier corresponding to the to-be-processed power data is established in the identifier table, and an association relation between the data identifier and the to-be-processed power data is established.
In one embodiment, the system comprises a plurality of processes, and the processes correspond to the users one by one;
the method further comprises the following steps:
responding to a data processing instruction triggered by a user in a system, adding a process lock to a current process corresponding to the user, and switching the state of the target data table to be an in-use state so as to prevent processes corresponding to other users from processing the power data in the target data table;
after the processing the power data in the target data table according to the processing mode information, the method further includes:
and releasing the process lock, and switching the state of the target data table to an unused state.
In one embodiment, the processing mode information includes: at least one of adding data, deleting data, and modifying data;
the processing the power data in the target data table according to the processing mode information includes:
if the processing mode information is data to be added, adding power data to be added to the tail end of the target data table; the power data to be increased is determined by the user;
if the processing mode information is deletion data, replacing the power data in the target data table with null data;
if the processing mode information is modification data, determining the content needing to be modified according to the data identification and the power data to be modified; the power data to be modified is determined by the user.
In one embodiment, if the processing mode information is addition data, before adding the power data to be added to the end of the target data table, the method further includes:
judging whether the size of the power data to be increased is larger than the remaining space of the second memory space;
if so, increasing the size of the second memory space to store the power data to be increased.
In one embodiment, if the processing mode information is deletion data, replacing the power data in the target data table with null data includes:
replacing null data of an end location of the target data table with the power data to store the power data to the end location;
and associating the data identifier corresponding to the power data with the terminal position, and deleting the data identifier corresponding to the power data.
In one embodiment, the determining, according to the data identifier and the power data to be modified, what needs to be modified includes:
judging whether the power data to be modified is the power data corresponding to the data identifier;
if not, determining the content needing to be modified as the power data to be modified;
if yes, determining the content needing to be modified as the power data to be modified and the data identification corresponding to the power data to be modified.
A data processing apparatus, the apparatus comprising:
the acquisition module is used for responding to a data processing instruction triggered by a user in the system and acquiring a data identifier of the power data to be processed from the first memory space; the data processing instruction comprises processing mode information aiming at the power data to be processed; the first memory space is a private memory space corresponding to the user;
the determining module is used for determining a target data table corresponding to the to-be-processed power data from a file system according to the data identification; the file system is arranged in a second memory space and comprises a plurality of data tables; the second memory space is a public memory space for storing the power data;
and the processing module is used for processing the electric power data in the target data table according to the processing mode information.
A computer device comprising a memory storing a computer program and a processor implementing the steps of the method described above when executing the computer program.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method.
According to the data processing method, the data processing device, the computer equipment and the storage medium, after a data processing instruction triggered by a user in a system is obtained, a data identifier of power data to be processed is obtained from a private first memory space corresponding to the user, a target data table corresponding to the power data to be processed is determined from a file system of a second memory space according to the data identifier, wherein the second memory space is a public memory space used for storing the power data, and finally the power data in the target data table is processed according to processing mode information in the data processing instruction. Compared with the traditional method for processing the electric power data by utilizing the open source memory bank and the like, the method and the device have the advantages that the electric power data are quickly, simply and conveniently processed through the data table and the memory space separating mode, and the electric power data processing efficiency is improved.
Drawings
FIG. 1 is a diagram of an application environment of a data processing method in one embodiment;
FIG. 2 is a flow diagram illustrating a data processing method according to one embodiment;
FIG. 3 is a diagram illustrating the structure of a data table in one embodiment;
FIG. 4 is a diagram illustrating a structure of a header of a data table in one embodiment;
FIG. 5 is a diagram illustrating the structure of a table body of the data table according to one embodiment;
FIG. 6 is a diagram illustrating a structure of an association between a data identifier and a data table in one embodiment;
FIG. 7 is a flowchart illustrating the processing steps of a multiprocess in one embodiment;
FIG. 8 is a block diagram showing the structure of a data processing apparatus according to an embodiment;
FIG. 9 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The data processing method provided by the application can be applied to the application environment shown in fig. 1. The terminal 102 may include a memory, and the terminal 102 may obtain a data processing instruction triggered by a user in the system, obtain a data identifier of the power data to be processed from the first memory space, determine a target data table corresponding to the power data to be processed from the file system located in the second memory space according to the data identifier, and process the power data in the target data table according to the processing mode information in the data processing instruction. In an embodiment, the terminal 102 may further include a server 104, where the first memory space and the second memory space may also be disposed in a memory of the server, and the terminal 102 may obtain the data table and power data therein from the server 104, and process the power data in the terminal 102. The terminal 102 communicates with the server 104 through a network, the terminal 102 may be, but is not limited to, various personal computers, notebook computers, smart phones, and tablet computers, and the server 104 may be implemented by an independent server or a server cluster formed by a plurality of servers.
In one embodiment, as shown in fig. 2, a data processing method is provided, which is described by taking the application of the method to the terminal in fig. 1 as an example, and includes the following steps:
step S202, responding to a data processing instruction triggered by a user in a system, and acquiring a data identifier of power data to be processed from a first memory space; the data processing instruction comprises processing mode information aiming at the power data to be processed; the first memory space is a private memory space corresponding to the user.
The user may be a user using the system, the system may be an electric power system, the first memory space may be a space created based on a memory in the system, for example, a part of the memory is separately partitioned from the memory to serve as the first memory space, and the first memory space may be a private memory space corresponding to the user. The user may trigger a data processing instruction in the system, for example, click a corresponding data processing button in the system, and the terminal 102 may respond to the data processing instruction triggered by the user in the system and obtain a data identifier of the power data to be processed from the first memory space, where the data processing instruction may include processing mode information of the power data to be processed. The data identifier may be an identifier for indicating corresponding to-be-processed power data, specifically, the data identifier may be an index, and the index may include a storage location of the to-be-processed power data, for example, the to-be-processed power data is stored in a certain data table.
In addition, the index can also support a default hash algorithm and a custom algorithm, the hash value of the field set corresponding to the index is calculated, and can be determined by an external user custom algorithm, the custom algorithm is defined according to service characteristics, for example, the index field is a terminal address, the front bits of the terminal address are fixed values of service information, and the rear 4 bits are incremental serial numbers, so that the rear 4 bits of the terminal address can be directly taken as the hash value, and the high-efficiency index algorithm is realized. Under the condition that no special appointed algorithm is available, a default hash algorithm is adopted, and the use is convenient.
Step S204, determining a target data table corresponding to the power data to be processed from the file system according to the data identification; the file system is arranged in the second memory space and comprises a plurality of data tables; the second memory space is a common memory space for storing power data.
The data identifier may be the data identifier corresponding to the to-be-processed power data stored in the first memory space in step S202, and the file system may store a plurality of data tables, and may also be used to manage the data tables, for example, add data, delete data, or modify data; the file system may be stored in the second memory space, and the second memory space may be a common memory space for storing power data, for example, the power data may be stored by using a data table in the file system. Wherein, a plurality of power data can be stored in the data table. The terminal 102 may determine, according to the data identifier, a target data table corresponding to the to-be-processed power data from the file system, that is, a data table in which the to-be-processed power data is located.
Specifically, as shown in fig. 3, fig. 3 is a schematic structural diagram of a data table in an embodiment. The internal structure of the data table in the file system can be divided into header data and body data, the header data is of a fixed-length structure, and the body data area can record electric power data according to the actual situation and can be dynamically expanded. As shown in fig. 4, fig. 4 is a schematic structural diagram of a header of a data table in one embodiment. The data structure of the header of the data table may include a plurality of fields, each of which may include a plurality of bytes, the table name supports an alias structure, and a "may be used between table name and alias": "separate, if no alias is defined, default alias to table name. As shown in fig. 5, fig. 5 is a schematic structural diagram of a data table body in an embodiment. The power data in a plurality of records may be included in the table body, a plurality of fields may be included in each record, and the field type may include a variety of definitions, such as a string of characters that may be of a specified length, numbers (supporting 1,2,4,8 lengths), floating point (supporting single and double precision), date type and binary type (supporting bitmap information and arbitrary structure), and the like. The data table may be stored in a file system in the second memory space in a compact structure, and may record a header start address, a body start address, and a length of each record. In addition, the file of the data physical carrier of the data table can be conveniently moved, the safe partition requirement of the power system can be conveniently realized, and the safe cross-partition of the file data can be realized; and in the power data file of the data table, the terminal 102 can ensure the reliability and integrity of the file through a plurality of crc check codes.
In one embodiment, as shown in fig. 6, fig. 6 is a schematic structural diagram of an association relationship between a data identifier and a data table in one embodiment. The data identifier may be an index, the system may include a plurality of indexes, and the index may be stored in the private memory in a loose map form, that is, in the first memory space. Wherein each index may include a plurality of hash values, each hash value may point to different record data, each record data may include a plurality of fields, and the field data storage may be stored in a structure of null identifier indicator (2 bytes) + length indicator (2 bytes) + data (specified by length); and the field type, the field data storage and the storage of the whole record can realize efficient batch warehousing and batch loading through an Open Cloud Computing Interface (OCCI).
In step S206, the power data in the target data table is processed according to the processing method information.
The processing mode information may be information in the data processing instruction, and the information may be used to instruct the terminal 102 to perform corresponding processing on the power data. The terminal 102 may process the power data in the target data table according to the processing method information. For example, data may be added, deleted, modified, etc. Wherein each process may employ a different approach.
In the data processing method, after a data processing instruction triggered by a user in a system is acquired, a data identifier of power data to be processed is acquired from a private first memory space corresponding to the user, a target data table corresponding to the power data to be processed is determined from a file system of a second memory space according to the data identifier, wherein the second memory space is a public memory space used for storing the power data, and finally the power data in the target data table is processed according to processing mode information in the data processing instruction. Compared with the traditional method for processing the electric power data by utilizing the open source memory bank and the like, the method and the device have the advantages that the electric power data are quickly, simply and conveniently processed through the data table and the memory space separating mode, and the electric power data processing efficiency is improved.
In one embodiment, in response to a data processing instruction triggered by a user, before acquiring a data identifier of power data to be processed from a first memory space corresponding to the user, the method further includes: detecting whether a data identifier corresponding to the electric power data to be processed exists in the identifier table; if not, a data identifier corresponding to the to-be-processed power data is created in the identifier table, and an association relation between the data identifier and the to-be-processed power data is established.
In this embodiment, the first memory space may be a private space in the memory of the terminal 102, and the first memory space may include an identifier table, where the data identifier may be stored in the identifier table, and the identifier table includes at least one data identifier; specifically, the indication table may be an index table, and the index table may include a plurality of indexes. The terminal 102 may create a data identifier as needed, and before responding to a data processing instruction triggered by a user, the terminal 102 may detect whether a data identifier corresponding to the to-be-processed power data exists in the indication table, and if so, may obtain a data table in which the to-be-processed power data is located according to the data identifier; if not, the terminal 102 may create a data identifier corresponding to the to-be-processed power data in the identifier table, and create an association relationship between the newly created data identifier and the to-be-processed power data.
Specifically, the data identifier may be an index, the terminal 102 supports multiple indexes, each index may be a separate field or may be multiple joint fields, when querying the record, the terminal 102 may match the most appropriate index according to a field condition of the query, for example, the coincidence degree of the field queried and the field indexed is the highest, if the index data does not exist before, the terminal 102 may create the index data, that is, the index data is created when it needs to be used.
With the present embodiment, the terminal 102 can generate the index as needed when the index is needed, thereby reducing redundancy in the identification table and improving efficiency of processing the power data.
In one embodiment, further comprising: and responding to a data processing instruction triggered by a user in the system, adding a process lock to the current process corresponding to the user, and switching the state of the target data table to be the in-use state so as to prevent processes corresponding to other users from processing the power data in the target data table.
In this embodiment, the system may include a plurality of processes, and each process may correspond to a user one by one; that is, each process may have a corresponding first memory space, the power data is stored in a mapping file, the index is not stored in a file, the user may be set up in a private memory, and each user may set up its own index in the process. Therefore, in the case of multiple processes, it is desirable to avoid multiple processes from processing power data simultaneously. The terminal 102 may respond to a data processing instruction triggered by a user in the system, add a process lock to a current process corresponding to the user, and switch a state of a target data table corresponding to the data processing instruction to an in-use state, thereby preventing processes corresponding to other users from processing power data in the target data table. The target data table may be a data table where the power data to be processed is located.
In one embodiment, after processing the power data in the target data table according to the processing mode information, the method further includes: and releasing the process lock, and switching the state of the target data table to an unused state.
In this embodiment, the terminal 102 may release the process lock after the data processing is completed in the current process, so as to switch the state of the target data table to an unused state, and the data table in the unused state may be processed by other processes.
Specifically, as shown in fig. 7, fig. 7 is a flowchart illustrating steps of a multi-process in one embodiment. The process 1 may create a memory base table, that is, the above-mentioned data table, in a file mapping manner, and then may create index data in the private memory of the process 1 as needed, the process 2 may open a file corresponding to the table name in a file mapping manner, so as to implement sharing of power data, and the process 2 may also create index data in the private memory thereof as needed, and the index data of each process may be different and do not affect each other. When any process processes the power data, the terminal 102 may add a process lock in a file lock manner, modify the state value of the data table at the same time, avoid multiple processes processing the power data at the same time, and may release the process lock and modify the state value after the processing is completed.
Through the embodiment, the terminal 102 can realize the mutual exclusion processing of the modification of the data table by using the process lock and the modification state value under the condition of a plurality of processes, so that the plurality of processes are prevented from processing the power data at the same time, and the safety of the power data is improved.
In one embodiment, processing the power data in the target data table according to the processing mode information includes: if the processing mode information is data increase, power data to be increased are added at the tail end of the target data table; the power data to be increased is determined by the user; if the processing mode information is the deleted data, replacing the power data in the target data table with the null data; if the processing mode information is modification data, determining the content needing to be modified according to the data identification and the power data to be modified; the power data to be modified is determined by the user.
In this embodiment, the processing mode information may be information in the data processing instruction, and the processing mode information may include: at least one of adding data, deleting data, and modifying data. The terminal 102 may perform different processing on the power data to be processed according to different processing mode information. If the processing mode information is adding data, the terminal 102 may add power data to be added at the end of the target data table, where the power data to be added may be determined by a user. Specifically, the terminal 102 may obtain a new memory location from the current location pointer, where the new memory location is at the original end, and directly copy the power data to be newly added in the recorded memory, that is, the target data table, so as to complete the addition of the power data. If the processing mode is to delete information, the terminal 102 may replace the power data in the target data table with null data, so as to delete the power data. Specifically, the deletion of the power data may be realized by replacing the target data table with data at different positions.
If the processing mode information is modification data, the terminal 102 may determine the content to be modified according to the data identifier and the power data to be modified, where the power data to be modified may be determined by a user. Specifically, in one embodiment, determining the content to be modified according to the data identifier and the power data to be modified includes: judging whether the power data to be modified is power data corresponding to the data identifier; if not, determining the content needing to be modified as the power data to be modified; and if so, determining the content needing to be modified as the power data to be modified and the data identification corresponding to the power data to be modified. The terminal 102 may determine whether only the power data or both the power data and its corresponding data identifier need to be modified according to whether the data identifier is related to the power data to be modified. The terminal 102 may locate a corresponding position in the corresponding data table that needs to be modified according to the content that needs to be modified, and then modify a corresponding field according to the power data to be modified, if the modified field is not an index, that is, the data identifier, the corresponding field does not need to be processed, and if there is some index related to the modified field, the current index record is reconstructed.
Through the embodiment, the terminal 102 can perform corresponding processing on the power data to be processed by using the data identifier and the data table according to different processing mode information, so that the efficiency of processing the power data is improved.
In one embodiment, if the processing mode information is the addition data, before adding the power data to be added to the end of the target data table, the method further includes: judging whether the size of the power data to be increased is larger than the remaining space of the second memory space; if so, increasing the size of the second memory space to store the power data to be increased.
In this embodiment, the terminal 102 may determine the remaining size of the second memory space before increasing the power data, so as to determine whether to increase the memory. The terminal 102 may determine whether the size of the power data to be added is larger than the remaining space of the second memory space, and if not, the terminal 102 may directly add the power data to be added to the data table; if not, the terminal 102 may increase the size of the second memory space, so as to store the power data to be increased. Specifically, the terminal 102 may increase the size of the second memory space according to a dichotomy, where the size of the memory allocated in the center may be 2 times of the current size when the memory needs to be expanded again due to insufficient storage.
Through the embodiment, when the memory space of the terminal 102 is insufficient, the size of the second memory space can be increased by using the corresponding method, so that the terminal 102 can add the power data to be added to the data table, and the processing efficiency of the power data is improved.
In one embodiment, if the processing mode information is delete data, replacing the power data in the target data table with null data includes: replacing null data of the end position of the target data table with the power data to store the power data to the end position; and associating the data identifier corresponding to the power data with the tail end position, and deleting the data identifier corresponding to the power data.
In this embodiment, the terminal 102 may delete the data in the target data table in a data replacement manner. The terminal 102 may replace the empty data at the end position of the target data table with the power data to store the power data at the end position in the target data table, associate the data identifier corresponding to the power data with the end position, and delete the data identifier corresponding to the power data to delete the power data. Specifically, the terminal 102 may locate the corresponding power data to be deleted according to the data identifier, then replace the last record with the record to be deleted by using a replacement method, during which the movement of the record memory is not involved, and then modify the corresponding index, i.e., the value of the data identifier, so as to change the index value corresponding to the power data to be deleted into the last record.
By the embodiment, the terminal 102 can delete the power data by using the replacement method, so that the efficiency of deleting the power data is improved, and the processing efficiency of the power data is improved.
In an embodiment, the multiple data tables may be further merged, and specifically, a memory table, that is, a simple calculation engine may be integrated in the data tables, and the terminal 102 may implement merging of simple operations, such as addition, subtraction, multiplication, division, and the like, between the multiple data tables through an api interface. The merged record takes the main key as the basis of the identification of the same record, the target table establishes an index according to the main key, the source table does not need the index, the source records are scanned in sequence, the index is used for positioning each source record to the record of the target table, then the operation fields of the target record and the source record are calculated and merged according to the expression, and finally the merging of 2 tables is completed.
By the embodiment, the terminal 102 can merge a plurality of data tables, and the power data is processed in the merged data table, so that the processing efficiency of the power data can be improved.
It should be understood that although the steps in the flowcharts of fig. 2 and 7 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2 and 7 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps.
In one embodiment, as shown in fig. 8, there is provided a data processing apparatus including: an obtaining module 500, a determining module 502 and a processing module 504, wherein:
an obtaining module 500, configured to obtain, in response to a data processing instruction triggered in the system by a user, a data identifier of to-be-processed power data from a first memory space; the data processing instruction comprises processing mode information aiming at the power data to be processed; the first memory space is a private memory space corresponding to the user.
A determining module 502, configured to determine, according to the data identifier, a target data table corresponding to the to-be-processed power data from the file system; the file system is arranged in the second memory space and comprises a plurality of data tables; the second memory space is a common memory space for storing power data.
And the processing module 504 is configured to process the power data in the target data table according to the processing mode information.
In one embodiment, the above apparatus further comprises: the detection module is used for detecting whether a data identifier corresponding to the to-be-processed power data exists in the identifier table; if not, a data identifier corresponding to the to-be-processed power data is created in the identifier table, and an association relation between the data identifier and the to-be-processed power data is established.
In one embodiment, the above apparatus further comprises: and the adding module is used for responding to a data processing instruction triggered by a user in the system, adding a process lock to the current process corresponding to the user, and switching the state of the target data table into an in-use state so as to prevent processes corresponding to other users from processing the power data in the target data table.
In one embodiment, the above apparatus further comprises: and the releasing module is used for releasing the process lock and switching the state of the target data table into an unused state.
In an embodiment, the processing module 504 is specifically configured to add the power data to be added to the end of the target data table if the processing mode information is the addition data; the power data to be increased is determined by the user; if the processing mode information is the deleted data, replacing the power data in the target data table with the null data; if the processing mode information is modification data, determining the content needing to be modified according to the data identification and the power data to be modified; the power data to be modified is determined by the user.
In one embodiment, the above apparatus further comprises: the judging module is used for judging whether the size of the power data to be increased is larger than the remaining space of the second memory space; if so, increasing the size of the second memory space to store the power data to be increased.
In an embodiment, the processing module 504 is specifically configured to replace the null data and the power data at the end position of the target data table, so as to store the power data to the end position; and associating the data identifier corresponding to the power data with the tail end position, and deleting the data identifier corresponding to the power data.
In an embodiment, the processing module 504 is specifically configured to determine whether the power data to be modified is power data corresponding to the data identifier; if not, determining the content needing to be modified as the power data to be modified; and if so, determining the content needing to be modified as the power data to be modified and the data identification corresponding to the power data to be modified.
For specific limitations of the data processing apparatus, reference may be made to the above limitations of the data processing method, which are not described herein again. The various modules in the data processing apparatus described above may be implemented in whole or in part by software, hardware, and combinations thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 9. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a data processing method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 9 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory in which a computer program is stored and a processor, which when executing the computer program implements a data processing method as described above.
In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, realizes the data processing method as described above.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method of data processing, the method comprising:
responding to a data processing instruction triggered by a user in a system, and acquiring a data identifier of power data to be processed from a first memory space; the data processing instruction comprises processing mode information aiming at the power data to be processed; the first memory space is a private memory space corresponding to the user;
determining a target data table corresponding to the to-be-processed power data from a file system according to the data identification; the file system is arranged in a second memory space and comprises a plurality of data tables; the second memory space is a public memory space for storing the power data;
and processing the power data in the target data table according to the processing mode information.
2. The method according to claim 1, wherein the first memory space includes an identification table, the data identifier is stored in the identification table, and the identification table includes at least one of the data identifiers;
before the data processing instruction triggered by the user is used for acquiring the data identifier of the power data to be processed from the first memory space corresponding to the user, the method further includes:
detecting whether a data identifier corresponding to the to-be-processed power data exists in the identifier table;
if not, a data identifier corresponding to the to-be-processed power data is established in the identifier table, and an association relation between the data identifier and the to-be-processed power data is established.
3. The method of claim 1, wherein the system includes a plurality of processes, the processes corresponding one-to-one to the users;
the method further comprises the following steps:
responding to a data processing instruction triggered by a user in a system, adding a process lock to a current process corresponding to the user, and switching the state of the target data table to be an in-use state so as to prevent processes corresponding to other users from processing the power data in the target data table;
after the processing the power data in the target data table according to the processing mode information, the method further includes:
and releasing the process lock, and switching the state of the target data table to an unused state.
4. The method of claim 1, wherein the processing mode information comprises: at least one of adding data, deleting data, and modifying data;
the processing the power data in the target data table according to the processing mode information includes:
if the processing mode information is data to be added, adding power data to be added to the tail end of the target data table; the power data to be increased is determined by the user;
if the processing mode information is deletion data, replacing the power data in the target data table with null data;
if the processing mode information is modification data, determining the content needing to be modified according to the data identification and the power data to be modified; the power data to be modified is determined by the user.
5. The method according to claim 4, wherein if the processing mode information is addition data, before adding the power data to be added at the end of the target data table, the method further comprises:
judging whether the size of the power data to be increased is larger than the remaining space of the second memory space;
if so, increasing the size of the second memory space to store the power data to be increased.
6. The method according to claim 4, wherein if the processing mode information is delete data, replacing the power data in the target data table with null data comprises:
replacing null data of an end location of the target data table with the power data to store the power data to the end location;
and associating the data identifier corresponding to the power data with the terminal position, and deleting the data identifier corresponding to the power data.
7. The method of claim 4, wherein determining the content to be modified according to the data identifier and the power data to be modified comprises:
judging whether the power data to be modified is the power data corresponding to the data identifier;
if not, determining the content needing to be modified as the power data to be modified;
if yes, determining the content needing to be modified as the power data to be modified and the data identification corresponding to the power data to be modified.
8. A data processing apparatus, characterized in that the apparatus comprises:
the acquisition module is used for responding to a data processing instruction triggered by a user in the system and acquiring a data identifier of the power data to be processed from the first memory space; the data processing instruction comprises processing mode information aiming at the power data to be processed; the first memory space is a private memory space corresponding to the user;
the determining module is used for determining a target data table corresponding to the to-be-processed power data from a file system according to the data identification; the file system is arranged in a second memory space and comprises a plurality of data tables; the second memory space is a public memory space for storing the power data;
and the processing module is used for processing the electric power data in the target data table according to the processing mode information.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010730402.2A CN111813783B (en) | 2020-07-27 | 2020-07-27 | Data processing method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010730402.2A CN111813783B (en) | 2020-07-27 | 2020-07-27 | Data processing method, device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111813783A true CN111813783A (en) | 2020-10-23 |
| CN111813783B CN111813783B (en) | 2024-03-26 |
Family
ID=72862636
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010730402.2A Active CN111813783B (en) | 2020-07-27 | 2020-07-27 | Data processing method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111813783B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112800047A (en) * | 2021-03-03 | 2021-05-14 | 百果园技术(新加坡)有限公司 | User associated data processing method, device, equipment and storage medium |
| WO2023185401A1 (en) * | 2022-03-28 | 2023-10-05 | 华为技术有限公司 | Data processing method, encoding and decoding accelerator and related device |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050096787A1 (en) * | 2003-11-05 | 2005-05-05 | Kim Jun H. | Tablet cassette control method of medication dispensing and packaging system |
| CN101089857A (en) * | 2007-07-24 | 2007-12-19 | 中兴通讯股份有限公司 | Internal store data base transaction method and system |
| CN102685221A (en) * | 2012-04-29 | 2012-09-19 | 华北电力大学（保定） | Distributed storage and parallel mining method for state monitoring data |
| CN105868023A (en) * | 2016-03-24 | 2016-08-17 | 联想(北京)有限公司 | Data processing method and computing node in distributed system |
| US20170117719A1 (en) * | 2015-10-23 | 2017-04-27 | Chicony Power Technology Co., Ltd. | Power supply system with identification code updating capability and method for charging an electronic device |
| CN106899772A (en) * | 2017-02-21 | 2017-06-27 | 深圳市彬讯科技有限公司 | The distributed equally loaded control method and calling system of a kind of calling service |
| CN107004195A (en) * | 2014-09-29 | 2017-08-01 | 加拿大皇家银行 | The safe handling of data |
| CN107665233A (en) * | 2017-07-24 | 2018-02-06 | 上海壹账通金融科技有限公司 | Database data processing method, device, computer equipment and storage medium |
| CN107943846A (en) * | 2017-11-01 | 2018-04-20 | 内蒙古科电数据服务有限公司 | Data processing method, device and electronic equipment |
| CN108829727A (en) * | 2018-05-09 | 2018-11-16 | 上海陆家嘴国际金融资产交易市场股份有限公司 | Date storage method, device, computer equipment and storage medium |
| CN109886025A (en) * | 2019-01-04 | 2019-06-14 | 平安科技（深圳）有限公司 | Management method, device, computer equipment and the storage medium of user identifier |
| CN110046202A (en) * | 2019-03-07 | 2019-07-23 | 中国人民解放军海军工程大学 | The integrated power system real time data releasing method of key value database based on memory |
| CN110276002A (en) * | 2019-06-26 | 2019-09-24 | 浙江大搜车软件技术有限公司 | Search for application data processing method, device, computer equipment and storage medium |
| CN110795395A (en) * | 2018-07-31 | 2020-02-14 | 阿里巴巴集团控股有限公司 | File deployment system and file deployment method |
-
2020
- 2020-07-27 CN CN202010730402.2A patent/CN111813783B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050096787A1 (en) * | 2003-11-05 | 2005-05-05 | Kim Jun H. | Tablet cassette control method of medication dispensing and packaging system |
| CN101089857A (en) * | 2007-07-24 | 2007-12-19 | 中兴通讯股份有限公司 | Internal store data base transaction method and system |
| CN102685221A (en) * | 2012-04-29 | 2012-09-19 | 华北电力大学（保定） | Distributed storage and parallel mining method for state monitoring data |
| CN107004195A (en) * | 2014-09-29 | 2017-08-01 | 加拿大皇家银行 | The safe handling of data |
| US20170117719A1 (en) * | 2015-10-23 | 2017-04-27 | Chicony Power Technology Co., Ltd. | Power supply system with identification code updating capability and method for charging an electronic device |
| CN105868023A (en) * | 2016-03-24 | 2016-08-17 | 联想(北京)有限公司 | Data processing method and computing node in distributed system |
| CN106899772A (en) * | 2017-02-21 | 2017-06-27 | 深圳市彬讯科技有限公司 | The distributed equally loaded control method and calling system of a kind of calling service |
| CN107665233A (en) * | 2017-07-24 | 2018-02-06 | 上海壹账通金融科技有限公司 | Database data processing method, device, computer equipment and storage medium |
| CN107943846A (en) * | 2017-11-01 | 2018-04-20 | 内蒙古科电数据服务有限公司 | Data processing method, device and electronic equipment |
| CN108829727A (en) * | 2018-05-09 | 2018-11-16 | 上海陆家嘴国际金融资产交易市场股份有限公司 | Date storage method, device, computer equipment and storage medium |
| CN110795395A (en) * | 2018-07-31 | 2020-02-14 | 阿里巴巴集团控股有限公司 | File deployment system and file deployment method |
| CN109886025A (en) * | 2019-01-04 | 2019-06-14 | 平安科技（深圳）有限公司 | Management method, device, computer equipment and the storage medium of user identifier |
| CN110046202A (en) * | 2019-03-07 | 2019-07-23 | 中国人民解放军海军工程大学 | The integrated power system real time data releasing method of key value database based on memory |
| CN110276002A (en) * | 2019-06-26 | 2019-09-24 | 浙江大搜车软件技术有限公司 | Search for application data processing method, device, computer equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| MICHAEL BADAMO等: "Identifying Power-Efficient Multicore Cache Hierarchies via Reuse Distance Analysis", 《HTTPS://DL.ACM.ORG/DOI/ABS/10.1145/2851503》, pages 1 - 30 * |
| 陈升: "统一时空电网数据模型与访问技术", 《计算机系统应用 》, pages 210 - 214 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112800047A (en) * | 2021-03-03 | 2021-05-14 | 百果园技术(新加坡)有限公司 | User associated data processing method, device, equipment and storage medium |
| CN112800047B (en) * | 2021-03-03 | 2024-04-05 | 百果园技术(新加坡)有限公司 | User associated data processing method, device, equipment and storage medium |
| WO2023185401A1 (en) * | 2022-03-28 | 2023-10-05 | 华为技术有限公司 | Data processing method, encoding and decoding accelerator and related device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111813783B (en) | 2024-03-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10101937B2 (en) | Apparatus and method for referencing dense and sparse information in multi-dimensional to linear address space translation | |
| CN110018998B (en) | File management method and system, electronic equipment and storage medium | |
| CN106610790B (en) | Method and device for deleting repeated data | |
| CN106708825B (en) | A kind of data file processing method and system | |
| CN105468642A (en) | Data storage method and apparatus | |
| CN111813783B (en) | Data processing method, device, computer equipment and storage medium | |
| CN111177143B (en) | Key value data storage method and device, storage medium and electronic equipment | |
| CN111381779A (en) | Data processing method, device, equipment and storage medium | |
| CN108664482B (en) | FLASH memory and storage file management method | |
| CN113282799B (en) | Node operation method, node operation device, computer equipment and storage medium | |
| CN114625696A (en) | File recovery method and device, electronic equipment and storage medium | |
| CN112783866A (en) | Data reading method and device, computer equipment and storage medium | |
| CN114924911B (en) | Method, device, equipment and storage medium for backing up effective data of Windows operating system | |
| CN113672162A (en) | Data storage method, device and equipment | |
| CN115113798B (en) | Data migration method, system and equipment applied to distributed storage | |
| CN115421648A (en) | Memory garbage collection method, device, equipment, storage medium and program product | |
| CN112416966A (en) | Ad hoc query method, apparatus, computer device and storage medium | |
| CN112835886A (en) | Data table field adding method and device | |
| CN113411395B (en) | Access request routing method, device, computer equipment and storage medium | |
| CN110347333A (en) | Improve method, apparatus, computer equipment and the storage medium of clone's mirror image performance | |
| CN117290302B (en) | Directory separation method, apparatus, computer device and storage medium | |
| CN117540056B (en) | Method, device, computer equipment and storage medium for data query | |
| CN116821102B (en) | Data migration method, device, computer equipment and storage medium | |
| US20240134837A1 (en) | Index entry generating methods, apparatuses, and devices for large object data block index | |
| CN111046241B (en) | Graph storage method and device for flow graph processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 86, room 406, No.1, Yichuang street, Zhongxin Guangzhou Knowledge City, Huangpu District, Guangzhou City, Guangdong Province Applicant after: Southern Power Grid Digital Grid Research Institute Co.,Ltd. Address before: 511458 Room 1301, Chengtou Building, 106 Fengze East Road, Nansha District, Guangzhou City, Guangdong Province (self-compiled 1301-12159) Applicant before: Southern Power Grid Digital Grid Research Institute Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |