CN115098454A - Data processing method and system - Google Patents
Data processing method and system Download PDFInfo
- Publication number
- CN115098454A CN115098454A CN202210884028.0A CN202210884028A CN115098454A CN 115098454 A CN115098454 A CN 115098454A CN 202210884028 A CN202210884028 A CN 202210884028A CN 115098454 A CN115098454 A CN 115098454A
- Authority
- CN
- China
- Prior art keywords
- file
- data
- state
- read
- written
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 24
- 238000000034 method Methods 0.000 claims abstract description 70
- 238000012545 processing Methods 0.000 claims abstract description 53
- 238000011084 recovery Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 14
- 238000004064 recycling Methods 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 description 24
- 238000010586 diagram Methods 0.000 description 11
- 230000003287 optical effect Effects 0.000 description 9
- 230000000694 effects Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000001960 triggered effect Effects 0.000 description 3
- 238000009825 accumulation Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000014759 maintenance of location Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a data processing method and a system, the method is applied to a current file sequence comprising a file to be written, a file to be read and a file to be processed, and the method comprises the following steps: judging whether the written data is detected; when the write-in data are detected, determining a file to be written corresponding to the write state, and writing the write-in data into the file to be written; when the write-in data is not detected, judging whether a reading instruction is detected or not; and when the reading instruction is detected, determining the file to be read corresponding to the reading state, and reading the cache data in the file to be read. Therefore, the method can improve the data processing efficiency, ensure that the data processing efficiency is not reduced when the buffer storage amount is large, and simultaneously, the method can also recycle the space utilization rate when the data information is recovered.
Description
Technical Field
The present application relates to the field of data processing, and in particular, to a data processing method and system.
Background
The unidirectional optical gate (also called unidirectional network gate) is usually deployed between networks with different security levels, and realizes unidirectional transmission of data from a low security domain to a high security domain through a unidirectional feedback-free transmission physical environment. At present, the file storage method capable of realizing file caching in the unidirectional optical gate still has the following defects:
firstly, the number of data block cache records and data blocks is very large, so that the efficiency is reduced when the data volume is large;
secondly, the data block cache records and the data blocks are stored separately, so that the position of reading 1 data block needs to be moved for at least 2 times, and particularly when a mechanical hard disk is used, the efficiency is lower when the data cache amount is larger;
and thirdly, after the data block is read, the space of the data block needs to be recovered, which requires the internal terminal to record each piece of recovered data block information in the program, and especially when the released space is smaller than the size of the data block to be written, the utilization rate of the released space is reduced.
Disclosure of Invention
An object of the embodiments of the present application is to provide a data processing method and system, which can improve data processing efficiency, ensure that the data processing efficiency is not reduced when the buffer size is large, and simultaneously, can also use the space utilization rate when recovering data information.
A first aspect of the embodiments of the present application provides a data processing method, which is applied to a current file sequence including a file to be written, a file to be read, and a file to be processed, and the method includes:
judging whether the written data is detected;
when the write-in data are detected, determining a file to be written corresponding to a write state, and writing the write-in data into the file to be written;
when the written data is not detected, judging whether a reading instruction is detected;
and when the reading instruction is detected, determining a file to be read corresponding to the reading state, and reading the cache data in the file to be read.
In the implementation process, the method is applied to the current file sequence comprising the file to be written, the file to be read and the file to be processed, and specifically, the method can preferentially judge whether the written data is detected; therefore, the method judges whether the circulation cycle of the three files of writing, reading and writing is started or the circulation cycle of the three files of reading, writing and reading is started, and further ensures that the data processing method can start to operate in an expected form. Then, when the write-in data is detected, determining a file to be written corresponding to the write state, and writing the write-in data into the file to be written; therefore, the process can realize the writing of data so that the written data waits to be read, and simultaneously can transmit the writing state to the next circulating file, thereby realizing the process circulation effect while realizing the data processing process, avoiding the accumulation of workload and improving the overall processing efficiency. Then, when the method does not detect the written data, judging whether a reading instruction is detected; therefore, the method can judge whether to start the circulation of the read, the empty and the write files when the circulation of the write, the read and the empty files is not started, thereby ensuring the correct and effective operation of the data processing method and avoiding the method failure caused by a non-write event. Finally, when a reading instruction is detected, determining a file to be read corresponding to the reading state, and reading cache data in the file to be read; the method can realize circulation of reading, emptying and writing while reading data, thereby realizing another data processing starting mode, and further improving the stability and the applicability of the method while improving the data processing efficiency.
Further, when the write data is written into the file to be written, or the cache data in the file to be read is read, the method further includes:
determining a file to be processed corresponding to the empty state;
and recovering the historical data space in the file to be processed.
In the implementation process, when writing the write-in data into a file to be written, or reading the cache data in the file to be read, determining a file to be processed corresponding to an empty state; and then, recovering the historical data space in the file to be processed. Therefore, by implementing the implementation mode, the space recovery can be carried out on the file to be processed while reading or writing, so that the simultaneous operation of reading, writing and recovering can be realized, and the subsequent circulation of reading, writing and empty states is facilitated.
Further, the method further comprises:
and when the written data is written in or the cache data is read, converting the read state into the empty state, converting the empty state into the write state, and converting the write state into the read state.
In the implementation process, the method can convert the read state into the empty state, convert the empty state into the write state and convert the write state into the read state after the write data is written or the cache data is read. Therefore, the method can circulate the states in the sequence of reading, emptying and writing, and then the files corresponding to the three state marks of reading, emptying and writing are taken as the file to be read, the file to be processed and the file to be written, so that the file types are changed, different data processing is performed on the three files in the next circulation, the content in the three files is ensured to be changed and alternated all the time, and the overall data processing efficiency is ensured.
Further, before the step of determining whether the written data is detected, the method further includes:
creating a current file sequence comprising a first file, a second file and a third file;
setting three states of a writing state, a reading state and an empty state on the first file;
when the written data is detected, determining a second file as a file to be processed corresponding to the empty state, and writing the written data into the first file;
when a reading instruction is detected, determining a third file as a file to be processed corresponding to the empty state, determining a second file as a file to be written corresponding to the writing state, and reading the written data in the first file; the first file is a file to be read, and the written data is cache data.
In the implementation process, the method needs to suggest a current file sequence in advance before judging whether the step of writing data is detected, and initially, three empty files are created and set as three files with sequential circulation, and a writing state, a reading state and an empty state are given to the first file; therefore, the process can create a circular flow by taking the 'three' as a base number, and the writing of the flow method foundation is realized through state overwriting, so that a complete data attribute foundation is provided for the subsequent progressive initialization flow. Then, when the written data is detected, determining the second file as a file to be processed corresponding to the empty state, and writing the written data into the first file; therefore, the method can detect the written data preferentially, and sequentially move the state tags when the written data appears, so that the effects of reading and writing a file and an empty file are achieved, the writing of the data and the space recovery and retention of a second file are ensured, and the smooth operation of the data processing method is further ensured. Then, when a reading instruction is detected, the method determines the third file as a file to be processed corresponding to the empty state, determines the second file as a file to be written corresponding to the writing state, and reads the written data in the first file; the first file is a file to be read, and the written data is cache data. Therefore, the method can promote the writing state and the empty state in the reading state, so that the three states are distributed on the three files, the initialization of data attributes and method flows can be realized in the data processing process, the parallelism of data processing and flow setting is guaranteed, and the overall data processing efficiency and stability are improved.
Further, the step of setting the writing state, the reading state and the empty state on the first file includes:
setting three states of a writing state, a reading state and an empty state on the first file, and recycling a historical data space in the first file.
In the implementation process, the method can set three states of a writing state, a reading state and a null state on the first file in the initial state, and recover the historical data space in the first file, so as to implement the initial initialization of the flow layer of the method, thereby avoiding the problem of incomplete initialization caused by data residue in the first file.
A second aspect of the embodiments of the present application provides a data processing system, where a current file sequence including a file to be written, a file to be read, and a file to be processed is set in the data processing system, and the data processing system includes:
a judging unit configured to judge whether write data is detected;
the writing unit is used for determining a file to be written corresponding to a writing state when the writing data is detected, and writing the writing data into the file to be written;
the judging unit is further configured to judge whether a read instruction is detected when the write data is not detected;
and the reading unit is used for determining the file to be read corresponding to the reading state and reading the cache data in the file to be read when the reading instruction is detected.
In the implementation process, a current file sequence including a file to be written, a file to be read and a file to be processed is set in the data processing system, and specifically, the data processing system can judge whether written data is detected through the judging unit; therefore, the circulation cycle of the three files of writing, reading and writing is judged to be started, and the circulation cycle of the three files of reading, writing and reading is also judged to be started, so that the data processing system can be ensured to start to operate in an expected form. Determining a file to be written corresponding to a writing state when the writing data is detected through the writing unit, and writing the writing data into the file to be written; therefore, the process can realize the writing of data so that the written data waits to be read, and simultaneously can transmit the writing state to the next circulating file, thereby realizing the process circulation effect while realizing the data processing process, avoiding the accumulation of workload and improving the overall processing efficiency. Judging whether a reading instruction is detected or not when the judging unit does not detect the written data; therefore, the system can judge whether to start the circulation of the read, the empty and the write files when the circulation of the write, the read and the empty files is not started, thereby ensuring the correct and effective operation of the data processing system and avoiding the system failure caused by non-write events. And when the reading unit detects the reading instruction, determining the file to be read corresponding to the reading state, and reading the cache data in the file to be read. The system can realize circulation of reading, emptying and writing while reading data, thereby realizing another data processing starting mode, and further improving the stability and the applicability of the system while improving the data processing efficiency.
Further, the data processing system further comprises:
a determining unit, configured to determine a to-be-processed file corresponding to an empty state when the write data is written into the to-be-written file or the cache data in the to-be-read file is read;
and the recovery unit is used for recovering the historical data space in the file to be processed.
In the implementation process, the data processing system may determine, by the determining unit, a to-be-processed file corresponding to the empty state when writing the write data into the to-be-written file or reading the cache data in the to-be-read file; and then, the historical data space in the file to be processed is recovered through a recovery unit. Therefore, by implementing the implementation mode, the system can perform space recovery on the file to be processed while reading or writing, so that simultaneous operation of reading, writing and recovery is realized, and subsequent circulation of reading, writing and space states is facilitated.
Further, the data processing system further comprises:
and the conversion unit is used for converting the read state into the empty state, converting the empty state into the write state and converting the write state into the read state after the write data is written or the cache data is read.
In the implementation process, the data processing system may convert the read state into the empty state, convert the empty state into the write state, and convert the write state into the read state after the write data is written or the cache data is read by the conversion unit. Therefore, the system can circulate the states in the sequence of reading, emptying and writing, and then the files corresponding to the three state marks of reading, emptying and writing are taken as the file to be read, the file to be processed and the file to be written, so that the file types are changed, different data processing is performed on the three files in the next circulation, the content in the three files is ensured to be changed and alternated all the time, and the overall data processing efficiency is ensured.
Further, the data processing system further comprises:
a creating unit configured to create a current file sequence including the first file, the second file, and the third file before the step of determining whether the write data is detected;
the setting unit is used for setting three states of a writing state, a reading state and an empty state on the first file;
the determining unit is further configured to determine a second file as a file to be processed corresponding to the empty state when write data is detected, and write the write data into the first file;
the determining unit is further configured to determine, when a read instruction is detected, a third file as a file to be processed corresponding to the empty state, determine a second file as a file to be written corresponding to the write state, and read the write data in the first file; the first file is a file to be read, and the written data is cache data.
In the implementation process, the data processing system may create, by the creating unit, a current file sequence including the first file, the second file, and the third file before the step of determining whether to detect the written data, and set, by the setting unit, three states, that is, a write state, a read state, and an empty state, on the first file; therefore, the process can create a circular flow by taking three as a base number, and the writing of the flow system foundation is realized through state overwriting, so that a complete data attribute foundation is provided for the subsequent progressive initialization flow. When the determining unit detects the written data, the second file is determined as a file to be processed corresponding to the empty state, and the written data is written into the first file; therefore, the system can detect the written data preferentially, and sequentially move the state tags when the written data appears, so that the effects of reading and writing a file and an empty file are achieved, the writing of the data and the space recovery and retention of the second file are ensured, and the smooth operation of the data processing system is further ensured. When the determining unit detects a reading instruction, the third file is determined as a file to be processed corresponding to the empty state, the second file is determined as a file to be written corresponding to the writing state, and the written data in the first file are read; the first file is a file to be read, and the written data is cache data. Therefore, the system can push the writing state and the empty state in the reading state, so that the three states are distributed on the three files, the initialization of data attributes and system flows can be realized in the data processing process, the parallelism of data processing and flow setting is guaranteed, and the overall data processing efficiency and stability are improved.
Further, the setting unit is specifically configured to set a write state, a read state, and an empty state on the first file, and to recycle a historical data space in the first file.
In the implementation process, the system can set three states of a writing state, a reading state and a null state on the first file in the initial state, and recovers the historical data space in the first file, so as to implement the initial initialization of the flow layer of the method, thereby avoiding the problem of incomplete initialization caused by data residue in the first file.
A third aspect of embodiments of the present application provides an electronic device, including a memory and a processor, where the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the data processing method described in any one of the first aspect of embodiments of the present application.
A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, which stores computer program instructions, and when the computer program instructions are read and executed by a processor, the computer program instructions perform the data processing method according to any one of the first aspect of the embodiments of the present application.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 is a schematic flowchart of a data processing method according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a data processing system according to an embodiment of the present application;
fig. 3 is a schematic diagram illustrating a unidirectional data stream before being stored in a circular queue file according to an embodiment of the present application;
fig. 4 is a schematic diagram illustrating a one-way session data being stored according to an embodiment of the present application;
fig. 5 is a schematic view of a file circulation relationship provided in an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
Example 1
Referring to fig. 1, fig. 1 is a flow chart illustrating a data processing method according to the present embodiment. The data processing method is applied to a current file sequence comprising a file to be written, a file to be read and a file to be processed, and specifically comprises the following steps:
s101, creating a current file sequence comprising the first file, the second file and the third file.
S102, setting three states of a writing state, a reading state and an empty state on the first file, and recycling a historical data space in the first file.
S103, when the written data is detected, determining the second file as a file to be processed corresponding to the empty state, and writing the written data into the first file.
S104, when a reading instruction is detected, determining the third file as a file to be processed corresponding to the empty state, determining the second file as a file to be written corresponding to the writing state, and reading the written data in the first file; the first file is a file to be read, and the written data is cache data.
S105, judging whether the written data is detected, if so, executing a step S106; if not, step S107 is executed.
S106, determining a file to be written corresponding to the writing state, writing the written data into the file to be written, and executing the step S109.
S107, judging whether a reading instruction is detected, if so, executing a step S108; if not, the flow is ended.
S108, determining the file to be read corresponding to the reading state, reading the cache data in the file to be read, and executing the step S109.
And S109, determining the file to be processed corresponding to the empty state.
And S110, recovering the historical data space in the file to be processed.
And S111, when the written data is written or the cache data is read, converting the read state into a null state, converting the null state into the write state, and converting the write state into the read state.
In this embodiment, the method adopts a design idea of combining a circular queue with a file cache, and takes a session ID and 3 sequence numbers as file names of three files, so as to form 1 file circular queue, that is, the files form a ring as follows.
Such as: "ID 1-1" ═ ID1-2 "═ ID 1-3" ═ ID1-1 ".
Specifically, a schematic diagram of the unidirectional data stream before being stored in the circular queue file is shown in fig. 3.
As can be seen, the unidirectional data flow in FIG. 3 would be stored in a file circular queue, e.g., "file circular queue ID 1" represents the file circular queue for Session ID1, and so on for the other queues.
In this embodiment, a schematic diagram after storing unidirectional session data is shown in fig. 4.
As shown in fig. 4, the data for each session is stored in two different files, in turn, wherein,
[ I "ID 1-1": "ID 1" represents the first file queue (corresponding to the 1 st unidirectional session), "-1" represents the first file in the file queue, with the file name: "ID 1-1";
② ID1-2 ": "ID 1" indicates the first file queue (corresponding to the 1 st unidirectional session), "-2" indicates the 2 nd file in the file queue, with the file name: "ID 1-2";
③ ID 1-3': "ID 1" indicates the first file queue (corresponding to the 1 st unidirectional session), "-3" indicates the 3 rd file in the file queue, with the file name: "ID 1-3";
1-1, 1-2, 1-3: "1" represents the 1 st unidirectional session, and "-1", "-2", "-3" represent data blocks 1, 2, 3 in the 1 st unidirectional session, respectively;
and analogizing the relationship between the file names and the data blocks of other file queues.
Therefore, through the thought, the file writing and the file reading can be carried out in different files, and the parameters of the file cache quantity and time are adjusted according to the service scene, so that the goal of optimally matching the service is achieved.
In this embodiment, the method can realize fast writing and reading of unidirectional data in a file by using a file circular queue technical idea.
The internal terminal program sets three files of 1 file circular queue into three states of readable file, writable file and empty file, which are abbreviated as: read, write, null;
the three states are circularly rotated on three files of ID1-1, ID1-2 and ID 1-3;
diagram of the circular relationship between the three states and the three files is shown in FIG. 5.
For example, the method (file circular queue read, write, empty state circular case) may include the following steps:
firstly, an initial state: the three states of read, write and empty are all on the ID1-1 file, corresponding to (r) in the figure.
The data writing state: after the data writing state is triggered, the empty state is pushed to move forward, namely, the empty state is ID1-2, the write state is ID1-1, and the read state is ID1-1, which corresponds to the second image.
Reading a data state: after the read data state is triggered, the write state and the empty state are pushed to move forward, namely, the empty state is ID1-3, the write state is ID1-2, and the read state is ID1-1, which corresponds to the third image.
(iv) empty state (recovery history data space): the empty state is triggered only by the write state and the read state, only as 1 isolated state between write to read.
During the continuous writing and reading of data, the three states of reading, writing and emptying are continuously circulated on the file names of ID1-1, ID1-2 and ID 1-3.
In this embodiment, the method circulates three states in three files in the process of special processing of the repetition cycle. At this time, since the history data is also stored in the file, the history data is directly emptied by operating the file of the operating system or is rebuilt after the file is deleted when the empty state is rotated.
In this embodiment, the execution subject of the method may be a computing system such as a computer and a server, and is not limited in this embodiment.
In this embodiment, an execution subject of the method may also be an intelligent device such as a smart phone and a tablet computer, which is not limited in this embodiment.
It can be seen that, by implementing the data processing method described in this embodiment, the file circular queue can be used in combination with the one-way data read-write feature, and the problems of multiple sessions, large data volume, and low efficiency of the recovery storage space can be effectively solved. Meanwhile, the writing, reading and empty (recycling historical data space) states of the three state file circular queues are not affected mutually, the writing, reading and empty (recycling historical data space) of file data are effectively separated, and the comprehensive data processing speed is improved. The continuity of the hard disk addresses stored and read by the file is good, and particularly, historical data recovery is completed once, so that the reliability and stability of data caching are further improved. Therefore, the method is simple and easy to implement, can effectively provide the comprehensive data caching capacity of the one-way optical gate, reduces the probability of loss of terminal data in the one-way optical gate caused by data congestion of a data receiver, improves the reliability of the one-way optical gate for providing services to the outside, and reduces the cost of system operation and maintenance.
Example 2
Referring to fig. 2, fig. 2 is a schematic structural diagram of a data processing system according to the present embodiment. As shown in fig. 2, the data processing system includes:
a judging unit 210 for judging whether write data is detected;
the writing unit 220 is configured to determine a file to be written corresponding to a writing state when the write data is detected, and write the write data into the file to be written;
the determining unit 210 is further configured to determine whether a read command is detected when the write data is not detected;
the reading unit 230 is configured to, when a reading instruction is detected, determine a file to be read corresponding to the reading state, and read cache data in the file to be read.
As an optional implementation, the data processing system further comprises:
the determining unit 240 is configured to determine a to-be-processed file corresponding to an empty state when writing the write data into the to-be-written file or reading cache data in the to-be-read file;
and the recovery unit 250 is used for recovering the historical data space in the file to be processed.
As an optional implementation, the data processing system further comprises:
the converting unit 260 is configured to convert the read state into an empty state, convert the empty state into the write state, and convert the write state into the read state after the write data is written or the cache data is read.
As an optional implementation, the data processing system further comprises:
a creating unit 270, configured to create a current file sequence including the first file, the second file, and the third file before the step of determining whether the written data is detected;
a setting unit 280 for setting three states of a write state, a read state, and an empty state on a first file;
the determining unit 240 is further configured to determine, when the write data is detected, the second file as a file to be processed corresponding to the empty state, and write the write data into the first file;
the determining unit 240 is further configured to, when the read instruction is detected, determine the third file as a to-be-processed file corresponding to the empty state, determine the second file as a to-be-written file corresponding to the write state, and read write data in the first file; the first file is a file to be read, and the written data is cache data.
As an optional implementation manner, the setting unit 280 is specifically configured to set three states, i.e., a write state, a read state, and an empty state, on the first file, and recycle the historical data space in the first file.
In this embodiment, the description in embodiment 1 may be referred to for explanation of the data processing system, and details are not repeated in this embodiment.
It can be seen that, by implementing the data processing system described in this embodiment, the file circular queue can be used in combination with the one-way data read-write feature, and the problems of multiple sessions, large data volume, and low efficiency of the recovery storage space can be effectively solved. Meanwhile, the writing, reading and empty (recovery historical data space) states of the file circulating queues in the three states are not influenced mutually, the writing, reading and empty (recovery historical data space) of file data are effectively separated, and the comprehensive data processing speed is improved. The continuity of the hard disk addresses stored and read by the file is good, and particularly, historical data recovery is completed once, so that the reliability and stability of data caching are further improved. Therefore, the method is simple and easy to implement, can effectively provide the comprehensive data caching capacity of the one-way optical gate, reduces the probability of loss of terminal data in the one-way optical gate caused by data congestion of a data receiver, improves the reliability of the one-way optical gate for providing services to the outside, and reduces the cost of system operation and maintenance.
An embodiment of the present application provides an electronic device, which includes a memory and a processor, where the memory is used to store a computer program, and the processor runs the computer program to make the electronic device execute the data processing method in embodiment 1 of the present application.
An embodiment of the present application provides a computer-readable storage medium, which stores computer program instructions, and when the computer program instructions are read and executed by a processor, the computer program instructions execute the data processing method in embodiment 1 of the present application.
In the several embodiments provided in the present application, it should be understood that the disclosed system and method may be implemented in other ways. The system embodiments described above are merely illustrative, and the flowcharts and block diagrams in the figures, for example, illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
Claims (10)
1. A data processing method is applied to a current file sequence comprising a file to be written, a file to be read and a file to be processed, and the method comprises the following steps:
judging whether the written data is detected;
when the write-in data are detected, determining a file to be written corresponding to a write state, and writing the write-in data into the file to be written;
when the written data is not detected, judging whether a reading instruction is detected;
and when the reading instruction is detected, determining a file to be read corresponding to the reading state, and reading the cache data in the file to be read.
2. The data processing method according to claim 1, wherein when the write data is written into the file to be written or the cache data in the file to be read is read, the method further comprises:
determining a file to be processed corresponding to the empty state;
and recovering the historical data space in the file to be processed.
3. The data processing method of claim 2, wherein the method further comprises:
and when the written data is written in or the cache data is read, converting the read state into the empty state, converting the empty state into the write state, and converting the write state into the read state.
4. The data processing method of claim 1, wherein before the step of determining whether write data is detected, the method further comprises:
creating a current file sequence comprising a first file, a second file and a third file;
setting three states of a writing state, a reading state and an empty state on the first file;
when the written data is detected, determining a second file as a file to be processed corresponding to the empty state, and writing the written data into the first file;
when a reading instruction is detected, determining a third file as a file to be processed corresponding to the empty state, determining a second file as a file to be written corresponding to the writing state, and reading the written data in the first file; the first file is a file to be read, and the written data is cache data.
5. The data processing method of claim 4, wherein the step of setting three states of a write state, a read state and an empty state on the first file comprises:
setting three states of a writing state, a reading state and an empty state on the first file, and recycling a historical data space in the first file.
6. A data processing system is characterized in that a current file sequence comprising a file to be written, a file to be read and a file to be processed is arranged in the data processing system, and the data processing system comprises:
a judging unit configured to judge whether write data is detected;
the writing unit is used for determining a file to be written corresponding to a writing state when the writing data is detected, and writing the writing data into the file to be written;
the judging unit is further configured to judge whether a read instruction is detected when the write data is not detected;
and the reading unit is used for determining the file to be read corresponding to the reading state and reading the cache data in the file to be read when the reading instruction is detected.
7. The data processing system of claim 6, further comprising:
a determining unit, configured to determine a to-be-processed file corresponding to an empty state when the write data is written into the to-be-written file or the cache data in the to-be-read file is read;
and the recovery unit is used for recovering the historical data space in the file to be processed.
8. The data processing system of claim 7, further comprising:
and the conversion unit is used for converting the read state into the empty state, converting the empty state into the write state and converting the write state into the read state after the write data is written or the cache data is read.
9. An electronic device, characterized in that the electronic device comprises a memory for storing a computer program and a processor for executing the computer program to cause the electronic device to perform the data processing method of any one of claims 1 to 5.
10. A readable storage medium, in which computer program instructions are stored, which, when read and executed by a processor, perform the data processing method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210884028.0A CN115098454A (en) | 2022-07-26 | 2022-07-26 | Data processing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210884028.0A CN115098454A (en) | 2022-07-26 | 2022-07-26 | Data processing method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115098454A true CN115098454A (en) | 2022-09-23 |
Family
ID=83298196
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210884028.0A Pending CN115098454A (en) | 2022-07-26 | 2022-07-26 | Data processing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115098454A (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6321304B1 (en) * | 1999-01-22 | 2001-11-20 | Apple Computer, Inc. | System and method for deleting read-only head entries in multi-processor computer systems supporting cache coherence with mixed protocols |
| CN102077165A (en) * | 2008-06-27 | 2011-05-25 | 佳能电子株式会社 | Peripheral device and image reading device |
| CN102467569A (en) * | 2010-11-16 | 2012-05-23 | 英业达股份有限公司 | Method for supporting concurrent read-write of file in data de-duplication |
| CN102521160A (en) * | 2011-12-22 | 2012-06-27 | 上海交通大学 | Write buffer detector, addressing method of written data and parallel channel write method |
| CN105142004A (en) * | 2015-08-28 | 2015-12-09 | 深圳Tcl数字技术有限公司 | Multi-media file reading method and multi-media file reading device |
| CN106354788A (en) * | 2016-08-23 | 2017-01-25 | 无锡天脉聚源传媒科技有限公司 | File reading method and device |
| US20180165214A1 (en) * | 2016-12-13 | 2018-06-14 | Advanced Micro Devices, Inc. | Dynamic cache bypassing |
| CN108491402A (en) * | 2018-01-17 | 2018-09-04 | 艾体威尔电子技术(北京)有限公司 | A kind of implementation method of lightweight file system |
| CN108763099A (en) * | 2018-04-18 | 2018-11-06 | 华为技术有限公司 | Startup method, apparatus, electronic equipment and the storage medium of system |
| CN111159124A (en) * | 2019-12-30 | 2020-05-15 | 浪潮电子信息产业股份有限公司 | Asynchronous write caching method, device and medium for Linux kernel file system |
| CN111880734A (en) * | 2020-07-24 | 2020-11-03 | 北京浪潮数据技术有限公司 | Data processing method, system, electronic equipment and storage medium |
| CN112685363A (en) * | 2020-12-30 | 2021-04-20 | 湖南博匠信息科技有限公司 | User file reading and writing method and device |
| CN113934454A (en) * | 2021-10-19 | 2022-01-14 | 广州小鹏汽车科技有限公司 | Data reading and writing method, micro control unit MCU chip, vehicle and storage medium |
-
2022
- 2022-07-26 CN CN202210884028.0A patent/CN115098454A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6321304B1 (en) * | 1999-01-22 | 2001-11-20 | Apple Computer, Inc. | System and method for deleting read-only head entries in multi-processor computer systems supporting cache coherence with mixed protocols |
| CN102077165A (en) * | 2008-06-27 | 2011-05-25 | 佳能电子株式会社 | Peripheral device and image reading device |
| CN102467569A (en) * | 2010-11-16 | 2012-05-23 | 英业达股份有限公司 | Method for supporting concurrent read-write of file in data de-duplication |
| CN102521160A (en) * | 2011-12-22 | 2012-06-27 | 上海交通大学 | Write buffer detector, addressing method of written data and parallel channel write method |
| CN105142004A (en) * | 2015-08-28 | 2015-12-09 | 深圳Tcl数字技术有限公司 | Multi-media file reading method and multi-media file reading device |
| CN106354788A (en) * | 2016-08-23 | 2017-01-25 | 无锡天脉聚源传媒科技有限公司 | File reading method and device |
| US20180165214A1 (en) * | 2016-12-13 | 2018-06-14 | Advanced Micro Devices, Inc. | Dynamic cache bypassing |
| CN108491402A (en) * | 2018-01-17 | 2018-09-04 | 艾体威尔电子技术(北京)有限公司 | A kind of implementation method of lightweight file system |
| CN108763099A (en) * | 2018-04-18 | 2018-11-06 | 华为技术有限公司 | Startup method, apparatus, electronic equipment and the storage medium of system |
| CN111159124A (en) * | 2019-12-30 | 2020-05-15 | 浪潮电子信息产业股份有限公司 | Asynchronous write caching method, device and medium for Linux kernel file system |
| CN111880734A (en) * | 2020-07-24 | 2020-11-03 | 北京浪潮数据技术有限公司 | Data processing method, system, electronic equipment and storage medium |
| CN112685363A (en) * | 2020-12-30 | 2021-04-20 | 湖南博匠信息科技有限公司 | User file reading and writing method and device |
| CN113934454A (en) * | 2021-10-19 | 2022-01-14 | 广州小鹏汽车科技有限公司 | Data reading and writing method, micro control unit MCU chip, vehicle and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 王永翔等: "机车故障诊断系统中的司机室显示屏", 《铁道学报》, 15 June 2006 (2006-06-15), pages 67 - 70 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11853549B2 (en) | Index storage in shingled magnetic recording (SMR) storage system with non-shingled region | |
| CN106802803B (en) | Application data processing method and device | |
| CN101208671B (en) | Managing message queues | |
| CN108319654B (en) | Computing system, cold and hot data separation method and device, and computer readable storage medium | |
| CN101419564B (en) | Method and device for recovering data by employing snapshot | |
| CN110147204B (en) | Metadata disk-dropping method, device and system and computer-readable storage medium | |
| CN101707633B (en) | Message-oriented middleware persistent message storing method based on file system | |
| CN102945278A (en) | Method and device for redoing logs of database records | |
| CN107168651B (en) | Small file aggregation storage processing method | |
| CN112988683A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN109614045B (en) | Metadata dropping method and device and related equipment | |
| CN116088770B (en) | Data management method, device, system, electronic equipment and storage medium | |
| CN104965835B (en) | A kind of file read/write method and device of distributed file system | |
| CN102298546A (en) | Method and computer for restoring deleted joint picture group (JPG) file from disk | |
| CN112883065A (en) | Data storage method and device and computer equipment | |
| CN109218799B (en) | Method, storage medium, device and system for quickly switching high-definition images of android television | |
| CN115098454A (en) | Data processing method and system | |
| CN111984198A (en) | Message queue implementation method and device and electronic equipment | |
| CN112799872B (en) | Erasure code encoding method and device based on key value pair storage system | |
| CN115421856A (en) | Data recovery method and device | |
| CN114647658A (en) | Data retrieval method, device, equipment and machine-readable storage medium | |
| CN114020688A (en) | Time sequence database snapshot data filing method and system | |
| CN112463741A (en) | Cleaning method for aggregated large files and related equipment | |
| CN111831206B (en) | Storage space switching method and back-end monitoring system | |
| CN113806249B (en) | Object storage sequence lifting method, device, terminal and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |