CN116881296A - Method, device, equipment and storage medium for monitoring and managing cache data - Google Patents
Method, device, equipment and storage medium for monitoring and managing cache data Download PDFInfo
- Publication number
- CN116881296A CN116881296A CN202310834547.0A CN202310834547A CN116881296A CN 116881296 A CN116881296 A CN 116881296A CN 202310834547 A CN202310834547 A CN 202310834547A CN 116881296 A CN116881296 A CN 116881296A
- Authority
- CN
- China
- Prior art keywords
- cache
- data
- preset
- cache data
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 75
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000009825 accumulation Methods 0.000 claims abstract description 65
- 230000008569 process Effects 0.000 claims abstract description 24
- 238000004590 computer program Methods 0.000 claims description 17
- 238000004519 manufacturing process Methods 0.000 abstract description 8
- 238000007726 management method Methods 0.000 description 34
- 238000004891 communication Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000001186 cumulative effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method, a device, equipment and a storage medium for monitoring and managing cache data. The method comprises the following steps: writing the cache data generated in the process of executing database sentences by the database into a preset cache space; the number of the preset cache spaces is at least one; when the writing quantity of the cache data in each preset cache space meets the disk writing condition, writing the cache data in the cache space into the disk; determining the data accumulation amount of cache data written into a disk under the current disk writing period of each preset cache space; determining whether to terminate the generation of the cache data by the database according to the data accumulation amount corresponding to each preset cache space, and obtaining a termination judgment result; and according to the termination judgment result, controlling the database to continuously generate or terminate generating the cache data so as to realize monitoring and management of the cache data. The embodiment of the invention realizes automatic monitoring and management of the cache data and solves the problem of business production safety caused by overlarge temporary files.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for monitoring and managing cache data.
Background
In the technical field of databases, temporary files in a database are often used to cache data generated during the process of executing database statements, such as order by, group by statements, and the like. Under the condition that some databases execute sentences, temporary files created by the databases can reach the size of tens or hundreds of GB, and the overlarge temporary files cause insufficient disk space, so that a series of serious problems such as transaction submission failure, active-standby switching, service interruption and the like are caused, and the service production safety is greatly influenced.
At present, the state of a magnetic disk is monitored, alarmed and repaired mainly by a mode of combining external monitoring tools with manual intervention. However, the method has obvious defects, when a large number of temporary files are written in the database instantaneously, real-time monitoring and intervention cannot be realized, and the problem of business production safety caused by overlarge temporary files cannot be fundamentally solved.
Disclosure of Invention
The invention provides a method, a device, equipment and a storage medium for monitoring and managing cache data, which are used for realizing automatic monitoring and management of the cache data and solving the problem of business production safety caused by overlarge temporary files.
According to an aspect of the present invention, there is provided a method for monitoring and managing cache data, the method including:
writing the cache data generated in the process of executing database sentences by the database into a preset cache space; the number of the preset cache spaces is at least one;
when the writing quantity of the cache data in each preset cache space meets the disk writing condition, writing the cache data in the cache space into the disk;
determining the data accumulation amount of the cache data written into the disk under the current disk writing period of each preset cache space;
determining whether to terminate the generation of the cache data by the database according to the data accumulation amount corresponding to each preset cache space, and obtaining a termination judgment result;
and controlling the database to continuously generate or terminate generating the cache data according to the termination judging result so as to realize monitoring and management of the cache data.
According to another aspect of the present invention, there is provided a monitoring management apparatus for caching data, including:
the cache data writing module is used for writing cache data generated in the process of executing database sentences in the database into a preset cache space; the number of the preset cache spaces is at least one;
The disk writing module is used for writing the cache data in the self cache space into the disk when the cache data writing quantity of each preset cache space meets the disk writing condition;
the data accumulation amount determining module is used for determining the data accumulation amount of the cache data written into the disk in the current disk writing period of each preset cache space;
the judging result determining module is used for determining whether to terminate the generation of the cache data by the database according to the data accumulation amount corresponding to each preset cache space to obtain a termination judging result;
and the monitoring management module is used for controlling the database to continuously generate or terminate generating the cache data according to the termination judgment result so as to realize monitoring management of the cache data.
According to another aspect of the present invention, there is provided an electronic apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor, so that the at least one processor can execute the method for monitoring and managing cache data according to any embodiment of the present invention.
According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to implement the method for monitoring and managing cache data according to any one of the embodiments of the present invention when the computer instructions are executed.
According to the technical scheme, cache data generated in the process of executing database sentences by a database are written into a preset cache space; when the writing quantity of the cache data in each preset cache space meets the disk writing condition, writing the cache data in the cache space into the disk; determining the data accumulation amount of cache data written into a disk under the current disk writing period of each preset cache space; determining whether to terminate the generation of the cache data by the database according to the data accumulation amount corresponding to each preset cache space, and obtaining a termination judgment result; and according to the termination judgment result, controlling the database to continuously generate or terminate generating the cache data so as to realize monitoring and management of the cache data. According to the technical scheme, the cache data generated by the database are aligned and written into the disk in a temporary writing mode of the preset cache space, whether the cache data are continuously generated is judged according to the data accumulation amount, automatic monitoring and management of the cache data are realized, and the safety problem of the business production process caused by overlarge temporary files is solved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a method for monitoring and managing cache data according to a first embodiment of the present invention;
FIG. 2 is a flowchart of a method for monitoring and managing cache data according to a second embodiment of the present invention;
FIG. 3 is a flowchart of a method for managing and monitoring cache data according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a device for monitoring and managing cache data according to a fourth embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device implementing a method for monitoring and managing cached data according to an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
Fig. 1 is a flowchart of a method for monitoring and managing cache data according to an embodiment of the present invention, where the method may be implemented by a device for monitoring and managing cache data, where the device for monitoring and managing cache data may be implemented in hardware and/or software, and the device for monitoring and managing cache data may be configured in an electronic device. As shown in fig. 1, the method includes:
s110, writing cache data generated in the process of executing database sentences by the database into a preset cache space; the number of the preset cache spaces is at least one.
The database statement executed by the database can be, for example, an order by statement or a group by statement; in addition to the ability to generate cached data during execution of the database statement, the database executing its internal database method may also generate cached data.
The preset buffer space may be a memory space for temporarily storing the buffer data. The preset cache space can be created in advance; for example, the preset CACHE space may be an IO CACHE (CACHE).
It should be noted that the number of the preset buffer spaces may be at least one. In the process of writing the cache data into each preset cache space, one time of cache data is generated and only written into one preset cache space, and the generated second time of cache data can be written into other preset cache spaces, but also written into one preset cache space, specifically, the generated cache data can be sequentially written into each preset cache space by adopting a polling mode.
It should be noted that, the CACHE space is the IO CACHE in the memory, one IO CACHE corresponds to one temporary file, and the CACHE is created when needed and destroyed when not needed. When the temporary file does not need to be deleted, the corresponding CACHE is destroyed.
Exemplary, if there is a preset CACHE space IO CACHE a, a preset CACHE space IO CACHE B, and a preset CACHE space IO CACHE C; generating 50K CACHE data in the process of executing database sentences by the database, and writing the 50K CACHE data into a preset CACHE space IO CACHE A; generating 75K CACHE data in the process of executing database sentences by the subsequent database, and writing the generated 75K CACHE data into a preset CACHE space IO CACHE B according to a polling rule; similarly, a certain amount of CACHE data generated in the process of executing the database statement next time by the database can be written into the preset CACHE space IO CACHE C according to the polling rule.
S120, when the writing quantity of the cache data in each preset cache space meets the disk writing condition, writing the cache data in the cache space into the disk.
The space memory sizes of the preset cache spaces may be the same or different, and correspondingly, the disk writing conditions corresponding to the cache spaces may be the same or different.
For example, for any preset buffer space, if the space memory size of the preset buffer space is 64K, the corresponding disc writing condition may be that when the writing amount of the buffer data written into the preset buffer space reaches 64K, the buffer data of 64K is written into the disc. The data accumulation amount of any preset CACHE space can represent the size of a single CACHE write temporary file.
It should be noted that, if the amount of the written cache data is smaller than the size of the space memory in the preset cache space in the process of writing the cache data into the preset cache space, the next writing of the cache data can be waited until the written cache data reaches the preset disk writing condition, that is, after occupying the space memory 64K, the cache data in the self cache space is written into the disk.
It should be noted that, in general, the buffered data is written to the disk in sequence, the data amount of the disk is small in write-once, the total number of times of writing is large, the data amount of the buffered data written to the disk is not fixed each time, and a certain performance is required to be consumed in order to acquire the temporary file size of the disk. And if the preset CACHE space is the IO CACHE, the preset CACHE space can record the data quantity and the writing times of the written CACHE data, and the temporary file size in the disk is not required to be determined by the consumption performance.
S130, determining the data accumulation amount of the cache data written into the disk in the current disk writing period of each preset cache space.
The data accumulation amount may be an accumulated data amount of cache data written into the disk in each disk writing period in the preset cache space.
For example, if the preset buffer space writes the buffer data of the disk at each disk writing period to 64K, the total number of writing times in the current disk writing period may be determined, and if the total number of writing times is 3, the data accumulation amount of the buffer data written into the disk by the preset buffer space is 192K.
And S140, determining whether to terminate the generation of the cache data by the database according to the data accumulation amount corresponding to each preset cache space, and obtaining a termination judgment result.
For example, if the data accumulation amount corresponding to each preset cache space meets the preset termination condition, determining that the termination database generates cache data, and obtaining a judgment result of terminating the cache data; if the data accumulation amount corresponding to each preset cache space does not meet the preset termination condition, determining that the database continues to generate cache data, and obtaining a judging result of continuing to generate the cache data.
The termination condition may be preset by a related technician, for example, the termination condition may be that a data accumulation amount of a preset number of preset buffer spaces exceeds a set buffer data amount threshold. The preset number and the buffer data amount threshold value can be preset by a related technician. The preset number may be set according to the total number of preset cache spaces, for example, the preset number may be 30% of the total number, and if the total number of preset cache spaces is 20, the preset number may be 6. For example, the buffer data amount threshold may be set to 16M.
And S150, controlling the database to continuously generate or terminate generating the cache data according to the termination judgment result so as to realize monitoring and management of the cache data.
For example, if the termination judgment result is that the database generates the cache data, a corresponding termination instruction may be generated and sent to the database to control the database to terminate generating the cache data, thereby implementing monitoring management of the cache data. If the termination judging result is that the database can continue to generate the cache data, the database can continue to generate the cache data without any response processing, so that the monitoring management of the cache data is realized.
According to the technical scheme, cache data generated in the process of executing database sentences by a database are written into a preset cache space; when the writing quantity of the cache data in each preset cache space meets the disk writing condition, writing the cache data in the cache space into the disk; determining the data accumulation amount of cache data written into a disk under the current disk writing period of each preset cache space; determining whether to terminate the generation of the cache data by the database according to the data accumulation amount corresponding to each preset cache space, and obtaining a termination judgment result; and according to the termination judgment result, controlling the database to continuously generate or terminate generating the cache data so as to realize monitoring and management of the cache data. According to the technical scheme, the cache data generated by the database are aligned and written into the disk in a temporary writing mode of the preset cache space, whether the cache data are continuously generated is judged according to the data accumulation amount, automatic monitoring and management of the cache data are realized, and the safety problem of the business production process caused by overlarge temporary files is solved.
Example two
Fig. 2 is a flowchart of a method for monitoring and managing cache data according to a second embodiment of the present invention, where the embodiment is optimized and improved based on the above technical solutions.
Further, determining whether to terminate the generation of the cache data by the database according to the data accumulation amount corresponding to each preset cache space, and obtaining a termination judgment result to be 'thinned' to determine the total writing times of the cache data written into the disk; if any data accumulation amount of the preset cache space meets the preset accumulation amount judgment condition or the total writing times meet the preset total writing times judgment condition, determining the total data amount of cache data written into the disk according to the corresponding data accumulation amount of each preset cache space; and determining whether to terminate the generation of the cache data by the database according to the total data amount, and obtaining a termination judgment result. To perfect the determination mode of the termination judgment result. In the embodiments of the present invention, the descriptions of other embodiments may be referred to in the portions not described in detail.
As shown in fig. 2, the method comprises the following specific steps:
s210, writing cache data generated in the process of executing database sentences by the database into a preset cache space; the number of the preset cache spaces is at least one.
S220, when the writing quantity of the cache data in each preset cache space meets the disk writing condition, the cache data in the cache space is written into the disk.
S230, determining the data accumulation amount of the cache data written into the disk in the current disk writing period of each preset cache space.
S240, determining the total number of writing times of the cache data written into the disk.
The total number of writing may be the sum of the number of times of writing the cache data into the disk until the current writing period is reached by all the preset cache spaces. The total number of writes may represent the number of times all CACHE writes to the temporary file, which is incremented.
S250, if any data accumulation amount of the preset cache space meets the preset accumulation amount judgment condition or the total writing times meet the preset total writing times judgment condition, determining the total data amount of the cache data written into the disk according to the corresponding data accumulation amount of each preset cache space.
The cumulative amount judgment condition may be preset by a person skilled in the art. For example, the accumulation amount judgment condition may be that the data accumulation amount of the preset buffer space is not less than a preset buffer data amount threshold. The buffer data amount threshold may be preset by a related technician. For example, the buffer data amount threshold is 16M.
The total data amount of the cache data may be the sum of the data accumulation amounts of all the preset cache spaces. Namely, under the current writing period, the total data amount of the cache data written into the disk in the cache space is preset. The total amount of data may represent the total size of all temporary files present, and when a temporary file does not need to be deleted, the temporary file size needs to be subtracted from the total amount of data, which is an addition or subtraction.
For example, if any data accumulation amount of the preset cache space meets a preset accumulation amount judgment condition, or the total writing times meet a preset total writing times judgment condition, determining the total data amount of cache data written into the disk; if any data accumulation amount in the preset cache space does not meet the preset accumulation amount judgment condition and the total writing times do not meet the preset total writing times judgment condition, continuing to want to cache the data in the preset cache space.
And S260, determining whether to terminate the generation of the cache data by the database according to the total data amount, and obtaining a termination judgment result.
For example, if the total data amount is not less than the preset total amount threshold value, the database is terminated to generate cache data; if the total data amount is smaller than the preset total amount threshold value, continuing to generate the cache data.
And S270, controlling the database to continuously generate or terminate generating the cache data according to the termination judgment result so as to realize monitoring and management of the cache data.
In an alternative embodiment, determining whether to terminate the database to generate the cached data according to the total data amount, to obtain a termination judgment result includes: if the total data quantity does not meet the preset total quantity judgment condition, determining to terminate the database to generate cache data; correspondingly, according to the termination judgment result, the control database continuously generates or terminates to generate the cache data so as to realize the monitoring management of the cache data, and the method comprises the following steps: if the termination judgment result is that the termination database generates the cache data, generating a statement termination instruction and sending the statement termination instruction to the database, and controlling the database to terminate generating the cache data according to the statement termination instruction so as to realize monitoring and management of the cache data.
The total amount judgment conditions may be preset by a person skilled in the art. For example, the total amount judgment condition may be that the total amount of data is smaller than a preset total amount threshold. The total amount threshold and the total number threshold may be preset by a person skilled in the art. For example, the total threshold may be 32M.
For example, if the total amount of data does not meet the preset total amount judgment condition, that is, the total amount of data is not less than the preset total amount threshold, it is determined that the generation of the cache data by the database is terminated. If the database is determined to be terminated to generate the cache data, generating a statement termination instruction, and sending the termination instruction to the database so as to control the database to terminate generating the cache data according to the statement termination instruction, thereby realizing monitoring and management of the cache data.
In an alternative embodiment, determining whether to terminate the database to generate the cached data according to the total data amount, to obtain a termination judgment result includes: if the total data quantity meets the preset total quantity judging condition, determining that the database continues to generate cache data; correspondingly, according to the termination judgment result, the control database continuously generates or terminates to generate the cache data so as to realize the monitoring management of the cache data, and the method comprises the following steps: if the termination judgment result is that the database continues to generate the cache data, the database is controlled to continuously generate the cache data so as to realize monitoring and management of the cache data.
For example, if the total amount of data meets a preset total amount judgment condition, that is, the total amount of data is smaller than a preset total amount threshold, it may be determined that the database may continue to generate the cache data. If the termination judgment result is that the database continues to generate the cache data, the database is controlled to continuously generate the cache data so as to realize monitoring and management of the cache data.
In order to further improve the effective management of the cache data and the security of the cache data management process, the performance condition of the disk can be fully considered in the process of monitoring and managing the cache data.
In an alternative embodiment, after the total data amount does not meet the preset total amount judgment condition, determining the available space capacity and the available space duty ratio of the magnetic disk in the current magnetic disk writing period; if the available space capacity meets the preset capacity judgment condition and the available space duty ratio meets the preset duty ratio judgment condition, determining that the database continues to generate the cache data.
The available space capacity can be the space capacity of the magnetic disk for storing the cache data in the current magnetic disk writing period; the available space ratio may be a ratio between the available space capacity of the disk at the current disk write cycle and the total space capacity of the disk.
The capacity judgment condition and the duty ratio judgment condition may be preset by a related technician, for example, the capacity judgment condition may be that the available space capacity is smaller than a preset capacity threshold; the duty ratio judgment condition may be that the available space duty ratio is smaller than a preset duty ratio threshold.
For example, if the available space capacity meets the preset capacity judgment condition, that is, the available space capacity is smaller than the preset capacity threshold, and the available space duty ratio meets the preset duty ratio judgment condition, that is, the available space duty ratio is smaller than the preset duty ratio threshold, it is determined that the database continues to generate the cache data.
Optionally, if the available space capacity does not meet the preset capacity judgment condition or the available space duty ratio does not meet the preset duty ratio judgment condition, the database is terminated to generate the cache data.
For example, if the available space capacity does not meet the preset capacity judgment condition, that is, the available space capacity is not smaller than the preset capacity threshold, or the available space duty ratio does not meet the preset duty ratio judgment condition, that is, the available space duty ratio is not smaller than the preset duty ratio threshold, the database is terminated to continue generating the cache data.
In one embodiment, the temporary data amount generated at a time is usually small, so the temporary data amount can be cached in the CACHE first, if the capacity of a single CACHE is 64K, the above flow is circulated, and after the CACHE capacity is full, the temporary data of the CACHE is written into the temporary file of the disk. One of the main functions of CACHE is to combine a small amount of IO, reduce the number of times of writing into a disk, and reduce the performance influence to a certain extent. Temporary data 30K are generated for the first time and are all cached in CACHE; the temporary data 32K is generated for the second time, or is completely cached in the CACHE; and generating temporary data 50K for the third time, wherein the CACHE capacity is 62K, 2K temporary data are cached in the CACHE, the CACHE capacity is full, and 64K temporary data cached in the CACHE are written into a disk temporary file (1 st disk write). The original 50K temporary data still remains 48K and is cached in CACHE. The temporary data 500K is generated for the fourth time (the case where the amount of single-generation data is large is small but also exists). The CACHE capacity has been used for 48K, at which time 16K temporary data is cached in the CACHE, the CACHE capacity is full, and the CACHE cached 64K temporary data is written to the disk temporary file (disk write 2). The original 500K temporary data still remains 484K, and 484K/64 k=7, so that the 64k×7=448K temporary data is written directly to the disk temporary file again (3 rd disk write). The original 500K temporary data still remains 500K-16K-448 k=36K and continues to be cached in the CACHE.
The writing amount of the temporary file of the magnetic disk of the single CACHE is 64k+64k+448k=576K, the writing times of the magnetic disk are 3 times, and the writing amount and the writing times have no strict corresponding relation. The significance of counting the number of writes is that each thread may create temporary files when considering concurrency, but the size of a single temporary file is small and the use is deleted. In this scenario, the condition for judging the cumulative amount of single CACHE data may not be satisfied, so that the judgment of the number of writing times is increased, and the judgment of whether the total size of the temporary file exceeds the limit after triggering is ensured.
According to the technical scheme, through determining the total data amount and the total writing times of the cache data written into the magnetic disk, when the fact that the data accumulation amount of any preset cache space meets the preset accumulation amount judgment condition is determined, whether the database generates the cache data is determined to be terminated or not according to the data accumulation amount corresponding to each preset cache space or the total writing times meets the preset total writing times judgment condition, and a termination judgment result is obtained, so that accurate judgment on whether the database generates the cache data is achieved, the accuracy of determining the termination judgment result is improved, automatic monitoring management of the cache data is achieved, and the problem of safety of a business production process caused by overlarge temporary files is solved.
Example III
Fig. 3 is a flowchart of a method for monitoring and managing cache data according to a third embodiment of the present invention. The present embodiment provides a preferred example based on the above-described embodiments.
As shown in fig. 3, the method comprises the following specific steps:
s301, writing the cache data generated in the process of executing database sentences in the database into a preset cache space.
S302, when the writing quantity of the cache data in each preset cache space meets the disk writing condition, the cache data in the cache space is written into the disk.
S303, determining the data accumulation amount of the cache data written into the disk in the current disk writing period of each preset cache space.
S304, determining the total writing times of the cache data written into the disk.
S305, if any data accumulation amount of the preset cache space meets the preset accumulation amount judgment condition or the total writing times meet the preset total writing times judgment condition, determining the total data amount of the cache data written into the disk according to the corresponding data accumulation amount of each preset cache space.
S306, if the total data quantity does not meet the preset total quantity judgment condition, determining to terminate the database to generate the cache data.
S307, if the database is determined to be terminated to generate the cache data, generating a statement termination instruction and sending the statement termination instruction to the database, and controlling the database to terminate generating the cache data according to the statement termination instruction so as to realize monitoring and management of the cache data.
S308, if the total data quantity meets the preset total quantity judgment condition, determining the available space capacity and the available space duty ratio of the magnetic disk in the current magnetic disk writing period.
S309, if the available space capacity meets the preset capacity judgment condition and the available space ratio meets the preset ratio judgment condition, determining that the database continues to generate the cache data.
And S310, if the available space capacity does not meet the preset capacity judgment condition or the available space ratio does not meet the preset ratio judgment condition, the database is terminated to generate the cache data.
Example IV
Fig. 4 is a schematic structural diagram of a device for monitoring and managing cache data according to a fourth embodiment of the present invention. The device for monitoring and managing cache data provided by the embodiment of the invention can be suitable for the situation of monitoring and managing the cache data generated in the process of executing database sentences on a database, and the device for monitoring and managing the cache data can be realized in a form of hardware and/or software, as shown in fig. 4, and specifically comprises: a cache data writing module 401, a disk writing module 402, a data accumulation amount determining module 403, a judgment result determining module 404, and a monitoring management module 405. Wherein,,
The cache data writing module 401 is configured to write cache data generated in a process of executing a database statement in a database into a preset cache space; the number of the preset cache spaces is at least one;
the disk writing module 402 is configured to write the cache data in the own cache space into a disk when the write amount of the cache data in each preset cache space meets a disk writing condition;
a data accumulation amount determining module 403, configured to determine a data accumulation amount of cache data written into the disk in the current disk writing period in each preset cache space;
the judgment result determining module 404 is configured to determine whether to terminate the database to generate the cache data according to the data accumulation amounts corresponding to the preset cache spaces, so as to obtain a termination judgment result;
and the monitoring management module 405 is configured to control the database to continue generating or terminate generating the cache data according to the termination judgment result, so as to implement monitoring management on the cache data.
According to the technical scheme, cache data generated in the process of executing database sentences by a database are written into a preset cache space; when the writing quantity of the cache data in each preset cache space meets the disk writing condition, writing the cache data in the cache space into the disk; determining the data accumulation amount of cache data written into a disk under the current disk writing period of each preset cache space; determining whether to terminate the generation of the cache data by the database according to the data accumulation amount corresponding to each preset cache space, and obtaining a termination judgment result; and according to the termination judgment result, controlling the database to continuously generate or terminate generating the cache data so as to realize monitoring and management of the cache data. According to the technical scheme, the cache data generated by the database are aligned and written into the disk in a temporary writing mode of the preset cache space, whether the cache data are continuously generated is judged according to the data accumulation amount, automatic monitoring and management of the cache data are realized, and the safety problem of the business production process caused by overlarge temporary files is solved.
Optionally, the determining module 404 includes:
the total number determining unit is used for determining the total number of writing of the cache data written into the magnetic disk;
a writing total amount determining unit, configured to determine, if there is any preset data accumulation amount of the cache space that satisfies a preset accumulation amount determining condition, or the writing total number of times satisfies a preset writing total number of times determining condition, a total amount of data of the cache data written to the disk according to the data accumulation amount corresponding to each preset cache space;
and the judging result determining unit is used for determining whether to terminate the generation of the cache data by the database according to the total data amount to obtain a termination judging result.
Optionally, the judgment result determining unit includes:
the data termination generating subunit is used for determining to terminate the database to generate cache data if the total data does not meet a preset total judgment condition;
accordingly, the monitoring management module 405 includes:
and the termination instruction generating unit is used for generating a statement termination instruction and sending the statement termination instruction to the database if the termination judgment result is that the database generation data is determined to be terminated, and controlling the database to terminate the generation of the cache data according to the statement termination instruction so as to realize the monitoring management of the cache data.
Optionally, the judgment result determining unit includes:
the cache data generation subunit is used for determining that the database continues to generate cache data if the total data quantity meets a preset total quantity judgment condition;
accordingly, the monitoring management module 405 includes:
and the cache data generation unit is used for controlling the database to continuously generate cache data if the termination judgment result is that the database continuously generates data, so as to realize monitoring and management of the cache data.
Optionally, the judgment result determining unit further includes:
a space capacity determining subunit, configured to determine, after the total data amount does not meet a preset total amount judgment condition and the total writing number does not meet a preset total number judgment condition, an available space capacity and an available space duty ratio of the disk in the current disk writing period;
and the first capacity condition judging subunit is used for determining that the database continuously generates cache data if the available space capacity meets a preset capacity judging condition and the available space duty ratio meets a preset duty ratio judging condition.
Optionally, the judgment result determining unit further includes:
And the second capacity condition judging subunit is used for terminating the database to generate cache data if the available space capacity does not meet the preset capacity judging condition or the available space duty ratio does not meet the preset duty ratio judging condition.
The device for monitoring and managing the cache data provided by the embodiment of the invention can execute the method for monitoring and managing the cache data provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example five
Fig. 5 shows a schematic diagram of an electronic device 50 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 5, the electronic device 50 includes at least one processor 51, and a memory, such as a Read Only Memory (ROM) 52, a Random Access Memory (RAM) 53, etc., communicatively connected to the at least one processor 51, in which the memory stores a computer program executable by the at least one processor, and the processor 51 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 52 or the computer program loaded from the storage unit 58 into the Random Access Memory (RAM) 53. In the RAM 53, various programs and data required for the operation of the electronic device 50 can also be stored. The processor 51, the ROM 52 and the RAM 53 are connected to each other via a bus 54. An input/output (I/O) interface 55 is also connected to bus 54.
Various components in the electronic device 50 are connected to the I/O interface 55, including: an input unit 56 such as a keyboard, a mouse, etc.; an output unit 57 such as various types of displays, speakers, and the like; a storage unit 58 such as a magnetic disk, an optical disk, or the like; and a communication unit 59 such as a network card, modem, wireless communication transceiver, etc. The communication unit 59 allows the electronic device 50 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunications networks.
The processor 51 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 51 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 51 performs the respective methods and processes described above, for example, a monitoring management method of cache data.
In some embodiments, the method of monitoring and managing cache data may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 58. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 50 via the ROM 52 and/or the communication unit 59. When the computer program is loaded into the RAM 53 and executed by the processor 51, one or more steps of the above-described monitoring management method of cache data may be performed. Alternatively, in other embodiments, the processor 51 may be configured to perform the method of monitoring and managing the cached data in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. The method for monitoring and managing the cache data is characterized by comprising the following steps:
writing the cache data generated in the process of executing database sentences by the database into a preset cache space; the number of the preset cache spaces is at least one;
when the writing quantity of the cache data in each preset cache space meets the disk writing condition, writing the cache data in the cache space into the disk;
determining the data accumulation amount of the cache data written into the disk under the current disk writing period of each preset cache space;
Determining whether to terminate the generation of the cache data by the database according to the data accumulation amount corresponding to each preset cache space, and obtaining a termination judgment result;
and controlling the database to continuously generate or terminate generating the cache data according to the termination judging result so as to realize monitoring and management of the cache data.
2. The method according to claim 1, wherein determining whether to terminate the database to generate the cache data according to the data accumulation amount corresponding to each preset cache space, to obtain a termination judgment result includes:
determining the total number of writing of cache data written into a disk;
if any data accumulation amount of the preset cache space meets a preset accumulation amount judgment condition or the total writing times meet a preset total writing times judgment condition, determining the total data amount of cache data written into a disk according to the corresponding data accumulation amount of each preset cache space;
and determining whether to terminate the generation of the cache data by the database according to the total data amount, and obtaining a termination judgment result.
3. The method according to claim 2, wherein determining whether to terminate the database to generate the cache data according to the total data amount, to obtain the termination judgment result, includes:
If the total data amount does not meet the preset total amount judgment condition, determining to terminate the database to generate cache data;
correspondingly, the controlling the database to continuously generate or terminate generating the cache data according to the termination judging result so as to realize monitoring and management of the cache data comprises the following steps:
if the termination judgment result is that the database is determined to be terminated to generate the cache data, generating a statement termination instruction and sending the statement termination instruction to the database, and controlling the database to terminate generating the cache data according to the statement termination instruction so as to realize monitoring management of the cache data.
4. The method according to claim 2, wherein determining whether to terminate the database to generate the cache data according to the total data amount, to obtain the termination judgment result, includes:
if the total data quantity meets a preset total quantity judging condition, determining that the database continues to generate cache data;
correspondingly, the controlling the database to continuously generate or terminate generating the cache data according to the termination judging result so as to realize monitoring and management of the cache data comprises the following steps:
and if the termination judging result is that the database continues to generate the cache data, controlling the database to continue to generate the cache data so as to realize monitoring and management of the cache data.
5. The method according to claim 4, wherein after the if the total amount of data does not satisfy a preset total amount judgment condition, the method further comprises:
determining the available space capacity and the available space duty ratio of the magnetic disk under the current magnetic disk writing period;
and if the available space capacity meets a preset capacity judgment condition and the available space ratio meets a preset ratio judgment condition, determining that the database continues to generate cache data.
6. The method of claim 5, wherein the method further comprises:
and if the available space capacity does not meet a preset capacity judgment condition or the available space ratio does not meet a preset ratio judgment condition, terminating the database to generate cache data.
7. A monitoring and managing apparatus for cache data, comprising:
the cache data writing module is used for writing cache data generated in the process of executing database sentences in the database into a preset cache space; the number of the preset cache spaces is at least one;
the disk writing module is used for writing the cache data in the self cache space into the disk when the cache data writing quantity of each preset cache space meets the disk writing condition;
The data accumulation amount determining module is used for determining the data accumulation amount of the cache data written into the disk in the current disk writing period of each preset cache space;
the judging result determining module is used for determining whether to terminate the generation of the cache data by the database according to the data accumulation amount corresponding to each preset cache space to obtain a termination judging result;
and the monitoring management module is used for controlling the database to continuously generate or terminate generating the cache data according to the termination judgment result so as to realize monitoring management of the cache data.
8. The apparatus of claim 7, wherein the determination module comprises:
a writing total amount determining unit, configured to determine, if a data accumulation amount of any preset cache space meets a preset accumulation amount judgment condition, a total data amount and a total writing number of cache data written into a disk according to a data accumulation amount corresponding to each preset cache space;
and the judging result determining unit is used for determining whether to terminate the generation of the cache data by the database according to the total data amount and the total writing times to obtain a termination judging result.
9. An electronic device, the electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the method of monitoring and managing cache data according to any one of claims 1 to 6.
10. A computer readable storage medium storing computer instructions for causing a processor to implement the method for monitoring and managing cache data according to any one of claims 1 to 6 when executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310834547.0A CN116881296A (en) | 2023-07-07 | 2023-07-07 | Method, device, equipment and storage medium for monitoring and managing cache data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310834547.0A CN116881296A (en) | 2023-07-07 | 2023-07-07 | Method, device, equipment and storage medium for monitoring and managing cache data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116881296A true CN116881296A (en) | 2023-10-13 |
Family
ID=88263679
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310834547.0A Pending CN116881296A (en) | 2023-07-07 | 2023-07-07 | Method, device, equipment and storage medium for monitoring and managing cache data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116881296A (en) |
-
2023
- 2023-07-07 CN CN202310834547.0A patent/CN116881296A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112540731B (en) | Data append writing method, device, equipment, medium and program product | |
| CN116594563A (en) | Distributed storage capacity expansion method and device, electronic equipment and storage medium | |
| CN116578502A (en) | Access request processing device, processing method, equipment and storage medium | |
| CN116881296A (en) | Method, device, equipment and storage medium for monitoring and managing cache data | |
| CN114564149B (en) | Data storage method, device, equipment and storage medium | |
| CN115718732A (en) | Disk file management method, device, equipment and storage medium | |
| CN115617549A (en) | Thread decoupling method and device, electronic equipment and storage medium | |
| CN113220554B (en) | Method and apparatus for detecting performance of program code | |
| CN116719621B (en) | Data write-back method, device, equipment and medium for mass tasks | |
| CN115442432B (en) | Control method, device, equipment and storage medium | |
| CN114610575B (en) | Method, apparatus, device and medium for calculating updated peak value of branch | |
| CN116431561B (en) | Data synchronization method, device, equipment and medium based on heterogeneous many-core accelerator card | |
| CN113722389B (en) | Data management method, device, electronic equipment and computer readable storage medium | |
| CN116954922A (en) | Distributed storage method, device, equipment and medium | |
| CN116501549A (en) | Data caching method and device, electronic equipment and storage medium | |
| CN116126249A (en) | Data reading method and device, electronic equipment and storage medium | |
| CN116450915A (en) | Application degradation method, device, electronic equipment and storage medium | |
| CN117555830A (en) | Flow control method, device, chip and storage medium | |
| CN113220233A (en) | Data reading method, device and system | |
| CN116226251A (en) | Data export method and device, electronic equipment and storage medium | |
| CN118051173A (en) | Storage process management method, device, equipment and medium | |
| CN117194113A (en) | Container snapshot generation method, device, equipment and medium | |
| CN117827949A (en) | Method and device for batch writing of data into database based on memory queue | |
| CN117193726A (en) | Parallel design method and device of software, electronic equipment and medium | |
| CN116450411A (en) | Method, device, equipment and storage medium for determining replication delay state |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |