CN107562782B - CIM (common information model) -based multi-level caching method, device and system - Google Patents
CIM (common information model) -based multi-level caching method, device and system Download PDFInfo
- Publication number
- CN107562782B CN107562782B CN201710606890.4A CN201710606890A CN107562782B CN 107562782 B CN107562782 B CN 107562782B CN 201710606890 A CN201710606890 A CN 201710606890A CN 107562782 B CN107562782 B CN 107562782B
- Authority
- CN
- China
- Prior art keywords
- cim
- pool
- expiration
- cim object
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The invention discloses a multi-level caching method, a device and a system based on a CIM (common information model), wherein the method comprises the following steps: establishing a two-level stepped cache pool formed by a soft reference pool and a weak reference pool, and caching the CIM object referenced by the reference pool, wherein the CIM object comprises a time stamp for recording the current access time of the CIM object; establishing a dynamic CIM object expiration checking thread; according to the utilization rate of the maximum configuration memory, the CIM object expiration checking thread periodically arranges the two-level stepped cache pool, and releases the checked expired CIM object. According to the invention, the lower-level daemon thread is arranged on the two-level stepped CIM object class cache pool, so that the available memory is utilized to the maximum extent to increase the CIM object cache amount, and the cache is ensured to be opened without influencing the normal operation of the system.
Description
Technical Field
The invention relates to the technical field of data transmission, in particular to a multi-level caching method, a multi-level caching device and a multi-level caching system based on a CIM (common information model).
Background
The Common Information Model (CIM) proposed by IEC 61970 series of standards is widely applied to power systems. When client applications such as topology analysis and data quality overall inspection of a large-scale power grid model are developed by adopting Java, multiple rounds of traversal processing need to be performed based on the hierarchical inclusion relationship of objects and the association relationship between the objects, that is, analysis with width priority or depth priority needs to be completed according to the object classification set and the mesh and tree relationships between the objects (each object constitutes one node in a data structure). During the analysis, some object data may be repeatedly accessed at certain stages, and then additional objects may be processed without being accessed for a longer period of time. The access requirements change as the analysis focuses on changes.
In order to complete access to the CIM object data, there are two main technical solutions in the prior art, one is that a client directly queries from a server through an interface every time the client needs to access the CIM object data, and the client does not cache itself. The other method is that the client establishes a complete set for the data required to be used each time, namely, all power grid CIM objects are cached, and references between the objects are established. The first scheme interacts with the server frequently, so that a large amount of network interaction operation is caused, on one hand, the server burden is increased, and on the other hand, the object data acquisition efficiency is low; in the second scheme, due to the mesh relationship of CIM objects, almost all objects are related based on "strong reference (when the reference of an object is held, garbage collection cannot release the referenced object)", and garbage collection of the Java virtual machine cannot release part of the infrequent objects, so that very large memory resources need to be configured, otherwise an out of memory (out of memory) exception is easily caused.
Disclosure of Invention
The invention mainly aims to provide a CIM model-based multi-level caching method, aiming at solving the problem that the referenced CIM object cannot be released in garbage collection of a CIM object cache pool which forms a two-level stair type by a soft reference pool and a weak reference pool.
In order to achieve the above purpose, the multi-level caching method for removing strong references of target objects based on a CIM model provided by the invention comprises the following steps:
s10, establishing a two-level stepped cache pool formed by the soft reference pool and the weak reference pool for caching the CIM object referenced by the reference pool, wherein the CIM object comprises a time stamp for recording the current access time of the CIM object;
s20, establishing a dynamic CIM object expiration check thread;
s30, according to the utilization rate of the maximum configuration memory, the CIM object expiration checking thread periodically arranges the two-level stepped cache pool, and releases the checked expired CIM object.
Preferably, the step of S30 includes:
s301, if the utilization rate of the maximum configuration memory is larger than a first threshold value, calculating the difference between the current time and the time scale of the CIM object pointed by each entry in the soft reference pool, and acquiring the time difference of the CIM object pointed by each entry in the soft reference pool;
s302, establishing a dynamic formula of the expiration time length, and comparing the time difference of the CIM object pointed by each item in the soft reference pool with the expiration time length to obtain the CIM object of which the time difference is greater than the expiration time length;
s303, arranging the CIM objects with the time difference larger than the expiration time limit duration into a weak reference pool.
Preferably, the dynamic formula of the expiration duration is:
wherein u is the memory usage rate in sorting, TiFor the time consumed by the last soft reference pool sorting process, SiIs the time interval from the end of the last soft reference buffer sorting operation to the start of the soft reference buffer sorting operation, texpIs the expiration time duration.
Preferably, the step of S30 further includes:
s304, if the utilization rate of the maximum configuration memory is larger than a second threshold value, releasing the CIM object referenced by the current weak reference pool.
Preferably, the second threshold is greater than the first threshold.
The invention also provides a multi-level cache device based on the CIM model, which comprises:
a first establishing module 10, configured to establish a two-level stepped cache pool formed by a soft reference pool and a weak reference pool, and configured to cache a CIM object referenced by the reference pool, where the CIM object includes a time stamp recording a current access time of the CIM object;
a second establishing module 20, configured to establish a dynamic CIM object expiration check thread;
and the expiration checking module 30 is configured to periodically arrange the two-level stepped cache pool by the CIM object expiration checking thread according to the utilization rate of the maximum configuration memory, and release the checked expired CIM object.
Preferably, the expiration checking module 30 includes:
a first obtaining unit 301, configured to calculate a difference between a current time and a time scale of a CIM object pointed to by each entry in the soft reference pool, and obtain a time difference of the CIM object pointed to by each entry in the soft reference pool, if a usage rate of the maximum configuration memory is greater than a first threshold;
a second obtaining unit 302, configured to establish a dynamic formula of the expiration time duration, and compare the time difference and the expiration time duration of the CIM object pointed by each entry in the soft reference pool to obtain a CIM object whose time difference is greater than the expiration time duration;
and the sorting unit 303 is configured to sort the CIM objects with the time difference greater than the expiration duration into the weak reference pool.
Preferably, the expiration checking module 30 further includes a releasing unit 304, configured to release the CIM object referenced by the current weak reference pool if the usage rate of the maximum configuration memory is greater than a second threshold.
The invention also provides a multi-level cache system based on the CIM model, which comprises a server and the multi-level cache device for removing the strong reference of the target object based on the CIM model, and is used for realizing the multi-level cache method for removing the strong reference of the target object based on the CIM model.
The technical scheme discloses a target object query access method, a device and a system based on a CIM model and a multi-level cache method, a device and a system for removing strong reference of CIM power grid model object data, and the two methods are combined to solve the problem of transmission and processing of dynamic cache data for removing strong reference of the CIM object, compared with the prior art, the technical scheme has the beneficial effects that:
the method is used in application software such as power grid CIM model data analysis and the like, and is particularly suitable for the situations of longitudinal level traversal analysis and transverse power grid topology traversal analysis of the model. For a large-scale power grid, because memory resources are limited or the data volume is huge, the whole power grid model cannot be loaded into the memory generally, and in this case, the dynamic cache processing method for removing strong reference of the CIM object has obvious advantages. The main body is as follows:
1) balancing access efficiency and memory usage. After the memory usage exceeds a certain limit (70%), an expiration period is dynamically set according to the memory usage rate, and the memory is effectively maintained at a reasonable level. When the utilization rate of the memory is high, the expiration period is shortened, the memory can be quickly vacated from the CIM object cache, and the weak reference buffer pool is emptied, so that the available memory is increased as much as possible; and when the utilization rate of the memory is low, the cache objects are not released, so that the cache data volume is increased, the hit rate of object retrieval is improved, and the server query is reduced.
2) Flexibly adapting to the total amount of memory configured for the application. If the total amount of the memory configured for the application is enough relative to the power grid model data to be analyzed, the data object managed by soft reference does not need to be released, and repeated query to the server is not needed; if the data volume of the power grid model to be analyzed is large and the data cannot be completely loaded into the memory, the program can still normally run, and the object data retrieval efficiency is improved through rotation of the two buffer areas.
3) The memory overflow is effectively prevented. With the soft-reference and weak-reference mechanisms, even under extreme memory constraints, there is no memory overflow because the resources held by both soft-references and weak-references are recoverable by the JVM garbage collector.
4) The use is simple. The application end of the object cache only needs to access the object data according to needs, does not need to consider how the data is loaded from the server, and does not need to actively release the data.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the structures shown in the drawings without creative efforts.
FIG. 1 is a flowchart of a method of an embodiment of a CIM-model-based multi-level caching method of the present invention;
FIG. 2 is a flowchart of the method of S30 according to the present invention;
FIG. 3 is a flowchart of a method of another embodiment of the S30 of the present invention;
FIG. 4 is a functional block diagram of an embodiment of a multi-level cache apparatus based on a CIM model according to the present invention;
FIG. 5 is a detailed diagram of the functional modules of the expiration checking module of the present invention;
FIG. 6 is a detailed diagram of the functional modules of another embodiment of the expiration checking module of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that, if directional indications (such as up, down, left, right, front, and back … …) are involved in the embodiment of the present invention, the directional indications are only used to explain the relative positional relationship between the components, the movement situation, and the like in a specific posture (as shown in the drawing), and if the specific posture is changed, the directional indications are changed accordingly.
In addition, if there is a description of "first", "second", etc. in an embodiment of the present invention, the description of "first", "second", etc. is for descriptive purposes only and is not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.
As shown in fig. 1-6, the present invention provides a multi-level caching method based on a CIM model, which includes the following steps:
s10, establishing a two-level stepped cache pool formed by the soft reference pool and the weak reference pool for caching the CIM object referenced by the reference pool, wherein the CIM object comprises a time stamp for recording the current access time of the CIM object;
s20, establishing a dynamic CIM object expiration check thread;
s30, according to the utilization rate of the maximum configuration memory, the CIM object expiration checking thread periodically arranges the two-level stepped cache pool, and releases the checked expired CIM object.
In this embodiment, the present invention further solves the problem on the basis of a target object query access method based on a CIM model, where the target object query access method based on the CIM model is as follows: generating JAVA classes corresponding to the CIM classes based on the CIM model, wherein the JAVA classes need to be directly or indirectly (through a parent class) inherited from a specific class, namely a CIM object (CIMObject), and the CIM object comprises the following properties: id, the unique identifier of the CIM object and a timestamp, which are used for recording the current access time of the CIM object, a CIM object cache pool is constructed by a target object access management class, a main reference pool is constructed by HashMap and is a soft reference pool, an auxiliary reference pool is a weak reference pool, a two-level stair type CIM object cache pool is formed by the soft reference pool and the weak reference pool, the keys of the entries of the two reference pools are the IDs of CIMObjects, the value of the entry in the soft reference pool is SoftReference < CIMObject >, the value of the entry in the weak reference pool is WeakReference < CIMObject > (namely, the data entry in the weak reference pool is the data pair (pair) of ID and SoftReference < CIMObject >); the object access management class receives a query access request of a target object based on a two-level stepped CIM object cache pool formed by a soft reference pool and a weak reference pool, and acquires a target object ID from the received query access request; matching the obtained target object ID with the ID of the CIM object referenced by the two-level stepped CIM object cache pool, namely matching the ID with the bin values of keys of entries of a soft reference pool and/or entries of a weak reference pool, if the matching is successful, obtaining the soft reference of the CIM object corresponding to the ID, then obtaining the 'strong' reference of the object by using a get method of the soft reference, obtaining the CIM object successfully, accessing the inquired CIM object, recording the current access time by the timestamp of the CIM object, and at the moment, locally accessing the CIM object by taking the CIM object held by the reference as an example for good storage; if the query is not successful, the target object is not stored in the CIM object cache pool, and the target object is turned to a server for query access; if the target object ID fails to be queried in the CIM object cache pool, turning to a server to execute query, if the query in the server is successful through the target object ID, generating a CIM object of a soft reference pool having the same ID as the target object, accessing the queried CIM object by using the ID of the CIM object and the entry of the soft reference pool of the CIM object, and recording the current access time by the time scale of the CIM object; if the query in the server fails through the target object ID, returning the failure of the query access request of the target object; the step of accessing the queried CIM object in the above method includes: judging whether the inquired CIM object is matched with the target object; if the CIM objects are matched, accessing the inquired CIM objects, and recording the current access time of the time scale of the CIM objects; and if not, the query access request returned to the target object fails.
According to the target object query access method based on the CIM model, the soft reference relation replaces the strong reference relation between CIM objects, and the problems that almost all objects are related based on 'strong reference (Strong reference, when the reference of the object is held, garbage collection cannot release the referenced object)' between the objects, and garbage collection of a Java virtual machine cannot release part of the unused objects, so that very large memory resources need to be configured, otherwise, an out-of-memory (OutOfmemory) exception is easily caused are solved; secondly, a soft reference pool (when object references are held, if garbage collection judges that the memory is insufficient, the referenced objects can be released) and a weak reference pool (Weak reference, if the garbage collection finds the weak references, the referenced objects can be released) are established to form a two-level stepped CIM object cache pool, and rapid localized retrieval is provided for multiple access of CIM objects by the application.
Furthermore, after the utilization rate of the maximum configuration memory reaches a certain limit, the object expiration check is started, and objects which are not accessed within the dynamically calculated expiration time duration are moved from the soft reference pool to the weak reference pool. The soft reference pool and the weak reference pool form cache levels with different reference strengths, so that the Java virtual machine can recycle the memory according to the needs, the effects of dynamically controlling the data caching amount and reducing the actual memory overhead of a larger power grid model are achieved, the memory overflow is effectively avoided, the interaction amount with the server is reduced, and the object query efficiency is improved. The main thread and the daemon thread belong to a process, the process fetches CIM object data to be used for analysis application, the two-level stepped CIM object cache pool is a cache system built in analysis application software, the maximum configuration memory refers to the maximum available memory configured for the operation of the analysis application software when an actual JAVA program operates, the server refers to a source for the analysis application software to acquire the CIM object data, and the system refers to a network system formed by any server and any terminal provided with the analysis application software. The invention solves the use problem of the memory configured by the software similar to the analysis application software, and reduces the interaction with other software and the server through the two-level stepped cache pool; and the maximum memory space configured by the software is cleared in time through the establishment and execution of the object expiration checking thread of the two-level stepped cache pool.
According to the technical scheme, a large number of CIM objects are repeatedly used for carrying out overall analysis and calculation of the power grid, the system originally needs to carry out analysis after data are obtained from the server for multiple times, and the interaction frequency of the server is reduced and the object data access efficiency is improved by adding two-stage cascaded CIM object caching processing.
Furthermore, the invention sets a guard thread with lower priority for the two-level stepped CIM object cache pool, namely an expiration check thread of the CIM object, and actively and periodically cleans the cache object by using the guard thread with lower priority, thereby ensuring that the available memory is maximally utilized to increase the cache amount of the CIM object and ensuring that the cache opening does not influence the memory available amount of the normal operation of the system. The daemon thread is an expiration check thread for opening a CIM object to a two-level stepped CIM object class cache pool; the lower priority of the daemon thread is also lower relative to the query access thread of the target object based on the CIM model. Meanwhile, due to the adoption of the two-level stepped cache pool of the CIM object, even under the condition of extreme memory limitation, the memory cannot overflow, and the resources held by the soft references and the weak references can be recycled by the JAVA class garbage collector.
The method is used in application software such as power grid CIM model data analysis and the like, and is particularly suitable for the situations of longitudinal level traversal analysis and transverse power grid topology traversal analysis of the model. For a large-scale power grid, because memory resources are limited or the data volume is huge, the whole power grid model cannot be loaded into the memory generally, and in this case, the dynamic cache processing method for removing strong reference of the CIM object has obvious advantages.
Further, the step of S30 includes:
s301, if the utilization rate of the maximum configuration memory is larger than a first threshold value, calculating the difference between the current time and the time scale of the CIM object pointed by each entry in the soft reference pool, and acquiring the time difference of the CIM object pointed by each entry in the soft reference pool;
s302, establishing a dynamic formula of the expiration time length, and comparing the time difference of the CIM object pointed by each item in the soft reference pool with the expiration time length to obtain the CIM object of which the time difference is greater than the expiration time length;
s303, arranging the CIM objects with the time difference larger than the expiration time limit duration into a weak reference pool.
Further, the dynamic formula of the expiration duration is as follows:
wherein u is the memory usage rate in sorting, TiFor the time consumed by the last soft reference pool sorting process, SiIs the time interval from the end of the last soft reference buffer sorting operation to the start of the soft reference buffer sorting operation, texpIs the expiration time duration.
In this embodiment, the first threshold is a threshold for actively sorting the cache objects in the soft reference pool, the expiration period duration is a dynamic formula related to the memory usage rate, and the expiration period is dynamically set according to the first threshold, so as to effectively maintain the memory at a reasonable level. When the memory utilization rate is high, the expiration period is shortened, and the storage space can be quickly obtained from the CIM object cache; and when the utilization rate of the memory is low, the cache objects are not released, the cache data volume is increased, the hit rate of object retrieval can be improved, and the interactive query with the server is reduced. Due to the setting of the first threshold value, the total amount of the memory configured as the application is flexibly adapted, and if the total amount of the memory configured to the application is enough relative to the power grid model data to be analyzed, the data object managed by soft reference does not need to be released, and repeated query to the server is not needed; if the data volume of the power grid model to be analyzed is large and the data cannot be completely loaded into the memory, the program can still normally run, and the object data retrieval efficiency is improved through rotation of the two buffer areas.
Further, the step of S30 further includes:
s304, if the utilization rate of the maximum configuration memory is larger than a second threshold value, releasing the CIM object referenced by the current weak reference pool.
Further, the second threshold is greater than the first threshold.
In this embodiment, the second threshold is a critical value for actively sorting the cache objects in the weak reference pool, and if the memory usage rate is greater than the second threshold, the system releases the weak reference pool to obtain the memory space. The second threshold is greater than the first threshold, so that the step S40 is only performed when the cache objects in the soft reference pool are sorted in advance, and the cache objects included in the weak reference pool are sorted: and the cache objects originally referenced by the weak reference pool and the outdated cache objects moved to the weak reference pool are sorted by the soft reference pool.
The present invention further provides a multi-level cache device based on the CIM model, where the multi-level cache device is used to implement the multi-level cache method for removing strong reference of a target object based on the CIM model, and the multi-level cache device for removing strong reference of a target object based on the CIM model adopts all the technical solutions of all the embodiments, so that the multi-level cache device at least has all the beneficial effects brought by the technical solutions of the embodiments, and details are not repeated herein. The multi-level cache device for removing the strong reference of the target object based on the CIM model comprises the following components:
a first establishing module 10, configured to establish a two-level stepped cache pool formed by a soft reference pool and a weak reference pool, and configured to cache a CIM object referenced by the reference pool, where the CIM object includes a time stamp recording a current access time of the CIM object;
a second establishing module 20, configured to establish a dynamic CIM object expiration check thread;
and the expiration checking module 30 is configured to periodically arrange the two-level stepped cache pool by the CIM object expiration checking thread according to the utilization rate of the maximum configuration memory, and release the checked expired CIM object.
Further, the expiration checking module 30 includes:
a first obtaining unit 301, configured to calculate a difference between a current time and a time scale of a CIM object pointed to by each entry in the soft reference pool, and obtain a time difference of the CIM object pointed to by each entry in the soft reference pool, if a usage rate of the maximum configuration memory is greater than a first threshold;
a second obtaining unit 302, configured to establish a dynamic formula of the expiration time duration, and compare the time difference and the expiration time duration of the CIM object pointed by each entry in the soft reference pool to obtain a CIM object whose time difference is greater than the expiration time duration;
and the sorting unit 303 is configured to sort the CIM objects with the time difference greater than the expiration duration into the weak reference pool.
Further, the expiration checking module 30 further includes a releasing unit 304, configured to release the CIM object referenced by the current weak reference pool if the usage rate of the maximum configuration memory is greater than a second threshold.
The invention also discloses a multi-level cache system based on CIM model, comprising a server, and further comprising the multi-level cache device for removing strong reference of target object based on CIM model as claimed in claims 6-8, for implementing the multi-level cache method for removing strong reference of target object based on CIM model as claimed in claims 1-5.
In the technical scheme, the invention provides a multi-level caching method based on a CIM (common information model), firstly, a soft reference relation is used for replacing a strong reference relation between CIM objects, and the problems that almost all objects are related based on 'strong reference (Strong reference, when the reference of the objects is held, garbage collection cannot release the referenced objects)' between the objects, the garbage collection of a Java virtual machine cannot release part of the unused objects, so that very large memory resources need to be configured, and otherwise, the abnormal condition of insufficient memory (OutOfmemory) is easily caused are solved; secondly, a two-level stepped cache pool of the CIM object class is established, wherein the cache pool is formed by a soft reference pool (when object reference is held, if garbage collection judges that the memory is insufficient, the referenced object can be released) and a weak reference pool (Weak reference, if garbage collection finds weak reference, the referenced object can be released, but because the operation priority of a garbage collection thread is low, if a processing thread with high priority is operated, the object pointed by the weak reference can be delayed to be released), and rapid localized retrieval is provided for accessing the CIM object for multiple times by application. And after the utilization rate of the memory reaches a certain limit, starting object expiration check, and moving the objects which are not accessed within the dynamically calculated expiration time duration from the soft reference pool to the weak reference pool. The soft reference pool and the weak reference pool form cache levels with different reference strengths, so that the Java virtual machine can recycle the memory according to the needs, the effects of dynamically controlling the data caching amount and reducing the actual memory overhead of a larger power grid model are achieved, the memory overflow is effectively avoided, the interaction amount with the server is reduced, and the object query efficiency is improved.
The invention of the technical scheme is characterized in that: starting a lower priority daemon process, periodically analyzing and sorting data in the soft reference pool, dynamically setting an expiration period by a preset threshold value of memory occupancy (ratio of memory used by a program to memory available by the program), and moving an expired object to the weak reference pool. Taking the memory occupancy rate of 70% as an example (70% is an empirical value), taking the memory occupancy rate as a predetermined threshold value for executing the soft reference pool to actively arrange the cache object by using the daemon process; taking the memory occupancy rate of 85% as an example (85% is an empirical value), the weak reference pool is used as a daemon process to execute a predetermined threshold value for actively sorting the cache objects.
The live-action operation steps of the technical scheme are as follows:
(1) and constructing a cache pool of the two-level stepped CIM object by using soft references and weak references to store the CIM object, wherein the CIM object comprises an ID (identity) for uniquely identifying the CIM object and a time stamp for recording the current access time of the CIM object, the soft reference pool is a main cache, and the weak reference pool is an auxiliary cache.
(2) A lower priority daemon is created for periodically traversing the analysis soft reference pool, dynamically setting an expiration deadline at a predetermined threshold of memory occupancy, and moving expired objects to the weak reference pool.
(3) And periodically traversing and analyzing the soft reference pool by a daemon thread with lower priority, dynamically setting an expiration deadline according to the use level of the memory, and moving an expired object to the weak reference pool.
(4) Judging whether the CIM object pointed by the current reference is expired, wherein the expiration condition is as follows: and judging whether the utilization rate of the maximum configuration memory reaches 70 percent, if the utilization rate of the maximum configuration memory is more than 70 percent, if so, judging again whether the utilization rate of the memory is more than 85 percent, if so, releasing cache data quoted by the weak reference pool, if more than 70 percent and less than or equal to 85 percent, sorting the soft reference pool, and if not, not sorting the soft reference pool.
(5) Calculating the dynamic expiration time duration used by the sorting operation, gradually querying the CIM object in the two-level stepped cache pool, acquiring the latest access time (the latest access time, namely the current access time, which can be understood as the time used by the current CIM object) of the queried CIM object, judging whether the CIM object pointed by the current reference is expired or not, if the expiration condition is satisfied, moving the object to the weak reference pool for management, and managing according to the relation of the dynamic expiration time duration of the memory usage rate:
(i＝1，2，3，…，T1＝0,S1＝10)
wherein u is the memory utilization rate during sorting; ti is the time consumed in the last soft citation pool arrangement process; siIs the time interval from the end of the last soft reference buffer sorting operation to the start of the soft reference buffer sorting operation. T when the memory consumption is greater than 70% for the first time and soft reference pool sorting is performedexpFor example, 10 is a dynamic expiration of memory usage, i.e., unused objects in 10 seconds are moved from the soft reference pool to the weak reference pool,
(6) if the accessed CIM object is retrieved to be in the weak reference pool, the CIM object is upgraded to be held in the soft reference pool, and the timestamp records the current access time of the CIM object in the soft reference pool.
The technical scheme discloses a target object query access method and device based on a CIM (common information model) and a multi-level cache method and device for removing strong reference of CIM power grid model object data, wherein the two methods are combined to solve the problems of transmission and processing of dynamic cache data for removing strong reference of the CIM object, and compared with the prior art, the technical scheme has the beneficial effects that:
the method is used in application software such as power grid CIM model data analysis and the like, and is particularly suitable for the situations of longitudinal level traversal analysis and transverse power grid topology traversal analysis of the model. For a large-scale power grid, because memory resources are limited or data volume is huge, the whole power grid model cannot be loaded into the memory generally, and in this case, the technical scheme has obvious advantages. The main body is as follows:
1) the access efficiency and the memory utilization rate are balanced, and after the memory usage exceeds a specific limit (70%), the expiration period is dynamically set according to the memory utilization rate, so that the memory is effectively maintained at a reasonable level. When the utilization rate of the memory is high, the expiration period is shortened, the memory can be quickly vacated from the CIM object cache, and the weak reference buffer pool is emptied, so that the available memory is increased as much as possible; and when the utilization rate of the memory is low, the cache objects are not released, so that the cache data volume is increased, the hit rate of object retrieval is improved, and the server query is reduced.
2) Flexibly adapting to the total amount of the memory configured for the application, if the total amount of the memory configured for the application is enough relative to the power grid model data to be analyzed, the data object managed by soft reference does not need to be released, and repeated query to the server is not needed; if the data volume of the power grid model to be analyzed is large and the data cannot be completely loaded into the memory, the program can still normally run, and the object data retrieval efficiency is improved through the rotation release of the two buffer areas.
3) The memory overflow is effectively prevented. With the soft-reference and weak-reference mechanisms, even under extreme memory constraints, there is no memory overflow because the resources held by both soft-references and weak-references are recoverable by the JVM garbage collector.
4) The use is simple. The application end of the object cache only needs to access the object data according to needs, does not need to consider how the data is loaded from the server, and does not need to actively release the data.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and all modifications and equivalents of the present invention, which are made by the contents of the present specification and the accompanying drawings, or directly/indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (9)
1. A multi-level caching method based on a CIM model is characterized by comprising the following steps:
s10, establishing a two-level stepped cache pool formed by the soft reference pool and the weak reference pool for caching the CIM object referenced by the reference pool, wherein the CIM object comprises a time stamp for recording the current access time of the CIM object;
s20, establishing a dynamic CIM object expiration check thread;
s30, according to the utilization rate of the maximum configuration memory, the CIM object expiration checking thread periodically arranges the two-level stepped cache pool, and releases the checked expired CIM object.
2. The CIM model-based multi-level caching method of claim 1, wherein the step of S30 comprises:
s301, if the utilization rate of the maximum configuration memory is larger than a first threshold value, calculating the difference between the current time and the time scale of the CIM object pointed by each entry in the soft reference pool, and acquiring the time difference of the CIM object pointed by each entry in the soft reference pool;
s302, establishing a dynamic formula of the expiration time length, and comparing the time difference of the CIM object pointed by each item in the soft reference pool with the expiration time length to obtain the CIM object of which the time difference is greater than the expiration time length;
s303, arranging the CIM objects with the time difference larger than the expiration time limit duration into a weak reference pool.
3. The CIM-model-based multi-level caching method of claim 2, wherein the dynamic formula of the expiration duration is as follows:
wherein u is the memory usage rate in sorting, TiFor the time consumed by the last soft reference pool sorting process, SiIs the time interval from the end of the last soft reference buffer sorting operation to the start of the soft reference buffer sorting operation, texpIs the expiration time duration.
4. The CIM model-based multi-level caching method according to any one of claims 1-3, wherein the step of S30 further comprises:
s304, if the utilization rate of the maximum configuration memory is larger than a second threshold value, releasing the CIM object referenced by the current weak reference pool.
5. The CIM model-based multi-level caching method of claim 4, wherein the second threshold is greater than the first threshold.
6. A multi-level cache device based on CIM model is characterized by comprising:
a first establishing module 10, configured to establish a two-level stepped cache pool formed by a soft reference pool and a weak reference pool, and configured to cache a CIM object referenced by the reference pool, where the CIM object includes a time stamp recording a current access time of the CIM object;
a second establishing module 20, configured to establish a dynamic CIM object expiration check thread;
and the expiration checking module 30 is configured to periodically arrange the two-level stepped cache pool by the CIM object expiration checking thread according to the utilization rate of the maximum configuration memory, and release the checked expired CIM object.
7. The CIM model-based multi-level caching apparatus of claim 6, wherein the expiration checking module 30 comprises:
a first obtaining unit 301, configured to calculate a difference between a current time and a time scale of a CIM object pointed to by each entry in the soft reference pool, and obtain a time difference of the CIM object pointed to by each entry in the soft reference pool, if a usage rate of the maximum configuration memory is greater than a first threshold;
a second obtaining unit 302, configured to establish a dynamic formula of the expiration time duration, and compare the time difference and the expiration time duration of the CIM object pointed by each entry in the soft reference pool to obtain a CIM object whose time difference is greater than the expiration time duration;
and the sorting unit 303 is configured to sort the CIM objects with the time difference greater than the expiration duration into the weak reference pool.
8. The CIM-model-based multi-level caching apparatus according to any one of claims 6 to 7, wherein the expiration checking module 30 further comprises a releasing unit 304, configured to release the CIM object referenced by the current weak reference pool if the maximum configured memory usage rate is greater than a second threshold.
9. A multi-level caching system based on a CIM model, comprising a server and a terminal, further comprising the multi-level caching apparatus based on the CIM model according to any one of claims 6 to 8, for implementing the multi-level caching method based on the CIM model according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710606890.4A CN107562782B (en) | 2017-07-24 | 2017-07-24 | CIM (common information model) -based multi-level caching method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710606890.4A CN107562782B (en) | 2017-07-24 | 2017-07-24 | CIM (common information model) -based multi-level caching method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107562782A CN107562782A (en) | 2018-01-09 |
| CN107562782B true CN107562782B (en) | 2020-02-04 |
Family
ID=60974172
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710606890.4A Active CN107562782B (en) | 2017-07-24 | 2017-07-24 | CIM (common information model) -based multi-level caching method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107562782B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117130792B (en) * | 2023-10-26 | 2024-02-20 | 腾讯科技（深圳）有限公司 | Processing method, device, equipment and storage medium for cache object |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102722432A (en) * | 2011-03-29 | 2012-10-10 | 国际商业机器公司 | Method and apparatus for tracing memory access |
| US8365164B1 (en) * | 2008-02-21 | 2013-01-29 | T-APP Ltd. | Portable software applications |
| CN105790259A (en) * | 2016-03-15 | 2016-07-20 | 华南理工大学 | CIM based power grid power flow data conversion method |
| CN106657282A (en) * | 2016-11-25 | 2017-05-10 | 许继电气股份有限公司 | Integration method and device for equipment operation state information of convertor station |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8495018B2 (en) * | 2011-06-24 | 2013-07-23 | International Business Machines Corporation | Transitioning application replication configurations in a networked computing environment |
-
2017
- 2017-07-24 CN CN201710606890.4A patent/CN107562782B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8365164B1 (en) * | 2008-02-21 | 2013-01-29 | T-APP Ltd. | Portable software applications |
| CN102722432A (en) * | 2011-03-29 | 2012-10-10 | 国际商业机器公司 | Method and apparatus for tracing memory access |
| CN105790259A (en) * | 2016-03-15 | 2016-07-20 | 华南理工大学 | CIM based power grid power flow data conversion method |
| CN106657282A (en) * | 2016-11-25 | 2017-05-10 | 许继电气股份有限公司 | Integration method and device for equipment operation state information of convertor station |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107562782A (en) | 2018-01-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8825721B2 (en) | Time-based object aging for generational garbage collectors | |
| Adya et al. | Efficient optimistic concurrency control using loosely synchronized clocks | |
| US7761487B2 (en) | Predicting out of memory conditions using soft references | |
| WO2018072408A1 (en) | Memory analysis method, device, system and computing device | |
| US20170147628A1 (en) | Transactional cache invalidation for inter-node caching | |
| US20130086131A1 (en) | Time-based object aging for generational garbage collectors | |
| CN111447102B (en) | SDN network device access method and device, computer device and storage medium | |
| US20080270482A1 (en) | Managing object lifetime for native/managed peers | |
| CN106021468B (en) | The update method and system of distributed caching and local cache | |
| CN106951179B (en) | Data migration method and device | |
| CN107153643B (en) | Data table connection method and device | |
| US10037269B2 (en) | Managing memory in a computer system | |
| CN110134738A (en) | Distributed memory system resource predictor method, device | |
| CN104917813A (en) | Resource request method and device | |
| CN107562782B (en) | CIM (common information model) -based multi-level caching method, device and system | |
| CN113157411A (en) | Reliable configurable task system and device based on Celery | |
| CN110716924B (en) | Method and device for deleting expired data | |
| US11216352B2 (en) | Method for automatically analyzing bottleneck in real time and an apparatus for performing the method | |
| CN116881288A (en) | Data query method, device and equipment | |
| US10824597B2 (en) | Cache folder identification method and device | |
| CN115408342A (en) | File processing method and device and electronic equipment | |
| CN114168594A (en) | Secondary index creating method, device, equipment and storage medium of horizontal partition table | |
| CN113672248A (en) | Patch acquisition method, device, server and storage medium | |
| CN117009439B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN107908470A (en) | A kind of queried access method, apparatus and system based on CIM model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20191218 Address after: 510080, Guangdong, Guangzhou province Yuexiu District Dongfeng East Road water all Gang No. 8, No. eighteen, building 6 Applicant after: GUANGDONG POWER GRID CO., LTD. INFORMATION CENTER Address before: 510060 building 6, No. 8, No. eighteen, Dongfeng Road, Dongfeng East Road, Guangzhou, Yuexiu District, Guangdong Applicant before: GUANGDONG POWER GRID CO., LTD. INFORMATION CENTER Applicant before: WeiHai CIMSTech Co., Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |