CN112148554A - Method and device for calculating occupation size of redis service data in real time - Google Patents
Method and device for calculating occupation size of redis service data in real time Download PDFInfo
- Publication number
- CN112148554A CN112148554A CN202010959350.6A CN202010959350A CN112148554A CN 112148554 A CN112148554 A CN 112148554A CN 202010959350 A CN202010959350 A CN 202010959350A CN 112148554 A CN112148554 A CN 112148554A
- Authority
- CN
- China
- Prior art keywords
- redis
- keywords
- data
- service
- keyword
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
Abstract
The invention relates to a method and a device for calculating the occupation size of redis service data in real time, which comprises the steps of establishing access to the redis service data connection; configuring keywords; obtaining the redis data in batches through the redis service, and obtaining a key value according to the redis data; separating and calculating redis data according to the type of the key value and the keyword, and caching the result of the size of the redis data corresponding to the calculated keyword; and after traversing the keywords, converting the size result of the redis data corresponding to the cached keywords into a unit and outputting the unit. The invention can know the occupation size proportion of each service and the sub-service in the redis service in real time and know the occupation size of each service in the redis service in real time, thereby leading related personnel to make a strategy in time along with the increase of the service volume and improving the working efficiency.
Description
Technical Field
The invention belongs to the technical field of data processing, and particularly relates to a method and a device for calculating the occupation size of redis service data in real time.
Background
Redis (remote Dictionary Server), a remote Dictionary service, is an open source log-type and Key-Value database written in ANSI C language, supporting network, based on memory and persistent, and provides API of multiple languages. The Redis is a high-performance open-source key-value database, has high read-write performance, can store mapping between keys (keys) and 5 different types of values (values), and can persist key value pair data stored in a memory to a hard disk, and a user can configure different persistence modes according to actual use scenes.
The redis caching technology is widely applied to each architecture system, and as the service volume of each subsystem rises, the redis service caching data also increases. The existing redis operation and maintenance technology cannot meet the requirements of personnel such as development teams and operation and maintenance teams in time, and can know the size proportion of each business and sub-business occupied in the redis service in real time. Because the occupation size of each service in the redis service cannot be known in real time, related personnel cannot timely make a response strategy along with the increase of the service volume, and the working efficiency is reduced.
Disclosure of Invention
In view of the above, the present invention provides a method and an apparatus for calculating an occupied size of redis service data in real time to solve the problem that as the traffic volume increases, the solution cannot be made in time, and the working efficiency is reduced.
In order to achieve the purpose, the invention adopts the following technical scheme: a method for calculating the occupation size of the redis service data in real time comprises the following steps:
establishing access to a redis service data connection;
configuring keywords;
judging whether the keywords are acquired or not, if so, acquiring the redis data in batches through the redis service, and acquiring a key value according to the redis data; otherwise, directly calculating all data in the redis service and caching;
separating the redis data according to the type of the key value and the keyword, calculating, and caching the redis data size result corresponding to the calculated keyword;
and after traversing the keywords, converting the size result of the redis data corresponding to the cached keywords into a unit and outputting the unit.
Further, the keywords include sub-service keywords and specified keywords.
Further, redis data was obtained in batch-specific quantities using keys, scan, and hscan.
Further, the types of the key value include:
hash value, value.
Further, separating and calculating the redis data according to the type and the keyword of the key value, including:
judging the type of the key value, and if the key value is the hash value, acquiring the redis data again and judging;
if the key value is a value, judging that the keyword is a sub-service keyword or an appointed keyword;
if the keyword is an instruction keyword, calculating the size of the redis data according to a preset rule;
and if the keyword is a sub-service keyword, calculating the size of a key, wherein the key corresponds to the size of the field, and the key corresponds to the size of the value of the field.
Further, caching the redis data size result corresponding to the calculated keyword includes:
caching the redis data size result, the sub-service keywords and the appointed keywords corresponding to each batch of calculated keywords to the map.
Further, the converting and outputting the redis data size result corresponding to the cached keyword by a unit includes:
converting the size result of the redis data corresponding to the keywords cached in the map into KB, MB and GB, and outputting the redis data to occupy the size of the redis service.
Further, the keyword includes a combination of a plurality of or more keywords.
The embodiment of the present application provides a device for calculating an occupation size of redis service data in real time, including:
the establishing module is used for establishing access to the redis service data connection;
the configuration module is used for configuring the keywords;
the acquisition module is used for acquiring the redis data in batches through the redis service and acquiring a key value according to the redis data;
the computing module is used for separating and computing the redis data according to the type of the key value and the keywords and caching the redis data size result corresponding to the computed keywords;
and the output module is used for converting the redis data size result corresponding to the cached keyword into a unit and outputting the unit after traversing the keyword.
Further, the configuration module includes:
and the keyword configuration unit is used for configuring the sub-service keywords and the specified keywords.
By adopting the technical scheme, the invention can achieve the following beneficial effects:
the invention provides a method for calculating the occupation size of redis service data in real time, which comprises the steps of establishing access to the redis service data connection; configuring keywords; obtaining the redis data in batches through the redis service, and obtaining a key value according to the redis data; separating and calculating redis data according to the type of the key value and the keyword, and caching the result of the size of the redis data corresponding to the calculated keyword; and after traversing the keywords, converting the size result of the redis data corresponding to the cached keywords into a unit and outputting the unit. The invention can know the occupation size proportion of each service and the sub-service in the redis service in real time and know the occupation size of each service in the redis service in real time, thereby leading related personnel to make a strategy in time along with the increase of the service volume and improving the working efficiency.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram illustrating steps of a method for calculating real-time occupation size of redis service data according to the present invention;
FIG. 2 is a flowchart illustrating a method for calculating real-time occupation size of redis service data according to the present invention;
fig. 3 is a schematic structural diagram of an apparatus for calculating an occupied size of redis service data in real time according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be described in detail below. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the examples given herein without any inventive step, are within the scope of the present invention.
A specific method and apparatus for calculating an occupation size of a redis service data in real time according to an embodiment of the present application will be described with reference to the accompanying drawings.
As shown in fig. 1, the method for calculating an occupation size of redis service data in real time provided in the embodiment of the present application includes:
s101, establishing access to redis service data connection;
and establishing an access redis service data connection by utilizing the redis service api.
S102, configuring keywords;
and configuring keywords in advance, wherein the keywords comprise sub-service keywords and specified keywords. The keywords are used as the calculation basis of the sub-service data in the differentiated service scene, such as the sub-service keywords ProductSeconddType, and the attribute values of the keywords do not need to be concerned; the specified keywords can also be configured, the keywords of the type are calculated independently, and when the redis service occupies the size, such as the key is BusiUpdateMsg, the BusiUpdateMsg is calculated in a fuzzy mode, and the key type is not needed to be worried about.
The keyword may be a combination of a plurality of keywords.
S103, judging whether the keywords are acquired or not, if so, acquiring the redis data in batches through the redis service, and acquiring a key value according to the redis data; otherwise, directly calculating all data in the redis service and caching;
and acquiring the calculated size data according to the specified quantity of the batch by using keys, scan and hscan, and acquiring by using a circulation mode.
S104, separating and calculating the redis data according to the type of the key value and the keyword, and caching the redis data size result corresponding to the calculated keyword;
the types of the key value include: hash value, value.
And S105, after traversing the keywords, converting the redis data size result corresponding to the cached keywords into a unit and outputting the unit.
The working principle of the method for calculating the occupation size of the redis service data in real time is as follows: as shown in fig. 2, a redis service api is utilized to establish access to a redis service data connection, configure a keyword, determine whether the keyword is obtained, if yes, obtain the redis data in batches through the redis service, and obtain a key value according to the redis data; and calculating the size of a key according to the keywords and the value, wherein the key corresponds to the field size, the key corresponds to the value of the field size, after traversing the first keyword, obtaining the next keyword, calculating the size of the redis data again until all keywords are traversed, and caching the redis data size result corresponding to the calculated keyword. When no key is configured, the redis service all data is calculated.
In some embodiments, separating and calculating the redis data according to the type of the key value and the keyword includes:
judging the type of the key value, and if the key value is the hash value, acquiring the redis data again and judging;
if the key value is a value, judging that the keyword is a sub-service keyword or an appointed keyword;
if the keyword is an instruction keyword, calculating the size of the redis data according to a preset rule;
and if the keyword is a sub-service keyword, calculating the size of a key, wherein the key corresponds to the size of the field, and the key corresponds to the size of the value of the field.
Preferably, the caching the redis data size result corresponding to the calculated keyword includes:
caching the redis data size result, the sub-service keywords and the appointed keywords corresponding to each batch of calculated keywords to the map.
Preferably, the converting and outputting the redis data size result corresponding to the cached keyword includes:
caching the redis data size result, the sub-service keywords and the appointed keywords corresponding to each batch of calculated keywords to the map.
Specifically, redis data with calculated size is obtained by utilizing keys, scan and hscan according to the specified amount of batches, the types of the keys are separated according to each key, the key size is accumulatively calculated, the key corresponds to the field size, the key corresponds to the value size of the field, the key is stored in a map [ key ] [ size ] cache, the map [ key ] [ size ] is traversed, size units (KB, MB and GB) are converted and calculated, and the key business data are output to occupy the redis service size.
As shown in fig. 3, an embodiment of the present application provides an apparatus for calculating an occupation size of redis service data in real time, including:
an establishing module 301, configured to establish access to a redis service data connection;
a configuration module 302 for configuring keywords;
the obtaining module 303 is configured to obtain the redis data in batches through the redis service, and obtain a key value according to the redis data;
the calculating module 304 is configured to separate and calculate the redis data according to the type of the key value and the keyword, and cache a redis data size result corresponding to the calculated keyword;
and the output module 305 is configured to convert the redis data size result corresponding to the cached keyword into a unit and output the unit after traversing the keyword.
The working principle of the device for calculating the occupation size of the redis service data in real time provided by the application is that the establishing module 301 establishes access to the redis service data connection; the configuration module 302 configures the keywords; the obtaining module 303 obtains the redis data in batches through the redis service, and obtains a key value according to the redis data; the calculation module 304 separates and calculates the redis data according to the type of the key value and the keyword, and caches a redis data size result corresponding to the calculated keyword; after traversing the keywords, the output module 305 converts the redis data size result corresponding to the cached keywords into units and outputs the units.
The embodiment of the application provides computer equipment, which comprises a processor and a memory connected with the processor;
the memory is used for storing a computer program, and the computer program is used for executing the method for calculating the occupation size of the redis service data in real time provided by any one of the above embodiments;
the processor is used to call and execute the computer program in the memory.
In summary, the present invention provides a method and an apparatus for calculating the size occupied by the redis service data in real time, including establishing access to the redis service data connection; configuring keywords; obtaining the redis data in batches through the redis service, and obtaining a key value according to the redis data; separating and calculating redis data according to the type of the key value and the keyword, and caching the result of the size of the redis data corresponding to the calculated keyword; and after traversing the keywords, converting the size result of the redis data corresponding to the cached keywords into a unit and outputting the unit. The invention can know the occupation size proportion of each service and the sub-service in the redis service in real time and know the occupation size of each service in the redis service in real time, thereby leading related personnel to make a strategy in time along with the increase of the service volume and improving the working efficiency.
It is to be understood that the embodiments of the method provided above correspond to the embodiments of the apparatus described above, and the corresponding specific contents may be referred to each other, which is not described herein again.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (10)
1. A method for calculating the occupation size of the redis service data in real time is characterized by comprising the following steps:
establishing access to a redis service data connection;
configuring keywords;
judging whether the keywords are acquired or not, if so, acquiring the redis data in batches through the redis service, and acquiring a key value according to the redis data; otherwise, directly calculating all data in the redis service and caching;
separating the redis data according to the type of the key value and the keyword, calculating, and caching the redis data size result corresponding to the calculated keyword;
and after traversing the keywords, converting the size result of the redis data corresponding to the cached keywords into a unit and outputting the unit.
2. The method of claim 1,
the keywords include sub-business keywords and specified keywords.
3. The method of claim 2,
redis data was obtained using keys, scan and hscan at the batch specified volumes.
4. The method of claim 3, wherein the type of key value comprises:
hash value, value.
5. The method of claim 4, wherein separating and calculating the redis data according to the type of the key value and the keyword comprises:
judging the type of the key value, and if the key value is the hash value, acquiring the redis data again and judging;
if the key value is a value, judging that the keyword is a sub-service keyword or an appointed keyword;
if the keyword is an instruction keyword, calculating the size of the redis data according to a preset rule;
and if the keyword is a sub-service keyword, calculating the size of a key, wherein the key corresponds to the size of the field, and the key corresponds to the size of the value of the field.
6. The method according to claim 1, wherein caching the redis data size result corresponding to the calculated keyword comprises:
caching the redis data size result, the sub-service keywords and the appointed keywords corresponding to each batch of calculated keywords to the map.
7. The method according to claim 6, wherein converting and outputting the redis data size result corresponding to the cached key into a unit comprises:
converting the size result of the redis data corresponding to the keywords cached in the map into KB, MB and GB, and outputting the redis data to occupy the size of the redis service.
8. The method of claim 1,
the keyword includes a combination of a plurality of keywords and/or a plurality of sub-business keywords.
9. An apparatus for calculating an occupation size of a redis service data in real time, comprising:
the establishing module is used for establishing access to the redis service data connection;
the configuration module is used for configuring the keywords;
the acquisition module is used for acquiring the redis data in batches through the redis service and acquiring a key value according to the redis data;
the computing module is used for separating and computing the redis data according to the type of the key value and the keywords and caching the redis data size result corresponding to the computed keywords;
and the output module is used for converting the redis data size result corresponding to the cached keyword into a unit and outputting the unit after traversing the keyword.
10. The apparatus of claim 9, wherein the configuration module comprises:
and the keyword configuration unit is used for configuring the sub-service keywords and the specified keywords.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010959350.6A CN112148554A (en) | 2020-09-14 | 2020-09-14 | Method and device for calculating occupation size of redis service data in real time |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010959350.6A CN112148554A (en) | 2020-09-14 | 2020-09-14 | Method and device for calculating occupation size of redis service data in real time |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112148554A true CN112148554A (en) | 2020-12-29 |
Family
ID=73892233
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010959350.6A Pending CN112148554A (en) | 2020-09-14 | 2020-09-14 | Method and device for calculating occupation size of redis service data in real time |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112148554A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112948404A (en) * | 2021-02-09 | 2021-06-11 | 国电南瑞科技股份有限公司 | System multi-source monitoring data classification modeling and circulation method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107357794A (en) * | 2016-05-10 | 2017-11-17 | 北京京东尚科信息技术有限公司 | Optimize the method and apparatus of the data store organisation of key value database |
| CN108280031A (en) * | 2017-12-22 | 2018-07-13 | 努比亚技术有限公司 | Redis cache cleaner method, server and computer readable storage medium |
| CN110083506A (en) * | 2019-04-16 | 2019-08-02 | 上海易点时空网络有限公司 | The method and device of cluster resource amount optimization |
| CN110737681A (en) * | 2019-10-15 | 2020-01-31 | 深圳前海环融联易信息科技服务有限公司 | Locking processing method and device for processing request, computer equipment and storage medium |
-
2020
- 2020-09-14 CN CN202010959350.6A patent/CN112148554A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107357794A (en) * | 2016-05-10 | 2017-11-17 | 北京京东尚科信息技术有限公司 | Optimize the method and apparatus of the data store organisation of key value database |
| CN108280031A (en) * | 2017-12-22 | 2018-07-13 | 努比亚技术有限公司 | Redis cache cleaner method, server and computer readable storage medium |
| CN110083506A (en) * | 2019-04-16 | 2019-08-02 | 上海易点时空网络有限公司 | The method and device of cluster resource amount optimization |
| CN110737681A (en) * | 2019-10-15 | 2020-01-31 | 深圳前海环融联易信息科技服务有限公司 | Locking processing method and device for processing request, computer equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 科创人: "redis数据量暴涨进行数据清理实战", pages 1 - 4, Retrieved from the Internet <URL:https://zhuanlan.zhihu.com/p/62009307> * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112948404A (en) * | 2021-02-09 | 2021-06-11 | 国电南瑞科技股份有限公司 | System multi-source monitoring data classification modeling and circulation method and system |
| CN112948404B (en) * | 2021-02-09 | 2022-11-15 | 国电南瑞科技股份有限公司 | Multi-source monitoring data classification modeling and transferring method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111177178B (en) | Data processing method and related equipment | |
| CN107038222B (en) | Database cache implementation method and system | |
| CN109656958B (en) | Data query method and system | |
| CN104639658A (en) | Realization method for accessing object storage by file system mounting | |
| CN106484709A (en) | A kind of auditing method of daily record data and audit device | |
| CN111586091B (en) | Edge computing gateway system for realizing computing power assembly | |
| CN110399268B (en) | Abnormal data detection method, device and equipment | |
| CN109739919A (en) | A kind of front end processor and acquisition system for electric system | |
| WO2020177536A1 (en) | Transaction deduplication method, transaction construction method, device and storage medium | |
| CN106844545A (en) | A kind of implementation method of the Database Systems with double engines based on stsndard SQL | |
| CN112860695A (en) | Monitoring data query method, device, equipment, storage medium and program product | |
| CN103716375A (en) | Distributed cache-based multistage financial data interaction method and system | |
| CN106502875A (en) | A kind of daily record generation method and system based on cloud computing | |
| CN108509453B (en) | Information processing method and device | |
| CN106649652A (en) | Construction method for on-line transformer substation main equipment monitoring system based on Hadoop | |
| CN108573029A (en) | A kind of method, apparatus and storage medium obtaining network access relational data | |
| CN105138686B (en) | A kind of instant application process for multistage storage data | |
| KR20220129085A (en) | Statistics Gathering Offloading | |
| CN109582649A (en) | A kind of metadata storing method, device, equipment and readable storage medium storing program for executing | |
| CN105095515A (en) | Bucket dividing method, device and equipment supporting fast query of Map-Reduce output result | |
| CN112148554A (en) | Method and device for calculating occupation size of redis service data in real time | |
| US11693858B2 (en) | Access path optimization | |
| CN111736907B (en) | Data analysis method of self-adaptive low-delay memory computing engine | |
| CN108628678A (en) | The determination method, apparatus and equipment of memory parameters | |
| CN110990472A (en) | Data deriving method and device based on Hbase |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |