CN115174289B - Log data storage method and device, storage medium and electronic device - Google Patents
Log data storage method and device, storage medium and electronic device Download PDFInfo
- Publication number
- CN115174289B CN115174289B CN202210571469.5A CN202210571469A CN115174289B CN 115174289 B CN115174289 B CN 115174289B CN 202210571469 A CN202210571469 A CN 202210571469A CN 115174289 B CN115174289 B CN 115174289B
- Authority
- CN
- China
- Prior art keywords
- log
- target
- stored
- interface
- amount
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000013500 data storage Methods 0.000 title description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000001514 detection method Methods 0.000 claims description 6
- 230000007423 decrease Effects 0.000 claims description 4
- 230000000694 effects Effects 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 9
- 230000003287 optical effect Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000005406 washing Methods 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000010411 cooking Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- 238000010408 sweeping Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/2803—Home automation networks
- H04L12/2823—Reporting information sensed by appliance or service execution status of appliance services in a home automation network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/215—Flow control; Congestion control using token-bucket
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/50—Queue scheduling
- H04L47/62—Queue scheduling characterised by scheduling criteria
- H04L47/625—Queue scheduling characterised by scheduling criteria for service slots or service orders
- H04L47/6275—Queue scheduling characterised by scheduling criteria for service slots or service orders based on priority
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Automation & Control Theory (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application discloses a method and a device for storing log data, a storage medium and an electronic device, and relates to the technical field of smart families, wherein the method for storing the log data comprises the following steps: obtaining target log parameters corresponding to target logs to be stored, wherein the target log parameters are used for indicating the stored log quantity of a target log interface for outputting the target logs in the terminal equipment; detecting a log storage state of the terminal equipment under the condition that the target log parameter meets the target parameter condition, wherein the log parameter meets the parameter condition and is used for indicating that the stored log quantity of the log interface exceeds the allowed log quantity stored by the log interface, and the log storage state is used for indicating the log storage condition of the terminal equipment; and storing the target log according to the log storage state. The method and the device solve the problems of more terminal equipment performance and the like consumed by outputting log data in the related technology, and realize the technical effect of reducing the terminal equipment performance consumed by outputting the log data.
Description
Technical Field
The application relates to the technical field of smart families, in particular to a method and a device for storing log data, a storage medium and an electronic device.
Background
In the prior art, a log component is used as a basic component for outputting and storing logs, so that a great number of callers are required, the calls are frequent, and the output quantity of the logs is often very large. Frequent log output may result in large local storage files, occupying more storage space of the terminal system; and frequent log output may result in a significant consumption of terminal device performance, which in turn may result in application churning. Therefore, the current log output technology cannot reasonably regulate and control the log output by the interface.
Aiming at the problems of more terminal equipment performance and the like consumed by outputting log data in the related art, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the application provides a method and a device for storing log data, a storage medium and an electronic device, which are used for at least solving the problems of more terminal equipment performance and the like consumed by outputting the log data in the related technology.
According to an embodiment of the present application, there is provided a method for storing log data, including: obtaining target log parameters corresponding to target logs to be stored, wherein the target log parameters are used for indicating the stored log quantity of a target log interface for outputting the target logs in terminal equipment;
Detecting a log storage state of the terminal equipment under the condition that the target log parameter meets a target parameter condition, wherein the log parameter meets the parameter condition and is used for indicating that the stored log quantity of a log interface exceeds the allowed log quantity stored by the log interface, and the log storage state is used for indicating the log storage condition of the terminal equipment;
and storing the target log according to the log storage state.
Optionally, the detecting the log storage state of the terminal device includes:
detecting a target log amount which is allowed to be stored currently by the terminal equipment;
and determining the log storage state according to the target log quantity.
Optionally, the detecting the target log amount that the terminal device is allowed to store currently includes one of the following:
acquiring a first log amount and a second log amount of the terminal equipment, wherein the first log amount is the total log amount allowed to be stored by the terminal equipment, and the second log amount is the current stored log amount of the terminal equipment; determining a difference between the first log amount and a second log amount as the target log amount;
and reading the remaining number of tokens from a token bucket as the target log amount, wherein the number of tokens in the token bucket gradually decreases from an initial value according to the number of logs stored by the terminal equipment.
Optionally, before the reading the remaining number of tokens from the token bucket as the target log amount, the method further includes: obtaining a product of a first proportional coefficient and a first log quantity as the initial value, wherein the first log quantity is the total log quantity allowed to be stored by the terminal equipment; generating tokens of the initial value in the token bucket;
after the obtaining the product of the first scale factor and the first log as the initial value, the method further includes: and generating tokens meeting the product value of a second proportionality coefficient and the first log quantity in the token bucket every other target time period until the generated token quantity meets the first log quantity.
Optionally, the determining the log storage state according to the target log amount includes:
determining that the log storage state is a first state when the target log amount is greater than or equal to a first threshold value, wherein the first state is used for indicating that the terminal equipment is allowed to continue to store the log output by the target log interface, and storing the target log when the log storage state is the first state;
And under the condition that the target log quantity is smaller than a first threshold value, determining the log storage state to be a second state, wherein the second state is used for indicating that the terminal equipment is not allowed to continue to store the log output by the target log interface.
Optionally, the obtaining the target log parameter corresponding to the target log to be stored includes:
analyzing a target interface identifier from the log identifier of the target log, wherein the interface identifier is used for indicating the target log interface;
and acquiring a third log quantity recorded for the target log interface as the target log parameter, and acquiring a second threshold value corresponding to the target interface identifier from the interface identifier and the threshold value with corresponding relation, wherein the third log quantity is the stored log quantity of the target log interface, and the target log parameter meets a target parameter condition under the condition that the third log quantity is larger than or equal to the second threshold value.
Optionally, the obtaining the target log parameter corresponding to the target log to be stored includes:
analyzing a target interface identifier from the log identifier of the target log, wherein the interface identifier is used for indicating the target log interface;
The method comprises the steps of obtaining a log quantity identifier distributed for a target log interface as a target log parameter, and obtaining a target log quantity identifier, wherein the log quantity identifier is used for marking whether the log quantity stored for the target log interface exceeds the log quantity allowed to be stored for the target log interface, the target log quantity identifier is used for marking that the log quantity stored for the target log interface exceeds the log quantity allowed to be stored for the target log interface, and the target log parameter meets a target parameter condition under the condition that the log quantity identifier is the target log identifier.
According to another embodiment of the present application, there is also provided a storage device for log data, including: the acquisition module is used for acquiring target log parameters corresponding to target logs to be stored, wherein the target log parameters are used for indicating the log quantity stored in a target log interface for outputting the target logs in the terminal equipment;
the detection module is used for detecting the log storage state of the terminal equipment under the condition that the target log parameter meets the target parameter condition, wherein the log parameter meets the parameter condition and is used for indicating that the log quantity stored by a log interface exceeds the log quantity allowed to be stored by the log interface, and the log storage state is used for indicating the log storage condition of the terminal equipment;
And the storage module is used for storing the target log according to the log storage state.
According to still another aspect of the embodiments of the present application, there is also provided a computer-readable storage medium having a computer program stored therein, wherein the computer program is configured to perform the above-described log data storage method when run.
According to still another aspect of the embodiments of the present application, there is further provided an electronic device including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the method for storing log data described above through the computer program.
In the embodiment of the application, the target log parameters corresponding to the target log to be stored are obtained, wherein the target log parameters are used for indicating the stored log quantity of a target log interface for outputting the target log in the terminal equipment; detecting a log storage state of the terminal equipment under the condition that the target log parameter meets the target parameter condition, wherein the log parameter meets the parameter condition and is used for indicating that the stored log quantity of the log interface exceeds the allowed log quantity stored by the log interface, and the log storage state is used for indicating the log storage condition of the terminal equipment; according to the log storage state, the target log is stored, namely, the target log parameter for indicating the stored log quantity of the target log interface for outputting the target log in the terminal equipment is obtained, if the target log parameter meets the target parameter condition, the stored log quantity of the target log interface can be indicated to exceed the allowed log quantity stored by the target log interface, namely, the expected log quantity which can be stored for the target log interface is stored for the target log interface, in this case, the log storage state of the terminal equipment is detected, and the log storage of the target log interface is controlled according to the log storage condition of the terminal equipment under the condition that the stored log quantity of the log interface exceeds the allowed log quantity stored by the log interface. By adopting the technical scheme, the problems of more terminal equipment performance consumed by outputting log data and the like in the related technology are solved, and the technical effect of reducing the terminal equipment performance consumed by outputting the log data is realized.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the application or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, and it will be obvious to a person skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a schematic diagram of a hardware environment of a method for storing log data according to an embodiment of the present application;
FIG. 2 is a flow chart of a method of storing log data according to an embodiment of the application;
FIG. 3 is a schematic diagram of invoking tokens in a token bucket according to an embodiment of the present application;
FIG. 4 is a timing diagram for generating tokens in a token bucket according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a method of storing log data according to an embodiment of the present application;
FIG. 6 is a schematic diagram of an exporter priority and token count process remaining in a token bucket according to an embodiment of the present application;
FIG. 7 is a schematic diagram of updating priorities of exporters according to an embodiment of the present application;
fig. 8 is a block diagram of a log data storage device according to an embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
According to an aspect of an embodiment of the present application, there is provided a method for storing log data. The log data storage method is widely applied to full-house intelligent digital control application scenes such as intelligent Home (Smart Home), intelligent Home equipment ecology, intelligent Home (Intelligence House) ecology and the like. Alternatively, in the present embodiment, the above-described method of storing log data may be applied to a hardware environment constituted by the terminal device 102 and the server 104 as shown in fig. 1. As shown in fig. 1, the server 104 is connected to the terminal device 102 through a network, and may be used to provide services (such as application services and the like) for a terminal or a client installed on the terminal, a database may be set on the server or independent of the server, for providing data storage services for the server 104, and cloud computing and/or edge computing services may be configured on the server or independent of the server, for providing data computing services for the server 104.
The network may include, but is not limited to, at least one of: wired network, wireless network. The wired network may include, but is not limited to, at least one of: a wide area network, a metropolitan area network, a local area network, and the wireless network may include, but is not limited to, at least one of: WIFI (Wireless Fidelity ), bluetooth. The terminal device 102 may not be limited to a PC, a mobile phone, a tablet computer, an intelligent air conditioner, an intelligent smoke machine, an intelligent refrigerator, an intelligent oven, an intelligent cooking range, an intelligent washing machine, an intelligent water heater, an intelligent washing device, an intelligent dish washer, an intelligent projection device, an intelligent television, an intelligent clothes hanger, an intelligent curtain, an intelligent video, an intelligent socket, an intelligent sound box, an intelligent fresh air device, an intelligent kitchen and toilet device, an intelligent bathroom device, an intelligent sweeping robot, an intelligent window cleaning robot, an intelligent mopping robot, an intelligent air purifying device, an intelligent steam box, an intelligent microwave oven, an intelligent kitchen appliance, an intelligent purifier, an intelligent water dispenser, an intelligent door lock, and the like.
In this embodiment, a method for storing log data is provided and applied to the terminal device, and fig. 2 is a flowchart of a method for storing log data according to an embodiment of the present application, as shown in fig. 2, where the flowchart includes the following steps:
step S202, obtaining target log parameters corresponding to target logs to be stored, wherein the target log parameters are used for indicating the stored log quantity of a target log interface for outputting the target logs in terminal equipment;
step S204, detecting a log storage state of the terminal equipment under the condition that the target log parameter meets a target parameter condition, wherein the log parameter meets the parameter condition and is used for indicating that the stored log quantity of a log interface exceeds the allowed log quantity of the log interface, and the log storage state is used for indicating the log storage condition of the terminal equipment;
and step S206, storing the target log according to the log storage state.
By the above steps, the target log parameter indicating the log amount stored by the target log interface for outputting the target log in the terminal device is obtained, and if the target log parameter meets the target parameter condition, it can be indicated that the log amount stored by the target log interface exceeds the log amount allowed to be stored by the target log interface, that is, the log amount expected to be stored by the target log interface is already stored by the target log interface, in this case, the log storage state of the terminal device is detected, and the log storage of the target log interface is controlled according to the log storage condition of the terminal device in the case that the log amount stored by the log interface exceeds the log amount allowed to be stored by the log interface. By adopting the technical scheme, the problems of more terminal equipment performance consumed by outputting log data and the like in the related technology are solved, and the technical effect of reducing the terminal equipment performance consumed by outputting the log data is realized.
In the technical solution provided in step S202, the log to be stored output by each log interface may be, but is not limited to, placed in a serial queue, and the first log to be stored in the serial queue may be, but is not limited to, obtained as a target log, and the amount of log stored in the terminal device for the target log interface outputting the target log may be obtained as a target log parameter. Through serializing the log interface of the log to be stored, the log output devices in different threads do not need to be synchronously locked for protection, and meanwhile, the method is not limited by a mode of setting a threshold value of the number of the received logs of the serial queue, so that the system performance resources are prevented from being exhausted due to the fact that the log quantity is too large at a certain time point, the smooth operation of the terminal equipment is ensured, and the user experience is improved.
Alternatively, in this embodiment, the log may be stored for a plurality of log interfaces in a terminal device, which may include, but is not limited to, a mobile terminal device (e.g., a cell phone, a tablet, etc.), a personal computer (Personal Computer), a server that needs to store the log, and so forth.
Alternatively, in this embodiment, the target log interface may be, but is not limited to, an interface for outputting a log, which is located in the terminal device and connected to a storage space in the terminal device, and may be, but is not limited to, outputting a log corresponding to a function module provided in the terminal device, and in the case where the function module in the terminal device outputs a log, the target log interface may be, but is not limited to, receiving the log output by the function module, and storing the received log output by the function module in the storage space of the terminal device.
In one exemplary example, the target log parameters may be obtained, but are not limited to, by: analyzing a target interface identifier from the log identifier of the target log, wherein the interface identifier is used for indicating the target log interface; and acquiring a third log quantity recorded for the target log interface as the target log parameter, and acquiring a second threshold value corresponding to the target interface identifier from the interface identifier and the threshold value with corresponding relation, wherein the third log quantity is the stored log quantity of the target log interface, and the target log parameter meets a target parameter condition under the condition that the third log quantity is larger than or equal to the second threshold value.
Optionally, in this embodiment, the log number or the log data amount stored for the target log interface outputting the target log may be used as the log identifier of the target log, and by recording the log amount stored for each log interface by the terminal device, the use condition of acquiring the performance resource of the terminal device is clearly implemented, so that the performance resource of the terminal device is more reasonably allocated.
Optionally, in this embodiment, the second threshold may be, but is not limited to, a log amount threshold for indicating that the terminal device is allowed to store for the target log interface, each log interface may be, but is not limited to, corresponding to a different log amount threshold, and/or the same log amount threshold allowed to store for the log interface, and the target log parameter may be, but is not limited to, satisfying the target parameter condition if the log amount that the target log interface has stored is greater than or equal to the second threshold.
In one exemplary example, the target log parameters may be obtained, but are not limited to, by: analyzing a target interface identifier from the log identifier of the target log, wherein the interface identifier is used for indicating the target log interface; the method comprises the steps of obtaining a log quantity identifier distributed for a target log interface as a target log parameter, and obtaining a target log quantity identifier, wherein the log quantity identifier is used for marking whether the log quantity stored for the target log interface exceeds the log quantity allowed to be stored for the target log interface, the target log quantity identifier is used for marking that the log quantity stored for the target log interface exceeds the log quantity allowed to be stored for the target log interface, and the target log parameter meets a target parameter condition under the condition that the log quantity identifier is the target log identifier.
Optionally, in this embodiment, the log quantity identifier may, but is not limited to, include a storage priority level identifier of the target log interface, and the storage priority level identifier may, but is not limited to, include High (High), LOW (LOW), and the like, and may, but is not limited to, allocate the storage priority level identifier of the target log interface to LOW if the log quantity stored for the target log interface is greater than or equal to the log quantity allowed to be stored for the target log interface for the first time, without recording a specific number or a specific data quantity of the log quantity stored for the target log interface, and save, by means of the interface identifier, a storage space of the terminal device required to record the stored log information; the storage priority identification of the target log interface may be assigned High, but is not limited to, in the case where the amount of log stored by the target log interface is less than the amount of log permitted to be stored by the target log interface.
In the technical solution provided in step S204, the log storage state of the terminal device may be detected, but not limited to, when the log amount stored in the log interface is greater than or equal to the log amount allowed to be stored in the log interface by the terminal device, where the log storage state may include, but is not limited to, the number of logs, the log data amount, and the like that the current remaining performance resource of the terminal device may support to store the log output by the log interface, so that performance of the terminal device that considers storing the log is achieved, and reasonability of storing the log output by each log interface is improved.
Optionally, in this embodiment, the target log parameter failing to meet the target parameter condition may be, but is not limited to, used to indicate that the log amount stored by the log interface is smaller than the log amount allowed to be stored by the terminal device for the log interface, where the target log is directly stored in the terminal device without detecting the log storage state of the terminal device.
In one exemplary instance, log storage status may be detected, but is not limited to, by: detecting a target log amount which is allowed to be stored currently by the terminal equipment; and determining the log storage state according to the target log quantity.
Optionally, in this embodiment, the target amount of log allowed to be stored currently by the terminal device may be, but is not limited to, an amount of performance resources required for indicating that the terminal device is currently used to store logs of each log interface, so as to comprehensively consider performance resources consumed by storing logs of the log interfaces stored in the terminal device, promote rationality of storing logs output by the log interfaces in the terminal device, avoid outputting a large amount of logs without limitation, and consume excessive performance resources of the terminal device.
In one exemplary example, detecting the target amount of logs that the terminal device is currently permitted to store may include, but is not limited to, one of the following:
mode one: acquiring a first log amount and a second log amount of the terminal equipment, wherein the first log amount is the total log amount allowed to be stored by the terminal equipment, and the second log amount is the current stored log amount of the terminal equipment; and determining a difference value between the first log amount and the second log amount as the target log amount.
Alternatively, in this embodiment, the first log amount may be, but is not limited to, a performance resource for indicating the terminal device, the second log amount may be, but is not limited to, a total data amount of logs or a total number of logs stored for each log interface included in the terminal device, and a difference between the first log amount and the second log amount may be, but is not limited to, determined as a target log amount that allows the terminal device to be currently allowed to store.
Mode two: and reading the remaining number of tokens from a token bucket as the target log amount, wherein the number of tokens in the token bucket gradually decreases from an initial value according to the number of logs stored by the terminal equipment.
Alternatively, in this embodiment, the number of tokens in the token bucket may be, but not limited to, used to record the amount of logs allowed to be stored by the terminal device, the total capacity of the token bucket may be, but not limited to, used to indicate the total amount of logs stored by the performance resource of the terminal device, and each log stored by the terminal device may be, but not limited to, one less than the number of tokens in the token bucket, so as to implement control over the log amount of the log interface stored by the terminal device.
FIG. 3 is a schematic diagram of invoking tokens in a token bucket according to an embodiment of the present application, where, as shown in FIG. 3, it is necessary to store the log output by the log interface in the terminal device, it may be, but is not limited to, first determining whether there is a token in the current token bucket, and if there is NO token in the current token bucket, returning NO (not allowed), and not allowing the log output by the log interface to be stored in the terminal device; if the current token bucket has tokens, the number of tokens in the token bucket is reduced by one every time a log is output, and YES (permission) is returned, so that whether the terminal equipment is permitted to store the output log of the log interface or not is controlled through the number of the tokens currently remained in the token bucket, the log is prevented from being stored all the time, and the performance resources of the terminal equipment required for storing the log are saved.
Alternatively, in this embodiment, the number of tokens in the token bucket may be, but not limited to, gradually decreased from an initial value according to the number of logs stored by the terminal device, and the initial value may be, but not limited to, the total number of tokens in the token bucket, or a part of the total number of tokens, etc., and the present application is not limited.
In one exemplary example, the token of the initial value may be generated, but is not limited to, by: obtaining a product of a first proportional coefficient and a first log quantity as the initial value, wherein the first log quantity is the total log quantity allowed to be stored by the terminal equipment; tokens of the initial value are generated in the token bucket.
Alternatively, in this embodiment, the product of the first scale factor and the first log may be obtained as the initial value, and the first scale factor may include, but is not limited to, a value that matches the performance of the terminal device, such as 1/2, 1/3, or 1/4 of the total capacity of the token bucket, which avoids generating a large number of tokens in the token bucket in a short time, and thus consumes a large amount of performance of the terminal device in a short time.
In one exemplary instance, all tokens may be generated in a token bucket by, but not limited to, the following: and generating tokens meeting the product value of a second proportionality coefficient and the first log quantity in the token bucket every other target time period until the generated token quantity meets the first log quantity.
Optionally, in this embodiment, the second scaling factor may not be limited to include a factor that meets the requirement of the log amount output by each log interface, but may not be limited to include 1/10, or 1/8, or 1/5, or the like, so that the number of tokens in the token bucket can be guaranteed to be fully used while meeting the log amount output by each log interface, thereby avoiding idle time that causes the terminal device to wait for storing the log of the log interface, and improving the utilization rate of the performance resource of the terminal device.
Fig. 4 is a timing diagram of generating tokens in a token bucket according to an embodiment of the present application, as shown in fig. 4, a timer may be, but is not limited to, a time interval for controlling the number of tokens in the token bucket. The total number of tokens in the token bucket may be, but is not limited to, 2400 (or 3600, or 1200, etc. the number of tokens matches the performance of the terminal device, the application is not limited in this regard), the number of tokens in the token bucket may be, but is not limited to, reset to a product of 1/2 (i.e., the first scale factor described above) and 2400 (i.e., the first log amount described above), i.e., 1200 (i.e., the initial value described above), may be, but is not limited to, traversing each exporter (i.e., the log interface described above) to reset the log amount (i.e., ca in fig. 4) stored by the terminal device for each exporter to 0, may be, but is not limited to, invoking a priority reset function to traverse each exporter and resetting the exporter priority (i.e., the log amount identifier described above) to equal High. The reset token function may be invoked, but is not limited to, every 100ms, placing the product of 1/10 (i.e., the second scaling factor described above) and 2400, i.e., 120 tokens, in the token manager until 2400 tokens are generated in the token manager.
In one exemplary example, determining the log storage status from the target log amount may include, but is not limited to, the following:
Case one: and under the condition that the target log quantity is larger than or equal to a first threshold value, determining the log storage state as a first state, wherein the first state is used for indicating the terminal equipment to allow the log output by the target log interface to be continuously stored, and under the condition that the log storage state is the first state, storing the target log.
Alternatively, in this embodiment, the performance resource of the terminal device may, but is not limited to, be capable of supporting the continuous storage of the log output by the target log interface, in the case where the target log amount is greater than or equal to the first threshold.
Optionally, in this embodiment, the first threshold may, but is not limited to, include 1/4, or 1/3, or 1/5 of the total number of tokens in the token bucket, or the number of tokens matching the performance of the terminal device, so that, in the case that the terminal device already stores enough logs output by the target log interfaces, the current performance of the terminal device is comprehensively considered in combination with the number of tokens currently remaining in the token bucket, and according to the actual log quantity situation stored for each log interface, the log quantity output by each log interface is dynamically adjusted and stored, without setting the log quantity output by each log interface stored by the terminal device to a fixed value, thereby improving the utilization rate of the performance resource of the terminal device.
And a second case: and under the condition that the target log quantity is smaller than a first threshold value, determining the log storage state to be a second state, wherein the second state is used for indicating that the terminal equipment is not allowed to continue to store the log output by the target log interface.
Optionally, in this embodiment, the target log amount is smaller than the first threshold, which may be, but not limited to, that the performance resources of the terminal device allocated for storing the log output by the target log interface have been consumed, and the current performance resources of the terminal device have not allowed to continue to store the log output by the target log interface, so that under the condition of storing the log output by each log interface, the control terminal device stores the log output by the log interface in combination with the performance resources of the terminal device, and the rationality of storing the log output by the log interface is improved.
In the technical solution provided in step S206, the target log may not be stored but not limited to in the second state, the target log may be discarded but not limited to, and so on, which realizes comprehensive consideration of performance resources of the terminal device, controls log quantity output by the log interface, and avoids outputting a large amount of invalid log, thereby consuming too many performances of the terminal device, causing equipment to be blocked, and affecting user experience.
In order to better understand the method for storing the log data, the following description is given with reference to an optional embodiment, but the method is not limited to the technical solution of the embodiment of the present application.
In this embodiment, a method for storing log data is provided, and fig. 5 is a schematic diagram of a method for storing log data according to an embodiment of the present application, as shown in fig. 5, mainly including the following steps:
step S502: the time stamp of the output log of the exporter (i.e., the log interface described above) may be obtained, but is not limited to, that exporter a, exporter B, exporter C, exporter D, etc. all want to export the log.
Step S504: the level check and the print parameter string may be, but is not limited to, checking whether the level of the exporter is a level that allows the log to be output, may be, but is not limited to, setting Warn or the like to a level that allows the log to be output, may be, but is not limited to, setting debug or the like to a level that does not allow the log to be output, and may be, but is not limited to, allowing the log of the exporter to be output in a case where the level of the exporter (such as Fatal) is greater than Warn.
Step S506: the logs are placed in a serial queue, which can but is not limited to receiving 3000 (or 5000, or 6000, etc. values, which the present application does not limit) at most, and each log can but is not limited to carrying log data, a level (level) of an exporter outputting the log, a timestamp, and a name of the exporter, etc.
Step S508: priority processing of the exporter may be performed by, but is not limited to, checking whether the exporter priority and the number of tokens remaining in the token bucket meet corresponding requirements.
Fig. 6 is a schematic diagram of a process for controlling the Priority of an exporter and the number of tokens remaining in a token bucket according to an embodiment of the present application, as shown in fig. 6, the Priority Manager Priority processor may, but is not limited to, obtain, according to a log Name of a log, the Priority (i.e. the above log quantity identifier) of an exporter outputting the log (i.e. the above target log interface), and in the case where the Priority is not LOW, that is, the Priority of the exporter is High at this time, allow the terminal device to store the log output by the exporter; under the condition that the priority is LOW, the method can be used for but is not limited to calling Token Manager to acquire the number of tokens currently remained in a Token bucket, and returning YES under the condition that the number of tokens currently remained in the Token bucket is greater than or equal to 1/4 of the total capacity of the tokens in the Token bucket (namely the first threshold value), namely, the performance resource of the terminal equipment allows the terminal equipment to continuously store the log output by the outputter; and returning to NO under the condition that the number of tokens currently remained in the token bucket is less than 1/4 of the total capacity of the tokens in the token bucket, namely, the performance resource of the terminal equipment does not allow the terminal equipment to continuously store the log output by the outputter.
Step S510: the method may include, but is not limited to, invoking a token generator to generate 2400 (i.e., the first log amount described above) tokens (or 3500, or 4000, etc. tokens matching the performance of the terminal device) in a token bucket, may, but is not limited to, starting to store the log output by the log interface in the case of 1200 (i.e., the initial value described above) tokens being generated, and may, but is not limited to, subtracting one token per log stored in the token bucket.
Step S512: the method can be used for, but is not limited to, calling a Processor to perform formatting and desensitizing processing on the log under the condition that the terminal equipment is allowed to continuously store the log output by the output device, removing sensitive information such as mailbox addresses and personal phones carried in the log, performing formatting processing on the log, and processing the format of the log into a format meeting the requirement of a required format.
Step S514: updating the priority of the exporter.
Fig. 7 is a schematic diagram of updating priorities of exporters according to an embodiment of the present application, as shown in fig. 7, the Priority Manager may update priorities of the respective exporters by, but not limited to, calculating Priority coefficients of the exporters, where the Priority coefficients P may be, but not limited to, calculated by p=ca/Ce, where Ca may be, but not limited to, a log amount accumulated by a terminal device for an exporter, ce may be, but not limited to, a threshold value of a log amount stored by the terminal device for an exporter, defined according to a total log output Cs per second, and assuming that M log exporters output the log amount threshold value ce=cs/M for each exporter.
The priority of the log Name in the cache may be updated to LOW if P is greater than 2.0 (or 1.0, or 2.5, or the like, which is a value matching the performance of the terminal device), or may be directly ended without updating the priority of the log Name in the cache if P is less than or equal to 2.0.
Step S516: the log output by the storage output device can be, but not limited to, put the character string formatted by the log into the Cache (Cache queue), take out and put into the Write (Write) queue in batches every 10 (or 20 or 15, etc. numerical values are not limited by the application), and then carry out Write to file (Write file) in batches from the Write queue.
Through the functional modules of the exporter log output queue, the priority manager, the token manager, the log formatting process, the file writing queue and the like, the dynamic adjustment of the log output quantity according to the mobile phone performance is realized so as to reduce the performance consumption caused by log output, ensure that the software has enough smooth use of resources, and promote the user experience. The main function of the exporter log output queue is to serialize all exporters, and synchronous lock protection is not needed for log exporters in different threads. And meanwhile, the maximum receiving number of the serial queue is set, so that the system performance resource exhaustion caused by too large log quantity at a certain time point is prevented. The main function of the priority manager is to update and acquire the priority coefficient, and dynamically update the priority information according to the actual output quantity of the output log. The main function of the token manager is to manage the token number of the exporter, and determine whether the exporter can continue to export logs according to the priority of the exporter and the actual number of tokens in the token bucket. The log formatting process mainly formats, desensitizes and controls the output of the control console for the log. File write queues are mainly used for serializing file writes, and one of the purposes of adopting queue processing is to avoid that synchronous lock protection causes larger performance consumption.
From the description of the above embodiments, it will be clear to a person skilled in the art that the method according to the above embodiments may be implemented by means of software plus the necessary general hardware platform, but of course also by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method of the various embodiments of the present application.
FIG. 8 is a block diagram of a log data storage device according to an embodiment of the present application; as shown in fig. 8, includes:
an obtaining module 82, configured to obtain a target log parameter corresponding to a target log to be stored, where the target log parameter is used to indicate a log amount stored in a target log interface that outputs the target log in a terminal device;
a detection module 84, configured to detect a log storage state of the terminal device when the target log parameter meets a target parameter condition, where the log parameter meets the parameter condition and indicates that an amount of log stored by a log interface exceeds an amount of log allowed to be stored by the log interface, and the log storage state is used to indicate a log storage condition of the terminal device;
And a storage module 86, configured to store the target log according to the log storage state.
By the above embodiment, the target log parameter indicating the log amount stored by the target log interface for outputting the target log in the terminal device is obtained, and if the target log parameter satisfies the target parameter condition, it may indicate that the log amount stored by the target log interface exceeds the log amount allowed to be stored by the target log interface, that is, the log amount expected to be stored by the target log interface has been stored for the target log interface, in which case, the log storage state of the terminal device is detected, and it is achieved that the log storage of the target log interface is controlled according to the log storage condition of the terminal device in the case that the log amount stored by the log interface exceeds the log amount allowed to be stored by the log interface. By adopting the technical scheme, the problems of more terminal equipment performance consumed by outputting log data and the like in the related technology are solved, and the technical effect of reducing the terminal equipment performance consumed by outputting the log data is realized.
Optionally, the detection module includes:
the detection unit is used for detecting the target log quantity which is allowed to be stored currently by the terminal equipment;
And the determining unit is used for determining the log storage state according to the target log quantity.
Optionally, the detection unit is configured to one of:
acquiring a first log amount and a second log amount of the terminal equipment, wherein the first log amount is the total log amount allowed to be stored by the terminal equipment, and the second log amount is the current stored log amount of the terminal equipment; determining a difference between the first log amount and a second log amount as the target log amount;
and reading the remaining number of tokens from a token bucket as the target log amount, wherein the number of tokens in the token bucket gradually decreases from an initial value according to the number of logs stored by the terminal equipment.
Optionally, the apparatus further includes:
a first processing module, configured to obtain, as the initial value, a product of a first scale factor and a first log quantity before the remaining number of tokens is read from the token bucket as the target log quantity, where the first log quantity is a total log quantity allowed to be stored by the terminal device; generating tokens of the initial value in the token bucket;
and the second processing module is used for generating tokens meeting the product value of the second proportionality coefficient and the first log quantity in the token bucket every other target time period after the product of the first proportionality coefficient and the first log quantity is obtained as the initial value until the generated token quantity meets the first log quantity.
Optionally, the determining unit is configured to:
determining that the log storage state is a first state when the target log amount is greater than or equal to a first threshold value, wherein the first state is used for indicating that the terminal equipment is allowed to continue to store the log output by the target log interface, and storing the target log when the log storage state is the first state;
and under the condition that the target log quantity is smaller than a first threshold value, determining the log storage state to be a second state, wherein the second state is used for indicating that the terminal equipment is not allowed to continue to store the log output by the target log interface.
Optionally, the acquiring module includes:
the first analyzing unit is used for analyzing a target interface identifier from the log identifier of the target log, wherein the interface identifier is used for indicating the target log interface;
the first obtaining unit is configured to obtain a third log amount recorded for the target log interface as the target log parameter, and obtain a second threshold value corresponding to the target interface identifier from interface identifiers and threshold values having a corresponding relationship, where the third log amount is a log amount stored by the target log interface, and the target log parameter satisfies a target parameter condition when the third log amount is greater than or equal to the second threshold value.
Optionally, the acquiring module includes:
the second analyzing unit is used for analyzing a target interface identifier from the log identifier of the target log, wherein the interface identifier is used for indicating the target log interface;
the second obtaining unit is configured to obtain a log quantity identifier allocated to the target log interface as the target log parameter, and obtain a target log quantity identifier, where the log quantity identifier is used to mark whether the log quantity stored for the target log interface exceeds the log quantity allowed to be stored for the target log interface, the target log quantity identifier is used to mark that the log quantity stored for the target log interface exceeds the log quantity allowed to be stored for the target log interface, and the target log parameter satisfies a target parameter condition when the log quantity identifier is the target log identifier.
Alternatively, in the present embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Alternatively, specific examples in this embodiment may refer to examples described in the foregoing embodiments and optional implementations, and this embodiment is not described herein.
It will be appreciated by those skilled in the art that the modules or steps of the application described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may alternatively be implemented in program code executable by computing devices, so that they may be stored in a memory device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than that shown or described, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps within them may be fabricated into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.
The foregoing is merely a preferred embodiment of the present application and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present application, which are intended to be comprehended within the scope of the present application.
Claims (10)
1. A method of storing log data, comprising:
obtaining target log parameters corresponding to target logs to be stored, wherein the target log parameters are used for indicating the stored log quantity of a target log interface for outputting the target logs in terminal equipment;
detecting a log storage state of the terminal equipment under the condition that the target log parameter meets a target parameter condition, wherein the log parameter meets the parameter condition and is used for indicating that the stored log quantity of a log interface exceeds the allowed log quantity stored by the log interface, and the log storage state is used for indicating the log storage condition of the terminal equipment;
and storing the target log according to the log storage state.
2. The method according to claim 1, wherein the detecting the log storage status of the terminal device comprises:
detecting a target log amount which is allowed to be stored currently by the terminal equipment;
and determining the log storage state according to the target log quantity.
3. The method of claim 2, wherein the detecting the target amount of logs the terminal device is currently permitted to store comprises one of:
Acquiring a first log amount and a second log amount of the terminal equipment, wherein the first log amount is the total log amount allowed to be stored by the terminal equipment, and the second log amount is the current stored log amount of the terminal equipment; determining a difference between the first log amount and a second log amount as the target log amount;
and reading the remaining number of tokens from a token bucket as the target log amount, wherein the number of tokens in the token bucket gradually decreases from an initial value according to the number of logs stored by the terminal equipment.
4. The method of claim 3, wherein the step of,
before the reading the remaining number of tokens from the token bucket as the target log amount, the method further comprises: obtaining a product of a first proportional coefficient and a first log quantity as the initial value, wherein the first log quantity is the total log quantity allowed to be stored by the terminal equipment; generating tokens of the initial value in the token bucket;
after the obtaining the product of the first scale factor and the first log as the initial value, the method further includes: and generating tokens meeting the product value of a second proportionality coefficient and the first log quantity in the token bucket every other target time period until the generated token quantity meets the first log quantity.
5. The method of claim 2, wherein said determining the log storage status from the target log amount comprises:
determining that the log storage state is a first state when the target log amount is greater than or equal to a first threshold value, wherein the first state is used for indicating that the terminal equipment is allowed to continue to store the log output by the target log interface, and storing the target log when the log storage state is the first state;
and under the condition that the target log quantity is smaller than a first threshold value, determining the log storage state to be a second state, wherein the second state is used for indicating that the terminal equipment is not allowed to continue to store the log output by the target log interface.
6. The method of claim 1, wherein the obtaining the target log parameter corresponding to the target log to be stored comprises:
analyzing a target interface identifier from the log identifier of the target log, wherein the interface identifier is used for indicating the target log interface;
and acquiring a third log quantity recorded for the target log interface as the target log parameter, and acquiring a second threshold value corresponding to the target interface identifier from the interface identifier and the threshold value with corresponding relation, wherein the third log quantity is the stored log quantity of the target log interface, and the target log parameter meets a target parameter condition under the condition that the third log quantity is larger than or equal to the second threshold value.
7. The method of claim 1, wherein the obtaining the target log parameter corresponding to the target log to be stored comprises:
analyzing a target interface identifier from the log identifier of the target log, wherein the interface identifier is used for indicating the target log interface;
the method comprises the steps of obtaining a log quantity identifier distributed for a target log interface as a target log parameter, and obtaining a target log quantity identifier, wherein the log quantity identifier is used for marking whether the log quantity stored for the target log interface exceeds the log quantity allowed to be stored for the target log interface, the target log quantity identifier is used for marking that the log quantity stored for the target log interface exceeds the log quantity allowed to be stored for the target log interface, and the target log parameter meets a target parameter condition under the condition that the log quantity identifier is the target log identifier.
8. A storage device for log data, comprising:
the acquisition module is used for acquiring target log parameters corresponding to target logs to be stored, wherein the target log parameters are used for indicating the log quantity stored in a target log interface for outputting the target logs in the terminal equipment;
The detection module is used for detecting the log storage state of the terminal equipment under the condition that the target log parameter meets the target parameter condition, wherein the log parameter meets the parameter condition and is used for indicating that the log quantity stored by a log interface exceeds the log quantity allowed to be stored by the log interface, and the log storage state is used for indicating the log storage condition of the terminal equipment;
and the storage module is used for storing the target log according to the log storage state.
9. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a stored program, wherein the program when run performs the method of any one of claims 1 to 7.
10. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to execute the method according to any of claims 1 to 7 by means of the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210571469.5A CN115174289B (en) | 2022-05-24 | 2022-05-24 | Log data storage method and device, storage medium and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210571469.5A CN115174289B (en) | 2022-05-24 | 2022-05-24 | Log data storage method and device, storage medium and electronic device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115174289A CN115174289A (en) | 2022-10-11 |
| CN115174289B true CN115174289B (en) | 2023-10-24 |
Family
ID=83484134
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210571469.5A Active CN115174289B (en) | 2022-05-24 | 2022-05-24 | Log data storage method and device, storage medium and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115174289B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110457258A (en) * | 2019-08-06 | 2019-11-15 | 北京字节跳动网络技术有限公司 | A kind of blog management method, device, equipment and medium |
| CN110716695A (en) * | 2019-09-12 | 2020-01-21 | 北京浪潮数据技术有限公司 | Node log storage method and system, electronic device and storage medium |
| CN111913885A (en) * | 2020-08-07 | 2020-11-10 | 腾讯科技（深圳）有限公司 | Log processing method and device, computer readable storage medium and equipment |
| CN112000630A (en) * | 2020-08-28 | 2020-11-27 | 苏州浪潮智能科技有限公司 | Log storage method, device and equipment and readable storage medium |
| CN112637284A (en) * | 2020-12-09 | 2021-04-09 | 北京金山云网络技术有限公司 | Redo log storage method and device, electronic device and storage medium |
-
2022
- 2022-05-24 CN CN202210571469.5A patent/CN115174289B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110457258A (en) * | 2019-08-06 | 2019-11-15 | 北京字节跳动网络技术有限公司 | A kind of blog management method, device, equipment and medium |
| CN110716695A (en) * | 2019-09-12 | 2020-01-21 | 北京浪潮数据技术有限公司 | Node log storage method and system, electronic device and storage medium |
| CN111913885A (en) * | 2020-08-07 | 2020-11-10 | 腾讯科技（深圳）有限公司 | Log processing method and device, computer readable storage medium and equipment |
| CN112000630A (en) * | 2020-08-28 | 2020-11-27 | 苏州浪潮智能科技有限公司 | Log storage method, device and equipment and readable storage medium |
| CN112637284A (en) * | 2020-12-09 | 2021-04-09 | 北京金山云网络技术有限公司 | Redo log storage method and device, electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115174289A (en) | 2022-10-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108683720B (en) | Container cluster service configuration method and device | |
| CN112346829B (en) | Method and equipment for task scheduling | |
| CN109343965A (en) | Resource adjusting method, device, cloud platform and server | |
| WO2021103790A1 (en) | Container scheduling method and apparatus, and non-volatile computer-readable storage medium | |
| CN110474852B (en) | Bandwidth scheduling method and device | |
| WO2019136963A1 (en) | Method and device for reclaiming memory | |
| CN111104227B (en) | Resource control method and device of K8s platform and related components | |
| EP2898412B1 (en) | Adaptive service timeouts | |
| KR101553650B1 (en) | Apparatus and method for load balancing in multi-core system | |
| CN110647392A (en) | Intelligent elastic expansion method based on container cluster | |
| CN111277640A (en) | User request processing method, device, system, computer equipment and storage medium | |
| CN113687873A (en) | Large-page memory configuration method, system and related device in cloud service page table | |
| CN115454322A (en) | Workload management using trained models | |
| CN112860387A (en) | Distributed task scheduling method and device, computer equipment and storage medium | |
| CN115174289B (en) | Log data storage method and device, storage medium and electronic device | |
| CN109753338B (en) | Virtual GPU (graphics processing Unit) utilization rate detection method and device | |
| CN109002381B (en) | Process communication monitoring method, electronic device and computer readable storage medium | |
| CN115174358B (en) | Monitoring processing method, system, equipment and storage medium for storage cluster interface | |
| CN111538629A (en) | Round-robin method and system of server sensor and related device | |
| CN109639843A (en) | Data transmission method and device | |
| CN111143073B (en) | Virtualized resource management method, device and storage medium | |
| CN114866563A (en) | Capacity expansion method, device, system and storage medium | |
| CN114860536A (en) | Monitoring method, monitoring system and related device of GPU card | |
| CN114461323A (en) | Card pause processing method and device, electronic equipment and storage medium | |
| CN112463416A (en) | Method, device, equipment and storage medium for processing data in message queue |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |