CN112035425B - Log storage method and device and computer system - Google Patents
Log storage method and device and computer system Download PDFInfo
- Publication number
- CN112035425B CN112035425B CN202011159844.2A CN202011159844A CN112035425B CN 112035425 B CN112035425 B CN 112035425B CN 202011159844 A CN202011159844 A CN 202011159844A CN 112035425 B CN112035425 B CN 112035425B
- Authority
- CN
- China
- Prior art keywords
- preset
- data
- log
- key
- processed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/113—Details of archiving
Abstract
The application discloses a log storage method, a log storage device and a computer system, wherein the method comprises the steps of obtaining a to-be-processed log in a preset format generated by a preset service platform and configuration data corresponding to the preset service platform, wherein the to-be-processed log comprises a key value pair and a preset identifier, and the key value pair is used for storing a historical operation record; analyzing the log to be processed, and identifying data keys and corresponding data values included in the log to be analyzed; when the included data key pair contains the preset data key, modifying the data value corresponding to the included preset data key according to the corresponding replacement rule; generating a historical operating record according to the corresponding meaning and the corresponding data value of each data key; and the historical operation record is stored to the index library address corresponding to the preset identification, so that the processing efficiency of the log is improved, the problem generated by the system by non-operation and maintenance personnel is conveniently processed, and the solution efficiency of the system when the problem occurs is improved.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a log storage method and device and a computer system.
Background
During the operation of the system, a system log is generated. In a period of small system scale, the generated system logs are few in number, the original texts of the generated logs can be directly stored, and the operation and maintenance work of the system logs can be manually performed by operation and maintenance personnel. However, with the rapid expansion of business requirements and the diversified development of services, the system becomes more and more complex, one system may include multiple devices and deploy multiple instances, which results in inconsistent data structures of generated logs, and when a problem occurs in the system, the problem needs to be manually located by operation and maintenance personnel, which is very difficult and inefficient.
Disclosure of Invention
In order to solve the defects of the prior art, the present invention provides a method and an apparatus for storing a log, and a computer system, so as to solve the above problems.
In order to achieve the above object, in a first aspect, the present invention provides a method for storing a log, where the method includes:
acquiring a log to be processed in a preset format generated by a preset service platform and configuration data corresponding to the preset service platform, wherein the log to be processed comprises key value pairs and preset identifications, the key value pairs are used for storing historical operating records of the preset service platform, and the configuration data comprises meanings corresponding to each data key, replacement rules corresponding to the preset data key and index library addresses corresponding to each preset identification;
analyzing the log to be processed, and identifying a data key and a corresponding data value included in the log to be analyzed;
when the included data key pair contains the preset data key, modifying the data value corresponding to the included preset data key according to the corresponding replacement rule;
generating a historical operating record according to the corresponding meaning and the corresponding data value of each data key;
and storing the historical operating record to an index library address corresponding to the preset identification.
In some embodiments, the storing the historical operating record to the index repository address corresponding to the preset identifier includes:
sending a data storage request to a preset system corresponding to the index base address so that the preset system can return data access permission according to the data storage request;
and when receiving the returned data access right, storing the historical operating record to the index library address.
In some embodiments, the method comprises:
receiving a log reading request sent by a user;
generating a visual log according to a historical operating record corresponding to the log reading request;
and returning the visual log to the user.
In some embodiments, the pending log is in Json format.
In some embodiments, the obtaining the to-be-processed log in the preset format generated by the preset service platform and the configuration data corresponding to the preset service platform includes:
acquiring logs to be processed in a preset format generated by a preset service platform and configuration data corresponding to the preset service platform by using a flash log acquisition system;
the analyzing the log to be processed and identifying the data key and the corresponding data value included in the log to be analyzed comprises:
and analyzing the log to be processed by using a Flink streaming computation engine, and identifying the data key and the corresponding data value included in the log to be analyzed.
In a second aspect, the present application provides a method for storing a log, where the method includes:
sending the generated to-be-processed log to a preset processing end so that the preset processing end can receive the to-be-processed log and obtain corresponding configuration data, wherein the to-be-processed log comprises key value pairs and preset identifications, the key value pairs are used for storing historical operation records of the preset service platform, and the configuration data comprise meanings corresponding to each data key, replacement rules corresponding to the preset data keys and index library addresses corresponding to each preset identification; analyzing the log to be processed, and identifying a data key and a corresponding data value included in the log to be analyzed; when the included data key pair contains the preset data key, modifying the data value corresponding to the included preset data key according to the corresponding replacement rule; and storing the historical operating record to an index library address corresponding to the preset identification.
In a third aspect, the present application provides a storage apparatus for a log, the apparatus comprising:
the system comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a log to be processed in a preset format generated by a preset service platform and configuration data corresponding to the preset service platform, the log to be processed comprises key value pairs and preset identifications, the key value pairs are used for storing historical operating records of the preset service platform, and the configuration data comprises meanings corresponding to each data key, replacement rules corresponding to the preset data keys and index library addresses corresponding to each preset identification;
the analysis module is used for analyzing the log to be processed and identifying the data key and the corresponding data value included in the log to be analyzed; when the included data key pair comprises a preset data key, modifying a data value corresponding to the included preset data key according to a corresponding replacement rule; generating a historical operating record according to the corresponding meaning and the corresponding data value of each data key;
and the storage module is used for storing the historical operating record to the index library address corresponding to the preset identifier.
In a fourth aspect, the present application provides a computer system comprising:
one or more processors;
and memory associated with the one or more processors for storing program instructions that, when read and executed by the one or more processors, perform operations comprising:
acquiring a log to be processed in a preset format generated by a preset service platform and configuration data corresponding to the preset service platform, wherein the log to be processed comprises key value pairs and preset identifications, the key value pairs are used for storing historical operating records of the preset service platform, and the configuration data comprises meanings corresponding to each data key, replacement rules corresponding to the preset data key and index library addresses corresponding to each preset identification;
analyzing the log to be processed, and identifying a data key and a corresponding data value included in the log to be analyzed;
when the included data key pair contains the preset data key, modifying the data value corresponding to the included preset data key according to the corresponding replacement rule;
generating a historical operating record according to the corresponding meaning and the corresponding data value of each data key;
and storing the historical operating record to an index library address corresponding to the preset identification.
The invention has the following beneficial effects:
the application provides a log storage method, which comprises the steps of obtaining a log to be processed in a preset format generated by a preset service platform and configuration data corresponding to the preset service platform, wherein the log to be processed comprises key value pairs and preset identifications, the key value pairs are used for storing historical operation records of the preset service platform, and the configuration data comprise meanings corresponding to each data key, replacement rules corresponding to the preset data keys and index base addresses corresponding to each preset identification; analyzing the log to be processed, and identifying a data key and a corresponding data value included in the log to be analyzed; when the included data key pair contains the preset data key, modifying the data value corresponding to the included preset data key according to the corresponding replacement rule; generating a historical operating record according to the corresponding meaning and the corresponding data value of each data key; the historical operation records are stored to the index base address corresponding to the preset identification, the format of the log to be processed generated by the service platform is preset, the processing efficiency of the log is improved, the stored analyzed log can be directly obtained when the log is required to be consulted subsequently, the log does not need to be analyzed again, the analyzed log can be understood by non-operation and maintenance personnel, the problem generated by the system can be conveniently processed by the non-operation and maintenance personnel, and the solution efficiency of the system when the problem occurs is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flow chart of a log storage system service provided by an embodiment of the present application;
FIG. 2 is a flow chart of a method provided by an embodiment of the present application;
FIG. 3 is a block diagram of an apparatus according to an embodiment of the present disclosure;
fig. 4 is a computer system structure diagram provided in the embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As described in the background art, in order to solve the above technical problems, the present application provides a log storage system, which includes a service platform for generating logs and a server for storing logs. The system can comprise a plurality of service platforms, and each service platform is preset with the same log format. Preferably, the log may be in Json format.
Example one
As shown in fig. 1, the process of performing log storage and query by the storage system includes:
firstly, a service platform generates a log with a preset format and sends the log to a server;
the generated log may include a prefix and a Json string. Each prefix uniquely corresponds to an index store. The Json string stores historical operating records of the platform in a key-value pair format. The historical operating record may include operating process parameters of the service platform, data operations executed, and the like, which are used to record the operating condition and the service condition of the platform, and this is not limited in the present application.
Secondly, the server receives the logs by using a flash log acquisition system to acquire configuration data corresponding to the service platform;
the method is characterized by comprising the following steps that (1) the Flume is a high-availability, high-reliability and distributed system for acquiring, aggregating and transmitting mass logs, and supports various data senders to be customized in a log system and used for collecting data; at the same time, flash provides the ability to simply process data and write to various data recipients.
Each service platform has corresponding configuration data, and the configuration data comprises data such as the meaning of each data key, the field type of each key value pair, whether each data key is displayed, the enumeration value corresponding to the preset data key, the meaning corresponding to the enumeration value and the like contained in a log generated by the service platform.
Wherein the field type includes time, value, text and other types. And the data value corresponding to the data key which is configured to be required to be displayed in the configuration data is displayed to the non-operation and maintenance personnel inquiring the log.
The meaning of each data key is used to indicate the contribution of the data value to which each data key corresponds. For example, the data value corresponding to a certain data key may be specified to represent the shopping amount of the user on the service platform, and the like, which is not limited in the present application.
The configuration data may include the meaning of the enumerated value corresponding to the preset data key. For example, the configuration data may specify that a value corresponding to a certain data key may be 1 or 0, and when the value corresponding to the data key is 0, the value represents that the purchase has failed, and the value 1 represents that the purchase has succeeded, and the like, which is not limited in the present application. The configuration data may further include a filtering rule, and the data value corresponding to the data key included in the filtering rule is not stored.
And step three, the server side obtains the data value corresponding to each data key contained in the log by using Flink streaming calculation, and determines the function of the data value corresponding to each data key according to the meaning of each data key.
Flink is an open source streaming framework, and the core of the Flink is a distributed streaming data streaming engine written in Java and Scale. Flink executes arbitrary stream data programs in a data parallel and pipelined manner, and Flink's pipelined runtime system can execute batch and stream processing programs.
And according to the action of the data value corresponding to each data key, the server side can obtain the historical operating record of the service platform contained in the log to be stored. For example, the data value corresponding to a certain data key is used to indicate the purchase amount of the user on the service platform, the server may determine the purchase amount of the user according to the data value corresponding to the data key in the log, and generate a purchase record, where the historical operating record includes the purchase record.
Preferably, the historical operating records may be generated in any format, which is not limited in this application.
Step four, the server side determines an index library corresponding to the log according to the prefix;
because each prefix uniquely corresponds to one index library, the server can determine the corresponding index library to be stored according to the prefix. The index library can be deployed on any storage system or device such as an elastic search, which is not limited in the present application.
And fifthly, the server requests data access authority from the index database, and when the data access authority is acquired, the historical operating records are stored in the corresponding index database.
Step six, the server receives a log query request of a user;
and the server determines the log to be read according to the log query request of the user and positions the index library where the log to be read is located.
And step seven, the server side generates a corresponding visual log according to the acquired corresponding historical operating record and a preset template, and returns the visual log to the user.
And the server side requests the index database for data access authority, and when the data access authority is acquired, the corresponding historical operating record is acquired from the index database.
Example two
Corresponding to the foregoing embodiment, the present application provides a log storage method, as shown in fig. 2, where the method includes:
210. acquiring a log to be processed in a preset format generated by a preset service platform and configuration data corresponding to the preset service platform, wherein the log to be processed comprises key value pairs and preset identifications, the key value pairs are used for storing historical operating records of the preset service platform, and the configuration data comprises meanings corresponding to each data key, replacement rules corresponding to the preset data key and index library addresses corresponding to each preset identification;
220. analyzing the log to be processed, and identifying a data key and a corresponding data value included in the log to be analyzed;
preferably, the acquiring the to-be-processed log in the preset format generated by the preset service platform and the configuration data corresponding to the preset service platform includes:
221. acquiring logs to be processed in a preset format generated by a preset service platform and configuration data corresponding to the preset service platform by using a flash log acquisition system;
the analyzing the log to be processed and identifying the data key and the corresponding data value included in the log to be analyzed comprises:
222. and analyzing the log to be processed by using a Flink streaming computation engine, and identifying the data key and the corresponding data value included in the log to be analyzed.
230. When the included data key pair contains the preset data key, modifying the data value corresponding to the included preset data key according to the corresponding replacement rule;
240. generating a historical operating record according to the corresponding meaning and the corresponding data value of each data key;
250. and storing the historical operating record to an index library address corresponding to the preset identification.
Preferably, the storing the historical operating record to the index library address corresponding to the preset identifier includes:
251. sending a data storage request to a preset system corresponding to the index base address so that the preset system can return data access permission according to the data storage request;
252. and when receiving the returned data access right, storing the historical operating record to the index library address.
Preferably, the log to be processed is in a Json format.
EXAMPLE III
In correspondence to the above-described embodiments, as shown in fig. 3, the present application provides a storage apparatus for a log, the apparatus including:
an obtaining module 310, configured to obtain a to-be-processed log in a preset format generated by a preset service platform and configuration data corresponding to the preset service platform, where the to-be-processed log includes a key value pair and a preset identifier, the key value pair is used to store a historical operating record of the preset service platform, and the configuration data includes a meaning corresponding to each data key, a replacement rule corresponding to the preset data key, and an index library address corresponding to each preset identifier;
the analysis module 320 is configured to analyze the log to be processed, and identify a data key and a corresponding data value included in the log to be analyzed; when the included data key pair comprises a preset data key, modifying a data value corresponding to the included preset data key according to a corresponding replacement rule; generating a historical operating record according to the corresponding meaning and the corresponding data value of each data key;
the storage module 330 is configured to store the historical operating record to an index library address corresponding to the preset identifier.
Preferably, the storage module 330 is further configured to send a data storage request to a preset system corresponding to the index repository address, so that the preset system returns a data access permission according to the data storage request; and when receiving the returned data access right, storing the historical operating record to the index library address.
Preferably, the apparatus further includes a receiving module 340, configured to receive a log reading request sent by a user; the analysis module 320 may further be configured to generate a visual log according to the historical operating record corresponding to the log reading request; and returning the visual log to the user.
Preferably, the log to be processed is in a Json format.
Preferably, the obtaining module 310 may be further configured to obtain, by using a Flume log collecting system, a to-be-processed log in a preset format generated by a preset service platform and configuration data corresponding to the preset service platform, and the parsing module 320 may be further configured to parse, by using a Flink stream computing engine, the to-be-processed log, and identify a data key and a corresponding data value included in the to-be-parsed log.
Example four
Corresponding to the method, the application provides a log storage method, which comprises the following steps:
sending the generated to-be-processed log to a preset processing end so that the preset processing end can receive the to-be-processed log and obtain corresponding configuration data, wherein the to-be-processed log comprises key value pairs and preset identifications, the key value pairs are used for storing historical operation records of the preset service platform, and the configuration data comprise meanings corresponding to each data key, replacement rules corresponding to the preset data keys and index library addresses corresponding to each preset identification; analyzing the log to be processed, and identifying a data key and a corresponding data value included in the log to be analyzed; when the included data key pair contains the preset data key, modifying the data value corresponding to the included preset data key according to the corresponding replacement rule; and storing the historical operating record to an index library address corresponding to the preset identification.
EXAMPLE five
An embodiment of the present application provides a computer system, including: one or more processors; and memory associated with the one or more processors for storing program instructions that, when read and executed by the one or more processors, perform operations comprising:
acquiring a log to be processed in a preset format generated by a preset service platform and configuration data corresponding to the preset service platform, wherein the log to be processed comprises key value pairs and preset identifications, the key value pairs are used for storing historical operating records of the preset service platform, and the configuration data comprises meanings corresponding to each data key, replacement rules corresponding to the preset data key and index library addresses corresponding to each preset identification;
analyzing the log to be processed, and identifying a data key and a corresponding data value included in the log to be analyzed;
when the included data key pair contains the preset data key, modifying the data value corresponding to the included preset data key according to the corresponding replacement rule;
generating a historical operating record according to the corresponding meaning and the corresponding data value of each data key;
and storing the historical operating record to an index library address corresponding to the preset identification. Fig. 4 illustrates an architecture of a computer system, which may include, in particular, a processor 1510, a video display adapter 1511, a disk drive 1512, an input/output interface 1513, a network interface 1514, and a memory 1520. The processor 1510, video display adapter 1511, disk drive 1512, input/output interface 1513, network interface 1514, and memory 1520 may be communicatively coupled via a communication bus 1530.
The processor 1510 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits, and is configured to execute related programs to implement the technical solution provided by the present Application.
The Memory 1520 may be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, or the like. The memory 1520 may store an operating system 1521 for controlling the operation of the computer system 1500, a Basic Input Output System (BIOS)1522 for controlling low-level operations of the computer system 1500. In addition, a web browser 1523, a data storage management 1524, an icon font processing system 1525, and the like may also be stored. The icon font processing system 1525 may be an application program that implements the operations of the foregoing steps in this embodiment of the application. In summary, when the technical solution provided by the present application is implemented by software or firmware, the relevant program codes are stored in the memory 1520 and called for execution by the processor 1510. The input/output interface 1513 is used for connecting an input/output module to realize information input and output. The i/o module may be configured as a component in a device (not shown) or may be external to the device to provide a corresponding function. The input devices may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices may include a display, a speaker, a vibrator, an indicator light, etc.
The network interface 1514 is used to connect a communication module (not shown) to enable the device to communicatively interact with other devices. The communication module can realize communication in a wired mode (such as USB, network cable and the like) and also can realize communication in a wireless mode (such as mobile network, WIFI, Bluetooth and the like).
The bus 1530 includes a path to transfer information between the various components of the device, such as the processor 1510, the video display adapter 1511, the disk drive 1512, the input/output interface 1513, the network interface 1514, and the memory 1520.
In addition, the computer system 1500 may also obtain information of specific extraction conditions from the virtual resource object extraction condition information database 1541 for performing condition judgment, and the like.
It should be noted that although the above devices only show the processor 1510, the video display adapter 1511, the disk drive 1512, the input/output interface 1513, the network interface 1514, the memory 1520, the bus 1530, etc., in a specific implementation, the devices may also include other components necessary for proper operation. Furthermore, it will be understood by those skilled in the art that the apparatus described above may also include only the components necessary to implement the solution of the present application, and not necessarily all of the components shown in the figures.
From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, or the like, and includes several instructions for enabling a computer device (which may be a personal computer, a cloud server, or a network device) to execute the method according to the embodiments or some parts of the embodiments of the present application.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the system or system embodiments are substantially similar to the method embodiments and therefore are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described system and system embodiments are only illustrative, wherein the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (6)
1. A method for storing logs, the method comprising:
acquiring a to-be-processed log in a preset format generated by a preset service platform and configuration data corresponding to the preset service platform by using a flash log acquisition system, wherein the to-be-processed log comprises a key value pair and a preset identifier, the key value pair is used for storing a historical operating record of the preset service platform, and the configuration data comprises a meaning corresponding to each data key, a replacement rule corresponding to the preset data key and an index library address corresponding to each preset identifier;
analyzing the log to be processed by using a Flink streaming computation engine, and identifying a data key and a corresponding data value included in the log to be processed;
when the included data key pair contains the preset data key, modifying the data value corresponding to the included preset data key according to the corresponding replacement rule;
generating a historical operating record according to the corresponding meaning and the corresponding data value of each data key;
storing the historical operating record to an index library address corresponding to the preset identification,
wherein the storing the historical operating record to the index library address corresponding to the preset identifier includes:
sending a data storage request to a preset system corresponding to the index base address so that the preset system can return data access permission according to the data storage request;
and when receiving the returned data access right, storing the historical operating record to the index library address.
2. The method according to claim 1, characterized in that it comprises:
receiving a log reading request sent by a user;
generating a visual log according to a historical operating record corresponding to the log reading request;
and returning the visual log to the user.
3. The method of claim 1, wherein the pending log is in a Json format.
4. A method for storing logs, the method comprising:
sending the generated to-be-processed log to a preset processing end so that the preset processing end can receive the to-be-processed log and obtain corresponding configuration data, wherein the to-be-processed log comprises key value pairs and preset identifications, the key value pairs are used for storing historical operation records of a preset service platform, and the configuration data comprise meanings corresponding to each data key, replacement rules corresponding to the preset data keys and index library addresses corresponding to each preset identification; analyzing the log to be processed, and identifying a data key and a corresponding data value included in the log to be processed; when the included data key pair contains the preset data key, modifying the data value corresponding to the included preset data key according to the corresponding replacement rule; storing the historical operating record to an index library address corresponding to the preset identifier, wherein the storing the historical operating record to the index library address corresponding to the preset identifier includes:
sending a data storage request to a preset system corresponding to the index base address so that the preset system can return data access permission according to the data storage request;
and when receiving the returned data access right, storing the historical operating record to the index library address.
5. An apparatus for storing a log, the apparatus comprising:
the system comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a log to be processed in a preset format generated by a preset service platform and configuration data corresponding to the preset service platform, the log to be processed comprises key value pairs and preset identifications, the key value pairs are used for storing historical operating records of the preset service platform, and the configuration data comprises meanings corresponding to each data key, replacement rules corresponding to the preset data keys and index library addresses corresponding to each preset identification;
the analysis module is used for analyzing the log to be processed and identifying the data key and the corresponding data value included in the log to be processed; when the included data key pair comprises a preset data key, modifying a data value corresponding to the included preset data key according to a corresponding replacement rule; generating a historical operating record according to the corresponding meaning and the corresponding data value of each data key;
the storage module is used for storing the historical operating record to an index base address corresponding to the preset identifier, and is also used for sending a data storage request to a preset system corresponding to the index base address so that the preset system can return a data access right according to the data storage request; and when receiving the returned data access right, storing the historical operating record to the index library address.
6. A computer system, the system comprising:
one or more processors;
and memory associated with the one or more processors for storing program instructions that, when read and executed by the one or more processors, perform operations comprising:
acquiring a log to be processed in a preset format generated by a preset service platform and configuration data corresponding to the preset service platform, wherein the log to be processed comprises key value pairs and preset identifications, the key value pairs are used for storing historical operating records of the preset service platform, and the configuration data comprises meanings corresponding to each data key, replacement rules corresponding to the preset data key and index library addresses corresponding to each preset identification;
analyzing the log to be processed, and identifying a data key and a corresponding data value included in the log to be processed;
when the included data key pair contains the preset data key, modifying the data value corresponding to the included preset data key according to the corresponding replacement rule;
generating a historical operating record according to the corresponding meaning and the corresponding data value of each data key;
storing the historical operating record to an index library address corresponding to the preset identification,
wherein the storing the historical operating record to the index library address corresponding to the preset identifier includes:
sending a data storage request to a preset system corresponding to the index base address so that the preset system can return data access permission according to the data storage request;
and when receiving the returned data access right, storing the historical operating record to the index library address.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011159844.2A CN112035425B (en) | 2020-10-27 | 2020-10-27 | Log storage method and device and computer system |
| CA3136021A CA3136021A1 (en) | 2020-10-27 | 2021-10-27 | Log storage method and device and computer system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011159844.2A CN112035425B (en) | 2020-10-27 | 2020-10-27 | Log storage method and device and computer system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112035425A CN112035425A (en) | 2020-12-04 |
| CN112035425B true CN112035425B (en) | 2021-11-09 |
Family
ID=73573377
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011159844.2A Active CN112035425B (en) | 2020-10-27 | 2020-10-27 | Log storage method and device and computer system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112035425B (en) |
| CA (1) | CA3136021A1 (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111241078A (en) * | 2020-01-07 | 2020-06-05 | 网易（杭州）网络有限公司 | Data analysis system, data analysis method and device |
| CN111526060A (en) * | 2020-06-16 | 2020-08-11 | 网易（杭州）网络有限公司 | Method and system for processing service log |
| CN111522786A (en) * | 2020-04-21 | 2020-08-11 | 中国建设银行股份有限公司 | Log processing system and method |
-
2020
- 2020-10-27 CN CN202011159844.2A patent/CN112035425B/en active Active
-
2021
- 2021-10-27 CA CA3136021A patent/CA3136021A1/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111241078A (en) * | 2020-01-07 | 2020-06-05 | 网易（杭州）网络有限公司 | Data analysis system, data analysis method and device |
| CN111522786A (en) * | 2020-04-21 | 2020-08-11 | 中国建设银行股份有限公司 | Log processing system and method |
| CN111526060A (en) * | 2020-06-16 | 2020-08-11 | 网易（杭州）网络有限公司 | Method and system for processing service log |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112035425A (en) | 2020-12-04 |
| CA3136021A1 (en) | 2022-04-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110221982B (en) | Performance test method, device and equipment of business system and readable storage medium | |
| CN111552627A (en) | User interface testing method and device, storage medium and computer equipment | |
| CN112306881A (en) | Simulation data generation method, device, equipment and storage medium | |
| CN108111364B (en) | Service system testing method and device | |
| CN110955409B (en) | Method and device for creating resources on cloud platform | |
| CN110858172A (en) | Automatic test code generation method and device | |
| CN110825731A (en) | Data storage method and device, electronic equipment and storage medium | |
| CN111125057B (en) | Method and device for processing service request and computer system | |
| CN113076729A (en) | Method and system for importing report, readable storage medium and electronic equipment | |
| CN111402000B (en) | Service page generation method, device and system and computer system | |
| CN112035425B (en) | Log storage method and device and computer system | |
| CN112416747A (en) | Test case execution method, device, equipment and medium | |
| CN110866492B (en) | Baseline branch identification method and device and computer system | |
| CN110347577B (en) | Page testing method, device and equipment thereof | |
| CN112214497A (en) | Label processing method and device and computer system | |
| CN113344023A (en) | Code recommendation method, device and system | |
| CN112380115A (en) | Regression testing method and device, electronic equipment and storage medium | |
| CN112988560A (en) | Method and device for testing system robustness | |
| CN108628909B (en) | Information pushing method and device | |
| CN116661758B (en) | Method, device, electronic equipment and medium for optimizing log framework configuration | |
| CN111680112B (en) | Data analysis method and device | |
| CN112307050B (en) | Identification method and device for repeated correlation calculation and computer system | |
| CN116483735B (en) | Method, device, storage medium and equipment for analyzing influence of code change | |
| CN117827624A (en) | Function test method, device and storage medium | |
| CN112948435A (en) | Data processing method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 834, Yingying building, No.99, Tuanjie Road, yanchuangyuan, Jiangbei new district, Nanjing, Jiangsu Province Applicant after: Nanjing Xingyun Digital Technology Co.,Ltd. Address before: Room 834, Yingying building, No.99, Tuanjie Road, yanchuangyuan, Jiangbei new district, Nanjing, Jiangsu Province Applicant before: Suning financial technology (Nanjing) Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |