CN111552675A - Information query method and device, computer equipment and storage medium - Google Patents
Information query method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN111552675A CN111552675A CN202010333803.4A CN202010333803A CN111552675A CN 111552675 A CN111552675 A CN 111552675A CN 202010333803 A CN202010333803 A CN 202010333803A CN 111552675 A CN111552675 A CN 111552675A
- Authority
- CN
- China
- Prior art keywords
- data
- information
- client
- target
- identity information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Mathematical Physics (AREA)
- Fuzzy Systems (AREA)
- Computer And Data Communications (AREA)
Abstract
The disclosure relates to an information query method, an information query device, computer equipment and a storage medium, and belongs to the technical field of information. The method comprises the following steps: receiving an operation information query request, wherein the operation information query request carries a target data path in a distributed coordination system; inquiring a data operation record table according to the target data path to obtain the identity information of the target client, wherein the data operation record table is used for recording the identity information of the client operating the data in the distributed coordination system and the data path of the operated data; and sending the identity information of the target client to the sending equipment of the operation information inquiry request. According to the method and the device, the identity information of the client operating the data in the distributed coordination system is quickly searched according to the data path, so that the fact that the client operates the data of the data path is known, the source tracing of the data operation in the distributed coordination system is achieved, the complexity is reduced, and the query efficiency is improved.
Description
Technical Field
The present disclosure relates to the field of information technologies, and in particular, to an information query method and apparatus, a computer device, and a storage medium.
Background
Zookeeper is a popular distributed coordination system, and the included data is extremely important, so that recording and monitoring are required for any type of operation such as creation, modification or deletion of data, and therefore, the subsequent query of the operation information of the data is facilitated.
In the related art, when operation information of data needs to be queried, a packet capture program is run on a physical machine where a service (Server) end of a Zookeeper is located, a Transmission Control Protocol (TCP) packet is captured through the packet capture program, then the TCP packet is reassembled, a Zookeeper request Protocol in the TCP packet is analyzed, and identity information such as an Internet Protocol (IP) address or port information of a client (client) operating on the data and a data path (path) of the operated data are acquired.
According to the technology, a TCP protocol needs to be analyzed when a TCP data packet is captured, the problems of disorder, retransmission and the like need to be correctly processed when the TCP data packet is recombined, the Zookeeper request protocol needs to be analyzed after the TCP data packet is recombined, the identity information of a client operating data can be obtained, the complexity is high, and the information query efficiency is low.
Disclosure of Invention
The present disclosure provides an information query method, apparatus, computer device and storage medium, which can improve the query efficiency of information. The technical scheme of the disclosure is as follows:
according to a first aspect of the embodiments of the present disclosure, there is provided an information query method, including:
receiving an operation information query request, wherein the operation information query request carries a target data path in a distributed coordination system;
inquiring a data operation record table according to the target data path to obtain the identity information of a target client, wherein the target client is a client for operating the data of the target data path, and the data operation record table is used for recording the identity information of the client for operating the data in the distributed coordination system and the data path of the operated data;
and sending the identity information of the target client to the sending equipment of the operation information inquiry request.
In one possible implementation manner, the generating process of the data operation record table includes:
acquiring a data operation transaction log generated when the distributed coordination system operates, wherein the data operation transaction log comprises a session identifier corresponding to a client performing data operation and a data path of operated data;
inquiring a session information table according to the session identification included in the data operation transaction log, wherein the session information table is used for recording the session identification corresponding to the current effective connection and the identity information of the client;
and correspondingly storing the inquired identity information of the client and the data path included in the data operation transaction log into the data operation record table.
In one possible implementation manner, the obtaining a transaction log of data operation generated when the distributed coordination system runs includes:
and receiving the data operation transaction log from a server of the distributed coordination system, wherein a log collection tool for collecting the data operation transaction log is operated on the server.
In one possible implementation manner, the generating of the session information table includes:
acquiring a first connection management log generated when the distributed coordination system operates, wherein the first connection management log comprises identity information of a client side establishing connection and a corresponding session identifier;
and correspondingly storing the identity information and the session identification of the client side in the first connection management log into the session information table.
In one possible implementation manner, the generating process of the session information table further includes:
acquiring a second connection management log generated when the distributed coordination system operates, wherein the second connection management log comprises identity information of a client side for closing connection and a corresponding session identifier;
and deleting the identity information and the session identification of the client terminal included in the second connection management log from the session information table.
In one possible implementation, the data operation transaction log further includes at least one of an operation type, an operation time, or a data content of performing the data operation;
correspondingly storing the inquired identity information of the client and the data path included in the data operation transaction log into the data operation record table, wherein the steps of:
and correspondingly storing the inquired identity information of the client, the data path and at least one of the operation type, the operation time or the data content in the data operation transaction log into the data operation record table.
In one possible implementation, the data operation record table and the session information table are stored in a relational database management system.
In one possible implementation, the data operation record table is further configured to record at least one of an operation type, an operation time, or a data content of the operation on the data in the distributed coordination system;
the querying a data operation record table according to the target data path to obtain the identity information of the target client includes:
inquiring the data operation record table according to the target data path to obtain identity information of the target client and at least one of target operation type, target operation time or target data content;
the sending device for sending the identity information of the target client to the operation information query request comprises:
and sending the identity information of the target client and at least one of the target operation type, the target operation time or the target data content to the sending equipment.
In one possible implementation, the identity information of the target client is at least one of an IP address or port information of the target client;
after querying a data operation record table according to the target data path to obtain the identity information of the target client, the method further includes:
determining target user information according to the identity information of the target client and a bastion machine log, wherein the bastion machine log is used for recording user information corresponding to any client;
and sending the target user information to the sending equipment of the operation information inquiry request.
According to a second aspect of the embodiments of the present disclosure, there is provided an information query apparatus, including:
the system comprises a receiving unit, a processing unit and a processing unit, wherein the receiving unit is configured to execute receiving of an operation information query request, and the operation information query request carries a target data path in a distributed coordination system;
the query unit is configured to query a data operation record table according to the target data path to obtain identity information of a target client, the target client is a client operating on data of the target data path, and the data operation record table is used for recording the identity information of the client operating on the data in the distributed coordination system and a data path of the operated data;
a sending unit configured to execute sending of the identity information of the target client to a sending device of the operation information query request.
In one possible implementation, the apparatus further includes a generating unit configured to perform:
acquiring a data operation transaction log generated when the distributed coordination system operates, wherein the data operation transaction log comprises a session identifier corresponding to a client performing data operation and a data path of operated data;
inquiring a session information table according to the session identification included in the data operation transaction log, wherein the session information table is used for recording the session identification corresponding to the current effective connection and the identity information of the client;
and correspondingly storing the inquired identity information of the client and the data path included in the data operation transaction log into the data operation record table.
In one possible implementation, the generating unit is configured to perform:
and receiving the data operation transaction log from a server of the distributed coordination system, wherein a log collection tool for collecting the data operation transaction log is operated on the server.
In one possible implementation, the generating unit is further configured to perform:
acquiring a first connection management log generated when the distributed coordination system operates, wherein the first connection management log comprises identity information of a client side establishing connection and a corresponding session identifier;
and correspondingly storing the identity information and the session identification of the client side in the first connection management log into the session information table.
In one possible implementation, the generating unit is further configured to perform:
acquiring a second connection management log generated when the distributed coordination system operates, wherein the second connection management log comprises identity information of a client side for closing connection and a corresponding session identifier;
and deleting the identity information and the session identification of the client terminal included in the second connection management log from the session information table.
In one possible implementation, the data operation transaction log further includes at least one of an operation type, an operation time, or a data content of performing the data operation;
the generating unit is configured to perform corresponding storage of the identity information of the queried client, the data path included in the data operation transaction log, and at least one of an operation type, an operation time, or a data content in the data operation record table.
In one possible implementation, the data operation record table and the session information table are stored in a relational database management system.
In one possible implementation, the data operation record table is further configured to record at least one of an operation type, an operation time, or a data content of the operation on the data in the distributed coordination system;
the query unit is configured to perform query on the data operation record table according to the target data path to obtain identity information of the target client and at least one of a target operation type, a target operation time or a target data content;
the sending unit is configured to perform sending identity information of the target client and at least one of the target operation type, the target operation time, or the target data content to the sending device.
In one possible implementation, the identity information of the target client is at least one of an IP address or port information of the target client;
the device further comprises:
the determining unit is configured to execute determining target user information according to the identity information of the target client and a bastion machine log, wherein the bastion machine log is used for recording user information corresponding to any client;
the sending unit is further configured to execute sending of the target user information to the sending device of the operation information query request.
According to a third aspect of embodiments of the present disclosure, there is provided a computer device comprising:
one or more processors;
one or more memories for storing the one or more processor-executable instructions;
wherein the one or more processors are configured to execute the instructions to implement the information query method as described in the first aspect or any one of the possible implementations of the first aspect.
According to a fourth aspect of embodiments of the present disclosure, there is provided a storage medium, wherein instructions of the storage medium, when executed by a processor of a computer device, enable the computer device to perform the information query method according to the first aspect or any one of the possible implementations of the first aspect.
According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product, wherein instructions of the computer program product, when executed by a processor of a computer device, enable the computer device to perform the information query method according to the first aspect or any one of the possible implementations of the first aspect.
The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:
after receiving the operation information query request, querying a data operation record table according to a target data path carried in the request, wherein the data operation record table is used for recording the identity information of a client operating data in the distributed coordination system and the data path of the operated data, so that the identity information of the target client corresponding to the target data path can be queried in the data operation record table, and the identity information of the target client can be returned to the sending equipment of the operation information query request. According to the technical scheme, the identity information of the client operating the data in the distributed coordination system is quickly searched according to the data path, so that the fact that the client operates the data of the data path is known, the tracing of the data operation in the distributed coordination system is achieved, the complexity is reduced, and the query efficiency is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.
FIG. 1 is a schematic diagram of an implementation environment of a method for querying information, according to an example embodiment;
FIG. 2 is a flow diagram illustrating a method of querying information in accordance with an exemplary embodiment;
FIG. 3 is a flow diagram illustrating a method of querying information in accordance with an exemplary embodiment;
FIG. 4 is a block diagram illustrating an information query device in accordance with an exemplary embodiment;
FIG. 5 is a block diagram illustrating a computer device 500 in accordance with an exemplary embodiment.
Detailed Description
In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
The user information to which the present disclosure relates may be information authorized by the user or sufficiently authorized by each party.
In order to facilitate a clearer understanding of the technical solutions of the embodiments of the present disclosure, the embodiments of the present disclosure are described with reference to a distributed coordination system.
The distributed coordination system may be a Zookeeper, a c (client)/s (server) architecture, the client may be a program installed on a terminal, the server may be one server, or a server cluster composed of a plurality of servers, or a cloud computing platform or a virtualization center, etc.
When a client of the distributed coordination system is installed on any terminal, a connection can be established between the client and a server of the distributed coordination system, and a write request can be sent to the server, where the write request may be a request for operating data, and the server can respond to the write request to perform corresponding operations on the data, including any type of operations such as creation, modification, or deletion.
Fig. 1 is a schematic diagram of an implementation environment of an information query method according to an exemplary embodiment, and referring to fig. 1, the implementation environment includes a terminal 101 and a server 102.
The terminal 101 is a terminal where a user is located, and the terminal 101 may be a smart phone, a tablet computer, a portable computer, a desktop computer, and the like. The terminal 101 may have a target application installed thereon, and is configured to provide a function of querying operation information of any data path in the distributed coordination system, where the operation information may include identity information of a client operating on data of the any data path. In the process of running the target application, the user may perform an operation on the interface of the target application by the terminal 101, and trigger the terminal 101 to send an operation information query request to the server 102.
The server 102 may be one server, a server cluster composed of a plurality of servers, a cloud computing platform, a virtualization center, or the like. The server 102 is configured to provide a background service for the target application, for example, after the server 102 receives an operation information query request sent by the terminal 101 based on the target application, the server 102 may query operation information of a data path carried by the operation information query request and return the operation information to the terminal 101.
The server 102 may be a server other than a server of a distributed coordination system (Zookeeper).
The terminal 101 may be connected to the server 102 through a wireless network or a wired network, so that data interaction between the terminal 101 and the server 102 is possible.
Fig. 2 is a flowchart illustrating an information query method according to an exemplary embodiment, where the information query method is used in a computer device, as shown in fig. 2, and includes the following steps.
In step S210, an operation information query request is received, where the operation information query request carries a target data path in the distributed coordination system.
In step S220, according to the target data path, a data operation record table is queried to obtain identity information of a target client, where the target client is a client that operates data of the target data path, and the data operation record table is used to record the identity information of the client that operates data in the distributed coordination system and a data path of the operated data.
In step S230, the identity information of the target client is sent to the sending device of the operation information query request.
According to the method provided by the embodiment of the disclosure, after the operation information query request is received, the data operation record table is queried according to the target data path carried in the request, and because the data operation record table is used for recording the identity information of the client operating the data in the distributed coordination system and the data path of the operated data, the identity information of the target client corresponding to the target data path can be queried in the data operation record table, so that the identity information of the target client can be returned to the sending equipment of the operation information query request. According to the technical scheme, the identity information of the client operating the data in the distributed coordination system is quickly searched according to the data path, so that the fact that the client operates the data of the data path is known, the tracing of the data operation in the distributed coordination system is achieved, the complexity is reduced, and the query efficiency is improved.
In one possible implementation, the generating of the data operation record table includes:
acquiring a data operation transaction log generated when the distributed coordination system operates, wherein the data operation transaction log comprises a session identifier corresponding to a client side performing data operation and a data path of operated data;
inquiring a session information table according to the session identification included in the data operation transaction log, wherein the session information table is used for recording the session identification corresponding to the current effective connection and the identity information of the client;
and correspondingly storing the inquired identity information of the client and the data path included in the data operation transaction log into the data operation record table.
In one possible implementation, the obtaining the transaction log of the data operation generated during the operation of the distributed coordination system includes:
the data operation transaction log is received from a server of the distributed coordination system, on which a log collection tool is run for collecting the data operation transaction log.
In one possible implementation, the generating of the session information table includes:
acquiring a first connection management log generated when the distributed coordination system runs, wherein the first connection management log comprises identity information of a client side establishing connection and a corresponding session identifier;
and correspondingly storing the identity information and the session identification of the client terminal included in the first connection management log into the session information table.
In one possible implementation manner, the generating process of the session information table further includes:
acquiring a second connection management log generated when the distributed coordination system operates, wherein the second connection management log comprises identity information of a client side for closing connection and a corresponding session identifier;
and deleting the identity information and the session identification of the client included in the second connection management log from the session information table.
In one possible implementation, the data operation transaction log further includes at least one of an operation type, an operation time, or a data content at which the data operation is performed;
the step of correspondingly storing the inquired identity information of the client and the data path included in the data operation transaction log into the data operation record table includes:
and correspondingly storing the identity information of the inquired client, the data path and at least one of the operation type, the operation time or the data content included in the data operation transaction log into the data operation record table.
In one possible implementation, the data operation record table and the session information table are stored in a relational database management system.
In one possible implementation, the data operation record table is further used for recording at least one of operation type, operation time or data content of the data in the distributed coordination system;
the querying a data operation record table according to the target data path to obtain the identity information of the target client includes:
inquiring the data operation record table according to the target data path to obtain identity information of the target client and at least one item of target operation type, target operation time or target data content;
the sending device for sending the identity information of the target client to the operation information query request includes:
and sending the identity information of the target client and at least one of the target operation type, the target operation time or the target data content to the sending equipment.
In one possible implementation, the identity information of the target client is at least one of an IP address or port information of the target client;
after querying the data operation record table according to the target data path and obtaining the identity information of the target client, the method further includes:
determining target user information according to the identity information of the target client and a bastion machine log, wherein the bastion machine log is used for recording user information corresponding to any client;
and sending the target user information to the sending equipment of the operation information inquiry request.
All the above optional technical solutions may be combined arbitrarily to form the optional embodiments of the present disclosure, and are not described herein again.
The flow shown in fig. 2 is a basic flow of the embodiment of the present disclosure, and a detailed flow of the embodiment of the present disclosure is described below based on the basic flow.
Fig. 3 is a flowchart illustrating an information query method according to an exemplary embodiment, where the information query method is used in a computer device, such as the server 102 in the corresponding embodiment of fig. 1, as shown in fig. 3, and the method includes the following steps.
In step S310, the computer device generates a data operation record table, which is used to record the identity information of the client operating on the data in the distributed coordination system and the data path of the operated data.
Wherein, the distributed coordination system can be Zookeeper. The client may be a client of the distributed coordination system, and the identity information of the client may be at least one of an IP address or port information of the client, that is, at least one of an IP address or port information of a terminal where the client is located. The data path refers to a path (path) of data in the distributed coordination system.
For the generation process of the data operation record table, in one possible implementation manner, the generation process of the data operation record table includes the following steps S311 to S313:
step S311, the computer device obtains a data operation transaction log generated when the distributed coordination system runs, where the data operation transaction log includes a session identifier corresponding to a client performing data operation and a data path of the operated data.
The session identifier (sessionId) may be allocated by a Server (Server) of the distributed coordination system after a Client (Client) of the distributed coordination system successfully establishes a connection with the Server (Server).
In one possible implementation, the first step may include: the data operation transaction log is received from a server of the distributed coordination system, on which a log collection tool is run for collecting the data operation transaction log. By receiving the data operation transaction log sent by the server of the distributed coordination system, the data operation transaction log can be analyzed by machines except the server of the distributed coordination system, the operation on the server of the distributed coordination system is only the transmission log, the core realization is completed on other machines, and the problem that the server of the distributed coordination system is interfered too much and the permission application is carried out can be avoided.
The server of the distributed coordination system can be operated with a log collection tool, and the log collection tool can collect data operation transaction logs generated in the operation process of the distributed coordination system in real time, and the log collection tool can also be called as a log real-time collection tool. The data operation transaction log may be a binary log, and may be parsed by a Zookeeper transaction log parsing tool. The data operation transaction log may record the session identifier and data path corresponding to the client at each data modification, creation, or deletion. Once a new data operation transaction log is collected, the log collection tool may send the currently collected data operation transaction log to another machine, for example, to a computer device executing the method provided by the embodiment of the present disclosure, so that the computer device may obtain the data operation transaction log.
In one possible implementation, the data operation transaction log further includes at least one of an operation type, an operation time, or a data content at which the data operation is performed. The operation type comprises any type of operation of creation, modification or deletion, the operation time refers to the time for operating data, and the data content refers to the content of the operated data. The data operation transaction log comprises at least one of operation type, operation time or data content besides the session identifier and the data path, so that more information can be obtained by analyzing the data operation log.
Step S312, the computer device queries a session information table according to the session identifier included in the data operation transaction log, where the session information table is used to record the session identifier corresponding to the current effective connection and the identity information of the client.
Taking any client of the distributed coordination system as an example, if connection is maintained between the client and the server of the distributed coordination system, the connection is effective connection, and the session identifier corresponding to the effective connection is the session identifier allocated by the server of the distributed coordination system after the connection between the client and the server is successfully established.
After the computer device obtains the data operation transaction log, the session identifier included in the data operation transaction log can be obtained, and then according to the session identifier, the computer device queries in a session information table to query whether the identity information of the client corresponding to the session identifier exists.
In a possible implementation manner, before querying the session information table according to the session identifier included in the data operation transaction log, the method provided by the embodiment of the present disclosure further includes: and analyzing the data operation transaction log through a transaction log analysis tool provided by the distributed coordination system to obtain a session identifier included in the data operation transaction log.
Because the transaction log is generally a binary log according to the operation, the transaction log is analyzed by a transaction log analysis tool provided by the distributed coordination system, and specific contents included in the log can be obtained. The transaction log analysis tool may be a logformat.
For the generation process of the session information table, in one possible implementation manner, the generation process of the session information table includes the following steps a to b:
step a, computer equipment acquires a first connection management log generated when the distributed coordination system runs, wherein the first connection management log comprises identity information of a client side establishing connection and a corresponding session identifier.
Similarly to the step one of obtaining the transaction log of the data operation, the step a may include: the first connection management log is received from a server of the distributed coordination system.
The log collection tool running on the server of the distributed coordination system can collect the connection management log generated when the distributed coordination system runs in real time, wherein the connection management log can be a human-readable text log, and can record the log of connection establishment and connection closing of the client each time, and the log comprises the identity information (at least one of IP address or port information) of the client and the session identifier distributed by the server after the connection is successfully established. Once collecting a new connection management log, the log collection tool may send the currently collected connection management log to another machine, such as a computer device executing the method provided by the embodiment of the present disclosure, so that the computer device may obtain the connection management log.
The connection management log may be a log of connection establishment of the client or a log of connection closing of the client, where the first connection management log in step a is a log of connection establishment of the client, and the second connection management log in step c is a log of connection closing of the client.
And step b, the computer equipment correspondingly stores the identity information and the session identification of the client end in the first connection management log into the session information table.
If the computer device acquires the first connection management log, it indicates that the client establishes a connection with the server of the distributed coordination system at present, and the computer device may acquire the identity information and the session identifier of the client included in the first connection management log and store the identity information and the session identifier in the session information table.
In a possible implementation manner, the generation process of the session information table further includes the following steps c to d:
and c, the computer equipment acquires a second connection management log generated when the distributed coordination system runs, wherein the second connection management log comprises the identity information of the client side for closing the connection and the corresponding session identification.
Similarly to the step one of obtaining the transaction log of the data operation, the step a may include: the second connection management log is received from a server of the distributed coordination system.
And d, the computer equipment deletes the identity information and the session identification of the client terminal included in the second connection management log from the session information table.
If the computer device acquires the second connection management log, it indicates that a client currently closes the connection with the server of the distributed coordination system, and when the client establishes the connection with the server, the computer device may acquire the first connection management log corresponding to the client, and store the identity information of the client and the corresponding session identifier included in the first connection management log corresponding to the client in the session information table.
It is to be understood that the identity information of the client and the corresponding session identifier included in the second connection management log in steps c to d may be different from or the same as the identity information of the client and the corresponding session identifier included in the first connection management log in steps a to b.
The computer device may generate the session information table by performing steps a to b, or steps c to d, or steps a to d a plurality of times. Through the generation process of the session information table, it can be known that the session information table can be an effective session information table, and the session information of the effective session is recorded. When any client is connected effectively, the Zookeeper ensures that the session identifier allocated to the client is not repeatedly allocated to a plurality of clients, so that a session information table for recording the session information of the effective session can be maintained, the session information comprises the session identifier and the identity information of the corresponding client, the session information is written in when the session is established, and the session information is deleted when the session is closed, so that the uniqueness of the identity of the client corresponding to the session identifier in the system can be ensured.
After the computer device generates the session information table, the session information table may be stored. In one possible implementation, the computer device may store the session information table in a relational database management system. The relational database management system may be MySQL, such as storing the session information table in a MySQL database. The session information table is stored persistently by using MySQL for later use, and can be queried by the MySQL without querying all logs, so that the query efficiency is improved. It is understood that the computer device may also store the session information table in other databases, which is not limited by the embodiments of the present disclosure.
Step S313, the computer device correspondingly stores the queried identity information of the client and the data path included in the data operation transaction log into the data operation record table.
If the computer device queries the identity information of the client in the session information table according to the session identifier included in the data operation transaction log, the computer device may combine the queried identity information of the client with the data path in the data operation transaction log and store the combined data path in the data operation transaction log in the data operation record table.
The transaction log further includes at least one of an operation type, an operation time, or a data content for the data operation. In one possible implementation, the third step includes: and correspondingly storing the inquired identity information of the client, the data path and at least one of the operation type, the operation time or the data content included in the data operation transaction log into the data operation record table.
Correspondingly, the data operation record table is further used for recording at least one of operation type, operation time or data content for operating data in the distributed coordination system. The operation type, the operation time or the data content is recorded besides the identity information and the data path of the client side in the data operation record table, so that the content of the operation information is enriched, the query of the operation content, the operation time or the data content can be provided besides the query of the identity information of the client side, and a querier can query more comprehensive operation information.
The server may generate the data operation record table by performing the step one to the step three a plurality of times. After the computer device generates the data operation record table, the data operation record table can be stored. Like the storage of the session information table, in one possible implementation, the server may store the data operation record table in a relational database management system. The data operation record table is stored persistently by using MySQL for later use, and can be queried by the MySQL, so that the query efficiency is improved. It is understood that the computer device may also store the data operation record table in other databases, which is not limited by the embodiment of the present disclosure.
It should be noted that step S310 is an optional step, which is a step that needs to be executed before the information query is performed, and is not required to be executed each time the information query is performed, so that it is ensured that the data operation record table is already generated when the information query is performed.
In step S320, the computer device receives an operation information query request, where the operation information query request carries a target data path in the distributed coordination system.
The operation information query request may be triggered by a user operating on a terminal, for example, the terminal may display an interface for querying operation information of any data path in the distributed coordination system, if the user wants to query operation information of a target data path in the distributed coordination system, the user may input the target data path on the interface, and trigger the terminal to carry the target data path in the operation information query request and send the operation information query request to the computer device, so that the computer device may receive the operation information query request.
In step S330, the computer device queries a data operation record table according to the target data path to obtain identity information of a target client, where the target client is a client that operates data of the target data path, and the data operation record table is used to record the identity information of the client that operates data in the distributed coordination system and a data path of the operated data.
After receiving the operation information query request, the computer device may query in the data operation record table according to a target data path carried in the operation information query request, to obtain the identity information of the client corresponding to the target data path, that is, the identity information of the target client.
In one possible implementation manner, the querying, by the computer device, the data operation record table according to the target data path to obtain the identity information of the target client includes: and inquiring the data operation record table according to the target data path to obtain the identity information of the target client and at least one item of the target operation type, the target operation time or the target data content.
For at least one of the operation type, the operation time, or the data content of the data in the distributed coordination system, the computer device may query, according to the target data path, at least one of the operation type, the operation time, or the data content corresponding to the target data path, that is, at least one of the target operation type, the target operation time, or the target data content, in the data operation record table.
In step S340, the computer device transmits the identity information of the target client to the transmitting device of the operation information query request.
The sending device of the operation information query request may refer to a terminal where a user triggering the operation information query request is located.
After the computer device queries the identity information of the corresponding target client in the data operation record table according to the target data path, the queried identity information of the target client can be used as a response to the operation information query request and returned to the sending device of the operation information query request.
In one possible implementation manner, sending the identity information of the target client to the sending device of the operation information query request includes: and sending the identity information of the target client and at least one of the target operation type, the target operation time or the target data content to the sending equipment.
At least one of the target operation type, the target operation time or the target data content is inquired in the data operation record table according to the target data path by the computer equipment, and when the computer equipment sends the identity information of the target client, at least one of the target operation type, the target operation time or the target data content can be sent at the same time, so that a user of the receiving equipment can know what type of operation is performed on the data of the target data path by the target client, when the operation is performed on the data of the target data path and the data content of the operation, and the comprehensiveness of the operation information is ensured.
In a possible implementation manner, after the computer device queries the data operation record table according to the target data path to obtain the identity information of the target client, the method provided in the embodiment of the present disclosure further includes: determining target user information according to the identity information of the target client and a bastion machine log, wherein the bastion machine log is used for recording user information corresponding to any client; and sending the target user information to the sending equipment of the operation information inquiry request. After the identity information of the client is determined, the specific user is further determined by combining other auxiliary information, so that the more accurate query requirement can be met.
The identity information of the target client is at least one of the IP address or the port information of the target client. The specific client can be determined according to the identity information of the target client, but if a specific user is further determined, the specific user can be determined by combining other auxiliary information, such as the bastion machine log, for example, the computer device can query the bastion machine log according to the identity information of the target client to obtain user information corresponding to the identity information of the target client, that is, the target user information, and then send the queried target user information to the sending device of the operation information query request, so that the user initiating the request can know which user operates the data of the target data path according to the target user information.
It should be noted that, in the embodiment of the present disclosure, the session information table is obtained by analyzing the connection management log generated during the Zookeeper runtime, the data operation object log generated during the Zookeeper runtime is analyzed, the session information table is queried to obtain the data operation record table, and then when the operation information of a certain data path needs to be queried (such as whether the operation information is modified, when the operation information is modified, or which client is modified), the required operation information can be quickly obtained by querying the data operation record table, so that the query efficiency is improved. In some possible embodiments, when operation information of a certain data path needs to be queried, the computer device may also directly query two types of logs generated during the Zookeeper operation, specifically, the data operation object log is parsed through a Zookeeper transaction log parsing tool, modification information of the data path is searched, a corresponding session identifier is obtained, a connection management log is searched through the session identifier, and identity information of a client that creates the session is obtained.
The technical scheme provided by the embodiment of the disclosure can be applied to the following scenes: the Zookeeper stores key data of a certain service, if a person modifies the service through a client of the Zookeeper to cause service failure, when tracing afterwards, the technical scheme can quickly find identity information of the client according to a data path, including at least one item of IP address or port information, and also can find information such as a modification time point, modified data content and the like, and if a responsible person is further found, other auxiliary information can be combined, such as bastion machine logs, so that a final responsible person can be found. The technical scheme enriches the tracing means of the Zookeeper writing request, the main processing work of the scheme is not completed on the Zookeeper server machine, and the influence on the Zookeeper server is reduced.
In the related technology, the identity information of the client is obtained through packet capturing, on one hand, a TCP protocol needs to be analyzed during packet capturing, problems such as disorder, retransmission and the like need to be correctly processed, and after TCP content is restored, a Zookeeper request protocol needs to be analyzed to obtain the identity information of the client, so that the complexity is high. On the other hand, the packet capturing needs to acquire the root authority of the physical machine, and the execution authority is too high; and the packet grabbing program needs to be resident in a physical machine where the Zookeeper server is located to operate, and unpredictable risks are caused to the stability of the Zookeeper.
On one hand, the technical scheme provided by the embodiment of the disclosure avoids the situation that the request recombination and the protocol analysis are too low on the bottom layer, and the processing is carried out on the upper layer. Based on the connection management log and the data operation transaction log generated by the Zookeeper, the identity information of the data modifier is acquired by using a transaction log analysis tool provided by the Zookeeper, so that the implementation complexity is reduced. On the other hand, the operation on the physical machine where the Zookeeper server is located only transmits the log, and the core implementation is completed on other machines, namely, the Zookeeper server is implemented outside the Zookeeper server, so that too much interference and permission application on the Zookeeper server are avoided, and the influence on the Zookeeper is reduced. In another aspect, the MySQL is used for saving the session information and the record of each operation of the data of the client, so that the query efficiency is optimized.
According to the method provided by the embodiment of the disclosure, after the operation information query request is received, the data operation record table is queried according to the target data path carried in the request, and because the data operation record table is used for recording the identity information of the client operating the data in the distributed coordination system and the data path of the operated data, the identity information of the target client corresponding to the target data path can be queried in the data operation record table, so that the identity information of the target client can be returned to the sending equipment of the operation information query request. According to the technical scheme, the identity information of the client operating the data in the distributed coordination system is quickly searched according to the data path, so that the fact that the client operates the data of the data path is known, the tracing of the data operation in the distributed coordination system is achieved, the complexity is reduced, and the query efficiency is improved.
Fig. 4 is a block diagram illustrating an information query device according to an example embodiment. Referring to fig. 4, the apparatus includes a receiving unit 401, an inquiring unit 402, and a transmitting unit 403.
The receiving unit 401 is configured to perform receiving an operation information query request, where the operation information query request carries a target data path in the distributed coordination system;
the query unit 402 is configured to perform query of a data operation record table according to the target data path to obtain identity information of a target client, where the target client is a client that operates on data of the target data path, and the data operation record table is used for recording identity information of clients that operate on data in the distributed coordination system and a data path of the operated data;
the sending unit 403 is configured to perform sending the identity information of the target client to the sending device of the operation information query request.
In one possible implementation, the apparatus further includes a generating unit configured to perform:
acquiring a data operation transaction log generated when the distributed coordination system operates, wherein the data operation transaction log comprises a session identifier corresponding to a client side performing data operation and a data path of operated data;
inquiring a session information table according to the session identification included in the data operation transaction log, wherein the session information table is used for recording the session identification corresponding to the current effective connection and the identity information of the client;
and correspondingly storing the inquired identity information of the client and the data path included in the data operation transaction log into the data operation record table.
In one possible implementation, the generating unit is configured to perform:
the data operation transaction log is received from a server of the distributed coordination system, on which a log collection tool is run for collecting the data operation transaction log.
In one possible implementation, the generating unit is further configured to perform:
acquiring a first connection management log generated when the distributed coordination system runs, wherein the first connection management log comprises identity information of a client side establishing connection and a corresponding session identifier;
and correspondingly storing the identity information and the session identification of the client terminal included in the first connection management log into the session information table.
In one possible implementation, the generating unit is further configured to perform:
acquiring a second connection management log generated when the distributed coordination system operates, wherein the second connection management log comprises identity information of a client side for closing connection and a corresponding session identifier;
and deleting the identity information and the session identification of the client included in the second connection management log from the session information table.
In one possible implementation, the data operation transaction log further includes at least one of an operation type, an operation time, or a data content at which the data operation is performed;
the generating unit is configured to perform corresponding storage of the identity information of the queried client, the data path included in the data operation transaction log, and at least one of an operation type, an operation time, or a data content in the data operation record table.
In one possible implementation, the data operation record table and the session information table are stored in a relational database management system.
In one possible implementation, the data operation record table is further used for recording at least one of operation type, operation time or data content of the data in the distributed coordination system;
the query unit 402 is configured to perform querying the data operation record table according to the target data path, to obtain identity information of the target client and at least one of a target operation type, a target operation time, or a target data content;
the sending unit 403 is configured to perform sending the identity information of the target client and at least one of the target operation type, the target operation time or the target data content to the sending device.
In one possible implementation, the identity information of the target client is at least one of an IP address or port information of the target client;
the device also includes:
the determining unit is configured to execute determining target user information according to the identity information of the target client and a bastion machine log, wherein the bastion machine log is used for recording user information corresponding to any client;
the sending unit 403 is further configured to perform sending the target user information to the sending device of the operation information inquiry request.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Fig. 5 is a block diagram illustrating a computer device 500 according to an exemplary embodiment, where the computer device 500 may have a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 501 and one or more memories 502, where the memory 502 stores at least one instruction, and the at least one instruction is loaded and executed by the processor 501 to implement the information query method provided by the above method embodiments. Certainly, the computer device may further have components such as a wired or wireless network interface, a keyboard, and an input/output interface, so as to perform input and output, and the computer device may further include other components for implementing the functions of the device, which is not described herein again.
In an exemplary embodiment, there is also provided a storage medium comprising instructions, such as a memory 502 comprising instructions, executable by a processor 501 of a computer device 500 to perform the above-described method. Alternatively, the storage medium may be a non-transitory computer readable storage medium, for example, a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, an optical data storage device, and the like.
In an exemplary embodiment, a computer program product is also provided, in which instructions are executable by the processor 501 of the computer device 500 to perform the above-described method.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (10)
1. An information query method, comprising:
receiving an operation information query request, wherein the operation information query request carries a target data path in a distributed coordination system;
inquiring a data operation record table according to the target data path to obtain the identity information of a target client, wherein the target client is a client for operating the data of the target data path, and the data operation record table is used for recording the identity information of the client for operating the data in the distributed coordination system and the data path of the operated data;
and sending the identity information of the target client to the sending equipment of the operation information inquiry request.
2. The information query method according to claim 1, wherein the generating process of the data operation record table comprises:
acquiring a data operation transaction log generated when the distributed coordination system operates, wherein the data operation transaction log comprises a session identifier corresponding to a client performing data operation and a data path of operated data;
inquiring a session information table according to the session identification included in the data operation transaction log, wherein the session information table is used for recording the session identification corresponding to the current effective connection and the identity information of the client;
and correspondingly storing the inquired identity information of the client and the data path included in the data operation transaction log into the data operation record table.
3. The information query method according to claim 2, wherein the generation process of the session information table includes:
acquiring a first connection management log generated when the distributed coordination system operates, wherein the first connection management log comprises identity information of a client side establishing connection and a corresponding session identifier;
and correspondingly storing the identity information and the session identification of the client side in the first connection management log into the session information table.
4. The information query method according to claim 3, wherein the generation process of the session information table further comprises:
acquiring a second connection management log generated when the distributed coordination system operates, wherein the second connection management log comprises identity information of a client side for closing connection and a corresponding session identifier;
and deleting the identity information and the session identification of the client terminal included in the second connection management log from the session information table.
5. The information query method of claim 2, wherein the data operation transaction log further comprises at least one of an operation type, an operation time, or a data content of performing a data operation;
correspondingly storing the inquired identity information of the client and the data path included in the data operation transaction log into the data operation record table, wherein the steps of:
and correspondingly storing the inquired identity information of the client, the data path and at least one of the operation type, the operation time or the data content in the data operation transaction log into the data operation record table.
6. The information query method of claim 1, wherein the data operation record table is further configured to record at least one of an operation type, an operation time, or a data content for operating on data in the distributed coordination system;
the querying a data operation record table according to the target data path to obtain the identity information of the target client includes:
inquiring the data operation record table according to the target data path to obtain identity information of the target client and at least one of target operation type, target operation time or target data content;
the sending device for sending the identity information of the target client to the operation information query request comprises:
and sending the identity information of the target client and at least one of the target operation type, the target operation time or the target data content to the sending equipment.
7. The information query method of claim 1, wherein the identity information of the target client is at least one of an IP address or port information of the target client;
after querying a data operation record table according to the target data path to obtain the identity information of the target client, the method further includes:
determining target user information according to the identity information of the target client and a bastion machine log, wherein the bastion machine log is used for recording user information corresponding to any client;
and sending the target user information to the sending equipment of the operation information inquiry request.
8. An information inquiry apparatus, comprising:
the system comprises a receiving unit, a processing unit and a processing unit, wherein the receiving unit is configured to execute receiving of an operation information query request, and the operation information query request carries a target data path in a distributed coordination system;
the query unit is configured to query a data operation record table according to the target data path to obtain identity information of a target client, the target client is a client operating on data of the target data path, and the data operation record table is used for recording the identity information of the client operating on the data in the distributed coordination system and a data path of the operated data;
a sending unit configured to execute sending of the identity information of the target client to a sending device of the operation information query request.
9. A computer device, comprising:
one or more processors;
one or more memories for storing the one or more processor-executable instructions;
wherein the one or more processors are configured to execute the instructions to implement the information query method of any one of claims 1 to 7.
10. A storage medium in which instructions, when executed by a processor of a computer device, enable the computer device to perform the information query method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010333803.4A CN111552675A (en) | 2020-04-24 | 2020-04-24 | Information query method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010333803.4A CN111552675A (en) | 2020-04-24 | 2020-04-24 | Information query method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111552675A true CN111552675A (en) | 2020-08-18 |
Family
ID=71998339
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010333803.4A Pending CN111552675A (en) | 2020-04-24 | 2020-04-24 | Information query method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111552675A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105786683A (en) * | 2016-03-03 | 2016-07-20 | 四川长虹电器股份有限公司 | Customized log collecting system and method |
| CN106209727A (en) * | 2015-04-29 | 2016-12-07 | 阿里巴巴集团控股有限公司 | A kind of session access method and apparatus |
| CN107480251A (en) * | 2017-08-14 | 2017-12-15 | 福建新大陆软件工程有限公司 | A kind of system for managing data access |
| CN110232057A (en) * | 2019-05-29 | 2019-09-13 | 掌阅科技股份有限公司 | Data rewind method, electronic equipment, storage medium |
| CN110580258A (en) * | 2019-09-20 | 2019-12-17 | 中国银行股份有限公司 | big data free query method and device |
| CN111026616A (en) * | 2019-12-24 | 2020-04-17 | 北京奇艺世纪科技有限公司 | Information processing method and device, electronic equipment and storage medium |
-
2020
- 2020-04-24 CN CN202010333803.4A patent/CN111552675A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106209727A (en) * | 2015-04-29 | 2016-12-07 | 阿里巴巴集团控股有限公司 | A kind of session access method and apparatus |
| CN105786683A (en) * | 2016-03-03 | 2016-07-20 | 四川长虹电器股份有限公司 | Customized log collecting system and method |
| CN107480251A (en) * | 2017-08-14 | 2017-12-15 | 福建新大陆软件工程有限公司 | A kind of system for managing data access |
| CN110232057A (en) * | 2019-05-29 | 2019-09-13 | 掌阅科技股份有限公司 | Data rewind method, electronic equipment, storage medium |
| CN110580258A (en) * | 2019-09-20 | 2019-12-17 | 中国银行股份有限公司 | big data free query method and device |
| CN111026616A (en) * | 2019-12-24 | 2020-04-17 | 北京奇艺世纪科技有限公司 | Information processing method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110149361B (en) | Internet business operation process backtracking method, system and device | |
| CN105447046A (en) | Distributed system data consistency processing method, device and system | |
| US10387370B2 (en) | Collecting test results in different formats for storage | |
| CN107872713A (en) | Short processing system for video, method and device | |
| US20220052976A1 (en) | Answer text processing methods and apparatuses, and key text determination methods | |
| JP4627539B2 (en) | Load test system, load test data creation method, and program thereof | |
| CN106251114B (en) | Method and device for realizing approval in application | |
| CN109829640A (en) | Recognition methods, device, computer equipment and the storage medium of enterprise's default risk | |
| US20190197140A1 (en) | Automation of sql tuning method and system using statistic sql pattern analysis | |
| CN112506969A (en) | BMC address query method, system, equipment and readable storage medium | |
| CN112165451A (en) | APT attack analysis method, system and server | |
| WO2022057525A1 (en) | Method and device for data retrieval, electronic device, and storage medium | |
| CN105184559B (en) | A kind of payment system and method | |
| CN105812432A (en) | Cloud file processing method and device | |
| CN112148920B (en) | Data management method | |
| CN111026637B (en) | Method and device for positioning problem code | |
| CN106982147B (en) | Communication monitoring method and device for Web communication application | |
| CN111552675A (en) | Information query method and device, computer equipment and storage medium | |
| CN108540471B (en) | Mobile application network traffic clustering method, computer readable storage medium and terminal | |
| CN115098738A (en) | Service data extraction method and device, storage medium and electronic equipment | |
| CN114625773A (en) | Financial data construction method and system based on business data | |
| CN114817256A (en) | Quick unified storage system of thing networking | |
| CN116451191A (en) | Information auditing method, device, electronic equipment and computer readable storage medium | |
| CN113778709A (en) | Interface calling method, device, server and storage medium | |
| CN108667685B (en) | Mobile application network flow clustering device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200818 |