CN109388536B - Data collection method and device - Google Patents
Data collection method and device Download PDFInfo
- Publication number
- CN109388536B CN109388536B CN201710665614.5A CN201710665614A CN109388536B CN 109388536 B CN109388536 B CN 109388536B CN 201710665614 A CN201710665614 A CN 201710665614A CN 109388536 B CN109388536 B CN 109388536B
- Authority
- CN
- China
- Prior art keywords
- program
- data
- business service
- output
- collecting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3068—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data format conversion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0751—Error or fault detection not based on redundancy
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3055—Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/466—Transaction processing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Stored Programmes (AREA)
- Monitoring And Testing Of Exchanges (AREA)
Abstract
The invention discloses a method and a device for collecting data, and relates to the technical field of computers. One embodiment of the method comprises: receiving a business service request, and configuring a data variable output by a collecting program and an output data content template; executing the collection program to record the data read by the collection program and executed by the business service program; and calling a template engine to format and output the recorded data. The implementation mode can solve the problems of low efficiency, low flexibility and inconvenient maintenance of the existing data collection.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a data collection method and device.
Background
At present, the fault condition must be handled in time during the running process of the application program, so that the normal operation and user experience of the service can be ensured, so that the fault point needs to be found and located at the first time, and the most important purpose for locating the fault is to master the running state of the program at any time, so that the capability of quickly and flexibly collecting the data during the running of the program is urgently needed. When data in the running process of the existing collection application is collected, data are collected at positions where faults may occur, the collected data are output according to a format agreed in advance, and meanwhile, when missing important data are found not to be reported, a collection program is rewritten and is released online.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art: in the prior art, data is output through program hard coding, so that a fault point without coded output data cannot be positioned when a fault occurs, the problem needs to be solved online, and the efficiency is low. Moreover, the content format of the program output data is fixed and inflexible. In addition, the data output code is coupled with the service code, and the maintenance is difficult.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and an apparatus for data collection, which can solve the problems of low efficiency, low flexibility and inconvenient maintenance of the existing data collection.
To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a data collection method including: receiving a business service request, and configuring a data variable output by a collecting program and an output data content template; executing the collection program to record the data read by the collection program and executed by the business service program; and calling a template engine to format and output the recorded data.
Optionally, before the configuring and collecting the data variables output by the program and the data content templates output by the program, the configuring and collecting method includes: and storing the characteristic data in the business service request in a thread local variable.
Optionally, the configuring and collecting the data variables output by the program and the data content templates output by the program includes: configuring data variables output by a collection program and an output data content template according to the execution process of a business service program; wherein, the collecting program is implanted in the business service program in advance.
Optionally, before the executing the collection program, the method includes: and determining whether a collection program pre-implanted into the business service program is in an open state.
Optionally, before the executing the collection program, the method includes: and determining and recording initial data or/and output data read by the collection program in the execution process of the business service program.
Optionally, the executing the collection program to record data read by the collection program and executed by the business service program includes: when determining to record the initial data read by the collecting program in the execution process of the business service program, executing the collecting program before executing the business service program, and recording the characteristic data in the business service request in the thread local variable read by the collecting program; when determining to record the output data read by the collecting program in the execution process of the business service program, executing the collecting program after executing the business service program, and recording the data read by the collecting program and output by the business service program; when determining to record the initial data and the output data read by the collecting program in the execution process of the business service program, executing the collecting program once before and after the business service program is executed, so as to record the characteristic data and the output data in the business service request in the thread local variables read by the collecting program.
In addition, according to an aspect of the embodiments of the present invention, there is provided an apparatus for data collection, including a configuration module, configured to receive a service request, configure data variables output by a collection program and data content templates output by the collection program; the reading module is used for executing the collection program so as to record the data, read by the collection program, executed by the business service program; and the output module is used for calling the template engine and formatting the recorded data.
Optionally, before the configuration module configures the data variables output by the collection program and the data content templates output by the collection program, the configuration module is further configured to: and storing the characteristic data in the business service request in a thread local variable.
Optionally, the configuring module configures the data variables output by the collecting program and the data content templates output by the collecting program, and includes: configuring data variables output by a collection program and an output data content template according to the execution process of a business service program; wherein, the collecting program is implanted in the business service program in advance.
Optionally, before the reading module executes the collection program, the reading module is further configured to: and determining whether a collection program pre-implanted into the business service program is in an open state.
Optionally, before the reading module executes the collection program, the reading module is further configured to: and determining and recording initial data or/and output data read by the collection program in the execution process of the business service program.
Optionally, the reading module executes the collection program to record data, read by the collection program, executed by the business service program, and the data includes: when determining to record the initial data read by the collecting program in the execution process of the business service program, executing the collecting program before executing the business service program, and recording the characteristic data in the business service request in the thread local variable read by the collecting program; when determining to record the output data read by the collecting program in the execution process of the business service program, executing the collecting program after executing the business service program, and recording the data read by the collecting program and output by the business service program; when determining to record the initial data and the output data read by the collecting program in the execution process of the business service program, executing the collecting program once before and after the business service program is executed, so as to record the characteristic data and the output data in the business service request in the thread local variables read by the collecting program.
According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any of the embodiments described above.
According to another aspect of the embodiments of the present invention, there is also provided a computer readable medium, on which a computer program is stored, which when executed by a processor implements the method of any of the above embodiments.
One embodiment of the above invention has the following advantages or benefits: the technical problems of low efficiency, low flexibility and inconvenient maintenance of the existing data collection are solved by completely configuring the content and the formatting treatment of the data output of the collection program, so that the output points of the collection program can be increased or reduced at any time, the format of the data output can be adjusted at any time, and the technical effect of rewriting the collection program is not needed.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of a main flow of a method of data collection according to an embodiment of the invention;
FIG. 2 is a schematic view of a main flow of a method of data collection according to a referential embodiment of the present invention;
FIG. 3 is a schematic diagram of the main modules of an apparatus for data collection according to an embodiment of the present invention;
FIG. 4 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 5 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.
Detailed Description
Exemplary embodiments of the invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a method of data collection according to an embodiment of the present invention, as shown in fig. 1, the method of data collection including:
step S101, receiving a business service request, and configuring data variables output by a collection program and an output data content template.
By way of example, the characteristic data in the service request may include a system ip address for performing the request, a name of an interface of the request, interface parameters of the request, an account number of the requesting user, a rating of the requesting user, and geographic data of the requesting user. Further, after receiving a business service request, feature data in the business service request may be stored in thread local variables. The feature data in the business service request is stored in the thread local variable, so that when the collection program needs to read the initial data of the business service program, the feature data in the thread local variable can be directly read. Meanwhile, due to the fact that thread local variables are used, the cost of transmission among all business service programs is saved by the characteristic data, the implementation of the characteristic data is hidden for the business service programs, and decoupling of the programs is facilitated.
Preferably, the configuration of the data variables output by the collection program and the data content template output by the collection program can be performed according to the execution process of the business service program. The data content template refers to an output data format. And the collection program is pre-embedded in the business service program. That is, the collection program is configured according to data executed by the business service program in order to be able to collect data of the business service program. Therefore, the data variables output by the collection program and the data content templates output by the collection program can be flexibly configured according to the execution process of the business service program.
Preferably, the AOP Around is used to plant a data collection procedure before and after the execution of the business service procedure. The AOP is an abbreviation of Aspect ordered Programming, and can realize the unified maintenance of program functions through a precompilation mode and a running-period dynamic proxy. In the invention, each part of the business logic can be isolated by utilizing the AOP, so that the coupling degree between each part of the business logic is reduced, the reusability of a program is improved, and the development efficiency is improved.
In another preferred embodiment, when configuring information (output data variables and output data content templates) for the collection program, the data specified by template output when executing the method name method in class ClassName may be configured, and the switch control (on state or off state) may be performed.
Step S102, executing a collection program to record the data read by the collection program and executed by the business service program.
Preferably, before executing the collection program, it is determined whether the collection program pre-embedded in the business service program is in an open state. And according to the judgment result, if the mobile terminal is in the open state, directly executing the collection program, and if the mobile terminal is not in the open state, exiting the process or setting the collection program to be in the open state.
Further, the collection program may be caused to read initial data or/and output data during execution of the business service program. That is, the initial data or/and output data of the execution of the business service program read by the collection program may be recorded.
Furthermore, if the initial data in the execution process of the business service program is read, before the business service program is executed, a collecting program is executed, and the characteristic data in the business service request in the thread local variable read by the collecting program is recorded. And if the output data in the execution process of the business service program is read, executing the collection program after the business service program is executed so as to record the data read by the collection program and output by the business service program. If the initial data and the output data before and after the business service program is executed need to be read, the collection program is executed once before and after the business service program is executed respectively so as to record the characteristic data and the output data in the business service request in the thread local variable read by the collection program.
Step S103, calling a template engine, and formatting and outputting the recorded data.
The template engine does not belong to a specific technical field, and is a cross-domain and cross-platform concept. The invention adopts the template engine to separate the interface and the data of the program, greatly improves the development efficiency, and makes code reuse easier due to good design.
According to the various embodiments described above, it can be seen that the present invention realizes that the output point of the collection program can be increased or decreased at any time and the format of the data output can be adjusted at any time without rewriting the collection program by fully configuring the contents of the data output of the collection program and the formatting process. In addition, the collection program is implanted before and after the business service program, so that the collection program and the business service program are completely isolated.
Fig. 2 is a schematic diagram of a main flow of a method of data collection according to a referenceable embodiment of the present invention, which may include:
step S201, receiving a service request, and storing the feature data in the service request in a thread local variable.
In an embodiment, the feature data in the service request may include a system ip address for performing the request, a name of an interface of the request, interface parameters of the request, an account number of the requesting user, a rating of the requesting user, and geographic data of the requesting user.
It is worth to be noted that, because thread local variables are used, the feature data saves the cost of transmission among the business service programs, and hides the implementation of the part of the business service programs, thereby facilitating the decoupling of the programs.
Step S202, determining whether a collection program pre-embedded in the business service program is in an open state, if so, executing step S203, otherwise, exiting the process.
Preferably, the AOP Around manner is used to plant a data collection procedure before and after execution of the business service procedure. The AOP is an abbreviation of Aspect ordered Programming, and can realize the unified maintenance of program functions through a precompilation mode and a running-period dynamic proxy. In the invention, each part of the business logic can be isolated by utilizing the AOP, so that the coupling degree between each part of the business logic is reduced, the reusability of a program is improved, and the development efficiency is improved.
It should be noted that, in this embodiment, when the collection program is in the off state, it indicates that the data collection operation is not required, and the process may be exited. However, sometimes the collection program is in the off state, which does not mean that the data collection operation is not required, but the data collection operation is required without starting the collection program, in which case the data collection program may be started first, and then step S203 is executed.
Step S203, according to the execution process of the business service program, configuring the data variables output by the collection program and the data content template output by the collection program. The data content template refers to an output data format.
It can be seen that, the data variables output by the collecting program and the data content templates output by the collecting program can be flexibly configured according to the execution process of the business service program. That is, if it is necessary to change the output data variables and the output data content templates, it is only necessary to modify the configuration information of the collection program (the output data variables and the output data content templates). Therefore, the problem that the collecting program needs to be rewritten when the data variable output by the collecting program is missing can not occur.
Further, when configuring information (output data variables and output data content templates) for the collection program, data specified by output template when executing the methodName method in class ClassName may be configured, and on-off control (on state or off state) may be performed.
Step S204, executing a collecting program to record the characteristic data in the business service request in the thread local variable read by the collecting program.
In step S204, initial input data is recorded before executing the business service program. Further, the start time (begintime) of the service program call can be recorded.
It should be noted that, the collection program reads the feature data in the business service request in the thread local variables according to the configuration information (the output data variables and the output data content templates) in step S203. That is, what data variables are output is specified in the configuration information, and the collector reads what data variables.
Step S205, executing the business service program until step S206 is completed.
In an embodiment, the collection procedure is executed once before and after the business service procedure is executed.
Step S206, the collection program is executed again to record the data read by the collection program and output by the business service program.
Wherein, the collecting program reads the data output by the business service program according to the configuration information (the output data variable and the output data content template) in step S203. That is, what data variables are output is specified in the configuration information, and the collector reads what output data. Preferably, a completion time (end) of executing the business service program, whether an exception (success) occurs during execution, and a return value (return) may be recorded.
Step S207, call the template engine, and format the output recorded data.
In addition, the present invention may refer to the implementation of the data collection method in the embodiment, which has been described in detail in the above data collection method, and therefore, the repeated content will not be described again.
According to the above-mentioned referenced embodiments, it can be seen that the data collection method of the present invention completely decouples the collection program and the service program, realizes complete separation of the two, and is more convenient for later-stage program maintenance. Moreover, the content of the collected program data output and the formatting process realize complete configuration, increase or decrease of the output points of the collected program and format change are realized without writing any code, and the program development efficiency and the flexibility of the collected program are improved.
Fig. 3 is a data collection apparatus 300 according to an embodiment of the present invention, as shown in fig. 3, including a configuration module 301, a reading module 302, and an output module 303. The configuration module 301 receives a service request, and configures the data variables output by the collection program and the data content templates output by the collection program. Then, the reading module 302 executes the collection program to record the data read by the collection program and executed by the business service program. Finally, the output module 303 invokes the template engine to format the output recorded data.
By way of example, the characteristic data in the service request may include a system ip address for performing the request, a name of an interface of the request, interface parameters of the request, an account number of the requesting user, a rating of the requesting user, and geographic data of the requesting user. Further, the configuration module 301 may store the feature data in the business service request in the thread local variable after receiving the business service request.
Preferably, the configuration module 301 configures the data variables output by the collection program and the data content templates output by the collection program according to the execution process of the business service program. The data content template refers to an output data format. And the collection program is pre-embedded in the business service program. That is, the collection program is configured according to data executed by the business service program in order to be able to collect data of the business service program. Therefore, the data variables output by the collection program and the data content templates output by the collection program can be flexibly configured according to the execution process of the business service program.
Preferably, the configuration module 301 uses the AOP Around to plant a data collection procedure before and after the business service procedure is executed.
In another preferred embodiment, when the configuration module 301 configures information (output data variables and output data content templates) for the collection program, it may configure data specified by template output when executing the methodName method in class ClassName, and may perform switch control (on state or off state).
As another embodiment, the reading module 302 needs to determine whether the collection program pre-embedded in the business service program is in an open state before executing the collection program. And according to the judgment result, if the collection program is in the open state, the collection program is directly executed, and if the collection program is not in the open state, the flow can be exited or the collection program is set to be in the open state.
Further, the reading module 302 can enable the collecting program to read initial data or/and output data in the process of executing the business service program. That is, the initial data or/and output data of the execution of the business service program read by the collection program may be recorded.
Further, if the reading module 302 is to read initial data in the process of executing the business service program, before executing the business service program, executing a collecting program, and recording the characteristic data in the business service request in the thread local variable read by the collecting program. If the reading module 302 is used for reading output data in the execution process of the business service program, after the business service program is executed, the collection program is executed again to record the data read by the collection program and output by the business service program. If the reading module 302 needs to read the initial data and the output data before and after the execution of the business service program, the collecting program is executed once before and after the execution of the business service program, so as to record the feature data and the output data in the business service request in the thread local variables read by the collecting program.
It should be noted that, in the implementation of the data collection device of the present invention, the data collection method is described in detail above, and therefore, the repeated content is not described again.
Fig. 4 illustrates an exemplary system architecture 400 to which the method of data collection or the apparatus of data collection of embodiments of the present invention may be applied.
As shown in fig. 4, the system architecture 400 may include terminal devices 401, 402, 403, a network 404, and a server 405. The network 404 serves as a medium for providing communication links between the terminal devices 401, 402, 403 and the server 405. Network 404 may include various types of connections, such as wire, wireless communication links, or fiber optic cables, to name a few.
A user may use terminal devices 401, 402, 403 to interact with a server 405 over a network 404 to receive or send messages or the like. The terminal devices 401, 402, 403 may have installed thereon various communication client applications, such as shopping-like applications, web browser applications, search-like applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 401, 402, 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 405 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the terminal devices 401, 402, 403. The backend management server may analyze and perform other processing on the received data such as the product information query request, and feed back a processing result (for example, target push information, product information — just an example) to the terminal device.
It should be noted that the method for collecting data provided in the embodiment of the present invention is generally executed by the server 405, and may also be executed on the terminal devices 401, 402, and 403 in the public place, and accordingly, the apparatus for collecting data is generally provided in the server 405, and may also be executed on the terminal devices 401, 402, and 403 in the public place.
It should be understood that the number of terminal devices, networks, and servers in fig. 4 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 5, shown is a block diagram of a computer system 500 suitable for use with a terminal device implementing an embodiment of the present invention. The terminal device shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 5, the computer system 500 includes a Central Processing Unit (CPU)501 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM503, various programs and data necessary for the operation of the system 500 are also stored. The CPU 501, ROM 502, and RAM503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The driver 510 is also connected to the I/O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 511. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 501.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes a configuration module, a read module, and an output module, where the names of the modules do not in some cases constitute a limitation on the modules themselves.
As another aspect, the present invention also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be separate and not assembled into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: receiving a business service request, and configuring a data variable output by a collecting program and an output data content template; executing the collection program to record the data read by the collection program and executed by the business service program; and calling a template engine to format and output the recorded data.
According to the technical scheme of the embodiment of the invention, the content of data output and the formatting process of the collecting program can be completely configured, so that the technical problems of low efficiency and flexibility and inconvenient maintenance of the existing data collection are solved, the output points of the collecting program can be increased or reduced at any time, the format of the data output can be adjusted at any time, and the technical effect of rewriting the collecting program is not needed.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method of data collection, comprising:
receiving a business service request, and configuring a data variable output by a collecting program and an output data content template;
the executing the collection program to record the data read by the collection program and executed by the business service program comprises the following steps: when determining to record the initial data read by the collecting program in the execution process of the business service program, executing the collecting program before executing the business service program, and recording the characteristic data in the business service request in the thread local variable read by the collecting program; when determining to record the output data read by the collecting program in the execution process of the business service program, executing the collecting program after executing the business service program, and recording the data read by the collecting program and output by the business service program; when determining to record the initial data and the output data read by the collecting program in the execution process of the business service program, executing the collecting program once before and after the business service program is executed so as to record the characteristic data and the output data in the business service request in the thread local variable read by the collecting program;
and calling a template engine to format and output the recorded data.
2. The method of claim 1, wherein the configuring step comprises, before the collecting the data variables output by the program and the data content templates output by the program:
and storing the characteristic data in the business service request in a thread local variable.
3. The method of claim 1 or 2, wherein the configuring and collecting the data variables output by the program and the data content templates output by the program comprises:
configuring data variables output by a collection program and an output data content template according to the execution process of a business service program; wherein, the collecting program is implanted in the business service program in advance.
4. The method of claim 1, wherein prior to executing the collection procedure, comprising:
and determining whether a collection program pre-implanted into the business service program is in an open state.
5. An apparatus for data collection, comprising:
the configuration module is used for receiving the business service request, and configuring the data variables output by the collection program and the data content template output by the collection program;
the reading module is used for executing the collection program, so as to record the data read by the collection program and executed by the business service program, and the data comprises: when determining to record the initial data read by the collecting program in the execution process of the business service program, executing the collecting program before executing the business service program, and recording the characteristic data in the business service request in the thread local variable read by the collecting program; when determining to record the output data read by the collecting program in the execution process of the business service program, executing the collecting program after executing the business service program, and recording the data read by the collecting program and output by the business service program; when determining to record the initial data and the output data read by the collecting program in the execution process of the business service program, executing the collecting program once before and after the business service program is executed so as to record the characteristic data and the output data in the business service request in the thread local variable read by the collecting program;
and the output module is used for calling the template engine and formatting the recorded data.
6. The apparatus of claim 5, wherein the configuration module, prior to collecting the data variables output by the program and the data content templates output, is further configured to:
and storing the characteristic data in the business service request in a thread local variable.
7. The apparatus of claim 5 or 6, wherein the configuration module is configured to collect data variables output by the program and data content templates output by the program, and comprises:
configuring data variables output by a collection program and an output data content template according to the execution process of a business service program; wherein, the collecting program is implanted in the business service program in advance.
8. The apparatus of claim 5, wherein before the reading module executes the collection procedure, the reading module is further configured to:
and determining whether a collection program pre-implanted into the business service program is in an open state.
9. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-4.
10. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710665614.5A CN109388536B (en) | 2017-08-07 | 2017-08-07 | Data collection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710665614.5A CN109388536B (en) | 2017-08-07 | 2017-08-07 | Data collection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109388536A CN109388536A (en) | 2019-02-26 |
| CN109388536B true CN109388536B (en) | 2022-06-07 |
Family
ID=65413512
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710665614.5A Active CN109388536B (en) | 2017-08-07 | 2017-08-07 | Data collection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109388536B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002149551A (en) * | 2000-11-08 | 2002-05-24 | Nec Software Hokkaido Ltd | Server management system |
| CN103399908A (en) * | 2013-07-30 | 2013-11-20 | 北京北纬通信科技股份有限公司 | Method and system for fetching business data |
| CN106648564A (en) * | 2015-10-30 | 2017-05-10 | 阿里巴巴集团控股有限公司 | Business data acquisition method and apparatus |
| CN106681882A (en) * | 2015-11-06 | 2017-05-17 | 上海瑞致软件有限公司 | IT-service concentrated monitoring and managing system based on Apriori algorithm |
| CN106919684A (en) * | 2017-03-02 | 2017-07-04 | 百度在线网络技术（北京）有限公司 | Data export method, device, equipment and storage medium |
-
2017
- 2017-08-07 CN CN201710665614.5A patent/CN109388536B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002149551A (en) * | 2000-11-08 | 2002-05-24 | Nec Software Hokkaido Ltd | Server management system |
| CN103399908A (en) * | 2013-07-30 | 2013-11-20 | 北京北纬通信科技股份有限公司 | Method and system for fetching business data |
| CN106648564A (en) * | 2015-10-30 | 2017-05-10 | 阿里巴巴集团控股有限公司 | Business data acquisition method and apparatus |
| CN106681882A (en) * | 2015-11-06 | 2017-05-17 | 上海瑞致软件有限公司 | IT-service concentrated monitoring and managing system based on Apriori algorithm |
| CN106919684A (en) * | 2017-03-02 | 2017-07-04 | 百度在线网络技术（北京）有限公司 | Data export method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109388536A (en) | 2019-02-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109995801B (en) | Message transmission method and device | |
| CN108874825B (en) | Abnormal data verification method and device | |
| CN111427701A (en) | Workflow engine system and business processing method | |
| CN111460129A (en) | Method and device for generating identification, electronic equipment and storage medium | |
| CN112596920A (en) | Message processing method and device, electronic equipment and storage medium | |
| CN116737662A (en) | Method, device, electronic equipment and storage medium for processing business data | |
| CN112948138A (en) | Method and device for processing message | |
| CN112685481A (en) | Data processing method and device | |
| CN109388536B (en) | Data collection method and device | |
| CN114070889B (en) | Configuration method, traffic forwarding device, storage medium, and program product | |
| CN113722007B (en) | Configuration method, device and system of VPN branch equipment | |
| CN113760487B (en) | Service processing method and device | |
| CN113779122B (en) | Method and device for exporting data | |
| CN110851194A (en) | Method and device for acquiring code for realizing new interface | |
| CN112099841A (en) | Method and system for generating configuration file | |
| CN113779018A (en) | Data processing method and device | |
| CN113760693A (en) | Method and apparatus for local debugging of microservice systems | |
| CN110858240A (en) | Front-end module loading method and device | |
| CN104268205A (en) | Method and device for obtaining information of installed applications in IOS system | |
| CN112311833B (en) | Data updating method and device | |
| CN117667289A (en) | Loading rendering method and device of application component | |
| CN114500485A (en) | Data processing method and device | |
| CN114880292A (en) | File classification method and device | |
| CN113778969A (en) | Log processing method and device | |
| CN115481087A (en) | Method and device for managing application |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |