CN115422012A - Data processing method and device, electronic equipment and computer readable medium - Google Patents
Data processing method and device, electronic equipment and computer readable medium Download PDFInfo
- Publication number
- CN115422012A CN115422012A CN202211192062.8A CN202211192062A CN115422012A CN 115422012 A CN115422012 A CN 115422012A CN 202211192062 A CN202211192062 A CN 202211192062A CN 115422012 A CN115422012 A CN 115422012A
- Authority
- CN
- China
- Prior art keywords
- terminal
- log file
- data processing
- standard error
- key
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 27
- 238000013515 script Methods 0.000 claims abstract description 41
- 238000012423 maintenance Methods 0.000 claims abstract description 38
- 238000012545 processing Methods 0.000 claims abstract description 37
- 238000012216 screening Methods 0.000 claims abstract description 27
- 238000000034 method Methods 0.000 claims description 36
- 238000004590 computer program Methods 0.000 claims description 19
- 230000006870 function Effects 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 18
- 230000003993 interaction Effects 0.000 claims description 12
- 230000008571 general function Effects 0.000 claims description 9
- 230000002688 persistence Effects 0.000 claims description 8
- 238000007405 data analysis Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 15
- 238000004891 communication Methods 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 230000036541 health Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000006837 decompression Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45591—Monitoring or debugging support
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Quality & Reliability (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application discloses a data processing method, a data processing device, electronic equipment and a computer readable medium, which relate to the technical field of data analysis, and one specific embodiment comprises the steps of responding to a received data processing request, obtaining a corresponding terminal identifier, and then calling a batch terminal operation and maintenance tool to send a target script execution instruction to a terminal corresponding to the terminal identifier; receiving execution result information returned by the terminal, and further persistently generating a tracking log file; and screening the tracking log file based on the preset identification to obtain and output a standard error log file. The batch operation and maintenance speed can be accelerated, and the working efficiency is improved.
Description
Technical Field
The present application relates to the field of data analysis technologies, and in particular, to a data processing method and apparatus, an electronic device, and a computer-readable medium.
Background
At present, with the increase of the number of operation and maintenance virtual machine devices, how to automatically complete related operations on a designated cluster machine in batches is a difficult problem, a remote host can be logged in through an SSH protocol, but when the remote host logs in, a user name and a password are required, and SSH key exchange is adopted to solve the problem of password login, so that password-free login is realized. The interaction problem encountered in the machine login process is processed by adopting expect, wherein the expect is a command special for processing interaction, and the interaction process can be written in a script by virtue of the expect so as to be automatically completed. And (4) completing relevant instruction operation in a specified cluster by configuring the host list. But when the operation and maintenance environment is very crude and the same person needs to operate multiple terminals simultaneously, the working efficiency is low.
In the process of implementing the present application, the inventor finds that at least the following problems exist in the prior art:
when the operation and maintenance environment is very crude and the same person needs to operate multiple terminals at the same time, the working efficiency is low.
Disclosure of Invention
In view of this, embodiments of the present application provide a data processing method and apparatus, an electronic device, and a computer readable medium, which can solve the existing problem of low work efficiency when the operation and maintenance environment is very crude and a same person needs to operate multiple terminals at the same time.
To achieve the above object, according to an aspect of an embodiment of the present application, there is provided a data processing method including:
responding to the received data processing request, acquiring a corresponding terminal identifier, and calling a batch terminal operation and maintenance tool to send a target script execution instruction to a terminal corresponding to the terminal identifier;
receiving execution result information returned by the terminal, and further persistently generating a tracking log file;
and screening the tracking log file based on the preset identification to obtain and output a standard error log file.
Optionally, before sending the target script execution instruction to the terminal corresponding to the terminal identifier, the method further includes:
executing a key exchange process; wherein,
the key exchange process comprises the following steps:
calling a target program, loading a preset general function library and an interoperation function library, and further defining an operation environment variable and a log variable;
and analyzing the target character string, initializing a global variable, receiving a host key of the terminal in batches, and registering the secret key in batches.
Optionally, the bulk receiving terminal host key includes:
and receiving the host names and the IP addresses of the terminals in batches.
Optionally, initializing global variables, including:
and acquiring a local task group directory, a user-defined full-scale host list file, a local private key file and a private key of the private key file, and initializing according to preset conditions.
Optionally, obtaining a standard error log file comprises:
and merging the screened tracking log files to obtain a standard error log file.
Optionally, the receiving of the execution result information returned by the terminal includes:
and remotely connecting the terminal in a text or graphic interface mode, and acquiring execution result information obtained after the terminal executes the target script execution instruction.
Optionally, screening the trace log file based on a preset identifier to obtain a standard error log file and outputting the standard error log file includes:
and searching whether a preset identifier exists in each row of the tracking log file, and generating a standard error log file based on the row with the preset identifier.
In addition, the present application also provides a data processing apparatus, including:
the interaction unit is configured to respond to the received data processing request, acquire the corresponding terminal identification and further call the batch terminal operation and maintenance tool to send the target script execution instruction to the terminal corresponding to the terminal identification;
the persistence unit is configured to receive the execution result information returned by the terminal and further persistently generate a tracking log file;
and the screening unit is configured to screen the tracking log file based on the preset identification so as to obtain a standard error log file and output the standard error log file.
Optionally, the apparatus further comprises a key exchange unit configured to:
executing a key exchange process; wherein,
the key exchange process is configured to:
calling a target program, loading a preset general function library and an interoperation function library, and further defining an operation environment variable and a log variable;
and analyzing the target character string, initializing the global variable, receiving the host key of the terminal in batches, and performing batch key registration.
Optionally, the key exchange unit is further configured to:
and receiving the host names and the IP addresses of the terminals in batches.
Optionally, the key exchange unit is further configured to:
and acquiring a local task group directory, a user-defined full-scale host list file, a local private key file and a private key of the private key file, and initializing according to preset conditions.
Optionally, the screening unit is further configured to:
and merging the screened tracking log files to obtain a standard error log file.
Optionally, the persistence unit is configured to:
and remotely connecting the terminal in a text or graphic interface mode, and acquiring execution result information obtained after the terminal executes the target script execution instruction.
Optionally, the screening unit is further configured to:
and searching whether a preset identifier exists in each row of the tracking log file, and generating a standard error log file based on the row with the preset identifier.
In addition, the present application also provides an electronic device for data processing, including: one or more processors; a storage device for storing one or more programs which, when executed by one or more processors, cause the one or more processors to implement the data processing method as described above.
In addition, the present application also provides a computer readable medium, on which a computer program is stored, which when executed by a processor implements the data processing method as described above.
To achieve the above object, according to still another aspect of embodiments of the present application, there is provided a computer program product.
A computer program product according to an embodiment of the present application includes a computer program, and when the computer program is executed by a processor, the computer program implements the data processing method according to an embodiment of the present application.
One embodiment of the above invention has the following advantages or benefits: the method comprises the steps that a corresponding terminal identification is obtained in response to a received data processing request, and then a batch terminal operation and maintenance tool is called to send a target script execution instruction to a terminal corresponding to the terminal identification; receiving execution result information returned by the terminal, and further persistently generating a tracking log file; and screening the tracking log file based on the preset identification to obtain and output a standard error log file. The batch operation and maintenance speed can be accelerated, and the working efficiency is improved.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a further understanding of the application and are not to be construed as limiting the application. Wherein:
FIG. 1 is a schematic diagram of the main flow of a data processing method according to one embodiment of the present application;
FIG. 2 is a schematic diagram of a main flow of a data processing method according to one embodiment of the present application;
FIG. 3 is a schematic flow chart diagram of a data processing method according to an embodiment of the present application;
FIG. 4 is a schematic diagram of the main elements of a data processing apparatus according to an embodiment of the present application;
FIG. 5 is an exemplary system architecture diagram to which embodiments of the present application may be applied;
fig. 6 is a schematic structural diagram of a computer system suitable for implementing the terminal device or the server according to the embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application to assist in understanding, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness. According to the technical scheme, the data acquisition, storage, use, processing and the like meet the relevant regulations of national laws and regulations.
Fig. 1 is a schematic diagram of a main flow of a data processing method according to an embodiment of the present application, and as shown in fig. 1, the data processing method includes:
and step S101, responding to the received data processing request, acquiring a corresponding terminal identifier, and further calling a batch terminal operation and maintenance tool to send a target script execution instruction to a terminal corresponding to the terminal identifier.
In this embodiment, an execution subject (for example, a server) of the data processing method may receive the data processing request by wired connection or wireless connection. When the execution body receives the data processing request, the terminal identifier carried in the request can be acquired. When the execution main body determines that the number of the terminals corresponding to the terminal identifier is multiple (for example, 100, the number of the terminals corresponding to the terminal identifier is not specifically limited in the embodiments of the present application), a batch terminal operation and maintenance tool (for example, an expect tool) may be called, and the batch terminal operation and maintenance tool is used to send a preset target script execution instruction to each terminal corresponding to the terminal identifier, so that each terminal executes the target script execution instruction, and thus, the corresponding service processing is completed.
And step S102, receiving the execution result information returned by the terminal, and further persistently generating a tracking log file.
Specifically, the receiving of the execution result information returned by the terminal includes:
and remotely connecting the terminal in a text or graphic interface mode, and acquiring execution result information obtained after the terminal executes a target script execution instruction.
And step S103, screening the tracking log file based on the preset identifier to obtain and output a standard error log file.
Specifically, screening the trace log file based on the preset identifier to obtain a standard error log file and outputting the standard error log file comprises:
and searching whether a preset identifier exists in each row of the tracking log file, and generating a standard error log file based on the row with the preset identifier.
The preset identifier may be, for example, a prefix or a suffix of the log file, and may be a combination of numbers or letters, or a combination of numbers and letters, and the content of the preset identifier is not specifically limited in this embodiment of the application.
Specifically, a standard error log file is obtained, comprising:
and merging the screened tracking log files to obtain a standard error log file.
For example, when performing trace and log merge, the execution of all commands in the script file is traced and output locally and persisted as a trace log file to facilitate error handling. And screening all tracking log files at the end of the script, screening out standard output and standard error logs required by a user, and merging the output of all hosts of the task operation into one log file to be displayed to the user.
The data processing method can be applied to an IT operation and maintenance automation working platform to improve the IT operation and maintenance efficiency and help IT operation and maintenance personnel to complete daily repetitive work (such as backup, configuration, data collection and the like). The configuration change detection automation means that once configuration parameters of the IT equipment change, a change process is triggered to be transferred to related technical personnel for confirmation, and discovery and maintenance configuration of IT operation and maintenance personnel are assisted through automatic detection. The automation of system health detection means that the health inspection of IT equipment hardware and an application system is automatically carried out regularly, and the health inspection and monitoring of the system are implemented by matching with an IT operation and maintenance team. The automatic generation of the maintenance report refers to the collection and analysis of logs of the system periodically and automatically, the recording of the running condition of the system, and the timely provision of the analysis report of the availability, performance and system resource utilization condition of the IT operation and maintenance through the periodical monitoring, analysis and summarization.
The execution main body of the embodiment acquires the corresponding terminal identification by responding to the received data processing request, and then calls the batch terminal operation and maintenance tool to send the target script execution instruction to the terminal corresponding to the terminal identification; receiving execution result information returned by the terminal, and further persistently generating a tracking log file; and screening the tracking log file based on the preset identification to obtain and output a standard error log file. The batch operation and maintenance speed can be accelerated, and the working efficiency is improved.
Fig. 2 is a schematic main flow diagram of a data processing method according to an embodiment of the present application, and as shown in fig. 2, the data processing method includes:
step S201, in response to receiving the data processing request, acquiring a corresponding terminal identifier.
The data processing request may be, for example, a request that requires a plurality of terminals to perform data query or data update, and the content of the data processing request is not specifically limited in the embodiments of the present application.
Step S202, a target program is called, a preset general function library and an interoperation function library are loaded, and then an operation environment variable and a log variable are defined.
An example target program may be a script intvrevoker. The general function library may be loaded by executing a script intvoset.sh, for example, and the interoperation function library may also be loaded by executing the script intvoset.sh, for example.
Step S203, parsing the target character string and initializing global variables, and then receiving the host keys of the terminals in batch to perform batch key registration.
The target string may be, for example, a certain parameter that is transmitted, such as parameter 1: a command line string.
Specifically, initializing global variables includes:
and acquiring a local task group directory, a user-defined full-scale host list file, a local private key file and a private key of the private key file, and initializing according to preset conditions. The preset condition may be, for example, a preset script called by a user.
Specifically, the bulk receiving terminal host key includes: and receiving the host names and the IP addresses of the terminals in batches.
And step S204, calling a batch terminal operation and maintenance tool to send a target script execution instruction to the terminal corresponding to the terminal identifier.
The batch terminal operation and maintenance tool may be an expect tool, for example, and the target script instruction set by the user in the batch terminal operation and maintenance tool is called to be sent to the corresponding terminal through the terminal interface corresponding to the terminal identifier, so that each terminal synchronously or asynchronously executes the received target script execution instruction, so as to return an execution result.
It can be understood that the target script execution instruction may be the same or different on each terminal corresponding to the terminal identifier, and may specifically be automatically adjusted according to the user requirement. For example, the execution main body may obtain user requirement information, generate a to-be-executed task of each terminal corresponding to the terminal identifier according to the user requirement information, generate a corresponding target script execution instruction according to the to-be-executed task corresponding to each terminal, and send the target script execution instruction to the corresponding terminal for the terminal to execute.
Further, the execution main body can obtain the thread identifications corresponding to the terminals corresponding to the terminal identifications, and then the threads corresponding to the thread identifications corresponding to the terminals can be called to execute the target script instructions corresponding to the terminals at preset time, so that the ordered execution of the terminal programs is not interfered with each other.
Step S205, receiving the execution result information returned by the terminal, and further generating the tracking log file persistently.
The execution result information may be, for example, a data query result, data query success information, or data query failure information returned by each terminal, or a data update result, data update success information, or data update failure information returned by each terminal.
And step S206, screening the tracking log file based on the preset identifier to obtain and output a standard error log file.
The method and the device for achieving the interaction between the programs and the server host solve the problem that some programs cannot achieve non-interactive operation by using the expect tool, can achieve interaction with the server host, can complete tasks and commands in the server host, and are convenient to complete relevant operations in a designated machine or a cluster. And the automatic interaction from the host key of the remote server to the batch verification secret-free login process is realized by expect, and some related operations can be performed after the interaction is passed. expect consists of a series of expect-send pairs: expect waits for a specific character, which may be, for example, a prompt, to be output in the output and then sends a specific response. The batch operation and maintenance speed can be accelerated, and the working efficiency is improved.
Fig. 3 is a schematic view of an application scenario of a data processing method according to an embodiment of the present application. The data processing method of the embodiment of the application can be applied to a batch operation and maintenance scene. The flow chart of fig. 3 is divided into two parts (1) and (2), wherein the upper part (1) of the flow chart utilizes a batch terminal operation and maintenance tool (expect) to realize key exchange, thereby creating conditions for the operation to be executed by the lower part (2) of fig. 3.
The process of realizing the key exchange comprises the following steps: sh is executed to call a corresponding program, so that a pre-configured general function library and an interoperation row function library are loaded, further, an environment variable is defined based on the general function library and the interoperation row function, for example, an environment variable program name is defined, further, a log variable and the like are defined based on the defined environment variable program name, and after configuration is completed, a contained character string is analyzed and global variable initialization is performed.
Wherein, the parameters involved in realizing the key exchange include: parameter 1: local task group directory, parameter 2: user-defined full-volume list file, parameter 3: local private key file, parameter 4: keys for private key files, etc. After the global variable is initialized, host list filtering is executed, and segmentation processing and saving are executed on the filtered list. And further receives a server host key and executes key login.
As shown in part (2) of FIG. 3, operations to implement key exchange include, but are not limited to: remote copy files, remote execution scripts, remote execution commands, remote copy decompression, etc., and all may be done in batches. When the tracing and log merging program is executed, the execution conditions of all commands in the script file can be traced and output to the local so as to be persisted as a tracing log file for facilitating error handling. And at the final stage of executing the script, screening all tracking log files, screening out standard output and standard error logs required by a user, and merging the output data of all hosts of the task operation into one log file to be displayed to the user.
The method and the device solve the problem that some programs cannot run interactively by using an expect tool, and are high in universality and mobility. The functions that can be realized by the data processing method of the embodiment of the application include, but are not limited to, basic functions of operation and maintenance task grouping, host list forward and backward screening, remote local bidirectional file copying, remote script execution, remote command execution, remote decompression, remote copying and the like, and functional combinations among the basic functions; the concurrency control can be realized, the concurrency of the host list is controlled, and the batch operation and maintenance speed is accelerated. The execution conditions of all commands in the script file can be tracked, the commands are output to the local, the commands are persisted into a tracking log file, and the tracking log file is combined into a log file through screening. The method supports remote user and password authentication, and also supports remote public key authentication and local private key authentication, so that the authentication is simpler and safer. The data processing method of the embodiment of the application is convenient and easy to use by combining Bash language with expect toolkit, can be executed in any Linux system with the solid state hybrid hard disk sshd, and does not need to depend on other components; the system list is a configurable system record list and can also be searched according to a regular expression; commands in the Shell script can be tracked to accurately locate problems in the program; concurrency can be achieved: specifically, the method can be executed in series or in batches, the batch is executed in parallel, and the parallelism is automatically calculated according to the number of CPUs.
Fig. 4 is a schematic diagram of main units of a data processing apparatus according to an embodiment of the present application. As shown in fig. 4, the data processing apparatus 400 includes an interaction unit 401, a persistence unit 402, and a filtering unit 403.
And the interaction unit 401 is configured to, in response to receiving the data processing request, obtain a corresponding terminal identifier, and further invoke the batch terminal operation and maintenance tool to send the target script execution instruction to the terminal corresponding to the terminal identifier.
And a persistence unit 402 configured to receive the execution result information returned by the terminal, and to persistently generate the trace log file.
And a screening unit 403 configured to screen the trace log file based on a preset identifier to obtain and output a standard error log file.
In some embodiments, the apparatus further comprises a key exchange unit, not shown in fig. 4, configured to: executing a key exchange process; wherein the key exchange process is configured to: calling a target program, loading a preset general function library and an interoperation function library, and further defining an operation environment variable and a log variable; and analyzing the target character string, initializing a global variable, receiving a host key of the terminal in batches, and registering the secret key in batches.
In some embodiments, the key exchange unit is further configured to: and receiving the terminal host names and the IP addresses in batches.
In some embodiments, the key exchange unit is further configured to: and acquiring a local task group directory, a user-defined full-scale host list file, a local private key file and a private key of the private key file, and initializing according to preset conditions.
In some embodiments, the screening unit 403 is further configured to: and merging the screened tracking log files to obtain a standard error log file.
In some embodiments, persistence unit 402 is configured to: and remotely connecting the terminal in a text or graphic interface mode, and acquiring execution result information obtained after the terminal executes a target script execution instruction.
In some embodiments, the screening unit 403 is further configured to: and searching whether a preset identifier exists in each row of the tracking log file, and generating a standard error log file based on the row with the preset identifier.
The data processing method and the data processing apparatus of the present application have corresponding relation in the specific implementation contents, and therefore, the repeated contents are not described again.
Fig. 5 shows an exemplary system architecture 500 to which the data processing method or the data processing apparatus of the embodiments of the present application may be applied.
As shown in fig. 5, the system architecture 500 may include terminal devices 501, 502, 503, a network 504, and a server 505. The network 504 serves to provide a medium for communication links between the terminal devices 501, 502, 503 and the server 505. Network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 501, 502, 503 to interact with a server 505 over a network 504 to receive or send messages or the like. The terminal devices 501, 502, 503 may have installed thereon various communication client applications, such as shopping-like applications, web browser applications, search-like applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 501, 502, 503 may be various electronic devices having data processing screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 505 may be a server providing various services, such as a background management server (for example only) providing support for data processing requests received by users using the terminal devices 501, 502, 503. The background management server can respond to the received data processing request, acquire the corresponding terminal identification, and then call the batch terminal operation and maintenance tool to send the target script execution instruction to the terminal corresponding to the terminal identification; receiving execution result information returned by the terminal, and further persistently generating a tracking log file; and screening the tracking log file based on the preset identification to obtain and output a standard error log file. The batch operation and maintenance speed can be accelerated, and the working efficiency is improved.
It should be noted that the data processing method provided in the embodiment of the present application is generally executed by the server 505, and accordingly, the data processing apparatus is generally disposed in the server 505.
It should be understood that the number of terminal devices, networks, and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 6, shown is a block diagram of a computer system 600 suitable for use in implementing a terminal device of an embodiment of the present application. The terminal device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU) 601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data necessary for the operation of the computer system 600 are also stored. The CPU601, ROM602, and RAM603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output section 607 including a signal processing section such as a Cathode Ray Tube (CRT), a liquid crystal credit authorization inquiry processor (LCD), and the like, and a speaker and the like; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.
In particular, according to embodiments disclosed herein, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, embodiments disclosed herein include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609 and/or installed from the removable medium 611. The above-described functions defined in the system of the present application are executed when the computer program is executed by the Central Processing Unit (CPU) 601.
It should be noted that the computer readable medium shown in the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present application may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes an interaction unit, a persistence unit, and a screening unit. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.
As another aspect, the present application also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs, and when the one or more programs are executed by one device, the device responds to a received data processing request, acquires a corresponding terminal identifier, and further calls a batch terminal operation and maintenance tool to send a target script execution instruction to a terminal corresponding to the terminal identifier; receiving execution result information returned by the terminal, and further persistently generating a tracking log file; and screening the tracking log file based on the preset identification to obtain and output a standard error log file.
The computer program product of the present application comprises a computer program which, when executed by a processor, implements the data processing method of the embodiments of the present application.
According to the technical scheme of the embodiment of the application, the batch operation and maintenance speed can be increased, and the working efficiency is improved.
The above-described embodiments are not intended to limit the scope of the present disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (16)
1. A data processing method, comprising:
responding to a received data processing request, acquiring a corresponding terminal identifier, and calling a batch terminal operation and maintenance tool to send a target script execution instruction to a terminal corresponding to the terminal identifier;
receiving execution result information returned by the terminal, and further persistently generating a tracking log file;
and screening the tracking log file based on a preset identifier to obtain a standard error log file and outputting the standard error log file.
2. The method according to claim 1, wherein before the sending the target script execution instruction to the terminal corresponding to the terminal identification, the method further comprises:
executing a key exchange process; wherein,
the key exchange process comprises the following steps:
calling a target program, loading a preset general function library and an interoperation function library, and further defining an operation environment variable and a log variable;
and analyzing the target character string, initializing a global variable, receiving a host key of the terminal in batches, and registering the secret key in batches.
3. The method of claim 2, wherein the bulk receiving terminal host key comprises:
and receiving the host names and the IP addresses of the terminals in batches.
4. The method of claim 2, wherein initializing the global variable comprises:
and acquiring a local task group directory, a user-defined full-scale host list file, a local private key file and a private key of the private key file, and initializing according to preset conditions.
5. The method of claim 1, wherein obtaining a standard error log file comprises:
and merging the screened tracking log files to obtain a standard error log file.
6. The method according to claim 1, wherein the receiving the execution result information returned by the terminal comprises:
and remotely connecting the terminal in a text or graphic interface mode, and acquiring execution result information obtained after the terminal executes the target script execution instruction.
7. The method of claim 1, wherein the screening the trace log file based on a preset identifier to obtain a standard error log file and outputting the standard error log file comprises:
and searching whether a preset identifier exists in each row of the tracking log file, and generating a standard error log file based on the row with the preset identifier.
8. A data processing apparatus, comprising:
the interaction unit is configured to respond to a received data processing request, acquire a corresponding terminal identifier and further call a batch terminal operation and maintenance tool to send a target script execution instruction to a terminal corresponding to the terminal identifier;
the persistence unit is configured to receive the execution result information returned by the terminal, and further persistently generate a tracking log file;
and the screening unit is configured to screen the tracking log file based on a preset identifier so as to obtain a standard error log file and output the standard error log file.
9. The apparatus of claim 8, wherein the apparatus further comprises a key exchange unit configured to:
executing a key exchange process; wherein,
the key exchange process is configured to:
calling a target program, loading a preset general function library and an interoperation function library, and further defining an operation environment variable and a log variable;
and analyzing the target character string, initializing a global variable, receiving a host key of the terminal in batches, and registering the secret key in batches.
10. The apparatus of claim 9, wherein the key exchange unit is further configured to:
and receiving the host names and the IP addresses of the terminals in batches.
11. The apparatus of claim 9, wherein the key exchange unit is further configured to:
and acquiring a local task group directory, a user-defined full-scale host list file, a local private key file and a private key of the private key file, and initializing according to preset conditions.
12. The apparatus of claim 8, wherein the screening unit is further configured to:
and merging the screened tracking log files to obtain a standard error log file.
13. The apparatus of claim 8, wherein the persistence unit is configured to:
and remotely connecting the terminal in a text or graphic interface mode, and acquiring execution result information obtained after the terminal executes the target script execution instruction.
14. An electronic device for data processing, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.
15. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-7.
16. A computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211192062.8A CN115422012A (en) | 2022-09-28 | 2022-09-28 | Data processing method and device, electronic equipment and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211192062.8A CN115422012A (en) | 2022-09-28 | 2022-09-28 | Data processing method and device, electronic equipment and computer readable medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115422012A true CN115422012A (en) | 2022-12-02 |
Family
ID=84206511
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211192062.8A Pending CN115422012A (en) | 2022-09-28 | 2022-09-28 | Data processing method and device, electronic equipment and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115422012A (en) |
-
2022
- 2022-09-28 CN CN202211192062.8A patent/CN115422012A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111190888A (en) | Method and device for managing graph database cluster | |
| CN110083455B (en) | Graph calculation processing method, graph calculation processing device, graph calculation processing medium and electronic equipment | |
| CN110221910B (en) | Method and apparatus for performing MPI jobs | |
| US9940113B2 (en) | Big data assistant | |
| CN109672722B (en) | Data deployment method and device, computer storage medium and electronic equipment | |
| US10067862B2 (en) | Tracking asynchronous entry points for an application | |
| CN113900834A (en) | Data processing method, device, equipment and storage medium based on Internet of things technology | |
| CN112882863A (en) | Method, device and system for recovering data and electronic equipment | |
| CN116893807A (en) | System and method for designing a robotic flow automation robot using a browser | |
| CN109597952B (en) | Webpage information processing method, system, electronic equipment and storage medium | |
| CN116737662A (en) | Method, device, electronic equipment and storage medium for processing business data | |
| CN112491940A (en) | Request forwarding method and device of proxy server, storage medium and electronic equipment | |
| CN110019059B (en) | Timing synchronization method and device | |
| CN112579406A (en) | Log call chain generation method and device | |
| CN111698109A (en) | Method and device for monitoring log | |
| CN115422012A (en) | Data processing method and device, electronic equipment and computer readable medium | |
| CN111176982B (en) | Test interface generation method and device | |
| CN110768855B (en) | Method and device for testing linkmzation performance | |
| CN111290870B (en) | Method and device for detecting abnormality | |
| CN113779122A (en) | Method and apparatus for exporting data | |
| CN113760693A (en) | Method and apparatus for local debugging of microservice systems | |
| CN110874302A (en) | Method and device for determining buried point configuration information | |
| CN116701053B (en) | Method, device, equipment and medium for restoring data backup of production environment database | |
| CN112947846B (en) | Batch processing task execution method and device of object storage system and electronic equipment | |
| CN112583614B (en) | Host cluster management method, device and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |