CN113377420A - Data interfacing method, device, equipment and storage medium between systems - Google Patents

Data interfacing method, device, equipment and storage medium between systems Download PDF

Info

Publication number
CN113377420A
CN113377420A CN202110637040.7A CN202110637040A CN113377420A CN 113377420 A CN113377420 A CN 113377420A CN 202110637040 A CN202110637040 A CN 202110637040A CN 113377420 A CN113377420 A CN 113377420A
Authority
CN
China
Prior art keywords
data
data file
source system
file
interfacing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110637040.7A
Other languages
Chinese (zh)
Inventor
张世荃
李懋东
刘权
孙森
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial and Commercial Bank of China Ltd ICBC
Original Assignee
Industrial and Commercial Bank of China Ltd ICBC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial and Commercial Bank of China Ltd ICBC filed Critical Industrial and Commercial Bank of China Ltd ICBC
Priority to CN202110637040.7A priority Critical patent/CN113377420A/en
Publication of CN113377420A publication Critical patent/CN113377420A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/70Software maintenance or management
    • G06F8/71Version control; Configuration management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60Protecting data
    • G06F21/62Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245Protecting personal data, e.g. for financial or medical purposes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60Protecting data
    • G06F21/64Protecting data integrity, e.g. using checksums, certificates or signatures

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioethics (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Hardware Design (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The specification relates to the technical field of data docking, and provides a method, a device, equipment and a storage medium for data docking between systems, wherein the method comprises the following steps: initiating an operation request to a specified page element of a corresponding page on a source system based on a preset operation sequence at regular time; acquiring data returned by the source system aiming at the operation request; storing the data as a data file with a specified format; and transmitting the data files to a target system in batch. The present specification may enable data interfacing between systems without the source system providing an API interface for the target system.

Description

Data interfacing method, device, equipment and storage medium between systems
Technical Field
The present disclosure relates to the field of data interfacing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for data interfacing between systems.
Background
Management Information systems (MIS for short) are widely used by IT departments in the banking industry, such as resource Management systems, configuration Management systems, operation and maintenance Management systems developed by departments, and the like, each System has ITs own data, and needs to use data of other systems.
For example, an operation and maintenance management system is newly developed in some department, and data of two old systems, namely a resource management system and a configuration management system, needs to be used as a calculation basis in near real time, so that the new system needs to be in data connection with the old system. However, the old system is developed earlier, the requirement for supplying data to the new system is not considered, and a specific Application Programming Interface (API) Interface is not reserved, so that data interfacing between the new system and the old system is difficult to realize.
Disclosure of Invention
An object of the embodiments of the present specification is to provide a method, an apparatus, a device, and a storage medium for data interfacing between systems, so as to implement data interfacing between systems under a condition that a source system does not provide an API interface for a target system.
In order to achieve the above object, in one aspect, an embodiment of the present specification provides a method for data interfacing between systems, including:
initiating an operation request to a specified page element of a page on a source system based on a preset operation sequence at regular time;
acquiring data returned by the source system aiming at the operation request;
storing the data as a data file with a specified format;
and transmitting the data files to a target system in batch.
In a preferred embodiment, before the bulk transmitting the data file to the target system, the method further includes:
and carrying out data integrity detection on the data rows and the data columns of the data file.
In a preferred embodiment, the performing data integrity check on the data rows and/or the data columns of the data file includes:
confirming whether the number of data columns of the data file is equal to that of the historical data file;
when the number of data columns of the data file is equal to that of the historical data file, determining a difference value between the number of data lines of the data file and the number of data lines of the historical data file;
determining an absolute value of a ratio of the difference to a number of data columns of the data file;
determining whether the absolute value is greater than a preset threshold;
and when the absolute value is not greater than a preset threshold value, confirming that the data of the data file is complete.
In a preferred embodiment, the method further comprises:
when the number of data columns of the data file is not equal to the number of data columns of the historical data file, or the absolute value is greater than a preset threshold value, determining that the data of the data file is incomplete;
and deleting the data file and acquiring the data file again.
In a preferred embodiment, before the initiating an operation request to a specified page element of a corresponding page on the source system based on the preset operation sequence at the timing, the method further includes:
providing a user name and a password ciphertext to the source system for the source system to log in and authenticate after decryption; and when decrypting the user name and the password ciphertext, the source system uses a decryption algorithm after code confusion processing.
In a preferred embodiment, before transmitting the data file to the target system, the method further includes:
and carrying out data desensitization treatment on the data file.
In a preferred embodiment, the specified format comprises a plain text format.
On the other hand, an embodiment of the present specification further provides an inter-system data interfacing apparatus, including:
the robot process automation module is used for initiating an operation request to a specified page element of a corresponding page on a source system at regular time based on a preset operation sequence and acquiring data returned by the source system aiming at the operation request;
the file transmission service module is used for storing the data into a data file with a specified format;
and the batch import module is used for transmitting the data files to the target system in batches.
In another aspect, the embodiments of the present specification further provide a computer device, which includes a memory, a processor, and a computer program stored on the memory, and when the computer program is executed by the processor, the computer program executes the instructions of the above method.
In another aspect, the present specification further provides a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor of a computer device to execute the instructions of the method.
As can be seen from the technical solutions provided by the embodiments of the present specification, the embodiments of the present specification may initiate an operation request to a specified page element of a corresponding page on a source system based on a preset operation sequence, and obtain data returned by the source system for the operation request, thereby implementing automatic simulation of a manual operation source system. On the basis, the data can be stored into a data file with a specified format and transmitted to the target system in batch, so that the data docking between the source system and the target system is realized under the condition that the source system does not provide an API (application program interface) for the target system, namely the light and quick data docking between the source system and the target system is realized.
Drawings
In order to more clearly illustrate the embodiments of the present specification or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present specification, and for those skilled in the art, other drawings can be obtained according to the drawings without any creative effort. In the drawings:
FIG. 1 illustrates a flow diagram of a method for data interfacing between systems in some embodiments of the present description;
FIG. 2 illustrates a flow diagram of data integrity checking in some embodiments of the present description;
FIG. 3 is a block diagram of an inter-system data interfacing apparatus in some embodiments of the present description;
FIG. 4 shows a block diagram of a computing device in some embodiments of the present description.
[ description of reference ]
1. A data docking device;
11. an RPA module;
12. a file transfer service module;
13. importing modules in batches;
21. a source system;
22. a source system database;
31. a target system;
32. a target system database;
402. a computer device;
404. a processor;
406. a memory;
408. a drive mechanism;
410. an input/output interface;
412. an input device;
414. an output device;
416. a presentation device;
418. a graphical user interface;
420. a network interface;
422. a communication link;
424. a communication bus.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only a part of the embodiments of the present specification, and not all of the embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments in the present specification without any inventive step should fall within the scope of protection of the present specification.
Embodiments of the present description relate to inter-system (e.g., MIS inter-system, etc.) data interfacing techniques. Wherein, data butt joint means: for the system a and the system B, if the system B uses the full amount of data of the system a or a part of the full amount of data (usually used for aggregating data of multiple systems for data analysis), the system a may normally and actively transmit data to the system B, or may normally and automatically acquire required data from the system a by the system B, and this process is called data interfacing. For convenience of description, in the embodiment of the present specification, the a system may be referred to as a source system (or a data provider), and the B system may be referred to as a target system (or a data demander). In the embodiment of the present specification, the source system may be a system adopting a browser and server architecture in general.
In some cases, the source system may not have a specific API interface reserved for the target system. In other cases, the source system sets a limit on the frequency of the call interface (e.g., 20 seconds) based on the performance capabilities of the front-end server, although reserving a specific API interface for the target system; as such, interfacing only a few thousand pieces of data through the API may take tens of hours, making it difficult to meet the data timeliness requirements. As such, in these cases, it is difficult to achieve data interfacing between the old and new systems between the source system and the target system.
In view of the above, to implement data interfacing between a source system and a target system in this case, the embodiments of the present specification provide an improved inter-system data interfacing method, which can be applied to any suitable computer device side. Referring to fig. 1, in some embodiments of the present disclosure, the inter-system data interfacing method may include the following steps:
s101, initiating an operation request to a designated page element of a corresponding page (or called as a graphical user interface) on a source system at regular time based on a preset operation sequence.
And S102, acquiring data returned by the source system aiming at the operation request.
S103, storing the data into a data file with a specified format.
And S104, transmitting the data files to a target system in batches.
In the embodiment of the present specification, an operation request may be initiated to a specified page element of a corresponding page on a source system based on a preset operation sequence, and data returned by the source system for the operation request may be acquired, so that an automated simulation of a manual operation source system is achieved. On the basis, the data can be stored into a data file with a specified format and transmitted to the target system in batch, so that under the condition that the source system does not provide an API (application programming interface) for the target system, data docking between the source system and the target system is realized, namely light-weight (the source system does not need to be modified) and agile (the API is provided for the target system relative to the source system, but the requirement of data timeliness is difficult to meet due to limited use) data docking between the source system and the target system is realized.
In this embodiment, step S101 and step S102 may be implemented based on a Robot Process Automation (RPA) technology. The RPA technology is a business process automation technology based on software robots, wherein a "robot" is a virtual concept, does not need an actual robot, and is only an execution body or an execution unit of a process. For example, in this embodiment of the present specification, such an execution unit may be referred to as an RPA module.
Through a series of operations executed on computer equipment in a human-simulated mode, the RPA module can automatically realize all the operation behaviors (such as mouse clicking, copying, pasting, data entry, webpage navigation, file opening and the like) of a human on a computer, and can continuously and repeatedly operate according to a certain rule. Because the core of the system is the simulation of human operation, the pressure applied to the source system by the system is similar to the normal operation of a person on the source system, no redundant load is generated, and no significant influence is caused on the source system. Therefore, in the embodiment of the present specification, a process of initiating an operation request to a specified page element of a corresponding page on a source system and acquiring data returned by the source system for the operation request is performed based on a preset operation sequence at regular time, that is, a process of automatically simulating a manual operation source system at regular time and automatically acquiring required data.
In the embodiment of the present specification, the operation sequence is all the operations and their ordering required for implementing data interfacing. For example, in one embodiment, all files under a specified path in the source system are obtained as an example. The sequence of operations may include: a series of operations to log in to the source system → a series of operations to open a specified path → a series of operations to read all files under a specified path → a series of operations to save the read files in a specified format, and so on. The operation sequence can be obtained in advance by the following steps 1 to 4. The whole operation process included in the operation sequence generally involves jumping among a plurality of pages, and each page needs to operate a specified page element to jump to the next page. Thus, an operation needs to be initiated to a specified page element of the corresponding page on the source system. It can be seen that the operation request in the embodiment of the present specification is actually an operation request sequence, that is, a plurality of operation requests are executed in sequence.
Step 1, solidifying the flow of manually executing the acquisition of the target data from the page of the source system. Firstly, all operation steps of the source system page need to be clearly executed manually, the execution success of the operation according to the steps is ensured, and then all the step list is fixed.
In some embodiments of the present description, there are two methods to obtain all operations, method 1: the engineer responsible for implementing data docking interviews users who manually execute the process, views and records the operation process of the users on the source system page, decomposes the process into steps and lists the steps into a table list, and therefore all operations are obtained. The method 2 comprises the following steps: and recording and storing each step of operation in the background by a software program when a user operates manually, and automatically forming a list.
And 2, identifying page elements operated in manual execution. I.e. to identify the page elements (i.e. operands) that are manipulated by the person performing each step, e.g. multi-selection boxes, drop-down boxes, buttons, etc.
There may be two ways how to identify page elements. The method comprises the following steps: on the basis of forming the list, an engineer in charge of data docking watches the operation process of the user on the source system page, visually judges and records what page elements are operated by the user at each step, and accordingly recognition is completed. The method 2 comprises the following steps: and recording and storing each step of operation in a background when a user operates manually through a software program, automatically judging the page elements of each step of operation according to the webpage HTML codes of the page elements related to the operation, and forming a list so as to finish the identification.
And 3, identifying the operation on the page elements in manual execution. And identifying what operation is performed on each page element in manual execution, such as clicking operation performed on a button element, pulling down operation and selecting operation performed on a pull-down box element, and the like.
There may be two approaches to how to identify operations on page elements. Method 1: on the basis of the step 2, the engineer in charge of data docking visually judges and records what operation the user has performed on each page element (for example, clicking, dragging and the like by using a left mouse button) on the operation process of viewing the page of the source system by the user. The method 2 comprises the following steps: and recording and storing each step of operation in the background when the user manually executes the operation through a software program, automatically judging the operation of each step on the corresponding page element according to a message hook (hook) of an operating system, and forming a list according to the operation.
And 4, writing an RPA module to automatically simulate the manual operation process. The RPA module refers to a computer program developed specifically for this purpose. Specifically, the RPA module can be implemented by using a programming language such as python or java, so that the manual operation steps are all automatically implemented.
In addition, in order to continuously and automatically transmit the data obtained by the RPA module to the transmission target system, it is necessary to preset a storage path for storing the data obtained by the RPA module, configure a frequency of operation of the RPA module (for example, it is set to execute once per hour or once per day, and the execution interval cannot be less than a time required for each execution of the RPA), configure a frequency of batch import operation (for example, it should be lower than the frequency of operation of the RPA once per hour or once per day, and the like), and configure Webdriver parameters (for example, timeout time of page loading, IP address of the source system, page size), and the like.
In this embodiment of the present specification, the obtaining of the data returned by the source system for the operation request is to read data from a database of the source system based on the operation request. The data retrieved from the source system may be saved as a data file in plain text format, such as Comma-Separated Values (CSV) format, etc. Therefore, the data file can be commonly used among various systems and programs, and the problems that the target system cannot be identified due to the text format problem and the like are solved. Furthermore, while the data obtained from the source system may be stored locally at the RPA module, this may not be conducive to direct import to the target system. Therefore, the RPA module can automatically upload the data file in the plain text format to a file upload service module such as an FTP server. Thus, the data files can be imported into the target system in batch (for example, the data files can be imported into a database of the target system in batch) through the written bat or shell scripts.
In other embodiments of the present disclosure, before the data file is transmitted to the target system in batch, data integrity detection may be performed on the data rows and/or the data columns of the data file to ensure that the data transmitted to the target system is complete data.
In some embodiments of the present disclosure, as shown in fig. 2, the performing data integrity check on the data rows and/or the data columns of the data file may include the following steps:
s201, determining the data column number of the data file. Because the data in the format data file such as CSV is arranged in a matrix form, the data column number and the data line number of the data file can be calculated.
S202, whether the number of data columns of the data file is equal to that of the historical data file is confirmed. When equal, step S203 is executed, otherwise step S207 and its subsequent steps are executed.
Since data in a format data file such as CSV is arranged in a matrix, the number of columns of the data file obtained each time is fixed in many cases. Therefore, the number of columns corresponding to the statistical history data file can be used as a reference value for evaluating whether the data file obtained each time is abnormal.
S203, when the number of the data columns of the data file is equal to that of the historical data file, determining a difference value between the number of the data lines of the data file and the number of the data lines of the historical data file.
S204, determining the absolute value of the ratio of the difference value to the number of data columns of the data file.
I.e. by means of a formula
Figure BDA0003105591920000081
The absolute value v is calculated. Wherein, c1For the number of data lines of the currently acquired data file, c0Is the data line number of the historical data file. The larger the absolute value is, the larger the difference between the data line number of the currently obtained data file and the data line number of the historical data file is; accordingly, the greater the probability that the currently obtained data file is an abnormal data file.
S205, determining whether the absolute value is larger than a preset threshold value.
And S206, when the absolute value is not greater than a preset threshold value, confirming that the data of the data file is complete. Therefore, no reacquisition is required. Of course, in this case, a log of processing success may also be generated and automatically notified to the user by way of an email or the like; the log format of the log may be, for example, a success flag bit time-stamped.
S207, when the number of the data columns of the data file is not equal to that of the historical data file, or the absolute value is larger than a preset threshold value, determining that the data of the data file is incomplete. Therefore, re-fetching is required to ensure the integrity of the data file.
And S208, deleting the data file and acquiring the data file again.
In other embodiments of the present disclosure, before initiating an operation request to a designated page element of a corresponding page on a source system at regular time based on a preset operation sequence, the source system generally needs to be logged in for login authentication. Therefore, a user name and password ciphertext (that is, both the user name and the password are ciphertext) can be provided to the source system before the source system performs login authentication after decryption; and when decrypting the user name and the password ciphertext, the source system uses a decryption algorithm after code confusion processing. Therefore, an attacker can be prevented from acquiring the encryption algorithm through means such as decompilation and the like, so that the attacker cannot acquire the user name and the password plaintext, and the data security is improved.
Code obfuscation in this specification refers to: for a program P to be protected (such as a decryption algorithm and the like), the program P is converted into P ' after obfuscation, and the output of P and P ' is consistent, but an attacker (the attacker refers to a person trying to illegally acquire information, such as a hacker trying to deduce a password plaintext by acquiring the decryption algorithm) has difficulty in acquiring key information (such as the decryption algorithm) in the program through P '. The code obfuscation in the embodiments of the present description may be static code obfuscation or dynamic code obfuscation, and may be specifically selected according to the actual application needs. Static obfuscation is typically performed by fixing code before program execution, such as squashing control flow, complicating control flow, and the like, and dynamic obfuscation is performed by changing code while executing, such as self-modifying code. In some embodiments of the present disclosure, the Encryption algorithm may use an asymmetric Encryption algorithm such as Data Encryption Standard (DES), Advanced Encryption Standard (AES), and the like.
Some field information in the data file may contain sensitive information, which is defined as: information that, once leaked, would cause economic or other losses to natural persons and/or organizations. The source system may provide such sensitive information, but the target system generally does not require sensitive information. Therefore, in order to further improve data security, before the data file is transmitted to the target system, data desensitization processing may be performed on the data file, that is, before the data file is transmitted to the target system, the data file may be read into a memory (for example, a csv _ read method of pandas may be used), and according to a preset data desensitization processing logic, sensitive information is deleted from the data file and stored as a new file. The data desensitization processing logic can pre-specify which field attributes are listed as sensitive information, and can be configured by a user. For example, in an embodiment of the present specification, field attribute columns such as "specific use of IT assets", "intranet IP address", and the like may be designated as sensitive information.
In addition, the number of columns is reduced to avoid the normal data file from deleting sensitive information. In some embodiments of the present specification, an empty column may be inserted in the original position of the field attribute column corresponding to the deleted sensitive information, and the column name of the empty column is set as the column name of the deleted column, so that the data structure of the data file after desensitization and the data structure of the data file before desensitization are consistent, that is, the number of columns is equal, and the column names are the same.
In other embodiments of the present disclosure, before the data file is transmitted to the target system in batch, the data file may be subjected to a life cycle check periodically, that is, whether the data file is expired is determined periodically. Wherein, the expiration means: the file has been imported into the database of the target system. Therefore, when the data files are transmitted in batches, the transmitted data files are all unexpired data files. And the expired data files can be deleted so as to reduce the occupation of storage resources.
While the process flows described above include operations that occur in a particular order, it should be appreciated that the processes may include more or less operations that are performed sequentially or in parallel (e.g., using parallel processors or a multi-threaded environment).
Corresponding to the method, the embodiment of the specification further provides a data docking device between systems. Referring to fig. 3, in some embodiments of the present specification, the data docking apparatus 1 may periodically initiate an operation request to a designated page element of a corresponding page on the source system 21 based on a preset operation sequence; acquiring data returned by the source system 21 for the operation request (namely acquiring data from a source system database 22 corresponding to the source system 21); storing the data as a data file with a specified format; the data files are transmitted to the target system 31 in batch (i.e. transmitted to the target system database 32 corresponding to the target system 31 in batch) for the target system 31 to use.
With continued reference to fig. 3, the data interfacing apparatus 1 may include an RPA module 11, a file transfer service module 12, and a bulk import module 13. Wherein:
the RPA module 11 may be configured to initiate an operation request to a specified page element of a corresponding page on a source system based on a preset operation sequence at regular time, and acquire data returned by the source system for the operation request;
the file transmission service module 12 is used for saving the data into a data file with a specified format;
and the batch import module 13 is used for transmitting the data files to the target system in batches.
In some embodiments of the present description, the RPA module 11 may further be configured to: and before the data files are transmitted to a target system in batch, carrying out data integrity detection on data rows and data columns of the data files.
In some embodiments of the present specification, the performing data integrity check on the data rows and/or the data columns of the data file includes:
confirming whether the number of data columns of the data file is equal to that of the historical data file;
when the number of data columns of the data file is equal to that of the historical data file, determining a difference value between the number of data lines of the data file and the number of data lines of the historical data file;
determining an absolute value of a ratio of the difference to a number of data columns of the data file;
determining whether the absolute value is greater than a preset threshold;
and when the absolute value is not greater than a preset threshold value, confirming that the data of the data file is complete.
In some embodiments of the present description, the RPA module 11 may further be configured to: when the number of data columns of the data file is not equal to the number of data columns of the historical data file, or the absolute value is greater than a preset threshold value, determining that the data of the data file is incomplete; and deleting the data file and acquiring the data file again.
In some embodiments of the present description, the RPA module 11 may further be configured to: providing a user name and a password ciphertext to a source system before initiating an operation request to a specified page element of a corresponding page on the source system based on a preset operation sequence at the timing so as to enable the source system to log in and authenticate after decryption; and when decrypting the user name and the password ciphertext, the source system uses a decryption algorithm after code confusion processing.
In some embodiments of the present description, the RPA module 11 may further be configured to: performing a data desensitization process on the data file prior to transmitting the data file to a target system.
In some embodiments of the present description, the specified format comprises a plain text format.
For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functions of the various elements may be implemented in the same one or more software and/or hardware implementations of the present description.
Embodiments of the present description also provide a computer device. As shown in FIG. 4, in some embodiments of the present description, the computer device 402 may include one or more processors 404, such as one or more Central Processing Units (CPUs) or Graphics Processors (GPUs), each of which may implement one or more hardware threads. The computer device 402 may also include any memory 406 for storing any kind of information such as code, settings, data, etc., and in a specific embodiment, a computer program running on the memory 406 and on the processor 404, the computer program when executed by the processor 404 may perform the instructions of the inter-system data interfacing method according to any of the above embodiments. For example, and without limitation, memory 406 may include any one or more of the following in combination: any type of RAM, any type of ROM, flash memory devices, hard disks, optical disks, etc. More generally, any memory may use any technology to store information. Further, any memory may provide volatile or non-volatile retention of information. Further, any memory may represent fixed or removable components of computer device 402. In one case, when the processor 404 executes the associated instructions, which are stored in any memory or combination of memories, the computer device 402 can perform any of the operations of the associated instructions. The computer device 402 also includes one or more drive mechanisms 408, such as a hard disk drive mechanism, an optical disk drive mechanism, etc., for interacting with any memory.
Computer device 402 may also include input/output interface 410(I/O) for receiving various inputs (via input device 412) and for providing various outputs (via output device 414). One particular output mechanism may include a presentation device 416 and an associated graphical user interface 418 (page). In other embodiments, input/output interfaces 410(I/O), input devices 412, and output devices 414 may also be excluded, as just one computer device in a network. Computer device 402 can also include one or more network interfaces 420 for exchanging data with other devices via one or more communication links 422. One or more communication buses 424 couple the above-described components together.
Communication link 422 may be implemented in any manner, such as over a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. Communication link 422 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products of some embodiments of the specification. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processor to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processor, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processor to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processor to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computer device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium which can be used to store information that can be accessed by a computer device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
As will be appreciated by one skilled in the art, embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
The embodiments of this specification may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The described embodiments may also be practiced in distributed computing environments where tasks are performed by remote processors that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment. In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of an embodiment of the specification. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims (10)

1. A method for interfacing data between systems, comprising:
initiating an operation request to a specified page element of a corresponding page on a source system based on a preset operation sequence at regular time;
acquiring data returned by the source system aiming at the operation request;
storing the data as a data file with a specified format;
and transmitting the data files to a target system in batch.
2. The method of claim 1, wherein prior to the bulk transfer of the data file to the target system, further comprising:
and carrying out data integrity detection on the data rows and the data columns of the data file.
3. The method for interfacing data between systems according to claim 2, wherein said performing data integrity check on data rows and/or data columns of said data file comprises:
confirming whether the number of data columns of the data file is equal to that of the historical data file;
when the number of data columns of the data file is equal to that of the historical data file, determining a difference value between the number of data lines of the data file and the number of data lines of the historical data file;
determining an absolute value of a ratio of the difference to a number of data columns of the data file;
determining whether the absolute value is greater than a preset threshold;
and when the absolute value is not greater than a preset threshold value, confirming that the data of the data file is complete.
4. The method of inter-system data interfacing of claim 3, further comprising:
when the number of data columns of the data file is not equal to the number of data columns of the historical data file, or the absolute value is greater than a preset threshold value, determining that the data of the data file is incomplete;
and deleting the data file and acquiring the data file again.
5. The method for interfacing data between systems according to claim 1, wherein before initiating an operation request to a designated page element of a corresponding page on a source system based on a preset operation sequence at the timing, the method further comprises:
providing a user name and a password ciphertext to the source system for the source system to log in and authenticate after decryption; and when decrypting the user name and the password ciphertext, the source system uses a decryption algorithm after code confusion processing.
6. The method of inter-system data interfacing of claim 1, further comprising, prior to transferring the data file to a target system:
and carrying out data desensitization treatment on the data file.
7. The method of inter-system data interfacing of claim 1, wherein said specified format comprises a plain text format.
8. An inter-system data interfacing apparatus, comprising:
the robot process automation module is used for initiating an operation request to a specified page element of a corresponding page on a source system at regular time based on a preset operation sequence and acquiring data returned by the source system aiming at the operation request;
the file transmission service module is used for storing the data into a data file with a specified format;
and the batch import module is used for transmitting the data files to the target system in batches.
9. A computer device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, performs the instructions of the method of any one of claims 1-7.
10. A computer storage medium on which a computer program is stored, characterized in that the computer program, when being executed by a processor of a computer device, executes instructions of a method according to any one of claims 1-7.
CN202110637040.7A 2021-06-08 2021-06-08 Data interfacing method, device, equipment and storage medium between systems Pending CN113377420A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110637040.7A CN113377420A (en) 2021-06-08 2021-06-08 Data interfacing method, device, equipment and storage medium between systems

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110637040.7A CN113377420A (en) 2021-06-08 2021-06-08 Data interfacing method, device, equipment and storage medium between systems

Publications (1)

Publication Number Publication Date
CN113377420A true CN113377420A (en) 2021-09-10

Family

ID=77576538

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110637040.7A Pending CN113377420A (en) 2021-06-08 2021-06-08 Data interfacing method, device, equipment and storage medium between systems

Country Status (1)

Country Link
CN (1) CN113377420A (en)

Similar Documents

Publication Publication Date Title
CN110457912B (en) Data processing method and device and electronic equipment
US9619654B2 (en) Application monitoring through collective record and replay
EP2663948B1 (en) Secure computing in multi-tenant data centers
US9026577B1 (en) Distributed workflow management system
Xiao et al. Achieving accountable MapReduce in cloud computing
US11126711B2 (en) System and method for implementing a log source value tool for security information event management
CN108334436A (en) Optimization method, device, equipment and the computer readable storage medium of application software
US10242221B1 (en) System and method for automatically securing sensitive data in public cloud using a serverless architecture
CN108694238A (en) Business data processing method, device based on block chain and storage medium
US20220253335A1 (en) Automated virtual machine resource management in container-supported many task computing
CN106899648A (en) A kind of data processing method and equipment
CN110245004A (en) Command executing method, device, equipment and computer readable storage medium
CN112365001B (en) Model generation method, device and server
CN111061798B (en) Configurable data transmission and monitoring method, equipment and medium
CN107294955B (en) Electronic file encryption middleware control system and method
US11563727B2 (en) Multi-factor authentication for non-internet applications
US20130066954A1 (en) Computer software analysis system, client computer, method of controlling operation of same and operation program therefor
US20190227916A1 (en) Methods, systems, apparatuses and devices for facilitating execution of test cases
WO2019157122A1 (en) Rf tag operating system with iot connector core
EP3842980B1 (en) System and method for automatically securing sensitive data in public cloud using a serverless architecture
CN114265577A (en) Service data processing method and device, computer equipment and storage medium
CN110858242A (en) Page skipping method and device
CN113377420A (en) Data interfacing method, device, equipment and storage medium between systems
CN115114642A (en) Financing lease verification method, device and equipment based on under-chain trusted computing
US10095478B2 (en) Computer implemented system and method for identifying project requirements

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination