CN111124370A - Data processing method and related equipment - Google Patents

Data processing method and related equipment Download PDF

Info

Publication number
CN111124370A
CN111124370A CN201911349789.0A CN201911349789A CN111124370A CN 111124370 A CN111124370 A CN 111124370A CN 201911349789 A CN201911349789 A CN 201911349789A CN 111124370 A CN111124370 A CN 111124370A
Authority
CN
China
Prior art keywords
script
target
file
jmx
data processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911349789.0A
Other languages
Chinese (zh)
Other versions
CN111124370B (en
Inventor
张熹
金向平
陈斌
佘亚飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Home Cloud Network Technology Co Ltd
Original Assignee
Shenzhen Home Cloud Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Home Cloud Network Technology Co Ltd filed Critical Shenzhen Home Cloud Network Technology Co Ltd
Priority to CN201911349789.0A priority Critical patent/CN111124370B/en
Publication of CN111124370A publication Critical patent/CN111124370A/en
Application granted granted Critical
Publication of CN111124370B publication Critical patent/CN111124370B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/20Software design
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • G06F16/258Data format conversion from or to a database
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/448Execution paradigms, e.g. implementations of programming paradigms
    • G06F9/4488Object-oriented
    • G06F9/449Object-oriented method invocation or resolution

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Stored Programmes (AREA)

Abstract

The embodiment of the application discloses a data processing method which is used for automatically converting a script into a database application. The method in the embodiment of the application comprises the following steps: the method comprises the steps of obtaining an ant xsl script and a jmeter jmx script, automatically converting an initial script into a target file which can be executed by a target database through the cooperation of the xsl script and the jmx script, and automatically converting the initial script written by developers so as to meet the requirements of different databases, so that operation and maintenance personnel are saved from participating in the specific work of script execution, and the overall work efficiency is improved.

Description

Data processing method and related equipment
Technical Field
The embodiment of the application relates to the field of databases, in particular to a data processing method and related equipment.
Background
The script is a strip of word command, which can be opened and viewed or edited by notebook, when the script program runs, the interpreter of the system translates the word command into machine recognizable instruction, and then executes the script.
The script operation in the prior art mainly adopts the following modes: the developer writes a database script and then manually copies the database script to a database to be operated, so that the system operates the script.
Disclosure of Invention
The embodiment of the application provides a data processing method and related equipment, which can automatically convert a script written by a developer into a file which can be executed by a database.
A first aspect of an embodiment of the present application provides a data processing method, including:
acquiring an extensible stylesheet language script (the format can be. xml) of ant and a Java management extension script (the format can be. jmx) of jemter;
calling jmx script through xsl script;
sending the target request to the interface through jmx script;
receiving an initial script sent by an interface, wherein the initial script is a script written by a developer and is associated with a target request;
saving the initial script to obtain a hypertext markup language file (the format can be html);
and converting the html file into a target file which can be executed by the target database.
Preferably, after the shell script calls the JQ tool to convert the html file into a target file which can be executed by the target database, the timed task of the jmx script is configured through jenkins, and the timed task is used for updating jmx script.
Preferably, after the shell script calls the JQ tool to convert the html file into a target file that can be executed by the target database, the method further includes:
putting the target file into a target database for execution to obtain an execution result; and saving the execution result.
Preferably, obtaining an anti extensible stylesheet language xsl script, a jerter jmx script and a computer shell layer shell script; and calling a JQ tool through the shell script to convert the html file into a target file which can be executed by a target database, and then backing up the target file.
Preferably, the html file is converted into the target file by a shell script and/or a JQ tool.
Preferably, the target database comprises a mystructured query language (mysql) database, and the target file comprises a structured query language file (which may be in the format of sql).
Preferably, the initial script is a data repair script or a version release script.
Preferably, the initial script is a script written by the developer for the current day.
A second aspect of the embodiments of the present application provides a data processing apparatus, including:
the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring an xsl script of the ant extensible stylesheet language and an jmx script of the jemter;
the calling unit is used for calling jmx scripts through the xsl scripts;
a sending unit, configured to send the target request to the interface through jmx script;
the receiving unit is used for receiving an initial script sent by the interface, the initial script is a script written by a developer, and the initial script is associated with the target request;
the saving unit is used for saving the initial script to obtain the html file;
and the conversion unit is used for converting the html file into a target file which can be executed by the target database.
Embodiments of the present application further provide a computer storage medium, in which instructions are stored, and when executed on a computer, the instructions cause the computer to perform the method according to the first aspect.
Embodiments of the present application also provide a computer program product, which when executed on a computer causes the computer to execute the method according to the first aspect.
According to the technical scheme, the embodiment of the application has the following advantages: in the embodiment of the application, the initial script is automatically converted into the target file which can be executed by the target database through the ant xsl script and the jmeter jmx script, and the initial script written by developers is automatically converted to adapt to the requirements of different databases, so that operation and maintenance personnel are saved to participate in the script to execute specific work, and the overall working efficiency is improved.
Drawings
FIG. 1 is a schematic flow chart of a data processing method according to an embodiment of the present application;
FIG. 2 is another schematic flow chart illustrating a data processing method according to an embodiment of the present application;
FIG. 3 is another schematic flow chart illustrating a data processing method according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a data processing apparatus in an embodiment of the present application.
Detailed Description
The embodiment of the application provides a data processing method and related equipment, which are used for automatically converting a script written by a developer into a file which can be executed by a database.
The data processing method in the embodiment of the application is applicable to both the linux environment and the windows environment, and is not limited herein, and is described below only in the linux environment.
Referring to fig. 1, an embodiment of a data processing method in the embodiment of the present application includes:
101. acquiring an xsl script and an jmx script;
developers or other workers write xsl scripts under the ant architecture and jmx scripts under the meter architecture.
In the embodiment of the present application, only the written scripts are divided into four types: the to-be-reviewed (review), the to-be-audited (edited), the to-be-executed (unexecuted), and the executed (executed) are examples, and the specific classification may be set as needed, which is not limited herein.
After writing, the data processing device obtains one type of xsl script and jmx script of the four types according to requirements.
102. Calling jmx script through xsl script;
the data processing device invokes jmx scripts with ant's sxl script.
103. Sending the target request to the interface through jmx script;
the interface in the embodiment of the present application is described only by taking the interface as an API interface, and it is understood that in practical application, the interface may be another interface, and is not limited herein.
And communicating with developers in advance to determine target information, wherein the target information can comprise an interface providing function, interface access parameters and interface return values. After determining the target information, the data processing device sends a target request to the interface through the jmx script, the target request being related to the target information.
104. Receiving an initial script sent by an interface;
after receiving the target request, the API interface sends an initial script to the data processing device according to target information determined by communication with the developer in advance, where the initial script is associated with the target request, the initial script is written by the developer, and the initial script may be written on the same day or within a certain period, and is set as required, and the specific details are not limited herein.
The initial script may be a script for data recovery or a script for version release, and it is understood that in actual application, the initial script may also be other scripts according to needs, and the specific details are not limited herein.
105. Saving the initial script to obtain an html file;
and after the data processing equipment receives the initial script sent by the API, storing the initial script to a specified directory, wherein the specified directory is set according to needs, and the specific path is not limited here.
Since the format of the saved file of the jmeter is html by default, the initial script becomes an html file after being saved.
106. And converting the html file into a target file which can be executed by the target database.
In this embodiment of the present application, there may be many ways to convert an html file into a target file that can be executed by a target database, and the html file may be converted into the target file that can be executed by the target database by executing a shell script, and the html file may also be converted into the target file that can be executed by the target database by using a JQ tool, which is understood that in practical applications, the html file may also be converted by other ways, for example: and the JQ tool can be called by the shell script to convert the html file into a target file which can be executed by the target database.
In this embodiment, the initial script is automatically converted into the target file that can be executed by the target database through the anti xsl script and the jmeter jmx script, and the initial script written by the developer is automatically converted to meet the requirements of different databases, so that the operation and maintenance personnel are saved from participating in the script to execute specific work, and the overall working efficiency is improved.
Referring to fig. 2, another embodiment of the data processing method in the embodiment of the present application includes:
201. acquiring an xsl script, an jmx script and a shell script;
202. calling jmx script through xsl script;
203. sending the target request to the interface through jmx script;
204. receiving an initial script sent by an interface;
205. saving the initial script to obtain an html file;
206. and converting the html file into a target file which can be executed by the target database.
The target database in the embodiment of the application can be a mysql database, and the target file is an sql file; the file system can also be an Access database, and the target file is a mdb file; it will be appreciated that in practical applications, the target database may also be other types of databases, and the target file may also be other types of files, such as: the target database may also be Microsoft's SQL database (MSSql), the target file is an.mdf file; the target database can also be a Paradox database, and the target file is a DB file; the target database can also be an Oracle database, and the target file is a DBF file; the target database can also be a FoxPro database, and the target file is a DBF file; the target database can also be an MS Works database, and the target file is a. wdb file; the details are not limited herein.
Steps 201 to 206 in this embodiment are similar to steps 101 to 106 in the embodiment shown in fig. 1, and are not described again here.
207. Configuring jmx timing tasks of scripts through jenkins;
the data processing device may set a calling time according to actual needs, and if the data processing device is applied to the test database, since the test database is frequently updated, the calling frequency may be increased, for example, 10 minutes per time, when the timing task is configured; assuming that the application can be set to 30 minutes per time when the data is automated, the specific timing task is set according to the actual requirement, and is not limited herein.
The script is updated through the timed task, so that the normal operation of the automatic test is ensured, and the failure of subsequent execution results due to the untimely update of the database script is avoided, so that the working progress is not influenced.
208. Putting the target file into a target database for execution to obtain an execution result;
as described in the foregoing step 206, assuming that the target file is the sql file, the target database is the mysql database, and the data processing device puts the sql file into the mysql database for execution, thereby obtaining an execution result.
The execution result may be success or failure, and is not limited herein.
209. Saving the execution result;
after the execution result is obtained, the execution result is stored, the storage mode can be various, the execution result can be recorded in a log mode, and the follow-up problem troubleshooting is facilitated.
210. And backing up the target file.
And the executed target file is backed up, so that the problem of executing the failed or missed script is conveniently checked.
In this embodiment, the initial script is automatically converted into the target file that can be executed by the target database through the anti xsl script, the jmeter jmx and the shell script, the initial script is automatically converted into the target file that can be executed by the target database by calling the JQ tool through the shell script, and the initial script written by the developer is automatically converted to meet the requirements of different databases, so that the operation and maintenance personnel are saved from participating in the specific work of script execution, and the overall working efficiency is improved.
The embodiment shown in fig. 2 can be applied to a production database, a test database and an automation database, please refer to fig. 3, wherein the specific flow shown in fig. 3 is similar to the corresponding steps of the embodiment shown in fig. 2, and is not limited herein.
In the embodiment of the application, a timing task can be set through jenkins, an ant tool is called for 5 minutes/time, a JQ tool is called for 10 minutes/time, specific time is set according to needs, and the specific time is not limited here.
With reference to fig. 4, the data processing method in the embodiment of the present application is described above, and a data processing device in the embodiment of the present application is described below, where an embodiment of the data processing device in the embodiment of the present application includes:
an obtaining unit 401, configured to obtain an xsl script of the extensible stylesheet language ant and an jmx script of the jemter;
a calling unit 402, configured to call jmx a script through an xsl script;
a sending unit 403, configured to send the target request to the interface through jmx script;
a receiving unit 404, configured to receive an initial script sent by an interface, where the initial script is a script written by a developer and is associated with a target request;
a saving unit 405, configured to save the initial script to obtain an html file;
and a converting unit 406, configured to convert the html file into a target file that can be executed by the target database.
In this embodiment, operations performed by each unit in the data processing apparatus are similar to those described in the foregoing embodiments shown in fig. 1 and fig. 2, and are not described again here.
In this embodiment, the initial script is automatically converted into a target file that can be executed by the target database by the conversion unit 406, and the initial script written by the developer is automatically converted to meet the requirements of different databases, so that the operation and maintenance personnel are saved from participating in the specific work of script execution, and the overall work efficiency is improved.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and various other media capable of storing program codes.

Claims (10)

1. A data processing method, comprising:
acquiring an xsl script of an ant extensible stylesheet language and an jmx script of a jemter;
calling the jmx script through the xsl script;
sending a target request to an interface through the jmx script;
receiving an initial script sent by the interface, wherein the initial script is a script written by a developer and is associated with the target request;
saving the initial script to obtain a hypertext markup language html file;
and converting the html file into a target file which can be executed by a target database.
2. The data processing method of claim 1, wherein after converting the html file into a target file executable by a target database, the method further comprises:
configuring a timing task of the jmx script through jenkins, wherein the timing task is used for updating the jmx script.
3. The data processing method of claim 1, wherein after converting the html file into a target file executable by a target database, the method further comprises:
putting the target file into the target database for execution to obtain an execution result;
and saving the execution result.
4. The data processing method of claim 1, wherein after converting the html file into a target file executable by a target database, the method further comprises:
and backing up the target file.
5. The data processing method of claim 1, wherein obtaining an extensible stylesheet language xsl script of ant and an jmx script of jemter comprises:
acquiring an extensible stylesheet language xsl script of ant, an jmx script of j emter and a shell script of a computer shell layer;
converting the html file into a target file which can be executed by a target database comprises the following steps:
and converting the html file into the target file through the shell script and/or the JQ tool.
6. The data processing method of claim 1, wherein the target database comprises a relational database management system (mysql) database, and the target file comprises a structured query language (sql) file.
7. The data processing method of claim 1, wherein the initial script is a data repair script or a version release script.
8. A data processing apparatus, characterized by comprising:
the acquisition unit is used for acquiring an xsl script of ant and an jmx script of jemter;
the calling unit is used for calling the jmx script through the xsl script;
a sending unit, configured to send a target request to an interface through the jmx script;
a receiving unit, configured to receive an initial script sent by the interface, where the initial script is a script written by a developer and is associated with the target request;
the saving unit is used for saving the initial script to obtain an html file;
and the conversion unit is used for converting the html file into a target file which can be executed by a target database.
9. A computer storage medium having stored therein instructions that, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 7.
10. A computer program product, characterized in that the computer program product, when executed on a computer, causes the computer to perform the method according to any of claims 1 to 7.
CN201911349789.0A 2019-12-24 2019-12-24 Data processing method and related equipment Active CN111124370B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911349789.0A CN111124370B (en) 2019-12-24 2019-12-24 Data processing method and related equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911349789.0A CN111124370B (en) 2019-12-24 2019-12-24 Data processing method and related equipment

Publications (2)

Publication Number Publication Date
CN111124370A true CN111124370A (en) 2020-05-08
CN111124370B CN111124370B (en) 2024-04-26

Family

ID=70502037

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911349789.0A Active CN111124370B (en) 2019-12-24 2019-12-24 Data processing method and related equipment

Country Status (1)

Country Link
CN (1) CN111124370B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112948480A (en) * 2021-04-21 2021-06-11 平安好医投资管理有限公司 Data extraction method and device, electronic equipment and storage medium
CN113282473A (en) * 2021-05-28 2021-08-20 平安国际智慧城市科技股份有限公司 Server performance testing method and device, computer equipment and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105868204A (en) * 2015-01-21 2016-08-17 中国移动(深圳)有限公司 Method and apparatus for converting script language SQL of Oracle
US20170206245A1 (en) * 2016-01-20 2017-07-20 Oracle International Corporation System and method for dynamic conversion of database accessing scripts during runtime in a mainframe rehosting platform
CN108710569A (en) * 2018-05-09 2018-10-26 深圳壹账通智能科技有限公司 Test cases generation method, data creation method, device, equipment and medium
CN109408528A (en) * 2018-10-25 2019-03-01 北京信安世纪科技股份有限公司 A kind of database script generation method, device, computing device and storage medium
CN109597729A (en) * 2018-10-29 2019-04-09 深圳壹账通智能科技有限公司 A kind of monitoring method and device based on analog subscriber test
CN110046095A (en) * 2019-03-18 2019-07-23 平安普惠企业管理有限公司 Based on the improved system integration method of testing process and device
CN110275822A (en) * 2019-04-26 2019-09-24 武汉众邦银行股份有限公司 Performance test methods, device, equipment and the storage medium of application programming interfaces

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105868204A (en) * 2015-01-21 2016-08-17 中国移动(深圳)有限公司 Method and apparatus for converting script language SQL of Oracle
US20170206245A1 (en) * 2016-01-20 2017-07-20 Oracle International Corporation System and method for dynamic conversion of database accessing scripts during runtime in a mainframe rehosting platform
CN108710569A (en) * 2018-05-09 2018-10-26 深圳壹账通智能科技有限公司 Test cases generation method, data creation method, device, equipment and medium
CN109408528A (en) * 2018-10-25 2019-03-01 北京信安世纪科技股份有限公司 A kind of database script generation method, device, computing device and storage medium
CN109597729A (en) * 2018-10-29 2019-04-09 深圳壹账通智能科技有限公司 A kind of monitoring method and device based on analog subscriber test
CN110046095A (en) * 2019-03-18 2019-07-23 平安普惠企业管理有限公司 Based on the improved system integration method of testing process and device
CN110275822A (en) * 2019-04-26 2019-09-24 武汉众邦银行股份有限公司 Performance test methods, device, equipment and the storage medium of application programming interfaces

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112948480A (en) * 2021-04-21 2021-06-11 平安好医投资管理有限公司 Data extraction method and device, electronic equipment and storage medium
CN112948480B (en) * 2021-04-21 2023-11-14 平安好医投资管理有限公司 Data extraction method, device, electronic equipment and storage medium
CN113282473A (en) * 2021-05-28 2021-08-20 平安国际智慧城市科技股份有限公司 Server performance testing method and device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN111124370B (en) 2024-04-26

Similar Documents

Publication Publication Date Title
US7898679B2 (en) Method and system for scheduling jobs in a computer system
CN108076098B (en) Service processing method and system
US5923833A (en) Restart and recovery of OMG-compliant transaction systems
CN100435101C (en) Apparatus and method for maintaining resource integrity in a software environment
CN109739482B (en) Service logic execution system and method based on dynamic language
CN103577551A (en) Method and device for submitting data to database in batch
CN109933632B (en) Data migration method, device and equipment for database
CN110716793B (en) Method, device, equipment and storage medium for executing distributed transaction
EP3722973A1 (en) Data processing method and device for distributed database, storage medium, and electronic device
CN103488793A (en) User behavior monitoring method based on information retrieval
CN112286661B (en) Task scheduling method and device, storage medium and terminal
CN111124370A (en) Data processing method and related equipment
US11983519B2 (en) Abort installation of firmware bundles
CN108920260B (en) Interaction method and device for heterogeneous system
CN113742043A (en) Asynchronous splitting method for server backend task
CN109857716B (en) System interaction log recording method and device, storage medium and server
US20060271929A1 (en) Method and system for providing job forecasting in a computer system
CN111427695A (en) Concurrent scheduling device for storage process in distributed database
CN113934575A (en) Big data backup system and method based on distributed copy
US8150821B2 (en) System and method for using generic utilities to perform database utilities on mainframe operated DB2 databases
CN111897794A (en) Database maintenance method and device, electronic equipment and storage medium
CN115599868B (en) Data real-time synchronous processing method, system, equipment and medium
CN112749156A (en) Data processing method, database management system and data processing equipment
CN111861381B (en) Access method and system of multi-fund channel
CN111651197B (en) Automatic warehouse moving method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant