CN110674171A - Script generation method and device and data processing method and device - Google Patents
Script generation method and device and data processing method and device Download PDFInfo
- Publication number
- CN110674171A CN110674171A CN201910865961.1A CN201910865961A CN110674171A CN 110674171 A CN110674171 A CN 110674171A CN 201910865961 A CN201910865961 A CN 201910865961A CN 110674171 A CN110674171 A CN 110674171A
- Authority
- CN
- China
- Prior art keywords
- data processing
- rule
- script
- target
- configuration file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013515 script Methods 0.000 title claims abstract description 97
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000003672 processing method Methods 0.000 title claims description 11
- 238000012545 processing Methods 0.000 claims abstract description 183
- 230000008569 process Effects 0.000 description 5
- 238000004590 computer program Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24564—Applying rules; Deductive queries
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
A script generation method and device, and data processing and device comprise: acquiring a rule configuration file generated in advance according to a target data processing rule; wherein the target data processing rule is formulated according to the data processing target; and reading the rule configuration file to obtain the data processing script. According to the method and the device, the rule configuration file generated according to the target data processing rule is obtained in advance, and then the rule configuration file is read to obtain the data processing script, so that the automatic generation of the data processing script is realized, the labor cost is saved, and the script generation efficiency is improved.
Description
Technical Field
The present disclosure relates to computer technologies, and in particular, to a script generating method and apparatus, and a data processing method and apparatus.
Background
With the advent of the big data age, data with large data volume often needs to be processed to show the desired data to users.
In the related art, data processing is often completed through scripts, and due to different projects, developers need to directly write or modify existing scripts to obtain scripts suitable for new projects.
However, both the direct writing method and the existing script modifying method are completely performed manually, which consumes labor cost and reduces the efficiency of script generation.
Disclosure of Invention
The application provides a script generation method and device and a data processing method and device, which can realize automatic generation of a data processing script, thereby saving labor cost and improving script generation efficiency.
The application provides a script generation method, which is characterized by comprising the following steps:
acquiring a rule configuration file generated in advance according to a target data processing rule; wherein the target data processing rule is formulated according to a data processing target;
and reading the rule configuration file to obtain a data processing script.
Before the obtaining of the rule configuration file generated in advance according to the data processing rule, the method further includes:
determining a data processing rule capable of realizing the target according to the data processing target, and taking the obtained data processing rule as a target data processing rule;
and acquiring the target data processing rule from a preset data processing rule set.
Before reading the rule configuration file to obtain the data processing script, the method further includes:
acquiring a scanning script which is compiled in advance and used for reading a rule configuration file;
the reading of the rule configuration file to obtain a data processing script comprises:
and reading the rule configuration file by using the scanning script to obtain the data processing script.
The application also provides a data processing method, which comprises the following steps:
acquiring target data to be processed;
processing the target data by using a data processing script to obtain a data processing result; wherein the data processing script is generated according to any one of the script generation methods.
After the data processing result is obtained, the method further comprises the following steps:
and when the data processing result shows that the target data meets the data processing rule corresponding to the data processing script, establishing the corresponding relation between the target data and the data processing rule.
The present application further provides a script generating apparatus, including:
the first acquisition module is used for acquiring a rule configuration file generated in advance according to a target data processing rule; wherein the target data processing rule is formulated according to a data processing target;
and the first processing module is used for reading the rule configuration file to obtain a data processing script.
The first processing module is further configured to determine a data processing rule capable of achieving the target according to the data processing target, and use the obtained data processing rule as a target data processing rule;
the first obtaining module is further configured to obtain the target data processing rule from a preset data processing rule set.
The first acquisition module is also used for acquiring a scanning script which is compiled in advance and used for reading the rule configuration file;
the first processing module is specifically configured to read the rule configuration file by using the scan script to obtain the data processing script.
The present application also provides a data processing apparatus, including:
the second acquisition module is used for acquiring target data to be processed;
the second processing module is used for processing the target data by using the data processing script to obtain a data processing result; wherein the data processing script is generated by the script generation apparatus according to any one of the above.
The second processing module is further configured to establish a corresponding relationship between the target data and the data processing rule when the data processing result indicates that the target data meets the data processing rule corresponding to the data processing script.
Compared with the prior art, this application includes: acquiring a rule configuration file generated in advance according to a target data processing rule; wherein the target data processing rule is formulated according to the data processing target; and reading the rule configuration file to obtain the data processing script. The rule configuration file generated according to the target data processing rule is obtained in advance, and the data processing script can be obtained by reading the rule configuration file, so that the automatic generation of the data processing script is realized, the labor cost is saved, and the script generation efficiency is improved.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the application. Other advantages of the application may be realized and attained by the instrumentalities and combinations particularly pointed out in the specification, claims, and drawings.
Drawings
The accompanying drawings are included to provide an understanding of the present disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the examples serve to explain the principles of the disclosure and not to limit the disclosure.
Fig. 1 is a schematic flowchart of a script generation method according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a data processing method according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a script generating apparatus according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application.
Detailed Description
The present application describes embodiments, but the description is illustrative rather than limiting and it will be apparent to those of ordinary skill in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are possible. Any feature or element of any embodiment may be used in combination with or instead of any other feature or element in any other embodiment, unless expressly limited otherwise.
The present application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The embodiments, features and elements disclosed in this application may also be combined with any conventional features or elements to form a unique inventive concept as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive aspects to form yet another unique inventive aspect, as defined by the claims. Thus, it should be understood that any of the features shown and/or discussed in this application may be implemented alone or in any suitable combination. Accordingly, the embodiments are not limited except as by the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.
Further, in describing representative embodiments, the specification may have presented the method and/or process as a particular sequence of steps. However, to the extent that the method or process does not rely on the particular order of steps set forth herein, the method or process should not be limited to the particular sequence of steps described. Other orders of steps are possible as will be understood by those of ordinary skill in the art. Therefore, the particular order of the steps set forth in the specification should not be construed as limitations on the claims. Further, the claims directed to the method and/or process should not be limited to the performance of their steps in the order written, and one skilled in the art can readily appreciate that the sequences may be varied and still remain within the spirit and scope of the embodiments of the present application.
An embodiment of the present application provides a script generation method, as shown in fig. 1, the method includes:
And 102, reading the rule configuration file to obtain a data processing script.
In an exemplary embodiment, the script generation method provided by the embodiment of the present application is applied to a Browser/Server (Browser/Server, B/S) structure.
In an exemplary embodiment, before obtaining the rule configuration file generated in advance according to the data processing rule, the method further includes:
first, a data processing rule that can achieve a data processing target is determined from the target, and the obtained data processing rule is taken as a target data processing rule.
Next, a target data processing rule is obtained from a preset data processing rule set.
In an exemplary embodiment, before reading the rule configuration file to obtain the data processing script, the method further includes:
and acquiring a scanning script which is written in advance and used for reading the rule configuration file.
Reading the rule configuration file to obtain a data processing script, comprising:
and reading the rule configuration file by using the scanning script to obtain a data processing script.
According to the script generation method provided by the embodiment of the application, the rule configuration file generated according to the target data processing rule is obtained in advance, and then the rule configuration file is read to obtain the data processing script, so that the automatic generation of the data processing script is realized, the labor cost is saved, and the script generation efficiency is improved.
An embodiment of the present application further provides a data processing method, including:
first, target data to be processed is acquired.
Secondly, processing the target data by using the data processing script to obtain a data processing result; wherein, the data processing script is generated according to the script generation method described in any one of the above embodiments.
In an exemplary embodiment, after obtaining the data processing result, the method further comprises:
and when the data processing result is that the target data meets the data processing rule corresponding to the data processing script, establishing a corresponding relation between the target data and the data processing rule.
According to the data processing method provided by the embodiment of the application, the utilized data processing script is automatically generated, so that the data is rapidly processed, and the data processing efficiency is improved.
An embodiment of the present application further provides a data processing method, as shown in fig. 2, including:
step 201, writing and maintaining a set of data analysis rule base (corresponding to the rule configuration file in the above embodiment).
Step 202, reading the rule base file through the scanning script to generate a dotting counter (corresponding to the data processing script in the above embodiment).
And 203, analyzing the system data, and dotting by using a dotting counter.
And step 204, storing the counted rules into a rule base table.
And step 205, returning data by connecting the table query rule base table with the data table.
The data processing method provided by the embodiment of the application can realize automatic processing of data, greatly reduce the maintenance cost of later data processing, and a World Wide Web (Web) terminal can display statistical data of various processing rules, including performance statistics, log analysis and the like, and can monitor the running state of a system or be used as an effective means for quickly and accurately positioning system faults.
An embodiment of the present application further provides a script generating apparatus, as shown in fig. 3, where the script generating apparatus 3 includes:
a first obtaining module 31, configured to obtain a rule configuration file generated in advance according to a target data processing rule; wherein the target data processing rule is formulated according to the data processing target.
The first processing module 32 is configured to read the rule configuration file to obtain the data processing script.
In an exemplary example, the first processing module 32 is further configured to determine a data processing rule capable of achieving the goal according to the data processing goal, and take the obtained data processing rule as the target data processing rule.
The first obtaining module 31 is further configured to obtain a target data processing rule from a preset data processing rule set.
In an exemplary embodiment, the first obtaining module 31 is further configured to obtain a scan script written in advance for reading the rule configuration file.
The first processing module 32 is specifically configured to read the rule configuration file by using the scan script to obtain the data processing script.
According to the script generation device provided by the embodiment of the application, the rule configuration file generated according to the target data processing rule is obtained in advance, and then the rule configuration file is read to obtain the data processing script, so that the automatic generation of the data processing script is realized, the labor cost is saved, and the script generation efficiency is improved.
In practical applications, the first obtaining module 31 and the first Processing module 32 are both implemented by a Central Processing Unit (CPU), a microprocessor Unit (MPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), or the like in the script generating device.
An embodiment of the present application further provides a data processing apparatus, as shown in fig. 4, including:
and a second obtaining module 41, configured to obtain target data to be processed.
A second processing module 42, configured to process the target data by using the data processing script to obtain a data processing result; wherein the data processing script is generated by the script generating device described in any one of the above embodiments.
In an exemplary embodiment, the second processing module 42 is further configured to establish a corresponding relationship between the target data and the data processing rule when the data processing result is that the target data satisfies the data processing rule corresponding to the data processing script.
According to the data processing device provided by the embodiment of the application, the utilized data processing script is automatically generated, so that the data is rapidly processed, and the data processing efficiency is improved.
In practical applications, the second obtaining module 41 and the second processing module 42 are implemented by a CPU, an MPU, a DSP, or an FPGA in the data processing apparatus.
An embodiment of the present application further provides a data processing apparatus, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the computer program is executed by the processor, the computer program implements the processing of any one of the methods described above.
Embodiments of the present application further provide a computer-readable storage medium, on which computer-executable instructions are stored, where the computer-executable instructions are used to execute the processing of any one of the methods described above.
It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
Claims (10)
1. A script generation method, comprising:
acquiring a rule configuration file generated in advance according to a target data processing rule; wherein the target data processing rule is formulated according to a data processing target;
and reading the rule configuration file to obtain a data processing script.
2. The method according to claim 1, wherein before the obtaining the rule configuration file generated in advance according to the data processing rule, the method further comprises:
determining a data processing rule capable of realizing the target according to the data processing target, and taking the obtained data processing rule as a target data processing rule;
and acquiring the target data processing rule from a preset data processing rule set.
3. The method of claim 1, wherein prior to reading the rule configuration file to obtain a data processing script, further comprising:
acquiring a scanning script which is compiled in advance and used for reading a rule configuration file;
the reading of the rule configuration file to obtain a data processing script comprises:
and reading the rule configuration file by using the scanning script to obtain the data processing script.
4. A data processing method, comprising:
acquiring target data to be processed;
processing the target data by using a data processing script to obtain a data processing result; wherein the data processing script is generated according to the script generation method of any one of claims 1-3.
5. The method of claim 4, wherein after obtaining the data processing result, further comprising:
and when the data processing result shows that the target data meets the data processing rule corresponding to the data processing script, establishing the corresponding relation between the target data and the data processing rule.
6. A script generation apparatus, comprising:
the first acquisition module is used for acquiring a rule configuration file generated in advance according to a target data processing rule; wherein the target data processing rule is formulated according to a data processing target;
and the first processing module is used for reading the rule configuration file to obtain a data processing script.
7. The apparatus of claim 6, wherein the first processing module is further configured to determine, according to the data processing target, a data processing rule that can achieve the target, and take the obtained data processing rule as a target data processing rule;
the first obtaining module is further configured to obtain the target data processing rule from a preset data processing rule set.
8. The apparatus of claim 6, wherein the first obtaining module is further configured to obtain a scan script written in advance for reading a rule configuration file;
the first processing module is specifically configured to read the rule configuration file by using the scan script to obtain the data processing script.
9. A data processing apparatus, comprising:
the second acquisition module is used for acquiring target data to be processed;
the second processing module is used for processing the target data by using the data processing script to obtain a data processing result; wherein the data processing script is generated by the script generating apparatus according to any one of claims 6-8.
10. The apparatus according to claim 9, wherein the second processing module is further configured to establish a corresponding relationship between the target data and the data processing rule when the data processing result indicates that the target data satisfies the data processing rule corresponding to the data processing script.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910865961.1A CN110674171A (en) | 2019-09-12 | 2019-09-12 | Script generation method and device and data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910865961.1A CN110674171A (en) | 2019-09-12 | 2019-09-12 | Script generation method and device and data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110674171A true CN110674171A (en) | 2020-01-10 |
Family
ID=69076777
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910865961.1A Pending CN110674171A (en) | 2019-09-12 | 2019-09-12 | Script generation method and device and data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110674171A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110826069A (en) * | 2019-11-05 | 2020-02-21 | 深信服科技股份有限公司 | Virus processing method, device, equipment and storage medium |
| CN111400353A (en) * | 2020-03-19 | 2020-07-10 | 中国建设银行股份有限公司 | Rule calculation method, device, equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1635497A (en) * | 2003-12-30 | 2005-07-06 | 英业达股份有限公司 | Hardware-level based test script automatic generating system and method |
| CN101631066A (en) * | 2008-07-14 | 2010-01-20 | 中兴通讯股份有限公司 | Method and system for realizing terminal quality of service of internet voice transmission system |
| CN108279866A (en) * | 2018-01-24 | 2018-07-13 | 马上消费金融股份有限公司 | Method, device and medium for arranging and executing business process |
| CN108958729A (en) * | 2017-05-19 | 2018-12-07 | 腾讯科技（深圳）有限公司 | A kind of data processing method, device and storage medium |
| CN109445797A (en) * | 2018-10-24 | 2019-03-08 | 北京奇虎科技有限公司 | Handle task executing method and device |
| CN110109897A (en) * | 2019-04-15 | 2019-08-09 | 深圳壹账通智能科技有限公司 | Database script generation method, device, computer equipment and storage medium |
-
2019
- 2019-09-12 CN CN201910865961.1A patent/CN110674171A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1635497A (en) * | 2003-12-30 | 2005-07-06 | 英业达股份有限公司 | Hardware-level based test script automatic generating system and method |
| CN101631066A (en) * | 2008-07-14 | 2010-01-20 | 中兴通讯股份有限公司 | Method and system for realizing terminal quality of service of internet voice transmission system |
| CN108958729A (en) * | 2017-05-19 | 2018-12-07 | 腾讯科技（深圳）有限公司 | A kind of data processing method, device and storage medium |
| CN108279866A (en) * | 2018-01-24 | 2018-07-13 | 马上消费金融股份有限公司 | Method, device and medium for arranging and executing business process |
| CN109445797A (en) * | 2018-10-24 | 2019-03-08 | 北京奇虎科技有限公司 | Handle task executing method and device |
| CN110109897A (en) * | 2019-04-15 | 2019-08-09 | 深圳壹账通智能科技有限公司 | Database script generation method, device, computer equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110826069A (en) * | 2019-11-05 | 2020-02-21 | 深信服科技股份有限公司 | Virus processing method, device, equipment and storage medium |
| CN111400353A (en) * | 2020-03-19 | 2020-07-10 | 中国建设银行股份有限公司 | Rule calculation method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107122296B (en) | Method and apparatus for data assertion for test interface | |
| CN109344066B (en) | Method, system and terminal for testing browser page | |
| CN107402878B (en) | Test method and device | |
| CN112416964A (en) | Data processing method, device and system, computer equipment and computer readable storage medium | |
| WO2018006721A1 (en) | Method and apparatus for storing log file | |
| CN111008246A (en) | Database log synchronization method and device, computer equipment and readable storage medium | |
| CN108228443B (en) | Web application testing method and device | |
| CN110674171A (en) | Script generation method and device and data processing method and device | |
| CN115203004A (en) | Code coverage rate testing method and device, storage medium and electronic equipment | |
| CN111367948B (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN115390847A (en) | Log processing method and device, computer readable storage medium and terminal | |
| CN109388564B (en) | Test method and device and electronic equipment | |
| CN108647102B (en) | Service request processing method and device of heterogeneous system and electronic equipment | |
| CN109901950A (en) | A kind of method and device for evading application crash | |
| CN110347597B (en) | Interface testing method and device of picture server, storage medium and mobile terminal | |
| CN109446462B (en) | Page-based data monitoring processing method, device, equipment and storage medium | |
| CN112256984A (en) | Method and device for acquiring interface background screenshot corresponding to webpage | |
| US10372513B2 (en) | Classification of application events using call stacks | |
| CN108600042B (en) | WiFi test method and device for electronic equipment, storage medium and test equipment | |
| CN109656805B (en) | Method and device for generating code link for business analysis and business server | |
| CN114500348A (en) | CDN gateway test method and system | |
| CN109688578B (en) | Method and equipment for adapting SIM card and mobile terminal equipment | |
| CN114385493A (en) | Performance test method and device, electronic equipment and storage medium | |
| CN109522124B (en) | Storage management system loading method, device, equipment and readable storage medium | |
| CN109522205B (en) | Simulation test method and device, computer storage medium and terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200110 |
|
| RJ01 | Rejection of invention patent application after publication |