CN111190954A - Power supply service command system based on button drawing technology - Google Patents
Power supply service command system based on button drawing technology Download PDFInfo
- Publication number
- CN111190954A CN111190954A CN202010027906.8A CN202010027906A CN111190954A CN 111190954 A CN111190954 A CN 111190954A CN 202010027906 A CN202010027906 A CN 202010027906A CN 111190954 A CN111190954 A CN 111190954A
- Authority
- CN
- China
- Prior art keywords
- conversion
- job
- execution module
- data
- power supply
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
Abstract
The invention relates to a power supply service command system based on a button drawing technology, which comprises a conversion design module, a Job execution module, a conversion execution module and a Job design module, wherein the conversion design module is used for carrying out conversion according to the power supply service command system; the conversion design module is used for developing conversion and operation in a graphical mode for the acquired power supply data and transmitting the graphical data to the Job execution module; the Job execution module is used for calling Job by using a command line and storing graphical data; the conversion execution module is used for calling Trans in a command line form and carrying out conversion analysis on the graphical data; and the Job design module is used for starting a plurality of card service processes to extract, distribute and supply power and receive when the system calls the cluster. The invention can provide high-efficiency data extraction, greatly reduce the workload and improve the working efficiency.
Description
Technical Field
The invention relates to the field of power systems, in particular to a power supply service command system based on a button drawing technology.
Background
For enterprise or industrial application, we will often encounter the use of various data processing, conversion, migration, and ETL tools, and it is essential that a key is one of ETL tools, has a very powerful function, and as an important component of Pentaho, it is now gradually increased in domestic project application, supports a graphical GUI design interface, and can then be circulated in a workflow form, and has a relatively stable expression in the aspects of making some simple or complex data extraction, quality detection, data cleaning, data conversion, data filtering, and so on, where most of us reduce a lot of workload and improve our work efficiency by applying it skillfully.
Reasonable use of clusters in a button may speed up execution and may also continue to be used in the event of partial server downtime. In the button, the cluster is composed of a master Carte server and a plurality of slave Carte servers. When the conversion is executed, the main server is responsible for distributing and tracking tasks and collecting result aggregates, the secondary server is responsible for specific conversion execution, and the button does not need to be deployed in the core server, so that no pressure is caused on the core server, and meanwhile, data below ten million levels can be efficiently processed.
Disclosure of Invention
In view of this, the present invention provides a power supply service command system based on a button drawing technology, which can provide an efficient data extraction, greatly reduce the workload, and improve the work efficiency.
The invention is realized by adopting the following scheme: a power supply service command system based on a button drawing technology comprises a conversion design module (Spoon), a Job execution module (Kitchen), a conversion execution module (Pan) and a Job design module (Carte); the conversion design module is used for developing conversion and operation in a graphical mode for the acquired power supply data and transmitting the graphical data to the Job execution module; the Job execution module is used for calling Job by using a command line and storing graphical data; the conversion execution module is used for calling Trans in a command line form and carrying out conversion analysis on the graphical data; and the Job design module is used for starting a plurality of card service processes to extract, distribute and supply power and receive when the system calls the cluster.
Preferably, the invention provides a working method of a power supply service command system based on a button drawing technology, which comprises the following steps:
step S1: acquiring power supply data, developing conversion and operation on the acquired power supply data in a graphical mode by the conversion design module, and transmitting the converted graphical data to the Job execution module;
step S2: the Job execution module calls Job storage graphical data by using a command line for analysis by the conversion execution module;
step S3: the conversion execution module calls Trans in a command line mode to analyze data;
step S4: and the Job design module starts a plurality of card service processes to extract, distribute and supply power and receive the data analyzed by the conversion execution module when the system calls the cluster.
Compared with the prior art, the invention has the following beneficial effects:
the invention can provide high-efficiency data extraction, greatly reduce the workload and improve the working efficiency.
Drawings
Fig. 1 is a block diagram of a system configuration according to an embodiment of the present invention.
Detailed Description
The invention is further explained below with reference to the drawings and the embodiments.
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
As shown in fig. 1, the present embodiment provides a power supply service command system based on a button drawing technology, which includes a conversion design module (Spoon), a Job execution module (Kitchen), a conversion execution module (Pan), and a Job design module (Carte); the conversion design module is used for developing conversion and operation in a graphical mode for the acquired power supply data and transmitting the graphical data to the Job execution module; the Job execution module is used for calling Job by using a command line and storing graphical data; the conversion execution module is used for calling Trans in a command line form and carrying out conversion analysis on the graphical data; and the Job design module is used for starting a plurality of card service processes to extract, distribute and supply power and receive when the system calls the cluster.
Preferably, the embodiment provides a working method of a power supply service command system based on a button drawing technology, including the following steps:
step S1: acquiring power supply data, developing conversion and operation on the acquired power supply data in a graphical mode by the conversion design module, and transmitting the converted graphical data to the Job execution module;
step S2: the Job execution module calls Job storage graphical data by using a command line for analysis by the conversion execution module;
step S3: the conversion execution module calls Trans in a command line mode to analyze data;
step S4: and the Job design module starts a plurality of card service processes to extract, distribute and supply power and receive the data analyzed by the conversion execution module when the system calls the cluster.
Preferably, in the present embodiment,
the keytte technology, an open-source ETL tool and pure java writing can be operated on Windows, Linux and Unix, and data extraction is efficient and stable.
Easy configuration: the method can be operated on Window, Linux and Unix, is green and does not need to be installed, and data extraction is efficient and stable.
Different databases: ETL toolset, which allows you to manage data from different databases.
Two types of scripts: transformation, which completes the underlying transformation for the data, and joba, which completes the control of the entire workflow.
Designing a graphical interface: the method realizes what service is done through graphical interface design without writing codes.
Timing function: the start module under Job has a timing function that can be timed daily, weekly, etc.
The button has four main components: spoon, Kitchen, Pan, Carte
Spoon-transform design tool (GUI approach): the method is a graphical interface, and can enable people to develop conversion and operation in a graphical mode. windows selection spoon. Sh for selecting Spoon
Kitchen — work (job) executor (command line mode): job may be called using a command line with Kitchen
Pan-transform actuator (command line approach): using Pan, Trans can be called in the form of a command line
Carte — job design tool (GUI mode): when a cluster (cluster) is called by a button to perform distributed distribution and processing tasks, a plurality of card service processes can be started to perform ETL (master) distribution tasks and receiving, running and submitting ETL tasks (slave).
JOB: it is actually a task stream in a button for calling transformation and other JOBs, consisting of an entry and a hop.
(1) And (4) hop: generally, only one Hop is arranged between the control and the control, and in operation, the Hop is mainly used for controlling the flow and has three states: one lock, one green pair, one red cross. Wherein:
locking: indicating that the next step must be performed regardless of whether the previous step was performed successfully or failed.
Green number matching: indicating that the previous step was successfully performed before the next step.
Red color difference number: indicating that the previous step failed to execute the next step.
(2) entry: in a specific execution process, after the jobstarts, the next time stamp updating is executed, dummy (placeholder, no operation is executed) is executed after the updating fails, and after the updating succeeds, the result is converted into sql and the result is output.
Transformation: the components are step and hop
Transformation defines a container for data operation, which is a process of data from input to output, and can be understood as a container with a granularity one level smaller than that of Job, a task is decomposed into Job, and then the Job needs to be decomposed into one or more transformations, and each Transformation only completes a part of work.
Step is the minimum unit inside the Transformation, and each Step completes a specific function.
Hop, for connecting Step in, is a graphical representation of a data stream.
The above description is only a preferred embodiment of the present invention, and all equivalent changes and modifications made in accordance with the claims of the present invention should be covered by the present invention.
Claims (2)
1. A power supply service command system based on a button drawing technology is characterized in that: the system comprises a conversion design module, a Job execution module, a conversion execution module and a Job design module; the conversion design module is used for developing conversion and operation in a graphical mode for the acquired power supply data and transmitting the graphical data to the Job execution module; the Job execution module is used for calling Job by using a command line and storing graphical data; the conversion execution module is used for calling Trans in a command line form and carrying out conversion analysis on the graphical data; and the Job design module is used for starting a plurality of card service processes to extract, distribute and supply power and receive when the system calls the cluster.
2. A working method of the power supply service command system based on the key drawing technology according to claim 1, characterized in that: the method comprises the following steps:
step S1: acquiring power supply data, developing conversion and operation on the acquired power supply data in a graphical mode by the conversion design module, and transmitting the converted graphical data to the Job execution module;
step S2: the Job execution module calls Job storage graphical data by using a command line for analysis by the conversion execution module;
step S3: the conversion execution module calls Trans in a command line mode to analyze data;
step S4: and the Job design module starts a plurality of card service processes to extract, distribute and supply power and receive the data analyzed by the conversion execution module when the system calls the cluster.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010027906.8A CN111190954A (en) | 2020-01-10 | 2020-01-10 | Power supply service command system based on button drawing technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010027906.8A CN111190954A (en) | 2020-01-10 | 2020-01-10 | Power supply service command system based on button drawing technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111190954A true CN111190954A (en) | 2020-05-22 |
Family
ID=70708768
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010027906.8A Pending CN111190954A (en) | 2020-01-10 | 2020-01-10 | Power supply service command system based on button drawing technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111190954A (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1420488A (en) * | 2001-08-07 | 2003-05-28 | 陈涛 | Vedio tape picture and text data generating and coding method and picture and text data playback device |
| US20140320875A1 (en) * | 2013-04-26 | 2014-10-30 | Canon Kabushiki Kaisha | Data processing apparatus, control method, and storage medium |
-
2020
- 2020-01-10 CN CN202010027906.8A patent/CN111190954A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1420488A (en) * | 2001-08-07 | 2003-05-28 | 陈涛 | Vedio tape picture and text data generating and coding method and picture and text data playback device |
| US20140320875A1 (en) * | 2013-04-26 | 2014-10-30 | Canon Kabushiki Kaisha | Data processing apparatus, control method, and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 崔有文 等: "基于KETTLE的数据集成研究", 《计算机技术与发展》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102004663B (en) | A kind of system and method for multi-task concurrent scheduling | |
| US6112196A (en) | Method and system for managing connections to a database management system by reusing connections to a database subsystem | |
| CN102467532A (en) | Task processing method and task processing device | |
| CN102306200B (en) | Device and method for concurrently applying incremental data manipulation statements | |
| CN102929585A (en) | Batch processing method and system supporting multi-master distributed data processing | |
| CN101471810B (en) | Method, device and system for implementing task in cluster circumstance | |
| CN104991952B (en) | A kind of method of intelligent data distribution flow engine and its synchrodata | |
| CN104077362A (en) | Online mass data processing system and method | |
| CN109643310A (en) | System and method for fast resampling in database | |
| CN110619014A (en) | ETL-based data extraction method | |
| CN108334557A (en) | A kind of aggregated data analysis method, device, storage medium and electronic equipment | |
| CN108519918A (en) | Distributed transaction processing method based on transaction chain | |
| CN105653356B (en) | Handle the method and device of multiserver concurrent operations | |
| CN110134533B (en) | System and method capable of scheduling data in batches | |
| CN105955820A (en) | Method, device, and system for job stream control | |
| CN105808619A (en) | Task redoing method based on influence analysis, influence analysis calculation device and one-key reset device | |
| CN107844566B (en) | Dump control method and system | |
| CN111190954A (en) | Power supply service command system based on button drawing technology | |
| CN109710679A (en) | Data pick-up method and device | |
| CN107153679B (en) | Extraction statistical method and system for semi-structured big data | |
| CN107145585A (en) | The automated import of data method and system of Hadoop data warehouses | |
| CN105630580A (en) | Scheduling platform based data summarizing method and data summarizing apparatus | |
| KR20050057834A (en) | Method for mass data processing through scheduler in multi processor system | |
| CN109491768A (en) | A kind of distributed transaction method for dynamically processing and system based on subtransaction stream | |
| CN102147750A (en) | Method and system for processing operation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200522 |
|
| RJ01 | Rejection of invention patent application after publication |