CN104573063A - Data analysis method based on big data - Google Patents
Data analysis method based on big data Download PDFInfo
- Publication number
- CN104573063A CN104573063A CN201510036086.8A CN201510036086A CN104573063A CN 104573063 A CN104573063 A CN 104573063A CN 201510036086 A CN201510036086 A CN 201510036086A CN 104573063 A CN104573063 A CN 104573063A
- Authority
- CN
- China
- Prior art keywords
- data
- model
- data mining
- code
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
Abstract
The invention provides a data analysis method based on big data. The data analysis method includes steps of receiving user-defined data mining process and component information to realize graphical data flow processing; generating codes conforming to Hadoop standards, converting a user-defined data mining process model into operable codes on the Hadoop; connecting data mining components to form the data mining process; utilizing a frame provided by Hadoop as a data mining process executing platform, submitting the codes to the executing frame and applying parallel computing capability of a cloud platform to parallelize the mining process. By the defined component model, the data mining process can be quickly and visually defined by users, and data access to various data storage systems is realized.
Description
Technical field
The present invention relates to data processing, particularly the large data analysis of one and method for digging.
Background technology
In the face of the data volume of rapid development, from data, how to excavate useful information becomes current most of data digging system problems faced.The data analysis of large data sets and digging system need the characteristic possessed to comprise (1) data adaptability: system can accept polytype data, avoid system to the strong requirement of the type of data stored, structure and data integrity, thus avoid the problem that the strong requirement of common data warehouse to data brings; (2) agility: system can adapt to data and increases progressively and upgrade application scenarios frequently; (3) analysis depth: the analysis to data analysis multi-angle, multiple sections is provided, can be convenient add complicated probability statistics and machine learning algorithm, adapt to diversiform data and analyze demand.
Relevant database provides the data analysis tool carrying out data analysis and excavation usually, but the data digging system based on relevant database exists following drawback:
(1) under huge data volume background, the loss of time that Data Migration brings is huge, and in such cases, it is to the more efficient method of computing system than Data Migration that computing power is shifted to data;
(2) data volume can only be made to narrow down in the acceptable scope of internal memory by the mode of sampling, the sampling of data can cause data message amount to be lost usually;
(3) ever-increasing data are easy to the continuous increase causing database index, and the hysteresis quality that index increases easily causes the processing speed of database to reduce.
Therefore Database Systems cannot meet the explosive growth of current big data quantity in data-handling efficiency and accessible data volume.For the problems referred to above existing in correlation technique, at present effective solution is not yet proposed.
Summary of the invention
For solving the problem existing for above-mentioned prior art, the present invention proposes a kind of data analysing method based on large data, comprising:
User-defined data mining process is received by visual interface, and the module information that configuration is relevant, realize patterned data flow process; Generate the code meeting Hadoop specification, user needs user-defined parameter by parameter configuration interface configuration; Then by the code that user-defined data mining process model conversion becomes to run on Hadoop, this conversion comprises Process model analysis, dependence analysis, Code Template parsing; Data mining assembly is coupled together composition data mining process, the data manipulation logic that described data mining component package is different, is divided into data mining algorithm assembly, connector assembly, User Defined assembly; Perform platform using the framework that Hadoop provides as data mining process, submit code to execution framework, use the computation capability of cloud platform to realize the parallelization of mining process.
Preferably, the method also comprises:
User defines the data mining process meeting process logic model by data mining process model component, and realize the conversion of logical model to physical model, each step data operation in mining process is abstracted into a data running node, be called logic node, build physical optics method by the input/output information of resolution logic node, user's configuration parameter, system component metamessage that node is corresponding, logical model through the backstage of system resolve convert physical model to after could perform;
User realizes the definition of logical model by the patterned way of model, data mining process comprises Data Collection, data prediction, data mining and result and shows, wherein in Data Collection, define one or more Data Source, and complete data extraction work in the process of implementation, by defining different connector assemblies to realize, the data in different pieces of information source are extracted; Comprise data scrubbing, data integration and data selection at data prediction, definition filtration, canonical matching component realize pre-service; Maintenance data mining algorithm performs mining algorithm to through pretreated data, finally execution result is showed user with the form of data or chart.
Preferably, the method also comprises: utilize code building engine to complete from logical model to can by the transfer process of executable code performing framework and perform, this conversion is divided into model analyzing and code building;
Described model analyzing comprises resolution logic model, nodal information according to the definition of data mining process model carrys out division operation sub-process, with data mining exercises node for division points, with sub-process structure task-set, and define the dependence between sub-process according to the order of connection of flow process;
Described code building, the sub-process obtained according to above model analyzing and dependence generating code, data mining assembly receives the input and output type information of coupled assembly, according to input and output type information and component code template, generate corresponding code, and the output after process is stored according to output mode, wherein component model comprises assembly ID, Code Template, user's defined parameters class and component metadata information, and assembly ID is used for the uniqueness of identified component; Code Template includes the Template Information relevant to performing platform; User Defined parameter class is the parameter that user inputs; Component description metamessage comprises component description, visual icon, template path data;
Described data mining process model is converted into Java executable code, finally generate one and be called that the main classes of class name is to control whole data mining process with user-defined data mining process name, and build excavation code by the information that configuration template provides according to the Task Dependent relation that model analyzing obtains.
The present invention compared to existing technology, has the following advantages:
By definitions component model, user can quick definition data mining process; Realize the visual definition to data mining process, and mining process is to the conversion of executable code; Achieve the data access to several data storage system.
Accompanying drawing explanation
Fig. 1 is the process flow diagram of the data analysing method based on large data according to the embodiment of the present invention.
Embodiment
Detailed description to one or more embodiment of the present invention is hereafter provided together with the accompanying drawing of the diagram principle of the invention.Describe the present invention in conjunction with such embodiment, but the invention is not restricted to any embodiment.Scope of the present invention is only defined by the claims, and the present invention contain many substitute, amendment and equivalent.Set forth many details in the following description to provide thorough understanding of the present invention.These details are provided for exemplary purposes, and also can realize the present invention according to claims without some in these details or all details.
The present invention proposes a kind of easily extensible data analysis and method for digging, by defining reusable data mining assembly, realizes component reusing technology to multi-data source, improves data mining process agility.
Fig. 1 is the data analysing method process flow diagram based on large data according to the embodiment of the present invention.
The method is implemented in the data analysis system with following system architecture, comprises data mining process model, code building engine, data mining component model, performs framework 4 modules.User defines data mining process by visual interface, and the module information that configuration is relevant, realize patterned data flow transaction module.Repeat in order to avoid allowing user to write versatility code, code building engine generates and meets Hadoop reputable code, and the parameter configuration interface configuration that user is provided by system needs user-defined parameter, and these parameters finally will be reflected in the final code generated.The code becoming to run at Hadoop from user-defined data mining process model conversion has a process analysis procedure analysis and code generation process, this process is completed by code building engine, mainly includes the work such as Process model analysis, dependence analysis, Code Template parsing.Data mining process is by coupling together formation by assembly, assembly, in order to realize different functions, will encapsulate different data manipulation logics.Data mining assembly is mainly divided into data mining algorithm assembly, connector assembly, User Defined assembly three major types.The framework that system provides using Hadoop performs platform as data mining process, by the form performing framework submission code, uses the computation capability of cloud platform to realize the parallelization of mining process.
Use the development approach based on model, the algorithm used in software process is extracted formation abstract model by the present invention, makes user without the need to being concerned about that special algorithm realizes details, and system excavates flow definition and algorithm assembly definition with the mode reduced data of facing assembly.The method increase the abstraction hierarchy of system, achieve low coupling, the high cohesion of data analysis system in a kind of extendible mode.
User defines data mining process by data mining process model component, and data mining process is a mining process meeting process logic model.In logical model, each step data operation in mining process is abstracted into a data running node, and this node can be described as logic node.System builds physical optics method by the input/output information of resolution logic node, user's configuration parameter, system component metamessage that node is corresponding.Logical process model and specific implementation technology have nothing to do.Physical model is then define from the angle of computer system, and implementation platform, the programming model of this model and system are relevant with MapReduce task scheduling strategy.Logical model could perform in systems in which after the parsing of the backstage of system converts physical model to.
Process model realizes the conversion of logical model to physical model.Meanwhile, user realizes the definition of logical model by the patterned way of model.Physical model is relevant to specific implementation, and physical model contains the related content such as data mining component model, template code generation model.
Data mining process comprises Data Collection, data prediction, data mining and result and shows several step.Data Collection mainly specifies adopted one or more Data Source, and completes data extraction work in the process of implementation, and user extracts the data in different pieces of information source by defining different connector assemblies to realize.Data prediction includes data scrubbing (denoising and removal inconsistent data), data integration (combination of multi-data source) and data selection (definition data filtering rule).User realizes this step by assemblies such as definition filtration, canonical couplings.Data mining maintenance data mining algorithm performs mining algorithm to through pretreated data.Finally, result shows that execution result is showed user with the form of data or chart by step.
Code building engine mainly completes from logical model to the transfer process that can be performed the executable code that framework performs, and roughly can be divided into model analyzing and code building two step.
The first step is model analyzing, groundwork is resolution logic model, nodal information according to the definition of data mining process model carrys out division operation sub-process, with data mining exercises node for division points, with sub-process structure task-set, and define the dependence between sub-process according to the order of connection of flow process.
Second step is code building, the sub-process obtained according to first step model analyzing and dependence generating code.Data mining assembly receives the input and output type information of coupled assembly.According to input and output type information and component code template, generate corresponding code, and the output after process is stored according to output mode.Component model comprises assembly ID, Code Template, user's defined parameters class and component metadata information.Assembly ID is used for the uniqueness of identified component; Code Template includes the Template Information relevant to performing platform; User Defined parameter class is the parameter that user inputs; Component description metamessage contains the metadata such as component description, visual icon, template path.Data mining process model is finally converted into Java executable code.These classes comprise data manipulation node class, data cube computation input and output class.Finally, system generates one and is called that the main classes of class name is to control whole data mining process with user-defined data mining process name, and builds excavation code by the information that configuration template provides according to the Task Dependent relation that model analyzing obtains.
Data mining assembly is the data manipulation unit that system realizes the function such as excavation, data cube computation, the specific data manipulation logic of component package.Expanded function is carried out by User Defined assembly.Data mining assembly defines the component metadata information meeting extension point schema rule in xml.
System to be standardized assembly life cycle in systems in which by definitions component model.Component model defines inlet flow and the output stream of this assembly, and accepts customer parameter by configuration interface.Component model defines the metamessage such as type, input interface, output interface, component type of assembly.By defining legal schema, assembly is present in system as the form of data mining process Plays assembly, and by system, it loaded, call, the life cycle management such as destruction.The definition of above-mentioned schema does not specify the realization of the algorithm logic of component internal, and the realization of algorithm defines mainly through the Code Template in the respective template path of each assembly.Code Template can need the specific function realized carry out implementation algorithm according to assembly, and the framework of this loose coupling is that the extensibility of system provides guarantee.User is by realizing self-defining Code Template to add Custom component.
Data mining algorithm component package mining algorithm logic.In data mining algorithm assembly, data store with SequenceFile type.This storage mode support is compressed, and customizable is the compression granularity compressed based on record or block.The data mining assembly extension point that user defines by frame of reference.The cluster process that data mining algorithm is packaged into data mining algorithm assembly by schema is divided into three steps: the first step realizes the conversion of file, for subsequent step does Data Collection; Second step carries out distributed Canopy algorithm to data, for determining the K Ge Cu center that K mean algorithm is initial; 3rd step carries out K mean iterative process according to user configured parameter.
In sum, the present invention proposes a kind of data analysis based on large data and method for digging, by definitions component model, user can quick definition data mining process; Realize the visual definition to data mining process, and mining process is to the conversion of executable code; Achieve the data access to several data storage system.
Obviously, it should be appreciated by those skilled in the art, above-mentioned of the present invention each module or each step can realize with general computing system, they can concentrate on single computing system, or be distributed on network that multiple computing system forms, alternatively, they can realize with the executable program code of computing system, thus, they can be stored in storage platform and be performed by computing system.Like this, the present invention is not restricted to any specific hardware and software combination.
Should be understood that, above-mentioned embodiment of the present invention only for exemplary illustration or explain principle of the present invention, and is not construed as limiting the invention.Therefore, any amendment made when without departing from the spirit and scope of the present invention, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.In addition, claims of the present invention be intended to contain fall into claims scope and border or this scope and border equivalents in whole change and modification.
Claims (3)
1., based on a data analysing method for large data, for carrying out excavating to large data and analyzing, it is characterized in that, comprising:
User-defined data mining process is received by visual interface, and the module information that configuration is relevant, realize patterned data flow process; Generate the code meeting Hadoop specification, user needs user-defined parameter by parameter configuration interface configuration; Then by the code that user-defined data mining process model conversion becomes to run on Hadoop, this conversion comprises Process model analysis, dependence analysis, Code Template parsing; Data mining assembly is coupled together composition data mining process, the data manipulation logic that described data mining component package is different, is divided into data mining algorithm assembly, connector assembly, User Defined assembly; Perform platform using the framework that Hadoop provides as data mining process, submit code to execution framework, use the computation capability of cloud platform to realize the parallelization of mining process.
2. method according to claim 1, is characterized in that, also comprises:
User defines the data mining process meeting process logic model by data mining process model component, and realize the conversion of logical model to physical model, each step data operation in mining process is abstracted into a data running node, be called logic node, build physical optics method by the input/output information of resolution logic node, user's configuration parameter, system component metamessage that node is corresponding, logical model through the backstage of system resolve convert physical model to after could perform;
User realizes the definition of logical model by the patterned way of model, data mining process comprises Data Collection, data prediction, data mining and result and shows, wherein in Data Collection, define one or more Data Source, and complete data extraction work in the process of implementation, by defining different connector assemblies to realize, the data in different pieces of information source are extracted; Comprise data scrubbing, data integration and data selection at data prediction, definition filtration, canonical matching component realize pre-service; Maintenance data mining algorithm performs mining algorithm to through pretreated data, finally execution result is showed user with the form of data or chart.
3. method according to claim 2, is characterized in that, also comprises: utilize code building engine to complete from logical model to can by the transfer process of executable code performing framework and perform, this conversion is divided into model analyzing and code building;
Described model analyzing comprises resolution logic model, nodal information according to the definition of data mining process model carrys out division operation sub-process, with data mining exercises node for division points, with sub-process structure task-set, and define the dependence between sub-process according to the order of connection of flow process;
Described code building, the sub-process obtained according to above model analyzing and dependence generating code, data mining assembly receives the input and output type information of coupled assembly, according to input and output type information and component code template, generate corresponding code, and the output after process is stored according to output mode, wherein component model comprises assembly ID, Code Template, user's defined parameters class and component metadata information, and assembly ID is used for the uniqueness of identified component; Code Template includes the Template Information relevant to performing platform; User Defined parameter class is the parameter that user inputs; Component description metamessage comprises component description, visual icon, template path data;
Described data mining process model is converted into Java executable code, finally generate one and be called that the main classes of class name is to control whole data mining process with user-defined data mining process name, and build excavation code by the information that configuration template provides according to the Task Dependent relation that model analyzing obtains.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510036086.8A CN104573063A (en) | 2015-01-23 | 2015-01-23 | Data analysis method based on big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510036086.8A CN104573063A (en) | 2015-01-23 | 2015-01-23 | Data analysis method based on big data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104573063A true CN104573063A (en) | 2015-04-29 |
Family
ID=53089125
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510036086.8A Pending CN104573063A (en) | 2015-01-23 | 2015-01-23 | Data analysis method based on big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104573063A (en) |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105243167A (en) * | 2015-11-10 | 2016-01-13 | 中国建设银行股份有限公司 | Data processing method and device |
| CN105389402A (en) * | 2015-12-29 | 2016-03-09 | 曙光信息产业(北京)有限公司 | Big-data-oriented ETL (Extraction-Transformation-Loading) method and device |
| CN105550268A (en) * | 2015-12-10 | 2016-05-04 | 江苏曙光信息技术有限公司 | Big data process modeling analysis engine |
| CN105893522A (en) * | 2016-03-30 | 2016-08-24 | 电子科技大学 | System for developing, generating and managing large-data analysis model business |
| CN106020811A (en) * | 2016-05-13 | 2016-10-12 | 乐视控股（北京）有限公司 | Development method and device of algorithm model |
| CN106326435A (en) * | 2016-06-02 | 2017-01-11 | 北京赛思信安技术股份有限公司 | System and method for visually and automatically analyzing and assembling big data |
| CN106407472A (en) * | 2016-11-01 | 2017-02-15 | 广西电网有限责任公司电力科学研究院 | Visual editing and management system for big data analysis and calculation task of order model |
| CN106484914A (en) * | 2016-10-26 | 2017-03-08 | 国云科技股份有限公司 | A kind of modular assembly method for quickly realizing data mining analysis |
| CN106502720A (en) * | 2016-09-26 | 2017-03-15 | 海尔优家智能科技（北京）有限公司 | A kind of data processing method and device |
| CN106547865A (en) * | 2016-11-01 | 2017-03-29 | 广西电网有限责任公司电力科学研究院 | A kind of convenient Distributed Calculation of big data supports system |
| CN106815019A (en) * | 2016-12-30 | 2017-06-09 | 国家计算机网络与信息安全管理中心 | The WEB interface integrated approach and device of Hadoop distributed algorithms |
| CN107451663A (en) * | 2017-07-06 | 2017-12-08 | 阿里巴巴集团控股有限公司 | Algorithm assembly, based on algorithm assembly modeling method, device and electronic equipment |
| CN107463614A (en) * | 2017-06-30 | 2017-12-12 | 中国科学院寒区旱区环境与工程研究所 | Eco-hydrological Model structure and parameter simulation method based on modeling framework |
| CN107766306A (en) * | 2016-08-17 | 2018-03-06 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus of data display |
| CN108073582A (en) * | 2016-11-08 | 2018-05-25 | 中移(苏州)软件技术有限公司 | A kind of Computational frame selection method and device |
| CN108121773A (en) * | 2017-12-05 | 2018-06-05 | 广东京信软件科技有限公司 | A kind of big data analysis task construction method based on visualization towed |
| CN108182063A (en) * | 2017-12-29 | 2018-06-19 | 福建南威软件有限公司 | A kind of implementation method of big data analysis visual configuration |
| CN109074061A (en) * | 2016-04-12 | 2018-12-21 | 霍尼韦尔国际公司 | Device and method for generating industrial process figure |
| CN110347575A (en) * | 2018-04-04 | 2019-10-18 | 中移（杭州）信息技术有限公司 | A kind of adjustment method based on big data component, device, terminal device and medium |
| CN110727729A (en) * | 2018-06-29 | 2020-01-24 | 贵州白山云科技股份有限公司 | Method and device for realizing intelligent operation |
| WO2020170049A1 (en) * | 2019-02-22 | 2020-08-27 | International Business Machines Corporation | Native code generation for cloud services |
| CN112181511A (en) * | 2020-08-26 | 2021-01-05 | 北京大学 | Executable information analysis flow interaction configuration generation method |
| CN113535831A (en) * | 2021-06-09 | 2021-10-22 | 福建升腾资讯有限公司 | Report form analysis method, device, equipment and medium based on big data |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101364176A (en) * | 2008-08-12 | 2009-02-11 | 北京航空航天大学 | BPEL visual model building system based on GMF |
| CN101398769A (en) * | 2008-10-28 | 2009-04-01 | 北京航空航天大学 | Processor resource integrating and utilizing method transparent to operating system |
| CN102033748A (en) * | 2010-12-03 | 2011-04-27 | 中国科学院软件研究所 | Method for generating data processing flow codes |
-
2015
- 2015-01-23 CN CN201510036086.8A patent/CN104573063A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101364176A (en) * | 2008-08-12 | 2009-02-11 | 北京航空航天大学 | BPEL visual model building system based on GMF |
| CN101398769A (en) * | 2008-10-28 | 2009-04-01 | 北京航空航天大学 | Processor resource integrating and utilizing method transparent to operating system |
| CN102033748A (en) * | 2010-12-03 | 2011-04-27 | 中国科学院软件研究所 | Method for generating data processing flow codes |
Non-Patent Citations (1)
| Title |
|---|
| 黄斌等: "基于MapReduce的数据挖掘平台设计与实现", 《计算机工程与设计》 * |
Cited By (37)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105243167B (en) * | 2015-11-10 | 2019-01-29 | 中国建设银行股份有限公司 | A kind of data processing method and device |
| CN105243167A (en) * | 2015-11-10 | 2016-01-13 | 中国建设银行股份有限公司 | Data processing method and device |
| CN105550268A (en) * | 2015-12-10 | 2016-05-04 | 江苏曙光信息技术有限公司 | Big data process modeling analysis engine |
| CN105389402A (en) * | 2015-12-29 | 2016-03-09 | 曙光信息产业(北京)有限公司 | Big-data-oriented ETL (Extraction-Transformation-Loading) method and device |
| CN105389402B (en) * | 2015-12-29 | 2019-04-12 | 曙光信息产业(北京)有限公司 | A kind of ETL method and apparatus towards big data |
| CN105893522A (en) * | 2016-03-30 | 2016-08-24 | 电子科技大学 | System for developing, generating and managing large-data analysis model business |
| CN105893522B (en) * | 2016-03-30 | 2019-12-31 | 电子科技大学 | Big data analysis model business development generation and management system |
| CN109074061B (en) * | 2016-04-12 | 2023-04-28 | 霍尼韦尔国际公司 | Apparatus and method for generating industrial process graphics |
| CN109074061A (en) * | 2016-04-12 | 2018-12-21 | 霍尼韦尔国际公司 | Device and method for generating industrial process figure |
| CN106020811A (en) * | 2016-05-13 | 2016-10-12 | 乐视控股（北京）有限公司 | Development method and device of algorithm model |
| CN106326435A (en) * | 2016-06-02 | 2017-01-11 | 北京赛思信安技术股份有限公司 | System and method for visually and automatically analyzing and assembling big data |
| CN107766306B (en) * | 2016-08-17 | 2021-05-07 | 阿里巴巴集团控股有限公司 | Data display method and equipment |
| CN107766306A (en) * | 2016-08-17 | 2018-03-06 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus of data display |
| CN106502720A (en) * | 2016-09-26 | 2017-03-15 | 海尔优家智能科技（北京）有限公司 | A kind of data processing method and device |
| CN106502720B (en) * | 2016-09-26 | 2019-11-08 | 海尔优家智能科技（北京）有限公司 | A kind of data processing method and device |
| CN106484914A (en) * | 2016-10-26 | 2017-03-08 | 国云科技股份有限公司 | A kind of modular assembly method for quickly realizing data mining analysis |
| CN106547865A (en) * | 2016-11-01 | 2017-03-29 | 广西电网有限责任公司电力科学研究院 | A kind of convenient Distributed Calculation of big data supports system |
| CN106407472A (en) * | 2016-11-01 | 2017-02-15 | 广西电网有限责任公司电力科学研究院 | Visual editing and management system for big data analysis and calculation task of order model |
| CN106407472B (en) * | 2016-11-01 | 2019-08-20 | 广西电网有限责任公司电力科学研究院 | A kind of the big data calculating analysis task visual edit and management system of order form mode |
| CN108073582B (en) * | 2016-11-08 | 2021-08-06 | 中移(苏州)软件技术有限公司 | Computing framework selection method and device |
| CN108073582A (en) * | 2016-11-08 | 2018-05-25 | 中移(苏州)软件技术有限公司 | A kind of Computational frame selection method and device |
| CN106815019B (en) * | 2016-12-30 | 2020-09-01 | 国家计算机网络与信息安全管理中心 | WEB interface integration method and device of Hadoop distributed algorithm |
| CN106815019A (en) * | 2016-12-30 | 2017-06-09 | 国家计算机网络与信息安全管理中心 | The WEB interface integrated approach and device of Hadoop distributed algorithms |
| CN107463614A (en) * | 2017-06-30 | 2017-12-12 | 中国科学院寒区旱区环境与工程研究所 | Eco-hydrological Model structure and parameter simulation method based on modeling framework |
| CN107451663A (en) * | 2017-07-06 | 2017-12-08 | 阿里巴巴集团控股有限公司 | Algorithm assembly, based on algorithm assembly modeling method, device and electronic equipment |
| CN107451663B (en) * | 2017-07-06 | 2021-04-20 | 创新先进技术有限公司 | Algorithm componentization, modeling method and device based on algorithm components and electronic equipment |
| CN108121773A (en) * | 2017-12-05 | 2018-06-05 | 广东京信软件科技有限公司 | A kind of big data analysis task construction method based on visualization towed |
| CN108182063A (en) * | 2017-12-29 | 2018-06-19 | 福建南威软件有限公司 | A kind of implementation method of big data analysis visual configuration |
| CN110347575A (en) * | 2018-04-04 | 2019-10-18 | 中移（杭州）信息技术有限公司 | A kind of adjustment method based on big data component, device, terminal device and medium |
| CN110727729A (en) * | 2018-06-29 | 2020-01-24 | 贵州白山云科技股份有限公司 | Method and device for realizing intelligent operation |
| US11150926B2 (en) | 2019-02-22 | 2021-10-19 | International Business Machines Corporation | Native code generation for cloud services |
| CN113454594A (en) * | 2019-02-22 | 2021-09-28 | 国际商业机器公司 | Native code generation for cloud services |
| GB2595994A (en) * | 2019-02-22 | 2021-12-15 | Ibm | Native code generation for cloud services |
| GB2595994B (en) * | 2019-02-22 | 2022-09-28 | Ibm | Native code generation for cloud services |
| WO2020170049A1 (en) * | 2019-02-22 | 2020-08-27 | International Business Machines Corporation | Native code generation for cloud services |
| CN112181511A (en) * | 2020-08-26 | 2021-01-05 | 北京大学 | Executable information analysis flow interaction configuration generation method |
| CN113535831A (en) * | 2021-06-09 | 2021-10-22 | 福建升腾资讯有限公司 | Report form analysis method, device, equipment and medium based on big data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104573063A (en) | Data analysis method based on big data | |
| US11487772B2 (en) | Multi-party data joint query method, device, server and storage medium | |
| CN102033748B (en) | Method for generating data processing flow codes | |
| CN112131636B (en) | Device and method for accurately mapping building information model and Internet of things monitoring item | |
| CN105893593B (en) | A kind of method of data fusion | |
| US11314808B2 (en) | Hybrid flows containing a continous flow | |
| CN105824974B (en) | The method and system of Data Analysis Services | |
| Ahmad et al. | A framework for architecture-driven migration of legacy systems to cloud-enabled software | |
| CN104268428A (en) | Visual configuration method for index calculation | |
| CN108573029B (en) | Method, device and storage medium for acquiring network access relation data | |
| CN111506950A (en) | BIM structure transformation increment information generation and storage system and method | |
| CN105912588A (en) | Visualization processing method and system for big data based on memory calculations | |
| Medina et al. | From composable design models to schedulability analysis with uml and the uml profile for marte | |
| CN108073582B (en) | Computing framework selection method and device | |
| CN104835015A (en) | Predecessor task-based workflow mining method | |
| CN114036183A (en) | Data ETL processing method, device, equipment and medium | |
| CN109063040A (en) | Client-side program collecting method and system | |
| CN113535831A (en) | Report form analysis method, device, equipment and medium based on big data | |
| CN104750610A (en) | Dominating degree based aberrance reduction method for message passing parallel program | |
| CN104331517A (en) | Retrieval method and retrieval device | |
| Hill et al. | Tools for continuously evaluating distributed system qualities | |
| CN111429105A (en) | Financial middle engine system and method | |
| CN115185489B (en) | Time sequence data algorithm configuration system and method | |
| CN113609633A (en) | Complex system collaborative simulation and verification evaluation system and method based on model | |
| Rahim et al. | Modular and distributed verification of SysML activity diagrams |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150429 |
|
| RJ01 | Rejection of invention patent application after publication |