CN110377582B8 - 一种ftp数据库与hdfs数据库自动互传数据的方法 - Google Patents

一种ftp数据库与hdfs数据库自动互传数据的方法 Download PDF

Info

Publication number
CN110377582B8
CN110377582B8 CN201910463038.5A CN201910463038A CN110377582B8 CN 110377582 B8 CN110377582 B8 CN 110377582B8 CN 201910463038 A CN201910463038 A CN 201910463038A CN 110377582 B8 CN110377582 B8 CN 110377582B8
Authority
CN
China
Prior art keywords
data
database
ftp
hdfs
processor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910463038.5A
Other languages
English (en)
Other versions
CN110377582B (zh
CN110377582A (zh
Inventor
徐剑
范联伟
余保华
徐圣吉
李贤军
张金国
展昭
吴磊
何宽宽
胡鸿超
杨培韬
胡刚
陈磊
梅存林
韦良春
夏磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sun Create Electronics Co ltd
Original Assignee
Sun Create Electronics Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sun Create Electronics Co ltd filed Critical Sun Create Electronics Co ltd
Priority to CN201910463038.5A priority Critical patent/CN110377582B8/zh
Publication of CN110377582A publication Critical patent/CN110377582A/zh
Publication of CN110377582B publication Critical patent/CN110377582B/zh
Application granted granted Critical
Publication of CN110377582B8 publication Critical patent/CN110377582B8/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/214—Database migration support
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本发明涉及一种FTP数据库与HDFS数据库自动互传数据的方法，包括如下步骤：数据抽取，GetFTP处理器查询并抽取到FTP数据库中的FTP数据，GetHDFS处理器查询并抽取到HDFS数据库中的HDFS数据；数据转换，对UpdateAttribute处理器进行属性配置，UpdateAttribute处理器将抽取到的FTP数据转换为HDFS数据，将抽取到的HDFS数据转换为FTP数据；数据加载，分别对PutHDFS处理器和PutFTP处理器进行属性配置，PutHDFS处理器将转换得到的HDFS数据加载到HDFS数据库的目标文件夹中，PutFTP处理器将转换得到的FTP数据加载到FTP数据库的目标文件夹中。本发明降低了数据转换的错误率，实现了数据的高效抽取与互传。
CN201910463038.5A 2019-05-30 2019-05-30 一种ftp数据库与hdfs数据库自动互传数据的方法 Active CN110377582B8 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910463038.5A CN110377582B8 (zh) 2019-05-30 2019-05-30 一种ftp数据库与hdfs数据库自动互传数据的方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910463038.5A CN110377582B8 (zh) 2019-05-30 2019-05-30 一种ftp数据库与hdfs数据库自动互传数据的方法

Publications (3)

Publication Number Publication Date
CN110377582A CN110377582A (zh) 2019-10-25
CN110377582B CN110377582B (zh) 2022-02-15
CN110377582B8 true CN110377582B8 (zh) 2022-08-26

Family

ID=68249608

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910463038.5A Active CN110377582B8 (zh) 2019-05-30 2019-05-30 一种ftp数据库与hdfs数据库自动互传数据的方法

Country Status (1)

Country Link
CN (1) CN110377582B8 (zh)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105912677A (zh) * 2016-04-13 2016-08-31 北京思特奇信息技术股份有限公司 一种基于oozie实现ftp与hdfs互传文件的方法和系统
WO2018234741A1 (en) * 2017-06-23 2018-12-27 Qio Technologies Ltd SYSTEMS AND METHODS FOR INTELLIGENCE OF INDUSTRIAL ASSETS BY DISTRIBUTED SYSTEMIC ANTICIPATION
CN109344189A (zh) * 2018-09-19 2019-02-15 浪潮软件集团有限公司 一种基于NiFi的大数据计算方法及装置

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11232187B2 (en) * 2016-01-13 2022-01-25 American Express Travel Related Services Company, Inc. Contextual identification and information security

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105912677A (zh) * 2016-04-13 2016-08-31 北京思特奇信息技术股份有限公司 一种基于oozie实现ftp与hdfs互传文件的方法和系统
WO2018234741A1 (en) * 2017-06-23 2018-12-27 Qio Technologies Ltd SYSTEMS AND METHODS FOR INTELLIGENCE OF INDUSTRIAL ASSETS BY DISTRIBUTED SYSTEMIC ANTICIPATION
CN109344189A (zh) * 2018-09-19 2019-02-15 浪潮软件集团有限公司 一种基于NiFi的大数据计算方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
nifi实例-ftp抽取文件到HDFS中;younge;《CSDN》;20161130;第1页 *

Also Published As

Publication number Publication date
CN110377582B (zh) 2022-02-15
CN110377582A (zh) 2019-10-25

Similar Documents

Publication Publication Date Title
MY195644A (en) Login Information Processing Method and Device
PH12019500872A1 (en) Function-as-a-service (faas) platform in blockchain networks
EP4039706A4 (en) ANTIBODIES TARGETING CLDN18.2, METHOD FOR PREPARATION AND USE
EP3544238A3 (en) Network node availability prediction based on past history data
WO2011104668A3 (en) A video processing system and a method for editing a video asset
WO2016123383A3 (en) Techniques for compact data storage of network traffic and efficient search thereof
GB2564517A (en) System and method for geofencing
WO2011113874A3 (en) System and method for communicating between different entities using different data portions for different channels
WO2012151047A3 (en) Methods and apparatus for gathering intelligence from itemized receipts
EP4255035A3 (en) Method for processing data on basis of network slice
WO2020138828A3 (ko) 사물 인터넷을 이용한 통합 서비스 플랫폼 서버 및 통합 서비스 제공 방법
EP3671575A3 (en) Neural network processing method and apparatus based on nested bit representation
WO2019157199A3 (en) Medical claim database relationship processing
EP4033492A4 (en) DATABASE CONSTRUCTION METHOD AND APPARATUS AND FILE RECOVERY METHOD AND APPARATUS
EP4043646A4 (en) REMOTE ASSISTANCE SERVER, REMOTE ASSISTANCE METHOD AND REMOTE ASSISTANCE SYSTEM
EP3896939A4 (en) RESOURCE DESCRIPTION FILE PROCESSING AND PAGE RESOURCE OBTAINING METHOD, AND INTERMEDIATE SERVER
EP4020294A4 (en) FORMAT VERIFICATION METHOD AND SYSTEM FOR SMART CERTIFICATE STORAGE CONTRACT, COMPUTER EQUIPMENT AND READABLE STORAGE MEDIUM
PH12019000378A1 (en) Systems and methods for converting human interactions to populate an electronic form
EP3971767A4 (en) METHOD FOR CONSTRUCTING A CONVOLUTIVE NEURONAL NETWORK MODEL BASED ON IMAGES OF AGRICULTURAL LAND, AND ASSOCIATED SYSTEM
CN103942280A (zh) 一种基于数据结构自动生成代码的方法
CN108681446B8 (zh) 基于组件服务的众创性气候算法管理系统及方法
EP3073398A3 (en) A method of modelling at least a part of a gas turbine engine
EA202190845A1 (ru) Нейронный блокчейн
CN110377582B8 (zh) 一种ftp数据库与hdfs数据库自动互传数据的方法
CN111213386B8 (zh) 视频传输服务器、视频传输方法及计算机存储介质

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CI03 Correction of invention patent
CI03 Correction of invention patent

Correction item: Address

Correct: No. 3366, Xiyou Road, High-tech Industrial Development Zone, Hefei City, Anhui Province, 230088

False: Building 14, Zone B, Digital Economy Industrial Park, No. 136 Yuhai East Road, Hangzhou Bay New District, Ningbo City, Zhejiang Province, 315336

Number: 07-01

Page: The title page

Volume: 38

Correction item: Address

Correct: No. 3366, Xiyou Road, High-tech Industrial Development Zone, Hefei City, Anhui Province, 230088

False: Building 14, Zone B, Digital Economy Industrial Park, No. 136 Yuhai East Road, Hangzhou Bay New District, Ningbo City, Zhejiang Province, 315336

Number: 07-01

Volume: 38