HK1173540A1 - 重複數據處理方法、裝置及系統 - Google Patents

重複數據處理方法、裝置及系統

Info

Publication number
HK1173540A1
HK1173540A1 HK13100790.7A HK13100790A HK1173540A1 HK 1173540 A1 HK1173540 A1 HK 1173540A1 HK 13100790 A HK13100790 A HK 13100790A HK 1173540 A1 HK1173540 A1 HK 1173540A1
Authority
HK
Hong Kong
Prior art keywords
repetitive data
processing repetitive
processing
data
repetitive
Prior art date
Application number
HK13100790.7A
Other languages
English (en)
Inventor
何昕
葉瑞海
吳協堯
張文波
Original Assignee
阿里巴巴集團控股有限公司 號郵箱
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 阿里巴巴集團控股有限公司 號郵箱 filed Critical 阿里巴巴集團控股有限公司 號郵箱
Publication of HK1173540A1 publication Critical patent/HK1173540A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Accounting & Taxation (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • Health & Medical Sciences (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Finance (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
HK13100790.7A 2011-06-17 2013-01-18 重複數據處理方法、裝置及系統 HK1173540A1 (zh)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201110164850.1A CN102831127B (zh) 2011-06-17 2011-06-17 重复数据处理方法、装置及系统

Publications (1)

Publication Number Publication Date
HK1173540A1 true HK1173540A1 (zh) 2013-05-16

Family

ID=47334270

Family Applications (1)

Application Number Title Priority Date Filing Date
HK13100790.7A HK1173540A1 (zh) 2011-06-17 2013-01-18 重複數據處理方法、裝置及系統

Country Status (7)

Country Link
US (1) US20130013597A1 (zh)
EP (1) EP2721477A4 (zh)
JP (1) JP6051212B2 (zh)
CN (1) CN102831127B (zh)
HK (1) HK1173540A1 (zh)
TW (1) TWI518530B (zh)
WO (1) WO2012174268A1 (zh)

Families Citing this family (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140023227A1 (en) * 2012-07-17 2014-01-23 Cubic Corporation Broken mag ticket authenticator
CN104239301B (zh) * 2013-06-06 2018-02-13 阿里巴巴集团控股有限公司 一种数据比对方法和装置
CN104077338B (zh) * 2013-06-25 2016-02-17 腾讯科技（深圳）有限公司 一种数据处理的方法及装置
CN104714956A (zh) * 2013-12-13 2015-06-17 国家电网公司 一种异构记录集对比方法及装置
CN104361050A (zh) * 2014-10-29 2015-02-18 中国建设银行股份有限公司 一种数据转化比对方法及装置
CN104391894A (zh) * 2014-11-11 2015-03-04 广州科腾信息技术有限公司 一种重复数据的检查处理方法
CN105677645B (zh) * 2014-11-17 2018-12-21 阿里巴巴集团控股有限公司 一种数据表比对方法和装置
CN105095367B (zh) * 2015-06-26 2018-12-28 北京奇虎科技有限公司 一种客户端数据的采集方法和装置
EP3115906A1 (en) 2015-07-07 2017-01-11 Toedt, Dr. Selk & Coll. GmbH Finding doublets in a database
CN105183835B (zh) * 2015-08-31 2018-09-04 小米科技有限责任公司 社交软件中信息标记的方法及装置
CN105787083A (zh) * 2016-03-02 2016-07-20 深圳市元征科技股份有限公司 数据处理方法及装置
CN105787130B (zh) * 2016-03-30 2019-09-27 北京金山安全软件有限公司 图片清理方法、装置和移动终端
CN106209840A (zh) * 2016-07-12 2016-12-07 中国银联股份有限公司 一种网络包去重方法及装置
CN106250424B (zh) * 2016-07-22 2019-12-03 杭州朗和科技有限公司 一种日志上下文内容的搜索方法、装置及系统
CN107688978B (zh) * 2016-08-05 2021-05-25 北京京东尚科信息技术有限公司 用于检测重复订单信息的方法及装置
CN107784022B (zh) * 2016-08-31 2020-09-15 北京国双科技有限公司 检测法律文书是否重复的方法及装置
CN108073521B (zh) * 2016-11-11 2021-10-08 深圳市创梦天地科技有限公司 一种数据去重的方法和系统
CN108153793A (zh) * 2016-12-02 2018-06-12 航天星图科技（北京）有限公司 一种原始数据处理方法
CN106503268B (zh) * 2016-12-07 2019-08-23 广东神马搜索科技有限公司 数据对比方法、装置和系统
CN108241615A (zh) * 2016-12-23 2018-07-03 中国电信股份有限公司 数据去重方法和装置
CN108280048B (zh) * 2017-01-05 2021-06-15 腾讯科技（深圳）有限公司 信息处理方法和装置
CN107084989B (zh) * 2017-03-27 2020-06-30 广州视源电子科技股份有限公司 一种aoi器件数据库的添加方法与系统
CN107025218B (zh) 2017-04-07 2021-03-02 腾讯科技（深圳）有限公司 一种文本去重方法和装置
CN108460098B (zh) * 2018-02-01 2023-04-07 北京百度网讯科技有限公司 信息推荐方法、装置和计算机设备
CN108921510A (zh) * 2018-06-27 2018-11-30 中国建设银行股份有限公司 银行业务远程自动审核方法及系统
CN109446190B (zh) * 2018-11-07 2022-11-01 湖北省标准化与质量研究院 一种标准元数据的数据处理方法
CN109885555B (zh) * 2019-01-07 2021-12-07 中国联合网络通信集团有限公司 一种用户信息管理方法及装置
CN109918518A (zh) * 2019-01-31 2019-06-21 平安科技（深圳）有限公司 图片查重方法、装置、计算机设备和存储介质
CN110012150B (zh) * 2019-02-20 2021-07-30 维沃移动通信有限公司 一种消息显示方法及终端设备
EP3963456A2 (en) 2019-04-30 2022-03-09 Clumio, Inc. Cloud-based data protection service
CN110555036A (zh) * 2019-08-21 2019-12-10 上海易点时空网络有限公司 基于异步处理的数据排重方法及装置
CN111158643A (zh) * 2019-11-29 2020-05-15 石化盈科信息技术有限责任公司 数据处理系统及方法
CN111651438A (zh) * 2020-04-28 2020-09-11 银江股份有限公司 基于MapDB的结构化数据去重方法、装置、设备及介质
CN111597178A (zh) * 2020-05-18 2020-08-28 山东浪潮通软信息科技有限公司 一种清洗重复数据的方法、系统、设备及介质
CN113259256B (zh) * 2021-07-15 2021-09-21 全时云商务服务股份有限公司 一种重复数据包过滤方法、系统及可读存储介质
CN115064237A (zh) * 2022-06-09 2022-09-16 山东浪潮智慧医疗科技有限公司 一种实现医院体检总结数据标准化的方法
CN117436496A (zh) * 2023-11-22 2024-01-23 深圳市网安信科技有限公司 基于大数据日志的异常检测模型的训练方法及检测方法

Family Cites Families (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5915250A (en) * 1996-03-29 1999-06-22 Virage, Inc. Threshold-based comparison
US6493709B1 (en) * 1998-07-31 2002-12-10 The Regents Of The University Of California Method and apparatus for digitally shredding similar documents within large document sets in a data processing environment
US6772196B1 (en) * 2000-07-27 2004-08-03 Propel Software Corp. Electronic mail filtering system and methods
US7660819B1 (en) * 2000-07-31 2010-02-09 Alion Science And Technology Corporation System for similar document detection
JP2003085946A (ja) * 2001-09-14 2003-03-20 Columbia Music Entertainment Inc データ記録装置およびデータ記録再生装置
JP2003122758A (ja) * 2001-10-11 2003-04-25 Canon Inc 画像検索方法及び装置
JP4065484B2 (ja) * 2001-11-06 2008-03-26 キヤノン株式会社 帳票検索システム
US20030101166A1 (en) * 2001-11-26 2003-05-29 Fujitsu Limited Information analyzing method and system
US20040107205A1 (en) * 2002-12-03 2004-06-03 Lockheed Martin Corporation Boolean rule-based system for clustering similar records
US7702673B2 (en) * 2004-10-01 2010-04-20 Ricoh Co., Ltd. System and methods for creation and use of a mixed media environment
US7873782B2 (en) * 2004-11-05 2011-01-18 Data Robotics, Inc. Filesystem-aware block storage system, apparatus, and method
JP2006134041A (ja) * 2004-11-05 2006-05-25 Canon Inc データ管理装置
WO2006052242A1 (en) * 2004-11-08 2006-05-18 Seirad, Inc. Methods and systems for compressing and comparing genomic data
CA2545232A1 (en) * 2005-07-29 2007-01-29 Cognos Incorporated Method and system for creating a taxonomy from business-oriented metadata content
US20070073592A1 (en) * 2005-09-28 2007-03-29 Redcarpet, Inc. Method and system for network-based comparision shopping
JP2007156845A (ja) * 2005-12-05 2007-06-21 Toshiba Corp データ検索装置、データ検索方法、およびプログラム
JP5105894B2 (ja) * 2006-03-14 2012-12-26 キヤノン株式会社 文書検索システム、文書検索装置及びその方法とプログラム、記憶媒体
US7478113B1 (en) * 2006-04-13 2009-01-13 Symantec Operating Corporation Boundaries
WO2008041173A2 (en) * 2006-10-02 2008-04-10 Nokia Corporation Method system and devices for network sharing or searching of resources
CA2710427C (en) * 2007-12-21 2018-04-24 Thomson Reuters Global Resources Systems, methods, and software for entity relationship resolution
EP2271981B1 (en) * 2008-03-31 2020-06-10 Sony Corporation Binding unit manifest file
US8838549B2 (en) * 2008-07-07 2014-09-16 Chandra Bodapati Detecting duplicate records
US8078646B2 (en) * 2008-08-08 2011-12-13 Oracle International Corporation Representing and manipulating RDF data in a relational database management system
JP5051061B2 (ja) * 2008-08-20 2012-10-17 富士通株式会社 情報検索装置
US8527522B2 (en) * 2008-09-05 2013-09-03 Ramp Holdings, Inc. Confidence links between name entities in disparate documents
JP2010191621A (ja) * 2009-02-17 2010-09-02 Fujitsu Ltd 電子カルテ管理システム、その方法、及びプログラム
CN102378969B (zh) * 2009-03-30 2015-08-05 惠普开发有限公司 拷贝卷中存储的数据的去重复
JP2010257019A (ja) * 2009-04-22 2010-11-11 Fujitsu Ltd 文書管理装置、文書管理方法およびそのプログラム
US8073865B2 (en) * 2009-09-14 2011-12-06 Etsy, Inc. System and method for content extraction from unstructured sources
US8732473B2 (en) * 2010-06-01 2014-05-20 Microsoft Corporation Claim based content reputation service
US20110295722A1 (en) * 2010-06-09 2011-12-01 Reisman Richard R Methods, Apparatus, and Systems for Enabling Feedback-Dependent Transactions

Also Published As

Publication number Publication date
EP2721477A4 (en) 2015-09-16
JP6051212B2 (ja) 2016-12-27
WO2012174268A1 (en) 2012-12-20
TW201301063A (zh) 2013-01-01
EP2721477A1 (en) 2014-04-23
CN102831127A (zh) 2012-12-19
JP2014517426A (ja) 2014-07-17
TWI518530B (zh) 2016-01-21
CN102831127B (zh) 2015-04-22
US20130013597A1 (en) 2013-01-10

Similar Documents

Publication Publication Date Title
HK1173540A1 (zh) 重複數據處理方法、裝置及系統
HK1161386A1 (zh) 種並行的數據處理方法、裝置和並行的數據處理系統
EP2755384A4 (en) RECEIVING DEVICE, RECEIVING METHOD, PROGRAM AND INFORMATION PROCESSING SYSTEM
EP2741497A4 (en) RECEIVING DEVICE, RECEIVING METHOD, PROGRAM, AND INFORMATION PROCESSING SYSTEM
EP2760200A4 (en) RECEIVING DEVICE, RECEIVING METHOD, PROGRAM, AND DATA PROCESSING SYSTEM
EP2940658A4 (en) INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING SYSTEM, AND INFORMATION PROCESSING METHOD
EP2590396A4 (en) INFORMATION PROCESSING SYSTEM, INFORMATION PROCESSING DEVICE, AND INFORMATION PROCESSING METHOD
EP2884477A4 (en) INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING METHOD AND INFORMATION PROCESSING SYSTEM
EP2693394A4 (en) INFORMATION PROCESSING SYSTEM, INFORMATION PROCESSING DEVICE, PICTURE DEVICE AND INFORMATION PROCESSING METHOD
EP2835933A4 (en) METHOD, DEVICE AND SYSTEM FOR IMPLEMENTING THE PROCESSING OF MULTIMEDIA DATA
EP2701363A4 (en) METHOD, DEVICE AND SYSTEM FOR PROCESSING CONTENT
EP2869509A4 (en) METHOD, APPARATUS AND SYSTEM FOR PROCESSING DATA PACKET
EP2704352A4 (en) METHOD, DEVICE AND SYSTEM FOR PROCESSING A ENCRYPTED TEXT
HK1199543A1 (zh) 音頻數據的處理方法、裝置和系統
EP2706432A4 (en) ACTUATING DEVICE, INFORMATION PROCESSING SYSTEM, AND INFORMATION PROCESSING METHOD
EP2787681A4 (en) DATA PROCESSING DEVICE, DATA PROCESSING METHOD, AND PROGRAM
EP2717485A4 (en) METHOD, DEVICE AND SYSTEM FOR SIGNAL PROCESSING
EP2879118A4 (en) INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING METHOD, AND SYSTEM
EP2782336A4 (en) INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING METHOD, INFORMATION PROCESSING DEVICE AND INFORMATION PROCESSING SYSTEM
HK1177353A1 (zh) 種數據處理的方法、裝置及系統
EP2683161A4 (en) TRANSMISSION DEVICE, INFORMATION PROCESSING, PROGRAM AND TRANSMISSION SYSTEM
EP2586203A4 (en) DATA PROCESSING APPARATUS, DATA PROCESSING SYSTEM, AND DATA PROCESSING METHOD
EP2773108A4 (en) RECEIVING DEVICE, RECEIVING METHOD, PROGRAM, AND INFORMATION PROCESSING SYSTEM
EP2693682A4 (en) DATA PROCESSING DEVICE, DATA PROCESSING METHOD AND PROGRAM THEREFOR
HK1181153A1 (zh) 種數據容災處理的方法和裝置

Legal Events

Date Code Title Description
PC Patent ceased (i.e. patent has lapsed due to the failure to pay the renewal fee)

Effective date: 20210622