JP5705338B2 - Ｍａｐｒｅｄｕｃｅ環境で機械学習アルゴリズムを処理するためのシステムおよび方法 - Google Patents

Ｍａｐｒｅｄｕｃｅ環境で機械学習アルゴリズムを処理するためのシステムおよび方法 Download PDF

Info

Publication number
JP5705338B2
JP5705338B2 JP2013555715A JP2013555715A JP5705338B2 JP 5705338 B2 JP5705338 B2 JP 5705338B2 JP 2013555715 A JP2013555715 A JP 2013555715A JP 2013555715 A JP2013555715 A JP 2013555715A JP 5705338 B2 JP5705338 B2 JP 5705338B2
Authority
JP
Japan
Prior art keywords
lop
computer
execution plan
statement
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2013555715A
Other languages
English (en)
Japanese (ja)
Other versions
JP2014510342A5 (https=
JP2014510342A (ja
Inventor
ヴァイシアナサン、シヴァクマール
ティアン、ユアンユアン
ゴティン、アモル
バーディック、ダグラス、アール
ペノール、エドウィン、ピー
レインワルド、ベルトルト
シンドワニ、ヴィカス
タティコンダ、シリシュ
クリシュナムルシー、ラジャセカール
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of JP2014510342A publication Critical patent/JP2014510342A/ja
Publication of JP2014510342A5 publication Critical patent/JP2014510342A5/ja
Application granted granted Critical
Publication of JP5705338B2 publication Critical patent/JP5705338B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/40—Transformation of program code
    • G06F8/41—Compilation
    • G06F8/42—Syntactic analysis
    • G06F8/427—Parsing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Devices For Executing Special Programs (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Complex Calculations (AREA)
JP2013555715A 2011-03-01 2012-02-29 Ｍａｐｒｅｄｕｃｅ環境で機械学習アルゴリズムを処理するためのシステムおよび方法 Active JP5705338B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US13/038,086 2011-03-01
US13/038,086 US8612368B2 (en) 2011-03-01 2011-03-01 Systems and methods for processing machine learning algorithms in a MapReduce environment
PCT/CA2012/050123 WO2012116449A1 (en) 2011-03-01 2012-02-29 Systems and methods for processing machine learning algorithms in a mapreduce environment

Publications (3)

Publication Number Publication Date
JP2014510342A JP2014510342A (ja) 2014-04-24
JP2014510342A5 JP2014510342A5 (https=) 2014-09-04
JP5705338B2 true JP5705338B2 (ja) 2015-04-22

Family

ID=46753913

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2013555715A Active JP5705338B2 (ja) 2011-03-01 2012-02-29 Ｍａｐｒｅｄｕｃｅ環境で機械学習アルゴリズムを処理するためのシステムおよび方法

Country Status (6)

Country Link
US (1) US8612368B2 (https=)
JP (1) JP5705338B2 (https=)
CN (1) CN103403700B (https=)
DE (1) DE112012000628T5 (https=)
GB (1) GB2502020A (https=)
WO (1) WO2012116449A1 (https=)

Families Citing this family (87)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8677366B2 (en) * 2011-05-31 2014-03-18 International Business Machines Corporation Systems and methods for processing hierarchical data in a map-reduce framework
US9361323B2 (en) * 2011-10-04 2016-06-07 International Business Machines Corporation Declarative specification of data integration workflows for execution on parallel processing platforms
US9201690B2 (en) 2011-10-21 2015-12-01 International Business Machines Corporation Resource aware scheduling in a distributed computing environment
US8924977B2 (en) 2012-06-18 2014-12-30 International Business Machines Corporation Sequential cooperation between map and reduce phases to improve data locality
US9471390B2 (en) 2013-01-16 2016-10-18 International Business Machines Corporation Scheduling mapreduce jobs in a cluster of dynamically available servers
US9152469B2 (en) 2013-01-28 2015-10-06 Hewlett-Packard Development Company, L.P. Optimizing execution and resource usage in large scale computing
CN103106183A (zh) * 2013-01-29 2013-05-15 福建天晴数码有限公司 基于mapreduce的大规模稀疏矩阵乘法运算的方法
US9339188B2 (en) 2013-03-04 2016-05-17 James Proud Methods from monitoring health, wellness and fitness with feedback
US9149189B2 (en) 2013-03-04 2015-10-06 Hello, Inc. User or patient monitoring methods using one or more analysis tools
US9392939B2 (en) 2013-03-04 2016-07-19 Hello Inc. Methods using a monitoring device to monitor individual activities, behaviors or habit information and communicate with a database with corresponding individual base information for comparison
US9345404B2 (en) 2013-03-04 2016-05-24 Hello Inc. Mobile device that monitors an individuals activities, behaviors, habits or health parameters
US9424508B2 (en) 2013-03-04 2016-08-23 Hello Inc. Wearable device with magnets having first and second polarities
US9298882B2 (en) 2013-03-04 2016-03-29 Hello Inc. Methods using patient monitoring devices with unique patient IDs and a telemetry system
US9367793B2 (en) 2013-03-04 2016-06-14 Hello Inc. Wearable device with magnets distanced from exterior surfaces of the wearable device
US9320434B2 (en) 2013-03-04 2016-04-26 Hello Inc. Patient monitoring systems and messages that send alerts to patients only when the patient is awake
US9704209B2 (en) 2013-03-04 2017-07-11 Hello Inc. Monitoring system and device with sensors and user profiles based on biometric user information
US9420856B2 (en) 2013-03-04 2016-08-23 Hello Inc. Wearable device with adjacent magnets magnetized in different directions
US20130281801A1 (en) 2013-03-04 2013-10-24 Hello Inc. System using patient monitoring devices with unique patient ID's and a telemetry system
US9526422B2 (en) 2013-03-04 2016-12-27 Hello Inc. System for monitoring individuals with a monitoring device, telemetry system, activity manager and a feedback system
US9532716B2 (en) 2013-03-04 2017-01-03 Hello Inc. Systems using lifestyle database analysis to provide feedback
US9427160B2 (en) 2013-03-04 2016-08-30 Hello Inc. Wearable device with overlapping ends coupled by magnets positioned in the wearable device by an undercut
US9427189B2 (en) 2013-03-04 2016-08-30 Hello Inc. Monitoring system and device with sensors that are responsive to skin pigmentation
US9420857B2 (en) 2013-03-04 2016-08-23 Hello Inc. Wearable device with interior frame
US9445651B2 (en) 2013-03-04 2016-09-20 Hello Inc. Wearable device with overlapping ends coupled by magnets
US9398854B2 (en) 2013-03-04 2016-07-26 Hello Inc. System with a monitoring device that monitors individual activities, behaviors or habit information and communicates with a database with corresponding individual base information for comparison
US9553486B2 (en) 2013-03-04 2017-01-24 Hello Inc. Monitoring system and device with sensors that is remotely powered
US9432091B2 (en) 2013-03-04 2016-08-30 Hello Inc. Telemetry system with wireless power receiver and monitoring devices
US9430938B2 (en) 2013-03-04 2016-08-30 Hello Inc. Monitoring device with selectable wireless communication
US9436903B2 (en) 2013-03-04 2016-09-06 Hello Inc. Wearable device with magnets with a defined distance between adjacent magnets
US9357922B2 (en) 2013-03-04 2016-06-07 Hello Inc. User or patient monitoring systems with one or more analysis tools
US9414651B2 (en) 2013-03-04 2016-08-16 Hello Inc. Wearable device with overlapping ends coupled by magnets operating in a temperature range of 200° F. to 400° F.
US9361572B2 (en) 2013-03-04 2016-06-07 Hello Inc. Wearable device with magnets positioned at opposing ends and overlapped from one side to another
US9848776B2 (en) 2013-03-04 2017-12-26 Hello Inc. Methods using activity manager for monitoring user activity
US9634921B2 (en) 2013-03-04 2017-04-25 Hello Inc. Wearable device coupled by magnets positioned in a frame in an interior of the wearable device with at least one electronic circuit
US9330561B2 (en) 2013-03-04 2016-05-03 Hello Inc. Remote communication systems and methods for communicating with a building gateway control to control building systems and elements
US9345403B2 (en) 2013-03-04 2016-05-24 Hello Inc. Wireless monitoring system with activity manager for monitoring user activity
US9159223B2 (en) 2013-03-04 2015-10-13 Hello, Inc. User monitoring device configured to be in communication with an emergency response system or team
US9737214B2 (en) 2013-03-04 2017-08-22 Hello Inc. Wireless monitoring of patient exercise and lifestyle
US9662015B2 (en) 2013-03-04 2017-05-30 Hello Inc. System or device with wearable devices having one or more sensors with assignment of a wearable device user identifier to a wearable device user
US9582748B2 (en) 2013-03-04 2017-02-28 Hello Inc. Base charging station for monitoring device
US9204798B2 (en) 2013-03-04 2015-12-08 Hello, Inc. System for monitoring health, wellness and fitness with feedback
US9530089B2 (en) 2013-03-04 2016-12-27 Hello Inc. Wearable device with overlapping ends coupled by magnets of a selected width, length and depth
US9406220B2 (en) 2013-03-04 2016-08-02 Hello Inc. Telemetry system with tracking receiver devices
US9354938B2 (en) 2013-04-10 2016-05-31 International Business Machines Corporation Sequential cooperation between map and reduce phases to improve data locality
US9342355B2 (en) 2013-06-20 2016-05-17 International Business Machines Corporation Joint optimization of multiple phases in large data processing
US10009581B2 (en) 2015-01-02 2018-06-26 Fitbit, Inc. Room monitoring device
US9610030B2 (en) 2015-01-23 2017-04-04 Hello Inc. Room monitoring device and sleep analysis methods
US9993197B2 (en) 2013-06-21 2018-06-12 Fitbit, Inc. Patient monitoring systems and messages that send alerts to patients only when the patient is awake
US10058290B1 (en) 2013-06-21 2018-08-28 Fitbit, Inc. Monitoring device with voice interaction
US9993166B1 (en) 2013-06-21 2018-06-12 Fitbit, Inc. Monitoring device using radar and measuring motion with a non-contact device
US10004451B1 (en) 2013-06-21 2018-06-26 Fitbit, Inc. User monitoring system
US9965512B2 (en) 2013-06-25 2018-05-08 Sap Se Operators for constants in aggregated formulas
KR20150033453A (ko) 2013-09-24 2015-04-01 주식회사 엘지씨엔에스 빅데이터 처리 방법, 이를 수행하는 빅데이터 처리 장치 및 이를 저장하는 기록매체
IN2013CH05422A (https=) * 2013-11-26 2015-05-29 Inmobi Pte Ltd
US9697475B1 (en) 2013-12-12 2017-07-04 Google Inc. Additive context model for entity resolution
US9910860B2 (en) 2014-02-06 2018-03-06 International Business Machines Corporation Split elimination in MapReduce systems
US20150278907A1 (en) * 2014-03-27 2015-10-01 Microsoft Corporation User Inactivity Aware Recommendation System
US9684493B2 (en) 2014-06-02 2017-06-20 International Business Machines Corporation R-language integration with a declarative machine learning language
US11094015B2 (en) 2014-07-11 2021-08-17 BMLL Technologies, Ltd. Data access and processing system
CN105302536A (zh) * 2014-07-31 2016-02-03 国际商业机器公司 MapReduce应用的相关参数的配置方法和装置
US10475290B2 (en) 2014-08-06 2019-11-12 Mido Play Inc. System for multiple jurisdiction lotteries with fraud detection
US9640028B2 (en) 2015-07-29 2017-05-02 Mido Play, Inc. Single platform system for multiple jurisdiction lotteries
US11244533B2 (en) 2014-08-06 2022-02-08 Lottery Now, Inc. Systems for multiple legal game providers and multiple jurisdictions with asynchronous meta games
US9734659B2 (en) 2014-08-06 2017-08-15 Mido Play Inc. Single platform system for multiple jurisdiction lotteries and social media
US12154413B2 (en) 2014-08-06 2024-11-26 Lottery Now, Inc. Systems for multiple legal game providers with digital ledger
US9659460B2 (en) 2015-06-03 2017-05-23 Mido Play Inc. Methods for multiple legal game providers and multiple jurisdictions with a single platform
US9836701B2 (en) 2014-08-13 2017-12-05 Microsoft Technology Licensing, Llc Distributed stage-wise parallel machine learning
CN105446896B (zh) * 2014-08-29 2018-05-04 国际商业机器公司 映射化简应用的缓存管理方法和装置
WO2016118036A1 (en) * 2015-01-19 2016-07-28 Huawei Technologies Co., Ltd. Systems and methods for selection of program implementation
US10540608B1 (en) 2015-05-22 2020-01-21 Amazon Technologies, Inc. Dynamically scaled training fleets for machine learning
US10496528B2 (en) 2015-08-31 2019-12-03 Microsoft Technology Licensing, Llc User directed partial graph execution
US10402469B2 (en) 2015-10-16 2019-09-03 Google Llc Systems and methods of distributed optimization
US10268461B2 (en) * 2015-11-23 2019-04-23 International Business Machines Corporation Global data flow optimization for machine learning programs
US10860947B2 (en) 2015-12-17 2020-12-08 Microsoft Technology Licensing, Llc Variations in experiment graphs for machine learning
US9715373B2 (en) * 2015-12-18 2017-07-25 International Business Machines Corporation Dynamic recompilation techniques for machine learning programs
US9916344B2 (en) 2016-01-04 2018-03-13 International Business Machines Corporation Computation of composite functions in a map-reduce framework
US20180089587A1 (en) 2016-09-26 2018-03-29 Google Inc. Systems and Methods for Communication Efficient Distributed Mean Estimation
US11196800B2 (en) 2016-09-26 2021-12-07 Google Llc Systems and methods for communication efficient distributed mean estimation
US10769549B2 (en) 2016-11-21 2020-09-08 Google Llc Management and evaluation of machine-learned models based on locally logged data
US10198291B2 (en) 2017-03-07 2019-02-05 International Business Machines Corporation Runtime piggybacking of concurrent jobs in task-parallel machine learning programs
WO2019032123A1 (en) 2017-08-11 2019-02-14 Visa International Service Association SYSTEMS AND METHODS FOR GENERATING DISTRIBUTED SOFTWARE USING AN UNREGRIBUTED SOURCE CODE
CN108960433B (zh) * 2018-06-26 2022-04-05 第四范式（北京）技术有限公司 用于运行机器学习建模过程的方法及系统
CN109657247B (zh) * 2018-12-19 2023-05-23 中科曙光国际信息产业有限公司 机器学习的自定义语法实现方法及装置
US11200238B2 (en) * 2019-01-28 2021-12-14 Roblox Corporation Computing cross products using map reduce
US11436533B2 (en) * 2020-04-10 2022-09-06 Capital One Services, Llc Techniques for parallel model training
US12236370B2 (en) 2020-08-24 2025-02-25 Samsung Electronics Co., Ltd Method and apparatus for federated learning
CA3214385C (en) 2021-06-11 2025-01-21 Iterate Studio, Inc. DATA PIPELINE AND ACCESS TO MULTIPLE MACHINE-LEARNED MODELS

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7231399B1 (en) * 2003-11-14 2007-06-12 Google Inc. Ranking documents based on large data sets
US7650331B1 (en) * 2004-06-18 2010-01-19 Google Inc. System and method for efficient large-scale data processing
US7844959B2 (en) 2006-09-29 2010-11-30 Microsoft Corporation Runtime optimization of distributed execution graph
US8190610B2 (en) * 2006-10-05 2012-05-29 Yahoo! Inc. MapReduce for distributed database processing
US7886046B1 (en) * 2008-05-16 2011-02-08 Google Inc. Methods and apparatus for predicting impact of proposed changes and implementations in distributed networks
US9110706B2 (en) * 2009-02-09 2015-08-18 Microsoft Technology Licensing, Llc General purpose distributed data parallel computing using a high level language
US8239847B2 (en) * 2009-03-18 2012-08-07 Microsoft Corporation General distributed reduction for data parallel computing
US20100241893A1 (en) * 2009-03-18 2010-09-23 Eric Friedman Interpretation and execution of a customizable database request using an extensible computer process and an available computing environment
US9268815B2 (en) * 2009-08-20 2016-02-23 Hewlett Packard Enterprise Development Lp Map-reduce and parallel processing in databases
JPWO2011090032A1 (ja) * 2010-01-20 2013-05-23 株式会社日立製作所 並列処理プログラム生成方法、並列処理プログラム生成プログラム、及び並列処理プログラム生成装置
US8356086B2 (en) * 2010-03-31 2013-01-15 Microsoft Corporation Distributed non-negative matrix factorization
JP5584914B2 (ja) * 2010-07-15 2014-09-10 株式会社日立製作所 分散計算システム
US9600250B2 (en) * 2010-10-08 2017-03-21 Microsoft Technology Licensing, Llc Declarative programming model with a native programming language

Also Published As

Publication number Publication date
WO2012116449A1 (en) 2012-09-07
US20120226639A1 (en) 2012-09-06
US8612368B2 (en) 2013-12-17
GB2502020A (en) 2013-11-13
CN103403700B (zh) 2016-02-03
GB201314958D0 (en) 2013-10-02
JP2014510342A (ja) 2014-04-24
CN103403700A (zh) 2013-11-20
DE112012000628T5 (de) 2013-11-14

Similar Documents

Publication Publication Date Title
JP5705338B2 (ja) Ｍａｐｒｅｄｕｃｅ環境で機械学習アルゴリズムを処理するためのシステムおよび方法
US9122523B2 (en) Automatic pipelining framework for heterogeneous parallel computing systems
US7598953B2 (en) Interpreter for simplified programming of graphics processor units in general purpose programming languages
US8239847B2 (en) General distributed reduction for data parallel computing
WO2019095873A1 (zh) 任务并行处理方法、装置、系统、存储介质及计算机设备
Boehm et al. SystemML's Optimizer: Plan Generation for Large-Scale Machine Learning Programs.
US10970130B2 (en) Composable and cancelable dataflow continuation passing
JP6598981B2 (ja) データ処理グラフのコンパイル
WO2021000971A1 (zh) 操作数据的生成方法、装置及相关产品
US12106097B2 (en) Using hardware-accelerated instructions
US20240028943A1 (en) Compilation of hybrid quantum computing programs
CN114416045A (zh) 自动生成算子的方法和装置
US20220229641A1 (en) Static versioning in the polyhedral model
Liu et al. From Datalog rules to efficient programs with time and space guarantees
Katel et al. High performance gpu code generation for matrix-matrix multiplication using mlir: some early results
CN120874913A (zh) 深度学习模型推理方法、装置、计算机设备和存储介质
Pereira et al. Compiling and optimizing OpenMP 4. X programs to OpenCL and SPIR
US12443399B1 (en) Method and system for code optimization based on statistical data
Smith Theory and practice of classical matrix-matrix multiplication for hierarchical memory architectures
Doroshenko et al. Automated design of parallel programs for heterogeneous platforms using algebra-algorithmic tools
US12547388B1 (en) Method and system for compiler generated external strategies
Mogers Guided rewriting and constraint satisfaction for parallel GPU code generation
US11922152B2 (en) Workload oriented constant propagation for compiler
Wasti et al. LoopStack: ML-friendly ML compiler stack
Khandelwal High Performance GPU Code Generation for Matrix-Matrix Multiplication using MLIR: Some Early Results

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20140718

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20140718

A871 Explanation of circumstances concerning accelerated examination

Free format text: JAPANESE INTERMEDIATE CODE: A871

Effective date: 20140718

A975 Report on accelerated examination

Free format text: JAPANESE INTERMEDIATE CODE: A971005

Effective date: 20140801

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20141014

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20150107

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20150203

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20150224

R150 Certificate of patent or registration of utility model

Ref document number: 5705338

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

S111 Request for change of ownership or part of ownership

Free format text: JAPANESE INTERMEDIATE CODE: R313113

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250