CA3154982C - Interactive machine learning - Google Patents

Interactive machine learning Download PDF

Info

Publication number
CA3154982C
CA3154982C CA3154982A CA3154982A CA3154982C CA 3154982 C CA3154982 C CA 3154982C CA 3154982 A CA3154982 A CA 3154982A CA 3154982 A CA3154982 A CA 3154982A CA 3154982 C CA3154982 C CA 3154982C
Authority
CA
Canada
Prior art keywords
user interface
prediction
machine learning
user
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CA3154982A
Other languages
English (en)
French (fr)
Other versions
CA3154982A1 (en
Inventor
Ishan Amlekar
Chantal BISSON-KROL
Zhen Lin
Seyednaser NOURASHRAFEDDIN
Sebastien Ouellet
Kevin Shen
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kinaxis Inc
Original Assignee
Kinaxis Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US16/697,620 external-priority patent/US12154013B2/en
Application filed by Kinaxis Inc filed Critical Kinaxis Inc
Publication of CA3154982A1 publication Critical patent/CA3154982A1/en
Application granted granted Critical
Publication of CA3154982C publication Critical patent/CA3154982C/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
    • G06F18/2178—Validation; Performance evaluation; Active pattern learning techniques based on feedback of a supervisor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/40—Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
    • G06F18/41—Interactive pattern learning with a human teacher
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/091—Active learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
    • G06N3/105—Shells for specifying net layout
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Medical Informatics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)
  • Machine Translation (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
CA3154982A 2019-10-15 2020-10-15 Interactive machine learning Active CA3154982C (en)

Applications Claiming Priority (7)

Application Number Priority Date Filing Date Title
US201962915076P 2019-10-15 2019-10-15
US62/915,076 2019-10-15
US16/697,620 2019-11-27
US16/697,620 US12154013B2 (en) 2019-10-15 2019-11-27 Interactive machine learning
US16/699,010 US12242954B2 (en) 2019-10-15 2019-11-28 Interactive machine learning
US16/699,010 2019-11-28
PCT/CA2020/051379 WO2021072537A1 (en) 2019-10-15 2020-10-15 Interactive machine learning

Publications (2)

Publication Number Publication Date
CA3154982A1 CA3154982A1 (en) 2022-04-22
CA3154982C true CA3154982C (en) 2024-05-28

Family

ID=75382129

Family Applications (1)

Application Number Title Priority Date Filing Date
CA3154982A Active CA3154982C (en) 2019-10-15 2020-10-15 Interactive machine learning

Country Status (5)

Country Link
US (2) US12242954B2 (enExample)
EP (1) EP4046086A4 (enExample)
JP (2) JP7245961B2 (enExample)
CA (1) CA3154982C (enExample)
WO (1) WO2021072537A1 (enExample)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11727284B2 (en) * 2019-12-12 2023-08-15 Business Objects Software Ltd Interpretation of machine learning results using feature analysis
US11615330B2 (en) * 2020-03-18 2023-03-28 Kyndryl, Inc. Virtual subject matter expert provisioning
US11748239B1 (en) * 2020-05-06 2023-09-05 Allstate Solutions Private Limited Data driven testing automation using machine learning
US11663296B2 (en) * 2020-09-02 2023-05-30 nbn co limited Automatic work order analysis
US12417407B2 (en) * 2021-02-03 2025-09-16 Royal Bank Of Canada System and method for machine learning monitoring
US12387108B2 (en) * 2021-04-21 2025-08-12 Google Llc Enhanced uncertainty management for optical communication systems
US20240104432A1 (en) * 2022-09-23 2024-03-28 X Development Llc Noisy ecological data enhancement via spatiotemporal interpolation and variance mapping

Family Cites Families (83)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11316763A (ja) 1998-05-01 1999-11-16 Mitsubishi Electric Corp 文書分類装置，文書分類方法及び記憶媒体
AUPS017302A0 (en) 2002-01-25 2002-02-14 Tritronics (Australia) Pty Ltd Performance monitoring system and method
US9785951B1 (en) 2006-02-28 2017-10-10 International Business Machines Corporation Scalable tuning engine
US7792353B2 (en) * 2006-10-31 2010-09-07 Hewlett-Packard Development Company, L.P. Retraining a machine-learning classifier using re-labeled training samples
US8108423B2 (en) 2008-10-03 2012-01-31 Disney Enterprises, Inc. System and method for ontology and rules based segmentation engine for networked content delivery
US20100138273A1 (en) 2008-12-01 2010-06-03 Arash Bateni Repeatability index to enhance seasonal product forecasting
JP4745424B2 (ja) 2009-06-05 2011-08-10 株式会社東芝 文書分類装置及び文書分類プログラム
US10102476B2 (en) 2009-06-30 2018-10-16 Agrigenetics, Inc. Application of machine learning methods for mining association rules in plant and animal data sets containing molecular genetic markers, followed by classification or prediction utilizing features created from these association rules
CN102792680B (zh) * 2010-02-19 2016-08-17 汤姆逊许可证公司 媒体内容空间导航
US20120303412A1 (en) 2010-11-24 2012-11-29 Oren Etzioni Price and model prediction system and method
US8386285B2 (en) 2011-05-25 2013-02-26 International Business Machines Corporation Demand modeling and prediction in a retail category
US20130166350A1 (en) 2011-06-28 2013-06-27 Smart Software, Inc. Cluster based processing for forecasting intermittent demand
US10192243B1 (en) 2013-06-10 2019-01-29 Groupon, Inc. Method and apparatus for determining promotion pricing parameters
US20180047071A1 (en) 2012-07-24 2018-02-15 Ebay Inc. System and methods for aggregating past and predicting future product ratings
JP2014067154A (ja) 2012-09-25 2014-04-17 Toshiba Corp 文書分類支援装置、方法及びプログラム
US9111228B2 (en) 2012-10-29 2015-08-18 Sas Institute Inc. System and method for combining segmentation data
US8924910B2 (en) 2012-11-09 2014-12-30 Analog Devices, Inc. Filter design tool
US20140236875A1 (en) 2012-11-15 2014-08-21 Purepredictive, Inc. Machine learning for real-time adaptive website interaction
US20140156346A1 (en) 2012-12-04 2014-06-05 Sap Ag Methods and systems for dynamic demand sensing
US9218574B2 (en) 2013-05-29 2015-12-22 Purepredictive, Inc. User interface for machine learning
US20140358828A1 (en) 2013-05-29 2014-12-04 Purepredictive, Inc. Machine learning generated action plan
US9489373B2 (en) 2013-07-12 2016-11-08 Microsoft Technology Licensing, Llc Interactive segment extraction in computer-human interactive learning
US9959675B2 (en) 2014-06-09 2018-05-01 Microsoft Technology Licensing, Llc Layout design using locally satisfiable proposals
US11100420B2 (en) 2014-06-30 2021-08-24 Amazon Technologies, Inc. Input processing for machine learning
US10452992B2 (en) 2014-06-30 2019-10-22 Amazon Technologies, Inc. Interactive interfaces for machine learning model evaluations
US9390086B2 (en) * 2014-09-11 2016-07-12 Palantir Technologies Inc. Classification system with methodology for efficient verification
US20160092519A1 (en) 2014-09-26 2016-03-31 Wal-Mart Stores, Inc. System and method for capturing seasonality and newness in database searches
US10002329B2 (en) 2014-09-26 2018-06-19 Facebook, Inc. Selection and modification of features used by one or more machine learned models used by an online system
US9760990B2 (en) 2014-12-14 2017-09-12 International Business Machines Corporation Cloud-based infrastructure for feedback-driven training and image recognition
US10453026B2 (en) 2015-03-06 2019-10-22 Walmart Apollo, Llc System and method for forecasting high-sellers using multivariate bayesian time series
US10713594B2 (en) 2015-03-20 2020-07-14 Salesforce.Com, Inc. Systems, methods, and apparatuses for implementing machine learning model training and deployment with a rollback mechanism
US20160358099A1 (en) 2015-06-04 2016-12-08 The Boeing Company Advanced analytical infrastructure for machine learning
US10121108B2 (en) 2015-07-31 2018-11-06 Bluvector, Inc. System and method for in-situ classifier retraining for malware identification and model heterogeneity
JP6555015B2 (ja) 2015-08-31 2019-08-07 富士通株式会社 機械学習管理プログラム、機械学習管理装置および機械学習管理方法
GB201517462D0 (en) 2015-10-02 2015-11-18 Tractable Ltd Semi-automatic labelling of datasets
US20180373208A1 (en) 2015-12-25 2018-12-27 Nec Corporation Cost function design system, cost function design method, and cost function design program
JP7114900B2 (ja) 2016-02-03 2022-08-09 日本電気株式会社 情報処理装置、情報処理方法、及び、プログラム
US10643182B2 (en) 2016-03-16 2020-05-05 Oracle International Corporation Resume extraction based on a resume type
US10672092B2 (en) 2016-04-22 2020-06-02 FiscalNote, Inc. Systems and methods for mapping to milestones in a policymaking process
CN105930934B (zh) 2016-04-27 2018-08-14 第四范式（北京）技术有限公司 展示预测模型的方法、装置及调整预测模型的方法、装置
WO2017190099A1 (en) 2016-04-28 2017-11-02 Atigeo Corp. Using forecasting to control target systems
US10489986B2 (en) 2018-01-25 2019-11-26 Ctrl-Labs Corporation User-controlled tuning of handstate representation model parameters
US11416680B2 (en) 2016-08-18 2022-08-16 Sap Se Classifying social media inputs via parts-of-speech filtering
US11188824B2 (en) 2017-02-17 2021-11-30 Google Llc Cooperatively training and/or using separate input and subsequent content neural networks for information retrieval
US20180247227A1 (en) 2017-02-24 2018-08-30 Xtract Technologies Inc. Machine learning systems and methods for data augmentation
JP6660900B2 (ja) 2017-03-06 2020-03-11 Kddi株式会社 モデル統合装置、モデル統合システム、方法およびプログラム
WO2018170595A1 (en) 2017-03-23 2018-09-27 Rubikloud Technologies Inc. Method and system for generation of adjustable automated forecasts for a promotion
CA3235875A1 (en) 2017-03-23 2018-09-27 Kinaxis Inc. Method and system for generation of at least one output analytic for a promotion
US10395210B2 (en) 2017-04-24 2019-08-27 Walmart Apollo, Llc System and method for using stores as receiving points for third party, E-commerce suppliers
US10255550B1 (en) 2017-06-07 2019-04-09 States Title, Inc. Machine learning using multiple input data types
US11068916B2 (en) 2017-06-26 2021-07-20 Kronos Technology Systems Limited Partnershi Using machine learning to predict retail business volume
US11928616B2 (en) 2017-09-18 2024-03-12 Kinaxis Inc. Method and system for hierarchical forecasting
US10108902B1 (en) 2017-09-18 2018-10-23 CS Disco, Inc. Methods and apparatus for asynchronous and interactive machine learning using attention selection techniques
US20190102693A1 (en) 2017-09-29 2019-04-04 Facebook, Inc. Optimizing parameters for machine learning models
US11748653B2 (en) 2017-10-05 2023-09-05 DataRobot, Inc. Machine learning abstraction
US11922440B2 (en) 2017-10-31 2024-03-05 Oracle International Corporation Demand forecasting using weighted mixed machine learning models
US10535138B2 (en) 2017-11-21 2020-01-14 Zoox, Inc. Sensor data segmentation
US20190156357A1 (en) 2017-11-22 2019-05-23 Staples, Inc. Advanced computational prediction models for heterogeneous data
US10402788B2 (en) 2017-11-29 2019-09-03 Aon Global Operations Ltd (Singapore Branch) Dashboard interface, platform, and environment for intelligent subscription product selection
CN110633745B (zh) 2017-12-12 2022-11-29 腾讯科技（深圳）有限公司 一种基于人工智能的图像分类训练方法、装置及存储介质
US10489126B2 (en) 2018-02-12 2019-11-26 Oracle International Corporation Automated code generation
US11017422B2 (en) 2018-03-01 2021-05-25 Adobe Inc. Dynamically generating digital product notifications over time based on product expiration data
US11599894B2 (en) 2018-06-29 2023-03-07 Tata Consultancy Services Limited Method and system for generating customer decision tree through machine learning
KR102814797B1 (ko) 2018-08-02 2025-05-30 삼성전자주식회사 메타-학습에 기반하여 기계학습의 모델을 선정하는 방법 및 장치
US10896237B2 (en) 2018-08-22 2021-01-19 International Business Machines Corporation Reducing database stress using cognitive data caching
WO2020051213A1 (en) 2018-09-05 2020-03-12 Trax Technology Solutions Pte Ltd. Systems and methods for monitoring products and aggregating product information
US10740656B2 (en) 2018-09-19 2020-08-11 Hughes Network Systems, Llc Machine learning clustering models for determining the condition of a communication system
US11107166B2 (en) 2018-09-25 2021-08-31 Business Objects Software Ltd. Multi-step day sales outstanding forecasting
US10997614B2 (en) 2018-10-09 2021-05-04 Oracle International Corporation Flexible feature regularization for demand model generation
US11295324B2 (en) 2018-10-26 2022-04-05 Target Brands, Inc. Method and system for generating disaggregated demand forecasts from ensemble demand forecasts
US11170391B2 (en) 2018-10-26 2021-11-09 Target Brands, Inc. Method and system for validating ensemble demand forecasts
US20200130425A1 (en) 2018-10-26 2020-04-30 Pilot Inc. Beadlock with filler ring
US20200184494A1 (en) 2018-12-05 2020-06-11 Legion Technologies, Inc. Demand Forecasting Using Automatic Machine-Learning Model Selection
JP2020094928A (ja) 2018-12-13 2020-06-18 本田技研工業株式会社 経路案内装置およびその制御方法、情報処理サーバ、ならびに経路案内システム
US20200210920A1 (en) 2019-01-02 2020-07-02 Legion Technologies, Inc. Machine Learning System for Demand Forecasting With Improved Date Alignment
US11568286B2 (en) 2019-01-31 2023-01-31 Fair Isaac Corporation Providing insights about a dynamic machine learning model
US11537825B2 (en) 2019-10-11 2022-12-27 Kinaxis Inc. Systems and methods for features engineering
US11526899B2 (en) 2019-10-11 2022-12-13 Kinaxis Inc. Systems and methods for dynamic demand sensing
US20220292308A1 (en) 2021-03-12 2022-09-15 DataRobot, Inc. Systems and methods for time series modeling
US20220318711A1 (en) 2021-03-31 2022-10-06 Coupa Software Incorporated Automated supply chain demand forecasting
EP4348941B1 (en) 2021-05-25 2026-07-01 Google LLC TIME SERIES ANOMALY DETECTION BY AUTOMATIC LEARNING
US20230091610A1 (en) 2021-09-13 2023-03-23 DataRobot, Inc. Systems and methods of generating and validating time-series features using machine learning
US20230401468A1 (en) 2022-05-17 2023-12-14 Dell Products L.P. Methods and systems for generating forecasts using an ensemble online demand generation forecaster

Also Published As

Publication number Publication date
JP2022548333A (ja) 2022-11-17
US20210110299A1 (en) 2021-04-15
EP4046086A4 (en) 2023-12-20
WO2021072537A1 (en) 2021-04-22
JP7624465B2 (ja) 2025-01-30
US12242954B2 (en) 2025-03-04
JP7245961B2 (ja) 2023-03-24
EP4046086A1 (en) 2022-08-24
US20240370723A1 (en) 2024-11-07
CA3154982A1 (en) 2022-04-22
JP2023052555A (ja) 2023-04-11

Similar Documents

Publication Publication Date Title
US12154013B2 (en) Interactive machine learning
US20240370723A1 (en) Interactive machine learning
US20240362409A1 (en) Data Insight Generation and Presentation
US20230376981A1 (en) Predictive systems and processes for product attribute research and development
US20220138621A1 (en) System and method for facilitating a machine learning model rebuild
EP4352670A1 (en) Resource allocation optimization for multi-dimensional machine learning environments
US10997500B1 (en) Neural network with re-ranking using engagement metrics
US12386488B2 (en) Apparatus and methods for model selection between a first model and a second model using projector inferencing
WO2024097683A1 (en) Game performance prediction across a device ecosystem
US12536201B2 (en) System and methods for varying optimization solutions using constraints based on an endpoint
US20260030285A1 (en) Document classification using free-form integration of machine learning models
US20250299208A1 (en) System and methods for varying optimization solutions using constraints
US12443967B2 (en) Apparatus and methods for high-order system growth modeling
US12596958B2 (en) Apparatus and methods for multiple stage process modeling
US12182677B1 (en) Apparatus and methods for conditioning raw data based on temporal interpolations to generate optimal extrapolations of an entity
US20260111498A1 (en) Training artificial intelligence (ai) engines for custom object description generation
US12608365B2 (en) System and method for setting and monitoring checkpoints in structured data
US12333318B1 (en) Methods and systems for generating a modified graphical user interface using an optimization protocol
US12619807B1 (en) Apparatus and method for generating a predicted output
Helskyaho et al. Introduction to machine learning
US12217627B1 (en) Apparatus and method for determining action guides
US20260140974A1 (en) Data enrichment using artificial intelligence-based interactive query generation
US20260017523A1 (en) Self-supervised learning for developing temporally agnostic transformers
US20260119207A1 (en) Apparatus and method for dynamic generation of visual elements in a user-interface
US20250285011A1 (en) System and methods for an adaptive machine learning model selection based on data complexity and user goals

Legal Events

Date Code Title Description
EEER Examination request

Effective date: 20220824

MPN Maintenance fee for patent paid

Free format text: FEE DESCRIPTION TEXT: MF (PATENT, 4TH ANNIV.) - STANDARD

Year of fee payment: 4

U00 Fee paid

Free format text: ST27 STATUS EVENT CODE: A-4-4-U10-U00-U101 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE REQUEST RECEIVED

Effective date: 20241010

U11 Full renewal or maintenance fee paid

Free format text: ST27 STATUS EVENT CODE: A-4-4-U10-U11-U102 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE FEE PAYMENT PAID IN FULL

Effective date: 20241010

MPN Maintenance fee for patent paid

Free format text: FEE DESCRIPTION TEXT: MF (PATENT, 5TH ANNIV.) - STANDARD

Year of fee payment: 5

U00 Fee paid

Free format text: ST27 STATUS EVENT CODE: A-4-4-U10-U00-U101 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE REQUEST RECEIVED

Effective date: 20251014

U11 Full renewal or maintenance fee paid

Free format text: ST27 STATUS EVENT CODE: A-4-4-U10-U11-U102 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE FEE PAYMENT PAID IN FULL

Effective date: 20251014