JP2019164761A - ラベル付きトレーニングデータの品質向上 - Google Patents
ラベル付きトレーニングデータの品質向上 Download PDFInfo
- Publication number
- JP2019164761A JP2019164761A JP2019013304A JP2019013304A JP2019164761A JP 2019164761 A JP2019164761 A JP 2019164761A JP 2019013304 A JP2019013304 A JP 2019013304A JP 2019013304 A JP2019013304 A JP 2019013304A JP 2019164761 A JP2019164761 A JP 2019164761A
- Authority
- JP
- Japan
- Prior art keywords
- classification
- item
- taxonomy
- hierarchical
- class labels
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012549 training Methods 0.000 title claims abstract description 29
- 238000000034 method Methods 0.000 claims abstract description 49
- 238000002372 labelling Methods 0.000 claims abstract description 44
- 238000011156 evaluation Methods 0.000 claims abstract description 28
- 238000010801 machine learning Methods 0.000 claims description 35
- 238000012795 verification Methods 0.000 claims description 26
- 238000004891 communication Methods 0.000 claims description 19
- 238000012790 confirmation Methods 0.000 claims description 4
- 238000013500 data storage Methods 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 abstract description 13
- 230000002776 aggregation Effects 0.000 abstract description 9
- 238000004220 aggregation Methods 0.000 abstract description 9
- 238000010200 validation analysis Methods 0.000 abstract 3
- 238000012545 processing Methods 0.000 description 15
- 238000005070 sampling Methods 0.000 description 11
- 230000004044 response Effects 0.000 description 6
- 238000013459 approach Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000004931 aggregating effect Effects 0.000 description 4
- 238000012550 audit Methods 0.000 description 4
- 230000002085 persistent effect Effects 0.000 description 3
- 238000012797 qualification Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000003908 quality control method Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 125000002066 L-histidyl group Chemical group [H]N1C([H])=NC(C([H])([H])[C@](C(=O)[*])([H])N([H])[H])=C1[H] 0.000 description 1
- 241000699670 Mus sp. Species 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000007717 exclusion Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012358 sourcing Methods 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0283—Price estimation or determination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/24323—Tree-organised classifiers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06311—Scheduling, planning or task assignment for a person or group
- G06Q10/063112—Skill-based matching of a person or a group to a task
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0633—Workflow analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06398—Performance of employee with respect to a job function
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0282—Rating or review of business operators or products
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Development Economics (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Entrepreneurship & Innovation (AREA)
- Economics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Finance (AREA)
- Accounting & Taxation (AREA)
- General Engineering & Computer Science (AREA)
- Marketing (AREA)
- Game Theory and Decision Science (AREA)
- General Business, Economics & Management (AREA)
- Educational Administration (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Medical Informatics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Computer Security & Cryptography (AREA)
Abstract
Description
(1)ドメインエキスパート34及び1つ以上のクラウドソーシングプラットフォーム36に対し、アダプタ38を通してタスクを割り当て、ワーカ資質評価40及び回答集約42をも行うマルチレベルワーカプラットフォーム32
(2)ジョブ依頼者が種々のサンプリング方式の中から選択できるようにする、統合ユーザインタフェースを備えるサンプリング方式インタフェース44
(3)ジョブ依頼者が種々のタイプのジョブ(例えば、フィルタジョブ48、ラベル付替えジョブ50及び監査ジョブ52)をローンチできるようにするジョブ処理インタフェース46
(4)クラウドソーシングからの集約結果及び機械学習モデル56の向上を表示するデータレポータダッシュボード54
Claims (20)
- アイテムにラベル付けを行う、コンピュータによる方法であって、
アイテムの説明を含むアイテム記録を受け付けるステップと、
前記アイテムについて、1つ以上の機械学習ベースの分類器に基づき、各クラスラベルに関連付けられるノードの連続したレベルを有する階層的分類タクソノミにおける分類を推論するステップであって、分類パスは前記階層的分類タクソノミにおけるクラスラベルのうちの1つ以上を含む、ステップと、
通信ネットワークを通して、クラウドソーシングシステムに参加している複数のワーカにラベル付けタスクを発行するステップであって、前記ラベル付けタスクは、前記アイテムの説明と、前記分類における1つ以上のクラスラベルとに少なくとも部分的に基づいて、前記分類を評価することを含む、ステップと、
前記クラウドソーシングシステムから評価決定を受け付けるステップと、
前記分類を検証して検証結果を得るステップであって、前記検証は、受け付けられた前記評価決定の集合に対する少なくとも1つの合意基準の適用を含む、ステップと、
前記検証結果に基づき、通信ネットワークを通して、前記分類におけるクラスラベルのうちの1つ以上に対応するデータを各宛先へ送るステップと
を含む方法。 - 前記推論は前記アイテム記録に基づく、請求項1に記載の方法。
- 前記分類は、前記階層的分類タクソノミの連続したレベルにおける各クラスラベルの順序付きシーケンスに対応する分類パスを含む、請求項1に記載の方法。
- 前記ラベル付けタスクは、前記アイテムの説明と、前記分類パスにおけるクラスラベルの順序付きシーケンスとに少なくとも部分的に基づいて、前記分類パスを確認することを含む、請求項3に記載の方法。
- 前記分類パスの確認は更に、前記アイテムの説明を含むオンライン検索クエリの結果に基づく、請求項4に記載の方法。
- 前記アイテム記録は前記アイテムに関連付けられる業者を含み、前記分類パスの確認は更に前記業者に基づく、請求項4に記載の方法。
- 前記アイテム記録は前記アイテムに関連付けられる価格を含み、前記分類パスの確認は更に前記価格に基づく、請求項4に記載の方法。
- 前記検証は、
第1の合意基準が満たされないときに、前記クラウドソーシングシステムに参加している少なくとも1人の別のワーカに前記ラベル付けタスクを発行することと、
少なくとも1人の前記別のワーカから各評価決定を受け付けることと
を含み、
前記適用は、受け付けられた前記評価決定の集合に対して第2の合意基準を適用することを含む、請求項1に記載の方法。 - 前記分類パスが妥当である場合に、前記分類における前記クラスラベルのうちの1つ以上のクラスラベルが、1つ以上の前記機械学習ベースの分類器のためのトレーニングデータとして指定される、請求項1に記載の方法。
- 前記分類が妥当ではない場合に、前記送るステップが、通信ネットワークを通して、ラベル付替えを行う少なくとも1人のドメインエキスパートに対して前記ラベル付けタスクを発行することを含む、請求項1に記載の方法。
- 少なくとも1人の前記ドメインエキスパートから、前記分類における1つ以上のクラスラベルのうち、ラベル付替えがなされたクラスラベルを受け付けるステップと、
前記分類において前記ラベル付替えがなされたクラスラベルを、1つ以上の前記機械学習ベースの分類器のためのトレーニングデータとして指定するステップと
を更に含む請求項10に記載の方法。 - 重複するタスクを前記発行の前にフィルタリングにより除外するステップを更に含む請求項1に記載の方法。
- 推論される前記分類は、前記階層的分類タクソノミ内の1つのレベルから、前記階層的分類タクソノミ内の連続したレベルを経て、前記階層的分類タクソノミ内の別のレベルへと延びるものである、請求項1に記載の方法。
- 前記階層的分類タクソノミ内の別のレベルは、前記階層的分類タクソノミ内の葉ノードレベルに対応するものである、請求項13に記載の方法。
- 推論される前記分類は、前記階層的分類タクソノミ内の連続したレベルを経て延び、葉ノードレベルの前に終了する、請求項1に記載の方法。
- 前記アイテム記録が製品の説明を含む、請求項1に記載の方法。
- プロセッサにより実行される実行可能命令を記憶するメモリ部を備える、コンピュータにより読出し可能なデータ記憶装置であって、
前記メモリ部は、
アイテムについて、1つ以上の機械学習ベースの分類器に基づき、各クラスラベルに関連付けられるノードの連続したレベルを有する階層的分類タクソノミにおける分類を推論するための実行可能命令であって、分類パスは前記階層的分類タクソノミにおけるクラスラベルのうちの1つ以上を含む、実行可能命令と、
通信ネットワークを通して、クラウドソーシングシステムに参加している複数のワーカにラベル付けタスクを発行するための実行可能命令であって、前記ラベル付けタスクは、前記アイテムの説明と、前記分類における1つ以上のクラスラベルとに少なくとも部分的に基づいて、前記分類を評価することを含む、実行可能命令と、
前記クラウドソーシングシステムからラベル付けタスクに関する評価決定を受け付けるための実行可能命令と、
前記分類を検証して検証結果を得るための実行可能命令であって、受け付けられた前記評価決定の集合に対して少なくとも1つの合意基準を適用するための実行可能命令を含む実行可能命令と、
前記検証結果に基づき、通信ネットワークを通して、前記分類におけるクラスラベルのうちの1つ以上に対応するデータを各宛先に送るための実行可能命令と
を有する、コンピュータにより読出し可能なデータ記憶装置。 - 前記分類は、前記階層的分類タクソノミの連続したレベルにおける各クラスラベルの順序付きシーケンスに対応する分類パスを含む、請求項17に記載のコンピュータにより読出し可能なデータ記憶装置。
- 通信ネットワークを通して、クラウドソーシングシステムに参加している複数のワーカにラベル付けタスクを発行するステップであって、前記ラベル付けタスクは、階層的分類タクソノミの連続したレベルにおける各クラスラベルの順序付きシーケンスを含む、推論された分類を、アイテムの説明と、分類パス内のクラスラベルとに少なくとも部分的に基づいて評価することを含む、ステップと、
前記クラウドソーシングシステムから各評価決定を受け付けるステップと
を行う通信インタフェースと、
前記分類を検証して検証結果を得るステップであって、前記検証は、受け付けられた前記評価決定の集合に対して少なくとも1つの合意基準を適用することを含む、ステップと、
前記検証結果に基づき、通信ネットワークを通して、前記分類におけるクラスラベルのうちの1つ以上に対応するデータを各宛先へ送るステップと
を行うプロセッサと
を備えるシステム。 - 前記分類パスが妥当ではない場合に、前記プロセッサは、通信ネットワークを通して、ラベル付替えを行う少なくとも1人のドメインエキスパートに前記ラベル付けタスクを送る、請求項19に記載のシステム。
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201862623448P | 2018-01-29 | 2018-01-29 | |
US62/623,448 | 2018-01-29 | ||
US16/243,342 | 2019-01-09 | ||
US16/243,342 US11803883B2 (en) | 2018-01-29 | 2019-01-09 | Quality assurance for labeled training data |
Publications (2)
Publication Number | Publication Date |
---|---|
JP2019164761A true JP2019164761A (ja) | 2019-09-26 |
JP7458148B2 JP7458148B2 (ja) | 2024-03-29 |
Family
ID=67392246
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
JP2019013304A Active JP7458148B2 (ja) | 2018-01-29 | 2019-01-29 | ラベル付きトレーニングデータの品質向上 |
Country Status (2)
Country | Link |
---|---|
US (1) | US11803883B2 (ja) |
JP (1) | JP7458148B2 (ja) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111288999A (zh) * | 2020-02-19 | 2020-06-16 | 深圳大学 | 一种基于移动终端的行人路网属性检测方法、装置和设备 |
KR102155748B1 (ko) * | 2020-01-03 | 2020-09-17 | 주식회사 크라우드웍스 | 인공지능 학습데이터 생성을 위한 크라우드소싱 기반 프로젝트의 작업 시간 대비 검수 시간을 이용한 검수 단가 자동 업데이트 방법 |
JP2021086613A (ja) * | 2019-11-27 | 2021-06-03 | クラウドワークス インコーポレイテッドCrowdWorks, Inc. | 本番作業に基づいた検証用クラウドソーシング作業を提供する方法及び装置 |
Families Citing this family (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11803883B2 (en) * | 2018-01-29 | 2023-10-31 | Nielsen Consumer Llc | Quality assurance for labeled training data |
US11868436B1 (en) | 2018-06-14 | 2024-01-09 | Amazon Technologies, Inc. | Artificial intelligence system for efficient interactive training of machine learning models |
US11875230B1 (en) | 2018-06-14 | 2024-01-16 | Amazon Technologies, Inc. | Artificial intelligence system with intuitive interactive interfaces for guided labeling of training data for machine learning models |
US11625641B2 (en) * | 2018-12-28 | 2023-04-11 | Esurance Insurance Services, Inc. | Machine learning based function testing |
US11270077B2 (en) * | 2019-05-13 | 2022-03-08 | International Business Machines Corporation | Routing text classifications within a cross-domain conversational service |
US20200387919A1 (en) * | 2019-06-10 | 2020-12-10 | International Business Machines Corporation | Product evaluation based on dynamic metrics |
CN112862105B (zh) * | 2019-11-28 | 2023-08-08 | 杭州海康威视数字技术股份有限公司 | 一种数据发布系统、方法及装置 |
CN111275079B (zh) * | 2020-01-13 | 2023-07-04 | 浙江大学 | 基于图神经网络的众包标签推测方法及系统 |
US11669753B1 (en) * | 2020-01-14 | 2023-06-06 | Amazon Technologies, Inc. | Artificial intelligence system providing interactive model interpretation and enhancement tools |
CN111444937B (zh) * | 2020-01-15 | 2023-05-12 | 湖州师范学院 | 一种基于集成tsk模糊分类器的众包质量提升的方法 |
US11537886B2 (en) | 2020-01-31 | 2022-12-27 | Servicenow Canada Inc. | Method and server for optimizing hyperparameter tuples for training production-grade artificial intelligence (AI) |
US11727285B2 (en) | 2020-01-31 | 2023-08-15 | Servicenow Canada Inc. | Method and server for managing a dataset in the context of artificial intelligence |
CN112101419A (zh) * | 2020-08-17 | 2020-12-18 | 清华大学 | 基于众包的高效分类系统及其创建、使用方法 |
CN112488222B (zh) * | 2020-12-05 | 2022-07-01 | 武汉中海庭数据技术有限公司 | 一种众包数据标注方法、系统、服务器及存储介质 |
CN113255339B (zh) * | 2021-05-06 | 2023-09-19 | 网易(杭州)网络有限公司 | 标注任务的处理方法、装置、电子设备及存储介质 |
US20220383225A1 (en) * | 2021-05-28 | 2022-12-01 | Adp, Inc. | Organizational Benchmarks |
US11783001B2 (en) * | 2021-07-08 | 2023-10-10 | Bank Of America Corporation | System and method for splitting a video stream using breakpoints based on recognizing workflow patterns |
US11531675B1 (en) * | 2021-07-19 | 2022-12-20 | Oracle International Corporation | Techniques for linking data to provide improved searching capabilities |
CN114493810B (zh) * | 2022-04-14 | 2022-07-05 | 成都信息工程大学 | 物联网数据处理方法、装置及介质 |
US20230350895A1 (en) * | 2022-04-29 | 2023-11-02 | Volvo Car Corporation | Computer-Implemented Method for Performing a System Assessment |
CN114880473A (zh) * | 2022-04-29 | 2022-08-09 | 支付宝(杭州)信息技术有限公司 | 一种标签分类方法、装置、存储介质及电子设备 |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2014509002A (ja) * | 2011-02-02 | 2014-04-10 | イーベイ インク. | 特定地域在庫検索用のメタデータ使用 |
US20140314311A1 (en) * | 2013-04-23 | 2014-10-23 | Wal-Mart Stores, Inc. | System and method for classification with effective use of manual data input |
JP2017505964A (ja) * | 2013-11-27 | 2017-02-23 | 株式会社Nttドコモ | 機械学習に基づく自動タスク分類 |
Family Cites Families (191)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7082426B2 (en) | 1993-06-18 | 2006-07-25 | Cnet Networks, Inc. | Content aggregation method and apparatus for an on-line product catalog |
US5664109A (en) | 1995-06-07 | 1997-09-02 | E-Systems, Inc. | Method for extracting pre-defined data items from medical service records generated by health care providers |
US5897622A (en) | 1996-10-16 | 1999-04-27 | Microsoft Corporation | Electronic shopping and merchandising system |
US7904333B1 (en) | 1996-10-25 | 2011-03-08 | Ipf, Inc. | Web-based electronic commerce (EC) enabled shopping network configured to allow members of a consumer product management team and authorized parties to communicate directly with consumers shopping at EC-enabled websites along the world wide web (WWW), using multi-mode virtual kiosks (MMVKS) driven by server-side components and managed by product team members |
US5864848A (en) | 1997-01-31 | 1999-01-26 | Microsoft Corporation | Goal-driven information interpretation and extraction system |
US6604086B1 (en) | 1998-07-20 | 2003-08-05 | Usa Technologies, Inc. | Electronic commerce terminal connected to a vending machine operable as a telephone |
US7197534B2 (en) | 1998-09-01 | 2007-03-27 | Big Fix, Inc. | Method and apparatus for inspecting the properties of a computer |
US6453347B1 (en) | 1999-10-29 | 2002-09-17 | Mcafee.Com, Inc. | Active marketing based on client computer configurations |
US6981040B1 (en) * | 1999-12-28 | 2005-12-27 | Utopy, Inc. | Automatic, personalized online information and product services |
US20040220926A1 (en) | 2000-01-03 | 2004-11-04 | Interactual Technologies, Inc., A California Cpr[P | Personalization services for entities from multiple sources |
WO2001063486A2 (en) | 2000-02-24 | 2001-08-30 | Findbase, L.L.C. | Method and system for extracting, analyzing, storing, comparing and reporting on data stored in web and/or other network repositories and apparatus to detect, prevent and obfuscate information removal from information servers |
EP1139264A3 (en) | 2000-03-29 | 2002-08-21 | Ford Motor Company | Order status inquiry and tracking |
US20050177785A1 (en) | 2000-05-25 | 2005-08-11 | Shrader Theodore J.L. | Client-side pricing agent for collecting and managing product price information over the internet |
JP2002014681A (ja) | 2000-06-28 | 2002-01-18 | Pioneer Electronic Corp | 情報提供システム |
US20020046248A1 (en) | 2000-10-13 | 2002-04-18 | Honeywell International Inc. | Email to database import utility |
EP1199652A1 (en) | 2000-10-16 | 2002-04-24 | Mail Morph Limited | Email processing |
US6772130B1 (en) | 2000-11-22 | 2004-08-03 | Pitney Bowes Inc. | Method for parcel tracking on the internet with e-mail notification |
US8230323B2 (en) | 2000-12-06 | 2012-07-24 | Sra International, Inc. | Content distribution system and method |
EP1223757B1 (en) | 2001-01-09 | 2006-03-22 | Metabyte Networks, Inc. | System, method, and software application for targeted advertising via behavioral model clustering, and preference programming based on behavioral model clusters |
US20020156817A1 (en) | 2001-02-22 | 2002-10-24 | Volantia, Inc. | System and method for extracting information |
US7546527B2 (en) | 2001-03-06 | 2009-06-09 | International Business Machines Corporation | Method and apparatus for repurposing formatted content |
IL142421A0 (en) | 2001-04-03 | 2002-03-10 | Linguistic Agents Ltd | Linguistic agent system |
US8095597B2 (en) | 2001-05-01 | 2012-01-10 | Aol Inc. | Method and system of automating data capture from electronic correspondence |
US7222090B2 (en) | 2001-08-29 | 2007-05-22 | Sedna Patent Services, Llc | Method and system for parsing purchase information from web pages |
US7197449B2 (en) | 2001-10-30 | 2007-03-27 | Intel Corporation | Method for extracting name entities and jargon terms using a suffix tree data structure |
US8689096B2 (en) | 2001-11-19 | 2014-04-01 | Hewlett-Packard Development Company, L.P. | Computer-based method, software module and computer program product for processing information in transaction-tax related applications |
US7321887B2 (en) | 2002-09-30 | 2008-01-22 | Sap Aktiengesellschaft | Enriching information streams with contextual content |
JP2005529385A (ja) | 2002-05-17 | 2005-09-29 | シンクロロジック インコーポレイテッド | 旅程データを構文解析するためのシステムおよび方法 |
US7319981B2 (en) | 2002-08-27 | 2008-01-15 | Michael Schwartzman | Multi-picture online commerce feature |
US20040064373A1 (en) | 2002-09-30 | 2004-04-01 | Shannon Robert W. J. | Point of sale receipt service |
US8070743B2 (en) | 2002-11-01 | 2011-12-06 | Valentx, Inc. | Devices and methods for attaching an endolumenal gastrointestinal implant |
US7024689B2 (en) | 2002-12-13 | 2006-04-04 | Intuit, Inc. | Granting access rights to unattended software |
US7219131B2 (en) | 2003-01-16 | 2007-05-15 | Ironport Systems, Inc. | Electronic message delivery using an alternate source approach |
US20040177120A1 (en) | 2003-03-07 | 2004-09-09 | Kirsch Steven T. | Method for filtering e-mail messages |
US7281029B2 (en) | 2003-05-13 | 2007-10-09 | Aol Llc, A Delaware Limited Liability Company | Method and system of capturing data for automating internet interactions |
US20050050099A1 (en) | 2003-08-22 | 2005-03-03 | Ge Information Systems | System and method for extracting customer-specific data from an information network |
US7819317B2 (en) | 2003-10-07 | 2010-10-26 | United States Postal Service | Bulk proof of delivery |
US7917548B2 (en) | 2003-11-14 | 2011-03-29 | Bottelle Memorial Institute | Universal parsing agent system and method |
US7836038B2 (en) | 2003-12-10 | 2010-11-16 | Google Inc. | Methods and systems for information extraction |
US7805344B2 (en) | 2004-03-12 | 2010-09-28 | Sybase, Inc. | System providing methodology for consolidation of financial information |
US20050210016A1 (en) | 2004-03-18 | 2005-09-22 | Zenodata Corporation | Confidence-based conversion of language to data systems and methods |
US20080147525A1 (en) | 2004-06-18 | 2008-06-19 | Gene Allen | CPU Banking Approach for Transactions Involving Educational Entities |
US7617176B2 (en) | 2004-07-13 | 2009-11-10 | Microsoft Corporation | Query-based snippet clustering for search result grouping |
GB0416857D0 (en) | 2004-07-29 | 2004-09-01 | Ingenico Uk Ltd | Electronic financial transactions |
WO2006013571A1 (en) | 2004-08-05 | 2006-02-09 | Viewscore Ltd. | System and method for ranking and recommending products or services by parsing natural-language text and converting it into numerical scores |
US20060122899A1 (en) | 2004-10-08 | 2006-06-08 | Advanced Commerce Strategies, Inc. | Comprehensive online shopping management system |
US7689037B2 (en) | 2004-10-22 | 2010-03-30 | Xerox Corporation | System and method for identifying and labeling fields of text associated with scanned business documents |
US20060095372A1 (en) | 2004-11-01 | 2006-05-04 | Sap Aktiengesellschaft | System and method for management and verification of invoices |
US20060143158A1 (en) | 2004-12-14 | 2006-06-29 | Ruhl Jan M | Method, system and graphical user interface for providing reviews for a product |
US7461056B2 (en) | 2005-02-09 | 2008-12-02 | Microsoft Corporation | Text mining apparatus and associated methods |
AU2005201758B2 (en) | 2005-04-27 | 2008-12-18 | Canon Kabushiki Kaisha | Method of learning associations between documents and data sets |
US20060265396A1 (en) | 2005-05-19 | 2006-11-23 | Trimergent | Personalizable information networks |
US7590647B2 (en) | 2005-05-27 | 2009-09-15 | Rage Frameworks, Inc | Method for extracting, interpreting and standardizing tabular data from unstructured documents |
US7739337B1 (en) | 2005-06-20 | 2010-06-15 | Symantec Corporation | Method and apparatus for grouping spam email messages |
US7546290B2 (en) | 2005-08-11 | 2009-06-09 | Marc Colando | Systems and methods for extracting and adapting data |
US8527504B1 (en) | 2005-09-01 | 2013-09-03 | At&T Intellectual Property Ii, L.P. | Data network content filtering using categorized filtering parameters |
US20070073580A1 (en) * | 2005-09-23 | 2007-03-29 | Redcarpet, Inc. | Method and system for delivering online sales promotions |
US20070073592A1 (en) | 2005-09-28 | 2007-03-29 | Redcarpet, Inc. | Method and system for network-based comparision shopping |
US7487912B2 (en) | 2005-09-28 | 2009-02-10 | First Data Corporation | Electronic receipting |
CN100442283C (zh) | 2005-10-20 | 2008-12-10 | 关涛 | 面向领域基于样本的互联网结构化数据抽取方法及其系统 |
US20070168464A1 (en) | 2005-12-13 | 2007-07-19 | Siemens Medical Solutions Health Services Corporation | System for Configuring a Data Exchange and Format Conversion System |
US7765212B2 (en) | 2005-12-29 | 2010-07-27 | Microsoft Corporation | Automatic organization of documents through email clustering |
US7899871B1 (en) | 2006-01-23 | 2011-03-01 | Clearwell Systems, Inc. | Methods and systems for e-mail topic classification |
US7644072B2 (en) | 2006-01-31 | 2010-01-05 | Perfect Market, Inc. | Generating a ranked list of search results via result modeling |
US7627641B2 (en) | 2006-03-09 | 2009-12-01 | Watchguard Technologies, Inc. | Method and system for recognizing desired email |
US8233751B2 (en) | 2006-04-10 | 2012-07-31 | Patel Nilesh V | Method and system for simplified recordkeeping including transcription and voting based verification |
US20070250390A1 (en) | 2006-04-24 | 2007-10-25 | Advanced Commerce Strategies, Inc. | Internet advertising method and system |
US8489689B1 (en) | 2006-05-31 | 2013-07-16 | Proofpoint, Inc. | Apparatus and method for obfuscation detection within a spam filtering model |
US7676465B2 (en) | 2006-07-05 | 2010-03-09 | Yahoo! Inc. | Techniques for clustering structurally similar web pages based on page features |
US7653576B2 (en) | 2006-08-01 | 2010-01-26 | International Business Machines Corporation | Method for pricing items |
US7788262B1 (en) | 2006-08-04 | 2010-08-31 | Sanika Shirwadkar | Method and system for creating context based summary |
US20090300482A1 (en) | 2006-08-30 | 2009-12-03 | Compsci Resources, Llc | Interactive User Interface for Converting Unstructured Documents |
US20080073429A1 (en) | 2006-09-26 | 2008-03-27 | Oesterling Christopher L | Method and system for post purchase monitoring |
US20080098300A1 (en) | 2006-10-24 | 2008-04-24 | Brilliant Shopper, Inc. | Method and system for extracting information from web pages |
US7689610B2 (en) | 2006-12-01 | 2010-03-30 | Computer Associates Think, Inc. | Automated grouping of messages provided to an application using string similarity analysis |
US8468244B2 (en) | 2007-01-05 | 2013-06-18 | Digital Doors, Inc. | Digital information infrastructure and method for security designated data and with granular data stores |
US7949670B2 (en) | 2007-03-16 | 2011-05-24 | Microsoft Corporation | Language neutral text verification |
US20080262940A1 (en) | 2007-03-29 | 2008-10-23 | Tsc Group | Purchase Order and Invoice Aggregator System for Sales Environment |
EP2151092A4 (en) | 2007-05-17 | 2010-07-21 | Fat Free Mobile Inc | METHOD AND SYSTEM FOR DESKTOP TAGGING OF A WEBSITE |
US20080306831A1 (en) | 2007-06-05 | 2008-12-11 | Jack Phillip Abraham | Systems and methods for facilitating purchase transactions in a network |
KR20090003397A (ko) | 2007-06-08 | 2009-01-12 | 엔에이치엔(주) | 정보 영역 지시에 의한 전자메일의 부가 정보 서비스 제공방법 및 그 시스템 |
US8725637B2 (en) | 2007-09-28 | 2014-05-13 | The Western Union Company | Methods and systems for generating invoices |
US8589366B1 (en) | 2007-11-01 | 2013-11-19 | Google Inc. | Data extraction using templates |
US20090171906A1 (en) | 2008-01-02 | 2009-07-02 | Research In Motion Limited | System and method for providing information relating to an email being provided to an electronic device |
US8676815B2 (en) | 2008-05-07 | 2014-03-18 | City University Of Hong Kong | Suffix tree similarity measure for document clustering |
US20090299887A1 (en) | 2008-06-03 | 2009-12-03 | Maya Shiran | System and method for detecting savings opportunities based on the price protection and return policies of retailers |
US20090313101A1 (en) | 2008-06-13 | 2009-12-17 | Microsoft Corporation | Processing receipt received in set of communications |
US8788350B2 (en) | 2008-06-13 | 2014-07-22 | Microsoft Corporation | Handling payment receipts with a receipt store |
US20090327268A1 (en) | 2008-06-27 | 2009-12-31 | Microsoft Corporation | Providing targeted information for entertainment-oriented searches |
US20100037177A1 (en) | 2008-08-08 | 2010-02-11 | Golsorkhi Andre O | Tool for capturing data across web domains |
US7809824B2 (en) | 2008-09-29 | 2010-10-05 | Yahoo! Inc. | Classification and cluster analysis spam detection and reduction |
US20100083095A1 (en) | 2008-09-29 | 2010-04-01 | Nikovski Daniel N | Method for Extracting Data from Web Pages |
US8065373B2 (en) | 2008-09-30 | 2011-11-22 | Pivot Solutions, Inc. | System and method for processing instant messages |
US8606649B2 (en) | 2008-10-31 | 2013-12-10 | Ellis Barlow Keener, JR. | Display of anomymous purchase information over the internet |
RU2686295C2 (ru) | 2008-12-17 | 2019-04-24 | Конинклейке Филипс Электроникс, Н.В. | Распределенные реестры пациентов для объединенных федеративных pacs |
US20100161527A1 (en) | 2008-12-23 | 2010-06-24 | Yahoo! Inc. | Efficiently building compact models for large taxonomy text classification |
US7783515B1 (en) | 2009-03-27 | 2010-08-24 | Bank Of America Corporation | Itemized receipt tracking system |
US20100257066A1 (en) | 2009-04-06 | 2010-10-07 | Bank Of America Corporation | Electronic receipts collection and management system |
EP2449521A4 (en) | 2009-06-30 | 2013-07-03 | Hewlett Packard Development Co | SELECTIVE EXTRACTION OF CONTENT |
US20110078724A1 (en) | 2009-09-30 | 2011-03-31 | Verizon Patent And Licensing, Inc. | Transactional advertising for television |
US8838485B2 (en) | 2009-10-26 | 2014-09-16 | Bay Photo, Inc. | Automated processing of digital media collections for distributed e-commerce |
US8666812B1 (en) | 2009-11-10 | 2014-03-04 | Google Inc. | Distributing content based on transaction information |
US8667015B2 (en) | 2009-11-25 | 2014-03-04 | Hewlett-Packard Development Company, L.P. | Data extraction method, computer program product and system |
US20110208787A1 (en) | 2010-01-11 | 2011-08-25 | Sidy Adam L | Method and device for continuously aggregating and redistributing personalized electronic information for display on multiple internet-enabled devices |
CA2788701A1 (en) | 2010-02-03 | 2011-08-11 | Arcode Corporation | Electronic message systems and methods |
US20110282734A1 (en) | 2010-04-07 | 2011-11-17 | Mark Zurada | Systems and methods used for publishing and aggregating real world and online purchases via standardized product information |
US9015139B2 (en) * | 2010-05-14 | 2015-04-21 | Rovi Guides, Inc. | Systems and methods for performing a search based on a media content snapshot image |
US9305263B2 (en) | 2010-06-30 | 2016-04-05 | Microsoft Technology Licensing, Llc | Combining human and machine intelligence to solve tasks with crowd sourcing |
US20120029963A1 (en) | 2010-07-31 | 2012-02-02 | Txteagle Inc. | Automated Management of Tasks and Workers in a Distributed Workforce |
EP2606438A4 (en) | 2010-08-20 | 2014-06-11 | Hewlett Packard Development Co | SYSTEMS AND METHODS SUITABLE FOR FILTERING THE CONTENT OF A WEB PAGE |
US20120047014A1 (en) | 2010-08-23 | 2012-02-23 | Yahoo! Inc. | Method and system for using email receipts for targeted advertising |
US8527436B2 (en) | 2010-08-30 | 2013-09-03 | Stratify, Inc. | Automated parsing of e-mail messages |
WO2012054788A1 (en) | 2010-10-21 | 2012-04-26 | Rillip Inc. | Method and system for performing a comparison |
US10115154B2 (en) | 2011-01-20 | 2018-10-30 | Martin Claude Lefebvre | Method and apparatus for inbound message management |
US20120203632A1 (en) | 2011-02-07 | 2012-08-09 | Marc Blum | Tracking and summarizing purchase information |
US20120203733A1 (en) | 2011-02-09 | 2012-08-09 | Zhang Amy H | Method and system for personal cloud engine |
US8666984B2 (en) | 2011-03-18 | 2014-03-04 | Microsoft Corporation | Unsupervised message clustering |
US9171088B2 (en) | 2011-04-06 | 2015-10-27 | Google Inc. | Mining for product classification structures for internet-based product searching |
US20120284150A1 (en) | 2011-05-02 | 2012-11-08 | Bradford Stanley | Inventory management system and method |
US20120284081A1 (en) | 2011-05-02 | 2012-11-08 | Fang Cheng | Methods and Apparatus for Gathering Intelligence from Itemized Receipts |
US20130009774A1 (en) | 2011-05-08 | 2013-01-10 | Anton Sabeta | Reminder method & system for merchandise returns |
US9646291B2 (en) | 2011-05-11 | 2017-05-09 | Visa International Service Association | Electronic receipt manager apparatuses, methods and systems |
US8458054B1 (en) | 2011-05-13 | 2013-06-04 | Amazon Technologies, Inc. | Method, medium, and system for suggesting images for items without images in listings data |
US8386285B2 (en) | 2011-05-25 | 2013-02-26 | International Business Machines Corporation | Demand modeling and prediction in a retail category |
US8661095B2 (en) | 2011-05-25 | 2014-02-25 | Juniper Networks, Inc. | Identifying scripting form of text commands for a network device |
CN102831121B (zh) | 2011-06-15 | 2015-07-08 | 阿里巴巴集团控股有限公司 | 一种网页信息抽取的方法和系统 |
US20120330971A1 (en) | 2011-06-26 | 2012-12-27 | Itemize Llc | Itemized receipt extraction using machine learning |
US9311599B1 (en) | 2011-07-08 | 2016-04-12 | Integral Ad Science, Inc. | Methods, systems, and media for identifying errors in predictive models using annotators |
US9875486B2 (en) | 2014-10-21 | 2018-01-23 | Slice Technologies, Inc. | Extracting product purchase information from electronic messages |
US9846902B2 (en) | 2011-07-19 | 2017-12-19 | Slice Technologies, Inc. | Augmented aggregation of emailed product order and shipping information |
US9563904B2 (en) | 2014-10-21 | 2017-02-07 | Slice Technologies, Inc. | Extracting product purchase information from electronic messages |
US8844010B2 (en) | 2011-07-19 | 2014-09-23 | Project Slice | Aggregation of emailed product order and shipping information |
US20130024282A1 (en) | 2011-07-23 | 2013-01-24 | Microsoft Corporation | Automatic purchase history tracking |
US9916538B2 (en) | 2012-09-15 | 2018-03-13 | Z Advanced Computing, Inc. | Method and system for feature detection |
US8738477B2 (en) | 2011-11-10 | 2014-05-27 | Connexive, Inc. | Method and apparatus for automated bill timeline |
US8903924B2 (en) | 2011-12-09 | 2014-12-02 | International Business Machines Corporation | Aggregating data in electronic communications |
US9606970B2 (en) | 2012-01-05 | 2017-03-28 | Data Record Science | Web browser device for structured data extraction and sharing via a social network |
US8744948B1 (en) | 2012-01-11 | 2014-06-03 | Intuit Inc. | Mobile data import for a financial return |
US20130197954A1 (en) * | 2012-01-30 | 2013-08-01 | Crowd Control Software, Inc. | Managing crowdsourcing environments |
US11568334B2 (en) | 2012-03-01 | 2023-01-31 | Figure Eight Technologies, Inc. | Adaptive workflow definition of crowd sourced tasks and quality control mechanisms for multiple business applications |
JP6100559B2 (ja) | 2012-03-05 | 2017-03-22 | 株式会社半導体エネルギー研究所 | 半導体記憶装置 |
US9092749B2 (en) | 2012-04-13 | 2015-07-28 | International Business Machines Corporation | Information governance crowd sourcing |
US9436738B2 (en) | 2012-04-19 | 2016-09-06 | Nant Holdings Ip, Llc | Mechanical Turk integrated IDE, systems and method |
US8812417B2 (en) | 2012-08-20 | 2014-08-19 | InsideSales.com, Inc. | Hierarchical based sequencing machine learning model |
US9461876B2 (en) | 2012-08-29 | 2016-10-04 | Loci | System and method for fuzzy concept mapping, voting ontology crowd sourcing, and technology prediction |
US9336193B2 (en) | 2012-08-30 | 2016-05-10 | Arria Data2Text Limited | Method and apparatus for updating a previously generated text |
US20140105508A1 (en) | 2012-10-12 | 2014-04-17 | Aditya Arora | Systems and Methods for Intelligent Purchase Crawling and Retail Exploration |
US9268860B2 (en) | 2012-12-02 | 2016-02-23 | At&T Intellectual Property I, L.P. | Methods, systems, and products for personalized monitoring of data |
US20140172767A1 (en) | 2012-12-14 | 2014-06-19 | Microsoft Corporation | Budget optimal crowdsourcing |
US9600788B2 (en) * | 2012-12-27 | 2017-03-21 | Xerox Corporation | Crowdsourcing directory system |
US9374369B2 (en) | 2012-12-28 | 2016-06-21 | Lookout, Inc. | Multi-factor authentication and comprehensive login system for client-server networks |
WO2014111948A1 (en) * | 2013-01-18 | 2014-07-24 | Hewlett-Packard Development Company, L. P | Task assignment in crowdsourcing |
US9183193B2 (en) | 2013-02-12 | 2015-11-10 | Xerox Corporation | Bag-of-repeats representation of documents |
US20140236577A1 (en) | 2013-02-15 | 2014-08-21 | Nec Laboratories America, Inc. | Semantic Representations of Rare Words in a Neural Probabilistic Language Model |
US9251474B2 (en) * | 2013-03-13 | 2016-02-02 | International Business Machines Corporation | Reward based ranker array for question answer system |
US9483741B2 (en) * | 2013-03-28 | 2016-11-01 | Wal-Mart Stores, Inc. | Rule-based item classification |
KR20140138512A (ko) | 2013-05-24 | 2014-12-04 | 대한민국(농촌진흥청장) | 구매정보 분석 장치 및 방법 |
US9594944B2 (en) * | 2013-09-25 | 2017-03-14 | Xerox Corporation | Methods and systems for monitoring a worker performing a crowdsourced task |
US20150095017A1 (en) | 2013-09-27 | 2015-04-02 | Google Inc. | System and method for learning word embeddings using neural language models |
US9767419B2 (en) | 2014-01-24 | 2017-09-19 | Microsoft Technology Licensing, Llc | Crowdsourcing system with community learning |
US20150254593A1 (en) * | 2014-03-10 | 2015-09-10 | Microsoft Corporation | Streamlined creation and utilization of reference human intelligence tasks |
US20150295869A1 (en) | 2014-04-14 | 2015-10-15 | Microsoft Corporation | Filtering Electronic Messages |
US9275418B2 (en) | 2014-05-16 | 2016-03-01 | Bank Of America Corporation | Providing e-receipts to customers |
US20160071048A1 (en) * | 2014-09-08 | 2016-03-10 | Xerox Corporation | Methods and systems for crowdsourcing of tasks |
US20160104188A1 (en) | 2014-10-08 | 2016-04-14 | Paribus Co. | Consumer price protection service |
WO2016064679A1 (en) | 2014-10-21 | 2016-04-28 | Slice Technologies, Inc. | Extracting product purchase information from electronic messages |
EP3210132A1 (en) | 2014-10-24 | 2017-08-30 | Google, Inc. | Neural machine translation systems with rare word processing |
US10409908B2 (en) | 2014-12-19 | 2019-09-10 | Google Llc | Generating parse trees of text segments using neural networks |
US20160232474A1 (en) * | 2015-02-05 | 2016-08-11 | Xerox Corporation | Methods and systems for recommending crowdsourcing tasks |
US11080587B2 (en) | 2015-02-06 | 2021-08-03 | Deepmind Technologies Limited | Recurrent neural networks for data item generation |
US10339440B2 (en) | 2015-02-19 | 2019-07-02 | Digital Reasoning Systems, Inc. | Systems and methods for neural language modeling |
US9313166B1 (en) | 2015-04-29 | 2016-04-12 | InsideSales.com, Inc. | Enhancement of machine learning techniques for an electronic message system |
US10515307B2 (en) | 2015-06-05 | 2019-12-24 | Google Llc | Compressed recurrent neural network models |
US10606946B2 (en) | 2015-07-06 | 2020-03-31 | Microsoft Technology Licensing, Llc | Learning word embedding using morphological knowledge |
US20170017635A1 (en) | 2015-07-17 | 2017-01-19 | Fido Labs Inc. | Natural language processing system and method |
US10089576B2 (en) | 2015-07-28 | 2018-10-02 | Microsoft Technology Licensing, Llc | Representation learning using multi-task deep neural networks |
WO2017083752A1 (en) | 2015-11-12 | 2017-05-18 | Google Inc. | Convolutional gated recurrent neural networks |
EP3371807B1 (en) | 2015-11-12 | 2023-01-04 | Google LLC | Generating target phoneme sequences from input speech sequences using partial conditioning |
WO2017090051A1 (en) | 2015-11-27 | 2017-06-01 | Giridhari Devanathan | A method for text classification and feature selection using class vectors and the system thereof |
US20170154258A1 (en) | 2015-11-30 | 2017-06-01 | National Institute Of Information And Communications Technology | Joint estimation method and method of training sequence-to-sequence model therefor |
US10984365B2 (en) | 2015-11-30 | 2021-04-20 | Microsoft Technology Licensing, Llc | Industry classification |
US10339470B1 (en) * | 2015-12-11 | 2019-07-02 | Amazon Technologies, Inc. | Techniques for generating machine learning training data |
US9792530B1 (en) * | 2015-12-28 | 2017-10-17 | Amazon Technologies, Inc. | Generating and using a knowledge base for image classification |
US10268671B2 (en) | 2015-12-31 | 2019-04-23 | Google Llc | Generating parse trees of text segments using neural networks |
US11263514B2 (en) | 2016-01-13 | 2022-03-01 | Google Llc | Processing and generating sets using recurrent neural networks |
US9799327B1 (en) | 2016-02-26 | 2017-10-24 | Google Inc. | Speech recognition with attention-based recurrent neural networks |
US10691999B2 (en) | 2016-03-16 | 2020-06-23 | Maluuba Inc. | Parallel-hierarchical model for machine comprehension on small data |
US10019438B2 (en) | 2016-03-18 | 2018-07-10 | International Business Machines Corporation | External word embedding neural network language models |
KR102431817B1 (ko) * | 2017-10-12 | 2022-08-12 | 삼성전자주식회사 | 사용자 발화를 처리하는 전자 장치 및 서버 |
US20190197483A1 (en) * | 2017-12-22 | 2019-06-27 | Microsoft Technology Licensing, Llc | Large-scale aggregation and verification of location data |
US11803883B2 (en) * | 2018-01-29 | 2023-10-31 | Nielsen Consumer Llc | Quality assurance for labeled training data |
-
2019
- 2019-01-09 US US16/243,342 patent/US11803883B2/en active Active
- 2019-01-29 JP JP2019013304A patent/JP7458148B2/ja active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2014509002A (ja) * | 2011-02-02 | 2014-04-10 | イーベイ インク. | 特定地域在庫検索用のメタデータ使用 |
US20140314311A1 (en) * | 2013-04-23 | 2014-10-23 | Wal-Mart Stores, Inc. | System and method for classification with effective use of manual data input |
JP2017505964A (ja) * | 2013-11-27 | 2017-02-23 | 株式会社Nttドコモ | 機械学習に基づく自動タスク分類 |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2021086613A (ja) * | 2019-11-27 | 2021-06-03 | クラウドワークス インコーポレイテッドCrowdWorks, Inc. | 本番作業に基づいた検証用クラウドソーシング作業を提供する方法及び装置 |
KR102155748B1 (ko) * | 2020-01-03 | 2020-09-17 | 주식회사 크라우드웍스 | 인공지능 학습데이터 생성을 위한 크라우드소싱 기반 프로젝트의 작업 시간 대비 검수 시간을 이용한 검수 단가 자동 업데이트 방법 |
WO2021137523A1 (ko) * | 2020-01-03 | 2021-07-08 | 주식회사 크라우드웍스 | 인공지능 학습데이터 생성을 위한 크라우드소싱 기반 프로젝트의 작업 시간 대비 검수 시간을 이용한 검수 단가 자동 업데이트 방법 |
CN111288999A (zh) * | 2020-02-19 | 2020-06-16 | 深圳大学 | 一种基于移动终端的行人路网属性检测方法、装置和设备 |
CN111288999B (zh) * | 2020-02-19 | 2021-08-31 | 深圳大学 | 一种基于移动终端的行人路网属性检测方法、装置和设备 |
Also Published As
Publication number | Publication date |
---|---|
JP7458148B2 (ja) | 2024-03-29 |
US20190236478A1 (en) | 2019-08-01 |
US11803883B2 (en) | 2023-10-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP7458148B2 (ja) | ラベル付きトレーニングデータの品質向上 | |
US11017038B2 (en) | Identification and evaluation white space target entity for transaction operations | |
CN107967575B (zh) | 一种人工智能保险咨询服务人工智能平台系统 | |
Fernández-García et al. | A recommender system for component-based applications using machine learning techniques | |
Vialardi et al. | A data mining approach to guide students through the enrollment process based on academic performance | |
Fahse et al. | Managing bias in machine learning projects | |
US11119979B2 (en) | Content based recommendations of file system save locations | |
US8949237B2 (en) | Detecting overlapping clusters | |
Tarmizi et al. | A review on student attrition in higher education using big data analytics and data mining techniques | |
Ali et al. | Trust-based requirements traceability | |
Paramesh et al. | Classifying the unstructured IT service desk tickets using ensemble of classifiers | |
US11900320B2 (en) | Utilizing machine learning models for identifying a subject of a query, a context for the subject, and a workflow | |
JP2024500464A (ja) | 動的ファセットランク付け | |
Kim | Implementation of artificial intelligence system and traditional system: a comparative study | |
Seeliger et al. | Learning of process representations using recurrent neural networks | |
US20200410387A1 (en) | Minimizing Risk Using Machine Learning Techniques | |
US20210065019A1 (en) | Using a dialog system for learning and inferring judgment reasoning knowledge | |
CN111159241A (zh) | 一种点击转化预估方法及装置 | |
Darias et al. | Using case-based reasoning for capturing expert knowledge on explanation methods | |
Tu et al. | Attention-aware answers of the crowd | |
US20200106988A1 (en) | Behavioral influence system in socially collaborative tools | |
US11210108B2 (en) | Guiding the installation process of sensor-based devices | |
Oneto et al. | Advances in artificial neural networks, machine learning and computational intelligence | |
Rebmann et al. | Multi-perspective identification of event groups for event abstraction | |
Calian et al. | Integrating memetic search into the BioHEL evolutionary learning system for large-scale datasets |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20190426 |
|
AA79 | Non-delivery of priority document |
Free format text: JAPANESE INTERMEDIATE CODE: A24379 Effective date: 20190604 |
|
A711 | Notification of change in applicant |
Free format text: JAPANESE INTERMEDIATE CODE: A711 Effective date: 20190606 |
|
A711 | Notification of change in applicant |
Free format text: JAPANESE INTERMEDIATE CODE: A711 Effective date: 20211022 |
|
A711 | Notification of change in applicant |
Free format text: JAPANESE INTERMEDIATE CODE: A711 Effective date: 20211105 |
|
A621 | Written request for application examination |
Free format text: JAPANESE INTERMEDIATE CODE: A621 Effective date: 20211222 |
|
A977 | Report on retrieval |
Free format text: JAPANESE INTERMEDIATE CODE: A971007 Effective date: 20230118 |
|
A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20230303 |
|
A601 | Written request for extension of time |
Free format text: JAPANESE INTERMEDIATE CODE: A601 Effective date: 20230601 |
|
A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20230803 |
|
A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20231107 |
|
A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20240206 |
|
TRDD | Decision of grant or rejection written | ||
A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20240216 |
|
A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20240318 |
|
R150 | Certificate of patent or registration of utility model |
Ref document number: 7458148 Country of ref document: JP Free format text: JAPANESE INTERMEDIATE CODE: R150 |